How Does eTools.ch Work?
The metasearch engine
eTools.ch has no own index or database – instead the
information is retrieved from several data sources simultaneously (federated search, search several search engines at once).
As the following illustration shows, a data source can be, among others,
a traditional web search engine, a web service or a database.
In the optimal case, eTools.ch receives a result from every data source within the timeout period.
Afterwards these results are processed, merged and returned to the user.
In simplified terms, the following steps are performed:
- The user enters the search terms and search options and starts the search.
- The search parameters get translated for each data source individually.
- All selected data sources are queried simultaneously.
- The returned results of every data source are transformed into a standardized
format and optionally filtered.
- The individual results are merged into a common result (data fusion):
- redundant information is removed;
- relevant information is weighted, ranked and sorted (similar to Borda count).
- The merged result is returned to the user.
Current Status of eTools.ch
eTools.ch is as fast as the slowest data source that currently is queried by eTools.ch.
Therefore the search settings let you limit the maximum time to wait for a query (by default 2 seconds).
If a data source does not respond anymore reliably, eTools.ch will automatically disable it temporary.
As soon as the data source is available again, it will be automatically re-enabled.
The following table reflects the current status of the 17 data sources that are queried by eTools.ch:
Currently, a complete search takes on average 0.78 seconds.
Refresh status
Bot Detection and Protection
eTools.ch receives millions of bot requests daily.
To keep the service running smoothly, we developed our own filters to detect and block these bots rather than relying on a third party protection service.
To date, we have blacklisted 4,881,596 suspicious IPs, which must pass a CAPTCHA to regain access.
In addition to the blacklisted IPs, we currently block 746 autonomous systems (AS), mostly cloud or hosting services with a high number of bot requests.
Every request is assigned a score reflecting the likelihood of it being a bot.
The lower the score, the more restrictions apply: fewer requests are allowed and sources may be excluded from search.
Your current score is 0.3 of 1.
Using a proxy, VPN, Tor, ad-blocking plugin, or privacy tool such as Apple's iCloud Private Relay or Brave's obfuscation features will lower your score.
We have no issue with these tools themselves; the problem is that bots use the same or similar techniques, making them harder to distinguish from legitimate users.
We appreciate your understanding as we fight against bots to keep eTools.ch running.
Technology of eTools.ch
eTools.ch is based on our proven search solution EnterpriseMetasearch®.
Possible Applications of EnterpriseMetasearch®
EnterpriseMetasearch® is modular, expandable and flexible configurable and thus can be
used for applications that need a federated search or an integration solution, for example:
- Enterprise Federated Search: an Internet/Intranet application that searches several
data sources at once, such as search engines, databases,
Enterprise Content Management (ECM)
or SAP systems.
- Vertical search portal (topic-specific search cockpit).
- Enterprise Application Integration (EAI).
- Spider or crawler to collect structured data.
- Enterprise (legacy) data migration or interface project.
Specific Qualities of the EnterpriseMetasearch® Technology
- The Web user-interface (GUI)
and the underlaying collector and aggregator
framework are completely separated from each other.
The strict separation of model- (business logic) and presentation layer
allows the autonomous use of the framework engine (e.g. in a
SOA environment).
- Scalable, lightweight and performance-optimized design, permitting to run on moderate hardware.
- Supports different protocols (e.g.
HTTP,
JDBC,
REST or
SOAP) and applications
(e.g. Apache Lucene or
SAP JCo).
- Allows conversion of results to other formats (e.g. Text,
XML or
PDF).
- Due to the flexible plug-in architecture, protocols and formats can be
easily adapted and expanded.
- Optimized for online operation, but can also be run as batch process
or operated as Unix/Windows-service.
- Platform independent (e.g. Unix, Linux, Windows or Mac OS X).
- Proposes spelling corrections in the form of «Did you mean: ...».
- Organizes search results into predefined categories (faceted search).
- Automatically organizes search results into thematic categories (text clustering):
Standards and Technologies of EnterpriseMetasearch®
- 100% Java (Java EE).
- Enterprise features, e.g.:
- Configurable pools (e.g. for worker threads and connections) and caches.
- Autonomous data source watchdog with status notification.
- Flexible application-clustering.
- Management and monitoring (JMX) with runtime statistics.
- Bot detection and protection.
- Extensible query parser.
- Unicode compliant localization (i18n).
- Robust multi-stage timeout handling.
- Supports widely used servlet engines and Java EE application servers, e.g.:
- Apache Tomcat
- Liferay Portal
- IBM WebSphere
- Oracle WebLogic Server
Interested?
Further information about our federated search solution can be found on the EnterpriseMetasearch® product page.
Please do not hesitate to contact us for any question.