Federation: Not an Answer to Pan-Enterprise Search
Many organizations have attempted to provide pan-enterprise search using a federated search infrastructure, where queries are federated out to the native search engines of the many disparate repositories in the enterprise. While this may be an appealing quick fix, this actually opens the enterprise up to significant risk.
Firstly, by federating search out to native search engines, one assumes that they are capable of effectively searching their own data. Given that many repositories rely on out-dated, end-of-life search products for their native search, this is not a reasonable assumption to make. These search engines rarely search all information in the repository, instead using sleight of hand techniques to perpetuate the illusion of performance. This is a significant problem now that the FRCP mandate that all Electronically Stored Information (ESI) be made searchable. Federated approaches therefore do not guarantee compliance.
Secondly, federation does not necessarily produce a single, coherent, relevance ranking. Each search engine to which a query is federated determines relevance ranking independently and each engine will return a separate list of results. Compiling these results into one coherent list is not trivial. Not only will each engine have determined the order of the returned results using different methods, but the weightings assigned to different results will also vary between search engines. Most legacy search engines, even if they are able to federate results out to other search engines, have no ability to intelligently handle the results they get back.
Finally, federation is not a scalable solution. Relying on native search engines means that an accurate results list cannot be built until one has all the results from all the source repositories. Consequently, the speed at which the results list is returned is dependent on the speed of the slowest engine. Furthermore, since every query has to be sent to all of the native search engines in a federated environment, this greatly increases not only network traffic, but also load on the repositories themselves. When this problem is combined with common legacy security techniques (such as unmapped methods) the network traffic demanded by even a modest intranet search can be crippling.
Autonomy's Approach
In contrast, Autonomy is able to directly search over 1,000 different content formats across 400 different content repositories, guaranteeing that all information has been searched whilst respecting security. Federation is not required which provides significant benefits in terms of performance and scalability, and ensures that the enterprise is FRCP compliant. Should federation be desired, Autonomy can provide this capability; however for full pan-enterprise deployments, federation is not recommended.
Summary: ...search with a single query. The Benefits The value of IDOL Federator on www.idg.net is significant. “Search is the key functional feature for our portal,” Effron says. With IDOL Federator, IDG has experienced higher user satisfaction, increased traffic to its advertiser-supported publication sites,...
Summary: ...needed to be accessible from different institutions but in a secure way. An additional problem was that most of the people working for PARS are not situated on the same location and often they are even from different institutions. This meant large amounts of documents had to be printed out and physically...
Summary: ...and the edited content was returned via email for eventual publishing. This ineffcient and time-consuming process introduced unnecessary cycle time, delayed the timely delivery of information, and slowed Web updates to once or twice a week. With Interwoven, simple-to-use templates streamline the publishing...
Summary: ...model has a different audience and buyer, and each responds differently to things like text, color, and layout,” says Shlauter. “The same types of things don’t necessarily work for every model, and even minor changes can have a bigger impact than you’d expect. You can’t really know what will...
Summary: ...had gone cold but we found him in hours with the search engine” Detective Gary Williams, South Yorkshire Police that even when they were able to locate the relevant information, the lack of metadata and incorrectly weighted search parameters meant that searches were often unrepeatable and information...
Summary: ...e-mails. Everyone using it has made the same comment—it makes their lives much easier,” says Rosenberg. Eisner’s internal accounting group is using WorkSite to eliminate the paper-pushing associated with compiling client content. For example, all content associated with a client’s tax return is...
Summary: ...more than 3.4 million products and services from over 7,800 vendors. The site logs an average of 75,000 searches every workday. Under this immense volume, GSA’s former search technology proved slow and ineffective—forcing customers to sift through as many as 1,000 results for some queries. The site’s...
Summary: ...court case decisions found in the system can be accessed by personalized views, created for court employees according to the role(s) they have in the processes. The Views are based on predefined lists of the decisions according to their role in the processes. The decisions can be searched by the associated...
Summary: ...resources and jumpstarting employee productivity—a feat that otherwise would have taken a hiring spree to achieve. “TeleForm was a huge improvement over the slow, inaccurate and inelegant method we had in place,” says Battle. “Every study meant development and implementation of a custom application....
Summary: ...was primarily motivated by cost (improving the cost efficiencies of disk storage within its environment), although performance of the overall system was an important secondary goal. As part of the evaluation process for an email archive solution, VUMC developed a "key criteria" list with 12 archive system...
Summary: ...version of each solution so we could start looking at more interesting ways of using technology,” says Farone. Improving disaster recovery and compliance As BCBSMA evaluated its ECM priorities, disaster recovery quickly rose to the top of the list. To ensure around-the-clock availability, the company...
Summary: ...transparently integrate with Microsoft applications, acceptance and use throughout the organization by all users becomes much easier and allows the organization to concentrate on the aspects of the implementation of the Strategic Framework in order to achieve a much faster Return on Investment. The component...
This is a small selection of the Autonomy case studies available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...Autonomy IDOL Server 7. To be FRCP-compliant, pan-enterprise search platforms: • Need to search ALL repositories • Cannot perform “jump- out ” - a sleight of hand used by some vendors to feign performance, in which the search engine stops looking across an index as soon as it is believed a large...
Summary: ...methods for audio are difficult to use, time-consuming and costly with the results often failing to meet the court’s expectations, leading to additional costs and delays with the risk of penalty. Manual transcriptions and reviews, which are prone to human error, are often inaccurate, slow, costly, and...
Summary: ...Intelligently selects the specialist most likely to return a query result, avoiding the need to bombard multiple systems with the same query • Automatically merges results from different search engines into one coherent set Conversation After users perform their queries, IDOL Retina engages the users...
Summary: ...defend that decision in court with audited, documented trail. The Query Journey function guides the user to select the right search methodology for the task at hand In helping users select the best algorithm for their needs, SPV clearly delineates the query pipeline, including concepts found, term frequencies...
Summary: ...using innovative visualization techniques like concept clouds and clustering, aid in quickly refining queries. Universal Search also provides methods to conceptually relate unstructured information, for example, documents can be previewed within the web interface, and users can be immediately presented...
Summary: ...Autonomy Interwoven - Social Media Analysis Product Brief. For instance, consumers often use different words to express the same idea, but this problem becomes especially pronounced in social media where the language is more conversational, rife with familiar expressions, slang, and varying emotional...
Summary: ...Autonomy Investigator & ECA’s exclusive automatic query guidance automatically suggests conceptually related search terms based on the initial query. This is especially helpful during discovery when the reviewer searches for one term and the engine is able to return a list of additional potentially...
Summary: ...of Civil Procedure (FRCP) and case law. Each step of the process is consistently executed on a single, easy-to-use, highly scalable and secure platform that quickly implements the workflow and processes necessary to demonstrate reasonable and good faith efforts to comply with the duty to preserve. The...
Summary: ...of ability to display and control native view options. The money and time saved by KeyView IDOL at load time is just the initial yield, the savings grow as the process continues. Keyview IDOL’s Viewing and Export capabilities eliminate the cost, time and problems associated with native applications...
Summary: ...While legacy technologies will likely continue to serve as the predominant stores for structured information, the methods for analyzing this businesscritical information is changing. With IDOL SPE, probabilistic modeling surfaces patterns in the data without prescriptive prior knowledge, and enables these...
Summary: ...Autonomy Business Console Product Brief. It offers insight to a granular level of reasoning via the intuitive graphical interface, highlighting sections of the document which correspond to each element of the query. Settings can then be finely tuned to ensure the desired results are returned to the end-user....
Summary: ...digital watermarking or similar methods of DRM and so is not undermined by pirates inserting random noise or employing format and codec changes. ACID can also treat audio and video channels of rich media both separately and as one using a range of detection methods. Because of this, ACID is not restricted...
This is a small selection of the Autonomy Product Briefs available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...in a document's words in order for it to be scored Legacy Search IDOL Server supports all known legacy search methods, including: • displaying recent additions to Document Management Serv er Users can click on the What’s New option at the top of the portlet. The portlet displays a list of documents...
Summary: ...Autonomy File System Fetch Technical Brief 0605. FlashSlave • Configurable directories or file lists with automatic recursion • Filtering of content based on file extensions and dates • Aggregation of native XML files • Batch processing • Import parameters compatible with any other Autonomy...
Summary: ...libraries, sites, site collections and systems, including rich media • List items from all list types, including: Wikipages Links Announcements CalendarItems Contacts Commentson a discussion board Tasks Surveyresponses Items from custom lists Blogentries Publishingpages plus list items in any other...
Summary: ...Autonomy PiB 4.3 08/03. The Retrieval Portlet submits the natural language query to one or more databases that have been set up, in order to find documents that are related to your query. The concept of the query is analyzed and documents that are relevant to this concept are returned to the user. • Legacy...
Summary: ...governance concerns have moved this technology to a strategic position in the enterprise. Governance and information risk issues came sharply into focus with the amendments to the Federal Rules of Civil Procedure (FRCP). As the uses of archiving evolve from managing mailbox volumes to addressing enterprise-wide...
Summary: ...includes a generic slave process “OmniSlave,” that can be utilized by the Import Module to handle any proprietary format. Included as standard, OmniSlave provides pre-built import DLLs that handle the Microsoft Office Suite (ppt, doc, rtf and xls file formats). Autonomy provides the developer with...
Summary: ...owners. Product specifications and features are subject to change without notice. Use of Autonomy software is under license. [AUT TB] 21.04.08 Such an approach not only enables market-leading search and retrieval operations (both in terms of accuracy and speed) through this unified retrieval method, but...
Summary: ...with DiSH. Features Content that is stored in IDOL server™ can optionally be returned as plain text in the normal way, with the loss of its original formatting but a decrease in time taken to return the content. Please note, it is necessary to have a local Oracle database or SQL*NET or NET8 to access...
Summary: ...Autonomy Stellent Fetch Technical Brief 0605. ImportSlave, OmniSlave, BinSlave & PDFSlave • Combine data from any number of tables into a single document • Support for multiple jobs performing different actions • Schedule jobs independently of each other • Extract data as any text based format...
Summary: ...Autonomy ODBC Fetch Technical Brief 0506. ImportSlave, OmniSlave, BinSlave & PDFSlave • Combine data from any number of tables into a single document • Support for multiple jobs performing different actions • Schedule jobs independently of each other • Extract data as any text based format including...
Summary: ...ImportSlave, OmniSlave, BinSlave & PDFSlave • Combine data from any number of tables into a single document • Support for multiple jobs performing different actions • Schedule jobs independently of each other • Extract data as any text based format including HTML & XML • Extract binary document...
This is a small selection of the Autonomy Technical Briefs available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...with common legacy security techniques (such as unmapped methods), the network traffic demanded by even a modest intranet search can be crippling. Furthermore, the speed at which the results list is returned is dependent on the speed of the slowest engine. 3. Federation does not necessarily produce a...
Summary: ...scheduled or ad hoc deletion It federates searches to different search engines, which often do not employ FRCP-compliant search methods Specifically, SharePoint’s native search places limitations in the following areas that do not comply with FRCP requirements: Large files not indexed: By default, the...
Summary: ...or ad hoc deletion It federates searches to different search engines, which often do not employ FRCP-compliant search methods Specifically, SharePoint’s native search places limitations in the following areas that do not comply with FRCP requirements: Large files not indexed: By default, the fetch will...
Summary: ...from normally scheduled or ad hoc deletion. • It federates searches to different search engines, which of ten do not employ FRCP-compliant search methods. Specifically, SharePoint’s native search places limitations in the following areas that do not comply with FRCP requirements: • Large files not...
Summary: ...average enterprise, that may mean cross-checking 10,000 results in order to return a results set of merely 10. Similar to the issues surrounding federation, unmapped security imposes an enormous burden on both the network and native repositories. Performance hit is felt even greater when using custom...
Summary: ...interactions. The requirement for a multitude of communications between modules negatively impacts performance. Complex architecture • Multiple query and result filters required for each security module slow down the query response time • Communication can potentially run insecurely across the network...
Summary: ...XML automatically. Autonomy’s approach allows organizations to overcome problems that are typically associated with XML by: Removing the need to manually insert XML tags Allowing interoperability between applications that use different XML tagging schemes Indexing native XML directly into the engine...
Summary: ...of today’s systems rely on legacy keyword and Boolean search techniques, whose limitations, shortfalls and over inclusiveness are well documented. Though these legacy methods are considered standard methods for eDiscovery and compliance, courts are beginning to question them and studies have shown that...
Summary: ...the business. So, while consumers use different channels and different touch points throughout their relationship with your brand, today’s businesses tend to be slow or simply unable to capitalize on the data. Leveraging Unstructured Data It’s all Data. Computers Can Process it. Organizations have...
Summary: ...you freely turn that document over or leave it out of the collection? On the whole, self-collection will likely fail to comply with the FRCP requirements of a systemized, repeatable, or defensible process. Self-collection is rarely systemized, as different criteria and search techniques will typically...
Summary: ...will retrieve only documents which contain every term so joined. Such queries generally return too little. If OR is used, then the search engine will return any and every document which contains any one or more of the so joined terms. Such queries generally return too much…” “In short, language...
Summary: ...acoustic information, Virage achieves a higher level of understanding through language modeling. Language modeling involves concept extraction in conjunction with acoustic-phonetic methods to achieve significantly greater accuracy and better results. Simple acoustic-phonetic methods alone fail to achieve...
This is a small selection of the Autonomy White Papers available, please visit our publications site at http://publications.autonomy.com/ for more information.
Related Case Studies
