The world's largest and most secure intelligence organizations have deployed Autonomy's Intellectual Asset Protection System (IAS) to safeguard their most sensitive information assets. Autonomy provides all aspects of security management, including front-end user authentication, back-end entitlement checking and secure encrypted communication between the IDOL Server and its client applications with 128-bit Block Tiny Encryption Algorithm (BTEA). IDOL's mapped security model is the only empirically proven index security model that scales in the enterprise.
There are three general security models currently available:
1. Unmapped Security
Unmapped security is the traditional method used by source repositories and search engines. For every potential match to a given query, a call is made via the native repository's API (e.g. Documentum) to ascertain the access privileges for that particular document. A single query consequently bombards the native repository with document privilege requests as the retrieval system attempts to assemble a relevant results list from thousands of candidate hits. This method presents significant performance and scalability problems.
Unmapped Security
Mapped Security
"Security is a key differentiator for IDOL. IDOL offers "mapped security" and near real-time synchronization of security entitlements with source content repositories - making it a great fit for highly secure search scenarios"
The Forrester WaveTM: Enterprise Search Platforms, Matthew Brown
Autonomy recommends mapped security but also offers the choice between mapped, unmapped and a hybrid of both. Autonomy also supplies plug-in sample code, so that customers, OEMs and partners are able to develop and implement their own form of security plug-in.
2. Cached Security
Cached security is the method of choice for legacy systems. Cached security marginally relieves the scalability problem of unmapped security by storing results for queries it has already seen. Consequently, when a user repeats a query, the result set can be retrieved from the cache rather than triggering a network-mediated request. However, this approach still relies on calling out across the network directly to the repository for each new query. In addition, it also misses potential results, as the result sets stored within its memory do not dynamically update new information.
3. Autonomy's Unique IAS Mapped Security
Only Autonomy offers mapped security - a highly configurable, secure, accurate, and fast method for respecting third party security entitlements. IDOL maps the underlying security model in the form of ACL, group, role, protective markings, etc. from all of the underlying repositories directly into the kernel of the IDOL engine itself, and stores the information in an encrypted field. As a result, IDOL does not need to send any requests across the network to the data stores when building up a results list. What the user is allowed to see is assessed "inline" within the IDOL kernel at speeds that exceed the response times of the native repository. Unlike other techniques, the security model is never out of date as the transitional signaling mechanism within the connector layer informs IDOL in real-time of any updates or changes to permissions within the underlying content.
Since IDOL's architecture is inherently modular by design, it requires multiple subsystems to communicate with each other, often across insecure networks. All communication between these processes may be encrypted (Secure Sockets Layer), so that packet sniffers who are able to break past a firewall are unable to read the content of traffic between IDOL modules. All of the system's modules are capable of operating in a secure communications mode providing, at minimal processing overhead, the protection of 128-bit encryption. Additionally, IDOL can leverage SSL for both aggregation and querying of content, including access to SSL encrypted sites.
"Autonomy fully meets the stringent requirements of managing records in a classified environment where security and access are critical."
IT Business Net, 2011
Scalability and Performance
The management of structured and unstructured content requires a platform that can meet the most rigorous performance requirements and be easily resized commensurate to business needs. IDOL scales to support the largest enterprise-wide and portal deployments in the world, with presence in virtually every vertical market. Since IDOL's scalability is based on its modular, distributed architecture, it can handle massive amounts of data on commodity dual-CPU servers. For instance, only a few hundred entry-level enterprise machines are required to support ChoicePoint's 10 billion record footprint. By comparison, a competitor uses 150,000 machines to handle the same amount of data.
A single IDOL engine can:
Support an estimated 30 million documents on 32-bit architectures and over 250 million on 64-bit platforms
Accurately index in excess of 60 GB/hour with guaranteed index commit times (i.e. how fast an asset can be queried after it is indexed) of sub 5ms
Execute over 2,000 queries per second, while querying the entire index for relevant information, with subsecond response times on a single machine with two CPUs when used against 30 million pieces of content
Support hundreds of thousands of enterprise users, or millions of web users, accessing hundreds of terabytes of data
Save storage space with an overall footprint of less than 30% of the original file size
This enhanced scalability results in hardware cost-savings as well as the ability to address larger volumes of content. Though IDOL scales extremely well on commodity servers, its flexible architecture can take full advantage of massive parallelism, SMP processing capabilities, 64-bit environments (such as Intel Itanium 64-bit architecture), software platforms (such as Solaris 10, Linux 64, Win64, etc), distributed server farms, and all common forms of external disk arrays (i.e. NAS, SAN etc) to further improve performance. This flexibility extends to being able to leverage one or a combination of these different environments.
How It Works
Content from various repositories is aggregated by connectors and then indexed into the IDOL Server or for dissemination across multiple IDOL Servers, through the Distributed Index Handler (DIH). The DIH can efficiently split and index copious quantities of data into multiple IDOL Server instances, optimizing performance by batching data, replicating all index commands and invoking dynamic load distribution. The DIH can perform data-dependent operations, such as distributing the content by date, which allows for more efficient querying. Performance is augmented by the Distributed Action Handler (DAH), a distribution server that allows the user to distribute action commands, such as querying, to IDOL Servers. Multiple copies of IDOL Servers, to which the DAH propagates actions, further ensure uninterrupted service in the event of server failure. For flexibility, both the DAH and the DIH can be configured to run in mirroring mode (IDOL Servers are exact copies of each other) and non-mirroring mode (each IDOL Server is configured differently and contains different data). In addition, the Distributed Service Handler (DiSH) component allows effective auditing, monitoring and alerting of all other Autonomy components.
Linear Scalability
Performance and capacity can be doubled by simply replicating the existing machine. This allows scaling predictions to be made without worry about bottlenecks.
Load Balancing
Data is automatically replicated across multiple servers and user requests are load-balanced across these replicas, guaranteeing performance, reducing latency and improving user-experience.
Mirroring / Failover
Automatically generated replicas are used to provide a pool of servers, the primary resource is automatically selected and the system switches to secondary systems if it fails so that service continues uninterrupted.
Distribution
For organizations that are geographically distributed, local replicas are automatically created and utilized where possible. Remote copies are only used when a local system fails, thereby building fault tolerance whilst maintaining the benefits of local performance and a reduction of resource overhead into a single, seamless service.
Adaptive Probabilistic Concept Caching
Frequently used concepts are maintained in memory and query results are returned as quickly and efficiently as possible.
Multi-dimensional Index & Query Throttling
By using a multi-dimensional index to provide valuable information to the distribution components, IDOL precludes bottlenecks and unbalanced peak loads during the indexing and query process.
Autonomy provides prioritized throttling based on:
Time: maximize index/query performance based on the time of day (i.e. work hours)
Location: prioritize activity based on the server landscape
Status: arbitrarily assign prioritized status for processing
"We have worked with Autonomy for a number of years due to their ability to offer a next-generation enterprise search platform that doesn't necessitate a trade-off between performance, security and scalability."
Mr. K. Sriram, Senior Vice President, Satyam Consulting and Enterprise Solutions Practice
Instruction-Level Parallelism
IDOL programmatically expresses itself as an expanding collection of operations. These operations can and are executed in serial pipeline form yet the inherent logic of simultaneously processing disparate forms of unstructured, semi-structured and structured data requires a high degree of parallelism. Not only does IDOL need to ingest multiple streams and types of data, it must also provide a real-time answer or decision against that data as it is indexed rather than force the user to wait an arbitrary period until serially accessed resources becomes available.
As a consequence IDOL has been designed with instruction-level parallelism (ILP) as the core of its process and operation model. ILP by definition is limited by the serial instruction model of scalar processors and thus Autonomy has been an extremely conscious early adopter of all forms of parallel architecture from multi-CPU, hyper-threading and now single die multi-core processing.
The engine's default process model is multi-threaded (using a configurable number of threads). IDOL operations can either be grouped by class, with indexing and querying performed by separate threads or for n-core models a single operation can be "atomized" into multiple threads. Concurrent querying and indexing is the default with no requirement whatsoever for "locking" any part of the indexes while querying takes place. All major multi-core manufacturers are supported, including Intel, AMD and the latest Niagara offerings from Sun Microsystems.
Classic scalar models that rely on Moore's predicted doubling of transistor density over 18 month intervals have already demonstrated wire and memory access latencies in addition to heat sealings. As a result, hardware manufacturers such as Intel have declared multi-core strategies as key to crossing the consumer "teraflop" threshold and aim to produce n-core 32 billion transistor die within the next 10 years. Autonomy is actively pursuing a Tera computing R&D simulation program in anticipation of increasing transistor and core density and the declared aim of such manufacturers. Autonomy is currently performing "coalition" simulations of split thread IDOL operations against n-core "battalion" processor units that blend general-purpose cores with more specialist cores such as those dedicated to signal processing. These blended core units are predicted to be the first consumer teraflop chips. Autonomy is developing process thread models that dynamically co-opt different core types to act in "coalition" to perform the simultaneous deconstruction and analysis of unstructured sources such as video that combine visual and auditory attributes.
Forthcoming Events for Security, Scalability and Performance
Summary: ...database backup to tape duration reduced by 13 percent on an average and up to 37 percent maximum. This results in a faster restore and additional application availability when required to restore from tape, as well as reduces load on the backup infrastructure. − Database query durations improved on...
Summary: ...needed to maintain full control over the firm’s information. Pre-set workspace templates and automated profiling ensure consistent organization and ease of search. A customized utility based on WorkSite Server with Caching reduces latency by pre-loading servers in satellite offices with documents for...
Summary: ...Clearwater Case Study. A customized utility based on WorkSite Server with Caching reduces latency by pre-loading servers in satellite offces with documents for better performance. WorkSite Communication Server auto-fles incoming and outgoing emails into addressable workspaces, reducing email overload...
Summary: ...and unstructured data located in systems from the acquired financial institution Benefits • Located and collected data from over 70,000 systems distributed geographically • Pre-Culled and deduplicated the over 500 TB of data • Provided a secure long-term archive environment that supports legal hold...
Summary: ...applications (i.e. Sigma, Documentum, PeopleSoft). • Adopt an online self-serve model where job applications are available for public submission 24x7. • Extend business hours with no direct labor and/or facility overhead. • Leverage Documentum content management and workflow system to support applicants’...
Summary: ...huge part of the way brown Rudnick works, and will make the new solution a powerful addition to our culture.” In addition, WorkSite’s collaborative features will help staff manage critical dates and resolve issues centrally, enabling distributed teams to securely share client case files, memoranda,...
Summary: ...program manager and technology lead on Athena. “K2’s taxonomy management capabilities and flexible APIs were important technical differentiators for us.” IDOL K2 provides lawyers at Freshfields with easy connectivity to more than 160,000 legal briefings, memoranda, forms, articles, practice notes,...
Summary: ...where a message might reside. Searches done in hours instead of days For the IT team, it’s much easier to produce messages on demand. “An area where HP Integrated Archive Platform has gone beyond expectations is the ease of use in doing searches and building queries,” Taylor says. “We have two...
Summary: ...the integrity of Dongkuk’s business and transaction data is protected, and should there be a service interruption, the data can be rapidly restored to ensure business continuity. In addition, because the software is designed to minimize impact on server CPU and I/O, it reduces the risk that back-up...
Summary: ...distributed authoring. “In a large organization you have to accept that you can’t have everything coming through one department because it causes a bottleneck. Yet, if you allow distributed authorship you must have workflow that controls approvals. TeamSite is quite effective at setting this up. It...
Summary: ...Roundarch: Cloud Data Protection - Simplified - Autonomy Case Study. POWER PROMOTE PROTECT Organization Roundarch Industry Consulting Solution Connected Backup and LiveVault Challenge: Tape-based backups were plaguing Roundarch with lengthy backup processes, and with data doubling year over year, backup...
Summary: ...any identical files stored on multiple machines, including email attachments, and backing up only one version. The data set is compressed further to reduce and optimize transmission and storage, after which it is encrypted to ensure secure transfer. The compression ratio means that the company is able...
This is a small selection of the Autonomy case studies available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...well on commodity servers, IDOL’s flexible architecture can further improve performance by taking full advantage of massive parallelism, SMP processing capabilities, 64-bit environments, software platforms (such as Solaris 10, Linux 64 and Win64), distributed server farms and all common forms of external...
Summary: ...modular architecture, it can handle massive amounts of data on commodity dual-CPU servers. IDOL delivers linear scalability through a multi-threaded, multi-instance approach with load-balancing to intelligently distribute the indexing and query workload. Mapped Security The biggest single constraint on...
Summary: ...LiquidBPM Engine is the core component of LiquidBPM. Its numerous features include caching, thread-pooling, persistence, clustering, load balancing, notification and scalability. The LiquidBPM Engine’s ability to automate processes improves efficiency and increases throughput by enforcing predictable...
Summary: ...user authentication, back-end entitlement checking and secure encrypted communication between the IDOL Server and its client applications with 128-bit Block Tiny Encryption Algorithm (BTEA). There are three security models offered within the market today – “Unmapped,” “Cached” and “Mapped”...
Summary: ...machines over the Internet or private network connection. Organizations can manage global backups from a single console with features such as grouping of servers for distributed administration, role-based administrations, and enterprise reporting. A simple agent can be installed on one or many machines...
Summary: ...are using virtualization to consolidate servers and reduce infrastructure costs. As the number of virtual machines grows, it becomes increasingly difficult to manage the backup process for virtual machines alongside the existing physical machines. A complicated backup management solution only increases...
Summary: ...minimizes the load on CPU processing and maximizes application availability. It uses as little as a tenth of the memory of competing solutions, which means it can be deployed on application or backup servers, and even virtual machines, without crippling performance.4 Data Protector software deduplication...
Summary: ...and recovery performance. HP StoreOnce’s thin, efficient footprint minimizes the load on CPU processing and maximizes application availability. It uses as little as a tenth of the memory of other available solutions, which means it can be deployed on application or backup servers, and even virtual machines,...
Summary: ...Stack App App Xen Hypercall API Xen Hypervisor Hardware Guest OS Guest OS Application Application Operating System Virtualization Layer Host Operating System Intel®Architecture CPU Memory NIC Disk MS Virtual Server app 1 app 2 HP‑UX 11i v2 Linux RH4 app 1 Windows app 1 app 1 Linux RH4 Windows app 1...
Summary: ...Today, more and more organizations are leveraging c.... Today, more and more organizations are leveraging cloud-based backup to lower costs and reduce the administrative overhead of managing the server backup process. Complex, traditional tape-based backup solutions often increase the burdens on IT staff,...
Summary: ...natural language query. Multiple data formats are available in a single folder, and results are downloadable for offline viewing. Enterprise Class Security Autonomy is the only vendor that can scale security to large numbers of users with complex repository security. Autonomy’s technology employs a...
Summary: ...Query (IQ) to form an understanding of the information on a user's screen, and proactively link users to related information on their secure corporate network, on the web or their local machine, entirely automatically. Implicit query constantly reads the information on the user’s desktop and dynamically...
This is a small selection of the Autonomy Product Briefs available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...AMD and the latest Niagara offerings from Sun Microsystems. Classic scalar models that rely on Moore’s predicted doubling of transistor density over 18 months intervals have already demonstrated wire and memory access latencies in addition to heat sealing’s. As a result, hardware manufactures such...
Summary: ...and a reduction of resource overhead into a single, seamless service. • Load Balancing: Data should automatically be replicated across multiple servers and user requests should be load balanced across these replicas, guaranteeing performance, reducing latency and improving user-experience. • Mirroring/...
Summary: ...application requirements, it is possible to have multiple instances of the Cardiff LiquidBPM Engine running as a cluster to distribute load and scale up. When used in this fashion, one can distribute load across multiple engines and also automatically failover across engines. In addition to horizontal...
Summary: ...i.e. N+1, N+M, Active/Passive, Active/ Active). Load Balancing The Autonomy infrastructure facilitates the use and management of dynamically growing data load. Data is automatically replicated across multiple servers and user requests are load balanced across these replicas, guaranteeing performance,...
Summary: ...is that mapped security mitigates this security challenge by respecting third-party security entitlements without either caching or using lastminute query time checks against the native repository. 6 Enterprise Search: Addressing Security and Entitlement Issues Ability to Choose Autonomy recommends mapped...
Summary: ...Distributed Environments EAS has a unique Parent/Child architecture that allows the archiving process and the archive service to be established in the optimum locations after such factors as network bandwidth, CPU consumption, anticipated demand, and archive policy have been accounted for.
...
Summary: ...CPU hardware—same as above (1GB of RAM dedicated to each WebLogic server) Database server hardware and software configuration n Database: Oracle 10g, deployed on a separate DB server machine n Database server OS:Windows 2003 Server n CPU: Intel Xeon 3GHz, 4 processors, 2 GB RAM III. Benchmark Results...
Summary: ...described below: 2.5.1 Large vocabulary recognition Using patented predictive technology the VoiceSuite module is able to provide the benefits of a large vocabulary speech recognition system without the overhead of a vast search space when considering sample audio. Figure 2: Time-first hypothosis extension...
Summary: ...through results encryption and image watermarking. Full audit tracking enables authorized personnel to gain immediate visibility into the history associated with any record in the system. Intellectual Asset Protection System (IAS) Virage’s Intellectual Asset Protection System (IAS) is a comprehensive...
Summary: ...and load balancing capabilities for reduced IT overhead and improved performance Architectural Improvements • Distributed link architecture • Intelligent repository storage • Distributed hot deployment mode • Distributed abridged mode 6 The Power of Autonomy IDOL these types of procedures can...
Summary: ...searched as easily as text Understands words in context for unrivalled accuracy Patented search technology drastically reduces CPU and memory requirements Automatic customization via additional text material Fixed-latency real-time mode for media monitoring operations Speech-To-Text Autonomy’s speech-to-text...
Summary: ...covering all of the repositories IDOL supports. Autonomy provides all aspects of security management including document and intradocument access control against user, group and role level entitlement. Encrypted inter-machine and intra-process communication protocols are woven into the fabric of Autonomy's...
This is a small selection of the Autonomy White Papers available, please visit our publications site at http://publications.autonomy.com/ for more information.
There do not seem to be any press releases related to this page in 2013 at the moment, please visit the news section on www.autonomy.com for the latest news.
Relevance
Date
Press Release
There do not seem to be any press releases related to this page in 2012 at the moment, please visit the news section on www.autonomy.com for the latest news.
Related Case Studies
RSS Feeds
Find us on Facebook
Follow us on Twitter
See us on YouTube
Find us on LinkedIn
