The world's largest and most secure intelligence organizations have deployed Autonomy's Intellectual Asset Protection System (IAS) to safeguard their most sensitive information assets. Autonomy provides all aspects of security management, including front-end user authentication, back-end entitlement checking and secure encrypted communication between the IDOL Server and its client applications with 128-bit Block Tiny Encryption Algorithm (BTEA). IDOL's mapped security model is the only empirically proven index security model that scales in the enterprise.
There are three general security models currently available:
1. Unmapped Security
Unmapped security is the traditional method used by source repositories and search engines. For every potential match to a given query, a call is made via the native repository's API (e.g. Documentum) to ascertain the access privileges for that particular document. A single query consequently bombards the native repository with document privilege requests as the retrieval system attempts to assemble a relevant results list from thousands of candidate hits. This method presents significant performance and scalability problems.
Unmapped Security
Mapped Security
"Security is a key differentiator for IDOL. IDOL offers "mapped security" and near real-time synchronization of security entitlements with source content repositories - making it a great fit for highly secure search scenarios"
The Forrester WaveTM: Enterprise Search Platforms, Matthew Brown
Autonomy recommends mapped security but also offers the choice between mapped, unmapped and a hybrid of both. Autonomy also supplies plug-in sample code, so that customers, OEMs and partners are able to develop and implement their own form of security plug-in.
2. Cached Security
Cached security is the method of choice for legacy systems. Cached security marginally relieves the scalability problem of unmapped security by storing results for queries it has already seen. Consequently, when a user repeats a query, the result set can be retrieved from the cache rather than triggering a network-mediated request. However, this approach still relies on calling out across the network directly to the repository for each new query. In addition, it also misses potential results, as the result sets stored within its memory do not dynamically update new information.
3. Autonomy's Unique IAS Mapped Security
Only Autonomy offers mapped security - a highly configurable, secure, accurate, and fast method for respecting third party security entitlements. IDOL maps the underlying security model in the form of ACL, group, role, protective markings, etc. from all of the underlying repositories directly into the kernel of the IDOL engine itself, and stores the information in an encrypted field. As a result, IDOL does not need to send any requests across the network to the data stores when building up a results list. What the user is allowed to see is assessed "inline" within the IDOL kernel at speeds that exceed the response times of the native repository. Unlike other techniques, the security model is never out of date as the transitional signaling mechanism within the connector layer informs IDOL in real-time of any updates or changes to permissions within the underlying content.
Since IDOL's architecture is inherently modular by design, it requires multiple subsystems to communicate with each other, often across insecure networks. All communication between these processes may be encrypted (Secure Sockets Layer), so that packet sniffers who are able to break past a firewall are unable to read the content of traffic between IDOL modules. All of the system's modules are capable of operating in a secure communications mode providing, at minimal processing overhead, the protection of 128-bit encryption. Additionally, IDOL can leverage SSL for both aggregation and querying of content, including access to SSL encrypted sites.
"One factor that has set the Autonomy search apart from the crowd is security. Whatever security exists on the application layer," she says, "Autonomy acknowledges it."
Carol Fineagan, CIO of EnergySolutions, CIO Magazine, July 2008
Scalability and Performance
The management of structured and unstructured content requires a platform that can meet the most rigorous performance requirements and be easily resized commensurate to business needs. IDOL scales to support the largest enterprise-wide and portal deployments in the world, with presence in virtually every vertical market. Since IDOL's scalability is based on its modular, distributed architecture, it can handle massive amounts of data on commodity dual-CPU servers. For instance, only a few hundred entry-level enterprise machines are required to support ChoicePoint's 10 billion record footprint. By comparison, a competitor uses 150,000 machines to handle the same amount of data.
A single IDOL engine can:
Support an estimated 30 million documents on 32-bit architectures and over 250 million on 64-bit platforms
Accurately index in excess of 60 GB/hour with guaranteed index commit times (i.e. how fast an asset can be queried after it is indexed) of sub 5ms
Execute over 2,000 queries per second, while querying the entire index for relevant information, with subsecond response times on a single machine with two CPUs when used against 30 million pieces of content
Support hundreds of thousands of enterprise users, or millions of web users, accessing hundreds of terabytes of data
Save storage space with an overall footprint of less than 30% of the original file size
This enhanced scalability results in hardware cost-savings as well as the ability to address larger volumes of content. Though IDOL scales extremely well on commodity servers, its flexible architecture can take full advantage of massive parallelism, SMP processing capabilities, 64-bit environments (such as Intel Itanium 64-bit architecture), software platforms (such as Solaris 10, Linux 64, Win64, etc), distributed server farms, and all common forms of external disk arrays (i.e. NAS, SAN etc) to further improve performance. This flexibility extends to being able to leverage one or a combination of these different environments.
How It Works
Content from various repositories is aggregated by connectors and then indexed into the IDOL Server or for dissemination across multiple IDOL Servers, through the Distributed Index Handler (DIH). The DIH can efficiently split and index copious quantities of data into multiple IDOL Server instances, optimizing performance by batching data, replicating all index commands and invoking dynamic load distribution. The DIH can perform data-dependent operations, such as distributing the content by date, which allows for more efficient querying. Performance is augmented by the Distributed Action Handler (DAH), a distribution server that allows the user to distribute action commands, such as querying, to IDOL Servers. Multiple copies of IDOL Servers, to which the DAH propagates actions, further ensure uninterrupted service in the event of server failure. For flexibility, both the DAH and the DIH can be configured to run in mirroring mode (IDOL Servers are exact copies of each other) and non-mirroring mode (each IDOL Server is configured differently and contains different data). In addition, the Distributed Service Handler (DiSH) component allows effective auditing, monitoring and alerting of all other Autonomy components.
Linear Scalability
Performance and capacity can be doubled by simply replicating the existing machine. This allows scaling predictions to be made without worry about bottlenecks.
Load Balancing
Data is automatically replicated across multiple servers and user requests are load-balanced across these replicas, guaranteeing performance, reducing latency and improving user-experience.
Mirroring / Failover
Automatically generated replicas are used to provide a pool of servers, the primary resource is automatically selected and the system switches to secondary systems if it fails so that service continues uninterrupted.
Distribution
For organizations that are geographically distributed, local replicas are automatically created and utilized where possible. Remote copies are only used when a local system fails, thereby building fault tolerance whilst maintaining the benefits of local performance and a reduction of resource overhead into a single, seamless service.
Adaptive Probabilistic Concept Caching
Frequently used concepts are maintained in memory and query results are returned as quickly and efficiently as possible.
Multi-dimensional Index & Query Throttling
By using a multi-dimensional index to provide valuable information to the distribution components, IDOL precludes bottlenecks and unbalanced peak loads during the indexing and query process.
Autonomy provides prioritized throttling based on:
Time: maximize index/query performance based on the time of day (i.e. work hours)
Location: prioritize activity based on the server landscape
Status: arbitrarily assign prioritized status for processing
"We have worked with Autonomy for a number of years due to their ability to offer a next-generation enterprise search platform that doesn't necessitate a trade-off between performance, security and scalability."
Mr. K. Sriram, Senior Vice President, Satyam Consulting and Enterprise Solutions Practice, 2007
Instruction-Level Parallelism
IDOL programmatically expresses itself as an expanding collection of operations. These operations can and are executed in serial pipeline form yet the inherent logic of simultaneously processing disparate forms of unstructured, semi-structured and structured data requires a high degree of parallelism. Not only does IDOL need to ingest multiple streams and types of data, it must also provide a real-time answer or decision against that data as it is indexed rather than force the user to wait an arbitrary period until serially accessed resources becomes available.
As a consequence IDOL has been designed with instruction-level parallelism (ILP) as the core of its process and operation model. ILP by definition is limited by the serial instruction model of scalar processors and thus Autonomy has been an extremely conscious early adopter of all forms of parallel architecture from multi-CPU, hyper-threading and now single die multi-core processing.
The engine's default process model is multi-threaded (using a configurable number of threads). IDOL operations can either be grouped by class, with indexing and querying performed by separate threads or for n-core models a single operation can be "atomized" into multiple threads. Concurrent querying and indexing is the default with no requirement whatsoever for "locking" any part of the indexes while querying takes place. All major multi-core manufacturers are supported, including Intel, AMD and the latest Niagara offerings from Sun Microsystems.
Classic scalar models that rely on Moore's predicted doubling of transistor density over 18 month intervals have already demonstrated wire and memory access latencies in addition to heat sealings. As a result, hardware manufacturers such as Intel have declared multi-core strategies as key to crossing the consumer "teraflop" threshold and aim to produce n-core 32 billion transistor die within the next 10 years. Autonomy is actively pursuing a Tera computing R&D simulation program in anticipation of increasing transistor and core density and the declared aim of such manufacturers. Autonomy is currently performing "coalition" simulations of split thread IDOL operations against n-core "battalion" processor units that blend general-purpose cores with more specialist cores such as those dedicated to signal processing. These blended core units are predicted to be the first consumer teraflop chips. Autonomy is developing process thread models that dynamically co-opt different core types to act in "coalition" to perform the simultaneous deconstruction and analysis of unstructured sources such as video that combine visual and auditory attributes.
Summary: ...the Vertica Database, Harvell and his team conducted extensive performance testing of the cluster. The test included a snapshot of their application running on a five-node cluster of inexpensive servers with 4 CPU AMD 2.6 GHz core processors with 64-bit 1 MB cache; 8 GB RAM; and ~750 GBs of usable space...
Summary: ...across the globally distributed organization ■ Inconsistent, ad hoc profi ling made content difficult to find ■ Large volumes of unmanaged email content caused email overload, undermined centralized management, and increased risk Benefits of Using Interwoven ■ Remote access and caching capabilities...
Summary: ...organization and search; version control to capture the complete evolution of investment models; comprehensive security and auditability to reduce risk; and reduced latency through content caching.
...
Summary: ...applications (i.e. Sigma, Documentum, PeopleSoft). • Adopt an online self-serve model where job applications are available for public submission 24x7. • Extend business hours with no direct labor and/or facility overhead. • Leverage Documentum content management and workflow system to support applicants’...
Summary: ...to our culture.” In addition, WorkSite’s collaborative features will help staff manage critical dates and resolve issues centrally, enabling distributed teams to securely share client case files, memoranda, attorneys’ notes, correspondence, tasks, and events. Instant Access for Mobile Workers To...
Summary: ...Innovations 1. Column store architecture 2. Aggressive compression 3. Concurrent load and query 4. Automatic database design 5. High availability without hardware redundancy 6. Runs on commodity hardware 7. Scale by adding inexpensive servers The company is also seeing big performance advantages with...
Summary: ...Vertica Case Study: hMetrix. In fact, we often ran it overnight, and still took several minutes when we upgraded to more powerful hardware running MySQL. That’s how shockingly fast Vertica is.” hMetrix’ standard system is a dual-socket, dual-core x86-64 CPU with 32 GB of RAM and one TB of disk space,...
Summary: ...are achieving a four-to-one compression ratio when moving data from our OLTP platform to Vertica. Finally, since customers pay by the terabyte with Vertica, there’s no extra cost for adding more CPUs, or for making hardware changes, or for building-in replication, high availability or disaster recovery....
Summary: ...Roundarch: Cloud Data Protection - Simplified - Autonomy Case Study. POWER PROMOTE PROTECT Organization Roundarch Industry Consulting Solution Connected Backup and LiveVault Challenge: Tape-based backups were plaguing Roundarch with lengthy backup processes, and with data doubling year over year, backup...
Summary: ...in Australia and Southeast Asia, including ABN Amro, AON Risk Services Australia/New Zealand, Commonwealth Bank of Australia, and St. George Bank. The company’s appliances were built from the ground up using modern distributed software architectures, Web interfaces, commodity hardware and open-source...
Summary: ...Information Technology The Application Web 2.0 application for digital content archiving The Benefits • Fast, simple and cost-effective storage and ad hoc querying of terabytes (and eventually petabytes) of customer data • Scales easily, by adding low-cost commodity servers • Ease of integration...
Summary: ...distributed authoring. “In a large organization you have to accept that you can’t have everything coming through one department because it causes a bottleneck. Yet, if you allow distributed authorship you must have workflow that controls approvals. TeamSite is quite effective at setting this up. It...
This is a small selection of the Autonomy case studies available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...well on commodity servers, IDOL’s flexible architecture can further improve performance by taking full advantage of massive parallelism, SMP processing capabilities, 64-bit environments, software platforms (such as Solaris 10, Linux 64 and Win64), distributed server farms and all common forms of external...
Summary: ...user authentication, back-end entitlement checking and secure encrypted communication between the IDOL Server and its client applications with 128-bit Block Tiny Encryption Algorithm (BTEA). There are three security models offered within the market today – “Unmapped,” “Cached” and “Mapped”...
Summary: ...disk and memory I/O. Heavy use of compression – Store data in 90% less table space CPUs are getting faster at a much greater rate than disk bandwidth is increasing, so Vertica replaces slower disk I/O with faster CPU cycles to encode data elements into a more compact form and query them. Vertica’s...
Summary: ...or many machines using a mass deployment script and template; agent upgrades are automatic. Data is transmitted – without manual intervention – via the Secure Sockets Layer (SSL) protocol, and encrypted at the source using 256-bit AES encryption with a unique key only accessible by its associated...
Summary: ...using a mass deployment script and template; agent upgrades are automatic. Data is transmitted—without manual intervention—via the Secure Sockets Layer (SSL) protocol, and encrypted at the source using 256-bit AES encryption with a unique key only accessible by its associated organization. LiveVault...
Summary: ...A market-leading secure, scalable cloud service Today, more and more organizations are moving to a cloud backup solution in order to lower costs and reduce the administrative overhead of managing the server backup process. A complicated backup solution only increases the burden on IT staff and limits...
Summary: ...largest enterprise-wide and portal deployments in the world, with presence in virtually every vertical market. Since IDOL’s scalability is based on its modular architecture, it can handle massive amounts of data on commodity dual-CPU servers. IDOL delivers linear scalability through a multi-threaded,...
Summary: ...LiquidBPM Engine is the core component of LiquidBPM. Its numerous features include caching, thread-pooling, persistence, clustering, load balancing, notification and scalability. The LiquidBPM Engine’s ability to automate processes improves efficiency and increases throughput by enforcing predictable...
Summary: ...with other Virage solutions or third-party systems specifications Minimum Configuration SmartEncode: Single Intel Pentium system equipped with 1 GHz CPU, 512MB RAM, 2 available PCI slots, Windows XP, 2000 or 2003 operating systems Advanced audio analysis: Dual Intel Pentium system equipped with 1 GHz...
Summary: ...Requirements Platforms Supported: • Microsoft Windows NT, Windows 2000 & XP • Linux • Sun Solaris • Any other POSIX compliant of UNIX on request Minimum Recommended Server Specifications: • Pentium CPU (Intel or Intel Compatible Processor) • 128 MB RAM J2EE-compliant application server support:...
Summary: ...recovery (DeltaRestore™) - TurboRestore Appliance™ onsite appliance (optional) - Proactive administration, monitoring, and notification - Data Shuttle Service for seeding and large data restores - Native, embedded data de-duplication - Encryption key escrow Benefits - Achieves cloud data protection...
Summary: ...Vertica Analytics Platform Release 5.0 Overview. ? B/R transfers substantially smaller data volumes than in prior releases. ? Capturing a backup snapshot of a live database results in lesser CPU processing overhead; use of encryption during data transfers is optional. ? The B/R operations are atomic and...
This is a small selection of the Autonomy Product Briefs available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...bottleneck, allowing scaling to support many thousands of queries per second and millions of users. • All of Autonomy’s modules take advantage of faster processors, symmetric multi-processing (SMP) capabilities, 64-bit environments (such as Intel Itanium 64-bit architecture) and software platforms...
Summary: ...and a reduction of resource overhead into a single, seamless service. • Load Balancing: Data should automatically be replicated across multiple servers and user requests should be load balanced across these replicas, guaranteeing performance, reducing latency and improving user-experience. • Mirroring/...
Summary: ...CPU hardware—same as above (1GB of RAM dedicated to each WebLogic server) Database server hardware and software configuration n Database: Oracle 10g, deployed on a separate DB server machine n Database server OS:Windows 2003 Server n CPU: Intel Xeon 3GHz, 4 processors, 2 GB RAM III. Benchmark Results...
Summary: ...processing. The concept behind MPP is that processors are tied to certain pieces of the data, the data is evenly distributed across the nodes and the number of processors can expand almost without limit, using relatively inexpensive, nonproprietary components. In summary, the Vertica Analytical Database...
Summary: ...Distributed Environments EAS has a unique Parent/Child architecture that allows the archiving process and the archive service to be established in the optimum locations after such factors as network bandwidth, CPU consumption, anticipated demand, and archive policy have been accounted for.
...
Summary: ...to configure the cache, and control aspects of its behavior. • Thread pooling. Threads are used by the Cardiff LiquidBPM Engine to analyze workfows. They are also used to execute tasks in a workfow that can be done concurrently. Threads tend to be an expensive computational resource. The engine utilizes...
Summary: ...versus manual partitioning, the bottom line seems to be that even on the new 64 bit architecture with 4 screaming Intel processors, and SQL 5 - the upper limit of MOSS’ content repository is 500GB. Assuming allocation for index data and other overhead this really allows less then 20 million objects...
Summary: ...a TCP/IP network. Nodes contain commodity, multi-core processors with 2 to 4 GB of RAM per core. Storage can be directly attached to each node, or can be SAN-based. In the Vertica Analytic Database, parallelism is optimized for star or snowflake data models. Fact tables are range partitioned across the...
Summary: ...a multi-threaded, multi-instance approach with loadbalancing to distribute the indexing and query workload. K2 v7 supports 64-bit architecture. Essentially, there is no document limit in K2 v7. Hundreds of servers can be linked together in a modular, distributed and replicated environment. This enhanced...
Summary: ...interface, but they are subject to the vicissitudes of source-system performance, network bottlenecks and dirty data. Finally, many BI tools maintain a local cache or an in-memory database to optimize query performance. Often, these caches can be updated incrementally at scheduled intervals, which can...
Summary: ...described below: 2.5.1 Large vocabulary recognition Using patented predictive technology the VoiceSuite module is able to provide the benefits of a large vocabulary speech recognition system without the overhead of a vast search space when considering sample audio. Figure 2: Time-first hypothosis extension...
Summary: ...through results encryption and image watermarking. Full audit tracking enables authorized personnel to gain immediate visibility into the history associated with any record in the system. Intellectual Asset Protection System (IAS) Virage’s Intellectual Asset Protection System (IAS) is a comprehensive...
This is a small selection of the Autonomy White Papers available, please visit our publications site at http://publications.autonomy.com/ for more information.
There do not seem to be any press releases related to this page in 2012 at the moment, please visit the news section on www.autonomy.com for the latest news.
Relevance
Date
Press Release
There do not seem to be any press releases related to this page in 2011 at the moment, please visit the news section on www.autonomy.com for the latest news.
Related Case Studies
RSS Feeds
