Share
Embed
Making the Content Conduit Faster: Search @ Emerson
For Emerson Process Management, Elasticsearch became the cornerstone technology and changed the game for indexing, searching, and retrieving critical documentation of customer quotes and orders. Indexing changed from months to hours, searches that took 40-50 seconds are now done in sub-seconds, and unhappy users are now ecstatic when retrieving critical customer order documentation.
Mark Heindselman | Elastic{ON} Tour Chicago | November 19, 2015
Elastic Co
November 19, 2015
Transcript
25 slides
-
Distributed Elasticsearch at Emerson Process Management Mark Heindselman, Director of
Knowledge Network and Information Services | Emerson Process Management 2 -
Agenda • What is a GAME CHANGER! • Who is
Emerson Process Management • Who is Redstone Content Solutions • Distributed Elastic Search Implementation Overview • The Emerson Process Management Story o Environment, Problem & Impact o Solution & Demo o Value o Future • Wrap Up 4 -
What is a GAME CHANGER! A newly introduced element or
factor that changes an existing situation or activity in a significant way 5 -
Emerson Process Management • A part of Emerson Electric •
35,000 Employees • Global Manufacturing locations • $10B Revenue • Leading Supplier of Products & Services for the Process Automation Industry ▪ Process Control Systems & Software ▪ Measurement & Analytical Instruments o Analytical, Flow, Level, Pressure, Temperature ▪ Control Valves & Regulators 6 -
Redstone Content Solutions Oracle Platinum Partner - WebCenter focus ▪
VAR & SI ▪ Public Sector Addendum ▪ Employee Staff that Started Stellent Partner Program Premier product training ▪ 3T™ Methodology ▪ Oracle Certified Training Delivery Partner Technical competency ▪ Java-Certified Developers ▪ PMP-Certified PMs 7 -
Distributed Elastic Search (DES) Overview • Elasticsearch search engine •
Redstone Content Solutions – Distributed Index Integration to Oracle WebCenter Content • DES is a Game Changing WebCenter Content Searching and Indexing Solution ▪ Oracle Validated Integration • WebCenter Content / Elasticsearch integration ▪ Indexing – Multiple indices definable o Rules-based and highly configurable o Indexing Engine is engineered for speed ▪ Searching o Intercept calls to GET_SEARCH_RESULTS service o Reroutes conditional queries to separate indices o Returns the result of the search to the WebCenter Results Interface 8 -
The Enterprise Content Management Global Environment Instance Name Size
# of Docs # of Contributors # of Users eDocs Asia 0.25 TB 0.250M 100 250 I eDocsDB 7.00 TB 7.000M programmatic 1000 I eDocs Europe 1.50 TB 1.250M 200 2000 I eDocs Eur Ext 1.00 TB 1.000M programmatic 5000 E eDocs/FishWeb 1.50 TB 1.650M 1500 & prog 6000 I eDocs Ext NA 1.80 TB 3.100M programmatic 60000 E eDocsReports 0.25 TB 0.200M programmatic 200 I MMIDocs 0.25 TB 0.200M 100 & prog 200 I OCM 6.00 TB 18.900M 1300 & prog 8000 Total 19.55 TB 33.55M 3200 & prog 65K E 11.6K I 9 E = external users I = internal users -
OCM Instance Environnent WebCenter Content Environment ▪ 10gR3 ▪ Physical
Server ▪ Dedicated SQL Server 10 Sales Order/Quote Repository ▪ 15,000,000 records ▪ 5 TB metadata index ▪ User and programmatic filing: o Check in Screens o WebDAV Drag & Drop from Desktop or from Outlook o eMail Listener o Oracle EBS o And Other Business Systems ▪ 150,000+ check-ins/month -
Our Problems Slow Searching – Metadata Index ▪ 45 seconds
for a search across the whole database ▪ 12 to 15 second for a search across 1M records Slow Indexing ▪ Months to Re-index o Started Dec 23 Completed Mar 26 -
Impact Dissatisfied users Long-running queries Increasing # of tickets for
slow searching High database utilization - impacted document loading Unable to add database fields because of slow index rebuild speed 12 -
We needed a solution that meet Our Objectives No UI
modifications No security group modifications No database architecture modifications 13 Our Goals Improve searching by 75% Improve index rebuild speed by 90% Improve user satisfaction -
Redstone’s Distributed Elastic Search Integration Single Master Index No change
for end users ▪ Same search and retrieval interface and results screens DES Integration with WebCenter ▪ Intercepts search prior to being passed SQL database ▪ Runs search against DES Index ▪ Runs multiple term searches simultaneously o SQL runs sequential searches for each term ▪ Passes the Search Results back to WebCenter results pages for display 14 -
The Results -‐ FAST Search Speed – Searching
all content with multiple terms in < 2 seconds – 1,000,000 records < 1 second Index Rebuild Speed – From 3 months to 9 hours for 17,000,000 records USERS ARE EXSTATIC! -
Value -‐ Distributed Elasticsearch Objectives No UI modifications
No security group modifications No database architecture modifications Goals Improve searching by 75% à 99% Improve index rebuild speed by 90% à 99% Improved WebCenter user satisfaction 17 -
950% Return On Investment WebCenter Usage –
6,000 employees – 1,440,000 searches/month – 17,280,000 searches/year – $40/hr (average fully loaded FTE) Time Saved in Doing Searches – 11 Seconds per Search – 2,247 Hours / Rebuild What that adds up to: – 17,280,000 searches – x 11 seconds – = 190,080,000 seconds – = 52,800 hours saved – x $40/hr – = $2,112,000 annual savings • Satisfied users • Mission Critical Application that the users can’t do with out! 18 -
Features/Benefits -‐ In-‐use 99% improved search and index rebuild
speeds Supports unlimited number of content items/search terms Rebuild segments of an index Create unlimited number of indices Create/rebuild search indices without restarting content server Minimal user impact during index rebuild Easy implementation and configuration Horizontal/vertical scalability for extreme performance 19 -
Features/Benefits -‐ In-‐use Support for distributed architecture for indices
and hardware Real-‐time feedback—estimates remaining index rebuild time Search remains operational during index rebuild Multi-‐threaded rebuild process Maintains current out-‐of-‐the-‐box index Add new documents/revisions or delete content during rebuilds Live reload—real-‐time tuning of the index rebuild process 20 -
Features/Benefits – Not In Use Yet Clustering Create/maintain
multiple indices based on content/metadata Index priority for documents matching multiple indices Ability to simultaneously rebuild multiple indices Flexible faceted search results – End users can filter search results – End users can control facets per search or saved search – Administrators can create different search facets per profile 21 -
Future of DES at Emerson We are or will be
use DES to index ALL our WebCenter Content instances In-‐production – OCM (18M items) metadata-‐only In-‐process – eDocs / Fishweb (2M items) GSA full-‐text – eDocsDB (5M items) metadata-‐only Future – eDocs Extranet (3M items) full-‐text – MMI Docs (200,000 items) full-‐text – eDocs Europe (2M items) full-‐text – eDocs Asia (300,000 items) full-‐text End-‐Game – All WebCenter Content Instances with a Master Index (32M items) to search them all at once 22 -
Future of DES at Emerson We are in discussions on
how we can use DES solution for eDiscovery purposes on – File Servers – SharePoint 23 -
Wrap Up & Summary • We Had a Major Indexing
and Searching Problems • Redstone Developed an Integration of Elastic Search with the Oracle WebCenter Content • The Integration required NO Changes for how the users were use to using the system • The solution was Game Changing for the users with Searching – it is FAST • The solution was Game Changing for the administrators with the Indexing - it is FAST • The Value of the Solution can be Quantified – in real time and money • The Value of the Solution is measured by the Users Response – Ecstatic, Satisfied, See the Application as Critical • Big Future for DES at Emerson Process Management 24 -
Customer Integration Solution & Technology With Problem Implementation Partner Solution
Mark Heindselman [email protected] 641 754-‐3161 John Klein [email protected] (563) 424-‐7563 [email protected]