Making the Content Conduit Faster: Search @ Emerson

For Emerson Process Management, Elasticsearch became the cornerstone technology and changed the game for indexing, searching, and retrieving critical documentation of customer quotes and orders. Indexing changed from months to hours, searches that took 40-50 seconds are now done in sub-seconds, and unhappy users are now ecstatic when retrieving critical customer order documentation.

Mark Heindselman | Elastic{ON} Tour Chicago | November 19, 2015

Transcript

2. Distributed Elasticsearch at Emerson Process Management
 Mark Heindselman, Director of Knowledge Network and Information Services | Emerson Process Management 2
3. Customer Integration Solution & Technology
 With Problem Implementation Partner Solution 3
4. Agenda • What is a GAME CHANGER! • Who is Emerson Process Management • Who is Redstone Content Solutions • Distributed Elastic Search Implementation Overview • The Emerson Process Management Story o Environment, Problem & Impact o Solution & Demo o Value o Future • Wrap Up 4
5. What is a GAME CHANGER! A newly introduced element or factor that changes an existing situation or activity in a significant way 5
6. Emerson Process Management • A part of Emerson Electric • 35,000 Employees • Global Manufacturing locations • $10B Revenue • Leading Supplier of Products & Services for the Process Automation Industry ▪ Process Control Systems & Software ▪ Measurement & Analytical Instruments o Analytical, Flow, Level, Pressure, Temperature ▪ Control Valves & Regulators 6
7. Redstone Content Solutions Oracle Platinum Partner - WebCenter focus ▪ VAR & SI ▪ Public Sector Addendum ▪ Employee Staff that Started Stellent Partner Program Premier product training ▪ 3T™ Methodology ▪ Oracle Certified Training Delivery Partner Technical competency ▪ Java-Certified Developers ▪ PMP-Certified PMs 7
8. Distributed Elastic Search (DES) Overview • Elasticsearch search engine • Redstone Content Solutions – Distributed Index Integration to Oracle WebCenter Content • DES is a Game Changing WebCenter Content Searching and Indexing Solution ▪ Oracle Validated Integration • WebCenter Content / Elasticsearch integration ▪ Indexing – Multiple indices definable o Rules-based and highly configurable o Indexing Engine is engineered for speed ▪ Searching o Intercept calls to GET_SEARCH_RESULTS service o Reroutes conditional queries to separate indices o Returns the result of the search to the WebCenter Results Interface 8
9. The Enterprise Content Management Global Environment 
 Instance  Name Size #  of  Docs #  of  Contributors #  of  Users eDocs  Asia 0.25  TB  0.250M 100 250  I eDocsDB 7.00  TB  7.000M programmatic 1000  I eDocs  Europe 1.50  TB  1.250M 200 2000  I eDocs  Eur  Ext 1.00  TB  1.000M programmatic 5000  E eDocs/FishWeb 1.50  TB  1.650M 1500  &  prog 6000  I eDocs  Ext  NA 1.80  TB  3.100M programmatic 60000  E eDocsReports 0.25  TB  0.200M programmatic 200  I MMIDocs 0.25  TB  0.200M 100  &  prog 200  I OCM 6.00  TB 18.900M 1300  &  prog 8000 Total 19.55  TB 33.55M 3200  &  prog 65K  E    11.6K  I 9 E  =  external  users      I  =  internal  users
10. OCM Instance Environnent WebCenter Content Environment ▪ 10gR3 ▪ Physical Server ▪ Dedicated SQL Server 10 Sales Order/Quote Repository ▪ 15,000,000 records ▪ 5 TB metadata index ▪ User and programmatic filing: o Check in Screens o WebDAV Drag & Drop from Desktop or from Outlook o eMail Listener o Oracle EBS o And Other Business Systems ▪ 150,000+ check-ins/month
11. Our Problems Slow Searching – Metadata Index ▪ 45 seconds for a search across the whole database ▪ 12 to 15 second for a search across 1M records Slow Indexing ▪ Months to Re-index o Started Dec 23 Completed Mar 26
12. Impact Dissatisfied users Long-running queries Increasing # of tickets for slow searching High database utilization - impacted document loading Unable to add database fields because of slow index rebuild speed 12
13. We needed a solution that meet Our Objectives No UI modifications No security group modifications No database architecture modifications 13 Our Goals Improve searching by 75% Improve index rebuild speed by 90% Improve user satisfaction
14. Redstone’s Distributed Elastic Search Integration Single Master Index No change for end users ▪ Same search and retrieval interface and results screens DES Integration with WebCenter ▪ Intercepts search prior to being passed SQL database ▪ Runs search against DES Index ▪ Runs multiple term searches simultaneously o SQL runs sequential searches for each term ▪ Passes the Search Results back to WebCenter results pages for display 14
15. The  Results  -­‐  FAST Search  Speed     – Searching    all  content    with  multiple     terms  in  <  2  seconds   – 1,000,000  records  <  1  second   Index  Rebuild  Speed       – From  3  months  to  9  hours  for   17,000,000  records   USERS  ARE  EXSTATIC!
17. Value  -­‐  Distributed  Elasticsearch Objectives   No  UI  modifications   No  security  group  modifications   No  database  architecture  modifications Goals     Improve  searching  by  75%  à 99%   Improve  index  rebuild  speed  by  90%  à 99%       Improved  WebCenter  user  satisfaction   17
18. 950%  Return  On  Investment   WebCenter  Usage     – 6,000  employees   – 1,440,000  searches/month   – 17,280,000  searches/year   – $40/hr  (average  fully  loaded  FTE)   Time  Saved  in  Doing  Searches   – 11  Seconds    per  Search     – 2,247  Hours  /  Rebuild   What  that  adds  up  to:     – 17,280,000  searches     – x  11  seconds       – =  190,080,000  seconds     – =  52,800  hours  saved   – x  $40/hr     – =  $2,112,000  annual  savings   • Satisfied  users   • Mission  Critical  Application   that  the  users  can’t  do  with   out! 18
19. Features/Benefits  -­‐  In-­‐use 99%  improved  search  and  index  rebuild   speeds   Supports  unlimited  number  of  content   items/search  terms   Rebuild  segments  of  an  index   Create  unlimited  number  of  indices   Create/rebuild  search  indices  without   restarting  content  server   Minimal  user  impact  during  index   rebuild   Easy  implementation  and  configuration   Horizontal/vertical  scalability  for   extreme  performance 19
20. Features/Benefits  -­‐  In-­‐use Support  for  distributed  architecture   for  indices  and  hardware   Real-­‐time  feedback—estimates   remaining  index  rebuild  time   Search  remains  operational  during   index  rebuild   Multi-­‐threaded  rebuild  process Maintains  current  out-­‐of-­‐the-­‐box   index   Add  new  documents/revisions  or   delete  content  during  rebuilds   Live  reload—real-­‐time  tuning  of  the   index  rebuild  process 20
21. Features/Benefits  –  Not  In  Use  Yet Clustering     Create/maintain  multiple  indices   based  on  content/metadata   Index  priority  for  documents   matching  multiple  indices Ability  to  simultaneously  rebuild   multiple  indices     Flexible  faceted  search  results   – End  users  can  filter  search  results   – End  users  can  control  facets  per   search  or  saved  search   – Administrators  can  create   different  search  facets  per  profile 21
22. Future  of  DES  at  Emerson We  are  or  will  be  use  DES  to  index  ALL  our  WebCenter  Content  instances   In-­‐production   – OCM  (18M  items)  metadata-­‐only   In-­‐process     – eDocs  /  Fishweb  (2M  items)  GSA  full-­‐text   – eDocsDB  (5M  items)  metadata-­‐only Future     – eDocs  Extranet  (3M  items)  full-­‐text   – MMI  Docs  (200,000  items)  full-­‐text   – eDocs  Europe  (2M  items)  full-­‐text   – eDocs  Asia  (300,000  items)  full-­‐text   End-­‐Game     – All  WebCenter  Content  Instances                        with  a  Master    Index  (32M  items)                        to  search  them  all  at  once 22
23. Future  of  DES  at  Emerson We  are  in  discussions  on  how  we  can  use  DES   solution  for  eDiscovery  purposes  on     – File  Servers   – SharePoint 23
24. Wrap Up & Summary • We Had a Major Indexing and Searching Problems • Redstone Developed an Integration of Elastic Search with the Oracle WebCenter Content • The Integration required NO Changes for how the users were use to using the system • The solution was Game Changing for the users with Searching – it is FAST • The solution was Game Changing for the administrators with the Indexing - it is FAST • The Value of the Solution can be Quantified – in real time and money • The Value of the Solution is measured by the Users Response – Ecstatic, Satisfied, See the Application as Critical • Big Future for DES at Emerson Process Management 24
25. Customer Integration Solution & Technology
 With Problem Implementation Partner Solution Mark  Heindselman   Mark.Heindselman@emerson.com   641  754-­‐3161 John  Klein   jklein@redstonecontentsolutioins.com   (563)  424-­‐7563 info@elastic.co