Goldman Sachs Building a Firm Wide Task List with Elastic

Stephen Coster explains how Elastic is used by Goldman Sachs to distribute and aggregate tasks for the whole banking and investment firm.

Stephen Coster | Elastic{ON} Tour | London

1. 1   Goldman Sachs Engineering   GS.com/Engineering Spring, 2015
2. 2   Elas(c{ON}   Goldman  Sachs   Building  a  firm  wide  single  task  list   Enterprise  Pla5orms  -­‐  Workflow   November  3,  2015   These  materials  (“Materials”)  are  confiden(al  and  for  discussion  purposes  only.    The  Materials  are  based  on  informa(on  that  we  consider  reliable,   but  Goldman  Sachs  does  not  represent  that  it  is  accurate,  complete  and/or  up  to  date,  and  it  should  not  be  relied  on  as  such.    The  Materials  do  not   cons(tute  advice  nor  is  Goldman  Sachs  recommending  any  ac(on  based  upon  them.    Opinions  expressed  may  not  be  those  of  Goldman  Sachs  unless   otherwise  expressly  noted.  As  a  condi(on  to  Goldman  Sachs  presen(ng  the  Materials  to  you,  you  agree  to  treat  the  Materials  in  a  confiden(al   manner  and  not  disclose  the  contents  thereof  without  the  permission  of  Goldman  Sachs.       ©  Copyright  2015  The  Goldman  Sachs  Group,  Inc.  All  rights  reserved.  
3. 3   Stephen Coster Vice President
4. 4   Building a firm wide single task list Problem n Develop  a  web  based  firm  wide  single  task  list   n Total  number  of  open  tasks  expected  to  be  in  the  region  of  five  million   n Distributed  to  38000  users  around  the  globe   n Update  latency  to  be  in  the  region  of  5-­‐10  seconds   n Data  to  be  sourced  from  mul(ple  produc(on  instances      
5. 5   Building a firm wide single task list Architecture Sequenced  Data  Stream   REST   n In  memory  elas(c  index   n Index  sequenced  stream  of  data   n Query,  created  from  client  provided  state,  determines  current  subset  of  end  user’s  full  dataset   n No  need  for  the  server  to  maintain  any  user  specific  session  state   n Use  an  offset  over  the  returned  data  set  to  enable  infinite  scroll  func(onality   n Server  side  facet  calcula(on  
6. 6   Building a firm wide single task list Scaling and Aggregation Sequenced   Source   ..n   So_ware  Load  Balancer   n Sequenced  data  stream  can  be  combined  from  mul(ple  different  sources  to  provide  a  unified   view  across  different  deployments   n No  requirement  for  user  state  to  be  stored  in  each  proxy  allowing  more  to  be  added  to   increase  capacity  with  ease   Sequenced   Source   Sequenced   Source  
7. 7   Learn more at GS.com/Engineering The term ‘engineer’ referenced in this section is neither a licensed engineer nor an individual offering engineering services to the general public under applicable law. These materials (“Materials”) are confidential and for discussion purposes only. The Materials are based on information that we consider reliable, but Goldman Sachs does not represent that it is accurate, complete and/or up to date, and it should not be relied on as such. The Materials do not constitute advice nor is Goldman Sachs recommending any action based upon them. Opinions expressed may not be those of Goldman Sachs unless otherwise expressly noted. As a condition to Goldman Sachs presenting the Materials to you, you agree to treat the Materials in a confidential manner and not disclose the contents thereof without the permission of Goldman Sachs. © Copyright 2015 The Goldman Sachs Group, Inc. All rights reserved.