Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Evolution of the Netflix API - QCon SF 2013

Evolution of the Netflix API - QCon SF 2013

Netflix streaming has been growing tremendously, now reaching more than 40M subscribers in over 40 countries. These subscribers are enjoying Netflix on more than 1,000 different device types, ranging from game consoles to set-top boxes, TVs and mobile devices. This is all made possible by a complex service-oriented architecture that handles billions of requests a day from those devices. At the center of that architecture is the Netflix API, which is the front door to the entire system.

This presentation will describe how the Netflix API evolved from a typical one-size-fits-all RESTful API designed to support public developers into a web service platform optimized to handle the diversity and variability of each device and user experience. The presentation will also address the challenges involving operations, deployment, performance, fault-tolerance, and rate of innovation at massive scale.

Presented at QCon SF 2013: http://www.infoq.com/presentations/netflix-api-evolution

Ben Christensen

November 11, 2013
Tweet

More Decks by Ben Christensen

Other Decks in Programming

Transcript

  1. Evolution of the Netflix API Ben Christensen Software Engineer –

    Edge & Playback Services at Netflix @benjchristensen ! ! ! ! http://techblog.netflix.com/ QCon San Francisco - November 2013
  2. Netflix accounts for 33% of Peak Downstream Internet Traffic in

    North America Netflix subscribers are watching more than 1 billion hours a month
  3. API traffic has grown from ~20 million/day in 2010 to

    >2 billion/day 0 500 1000 1500 2000 2010 2011 2012 Today millions of API requests per day
  4. At the start … 0 500 1000 1500 2000 2008

    2009 2010 2011 2012 Today millions of API requests per day
  5. 2008 API Launch Targeted 100% at External Developers Purpose A

    “Thousand Flowers” of 3rd party innovation ! Audience External Developers
  6. 0 500 1000 1500 2000 2008 2009 2010 2011 2012

    Today In 2011 … millions of API requests per day
  7. Netflix API Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  8. Netflix API Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  9. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  10. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R User request blocked by latency in single network call
  11. At high volume all request threads can block in seconds

    User Request Dependency A Dependency D Dependency G Dependency J Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R User Request User Request User Request User Request User Request User Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  12. ! Dozens of dependencies. ! One going bad takes everything

    down. ! 99.99%30 = 99.7% uptime ! 0.3% of 1 billion = 3,000,000 failures ! 2+ hours downtime/month ! ! ! Reality is generally worse.
  13. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R User Request User Request User Request User Request User Request User Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  14. cy D dency G ependency J Dependency M Dependency B

    Dependency E Dependency H Dependency K Dependency N Dependency C Dependency F Dependency I Dependency L Dependency O User Request User Request User Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Request - TCP/HTTP, latency, 4xx, 5xx, etc Deserialization - JSON/XML/Thrift/Protobuf/etc Logic - argument validation, caches, metrics, logging, multivariate testing, routing, etc Serialization - URL and/or body generation Logic - validation, decoration, object model, caching, metrics, logging, etc
  15. "Timeout guard" daemon prio=10 tid=0x00002aaacd5e5000 nid=0x3aac runnable [0x00002aaac388f000] java.lang.Thread.State: RUNNABLE!

    at java.net.PlainSocketImpl.socketConnect(Native Method)! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)! - locked <0x000000055c7e8bd8> (a java.net.SocksSocketImpl)! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)! at java.net.Socket.connect(Socket.java:579)! at java.net.Socket.connect(Socket.java:528)! at java.net.Socket.(Socket.java:425)! at java.net.Socket.(Socket.java:280)! at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)! at org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory$1.doit(ControllerThreadSocketFactory.java:91)! at org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory$SocketTask.run(ControllerThreadSocketFactory.java:158) at java.lang.Thread.run(Thread.java:722) [Sat Jun 30 04:01:37 2012] [error] proxy: HTTP: disabled connection for (127.0.0.1) > 80% of requests rejected Median Latency
  16. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  17. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  18. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  19. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  20. 2 minutes of request rate to show relative changes in

    traffic circle color and size represent health and traffic volume
  21. 2 minutes of request rate to show relative changes in

    traffic circle color and size represent health and traffic volume hosts reporting from cluster
  22. last minute latency percentiles 2 minutes of request rate to

    show relative changes in traffic circle color and size represent health and traffic volume hosts reporting from cluster
  23. last minute latency percentiles 2 minutes of request rate to

    show relative changes in traffic circle color and size represent health and traffic volume hosts reporting from cluster Circuit-breaker status
  24. last minute latency percentiles Request rate 2 minutes of request

    rate to show relative changes in traffic circle color and size represent health and traffic volume hosts reporting from cluster Circuit-breaker status
  25. Error percentage of last 10 seconds last minute latency percentiles

    Request rate 2 minutes of request rate to show relative changes in traffic circle color and size represent health and traffic volume hosts reporting from cluster Error percentage of last 10 seconds Circuit-breaker status
  26. last minute latency percentiles Request rate 2 minutes of request

    rate to show relative changes in traffic circle color and size represent health and traffic volume hosts reporting from cluster Error percentage of last 10 seconds Rolling 10 second counters with 1 second granularity Failures/Exceptions Thread-pool Rejections Thread timeouts Successes Short-circuited (rejected) Circuit-breaker status
  27. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  28. User Request Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R System Relationship Over Network without Bulkhead
  29. Netflix API Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  30. Netflix API Dependency A Dependency D Dependency G Dependency J

    Dependency M Dependency P Dependency B Dependency E Dependency H Dependency K Dependency N Dependency Q Dependency C Dependency F Dependency I Dependency L Dependency O Dependency R
  31. One Size Fits All RESTful API 0 500 1000 1500

    2000 2010 2011 2012 Today millions of API requests per day
  32. One Size Fits All RESTful API 0 500 1000 1500

    2000 2010 2011 2012 Today millions of API requests per day
  33. One Size Fits All RESTful API 0 500 1000 1500

    2000 2010 2011 2012 Today millions of API requests per day 1000+ Devices
  34. ... to collapse network traffic into coarse API calls ...

    nested, conditional, concurrent execution
  35. Concurrency without each engineer reading and re-reading this → !

    (awesome book ... everybody isn’t going to - or should have to - read it though, that’s the point)
  36. What if the implementation needs to change from synchronous to

    asynchronous? ! How should the client execute that method without blocking? spawn a thread? public  Data  getData(); Owner of api should retain control of concurrency behavior.
  37. public  void  getData(Callback<T>  c);   ! public  Future<T>  getData();  

    ! public  Future<List<Future<T>>>  getData();   ! ! other options ... ? public  Data  getData();
  38. Iterable pull Observable push T next() throws Exception returns; onNext(T)

    onError(Exception) onCompleted() Reactive Programming with Rx
  39. Iterable pull Observable push T next() throws Exception returns; onNext(T)

    onError(Exception) onCompleted() (Functional) Reactive Programming with RxJava
  40. Iterable pull Observable push T next() throws Exception returns; onNext(T)

    onError(Exception) onCompleted() (Functional) Reactive Programming with RxJava
  41. Iterable pull Observable push T next() throws Exception returns; onNext(T)

    onError(Exception) onCompleted() (Functional) Reactive Programming with RxJava
  42. Iterable pull Observable push T next() throws Exception returns; onNext(T)

    onError(Exception) onCompleted() !  //  Iterable<String>      //  that  contains  75  Strings    getDataFromLocalMemory()      .skip(10)      .take(5)      .map({  s  -­‐>          return  s  +  "_transformed"})      .forEach(            {  println  "next  =>  "  +  it})   !  //  Observable<String>      //  that  emits  75  Strings    getDataFromNetwork()      .skip(10)      .take(5)      .map({  s  -­‐>          return  s  +  "_transformed"})      .subscribe(            {  println  "onNext  =>  "  +  it})  
  43. Iterable pull Observable push T next() throws Exception returns; onNext(T)

    onError(Exception) onCompleted() !  //  Iterable<String>      //  that  contains  75  Strings    getDataFromLocalMemory()      .skip(10)      .take(5)      .map({  s  -­‐>          return  s  +  "_transformed"})      .forEach(            {  println  "onNext  =>  "  +  it})   !  //  Observable<String>      //  that  emits  75  Strings    getDataFromNetwork()      .skip(10)      .take(5)      .map({  s  -­‐>          return  s  +  "_transformed"})      .subscribe(            {  println  "onNext  =>  "  +  it})  
  44. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() String s = getData(args); if (s.equals(x)) { // do something } else { // do something else }
  45. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Iterable<String> values = getData(args); for (String s : values) { if (s.equals(x)) { // do something } else { // do something else } }
  46. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Future<String> s = getData(args); if (s.get().equals(x)) { // do something } else { // do something else }
  47. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Future<String> s = getData(args); if (s.get().equals(x)) { // do something } else { // do something else }
  48. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Future<String> s = getData(args); Futures.addCallback(s, new FutureCallback<String> { public void onSuccess(String s) { if (s.equals(x)) { // do something } else { // do something else } } }, executor);
  49. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Future<String> s = getData(args); Futures.addCallback(s, new FutureCallback<String> { public void onSuccess(String s) { if (s.equals(x)) { // do something } else { // do something else } } }, executor);
  50. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Future<String> s = getData(args); Futures.addCallback(s, new FutureCallback<String> { public void onSuccess(String s) { if (s.equals(x)) { // do something } else { // do something else } } }, executor);
  51. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() CompletableFuture<String> s = getData(args); s.thenApply((v) -> { if (v.equals(x)) { // do something } else { // do something else } });
  52. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() CompletableFuture<String> s = getData(args); s.thenApply((v) -> { if (v.equals(x)) { // do something } else { // do something else } });
  53. Future<String> s = getData(args); s.map({ s -> if (s.equals(x)) {

    // do something } else { // do something else } }); Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData() Observable<T> getData()
  54. Future<String> s = getData(args); s.map({ s -> if (s.equals(x)) {

    // do something } else { // do something else } }); Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData() Observable<T> getData()
  55. Future<String> s = getData(args); s.map({ s -> if (s.equals(x)) {

    // do something } else { // do something else } }); Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData() Observable<T> getData()
  56. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable<String> s = getData(args); s.map({ s -> if (s.equals(x)) { // do something } else { // do something else } });
  57. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable<String> s = getData(args); s.map({ s -> if (s.equals(x)) { // do something } else { // do something else } });
  58. Single Multiple Sync T getData() Iterable<T> getData() Async Future<T> getData()

    Observable<T> getData() Observable<String> s = getData(args); s.map({ s -> if (s.equals(x)) { // do something } else { // do something else } });
  59. class  VideoService  {        def  VideoList  getPersonalizedListOfMovies(userId);  

         def  VideoBookmark  getBookmark(userId,  videoId);        def  VideoRating  getRating(userId,  videoId);        def  VideoMetadata  getMetadata(videoId);   } class  VideoService  {        def  Observable<VideoList>  getPersonalizedListOfMovies(userId);        def  Observable<VideoBookmark>  getBookmark(userId,  videoId);        def  Observable<VideoRating>  getRating(userId,  videoId);        def  Observable<VideoMetadata>  getMetadata(videoId);   } ... create an observable api: instead of a blocking api ...
  60. “a library for composing asynchronous and event-based programs using observable

    sequences for the Java VM” A Java port of Rx (Reactive Extensions) https://rx.codeplex.com (.Net and Javascript by Microsoft) RxJava http://github.com/Netflix/RxJava
  61. client code treats all interactions with the api as asynchronous

    ! ! the api implementation chooses whether something is blocking or non-blocking and what resources it uses
  62. api.servletResponse.writer.print("Hello") $runScript.py -e PROD sample.groovy Hello ################################################################################################ ####### Script Logger

    (enabled via debug=true request parameter) ################################################################################################ [0ms] INFO: User => NULL ################################################################################################ ####### Script Logger (enabled via debug=true request parameter) ################################################################################################
  63. api.servletResponse.writer.print("Hello") $runScript.py -e PROD sample.groovy Hello ################################################################################################ ####### Script Logger

    (enabled via debug=true request parameter) ################################################################################################ [0ms] INFO: User => NULL ################################################################################################ ####### Script Logger (enabled via debug=true request parameter) ################################################################################################
  64. api.servletResponse.writer.print("Hello") $runScript.py -e PROD sample.groovy Hello ################################################################################################ ####### Script Logger

    (enabled via debug=true request parameter) ################################################################################################ [0ms] INFO: User => NULL ################################################################################################ ####### Script Logger (enabled via debug=true request parameter) ################################################################################################
  65. api.servletResponse.writer.print("Hello") $runScript.py -e PROD sample.groovy Hello ################################################################################################ ####### Script Logger

    (enabled via debug=true request parameter) ################################################################################################ [0ms] INFO: User => NULL ################################################################################################ ####### Script Logger (enabled via debug=true request parameter) ################################################################################################
  66. api.servletResponse.writer.print("Hello") $runScript.py -e PROD sample.groovy Hello ################################################################################################ ####### Script Logger

    (enabled via debug=true request parameter) ################################################################################################ [0ms] INFO: User => NULL ################################################################################################ ####### Script Logger (enabled via debug=true request parameter) ################################################################################################
  67. api.servletResponse.writer.print("Hello") $runScript.py -e PROD sample.groovy Hello ################################################################################################ ####### Script Logger

    (enabled via debug=true request parameter) ################################################################################################ [0ms] INFO: User => NULL ################################################################################################ ####### Script Logger (enabled via debug=true request parameter) ################################################################################################
  68. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } }
  69. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy
  70. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy {"alias":"bob","name":"Old"}
  71. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy {"alias":"bob","name":"Old"}
  72. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy {"alias":"bob","name":"Old"}
  73. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy {"alias":"bob","name":"Old"}
  74. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy {"alias":"bob","name":"Old"}
  75. public class HelloEndpoint extends APIEndpoint { @Override public void execute(APIRequest

    api) throws Throwable { // get a request parameter from servlet request String alias = api.servletRequest.getParameter("name"); ! // set content type and write something to servlet response api.servletResponse.setContentType("application/json"); api.servletResponse.writer.print( JsonUtility.toJson( [ alias : alias, name : api.user.firstName ] ) ) ! } } runScript.py -e TEST --userId 1189658154 --args "name=bob" HelloEndpoint.groovy {"alias":"bob","name":"Old"}
  76. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  77. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  78. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  79. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  80. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  81. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  82. public Observable getVideoSummary(APIVideo video) { // get id, title def

    seed = [id: video.id, title : video.getTitle(APIVideo.TitleType.REGULAR)] ! // get bookmark def bookmarkObservable = getBookmark(video) ! // get artwork def artworkObservable = getArtworkImageUrl(video) ! // merge them and accumulate into the seed. return Observable.merge(bookmarkObservable, artworkObservable) .reduce(seed, { aggregate, current -> aggregate << current}) .map({ [ (video.id.toString()) : it]}) }
  83. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:38:51 UTC 2013", "revision": 1, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  84. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:38:51 UTC 2013", "revision": 1, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  85. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  86. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" } $ activateScript.py -e TEST --revision 2 /test/hello { "active": true, "allocationPercentage": 100, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "previousRevision": null, "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  87. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" } $ activateScript.py -e TEST --revision 2 /test/hello { "active": true, "allocationPercentage": 100, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "previousRevision": null, "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  88. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" } $ activateScript.py -e TEST --revision 2 /test/hello { "active": true, "allocationPercentage": 100, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "previousRevision": null, "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  89. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" } $ activateScript.py -e TEST --revision 2 /test/hello { "active": true, "allocationPercentage": 100, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "previousRevision": null, "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  90. $ uploadScript.py -e TEST /test/hello sample.groovy { "active": false, "allocationPercentage":

    0, "creationDate": "Mon Nov 11 05:40:26 UTC 2013", "revision": 2, "userAuthorizationRequired": true, "userAuthorizationType": "https" } $ activateScript.py -e TEST --revision 3 /test/hello { "active": true, "allocationPercentage": 100, "creationDate": "Mon Nov 11 05:42:05 UTC 2013", "previousRevision": 2, "revision": 3, "userAuthorizationRequired": true, "userAuthorizationType": "https" }
  91. millions of API requests per day 0 500 1000 1500

    2000 2008 2009 2010 2011 2012 Today
  92. /ps3/home Dependency F 10 Threads Dependency G 10 Threads Dependency

    H 10 Threads Dependency I 5 Threads Dependency J 8 Threads Dependency A 10 Threads Dependency B 8 Threads Dependency C 10 Threads Dependency D 15 Threads Dependency E 5 Threads Dependency K 15 Threads Dependency L 4 Threads Dependency M 5 Threads Dependency N 10 Threads Dependency O 10 Threads Dependency P 10 Threads Dependency Q 8 Threads Dependency R 10 Threads Dependency S 8 Threads Dependency T 10 Threads /android/home /tv/home Functional Reactive Dynamic Endpoints Asynchronous Java API
  93. /ps3/home Dependency F 10 Threads Dependency G 10 Threads Dependency

    H 10 Threads Dependency I 5 Threads Dependency J 8 Threads Dependency A 10 Threads Dependency B 8 Threads Dependency C 10 Threads Dependency D 15 Threads Dependency E 5 Threads Dependency K 15 Threads Dependency L 4 Threads Dependency M 5 Threads Dependency N 10 Threads Dependency O 10 Threads Dependency P 10 Threads Dependency Q 8 Threads Dependency R 10 Threads Dependency S 8 Threads Dependency T 10 Threads /android/home /tv/home Functional Reactive Dynamic Endpoints Asynchronous Java API Hystrix fault-isolation layer
  94. + Observable<User>  u  =  new  GetUserCommand(id).observe();   Observable<Geo>  g  =

     new  GetGeoCommand(request).observe();   ! Observable.zip(u,  g,  {user,  geo  -­‐>                    return  [username:  user.getUsername(),                                    currentLocation:  geo.getCounty()]         }) RxJava in Hystrix 1.3+ https://github.com/Netflix/Hystrix
  95. Optimizing the Netflix API http://techblog.netflix.com/2013/01/optimizing-netflix-api.html ! Why REST Keeps Me

    Up At Night http://blog.programmableweb.com/2012/05/15/why-rest-keeps-me-up-at-night/ ! Functional Reactive in the Netflix API with RxJava http://techblog.netflix.com/2013/02/rxjava-netflix-api.html ! Application Resilience in a Service-oriented Architecture http://programming.oreilly.com/2013/06/application-resilience-in-a-service-oriented-architecture.html ! Netflix API on Tech Blog http://techblog.netflix.com/search/label/api jobs.netflix.com Ben Christensen @benjchristensen http://www.linkedin.com/in/benjchristensen