Scaling with Spring & Cloud Foundry

this talk looks at how to scale applications. It looks at horizontal duplication, data partitioning and microservices, all set against the backdrop of Spring Integration, Spring AMQP, Spring Session, Spring Data, Spring MVC, Spring Cloud and the Netflix OSS stack.

Transcript

1. S C A L I N G 

   W I T H
   Josh Long
   @starbuxman
   jlong@pivotal.io
   github.com/joshlong
   (⻰龙之春)
   S P R I N G
   

   View Slide

2. Spring Developer Advocate
   @Starbuxman
   Josh Long (⻰龙之春)
   @starbuxman jlong@pivotal.io
   |
   Jean Claude
   van Damme! Java mascot Duke some thing’s I’ve authored...
   

   View Slide

3. @Starbuxman
   

   View Slide

4. BUILDING ADAPTIVE APPLICATIONS IS HARD
   built on Cloud Foundry
   !
   code will be open sourced.
   W H Y S C A L E ?
   

   View Slide

5. Moore’s Law no longer works @Starbuxman
   § processing can’t scale up
   § concurrent, horizontal architectures are easier to scale
   § “process”-style concurrency is easy to scale still
   Moore's law is the observation that, over the
   history of computing hardware, the number of
   transistors in a dense integrated circuit doubles
   approximately every two years. The law is
   named after Gordon E. Moore, co-founder of the
   Intel Corporation, who described the trend in his
   1965 paper.!
   !
   http://en.wikipedia.org/wiki/Moore's_law
   

   View Slide

6. data @Starbuxman
   44000%
   larger in 2020 than 2009
   data production is expected to be
   :
   

   View Slide

7. systems are increasingly complex @Starbuxman
   § a complex system today has a lot of moving parts
   § security
   § multiple clients (iOS, Android, Windows Mobile, etc.)
   § multiple (business) domains
   § integration between systems
   § requires more people working on the same problem
   

   View Slide

8. mobile
   More than 

   1.5 MILLION
   activations
   daily *
   @Starbuxman
   * http://www.androidcentral.com/larry-page-15-million-android-devices-activated-every-day
   

   View Slide

9. social: a connected world in 60 seconds @Starbuxman
   3125
   photos uploaded
   7630
   messages sent
   7610
   searches
   2MM
   videos viewed
   2000
   checkins
   175k
   tweets
   1090
   visitors
   700k
   messages sent
   * source: visual.ly/60-seconds-social-media
   

   View Slide

10. the internet of things @Starbuxman
    “In five years, by 2018, Earth will be home to 7.6
    billion people, says the United Nations. By
    contrast, some 25 billion devices will be
    connected by 2015, and 50 billion by 2020,
    says Cisco.”!
    !
    http://www.businessinsider.com/what-you-need-to-know-about-the-
    internet-of-things-2013-3?op=1#ixzz3FxCafwWe
    § IPv6 gives us more addresses
    § devices are getting smaller,
    more ubiquitous
    § “devices” include homes
    appliances (refrigerators,
    washers, coffee machines,
    dryers), roads, air pollution
    monitors, (human) body
    monitors, etc
    

    View Slide

11. how to think about scale? @Starbuxman
    Chris Richardson (http://microservices.io/articles/scalecube.html) introduced me to this “scale cube” 

    from The Art of Scaling Software
    

    View Slide

12. BUILDING ADAPTIVE APPLICATIONS IS HARD
    built on Cloud Foundry
    !
    code will be open sourced.
    X - A X I S
    H O R I Z O N TA L D U P L I C AT I O N
    

    View Slide

13. STATELESS APPS

    SCALE
    

    View Slide

14. no state and lots of gain @Starbuxman
    § obvious: no state means no sharing
    § no sharing means that applications can be scaled horizontally easily
    § requires very little:
    § HTTP load balancers are ubiquitous.
    § message queues (like RabbitMQ) make effective load balancers
    

    View Slide

15. D E M O
    R A B B I T M Q P I N G P O N G
    

    View Slide

16. WHAT IF I
    HAVE SOME STATE?
    

    View Slide

17. http sessions? @Starbuxman
    § Spring Session
    § useful in a PaaS
    § useful when you need state
    § useful when you need durable, replicated state
    § pluggable: Redis out-of-the-box, but feel free to bring your own
    

    View Slide

18. D E M O
    R E D I S - B A C K E D H T T P S E S S I O N S
    

    View Slide

19. PAAS: 

    P L AT F O R M - A S - A - S E RV I C E
    

    View Slide

20. why PaaS? @Starbuxman
    Imagine if architects had to be the
    janitor for every building they designed.
    This is how the development team felt
    prior to moving to Windows Azure.
    Duncan Mackenzie Nov 07, 2011
    http://www.infoq.com/articles/Channel-9-Azure
    “ ”
    

    View Slide

21. The Impact of the Cloud @Starbuxman
    § Spring Boot makes it dead simple to stand up services.

    (Where do they live? Who runs them?)
    § Things get Distributed REALLY quickly! CF provides a way to simplify
    !
    !
    § Manifests are are the ultimate installer. 

    (cf push an entire distributed system!)
    § Spring Cloud PaaS connectors simplify service-consumption
    > cf push hystrix.jar
    > cf push …
    

    View Slide

22. D E M O
    S I M P L E S C A L I N G O N T H E C L O U D
    

    View Slide

23. BUILDING ADAPTIVE APPLICATIONS IS HARD
    built on Cloud Foundry
    !
    code will be open sourced.
    Z - A X I S
    D ATA PA RT I T I O N I N G
    

    View Slide

24. C A P & N O S Q L
    

    View Slide

25. Brewer’s Conjecture (CAP) @Starbuxman
    Many datastores provide some of the following three characteristics: !
    !
    § Consistency !
    § Availability !
    § Partitionability 

    !
    clarification #1: in a system with no network partitions (such as a single-
    node RDBMS), then there's no need to sacrifice C & A.!
    

    clarification #2: availability is a continuous value: 0-100%. there are many
    levels of consistency, and even partitions have nuances, including
    disagreement within the system about whether a partition exists.!
    

    View Slide

26. choose the best store for the job @Starbuxman
    

    View Slide

27. NoSQL @Starbuxman
    

    View Slide

28. S P R I N G D ATA 

    R E P O S I TO R I E S
    

    View Slide

29. How it Works in Rails @Starbuxman
    class Car < ActiveRecord

    end
    car = Car.new
    cars = car.find_cars_by_id(232) 

    # where did this method come from?
    # and then magic happens
    

    View Slide

30. Using Spring Data Repositories @Starbuxman
    •Spring Data Neo4J @EnableNeo4jRepositories
    •Spring Data JPA @EnableJpaRepositories
    •Spring Data MongoDB @EnableMongoRepositories
    •Spring Data GemFire @EnableGemfireRepositories
    @Configuration
    @EnableTransactionManagement
    @ComponentScan
    @EnableJpaRepositories( basePackageClasses = BlogRepository.class)
    public class ServiceConfiguration {
    !
    @Bean public DataSource dataSource(){ .. }
    @Bean public PlatformTransactionManager transactionManager(){ .. }
    }
    

    View Slide

31. Custom Repository @Starbuxman
    Keyword Sample Resulting MongoDB Query *
    GreaterThan findByAgeGreaterThan(int
    age)
    {"age" : {"$gt" : age}}
    LessThan findByAgeLessThan(int age) {"age" : {"$lt" : age}}
    Between findByAgeBetween(int from,
    int to)
    {"age" : {"$gt" : from, "$lt" :
    to}}
    NotNull findByFirstnameNotNull() {”firstname" : {"$ne" : null}}
    Null findByFirstnameNull() {”firstname" : null}
    Like findByFirstnameLike(String
    name)
    "firstname" : firstname}
    (regex)
    

    View Slide

32. M O N G O D B
    

    View Slide

33. Spring Data MongoDB @Starbuxman
    § GridFS integration
    § GIS integration
    § Document mapping
    

    View Slide

34. who’s using MongoDB? @Starbuxman
    § Mailbox.app: https://tech.dropbox.com/2013/09/scaling-mongodb-at-mailbox/
    § eHarmony: https://www.mongodb.com/presentations/big-dating-eharmony-0?
    _ga=1.259505294.567221685.1413121358
    § Expedia: https://www.mongodb.com/presentations/building-expedia
    %E2%80%99s-travel-graph-using-mongodb?
    _ga=1.26276665.567221685.1413121358
    

    View Slide

35. D E M O
    M O N G O D B G I S & 

    FA C E B O O K P L A C E S
    

    View Slide

36. R E D I S
    

    View Slide

37. Spring Data Redis @Starbuxman
    § key/value store
    § data structures
    § sets
    § queues
    § lists
    § maps
    § CacheManager implementation
    § memcached client
    

    View Slide

38. who’s using Redis? @Starbuxman
    § Twitter: http://www.infoq.com/presentations/Real-Time-Delivery-Twitter
    § Sina Weibo http://www.xdata.me/?p=353
    § GitHub https://github.com/blog/530-how-we-made-github-fast
    § Snapchat https://twitter.com/robustcloud/status/448503100056535040
    § Pinterest http://engineering.pinterest.com/post/55272557617/building-a-follower-
    model-from-scratch
    

    View Slide

39. C O U C H B A S E
    

    View Slide

40. Spring Data Couchbase @Starbuxman
    § keyed document access
    § sort of like a mix of Redis
    and MongoDB
    § horizontally scalable
    !
    @Configuration
    @EnableCouchbaseRepositories
    public class Application 

    extends AbstractCouchbaseConfiguration {
    !
    @Override
    protected List bootstrapHosts() {
    return Arrays.asList( “127.0.0.1" );
    }
    !
    @Override
    protected String getBucketName() {
    return "default";
    }
    !
    @Override
    protected String getBucketPassword() {
    return "";
    }
    !
    }
    

    View Slide

41. who’s using Couchbase? @Starbuxman
    § AOL: http://www.couchbase.com/ad_platforms
    § Playtika: http://www.couchbase.com/social-gaming
    

    View Slide

42. N E O 4 J
    

    View Slide

43. complexity vs performance @Starbuxman
    

    View Slide

44. who’s using Neo4j? @Starbuxman
    

    View Slide

45. the evolution of search
    Pre-1999
    WWW Indexing
    Atomic Data
    1999 - 2012
    Google Invents
    PageRank
    Simple
    Connected Data
    2012-?
    Google Launches the

    Knowledge Graph
    Rich
    Connected Data
    @Starbuxman
    

    View Slide

46. Recommenda)ons
    @Starbuxman
    

    View Slide

47. Graph Search! @Starbuxman
    

    View Slide

48. What the Cypher Query Looks Like: @Starbuxman
    MATCH (person:Person)-[:IS_FRIEND_OF]->(friend),
    (friend)-[:LIKES]->(restaurant),
    (restaurant)-[:LOCATED_IN]->(loc:Location),
    (restaurant)-[:SERVES]->(type:Cuisine)
    !
    WHERE person.name = 'Philip' AND loc.location='New York' AND
    type.cuisine='Sushi'
    !
    RETURN restaurant.name
    * Cypher query language example
    http://maxdemarzi.com/?s=facebook
    

    View Slide

49. What the Search Looks Like: @Starbuxman
    

    View Slide

50. D E M O
    N E O 4 J T W I T T E R
    

    View Slide

51. H A D O O P
    

    View Slide

52. spring for
    Surviving the Big Data
    Wild-West with

    Spring for Hadoop
    @Starbuxman
    

    View Slide

53. S P R I N G X D
    

    View Slide

54. stream processing, data ingestion & integration
    But How Do You Process Data Realtime?
    @metamarkets founder Michael E. Driscoll:
    @Starbuxman
    

    View Slide

55. stream processing, data ingestion & integration @Starbuxman
    Introducing Spring XD
    sources
    sinks
    

    View Slide

56. D E M O
    S P R I N G X D A N D P I V O TA L H D
    

    View Slide

57. View Slide

58. BUILDING ADAPTIVE APPLICATIONS IS HARD
    built on Cloud Foundry
    !
    code will be open sourced.
    Y- A X I S
    B O U N D E D C O N T E X T S
    

    View Slide

59. micro- vs. monolith… is not a new discussion @Starbuxman
    From: kt4@prism.gatech.EDU (Ken Thompson)
    Subject: Re: LINUX is obsolete
    Date: 3 Feb 92 23:07:54 GMT
    Organization: Georgia Institute of Technology
    I would generally agree that microkernels are probably the wave of
    the future. However, it is in my opinion easier to implement a
    monolithic kernel. It is also easier for it to turn into a mess in
    a hurry as it is modified.
    Regards,
    Ken
    

    View Slide

60. hold on a tick..
    …didn’t the monolith win?
    @Starbuxman
    

    View Slide

61. so what’s so bad about a monolith? @Starbuxman
    (does your monolith drive you to drink?)
    

    View Slide

62. boardroom agility pushes tech agility @Starbuxman
    § boardroom agility manifest in technology:
    • 2-pizza box teams are a result of eschewing organizational norms
    !
    § easier to scale (in development teams, and at runtime)
    !
    § shorter iterations:
    • small services > 

    continuous integration > 

    shorter release cycles > 

    deployment automation
    

    View Slide

63. the elegant microservice @Starbuxman
    

    View Slide

64. problems with microservices @Starbuxman
    § hard to deploy (devops!)
    § hard to tease into separate deployable modules (Boot!)
    § lots of moving parts introduces complexity (PaaS & Spring Cloud!)
    

    View Slide

65. W H Y B O O T
    

    View Slide

66. harder to tease into separate microservices? …No. @Starbuxman
    import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.web.bind.annotation.*
    !
    // assumes org.springframework.boot:spring-boot-starter-web on CLASSPATH
    @Configuration
    @RestController
    @EnableAutoConfiguration
    public class GreetingsController {
    !
    @RequestMapping("/hi/{name}")
    String hello(@PathVariable String name) {
    return "Hello, " + name + "!";
    }
    !
    public static void main(String[] args) {
    SpringApplication.run(GreetingsController.class, args);
    }
    }
    

    View Slide

67. managing many processes with a PaaS @Starbuxman
    § services are explicit about what they bundle
    § services are attached resources (locally or remote, who cares)
    § configuration is external
    § scaling is easy
    § isolation is provided at the process level
    

    View Slide

68. emergent patterns of microservices @Starbuxman
    § distributed / versioned configuration
    § service registration + discovery
    § client-side routing, service-to-service calls
    § load-balancing
    § minimizing failure cascades
    § proxies
    

    View Slide

69. Standing on the Shoulders of
    Spring &
    @Starbuxman
    

    View Slide

70. C O N F I G - S E RV E R
    

    View Slide

71. R E F R E S H - A B L E
    C O N F I G U R AT I O N
    

    View Slide

72. S E RV I C E R E G I S T R AT I O N
    & D I S C O V E RY
    W I T H E U R E K A
    http://techblog.netflix.com/2012/09/eureka.html
    

    View Slide

73. M A N A G I N G
    FA I L U R E S W I T H
    H Y S T R I X
    http://techblog.netflix.com/2012/11/hystrix.html
    

    View Slide

74. D Y N A M I C R O U T I N G
    W I T H Z U U L
    http://techblog.netflix.com/2012/11/hystrix.html
    

    View Slide

75. Bookmark.. @Starbuxman
    § The Netflix Techblog http://techblog.netflix.com
    § Fred Georges on Programmer Anarchy 

    http://www.infoq.com/news/2012/02/programmer-anarchy
    § Matt Stine’s CF + Microservices: a Mutualistic Symbiotic Relationship 

    http://www.youtube.com/watch?v=RGZefc92tZs
    § Martin Fowler’s article - http://martinfowler.com/articles/microservices.html
    

    View Slide

76. Bookmark.. @Starbuxman
    § Former Netflix DevOps Guru Adrian Cockroft on DevOps + MS

    http://www.infoq.com/interviews/adrian-cockcroft-microservices-devops
    § Bootiful Applications with Spring Boot

    http://http://www.youtube.com/watch?v=eCos5VTtZoI
    § Chris Richardson’s http://microservices.io site and his 

    Decomposing Applications for Scalability talks
    § github.com/joshlong/scaling-software-talk
    

    View Slide

77. References
    spring.io/guides
    github.com/spring-cloud/
    github.com/spring-cloud-samples/
    github.com/joshlong/spring-doge
    github.com/joshlong/spring-doge-microservice
    docs.spring.io/spring-boot/
    !
    Questions?
    Josh Long
    @starbuxman
    jlong@pivotal.io
    github.com/joshlong
    (⻰龙之春)
    

    View Slide