Apache Brooklyn - What it is (2015)

...and why you should join in. Presented at ApacheCon Core Europe 2015, this is an introduction and demonstration of the Apache Brooklyn project.

Transcript

1. apache brooklyn WHAT IT IS AND WHY YOU SHOULD JOIN IN Richard Downer richard@apache.org Presented at ApacheCon Europe 2015
2. apache brooklyn A BRIEF HISTORY OF BROOKLYN • Brooklyn  was  started  by  Cloudso3  (my  employer)   • Open  sourced  in  April  2012  (Apache  2  license)   • Joined  the  Apache  Incubator  in  May  2014   • Voted  to  graduate  the  Incubator  in  October  2015  -­‐  resoluHon  to  be   presented  to  the  next  ASF  Board  MeeHng
3. apache brooklyn THE ONLY SLIDE ABOUT THE COMPANY • Cloudso3  use  Apache  Brooklyn  as  the  base  for
 AMP:  ApplicaHon  Management  PlaOorm   • Provide  commercial  support  and  special  integraHons  for  Brooklyn   • Brooklyn  is  not  “open  core”:  Cloudso3  is  commiSed  to  a   comprehensive  and  expanding  Apache  Brooklyn  feature  list  and   codebase   …but  we  won’t  talk  about  Cloudso3  any  more
4. apache brooklyn THE PROBLEM WITH BROOKLYN: How  to   describe  it  in   one  sentence
5. APPLICATION MODELLING,
 MONITORING AND MANAGEMENT
6. BLUEPRINT FOR SUCCESS! Model, monitor and manage your applications with policy-based automation
7. apache brooklyn THESE TAGLINES • Are  accurate   • Are  short  (mostly)   • Fail  to  provide  any  useful  insight  into  what  Brooklyn   actually  does
8. apache brooklyn LET’S DESCRIBE BY EXAMPLE • A  simple  web  applicaHon:   • JBoss  web  app  container   • MySQL  database JBoss MySQL
9. apache brooklyn IN DEVELOPMENT… • You  could  set  up  your  JBoss  and  MySQL:   • Run  them  on  localhost   • Use  Vagrant  to  start  to  two  virtual  machines   • Provision  a  couple  of  cloud  instances   • JBoss  needs  to  locate  MySQL   • Easy  enough  to  configure  this  by  hand
10. apache brooklyn BUT WHAT ABOUT
 QA AND PRODUCTION? • A  single  web  server  and  a  single   database  is  not  a  producHon  grade   configuraHon!   • It’s  not  scalable:  it  can’t  handle  heavy   load   • It’s  not  resilient:  it  can’t  handle  failures   • So  we  add  mulHple  JBosses   • …and  a  load  balancer   • …and  a  MySQL  standby  node JBoss MySQL JBoss JBoss Load Balancer MySQL Hot Standby
11. apache brooklyn BUT WAIT, THERE’S STILL MORE • This  is  just  for  a  simple  applicaHon  of  a  web  server  and  a  database!   • What  about  something  with:   • MulHple  Hers  and  services   • Connected  by  a  message  queue   • With  mulHple  types  of  database  backend  -­‐
 SQL  and  NoSQL
12. apache brooklyn MORE COMPLEXITY JBoss MySQL JBoss JBoss Load Balancer MySQL Hot Standby Service B NoSQL store shard Service A NoSQL store shard NoSQL store shard Message Broker
13. apache brooklyn THE FIRST PROBLEM • How  do  you  deploy  an  app  with  mulHple  components?   • How  do  you  get  the  servers  to  deploy  onto?   • How  do  you  get  the  so3ware  onto  servers?   • How  do  you  configure  the  so3ware  pieces  to  talk  to  each  other?   • How  do  you  make  this  process  fast,  easy  and  automatable?   With  apache brooklyn,  of  course!
14. apache brooklyn THE FIRST PILLAR OF BROOKLYN Deployment and wiring
15. apache brooklyn OBLIGATORY PAGE FULL OF LOGOS
16. apache brooklyn BLUEPRINTS: HOW TO DEPLOY WITH BROOKLYN • Describe  your  applicaHon  to  Brooklyn  -­‐
 make  a  blueprint   • Describe  the  components  you  are  using   • Describe  how  they  must  be  configured   • Describe  how  they  relate  to  each  other   • Describe  where  they  are  to  be  deployed
17. apache brooklyn BLUEPRINT FOR OUR SIMPLE EXAMPLE name: My Web Application location: AWS_eu-west-1 services: - serviceType: brooklyn.entity.database.mysql.MySqlNode id: db name: My DB brooklyn.config: creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script - serviceType: brooklyn.entity.webapp.jboss.JBoss7Server name: My Web brooklyn.config: wars.root: http://bit.ly/brooklyn-example-helloworld-war java.sysprops: brooklyn.example.db.url: > $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n")
18. apache brooklyn A DEMONSTRATION
 OF DEPLOYMENT Watching  at  home?  Go  to:
 hSp:/ /youtu.be/0iJ18tlaOQk
19. apache brooklyn ANATOMY OF AN APPLICATION Application Entity
20. apache brooklyn ANATOMY OF AN ENTITY
21. apache brooklyn ANATOMY OF AN ENTITY
22. apache brooklyn ANATOMY OF AN ENTITY
23. apache brooklyn ANATOMY OF AN ENTITY
24. apache brooklyn ANATOMY OF AN ENTITY
25. apache brooklyn ANATOMY OF AN ENTITY
26. apache brooklyn MAKE IT A LOAD-BALANCED WEB CLUSTER name: My Web Application location: AWS_eu-west-1 services: - serviceType: brooklyn.entity.database.mysql.MySqlNode id: db name: My DB brooklyn.config: creationScriptUrl: https://bit.ly/brooklyn-visitors-creation-script - serviceType: brooklyn.entity.webapp.jboss.JBoss7Server name: My Web brooklyn.config: wars.root: http://bit.ly/brooklyn-example-helloworld-war java.sysprops: brooklyn.example.db.url: > $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") brooklyn.entity.webapp.ControlledDynamicWebAppCluster
27. apache brooklyn MAKE IT A LOAD-BALANCED WEB CLUSTER
28. apache brooklyn LOCATIONS • Fully  “cloud  aware”  using
 Apache  jclouds   • Amazon  EC2,  CloudStack,  OpenStack,  So3Layer,  Google  GCE,  …   • Provisions  instances  on  demand,  installs  and  customises;  de-­‐provisions   when  no  longer  required   • Or  use  “BYON”  -­‐  bring  your  own  nodes   • Or,  for  tesHng,  deploy  to  localhost
29. apache brooklyn LOCATIONS • MulHple  locaHons   • MulH-­‐geography  deployments  reduce  latency  to  InternaHonal  users   • “Fabrics”  replicate  an  applicaHon  topology  into  different  regions   • Geography-­‐aware  DNS  routes  visitors  to  closest  server   • Availability  zone  awareness   • Clusters  can  distribute  their  members  across  availability  zones
30. apache brooklyn THE SECOND PILLAR OF BROOKLYN Deployment and wiring Runtime management
31. apache brooklyn WHAT IS
 RUNTIME MANAGEMENT? • Deployment  is  merely  the  opening  move  in  the  game   • RunHme  management  is…   • InstrucHons  to  change  the  deployed  applicaHon   • Monitor  the  health  of  all  the  components  and  react  to  failures   • Monitor  the  load  on  the  components  and  react  to  rising  and  falling   demand   • OpHmise  for  cost,  responsiveness,  and  more
32. apache brooklyn POLICIES • Something  that  will  make  changes  to  the  applicaHon  without  requiring   operator  intervenHon   • Policies  are  aSached  to  an  enHty   • Monitor  the  enHty’s  sensor  data  and  other  informaHon   • Makes  changes  to  the  applicaHon  by  invoking  effectors  on  the  enHty
33. apache brooklyn POLICIES DEMONSTRATION & DESCRIPTIONS Watching  at  home?  Go  to:
 hSp:/ /youtu.be/-­‐WdbiDpZ8-­‐g
34. apache brooklyn BLUEPRINT FOR A POLICY brooklyn.policies: - type: brooklyn.policy.ha.ServiceReplacer - type: brooklyn.policy.autoscaling.AutoScalerPolicy brooklyn.config: metric: $brooklyn:sensor("brooklyn.entity.webapp.ControlledDynamicWebAppCluster", "webapp.reqs.perSec.windowed") metricLowerBound: 10 metricUpperBound: 100 minPoolSize: 2 maxPoolSize: 5 dynamiccluster.zone.enable: true dynamiccluster.numAvailabilityZones: 2 memberSpec: $brooklyn:entitySpec: type: brooklyn.entity.webapp.jboss.JBoss7Server brooklyn.enrichers: - type: brooklyn.policy.ha.ServiceFailureDetector brooklyn.policies: - type: brooklyn.policy.ha.ServiceRestarter brooklyn.config: failOnRecurringFailuresInThisDuration: 30 minutes
35. apache brooklyn MORE POLICIES • OpHmise  to  minimise  latency  to  the  users:
 enHHes  are  moved  to  locaHons  close  to  the  users  on  the  network   (“follow  the  sun”)   • OpHmise  to  minimise  costs:
 enHHes  are  moved  to  locaHons  offering  the  lowest  prices  (“follow  the   moon”)   • Policies  can  be  based  on  anything  measurable!
36. apache brooklyn THE FOUNDATION OF BROOKLYN Deployment and wiring Runtime management Autonomic  compu+ng
37. apache brooklyn hSps:/ /en.wikipedia.org/wiki/Autonomic_compuHng Autonomic  compuHng  refers  to  the  self-­‐managing  characterisHcs  of  distributed  compuHng   resources,  adapHng  to  unpredictable  changes  while  hiding  intrinsic  complexity  to  operators  and   users  […]  The  system  makes  decisions  on  its  own,  using  high-­‐level  policies;  it  will  constantly  check   and  opHmize  its  status  and  automaHcally  adapt  itself  to  changing  condiHons.  An  autonomic   compuHng  framework  is  composed  of  autonomic  components  (AC)  interacHng  with  each  other  […]   with  sensors  (for  self-­‐monitoring),  effectors  (for  self-­‐adjustment),  knowledge  and  planner/adapter   for  exploiHng  policies  based  on  self-­‐  and  environment  awareness.
38. apache brooklyn THE BLUFFER’S GUIDE TO AUTONOMIC COMPUTING IN BROOKLYN • The  autonomic  components  are  the  enHHes   • EnHHes  have:   • Sensors,  which  provide  data  to  the  external  world   • Effectors,  which  cause  the  enHty  to  change  in  some  way   • Sensor  data  drives  policies;
 policies  drive  effectors  to  make  changes;
 a  conHnually-­‐adapHng  feedback  loop   • Management  can  happen  locally  at  the  enHty;
 or  be  escalated  up  the  tree  to  be  managed  there
39. apache brooklyn THE STATUS OF APACHE BROOKLYN
40. apache brooklyn BREAKING NEWS • Voted  to  graduate  the  Incubator  in  October  2015  -­‐  resoluHon  to  be   presented  to  the  next  ASF  Board  MeeHng
41. apache brooklyn STATUS UPDATE • Entered  Apache  Incubator  in  May  2014   • Current  version  is  0.8.0   • Released  14th  September  2015   • Includes  source  and  binary  releases   • Our  third  release  in  the  Apache  Incubator   • Code  is  sHll  in  rapid  development  but  stabilising  as  we  approach   version  1.0   • Expected  early  2016
42. apache brooklyn STATUS UPDATE • 9  commiSers  &  PPMC  members   • 10  mentors   • A  number  of  addiHonal  developers  and  contributors   • A  number  of  commercial  customers  (via  Cloudso3)   • A  number  of  academic  users   • We  sHll  a  small  community  -­‐  but  growing  all  the  Hme!
43. apache brooklyn HOW TO HELP • We  need  a  bigger  and  more  diverse  community   • …so  please  join  us!   • Download  and  run,  try  out  deployments   • Share  your  experiences  on  the  mailing  list   • Bug  reports,  code  contribuHons,  documentaHon  contribuHons   • Tell  us  how  to  make  it  be1er!
44. apache brooklyn WHERE TO FIND US • Official  website:  hSps:/ /brooklyn.incubator.apache.org   • Mailing  list:
 dev-­‐subscribe@brooklyn.incubator.apache.org
 hSps:/ /mail-­‐archives.apache.org/mod_mbox/incubator-­‐brooklyn-­‐dev/   • Source  code:
 hSps:/ /github.com/apache/incubator-­‐brooklyn
 or  hSps:/ /git-­‐wip-­‐us.apache.org/repos/asf?p=incubator-­‐brooklyn.git   • IRC  channel:  #brooklyncentral  on  Freenode   • Social  Media:  @ApacheBrooklyn,  YouTube,  Google+,  Facebook
45. apache brooklyn CLOUDSOFT ARE HIRING! • Cloudso3  are  looking  for  great  so3ware  engineers   • To  work  on  Apache  Brooklyn  and  other  open  source  projects,  and  with  our   commercial  clients  who  are  puzng  it  into  producHon   • Brooklyn  is  wriSen  in  Java  with  a  JavaScript  front-­‐end,  but  Java/JavaScript   experience  not  an  issue  -­‐  because  we  know  that  great  so3ware  engineers   can  adapt  and  learn   • LocaHon  not  an  issue  -­‐  we  have  a  distributed  team   jobs@cloudso3.io
46. apache brooklyn More  Brooklyn  at  ApacheCon   Running  Docker  in  ProducHon  with
 Apache  Brooklyn  and  Clocker  (Andrew  Kennedy)
 Tomorrow,  14:30  @  Kond   Apache  Brooklyn  Hacks
 Tomorrow,  15:30  @  Petofi   Thank  You!   Richard  Downer  <richard@apache.org>
 TwiSer:  @FronHerTown