Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dropwizard

 Dropwizard

A presentation on jhug (http://www.jhug.gr), October 2013 about dropwizard and modern java stacks

Spyros Anastasopoulos

October 12, 2013
Tweet

More Decks by Spyros Anastasopoulos

Other Decks in Programming

Transcript

  1. Dropwizard
    JHUG October 2013
    Anastasopoulos Spyros
    @anastasop

    View Slide

  2. Introduction
    Dropwizard is a Java framework for developing ops-
    friendly, high-performance, RESTful web services.
    A modern standalone stack like play and twitter libraries
    (commons, ostrich, server) with a cloud/devops orientation
    Named after a comic strip http://gunshowcomic.com/316
    A new tool to solve a recurring problem today. Tools,
    problems, environments, techniques and assumptions
    never stay constant but evolve.

    View Slide

  3. Dropwizard (what & how)

    View Slide

  4. The stack
    Restful web services are composed of:
    ● REST API endpoints
    ● templates and their rendering engines
    ● database access libraries
    ● additional servlets and filters
    ● special admin servlets like /abort, /reload
    ● lifecycle bound objects (executors or connection pools)
    ● health checks (periodically running application tests)
    ● metrics monitors (check behavior in production)
    For all the above, there are well known implementations
    that provide APIs for using, integrating and extending them.

    View Slide

  5. The stack
    Dropwizard, a little bit more than opinionated glue code for
    ● Jetty - embedded, tunable web server
    ● Jersey - JAX-RS, higher level abstraction over servlets
    ● Jackson - JSON processor
    ● Logback - successor of log4j for logging
    ● Metrics - a library for application runtime metrics
    ● Liquibase - a tool for database refactoring/migration
    ● Hibernate - the well known ORM
    ● JDBI - an alternative database access API
    ● Mustache, Freemarker - template engines

    View Slide

  6. Metrics - mind the gap
    If you don’t measure it, you can’t optimize it. Compare
    A. Our code uses HAL and runs on death star
    B. At ~2,000 req/sec, latency jumps from 13ms to 453ms.
    Provides
    Gauge, Counter, Meter, Histogram, Timer, Health Check
    programmatically or instrumented (@Timed, @Metered,
    @Gauge)
    Monitoring
    files, logs, http/json, monitoring systems(Graphite, Ganglia)

    View Slide

  7. Liquibase - database refactoring
    ● describe the changes







    ● liquibase.sh {update, diff, status, tag, dump, rollback, ..}
    ● maintains a table DATABASECHANGELOG with the changes
    ○ ID AUTHOR FILENAME DATEEXECUTED TAG LIQUIBASEVERSION

    View Slide

  8. Dropwizard usage
    Create a maven project with dropwizard’s dependencies
    ● write code following dropwizard’s conventions and API
    ● provide the service configuration in a single yaml file
    ● java -jar service.jar server configuration.yaml
    From my experience people new to dropwizard either love
    it because of all these features or hate it because of a
    single one of them (each one gets equal 33.33%
    disapproval) More on these later.

    View Slide

  9. Configuration
    Everything goes into one, global yaml file. No other way to
    configure. No .properties or .xml files in the classpath or
    in META-INF/ or in “hidden/non standard” server places
    logging:
    level: INFO
    http:
    port: 8080
    database:
    driverClass: org.postgresql.Driver
    httpClient:
    maxConnections: 10
    Dropwizard provides sensible defaults for most uses

    View Slide

  10. API
    Underneath, it’s still jersey. For standard JEE you just pack
    this in a war and deploy
    @Path("/") @Produces(MediaType.APPLICATION_JSON)
    public class TeamsServiceResource {
    @Context private RedisPool pool;
    @GET @Path("{name}")
    @Produces({MediaType.APPLICATION_JSON})
    public Team getTeam(@PathParam("name") String name) {
    // implementation omitted
    }
    }

    View Slide

  11. API
    For dropwizard, first you create a simple pojo to hold your
    configuration. The configuration.yaml file will be
    deserialized to this pojo using Jackson and will be validated
    with hibernate validator(@NotNull, @NotEmpty, @Min, ...)
    Then extend dropwizard’s default service, to create yours
    public class TeamsService extend
    Service {
    // override what is needed
    }

    View Slide

  12. API
    Dropwizard has a modular architecture. Each unit of
    reusability is called a bundle. Think of them as modules or
    plugins. A service out of the box uses only the bundles for
    jetty, jersey and metrics. Additional ones (hibernate, views,
    liquibase) must be declared for a service
    public void initialize(
    Bootstrap bootstrap) {
    bootstrap.setName("dropwizard-example");
    bootstrap.addBundle(new AssetsBundle("/assets/", "/"));
    bootstrap.addBundle(new ViewBundle());
    bootstrap.addBundle(new MigrationsBundle());
    }

    View Slide

  13. API
    Then declare what your service will use from all the things
    bundles provide. These are gathered in an environment.
    Note: be very explicit. No shortcuts like autowiring
    public void run(
    TeamsServiceConfiguration conf, Environment env) {
    env.addResource(TeamsServiceResource.class);
    env.addResource(new TeamsServiceResource(“custom”);
    env.addHealthCheck(new TeamsHealthCheck(“teamshealth”));
    env.managedExecutorService(executor);
    env.addTask(new ShutdownTask(“shutdown”);
    env.addServlet(new MyViewServlet());
    }

    View Slide

  14. API
    $ mvn package
    $ java -jar target/service.jar server configuration.yaml
    Service Endpoints
    GET http://localhost:8080/teams/
    GET http://localhost:8080/teams/{name}
    POST http://localhost:8080/teams/{name}
    Admin Console
    POST http://localhost:8081/tasks/gc (dropwizard)
    POST http://localhost:8081/tasks/shutdown (user)
    GET http://localhost:8081/ping
    GET http://localhost:8081/healthcheck
    GET http://localhost:8081/threads
    GET http://localhost:8081/metrics

    View Slide

  15. Other goodies
    Bundles
    ● easy testing for jersey representations and resources
    ● OAuth2 authentication (no token exchange, only auth)
    ● database migrations with liquibase
    ● instrumented rest client with Apache http or jersey
    Commands
    manage applications with custom cli arguments and code
    java -jar hello-world.jar db status helloworld.yml
    java -jar hello-world.jar queues persist helloworld.yml

    View Slide

  16. API - Notes
    Bundles organize configuration, build environments and
    extend dropwizard. They usually are build on jersey’s SPI
    // from views bundle
    env.addProvider(ViewMessageBodyWriter.class);
    // from hibernate bundle
    DatabaseConfiguration dbConfig =
    getDatabaseConfiguration(configuration);
    sessionFactory = sessionFactoryFactory.build(
    environment, dbConfig, entities);
    env.addProvider(new
    UnitOfWorkResourceMethodDispatchAdapter(sessionFactory));

    View Slide

  17. API - Notes
    Dropwizard tries to keep the application footprint as small
    as possible. You must declare everything you are going to
    use.
    Sometimes this is simple. Templates for example:
    public class TeamView extends View {
    public TeamView(String name, List members) {
    // the view file goes on resources/same package
    super("Team.mustache");
    }
    }

    View Slide

  18. API - Notes
    But sometimes it can be tedious.
    // hibernate bundle
    protected HibernateBundle(
    Class> entity, Class>... entities) {...}
    The bundle builds the sessionFactory entirely
    programmatically. It consults only the yaml configuration
    not hibernate.cfg.xml. To make it easier dropwizard
    provides AbstractDao and @UnitOfWork.

    View Slide

  19. API - Notes
    Good integration example: liquibase - database refactoring




    ● No additional configuration like liquibase.properties
    ● src/main/resources/migrations.xml packed with service
    ● Liquibase runner integrated as dropwizard commands
    ○ java -jar hello-world.jar db migrate helloworld.yml
    ○ no need for separate cli tool or maven plugin
    ○ more script control than mvn lifecycle or web listener

    View Slide

  20. tl;dr
    Dropwizard is a java stack for building, deploying and
    monitoring RESTful web applications. It combines the
    power of jetty, jersey and metrics to provide a simple,
    coherent and powerful toolbox for developers and devops.
    Main features
    ● single standalone jar for deployment (embedded jetty)
    ● centralized application configuration using yaml
    ● developer friendly as it’s based on a proven java stack
    ● devops friendly with metrics, health checks and tasks
    ● admin friendly with commands
    ● creates production ready services very fast

    View Slide

  21. Dropwizard (why)

    View Slide

  22. Dropwizard: love it or leave it
    ● write code following dropwizard’s conventions and API
    ● provide the service configuration in a single yaml file
    ● java -jar service.jar configuration.yaml

    View Slide

  23. java is not so simple
    Traditional java
    ● It is used a lot in enterprise computing
    ○ bureaucratic, heavy, not friendly
    ● Ruby, Python, Javascript devs view of java
    ○ all the above + difficult, rigid, not very expressive
    Today java is having a renaissance.
    ● Many companies start using more and more java
    ○ codebase size and performance scales better
    ● Transition should be easy and smooth
    ○ traditional java (JEE, spring) does not help

    View Slide

  24. java has not fun lately
    To write a web application in say Ruby/Sinatra start with:
    get '/hello/:name' do
    # matches "GET /hello/foo" and "GET /hello/bar"
    # params[:name] is 'foo' or 'bar'
    "Hello #{params[:name]}!"
    end
    To write a web application in java that uses HTTP, MySQL
    and JSON first write code for a PublishedNetworkService
    that uses a RequestResponseProtocol over a
    ReliableStreamTransport, persists data to a
    StructuredDataStore and returns a RenderableMessage.
    Then configure the framework to wire HTTP, JDBC, JSON.

    View Slide

  25. dropwizard is simple
    It does not try to hide important things
    @Path("/") @Produces(MediaType.APPLICATION_JSON)
    public class TeamsServiceResource {
    @GET @Path("{name}")
    @Produces({MediaType.APPLICATION_JSON})
    public Team getMembers(@PathParam("name") String name) {}
    }
    It does not try to do magic behind the scenes
    env.addResource(TeamsServiceResource.class);
    Spring and JEE, try to simplify things but...

    View Slide

  26. but developers have evolved
    ● Open source movement. Developers love it
    ○ search repos, use best solutions, contribute back
    ● Libraries are a better paradigm than frameworks
    ○ work the same in different stacks and languages
    ● Some java techniques are abused by large frameworks
    ○ class scanning, runtime code generation, autowiring
    ● Developers prefer to assemble their own stacks
    ○ simpler, easier to use, understand and explain
    ○ see and handle the SQL generated by the ORM
    ○ know how the object graph is constructed
    ○ track HTTP request, response handling
    ○ debug and profile applications better

    View Slide

  27. Dropwizard: love it or leave it
    ● write code following dropwizard’s conventions and API
    ● provide the service configuration in a single yaml file
    ● java -jar service.jar configuration.yaml

    View Slide

  28. configuration
    A well-written document packaged with the application that
    describes the proper runtime environment. Essentially, it is
    a set of conventions that must be manually satisfied.
    Examples
    JEE: web.xml, application-server.xml, JNDI
    Spring: applicationContext.xml, runtime.properties
    GNU: ./configure; make; make install
    Microsoft: .MSI, Windows registry
    Note the last 2 are automated but not 100% bullet proof

    View Slide

  29. configuration
    - Who writes the configuration?
    - The developer of the application
    - Who satisfies the configuration?
    - The owner of the runtime environment
    There must be some common conventions so that these
    two can come to agreement. The most successful so far is
    JEE
    “The application is written according to JEE, packaged as a
    standard EAR and runs after deployment in a JEE server”

    View Slide

  30. configuration
    Is the runtime environment fixed during the application
    lifecycle, from prototype to production and beyond?
    ● mvn jetty:run - properties in pom.xml
    ● docker run - DockerFile
    ● vagrant up - VagrantFile
    ● JEE server - vendor specific xml files
    ● hosted datacenter - all the above
    ● cloud - git, cli + all the above
    An example using vagrant is Mozilla/Kuma the
    python/django framework that powers MDN

    View Slide

  31. configuration
    Who are the users of an application?
    ● Customers
    ● Developers
    ● QA
    ● Management
    ● Monitoring and Continuous Integration Tools
    ● An application who depends on one of your services
    ● A service that depends on one of your services
    ● The datacenter/cloud provisioner
    Clearly each one of these has different requirements and
    expectations regarding acquiring, configuring and running
    the application.

    View Slide

  32. configuration - tl;dr
    Given that the configuration problem is multifaceted and
    complex, dropwizard adopts a simple solution, a single
    yaml file, which is not the best for each different case but it
    suits all cases in the most uniform and straightforward way.
    The hope of the IT world is that progress in virtualization
    technology will make explicit configuration obsolete but we
    are not there yet.

    View Slide

  33. Dropwizard: love it or leave it
    ● write code following dropwizard’s conventions and API
    ● provide the service configuration in a single yaml file
    ● java -jar service.jar configuration.yaml

    View Slide

  34. one JAR for deploying the application
    ● service runs as an isolated single unix process
    ● can use existing unix process management tools
    ● no PermGen issues with hot deployment of WARs
    ● no application server configuration and maintenance
    ● no arcane deployment tools
    ● easy dependencies resolution, no classloader troubles
    ● no hidden application logs
    ● no GC tuning to work with multiple applications

    View Slide

  35. application server features
    ● run each application in an isolated environment
    ○ protect data and code of an application
    ● isolate the applications from the network
    ○ slow clients, connections failures and timeouts
    ● handle the details of HTTP
    ○ URL rewrites, redirections, static content
    ● centralized functionality for applications
    ○ security, caching, logging, basic error handling
    ● manage resources for applications
    ○ threads, database connections, queues, sockets
    ● provision applications
    ○ start/stop, scaling, load-balancing, fault tolerance etc

    View Slide

  36. application servers - now and then
    machines
    Then
    ● machines were big and expensive
    ● more applications than machines
    ● application servers
    ○ total provisioning of the machines cluster
    ○ multiplexing many applications on few machines
    Now
    ● smaller, cheaper, more capable machines
    ○ an application may run on a dedicated machine
    ● big datacenters, on site or on the cloud
    ● web scaling
    ○ more machines for traffic, replication, fault tolerance

    View Slide

  37. application servers - now and then
    applications
    Then
    ● big monolithic applications
    ○ all parts written using the same stack, JEE
    Now
    ● Write reusable services
    ○ use best available stack for each service
    ○ provision, deploy, monitor each one independently
    ● Assemble applications from services
    ○ loose coupling, reusability, distribution
    ● Applications are big but individual services need not
    ○ reusability at service level not only at code level

    View Slide

  38. application servers - now and then
    developers
    Then
    ● binding to a single vendor or standard seemed good
    ○ everybody uses the standard after all
    ● servers and standards are big and hard to master
    ○ but so are our applications
    Now
    ● Open source movement. Developers love it
    ○ search repos, use best solutions, contribute back
    ● Developers prefer to assemble their own stacks
    ○ simpler to use and understand
    ● Libraries are a better paradigm than frameworks
    ○ work the same in different stacks and languages

    View Slide

  39. application servers - now and then
    the modern datacenter
    Nowadays the whole datacenter, with its many distributed
    systems is the new application server
    The application gets the same support as with an
    application server but in a coarse-grain manner
    Static web sites
    Background workers
    User databases
    Analytics databases
    Web frontends
    API backpoints (Dropwizard)
    Message Queues
    load balancing
    fault tolerance
    job schedulers
    provisioners
    monitors
    worker machines
    for services
    proxies
    caches

    View Slide

  40. References
    Examples
    Dropwizard - the project site with excellent documentation
    Dropwizard-Jedis - An example application with Redis
    Articles for modern java stacks
    The Square stack - the java stack used at Square
    Heroku and Java - introduction for java developers
    The Twitter stack - the stack used at Twitter
    Talks
    Make features, not WAR - a presentation on InfoQ
    Autoscaling java Google I/O 2013 - dropwizard is compliant
    Twitter commons at Airbnb - a stack similar to dropwizard

    View Slide