Introduction Dropwizard is a Java framework for developing ops- friendly, high-performance, RESTful web services. A modern standalone stack like play and twitter libraries (commons, ostrich, server) with a cloud/devops orientation Named after a comic strip http://gunshowcomic.com/316 A new tool to solve a recurring problem today. Tools, problems, environments, techniques and assumptions never stay constant but evolve.
The stack Restful web services are composed of: ● REST API endpoints ● templates and their rendering engines ● database access libraries ● additional servlets and filters ● special admin servlets like /abort, /reload ● lifecycle bound objects (executors or connection pools) ● health checks (periodically running application tests) ● metrics monitors (check behavior in production) For all the above, there are well known implementations that provide APIs for using, integrating and extending them.
The stack Dropwizard, a little bit more than opinionated glue code for ● Jetty - embedded, tunable web server ● Jersey - JAX-RS, higher level abstraction over servlets ● Jackson - JSON processor ● Logback - successor of log4j for logging ● Metrics - a library for application runtime metrics ● Liquibase - a tool for database refactoring/migration ● Hibernate - the well known ORM ● JDBI - an alternative database access API ● Mustache, Freemarker - template engines
Metrics - mind the gap If you don’t measure it, you can’t optimize it. Compare A. Our code uses HAL and runs on death star B. At ~2,000 req/sec, latency jumps from 13ms to 453ms. Provides Gauge, Counter, Meter, Histogram, Timer, Health Check programmatically or instrumented (@Timed, @Metered, @Gauge) Monitoring files, logs, http/json, monitoring systems(Graphite, Ganglia)
Liquibase - database refactoring ● describe the changes
● liquibase.sh {update, diff, status, tag, dump, rollback, ..} ● maintains a table DATABASECHANGELOG with the changes ○ ID AUTHOR FILENAME DATEEXECUTED TAG LIQUIBASEVERSION
Dropwizard usage Create a maven project with dropwizard’s dependencies ● write code following dropwizard’s conventions and API ● provide the service configuration in a single yaml file ● java -jar service.jar server configuration.yaml From my experience people new to dropwizard either love it because of all these features or hate it because of a single one of them (each one gets equal 33.33% disapproval) More on these later.
Configuration Everything goes into one, global yaml file. No other way to configure. No .properties or .xml files in the classpath or in META-INF/ or in “hidden/non standard” server places logging: level: INFO http: port: 8080 database: driverClass: org.postgresql.Driver httpClient: maxConnections: 10 Dropwizard provides sensible defaults for most uses
API Underneath, it’s still jersey. For standard JEE you just pack this in a war and deploy @Path("/") @Produces(MediaType.APPLICATION_JSON) public class TeamsServiceResource { @Context private RedisPool pool; @GET @Path("{name}") @Produces({MediaType.APPLICATION_JSON}) public Team getTeam(@PathParam("name") String name) { // implementation omitted } }
API For dropwizard, first you create a simple pojo to hold your configuration. The configuration.yaml file will be deserialized to this pojo using Jackson and will be validated with hibernate validator(@NotNull, @NotEmpty, @Min, ...) Then extend dropwizard’s default service, to create yours public class TeamsService extend Service { // override what is needed }
API Dropwizard has a modular architecture. Each unit of reusability is called a bundle. Think of them as modules or plugins. A service out of the box uses only the bundles for jetty, jersey and metrics. Additional ones (hibernate, views, liquibase) must be declared for a service public void initialize( Bootstrap bootstrap) { bootstrap.setName("dropwizard-example"); bootstrap.addBundle(new AssetsBundle("/assets/", "/")); bootstrap.addBundle(new ViewBundle()); bootstrap.addBundle(new MigrationsBundle()); }
API Then declare what your service will use from all the things bundles provide. These are gathered in an environment. Note: be very explicit. No shortcuts like autowiring public void run( TeamsServiceConfiguration conf, Environment env) { env.addResource(TeamsServiceResource.class); env.addResource(new TeamsServiceResource(“custom”); env.addHealthCheck(new TeamsHealthCheck(“teamshealth”)); env.managedExecutorService(executor); env.addTask(new ShutdownTask(“shutdown”); env.addServlet(new MyViewServlet()); }
API $ mvn package $ java -jar target/service.jar server configuration.yaml Service Endpoints GET http://localhost:8080/teams/ GET http://localhost:8080/teams/{name} POST http://localhost:8080/teams/{name} Admin Console POST http://localhost:8081/tasks/gc (dropwizard) POST http://localhost:8081/tasks/shutdown (user) GET http://localhost:8081/ping GET http://localhost:8081/healthcheck GET http://localhost:8081/threads GET http://localhost:8081/metrics
Other goodies Bundles ● easy testing for jersey representations and resources ● OAuth2 authentication (no token exchange, only auth) ● database migrations with liquibase ● instrumented rest client with Apache http or jersey Commands manage applications with custom cli arguments and code java -jar hello-world.jar db status helloworld.yml java -jar hello-world.jar queues persist helloworld.yml
API - Notes Bundles organize configuration, build environments and extend dropwizard. They usually are build on jersey’s SPI // from views bundle env.addProvider(ViewMessageBodyWriter.class); // from hibernate bundle DatabaseConfiguration dbConfig = getDatabaseConfiguration(configuration); sessionFactory = sessionFactoryFactory.build( environment, dbConfig, entities); env.addProvider(new UnitOfWorkResourceMethodDispatchAdapter(sessionFactory));
API - Notes Dropwizard tries to keep the application footprint as small as possible. You must declare everything you are going to use. Sometimes this is simple. Templates for example: public class TeamView extends View { public TeamView(String name, List members) { // the view file goes on resources/same package super("Team.mustache"); } }
API - Notes But sometimes it can be tedious. // hibernate bundle protected HibernateBundle( Class> entity, Class>... entities) {...} The bundle builds the sessionFactory entirely programmatically. It consults only the yaml configuration not hibernate.cfg.xml. To make it easier dropwizard provides AbstractDao and @UnitOfWork.
API - Notes Good integration example: liquibase - database refactoring
● No additional configuration like liquibase.properties ● src/main/resources/migrations.xml packed with service ● Liquibase runner integrated as dropwizard commands ○ java -jar hello-world.jar db migrate helloworld.yml ○ no need for separate cli tool or maven plugin ○ more script control than mvn lifecycle or web listener
tl;dr Dropwizard is a java stack for building, deploying and monitoring RESTful web applications. It combines the power of jetty, jersey and metrics to provide a simple, coherent and powerful toolbox for developers and devops. Main features ● single standalone jar for deployment (embedded jetty) ● centralized application configuration using yaml ● developer friendly as it’s based on a proven java stack ● devops friendly with metrics, health checks and tasks ● admin friendly with commands ● creates production ready services very fast
Dropwizard: love it or leave it ● write code following dropwizard’s conventions and API ● provide the service configuration in a single yaml file ● java -jar service.jar configuration.yaml
java is not so simple Traditional java ● It is used a lot in enterprise computing ○ bureaucratic, heavy, not friendly ● Ruby, Python, Javascript devs view of java ○ all the above + difficult, rigid, not very expressive Today java is having a renaissance. ● Many companies start using more and more java ○ codebase size and performance scales better ● Transition should be easy and smooth ○ traditional java (JEE, spring) does not help
java has not fun lately To write a web application in say Ruby/Sinatra start with: get '/hello/:name' do # matches "GET /hello/foo" and "GET /hello/bar" # params[:name] is 'foo' or 'bar' "Hello #{params[:name]}!" end To write a web application in java that uses HTTP, MySQL and JSON first write code for a PublishedNetworkService that uses a RequestResponseProtocol over a ReliableStreamTransport, persists data to a StructuredDataStore and returns a RenderableMessage. Then configure the framework to wire HTTP, JDBC, JSON.
dropwizard is simple It does not try to hide important things @Path("/") @Produces(MediaType.APPLICATION_JSON) public class TeamsServiceResource { @GET @Path("{name}") @Produces({MediaType.APPLICATION_JSON}) public Team getMembers(@PathParam("name") String name) {} } It does not try to do magic behind the scenes env.addResource(TeamsServiceResource.class); Spring and JEE, try to simplify things but...
but developers have evolved ● Open source movement. Developers love it ○ search repos, use best solutions, contribute back ● Libraries are a better paradigm than frameworks ○ work the same in different stacks and languages ● Some java techniques are abused by large frameworks ○ class scanning, runtime code generation, autowiring ● Developers prefer to assemble their own stacks ○ simpler, easier to use, understand and explain ○ see and handle the SQL generated by the ORM ○ know how the object graph is constructed ○ track HTTP request, response handling ○ debug and profile applications better
Dropwizard: love it or leave it ● write code following dropwizard’s conventions and API ● provide the service configuration in a single yaml file ● java -jar service.jar configuration.yaml
configuration A well-written document packaged with the application that describes the proper runtime environment. Essentially, it is a set of conventions that must be manually satisfied. Examples JEE: web.xml, application-server.xml, JNDI Spring: applicationContext.xml, runtime.properties GNU: ./configure; make; make install Microsoft: .MSI, Windows registry Note the last 2 are automated but not 100% bullet proof
configuration - Who writes the configuration? - The developer of the application - Who satisfies the configuration? - The owner of the runtime environment There must be some common conventions so that these two can come to agreement. The most successful so far is JEE “The application is written according to JEE, packaged as a standard EAR and runs after deployment in a JEE server”
configuration Is the runtime environment fixed during the application lifecycle, from prototype to production and beyond? ● mvn jetty:run - properties in pom.xml ● docker run - DockerFile ● vagrant up - VagrantFile ● JEE server - vendor specific xml files ● hosted datacenter - all the above ● cloud - git, cli + all the above An example using vagrant is Mozilla/Kuma the python/django framework that powers MDN
configuration Who are the users of an application? ● Customers ● Developers ● QA ● Management ● Monitoring and Continuous Integration Tools ● An application who depends on one of your services ● A service that depends on one of your services ● The datacenter/cloud provisioner Clearly each one of these has different requirements and expectations regarding acquiring, configuring and running the application.
configuration - tl;dr Given that the configuration problem is multifaceted and complex, dropwizard adopts a simple solution, a single yaml file, which is not the best for each different case but it suits all cases in the most uniform and straightforward way. The hope of the IT world is that progress in virtualization technology will make explicit configuration obsolete but we are not there yet.
Dropwizard: love it or leave it ● write code following dropwizard’s conventions and API ● provide the service configuration in a single yaml file ● java -jar service.jar configuration.yaml
one JAR for deploying the application ● service runs as an isolated single unix process ● can use existing unix process management tools ● no PermGen issues with hot deployment of WARs ● no application server configuration and maintenance ● no arcane deployment tools ● easy dependencies resolution, no classloader troubles ● no hidden application logs ● no GC tuning to work with multiple applications
application server features ● run each application in an isolated environment ○ protect data and code of an application ● isolate the applications from the network ○ slow clients, connections failures and timeouts ● handle the details of HTTP ○ URL rewrites, redirections, static content ● centralized functionality for applications ○ security, caching, logging, basic error handling ● manage resources for applications ○ threads, database connections, queues, sockets ● provision applications ○ start/stop, scaling, load-balancing, fault tolerance etc
application servers - now and then machines Then ● machines were big and expensive ● more applications than machines ● application servers ○ total provisioning of the machines cluster ○ multiplexing many applications on few machines Now ● smaller, cheaper, more capable machines ○ an application may run on a dedicated machine ● big datacenters, on site or on the cloud ● web scaling ○ more machines for traffic, replication, fault tolerance
application servers - now and then applications Then ● big monolithic applications ○ all parts written using the same stack, JEE Now ● Write reusable services ○ use best available stack for each service ○ provision, deploy, monitor each one independently ● Assemble applications from services ○ loose coupling, reusability, distribution ● Applications are big but individual services need not ○ reusability at service level not only at code level
application servers - now and then developers Then ● binding to a single vendor or standard seemed good ○ everybody uses the standard after all ● servers and standards are big and hard to master ○ but so are our applications Now ● Open source movement. Developers love it ○ search repos, use best solutions, contribute back ● Developers prefer to assemble their own stacks ○ simpler to use and understand ● Libraries are a better paradigm than frameworks ○ work the same in different stacks and languages
application servers - now and then the modern datacenter Nowadays the whole datacenter, with its many distributed systems is the new application server The application gets the same support as with an application server but in a coarse-grain manner Static web sites Background workers User databases Analytics databases Web frontends API backpoints (Dropwizard) Message Queues load balancing fault tolerance job schedulers provisioners monitors worker machines for services proxies caches
References Examples Dropwizard - the project site with excellent documentation Dropwizard-Jedis - An example application with Redis Articles for modern java stacks The Square stack - the java stack used at Square Heroku and Java - introduction for java developers The Twitter stack - the stack used at Twitter Talks Make features, not WAR - a presentation on InfoQ Autoscaling java Google I/O 2013 - dropwizard is compliant Twitter commons at Airbnb - a stack similar to dropwizard
anastasop
octu0
k0kubun
pineapplecandy
soutaro
ivargrimstad
torounit
ancaghenade
yukana
gishi_yama
little_rubyist
khmer495
sagawa23
paulrobertlloyd
iamctodd
jensimmons
addyosmani
mojombo
chrisshort
chrislema
mclloyd
holman
jrom
akosma
lara