.consulting .solutions .partnership Good for Karma: Configuration at Runtime Alexander Schwartz, Principal IT Consultant JFall 2015 Ede (NL) – 5. November 2015 

Configuration at Runtime © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 2 Everyday configuration changes 1 Archaius: Accessing changing configurations 2 etcd: Central Configuration Store 3 Togglz: Activating new Functionality on demand 4 Better with Configuration at Runtime? 5 

Configuration at Runtime © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 5 Everyday configuration changes 1 Archaius: Accessing changing configurations 2 etcd: Central Configuration Store 3 Togglz: Activating new Functionality on demand 4 Better with Configuration at Runtime? 5 

Everyday configuration changes If a configuration change requires a reboot, it requires a maintenance window © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 6 The customer database has a new IP address, please change the configuration of the on-line registration. No problem, I've changed it. I just have to re-start the application tonight! Am I a hero because I get up in the middle of the night to restart the application? 

Everyday configuration changes When new features are activated by a deployment, a fallback is difficult © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 7 The new feature in the application, could you please activate it? Ok, we roll out the new application version now. Then all users can access it. Should you really do this on a Friday afternoon? 

Everyday configuration changes Configurations at runtime saves time and hassle – and avoids maintenance windows © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 8 • Change application‘s parameters at runtime  target-adresses for services  names of folders  timeouts and business parameters • Central Configuration Store  configuration parameters  Service-Discovery • Activating new Features  for named users or groups  for a percentage of users  timed activation 

Everyday configuration changes Karma: Each of our actions has a consequence for ourselves © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 9 • We are responsible for our actions • Configuration changes should lead to the desired result – without affecting the users or other stakeholders • Insight: We know that configurations change • Kindness: We treat our customers with respect and don‘t need maintenance windows • Modesty: We don‘t want to be “heroes” that get up in the middle of the night Source: unknown 

Configuration at Runtime © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 10 Everyday configuration changes 1 Archaius: Accessing changing configurations 2 etcd: Central Configuration Store 3 Togglz: Activating new Functionality on demand 4 Better with Configuration at Runtime? 5 

Netflix Archaius Netflix Archaius manages access to parameters for Java clients © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 11 1. Loading configuration data from: property files, URLs, databases 2. Cascading Configurations for: standard configurations, different environments, single servers 3. Type safe access to the current value 4. Call backs, when a value changes Source: http://en.wikipedia.org/wiki/Archaius_tigris (CC) Hans Stieglitz 

Netflix Archaius Run Java with an environment parameter © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 12 java -Darchaius.deployment.environment=test ... # database.properties db.url=jdbc:postgresql://localhost:5432/standard-db # database-test.properties db.url=jdbc:postgresql://localhost:5432/test-db // Java Startup ConfigurationManager.loadCascadedPropertiesFromResources("database"); 

Netflix Archaius Pass a configuration file as a parameter © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 13 java -Darchaius.configurationSource.additionalUrls=file:///.../archaius.properties -Darchaius.fixedDelayPollingScheduler.delayMills=1000 -Darchaius.fixedDelayPollingScheduler.initialDelayMills=1000 –jar application.jar # archaius.properties app.timeout=1000 app.url=http://myserver:3000/ 

Netflix Archaius Archaius provides type safe access to the current value © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 14 The call back is activated every time a parameter changes its value. // construct type safe accessor for property DynamicStringProperty property = DynamicPropertyFactory.getInstance() .getStringProperty("db.url", "default value"); // access current value String url = property.get(); // get notified once the value changes property.addCallback(() -> { System.out.println("property has changed: " + property.get()); }); 

Netflix Archaius Pros and Cons Archaius © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 15 Pro: • Configurations can be rolled out as files as usual (for example using Chef, Puppet, Ansible and Salt) • Different storage back ends apart from files are supported • Hierarchical structure simplifies the setup • Proven library based on Apache Commons Configuration • Call backs enable complex reconfiguration Contra: • Changing multiple stores at the same time can be challenging Related projects: Apache DeltaSpike, Apache Commons Configuration 2.0 und Archaius 2.0, Apache Tamaya 

Configuration at Runtime 16 © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 Everyday configuration changes 1 Archaius: Accessing changing configurations 2 etcd: Central Configuration Store 3 Togglz: Activating new Functionality on demand 4 Better with Configuration at Runtime? 5 

CoreOS etcd etcd is a central configuration store © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 17 Can be used for configuration and service discovery • Easy to use: HTTP/REST with JSON • Reliable: highly available and consistent • Secure: optionally with SSL-Client-Certificates • Fast: several thousand write operations per second Operations: GET/PUT/DELETE/CAS/WAIT/TTL # PUT curl -L -X PUT http://127.0.0.1:4001/v2/keys/foo-service/container1 –d value="localhost:1111" # GET curl -L http://127.0.0.1:4001/v2/keys/foo-service/container1 # WAIT curl -L http://127.0.0.1:4001/v2/keys/foo-service?wait=true\&recursive=true 

CoreOS etcd + Hightower´s confd When a value changes in etcd, confd will update the configuration file © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 18 confd 

CoreOS etcd + Hightower´s confd When a value changes in etcd, confd will update the configuration file © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 19 • Service description (*.toml) • Template for a configuration file (*.tmpl) # nginx.toml [template] src = "nginx.tmpl" dest = "/etc/nginx/sites-enabled/app.conf" keys = [ "/foo-service" ] check_cmd = "/usr/sbin/nginx -t" reload_cmd = "/usr/sbin/service nginx reload" # nginx.tmpl upstream app_pool { {{ range getvs "/foo-service/*" }} server {{ . }}; {{ end }} } confd 

CoreOS etcd + Jurmous´ Java Client Jurmos Java client provides a Java API for etcd © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 20 • Java-API for the full etcd functionality • Implemented on top of Netty (for asynchronous, non-blocking IO) // Simple put etcd.put("/foo-service/container1","localhost:1111").send(); // Wait for next change on foo EtcdResponsePromise promise = etcd.getDir("/foo-service/") .recursive().waitForChange().send(); // Java 8 lambda construction promise.addListener(promise -> { // do something with change }); 

CoreOS etcd Pros und Cons etcd © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 21 Pro: • Easy to access (REST), simple installation (Go-Binary), transport security (SSL) • Changes are published immediately to all clients • Generic structure for flexible usage Contra: • No access controls on a client level • Young open source project Add-ons and alternatives: • Vulcan (HTTP Load Balancer, stores its configuration in etcd) • SkyDNS (service discovery via DNS, stores its information in etcd) • Apache Zookeeper (central configuration store and synchronization) • Hazelcast (leader election, distributed locks, topics, in-memory-data-grid) • Netflix Eureka (service discovery) • Hashicorp Consul (service discovery and configuration store) 

Configuration at Runtime 22 © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 Everyday configuration changes 1 Archaius: Accessing changing configurations 2 etcd: Central Configuration Store 3 Togglz: Activating new Functionality on demand 4 Better with Configuration at Runtime? 5 

Togglz: Feature Toggles for Java Togglz switches functionality on and off at runtime © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 23 Scenarios: • Not yet released functions are active only in test environments • Activation and deactivation can be carried out via a web front end • Rules show new functionality only to some users or activate it at a given time 

Togglz: Feature Toggles for Java Togglz has a Web UI © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 24 Status and rollout strategy can be changed for each feature at runtime. 

Togglz: Feature Toggles for Java Feature toggles are queried in the source code © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 25 Enums declare features in Java code. They are queried in the code before the new code is called. // declaration and documentation of features public enum Features implements Feature { @Label("First Feature") FEATURE_ONE; public boolean isActive() { return FeatureContext.getFeatureManager().isActive(this); } } // making use of toggles within the code if(Features.FEATURE_ONE.isActive()) { // do new exciting stuff here } 

Togglz: Feature Toggles for Java Pros and Cons Feature Toggles © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 26 • Feature toggles are a known and well understood pattern • Multiple implementations for different programming languages exist • Feature toggles increase the complexity of your tests • When a feature is fully rolled out, you’ll need to remove the feature toggle It might be worth the additional effort, • if you have frequent releases (Continuous Delivery), where unfinished features would otherwise block finished features • if feature toggles help you to avoid long running feature branches that lead to a big merge at the end • if an early gradual roll out provides information for the completion of the feature 

Configuration at Runtime 27 © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 Everyday configuration changes 1 Archaius: Accessing changing configurations 2 etcd: Central Configuration Store 3 Togglz: Activating new Functionality on demand 4 Better with Configuration at Runtime? 5 

Configuration at Runtime Better with Configuration at Runtime? Yes! © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 28 Access to a changing configuration • Helps already during development of the application • Minimal initial effort, great flexibility for production Central Configuration Store • Cross-application configuration • Service discovery in cloud scenarios Feature Toggles • If it simplifies development and deployment • If early feedback from production helps to improve the feature @ahus1de 

Configuration at Runtime Links © msg | Good for Karma: Configuration at Runtime | Alexander Schwartz | JFall Ede | 5. November 2015 29 Netflix Archaius: https://github.com/Netflix/archaius Apache Commons Configuration: http://commons.apache.org/proper/commons-configuration/ Apache Tamaya: http://tamaya.incubator.apache.org/ Overview of different existing solutions: http://javaeeconfig.blogspot.de/2014/08/ overview-of-existing-configuration.html Saltstack and Consul Examples https://github.com/ahus1/saltconsul-examples Togglz: http://www.togglz.org/ Übersicht über Feature Toggle Implementierungen: http://www.beautifulbuilds.com/feature-toggle-frameworks-list/ @ahus1de CoreOS etcd: https://github.com/coreos/etcd Jurnous etcdj: https://github.com/jurmous/etcd4j Vulcan: http://www.vulcanproxy.com/ SkyDNS: https://github.com/skynetservices/skydns Netflix Eureka: https://github.com/Netflix/eureka Hazelcast: http://hazelcast.com/ Apache Zookeeper: https://zookeeper.apache.org/ Netflix exhibitor: https://github.com/Netflix/exhibitor HashCorp Consul: http://consul.io/ 

.consulting .solutions .partnership Alexander Schwartz Principal IT Consultant +49 171 5625767 [email protected] @ahus1de msg systems ag (Headquarters) Robert-Buerkle-Str. 1, 85737 Ismaning Germany www.msg-systems.com