The Song of JBoss and Chef

Slide 1

Slide 1 text

The Song of JBoss and Chef The Song of JBoss and Chef DevOpsDays Warsaw 2016 Tomasz Torcz

Slide 2

Slide 2 text

Agenda - who am I? - what is JBoss? What is Chef? - why is it complicated? - pro & cons

Slide 3

Slide 3 text

Who am I?

Slide 4

Slide 4 text

Banks. Banks are special

Slide 5

Slide 5 text

JBoss? The JBoss Enterprise Application Platform is a Java EE-based application server runtime platform used for building, deploying, and hosting highly-transactional Java applications and services.

Slide 6

Slide 6 text

Chef Configuration management tool written in Ruby and Erlang. It uses a pure-Ruby, domain-specific language for writing system configuration "recipes".

Slide 7

Slide 7 text

The Song of The Song of JBoss and Chef JBoss and Chef

Slide 8

Slide 8 text

The Song of The Song of JBoss and Chef JBoss and Chef

Slide 9

Slide 9 text

Why did we care? - already automated lower levels of stack Paweł Bartusch @ https://youtu.be/U50w_ZoQygA - needed for full CI/CD pipeline Waldemar Piszczewiat @ https://youtu.be/9-X8qytNznA - crucial to have accounting git blame

Slide 10

Slide 10 text

Architecture

Slide 11

Slide 11 text

Architecture vroom! vrooom!

Slide 12

Slide 12 text

Workflow - multiple teams, separation of duties - internal developers - external vendors

Slide 13

Slide 13 text

How do you configure JBoss?

Slide 14

Slide 14 text

1. Web Console

Slide 15

Slide 15 text

How do you configure JBoss? 1. Web Console 2. Command Line Interface

Slide 16

Slide 16 text

How do you configure JBoss? 1. Web Console 2. Command Line Interface 3. XML files (domain.xml, host*.xml, etc.)

Slide 17

Slide 17 text

How do you configure JBoss? 1. Web Console 2. Command Line Interface 3. XML files (domain.xml, host*.xml, etc.)

Slide 18

Slide 18 text

JBoss CLI "logging": { … "root-logger": { "handlers": "[\"File_rotating\"]", "level": "INFO" ~/bin/jboss-cli.sh --controller=10.14.x.y:9999 --connect –commands='/profile=full- ha/subsystem=logging/root-logger=ROOT:write- attribute(name=handlers, value=["File_rotating"])'

Slide 19

Slide 19 text

Reading the JBoss output { "outcome" => "failed", "failure-description" => {"host-failure- descriptions" => {"vhost02" => "JBAS014749: Operation handler failed: null"}}, "rolled-back" => true }

Slide 20

Slide 20 text

Reading the JBoss output eval(JBoss’ output) … success! => {"outcome"=>"failed", "failure-description"=>{ "host-failure-descriptions"=>{ "vhost02"=>"JBAS014749: Operation handler failed: null" } }, "rolled-back"=>true}

Slide 21

Slide 21 text

Reading the JBoss output "extension" => { "org.jboss.as.webservices" => undefined, … NameError: undefined local variable or method `undefined' for main:Object

Slide 22

Slide 22 text

Reading the JBoss output "extension" => { "org.jboss.as.webservices" => undefined, … NameError: undefined local variable or method `undefined' for main:Object sed 's/undefined/{}/g' … success!

Slide 23

Slide 23 text

Reading the JBoss output "content" => [{"hash" => bytes { 0x87, 0x17, 0x0a, 0xd7, 0xff, 0x72 }}] SyntaxError: (eval):1: syntax error, unexpected =>, expecting end-of-input "content" => [{"hash" => bytes { 0x87, 0x... ^ sed 's/{"hash" => bytes {//g' … success!

Slide 24

Slide 24 text

JBoss output massaging function c.gsub!('=> undefined', '=> {}') c.gsub!(/([[:blank:]]*)(-?\b[0-9]+)L\b(,?)/, '\1\2\3') # 1234L -> "1234" c.gsub!('expression "', '"expression ') c.gsub!('{"hash" => bytes {', '') # \_ 'hash => bytes' to array c.gsub!(/([ \t]+)}}(\],?$)/,'\1\2') # / c.gsub!(/'/, '\\\\' + "'") # don't ask :/ c.gsub!(/([[:blank:]]*$)(". +"[[:blank:]]=\>[[:blank:]]".+")$(,?)/, '\2\3') c.gsub!(/(.+\[)$(.+)$(\].+)/,'\1\2\3') c.gsub!(/"/, '\'')

Slide 25

Slide 25 text

JBoss output massaging function commit 3b54e51a0fc9149330e32c42fd784ad3b20850ad Author: Bartosz Trudnowski Date: Tue Feb 23 10:00:58 2016 +0100 - c.gsub!(/([[:blank:]]$)(".+"[[:blank:]]=\>[[:blank:]]".+")$(,?)/, '\2\3') + c.gsub!(/([[:blank:]]*$)(".+"[[:blank:]]=\>[[:blank:]]".+")$(,?)/, '\2\3')

Slide 26

Slide 26 text

JBoss output format

Slide 27

Slide 27 text

Other JBoss annoyances - config keywords incoherence - punctuation frenzy

Slide 28

Slide 28 text

The good and the bad What lessons did we learn?

Slide 29

Slide 29 text

PROD cluster

Slide 30

Slide 30 text

PROD cluster 1 hour later

Slide 31

Slide 31 text

Handover bph_jboss_srv bph_jboss_ml bph_jboss_app

Slide 32

Slide 32 text

Handover – number of commits bph_jboss_srv bph_jboss_ml bph_jboss_app 80 Tomasz Torcz 254 Tomasz Torcz 50 Tomasz Torcz 9 Waldemar Piszczewiat 67 Bartosz Trudnowski 10 Bartosz Trudnowski 4 Krzysztof Kowalczyk 17 Waldemar Piszczewiat 7 Waldemar Piszczewiat 4 Bartosz Trudnowski 4 Bartosz Dempc 6 Krzysztof Kowalczyk 3 Waldemar Buczak 1 Waldemar Buczak 1 Grzegorz Prusik 1 Grzegorz Krysik

Slide 33

Slide 33 text

Data bags – number of commits dev test prod 218 Waldemar Piszczewiat 273 Waldemar Piszczewiat 40 Bartosz Dempc 53 Tomasz Torcz 194 Bartosz Dempc 17 Paweł Rodziewicz 8 Bartosz Trudnowski 107 Grzegorz Prusik 13 Grzegorz Prusik 5 Technical CICD account 49 Rafał Markowski 6 Michał Szor 3 Waldemar Buczak 43 Grzegorz Krysik 6 Waldemar Piszczewiat 2 Bartosz Dempc 24 Grzegorz Słaby 5 Grzegorz Słaby … …

Slide 34

Slide 34 text

Reading the Chef output * execute[Updating SG01 property spring.profiles.active to cbpm,sit,mockReport,localStorage] action run (skipped due to not_if)

Slide 35

Slide 35 text

Managable config - many similar, but maybe different clusters? - json-diff - testing, moving configuration - copy&paste - easy to revert

Slide 36

Slide 36 text

Diverging templates cicd-dev-apps/dev/ … /GsbTytan.properties.erb cicd-uat-apps/dk1/ … /GsbTytan.properties.erb cicd-uat-apps/dk2/ … /GsbTytan.properties.erb cicd-uat-apps/tg/ … /GsbTytan.properties.erb cicd-sit-apps/sit/ … /GsbTytan.properties.erb cicd-sit-apps/sitstab/ … /GsbTytan.properties.erb cicd-prod-apps/prod/ … /GsbTytan.properties.erb - moved to external git repos

Slide 37

Slide 37 text

The Song of JBoss and Chef The Song of JBoss and Chef DevOpsDays Warsaw 2016 Tomasz Torcz