Behind the scenes of the new Groovy website and documentation

The new Groovy website is the perfect example of the ""eating your own dogfood"" mantra. In this talk, we will explain the technologies used to build the new Groovy website, including Gradle, the Groovy MarkupTemplateEngine, Asciidoctor and automation of publication through TeamCity. You will learn how we put all those pieces together to create a toolchain that allows us to release new versions of Groovy will a fully tested documentation, as well as the generation of a static website only using Groovy technologies!

Transcript

1. Guillaume Laforge 

   @glaforge
   Behind the new
   Groovy website
   

   View Slide

2. Guillaume Laforge
   Groovy project lead
   @glaforge
   http://glaforge.appspot.com
   

   View Slide

3. Cédric Champeau
   Groovy core developer
   @cedricchampeau
   http://melix.github.io/blog/
   

   View Slide

4. Cédric Champeau
   Groovy core developer
   @cedricchampeau
   http://melix.github.io/blog/
   

   View Slide

5. Documentation
   

   View Slide

6. Before
   

   View Slide

7. View Slide

8. View Slide

9. After
   

   View Slide

10. After
    

    View Slide

11. After
    

    View Slide

12. After
    

    View Slide

13. After
    

    View Slide

14. @glaforge
    New documentation goals
    • One documentation version per release
    • Integrated in the build process
    • Should be easy to write
    • Should be tested
    10
    

    View Slide

15. Executable
    documentation
    

    View Slide

16. @glaforge
    How? Asciidoctor to the rescue!
    • Markup based documentation engine
    • Derived from Asciidoc
    • Lightweight syntax
    • Feature rich
    • Highly customizable
    12
    

    View Slide

17. @glaforge
    How? Asciidoctor to the rescue!
    13
    = Syntax

    

    This chapter covers the syntax of the Groovy programming
    language.

    The grammar of the language derives from the Java grammar,

    but enhances it with specific constructs for Groovy, and
    allows certain simplifications.

    

    == Comments

    

    === Single line comment

    

    Single line comments start with `//` and can be found at
    any position in the line.

    The characters following `//`, till the end of the line,
    are considered part of the comment.
    

    View Slide

18. @glaforge
    Directory structure
    14
    src/spec/
    assets
    doc
    test
    test-resources
    

    View Slide

19. @glaforge
    Directory structure
    14
    src/spec/
    assets
    doc
    test
    test-resources
    images, css, javascript…
    

    View Slide

20. @glaforge
    Directory structure
    14
    src/spec/
    assets
    doc
    test
    test-resources
    images, css, javascript…
    asciidoctor source files
    

    View Slide

21. @glaforge
    Directory structure
    14
    src/spec/
    assets
    doc
    test
    test-resources
    images, css, javascript…
    asciidoctor source files
    documentation specific tests
    

    View Slide

22. @glaforge
    Directory structure
    14
    src/spec/
    assets
    doc
    test
    test-resources
    images, css, javascript…
    asciidoctor source files
    documentation specific tests
    test sources
    

    View Slide

23. Every single snippet
    of code you see in
    the documentation
    is tested
    

    View Slide

24. OMG! Executable
    documentation!
    

    View Slide

25. @glaforge
    Including external snippets of code
    17
    [source,groovy]

    ----

    include::src/spec/test/SyntaxTest.groovy[tags=single_line_comment,indent=0]

    ----
    

    View Slide

26. @glaforge
    Including external snippets of code
    17
    [source,groovy]

    ----

    include::src/spec/test/SyntaxTest.groovy[tags=single_line_comment,indent=0]

    ----
    Groovy syntax highlighting
    for the block of code
    

    View Slide

27. @glaforge
    Including external snippets of code
    17
    [source,groovy]

    ----

    include::src/spec/test/SyntaxTest.groovy[tags=single_line_comment,indent=0]

    ----
    Groovy syntax highlighting
    for the block of code
    include directive
    

    View Slide

28. @glaforge
    Including external snippets of code
    17
    [source,groovy]

    ----

    include::src/spec/test/SyntaxTest.groovy[tags=single_line_comment,indent=0]

    ----
    Groovy syntax highlighting
    for the block of code
    include directive source file to include
    

    View Slide

29. @glaforge
    Including external snippets of code
    17
    [source,groovy]

    ----

    include::src/spec/test/SyntaxTest.groovy[tags=single_line_comment,indent=0]

    ----
    Groovy syntax highlighting
    for the block of code
    include directive source file to include just include a portion
    

    View Slide

30. @glaforge
    Snippet demarcation
    18
    void testSingleLineComment() {

    // tag::single_line_comment[]

    // a standalone single line comment

    println "hello" // a comment till the end of the line

    // end::single_line_comment[]

    }
    

    View Slide

31. @glaforge
    Snippet demarcation
    18
    void testSingleLineComment() {

    // tag::single_line_comment[]

    // a standalone single line comment

    println "hello" // a comment till the end of the line

    // end::single_line_comment[]

    }
    tag::tag_name[]
    end::tag_name[]
    

    View Slide

32. @glaforge
    Narrative
    19
    They can be seen as *interfaces* carrying both *default implementations*
    and *state*. A trait is defined using the `trait` keyword:

    

    [source,groovy]

    ----

    include::src/spec/test/TraitsSpecTest.groovy[tags=flying_simple,indent=0]

    ----

    <1> declaration of a trait

    <2> declaration of a method inside a trait
    

    View Slide

33. @glaforge
    Callouts
    20
    class TraitsSpecTest extends GroovyTestCase {

    void testTraitDeclaration() {

    assertScript '''
    // tag::flying_simple[]

    trait FlyingAbility { // <1>

    String fly() { "I'm flying!" } // <2>

    }

    // end::flying_simple[]

    

    // tag::bird[]

    class Bird implements FlyingAbility {} // <1>

    def b = new Bird() // <2>

    assert b.fly() == "I'm flying!" // <3>

    // end::bird[]

    '''

    }
    

    View Slide

34. @glaforge
    Narrative and callouts rendered
    21
    

    View Slide

35. @glaforge
    Gradle
    22
    apply plugin: 'org.asciidoctor.gradle.asciidoctor'

    

    asciidoctor {

    def (full, major, minor, patch, flavor) =
    (groovyVersion =~ /(\d+)\.(\d++)\.(\d+)(?:-(.+))?/)[0]

    logDocuments = true

    sourceDir = project.file('src/spec/doc')

    options = [

    attributes: [

    'rootProjectDir': rootProject.projectDir,

    'source-highlighter': 'prettify',

    groovyversion: groovyVersion,

    'groovy-major-version': major,

    'groovy-minor-version': minor,

    'groovy-patch-version': patch,

    'groovy-full-version': groovyVersion,

    'groovy-short-version': "${major}.${minor}",

    doctype: 'book',

    revnumber: groovyVersion,

    icons: 'font',

    toc2: '',

    specfolder: 'src/spec/doc',

    linkcss: '',

    stylesheet: "assets/css/style.css",

    encoding: 'utf-8',

    toclevels: 10,

    numbered: ''

    ]

    ]

    }
    

    View Slide

36. @glaforge
    In summary
    • Documentation rewritten from scratch
    – help still needed to fill the many gaps though!
    • Narrative written using Asciidoctor
    • Code must belong to test cases
    • Makes guarantees that doc is up-to-date
    • Integrated through Gradle
    23
    

    View Slide

37. Website
    

    View Slide

38. After…
    

    View Slide

39. After…
    

    View Slide

40. After…
    

    View Slide

41. After…
    

    View Slide

42. After…
    

    View Slide

43. @glaforge
    Objectives
    • Facelifting and responsiveness
    • Easy maintenance / hosting
    • Well organized
    • Make contributions easy
    • Eat your own dog food
    26
    

    View Slide

44. @glaforge
    Responsiveness
    27
    

    View Slide

45. @glaforge
    Responsiveness
    27
    Drop-down menu
    

    View Slide

46. @glaforge
    Responsiveness
    27
    Drop-down menu
    Readable from any modern
    browsers, both desktop
    and mobile
    

    View Slide

47. @glaforge
    Responsiveness
    27
    Drop-down menu
    Lead to Github’s
    online editor!
    Readable from any modern
    browsers, both desktop
    and mobile
    

    View Slide

48. @glaforge
    Technical solution
    • Bootstrap theme by Damien Vitrac
    • Fully static
    • Uses a Groovy DSL for contents
    • Forkable on GitHub
    • Uses the Markup Template Engine
    • Baked with Gradle
    28
    

    View Slide

49. @glaforge
    Markup template engine
    • Introduced in Groovy 2.3
    • Fully fledged template engine
    • Primarily aimed at generating markup contents
    • Templates look like Groovy builders
    • Statically compiled templates
    • Optional type checking
    29
    

    View Slide

50. @glaforge
    Markup template engine
    • No more closing tag nightmare
    • Can be very fast once templates are compiled
    • Support for i18n
    • Used in both Spring Boot and Ratpack
    30
    

    View Slide

51. @glaforge
    A basic template
    31
    downloads.each { dl ->
    div(class:'download') {

    a(href:dl.url, "Groovy $dl.version")
    }

    }
    

    View Slide

52. @glaforge
    Template rendered
    32
    

    Groovy 2.3.8

    

    

    Groovy 2.3.9

    
    

    View Slide

53. @glaforge
    Includes
    33
    include template: 'other_template.tpl'

    include unescaped: 'raw.txt'

    include escaped: 'to_be_escaped.txt'
    

    View Slide

54. @glaforge
    Layouts
    34
    layout 'layouts/main.groovy', true,

    pageTitle: 'The Groovy programming language',

    mainContent: contents {

    div(id: 'content') {

    include unescaped: 'html/index.html'

    

    section(class: "row colset-3-article") {

    h1 { strong "Groovy events you shouldn't miss!" }

    allEvents.keySet().take(3).each { String eventName ->

    Event event = allEvents[eventName]

    article {

    div(class: 'content') {

    …
    

    View Slide

55. @glaforge
    Layout
    35
    layout 'layouts/page.groovy', true,

    mainContent: contents {

    // 'Content'

    div {

    include template: 'includes/topmenu.groovy'

    

    // main contents goes here!

    mainContent()
    

    include template: 'includes/bottommenu.groovy'

    }

    }
    

    View Slide

56. Starting the
    engine
    

    View Slide

57. @glaforge
    Setting up the engine
    37
    def tplConf = new TemplateConfiguration(
    autoIndent : true,
    autoNewLine: true
    ) 

    def classLoader = …
    

    tplEngine = new MarkupTemplateEngine(

    classLoader, tplConf,

    new CachingTemplateResolver())
    

    View Slide

58. @glaforge
    Template resolver
    • Allows…
    – custom template resolution/loading
    – overriding i18n behavior
    – custom caching strategies
    38
    

    View Slide

59. @glaforge
    Rendering
    39
    tplEngine.createTemplateByPath("pages/${page}.groovy")

    .make(model)
    

    View Slide

60. @glaforge
    Markup template engine
    • Even if a template contains unresolved variables
    – Statically compiled
    – Uses type checking extensions under the hood
    – Allows optional type checking of model
    – Even faster if type checked
    40
    

    View Slide

61. @glaforge
    Type checked models
    41
    modelTypes = { List pages }
    

    pages.each { page ->
    p("Page title: $page.title")
    p(page.text)

    }
    

    View Slide

62. @glaforge
    A DSL for the model
    • Templates generate contents
    • The model describes the website
    • Our model is described using a Groovy DSL
    42
    

    View Slide

63. @glaforge
    Sitemap.groovy
    43
    menu {

    group('Groovy') {

    item ‘Learn', 'learn.html'

    item 'Documentation', 'documentation.html'

    item 'Download', 'download.html'

    item 'Community', 'community.html'

    item 'Ecosystem', 'ecosystem.html'

    }

    

    group('About') {

    item 'Contributing', 'contribute.html'

    item 'Source code', 'https://github.com/groovy/groovy-core'

    item 'Build status', 'buildstatus.html'

    item 'Books', 'learn.html#books'

    item 'Sponsors', 'sponsors.html'

    item 'FAQ', 'faq.html'

    item 'Search', 'search.html'

    }
    …
    

    View Slide

64. @glaforge
    Sitemap.groovy
    44
    ecosystem {

    project('Grails') {

    description '''Grails is an Open Source, full stack,
    web application framework for the JVM.
    It takes advantage of the Groovy programming
    language and convention over configuration
    to provide a productive and stream-lined
    development experience.'''

    url 'http://grails.org/'

    logo 'img/ecosystem/grails.png'

    }

    

    project('Gradle') {

    description '''Gradle is build automation evolved.
    Gradle can automate the building, testing,
    publishing, deployment and more of software
    packages or other types of projects such as
    generated static websites, generated documentation
    or indeed anything else.'''

    url 'http://gradle.org'

    logo 'img/ecosystem/gradle.gif'

    }

    …
    

    View Slide

65. Baking…
    

    View Slide

66. @glaforge
    Baking
    • sitemap.groovy contains the site “data”
    • MarkupTemplateEngine converts it to pages
    • Gradle
    – Compiles the generator
    – Generates the website
    – Checks for dead links in generated contents
    46
    

    View Slide

67. @glaforge
    Deployment
    • Gradle generates a static website
    • Can be deployed anywhere
    • For continuous deployment, we use TeamCity
    47
    

    View Slide

68. @glaforge
    TeamCity at Groovy
    • CI Server for Groovy
    • Tests, builds and releases Groovy
    • Build plans for JDK 5, 6, 7, 8 and even 9
    • Tests every pull request before merge
    • We test early releases of the JDK (EA and sources)
    • http://ci.groovy-lang.org?guest=1
    48
    

    View Slide

69. View Slide

70. @glaforge
    TeamCity at Groovy
    • Server is sponsored by
    • Deployment of Groovy is done through
    • Releasing Groovy is:
    – Done through Artifactory plugin
    – One form + one click
    50
    

    View Slide

71. @glaforge
    Website deployment
    A commit on the master branch of website…
    • Triggers a build
    • Which generates a static website
    • Copied to the server if the build is successful
    51
    

    View Slide

72. @glaforge
    A word on GVM
    When a release is done:
    • GVM is notified
    • Download is made available in GVM
    • Done through a REST API
    • API Called directly from TeamCity
    • Thanks to... a Groovy script!
    52
    

    View Slide

73. @glaforge
    Groovy build-step in TeamCity
    • Groovy Build Step plugin
    – Allows you to write a build step as a Groovy script
    – Can be used to deploy, copy resources, web hooks ...
    53
    

    View Slide

74. Summary
    

    View Slide

75. Executable
    documentation
    

    View Slide

76. New website
    

    View Slide

77. Simplified
    deployment
    

    View Slide

78. Thanks for
    your attention!
    

    View Slide

79. Q & A
    

    View Slide

80. @glaforge
    Picture credits
    London: http://www.100percentoptical.com/images/2014/10/london.jpg
    Vomit:; http://youoffendmeyouoffendmyfamily.com/wordpress/wp-content/uploads/2011/09/vomit1.jpg
    Dinos: http://vergapipe.com/wp-content/uploads/2014/11/dinosaurs_wallpapers_free_download.jpg
    Future: http://www.hdwallsource.com/stunning-future-wallpaper-29256.html/stunning-future-wallpaper-29256
    Gears: http://gearsoffate.com/wp-content/uploads/2014/09/Gears2.jpg
    Surprise: http://www.mentalactif.com/wp-content/uploads/2013/03/SURPRISE.jpg
    Plane: http://i.imgur.com/esCemkx.jpg
    Baking: http://www.nlinnovators.nl/wp-content/blogs.dir/1/files_mf/1409922073Navy_baking_bread.jpg
    60
    

    View Slide