HCL: A human-friendly language for developers and operators

Description
--------------
In 2018, HashiCorp Configuration Language (HCL) was second on GitHub’s list of fastest-growing languages.

Anubhav Mishra explores the history behind the creation of HCL and explains what has made it a popular language of choice, used by tools like HashiCorp Terraform and GitHub Actions. Along the way, he details the language’s syntax and engine behind HCL and showcases real-world examples using HCL to express production infrastructure, and outlines the benefits of doing so.

Live demos include:

* Using HCL as the language for operators practicing infrastructure as code using Terraform
* Using HCL as the language for developers using GitHub Actions to create an end-to-end pipeline for their organization

What you'll learn
--------------------
Understand HCL syntax and real-world use cases around the language
Learn the history behind the creation of the language

Transcript

1. HCL (HashiCorp Configuration Language)
   A Human Friendly Language for Developers and Operators
   Anubhav Mishra
   Team Lead, Developer Advocacy, OOCTO at HashiCorp
   

   View Slide

2. $ whoami
   @build1point0
   Anubhav Mishra
   Team Lead, Developer Advocacy, OOCTO
   Atlan&s
   

   View Slide

3. Gopher Artwork by Ashley McNamara
   

   View Slide

4. View Slide

5. HCL (HashiCorp Configuration Language)
   @build1point0
   
   ▪ Created in July 31st, 2014 by HashiCorp
   ▪ Used by HashiCorp Projects: Terraform, Consul, Nomad, Vault, 

   consul-template, envconsul
   ▪ Used by Github Actions
   ~45 million downloads/year
   

   View Slide

6. @build1point0
   
   

   View Slide

7. ⁄
   History
   

   View Slide

8. @build1point0
   
   HashiCorp in 2014
   Ruby JSON JSON ?
   

   View Slide

9. @build1point0
   
   HashiCorp in 2014
   

   View Slide

10. @build1point0
    
    UCL (Universal Configuration Language)
    UCL is heavily infused by NGINX configuration as the example of a convenient configuration system.
    However, UCL is fully compatible with JSON format and is able to parse JSON files.
    CODE EDITOR
    param = value;
    section {
    param = value;
    param1 = value1;
    flag = true;
    number = 10k;
    time = 0.2s;
    string = "something";
    subsection {
    host = {
    host = "hostname";
    port = 900;
    }
    …
    

    View Slide

11. @build1point0
    
    UCL (Universal Configuration Language)
    UCL is heavily infused by NGINX configuration as the example of a convenient configuration system.
    However, UCL is fully compatible with JSON format and is able to parse JSON files.
    CODE EDITOR
    {
    "param": "value",
    "param1": "value1",
    "flag": true,
    "subsection": {
    "host": [
    {
    "host": "hostname",
    "port": 900
    },
    {
    "host": "hostname",
    "port": 901
    }
    

    View Slide

12. @build1point0
    
    HCL is created in 2014
    

    View Slide

13. ⁄
    HCL v1
    

    View Slide

14. HCL (HashiCorp Configuration Language)
    @build1point0
    
    ▪ Configuration language
    ▪ Structured language built for both humans and machines
    ▪ Built for command line tools
    ▪ Targeting DevOps tools, servers, etc. (Terraform)
    

    View Slide

15. HCL
    Version 1
    CODE EDITOR
    resource "aws_instance" "example" {
    ami = "ami-2757f631"
    instance_type = "t2.micro"
    }
    @build1point0
    
    

    View Slide

16. HCL
    Version 1
    CODE EDITOR
    job "example" {
    datacenters = ["dc1"]
    type = "service"
    group "cache" {
    count = 1
    task "redis" {
    driver = "docker"
    config {
    image = "redis:3.2"
    port_map {
    db = 6379
    }
    }
    }
    }
    @build1point0
    
    

    View Slide

17. HCL
    Version 1
    CODE EDITOR
    resource "aws_instance" "example" {
    ami = “${var.ami}”
    instance_type = "t2.micro"
    }
    resource "aws_instance" “example2” {
    ami = “${var.ami}”
    instance_type = "t2.micro"
    }
    @build1point0
    
    

    View Slide

18. HIL (HashiCorp Interpolation Language)
    @build1point0
    
    ▪ HIL (HashiCorp Interpolation Language) is a lightweight embedded
    language used primarily for configuration interpolation.
    ▪ HIL is built to interpolate any string, but is in use by HashiCorp primarily
    with HCL.
    ▪ HIL enables the configuration to be able to reference values from
    elsewhere. Example: foo = "hi ${var.world}"
    

    View Slide

19. HCL
    Version 1
    CODE EDITOR
    resource "aws_instance" "example" {
    ami = "ami-2757f631"
    instance_type = "t2.micro"
    }
    data "aws_route53_zone" "default" {
    name = "hashicorp.live."
    }
    resource "aws_route53_record" "web_server" {
    zone_id = "${data.aws_route53_zone.default.zone_id}"
    name = "example.hashicorp.live"
    type = "A"
    ttl = "5"
    records = ["${aws_instance.example.public_ip}"]
    }
    @build1point0
    
    

    View Slide

20. @build1point0
    
    HCL JSON
    % backwards compatible
    
    

    View Slide

21. HCL <>
    JSON
    CODE EDITOR
    variable "ami" {
    description = "the AMI to use"
    }
    {
    "variable": {
    "ami": {
    "description": "the AMI to use"
    }
    }
    }
    @build1point0
    
    

    View Slide

22. @build1point0
    
    Why no JSON?
    

    View Slide

23. Why HCL?
    @build1point0
    
    ▪ Prior to HCL, HashiCorp tools did used languages like Ruby and data
    structure languages such as JSON
    ▪ Some people wanted human-friendly language and other wanted
    machine-friendly languages
    ▪ JSON fits the bill here, but is fairly verbose and most importantly
    doesn't support comments.
    ▪ HashiCorp created HCL and made is JSON-compatible.
    ▪ HCL for HashiCorp tools and JSON is the interoperability layer.
    

    View Slide

24. JSON as the Interoperability Layer
    @build1point0
    
    ▪ Languages like Python, Ruby, etc can manipulate JSON easily
    ▪ Machines can generate JSON easily
    

    View Slide

25. @build1point0
    
    I love my YAMLS!
    

    View Slide

26. Why HCL?
    @build1point0
    
    ▪ YAML was hard for beginners
    ▪ Hard to determine the actual structures, lots of guess work involved.
    

    View Slide

27. Our Findings
    @build1point0
    
    ▪ The success of a new language depends on the tooling around it.
    – With HCL, the adoption was OK, thanks to Terraform.
    ▪ There is no perfect language
    – Programmers want both human friendly and machine readable code.
    ▪ HCL is a common configuration language.
    

    View Slide

28. Demo
    HCL v1
    

    View Slide

29. This wasn’t enough
    @build1point0
    
    

    View Slide

30. HCL v1 Challenges
    @build1point0
    
    ▪ Unhelpful error messages.
    ▪ Use of string interpolation for non-string results (HCL + HIL)
    ▪ Lack of rich type system. Eg: Use of strings vs maps, lists, etc.
    ▪ The ability to define complex logic. Eg: web applications, networks, etc.
    – Need for iteration constructs like for loops.
    – Need for templates.
    

    View Slide

31. ⁄
    HCL2
    

    View Slide

32. HCL2
    Error Messages
    TERMINAL
    # HCL v1
    __builtin_StringToInt: strconv.ParseInt: parsing "foo": invalid syntax in:
    ${1 + var.example}
    # HCL2
    Error: Unsupported block type
    on example.tf line 4, in resource "aws_instance" "example":
    2: provisionr "local-exec" {
    Blocks of type "provisionr" are not expected here. Did you mean "provisioner"?
    @build1point0
    
    

    View Slide

33. HCL2
    Expressions Outside of
    Interpolation (HCL + HIL)
    CODE EDITOR
    # HCL v1
    variable "ami" {
    }
    variable "instance_type" {
    }
    resource "aws_instance" "example" {
    ami = "${var.ami}"
    instance_type = "${var.instance_type}"
    .....
    }
    # HCL2
    variable "ami" {
    }
    variable "instance_type" {
    }
    resource "aws_instance" "example" {
    ami = var.ami
    instance_type = var.instance_type
    .....
    }
    @build1point0
    
    

    View Slide

34. HCL2
    Comprehensive List and
    Map Support
    CODE EDITOR
    # HCL v1
    resource "aws_instance" "example" {
    .....
    # Instead, it's necessary to use the list function to "trick" in HIL
    vpc_security_group_ids = "${var.security_group_id != "" ?
    list(var.security_group_id) : list()}"
    }
    # HCL2
    resource "aws_instance" "example" {
    .....
    vpc_security_group_ids = var.security_group_id != "" ?
    [var.security_group_id] : []
    }
    @build1point0
    
    

    View Slide

35. HCL2
    Ability to Define Complex
    Logic
    CODE EDITOR
    # HCL2
    resource "aws_instance" "example" {
    …..
    tags = {for def in var.standard_tags: def.name =>
    def.value}
    }
    output "instance_public_ips" {
    value = {for inst in aws_instance.example:
    inst.tags.name => inst.public_ip...}
    }
    @build1point0
    
    

    View Slide

36. HCL2
    Templates
    CODE EDITOR
    # HCL2
    output "server_config" {
    value = <%{ for ip_addr in server_ips }server ${ ip_addr };%{ endfor }
    EOT
    }
    # Output
    server 10.0.0.1;
    server 10.0.0.2;
    …..
    @build1point0
    
    

    View Slide

37. Demo
    HCL2
    

    View Slide

38. @build1point0
    
    Github Actions
    

    View Slide

39. Demo
    Github Actions
    

    View Slide

40. Future Work
    @build1point0
    
    ▪ Continue to improve HCL (HCL2)
    ▪ Publish HCL specification on a website
    ▪ VS Code language server and LSP (Language Server Protocol)
    

    View Slide

41. Links
    @build1point0
    
    ▪ HCL: https://github.com/hashicorp/hcl
    ▪ HIL: https://github.com/hashicorp/hil
    ▪ HCL2: https://github.com/hashicorp/hcl2
    ▪ gohcl: https://godoc.org/github.com/hashicorp/hcl2/gohcl

    

    View Slide

42. Thank You
    Questions?
    @build1point0
    www.hashicorp.com
    

    View Slide