HCL: A human-friendly language for developers and operators

Description
--------------
In 2018, HashiCorp Configuration Language (HCL) was second on GitHub’s list of fastest-growing languages.

Anubhav Mishra explores the history behind the creation of HCL and explains what has made it a popular language of choice, used by tools like HashiCorp Terraform and GitHub Actions. Along the way, he details the language’s syntax and engine behind HCL and showcases real-world examples using HCL to express production infrastructure, and outlines the benefits of doing so.

Live demos include:

* Using HCL as the language for operators practicing infrastructure as code using Terraform
* Using HCL as the language for developers using GitHub Actions to create an end-to-end pipeline for their organization

What you'll learn
--------------------
Understand HCL syntax and real-world use cases around the language
Learn the history behind the creation of the language

Transcript

1. HCL (HashiCorp Configuration Language) A Human Friendly Language for Developers and Operators Anubhav Mishra Team Lead, Developer Advocacy, OOCTO at HashiCorp
2. $ whoami @build1point0 Anubhav Mishra Team Lead, Developer Advocacy, OOCTO Atlan&s
3. Gopher Artwork by Ashley McNamara
5. HCL (HashiCorp Configuration Language) @build1point0  ▪ Created in July 31st, 2014 by HashiCorp ▪ Used by HashiCorp Projects: Terraform, Consul, Nomad, Vault, 
 consul-template, envconsul ▪ Used by Github Actions ~45 million downloads/year
6. @build1point0 
7. ⁄ History
8. @build1point0  HashiCorp in 2014 Ruby JSON JSON ?
9. @build1point0  HashiCorp in 2014
10. @build1point0  UCL (Universal Configuration Language) UCL is heavily infused by NGINX configuration as the example of a convenient configuration system. However, UCL is fully compatible with JSON format and is able to parse JSON files. CODE EDITOR param = value; section { param = value; param1 = value1; flag = true; number = 10k; time = 0.2s; string = "something"; subsection { host = { host = "hostname"; port = 900; } …
11. @build1point0  UCL (Universal Configuration Language) UCL is heavily infused by NGINX configuration as the example of a convenient configuration system. However, UCL is fully compatible with JSON format and is able to parse JSON files. CODE EDITOR { "param": "value", "param1": "value1", "flag": true, "subsection": { "host": [ { "host": "hostname", "port": 900 }, { "host": "hostname", "port": 901 }
12. @build1point0  HCL is created in 2014
13. ⁄ HCL v1
14. HCL (HashiCorp Configuration Language) @build1point0  ▪ Configuration language ▪ Structured language built for both humans and machines ▪ Built for command line tools ▪ Targeting DevOps tools, servers, etc. (Terraform)
15. HCL Version 1 CODE EDITOR resource "aws_instance" "example" { ami = "ami-2757f631" instance_type = "t2.micro" } @build1point0 
16. HCL Version 1 CODE EDITOR job "example" { datacenters = ["dc1"] type = "service" group "cache" { count = 1 task "redis" { driver = "docker" config { image = "redis:3.2" port_map { db = 6379 } } } } @build1point0 
17. HCL Version 1 CODE EDITOR resource "aws_instance" "example" { ami = “${var.ami}” instance_type = "t2.micro" } resource "aws_instance" “example2” { ami = “${var.ami}” instance_type = "t2.micro" } @build1point0 
18. HIL (HashiCorp Interpolation Language) @build1point0  ▪ HIL (HashiCorp Interpolation Language) is a lightweight embedded language used primarily for configuration interpolation. ▪ HIL is built to interpolate any string, but is in use by HashiCorp primarily with HCL. ▪ HIL enables the configuration to be able to reference values from elsewhere. Example: foo = "hi ${var.world}"
19. HCL Version 1 CODE EDITOR resource "aws_instance" "example" { ami = "ami-2757f631" instance_type = "t2.micro" } data "aws_route53_zone" "default" { name = "hashicorp.live." } resource "aws_route53_record" "web_server" { zone_id = "${data.aws_route53_zone.default.zone_id}" name = "example.hashicorp.live" type = "A" ttl = "5" records = ["${aws_instance.example.public_ip}"] } @build1point0 
20. @build1point0  HCL JSON % backwards compatible
21. HCL <> JSON CODE EDITOR variable "ami" { description = "the AMI to use" } { "variable": { "ami": { "description": "the AMI to use" } } } @build1point0 
22. @build1point0  Why no JSON?
23. Why HCL? @build1point0  ▪ Prior to HCL, HashiCorp tools did used languages like Ruby and data structure languages such as JSON ▪ Some people wanted human-friendly language and other wanted machine-friendly languages ▪ JSON fits the bill here, but is fairly verbose and most importantly doesn't support comments. ▪ HashiCorp created HCL and made is JSON-compatible. ▪ HCL for HashiCorp tools and JSON is the interoperability layer.
24. JSON as the Interoperability Layer @build1point0  ▪ Languages like Python, Ruby, etc can manipulate JSON easily ▪ Machines can generate JSON easily
25. @build1point0  I love my YAMLS!
26. Why HCL? @build1point0  ▪ YAML was hard for beginners ▪ Hard to determine the actual structures, lots of guess work involved.
27. Our Findings @build1point0  ▪ The success of a new language depends on the tooling around it. – With HCL, the adoption was OK, thanks to Terraform. ▪ There is no perfect language – Programmers want both human friendly and machine readable code. ▪ HCL is a common configuration language.
28. Demo HCL v1
29. This wasn’t enough @build1point0 
30. HCL v1 Challenges @build1point0  ▪ Unhelpful error messages. ▪ Use of string interpolation for non-string results (HCL + HIL) ▪ Lack of rich type system. Eg: Use of strings vs maps, lists, etc. ▪ The ability to define complex logic. Eg: web applications, networks, etc. – Need for iteration constructs like for loops. – Need for templates.
31. ⁄ HCL2
32. HCL2 Error Messages TERMINAL # HCL v1 __builtin_StringToInt: strconv.ParseInt: parsing "foo": invalid syntax in: ${1 + var.example} # HCL2 Error: Unsupported block type on example.tf line 4, in resource "aws_instance" "example": 2: provisionr "local-exec" { Blocks of type "provisionr" are not expected here. Did you mean "provisioner"? @build1point0 
33. HCL2 Expressions Outside of Interpolation (HCL + HIL) CODE EDITOR # HCL v1 variable "ami" { } variable "instance_type" { } resource "aws_instance" "example" { ami = "${var.ami}" instance_type = "${var.instance_type}" ..... } # HCL2 variable "ami" { } variable "instance_type" { } resource "aws_instance" "example" { ami = var.ami instance_type = var.instance_type ..... } @build1point0 
34. HCL2 Comprehensive List and Map Support CODE EDITOR # HCL v1 resource "aws_instance" "example" { ..... # Instead, it's necessary to use the list function to "trick" in HIL vpc_security_group_ids = "${var.security_group_id != "" ? list(var.security_group_id) : list()}" } # HCL2 resource "aws_instance" "example" { ..... vpc_security_group_ids = var.security_group_id != "" ? [var.security_group_id] : [] } @build1point0 
35. HCL2 Ability to Define Complex Logic CODE EDITOR # HCL2 resource "aws_instance" "example" { ….. tags = {for def in var.standard_tags: def.name => def.value} } output "instance_public_ips" { value = {for inst in aws_instance.example: inst.tags.name => inst.public_ip...} } @build1point0 
36. HCL2 Templates CODE EDITOR # HCL2 output "server_config" { value = <<EOT %{ for ip_addr in server_ips }server ${ ip_addr };%{ endfor } EOT } # Output server 10.0.0.1; server 10.0.0.2; ….. @build1point0 
37. Demo HCL2
38. @build1point0  Github Actions
39. Demo Github Actions
40. Future Work @build1point0  ▪ Continue to improve HCL (HCL2) ▪ Publish HCL specification on a website ▪ VS Code language server and LSP (Language Server Protocol)
41. Links @build1point0  ▪ HCL: https://github.com/hashicorp/hcl ▪ HIL: https://github.com/hashicorp/hil ▪ HCL2: https://github.com/hashicorp/hcl2 ▪ gohcl: https://godoc.org/github.com/hashicorp/hcl2/gohcl

42. Thank You Questions? @build1point0 www.hashicorp.com