Getting robust VCL with falco

by ysugimoto

Slide 1

Slide 1 text

2022/03/29 Fastly User Meetup @ysugimoto Getting robust VCL with falco

Slide 2

Slide 2 text

ysugimoto ೔ຊܦࡁ৽ฉࣾ Software Engineer / SRE Yoshiaki Sugimoto

Slide 3

Slide 3 text

falco is a VCL parser and linter optimized for Fastly

Slide 4

Slide 4 text

falco is a VCL parser and linter optimized for Fastly Not

Slide 5

Slide 5 text

Fastly Linter Command = falco

Slide 6

Slide 6 text

How do we automate applying production VCLs?

Slide 7

Slide 7 text

Deployment Flows at Nikkei 1. Using Terraform Provider 2. Custom VCLs using Fastly API

Slide 8

Slide 8 text

Apply custom VCLs using Fastly API 1. Build VCLs using template engine 2. Clone new version 3. Delete existing custom VCLs 4. Upload new VCL fi les ← Validate VCL 5. Activate new version CI/CD

Slide 9

Slide 9 text

• ίϯιʔϧ͔Β֬ೝͰ͖Δ͕CI/CD PipelineͰ͸ஞҰ֬ೝ͍ͯ͠ΒΕͳ͍ • Activateʹࣦഊͨ͠৔߹ɺෆཁͳόʔδϣϯ͕࡞ΒΕͯ͠·͏ VCL Errors in Fastly Console

Slide 10

Slide 10 text

• ߦ൪߸͸Fastlyͷ಺෦ॲཧʢϚΫϩల։ͳͲʣΛऴ͑ͨ͋ͱͷߦ਺Ͱ͋Δͨ Ίɺ࣮ίʔυͱରԠ͍ͯ͠ͳ͍ • ϑΝΠϧ໊΋ग़ͳ͍ͨΊɺෳ਺ϞδϡʔϧϑΝΠϧ͕͋Δ৔߹ಛఆ͕ࠔ೉ Ambiguous Error Reporting … sub vcl_recv { #FASTLY RECV set req.backend = origin_1 return (pass); } … Missing Semicolon!

Slide 11

Slide 11 text

VCL statistics in ೔ܦిࢠ൛ 44 submodule inclusion 8,844 Lines (※ include comments) 88 subroutines 124 backends 25 Directors ※ 2022/03/29࣌఺

Slide 12

Slide 12 text

Improve CI/CD pipeline 1. Build VCLs using template engine 2. Lint built VCLs with falco ← Validate VCL 3. Clone new version 4. Delete existing custom VCLs 5. Upload new VCL fi les ← Ensured valid VCL 6. Activate new version CI/CD

Slide 13

Slide 13 text

VCL is a programming language?

Slide 14

Slide 14 text

Varnish Configuration Language

Slide 15

Slide 15 text

Configuration Design and Best Practices From a syntax perspective, the configuration language should offer the following: 1. Syntax highlighting in editors 2. Linter 3. Automatic syntax formatter https://sre.google/workbook/con fi guration-design/

Slide 16

Slide 16 text

From a syntax perspective, the configuration language should offer the following: 1. Syntax highlighting in editors 2. Linter 3. Automatic syntax formatter Configuration Design and Best Practices https://sre.google/workbook/con fi guration-design/

Slide 17

Slide 17 text

falco aims to cover the linter

Slide 18

Slide 18 text

Underlying falco features • Completely runs on local machine / CI • Basic syntax check • Strict type check • Suggest Fastly recommended ways • Assuming problems in VCL

Slide 19

Slide 19 text

• ߦ൪߸ɺϑΝΠϧ໊ͱڞʹ໰୊ͷ͋ΔՕॴΛදࣔ • VCLͷγϯλοΫεؒҧ͍Λࣄલʹݕग़ Basic syntax check … sub vcl_recv { #FASTLY RECV set req.backend = origin_1 return (pass); } … Missing Semicolon!

Slide 20

Slide 20 text

• ʢϦςϥϧʣ୅ೖܕͷҧ͍ΛΤϥʔͱͯ͠Ϩϙʔτ • ؔ࿈͢ΔFastlyͷυΩϡϝϯτ΁ͷϦϯΫΛఏࣔ Strict type check

Slide 21

Slide 21 text

• Fastlyͷਪ঑͢ΔઃఆΛఏҊ • ؔ࿈͢ΔFastlyͷυΩϡϝϯτ΁ͷϦϯΫΛఏࣔ Suggest Fastly recommended ways

Slide 22

Slide 22 text

• ໰୊ʹͳΓͦ͏ͳهड़ʹରͯ͠ܯࠂʢΤϥʔʹ͸͠ͳ͍ʣ Assuming problems in VCL

Slide 23

Slide 23 text

Linter Features

Slide 24

Slide 24 text

Linter Features • جຊతͳϧʔϧ + ӡ༻ܦݧ͔ΒಘͨϧʔϧηοτΛ࣮૷ • Varnish༝དྷʢʁʣͷUndocumentedͳSpecΛؚΊͰ͖Δ ݶΓαϙʔτ

Slide 25

Slide 25 text

Linter Rules • Root Statementsͷॏෳఆٛ • Backend/Directorͷඞਢఆٛݕࠪ • Boilerplate Macroͷهड़࿙Ε • ϩʔΧϧม਺ͷγϯλοΫεݕࠪ • ୅ೖ࣌ͷܕݕࠪ • Statementͷར༻Մೳείʔϓݕࠪ • ҙਤ͠ͳ͍Literalͷར༻ • ະ࢖༻ม਺ɾStatementͷݕग़ • ૊ΈࠐΈؔ਺ͷҾ਺ͷਖ਼౰ੑݕࠪ • ਖ਼نදݱΩϟϓνϟͷ্ॻ͖ܯࠂ • ਖ਼نදݱͷਖ਼౰ੑݕࠪ • Include͞ΕͨϞδϡʔϧͷ࠶ؼݕࠪ • IPܕ/CIDRͷਖ਼౰ੑݕࠪ • ૊ΈࠐΈม਺ͷset/unsetͷՄ൱ݕࠪ • Fastlyͷ੍ݶࣄ߲ݕࠪ • Subroutineͷείʔϓೝࣝ https://github.com/ysugimoto/falco/blob/main/docs/rules.md

Slide 26

Slide 26 text

Error Levels • ERROR • Activate࣌ʹΤϥʔʹͳΔͷͰमਖ਼͕ඞཁ • WARNING • ActivateՄೳ͕ͩ༧ظ͠ͳ͍ৼΔ෣͍͕ى͜ΔՄೳੑ͕͋Δ • INFO • ActivateՄೳɺFastlyਪ঑ͷهड़ͷఏҊͳͲ

Slide 27

Slide 27 text

Remote Snippets Support • Edge Dictionary • Access Control Lists Fastly্ʹొ࿥͍ͯ͠ΔεχϖοτΛؚΊͯLinting