初探 Infrastructure as Code 工具 Pulumi

想必大家對於 Infrastructure as Code 簡稱 (IaC) 並不陌生，而這個名詞在很早以前就很火熱，本篇最主要介紹為什麼我們要導入 IaC，以及該選擇哪些工具來管理雲平台 (AWS, GCP, Azure 等…)。觀看現在很火紅的 Terraform 及後起之秀 Pulumi 是大家可以作為選擇的參考，而底下會來歸納優缺點及技術比較，以及為什麼我最後會選擇 Pulumi。這兩套都是由 Go 語言所開發，現在選擇工具前，都要先考慮看看什麼語言寫的，以及整合進團隊自動化部署流程難易度。

Transcript

1. Infrastructure as Code (IaC)
   Bo-Yi Wu
   
   
   2021/11/16
   
   
   Pulumi vs Terraform
   

   View Slide

2. About me
   • Software Engineer in Mediatek
   
   
   • Member of Drone CI/CD Platform
   
   
   • Member of Gitea Platform
   
   
   • Member of Gin Golang Framework
   
   
   • Maintain Some GitHub Actions Plugins.
   

   View Slide

3. What is
   Infrastructure
   
   
   As Code (IaC)?
   

   View Slide

4. Infrastructure as code is about
   bringing software engineering
   principles and approaches into
   the cloud infrastructure space.
   IUUQTXXXQVMVNJDPNXIBUJTXIBUJTJOGSBTUSVDUVSFBTDPEF
   

   View Slide

5. 6QMPBE%BUBUP"844TUPSBHF
   
   
   
   

   View Slide

6. View Slide

7. "84$MPVE%BTICPBSE
   

   View Slide

8. 5FTUJOH
   4UBHJOH
   1SPEVDUJPO
   

   View Slide

9. *OUFHSBUJPO5FTUJOHPS4NPLF5FTUJOH
   

   View Slide

10. Move to
    
    
    GCP or Azure?
    

    View Slide

11. 為什麼需要 IaC
    
    
    (避免⼈為操作失敗?)
    

    View Slide

12. IaC 帶來的好處?
    • 建置 CI/CD ⾃動化 (不⽤依賴 UI 操作)
    
    
    • 版本控制 (審核避免錯誤)
    
    
    • 重複使⽤ (減少建置時間)
    
    
    • 環境⼀⾄性 (測試及正式)
    
    
    • 團隊成長 (分享學習資源)
    

    View Slide

13. IUUQTHJUIVCDPNEBUBTUBDLUWEBUBFOHJOFFSSPBENBQ
    

    View Slide

14. ⼯具選擇?
    

    View Slide

15. Terraform
    • Domain Speci
    fi
    c Language of IaC Tool
    
    
    • HCL for Terraform
    FBTZUPSFBEBOEXSJUF
    

    View Slide

16. data "aws_ami" "ubuntu_16_04_docker" {
    
    
    f
    i
    lter {
    
    
    name = "name"
    
    
    values = ["app
    -
    base
    -
    docker
    -
    image
    -
    *"]
    
    
    }
    
    
    f
    i
    lter {
    
    
    name = "virtualization
    -
    type"
    
    
    values = ["hvm"]
    
    
    }
    
    
    owners = ["161654634948"] # Canonical
    
    
    }
    
    
    resource "aws_instance" "foobar_api_01" {
    
    
    ami = data.aws_ami.ubuntu_16_04_docker.id
    
    
    instance_type = "t2.small"
    
    
    vpc_security_group_ids = [aws_security_group.foobar_api.id]
    
    
    key_name = aws_key_pair.foobar_deploy.key_name
    
    
    subnet_id = aws_subnet.foobar_a.id
    
    
    iam_instance_prof
    i
    le = aws_iam_instance_prof
    i
    le.ec2_role.id
    
    
    tags = {
    
    
    Name = "foobar api 01"
    
    
    Project = "foobar"
    
    
    Environment = var.environment["production"]
    
    
    }
    
    
    }
    HCL 語法
    

    View Slide

17. What if you need more logic?
    • Conditionals
    
    
    • If / else & logical operators
    
    
    • Loops
    
    
    • Functions
    
    
    • re-use logic
    

    View Slide

18. _, err
    : =
    s3.NewBucket(ctx, "wms
    -
    bucket", &s3.BucketArgs{
    
    
    Acl: pulumi.String("private"),
    
    
    Bucket: pulumi.String("objects.wms"),
    
    
    ForceDestroy: pulumi.Bool(false),
    
    
    LifecycleRules: s3.BucketLifecycleRuleArray{
    
    
    &s3.BucketLifecycleRuleArgs{
    
    
    Enabled: pulumi.Bool(true),
    
    
    Expiration: &s3.BucketLifecycleRuleExpirationArgs{
    
    
    Days: pulumi.Int(7),
    
    
    },
    
    
    Pref
    i
    x: pulumi.String("backup"),
    
    
    Tags: pulumi.StringMap{
    
    
    "autoclean": pulumi.String("true"),
    
    
    "name": pulumi.String("database"),
    
    
    },
    
    
    },
    
    
    },
    
    
    }, pulumi.Protect(true))
    AWS S3 using Golang
    

    View Slide

19. Con
    fi
    guration by ENV
    
    
    (AWS Global and China)
    

    View Slide

20. View Slide

21. Amazon FSx for Lustre
    
    
    (Don’t support in China)
    

    View Slide

22. Database struct {
    
    
    Driver string `envconf
    i
    g:"APP_DATABASE_DRIVER"`
    
    
    Username string `envconf
    i
    g:"APP_DATABASE_USERNAME"`
    
    
    Password string `envconf
    i
    g:"APP_DATABASE_PASSWORD"`
    
    
    Name string `envconf
    i
    g:"APP_DATABASE_NAME"`
    
    
    }
    Conf
    i
    g struct {
    
    
    Logging Logging
    
    
    Server Server
    
    
    Database Database
    
    
    Redis Redis
    
    
    Storage Storage
    
    
    Queue Queue
    
    
    }
    

    View Slide

23. Pulumi Support Language
    • Golang
    
    
    • C#
    
    
    • Python
    
    
    • JavaScript
    
    
    • TypeScript
    
    
    • F#
    
    
    • VB
    

    View Slide

24. View Slide

25. Terraform
    
    
    就不能⽤程式語⾔來寫?
    

    View Slide

26. View Slide

27. View Slide

28. https:
    / /
    github.com/hashicorp/terraform
    -
    cdk
    

    View Slide

29. Pulumi
    
    
    Cloud Provider
    

    View Slide

30. View Slide

31. Converting From Terraform
    
    
    Using tf2pulumi Tool
    IUUQTXXXQVMVNJDPNEPDTHVJEFTBEPQUJOHGSPN@UFSSBGPSN
    

    View Slide

32. View Slide

33. Using Pulumi and Terraform
    
    
    Side-by-Side from
    
    
    Terraform state
    import * as pulumi from "@pulumi/pulumi"
    ;
    
    import * as terraform from "@pulumi/terraform"
    ;
    
    const con
    fi
    g = new pulumi.Con
    fi
    g()
    ;
    
    const tfeToken = con
    fi
    g.requireSecret("tfeToken")
    ;
    
    const networkState = new terraform.state.RemoteStateReference("network",
    {
    
    backendType: "remote"
    ,
    
    token: tfeToken
    ,
    
    organization: "acme"
    ,
    
    workspaces:
    {
    
    name: "production-network"
    }
    ,
    
    })
    ;
    
    

    View Slide

34. ⾃⾏開發整合⼯具?
    
    
    (Automation API)
    

    View Slide

35. https:
    / / w w w
    .pulumi.com/blog/automation
    -
    api/
    

    View Slide

36. YA! 不⽤使⽤ CLI Tool
    
    
    (pulumi command)
    

    View Slide

37. Automation API allows you to embed Pulumi within your application code
    

    View Slide

38. 1. Database Migration
    

    View Slide

39. _, err = rds.NewClusterInstance(ctx, "dbInstance", &rds.ClusterInstanceArgs{
    
    
    ClusterIdentif
    i
    er: cluster.ClusterIdentif
    i
    er,
    
    
    InstanceClass: rds.InstanceType_T3_Small,
    
    
    Engine: rds.EngineTypeAuroraMysql,
    
    
    EngineVersion: pulumi.String("5.7.mysql_aurora.2.03.2"),
    
    
    PubliclyAccessible: pulumi.Bool(true),
    
    
    DbSubnetGroupName: subnetGroup.Name,
    
    
    })
    
    
    if err
    ! =
    nil {
    
    
    return err
    
    
    }
    
    
    ctx.Export("host", cluster.Endpoint)
    
    
    ctx.Export("dbName", dbName)
    
    
    ctx.Export("dbUser", dbUser)
    
    
    ctx.Export("dbPass", dbPass)
    
    
    

    View Slide

40. / /
    create our stack with an "inline" Pulumi program (deployFunc)
    
    
    stack
    : =
    auto.UpsertStackInlineSource(ctx, stackName, projectName, deployFunc)
    
    
    / /
    run the update to deploy our database
    
    
    res, err
    : =
    stack.Up(ctx, stdoutStreamer)
    
    
    fmt.Println("Update succeeded!")
    
    
    / /
    get the connection info
    
    
    host
    : =
    res.Outputs["host"].Value
    
    
    dbName
    : =
    res.Outputs["dbName"].Value
    
    
    dbUser
    : =
    res.Outputs["dbUser"].Value
    
    
    dbPass
    : =
    res.Outputs["dbPass"].Value
    
    
    / /
    establish db connection
    
    
    db
    : =
    sql.Open("mysql", fmt.Sprintf("%s:%s@tcp(%s:3306)/%s", dbUser, dbPass, host, dbName))
    defer db.Close()
    
    
    / /
    run our database "migration"
    
    
    fmt.Println("creating table
    . . .
    ")
    
    
    db.Query(`
    
    
    CREATE TABLE IF NOT EXISTS hello_pulumi(
    
    
    id int(9) NOT NULL,
    
    
    color varchar(14) NOT NULL,
    
    
    PRIMARY KEY(id)
    
    
    );
    
    
    `)
    
    
    

    View Slide

41. 2. RESTful API Service
    

    View Slide

42. ctx
    : =
    context.Background()
    
    
    stackName
    : =
    createReq.ID
    
    
    program
    : =
    createPulumiProgram(createReq.Content)
    
    
    s, err
    : =
    auto.NewStackInlineSource(ctx, stackName, project, program)
    
    
    if err
    ! =
    nil {
    
    
    / /
    if stack already exists, 409
    
    
    if auto.IsCreateStack409Error(err) {
    
    
    w.WriteHeader(409)
    
    
    fmt.Fprintf(w, fmt.Sprintf("stack %q already exists", stackName))
    
    
    return
    
    
    }
    
    
    w.WriteHeader(500)
    
    
    fmt.Fprintf(w, err.Error())
    
    
    return
    
    
    }
    
    
    s.SetConf
    i
    g(ctx, "aws:region", auto.Conf
    i
    gValue{Value: "us
    -
    west-2"})
    
    
    / /
    deploy the stack
    
    
    / /
    we'll write all of the update logs to st out so we can watch requests get processed
    
    
    upRes, err
    : =
    s.Up(ctx, optup.ProgressStreams(os.Stdout))
    
    
    if err
    ! =
    nil {
    
    
    w.WriteHeader(500)
    
    
    fmt.Fprintf(w, err.Error())
    
    
    return
    
    
    }
    
    
    

    View Slide

43. State Management
    

    View Slide

44. %FGBVMUJO$MPVE %FGBVMUJO-PDBM
    IUUQTFOXJLJQFEJBPSHXJLJ5FBN@DPO
    fl
    JDU
    

    View Slide

45. Why I Choose
    
    
    Pulumi?
    

    View Slide

46. 開發者
    • 可以使⽤熟悉的語⾔, 不⽤學習新語法
    
    
    • 模組及跨平台 (AWS, GCP, Azure)
    
    
    • 減少複製流程, ⾃由建立模組清單
    

    View Slide

47. DevOps 團隊
    • Infrastructure as Code (可審核檢視)
    
    
    • Multi-Cloud DevOps (⽤⼀種⽅式管理多個雲平台)
    
    
    • Deploy Continuously (⾃動化部署)
    

    View Slide

48. 資安團隊
    • Policy as Code
    
    
    • Built-In Secrets (加密特定資料: DB 密碼 ..)
    
    
    • Enforce Standards (同⼀套流程管理系統架構)
    

    View Slide

49. Pulumi
    Architecture
    

    View Slide

50. Pulumi Overview
    IUUQTXXXQVMVNJDPNEPDTJOUSPDPODFQUT
    

    View Slide

51. Pulumi Overview
    3VOUIF1VMVNJ$-*DPNNBOE QVMVNJVQ
    
    

    View Slide

52. name: footer
    
    
    runtime: go
    
    
    description: A minimal AWS Go Pulumi program
    
    
    template:
    
    
    conf
    i
    g:
    
    
    aws:region:
    
    
    description: AWS region
    
    
    default: ap
    -
    southeast-1
    conf
    i
    g:
    
    
    aws:prof
    i
    le: footer
    
    
    aws:region: ap
    -
    southeast-1
    
    
    project:environment: production
    
    
    project:name: footer
    
    
    pulumi:template: aws
    -
    go
    
    
    1SPKFDU :".-
    4UBDL :".-
    
    

    View Slide

53. package main
    
    
    import (
    
    
    "github.com/pulumi/pulumi
    -
    aws/sdk/v4/go/aws/s3"
    
    
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
    
    
    )
    
    
    func main() {
    
    
    pulumi.Run(func(ctx
    *
    pulumi.Context) error {
    
    
    / /
    Create an AWS resource (S3 Bucket)
    
    
    bucket, err
    : =
    s3.NewBucket(ctx, "my
    -
    bucket", nil)
    
    
    if err
    ! =
    nil {
    
    
    return err
    
    
    }
    
    
    / /
    Export the name of the bucket
    
    
    ctx.Export("bucketName", bucket.ID())
    
    
    return nil
    
    
    })
    
    
    }
    NBJOQSPHSBN
    

    View Slide

54. Pulumi CLI Console
    

    View Slide

55. Pulumi Cloud Console
    

    View Slide

56. Pulumi Cloud Console
    

    View Slide

57. Pulumi Cloud Console
    

    View Slide

58. Pulumi Cloud Console
    

    View Slide

59. 快速建置簡單網⾴
    
    
    使⽤ AWS S3 + Static
    Website
    

    View Slide

60. pulumi.Run(func(ctx
    *
    pulumi.Context) error {
    
    
    / /
    Create an AWS resource (S3 Bucket)
    
    
    bucket, err
    : =
    s3.NewBucket(ctx, "my
    -
    bucket", &s3.BucketArgs{
    
    
    Website: s3.BucketWebsiteArgs{
    
    
    IndexDocument: pulumi.String("index.html"),
    
    
    },
    
    
    })
    
    
    if err
    ! =
    nil {
    
    
    return err
    
    
    }
    
    
    site
    : =
    getEnv(ctx, "s3:siteDir", "content")
    
    
    f
    i
    les, err
    : =
    ioutil.ReadDir(site)
    
    
    if err
    ! =
    nil {
    
    
    return err
    
    
    }
    
    
    for _, item
    : =
    range f
    i
    les {
    
    
    name
    : =
    item.Name()
    
    
    if _, err = s3.NewBucketObject(ctx, name, &s3.BucketObjectArgs{
    
    
    Bucket: bucket.Bucket,
    
    
    Source: pulumi.NewFileAsset(f
    i
    lepath.Join(site, name)),
    
    
    Acl: pulumi.String("public
    -
    read"),
    
    
    ContentType: pulumi.String(mime.TypeByExtension(path.Ext(f
    i
    lepath.Join(site, name)))),
    
    
    }); err
    ! =
    nil {
    
    
    return err
    
    
    }
    
    
    }
    
    
    / /
    Export the name of the bucket
    
    
    ctx.Export("bucketID", bucket.ID())
    
    
    ctx.Export("bucketName", bucket.Bucket)
    
    
    ctx.Export("bucketEndpoint", bucket.WebsiteEndpoint)
    
    
    return nil
    
    
    })
    

    View Slide

61. Pulumi Pricing
    

    View Slide

62. IUUQTXXXQVMVNJDPNQSJDJOH
    

    View Slide

63. 免費⽅案
    IUUQTXXXQVMVNJDPNQSJDJOHGBRQSPEVDU
    

    View Slide

64. Convince Your Boss
    IUUQTXXXQVMVNJDPNQSJDJOHGBRQSPEVDU
    

    View Slide

65. Thank You
    

    View Slide