Upgrade to Pro — share decks privately, control downloads, hide ads and more …

初探 Infrastructure as Code 工具 Pulumi

Bo-Yi Wu
November 16, 2021

初探 Infrastructure as Code 工具 Pulumi

想必大家對於 Infrastructure as Code 簡稱 (IaC) 並不陌生,而這個名詞在很早以前就很火熱,本篇最主要介紹為什麼我們要導入 IaC,以及該選擇哪些工具來管理雲平台 (AWS, GCP, Azure 等…)。觀看現在很火紅的 Terraform 及後起之秀 Pulumi 是大家可以作為選擇的參考,而底下會來歸納優缺點及技術比較,以及為什麼我最後會選擇 Pulumi。這兩套都是由 Go 語言所開發,現在選擇工具前,都要先考慮看看什麼語言寫的,以及整合進團隊自動化部署流程難易度。

Bo-Yi Wu

November 16, 2021
Tweet

More Decks by Bo-Yi Wu

Other Decks in Programming

Transcript

  1. Infrastructure as Code (IaC)
    Bo-Yi Wu


    2021/11/16


    Pulumi vs Terraform

    View full-size slide

  2. About me
    • Software Engineer in Mediatek


    • Member of Drone CI/CD Platform


    • Member of Gitea Platform


    • Member of Gin Golang Framework


    • Maintain Some GitHub Actions Plugins.

    View full-size slide

  3. What is
    Infrastructure


    As Code (IaC)?

    View full-size slide

  4. Infrastructure as code is about
    bringing software engineering
    principles and approaches into
    the cloud infrastructure space.
    IUUQTXXXQVMVNJDPNXIBUJTXIBUJTJOGSBTUSVDUVSFBTDPEF

    View full-size slide

  5. 6QMPBE%BUBUP"844TUPSBHF



    View full-size slide

  6. "84$MPVE%BTICPBSE

    View full-size slide

  7. 5FTUJOH
    4UBHJOH
    1SPEVDUJPO

    View full-size slide

  8. *OUFHSBUJPO5FTUJOHPS4NPLF5FTUJOH

    View full-size slide

  9. Move to


    GCP or Azure?

    View full-size slide

  10. 為什麼需要 IaC


    (避免⼈為操作失敗?)

    View full-size slide

  11. IaC 帶來的好處?
    • 建置 CI/CD ⾃動化 (不⽤依賴 UI 操作)


    • 版本控制 (審核避免錯誤)


    • 重複使⽤ (減少建置時間)


    • 環境⼀⾄性 (測試及正式)


    • 團隊成長 (分享學習資源)

    View full-size slide

  12. IUUQTHJUIVCDPNEBUBTUBDLUWEBUBFOHJOFFSSPBENBQ

    View full-size slide

  13. ⼯具選擇?

    View full-size slide

  14. Terraform
    • Domain Speci
    fi
    c Language of IaC Tool


    • HCL for Terraform
    FBTZUPSFBEBOEXSJUF

    View full-size slide

  15. data "aws_ami" "ubuntu_16_04_docker" {


    f
    i
    lter {


    name = "name"


    values = ["app
    -
    base
    -
    docker
    -
    image
    -
    *"]


    }


    f
    i
    lter {


    name = "virtualization
    -
    type"


    values = ["hvm"]


    }


    owners = ["161654634948"] # Canonical


    }


    resource "aws_instance" "foobar_api_01" {


    ami = data.aws_ami.ubuntu_16_04_docker.id


    instance_type = "t2.small"


    vpc_security_group_ids = [aws_security_group.foobar_api.id]


    key_name = aws_key_pair.foobar_deploy.key_name


    subnet_id = aws_subnet.foobar_a.id


    iam_instance_prof
    i
    le = aws_iam_instance_prof
    i
    le.ec2_role.id


    tags = {


    Name = "foobar api 01"


    Project = "foobar"


    Environment = var.environment["production"]


    }


    }
    HCL 語法

    View full-size slide

  16. What if you need more logic?
    • Conditionals


    • If / else & logical operators


    • Loops


    • Functions


    • re-use logic

    View full-size slide

  17. _, err
    : =
    s3.NewBucket(ctx, "wms
    -
    bucket", &s3.BucketArgs{


    Acl: pulumi.String("private"),


    Bucket: pulumi.String("objects.wms"),


    ForceDestroy: pulumi.Bool(false),


    LifecycleRules: s3.BucketLifecycleRuleArray{


    &s3.BucketLifecycleRuleArgs{


    Enabled: pulumi.Bool(true),


    Expiration: &s3.BucketLifecycleRuleExpirationArgs{


    Days: pulumi.Int(7),


    },


    Pref
    i
    x: pulumi.String("backup"),


    Tags: pulumi.StringMap{


    "autoclean": pulumi.String("true"),


    "name": pulumi.String("database"),


    },


    },


    },


    }, pulumi.Protect(true))
    AWS S3 using Golang

    View full-size slide

  18. Con
    fi
    guration by ENV


    (AWS Global and China)

    View full-size slide

  19. Amazon FSx for Lustre


    (Don’t support in China)

    View full-size slide

  20. Database struct {


    Driver string `envconf
    i
    g:"APP_DATABASE_DRIVER"`


    Username string `envconf
    i
    g:"APP_DATABASE_USERNAME"`


    Password string `envconf
    i
    g:"APP_DATABASE_PASSWORD"`


    Name string `envconf
    i
    g:"APP_DATABASE_NAME"`


    }
    Conf
    i
    g struct {


    Logging Logging


    Server Server


    Database Database


    Redis Redis


    Storage Storage


    Queue Queue


    }

    View full-size slide

  21. Pulumi Support Language
    • Golang


    • C#


    • Python


    • JavaScript


    • TypeScript


    • F#


    • VB

    View full-size slide

  22. Terraform


    就不能⽤程式語⾔來寫?

    View full-size slide

  23. https:
    / /
    github.com/hashicorp/terraform
    -
    cdk

    View full-size slide

  24. Pulumi


    Cloud Provider

    View full-size slide

  25. Converting From Terraform


    Using tf2pulumi Tool
    IUUQTXXXQVMVNJDPNEPDTHVJEFTBEPQUJOHGSPN@UFSSBGPSN

    View full-size slide

  26. Using Pulumi and Terraform


    Side-by-Side from


    Terraform state
    import * as pulumi from "@pulumi/pulumi"
    ;

    import * as terraform from "@pulumi/terraform"
    ;

    const con
    fi
    g = new pulumi.Con
    fi
    g()
    ;

    const tfeToken = con
    fi
    g.requireSecret("tfeToken")
    ;

    const networkState = new terraform.state.RemoteStateReference("network",
    {

    backendType: "remote"
    ,

    token: tfeToken
    ,

    organization: "acme"
    ,

    workspaces:
    {

    name: "production-network"
    }
    ,

    })
    ;

    View full-size slide

  27. ⾃⾏開發整合⼯具?


    (Automation API)

    View full-size slide

  28. https:
    / / w w w
    .pulumi.com/blog/automation
    -
    api/

    View full-size slide

  29. YA! 不⽤使⽤ CLI Tool


    (pulumi command)

    View full-size slide

  30. Automation API allows you to embed Pulumi within your application code

    View full-size slide

  31. 1. Database Migration

    View full-size slide

  32. _, err = rds.NewClusterInstance(ctx, "dbInstance", &rds.ClusterInstanceArgs{


    ClusterIdentif
    i
    er: cluster.ClusterIdentif
    i
    er,


    InstanceClass: rds.InstanceType_T3_Small,


    Engine: rds.EngineTypeAuroraMysql,


    EngineVersion: pulumi.String("5.7.mysql_aurora.2.03.2"),


    PubliclyAccessible: pulumi.Bool(true),


    DbSubnetGroupName: subnetGroup.Name,


    })


    if err
    ! =
    nil {


    return err


    }


    ctx.Export("host", cluster.Endpoint)


    ctx.Export("dbName", dbName)


    ctx.Export("dbUser", dbUser)


    ctx.Export("dbPass", dbPass)


    View full-size slide

  33. / /
    create our stack with an "inline" Pulumi program (deployFunc)


    stack
    : =
    auto.UpsertStackInlineSource(ctx, stackName, projectName, deployFunc)


    / /
    run the update to deploy our database


    res, err
    : =
    stack.Up(ctx, stdoutStreamer)


    fmt.Println("Update succeeded!")


    / /
    get the connection info


    host
    : =
    res.Outputs["host"].Value


    dbName
    : =
    res.Outputs["dbName"].Value


    dbUser
    : =
    res.Outputs["dbUser"].Value


    dbPass
    : =
    res.Outputs["dbPass"].Value


    / /
    establish db connection


    db
    : =
    sql.Open("mysql", fmt.Sprintf("%s:%s@tcp(%s:3306)/%s", dbUser, dbPass, host, dbName))
    defer db.Close()


    / /
    run our database "migration"


    fmt.Println("creating table
    . . .
    ")


    db.Query(`


    CREATE TABLE IF NOT EXISTS hello_pulumi(


    id int(9) NOT NULL,


    color varchar(14) NOT NULL,


    PRIMARY KEY(id)


    );


    `)


    View full-size slide

  34. 2. RESTful API Service

    View full-size slide

  35. ctx
    : =
    context.Background()


    stackName
    : =
    createReq.ID


    program
    : =
    createPulumiProgram(createReq.Content)


    s, err
    : =
    auto.NewStackInlineSource(ctx, stackName, project, program)


    if err
    ! =
    nil {


    / /
    if stack already exists, 409


    if auto.IsCreateStack409Error(err) {


    w.WriteHeader(409)


    fmt.Fprintf(w, fmt.Sprintf("stack %q already exists", stackName))


    return


    }


    w.WriteHeader(500)


    fmt.Fprintf(w, err.Error())


    return


    }


    s.SetConf
    i
    g(ctx, "aws:region", auto.Conf
    i
    gValue{Value: "us
    -
    west-2"})


    / /
    deploy the stack


    / /
    we'll write all of the update logs to st out so we can watch requests get processed


    upRes, err
    : =
    s.Up(ctx, optup.ProgressStreams(os.Stdout))


    if err
    ! =
    nil {


    w.WriteHeader(500)


    fmt.Fprintf(w, err.Error())


    return


    }


    View full-size slide

  36. State Management

    View full-size slide

  37. %FGBVMUJO$MPVE %FGBVMUJO-PDBM
    IUUQTFOXJLJQFEJBPSHXJLJ5FBN@DPO
    fl
    JDU

    View full-size slide

  38. Why I Choose


    Pulumi?

    View full-size slide

  39. 開發者
    • 可以使⽤熟悉的語⾔, 不⽤學習新語法


    • 模組及跨平台 (AWS, GCP, Azure)


    • 減少複製流程, ⾃由建立模組清單

    View full-size slide

  40. DevOps 團隊
    • Infrastructure as Code (可審核檢視)


    • Multi-Cloud DevOps (⽤⼀種⽅式管理多個雲平台)


    • Deploy Continuously (⾃動化部署)

    View full-size slide

  41. 資安團隊
    • Policy as Code


    • Built-In Secrets (加密特定資料: DB 密碼 ..)


    • Enforce Standards (同⼀套流程管理系統架構)

    View full-size slide

  42. Pulumi
    Architecture

    View full-size slide

  43. Pulumi Overview
    IUUQTXXXQVMVNJDPNEPDTJOUSPDPODFQUT

    View full-size slide

  44. Pulumi Overview
    3VOUIF1VMVNJ$-*DPNNBOE QVMVNJVQ

    View full-size slide

  45. name: footer


    runtime: go


    description: A minimal AWS Go Pulumi program


    template:


    conf
    i
    g:


    aws:region:


    description: AWS region


    default: ap
    -
    southeast-1
    conf
    i
    g:


    aws:prof
    i
    le: footer


    aws:region: ap
    -
    southeast-1


    project:environment: production


    project:name: footer


    pulumi:template: aws
    -
    go


    1SPKFDU :".-
    4UBDL :".-

    View full-size slide

  46. package main


    import (


    "github.com/pulumi/pulumi
    -
    aws/sdk/v4/go/aws/s3"


    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"


    )


    func main() {


    pulumi.Run(func(ctx
    *
    pulumi.Context) error {


    / /
    Create an AWS resource (S3 Bucket)


    bucket, err
    : =
    s3.NewBucket(ctx, "my
    -
    bucket", nil)


    if err
    ! =
    nil {


    return err


    }


    / /
    Export the name of the bucket


    ctx.Export("bucketName", bucket.ID())


    return nil


    })


    }
    NBJOQSPHSBN

    View full-size slide

  47. Pulumi CLI Console

    View full-size slide

  48. Pulumi Cloud Console

    View full-size slide

  49. Pulumi Cloud Console

    View full-size slide

  50. Pulumi Cloud Console

    View full-size slide

  51. Pulumi Cloud Console

    View full-size slide

  52. 快速建置簡單網⾴


    使⽤ AWS S3 + Static
    Website

    View full-size slide

  53. pulumi.Run(func(ctx
    *
    pulumi.Context) error {


    / /
    Create an AWS resource (S3 Bucket)


    bucket, err
    : =
    s3.NewBucket(ctx, "my
    -
    bucket", &s3.BucketArgs{


    Website: s3.BucketWebsiteArgs{


    IndexDocument: pulumi.String("index.html"),


    },


    })


    if err
    ! =
    nil {


    return err


    }


    site
    : =
    getEnv(ctx, "s3:siteDir", "content")


    f
    i
    les, err
    : =
    ioutil.ReadDir(site)


    if err
    ! =
    nil {


    return err


    }


    for _, item
    : =
    range f
    i
    les {


    name
    : =
    item.Name()


    if _, err = s3.NewBucketObject(ctx, name, &s3.BucketObjectArgs{


    Bucket: bucket.Bucket,


    Source: pulumi.NewFileAsset(f
    i
    lepath.Join(site, name)),


    Acl: pulumi.String("public
    -
    read"),


    ContentType: pulumi.String(mime.TypeByExtension(path.Ext(f
    i
    lepath.Join(site, name)))),


    }); err
    ! =
    nil {


    return err


    }


    }


    / /
    Export the name of the bucket


    ctx.Export("bucketID", bucket.ID())


    ctx.Export("bucketName", bucket.Bucket)


    ctx.Export("bucketEndpoint", bucket.WebsiteEndpoint)


    return nil


    })

    View full-size slide

  54. Pulumi Pricing

    View full-size slide

  55. IUUQTXXXQVMVNJDPNQSJDJOH

    View full-size slide

  56. 免費⽅案
    IUUQTXXXQVMVNJDPNQSJDJOHGBRQSPEVDU

    View full-size slide

  57. Convince Your Boss
    IUUQTXXXQVMVNJDPNQSJDJOHGBRQSPEVDU

    View full-size slide