Foods in Rosemont, IL • Previously: ◦ PwC: Consultant, Digital Tech. & Cloud Practice ◦ McDonald’s Corp.: Infrastructure Analyst • Certified AWS Solutions Architect Associate, and Developer Associate • I live in Chicago, IL • When I’m not trying to absorb as much as I can about DevOps & cloud, I enjoy drinking coffee, reading, and photography. https://matt.adorjan.co [email protected]
statistics between regions. • The goal is to provide AWS users with the data to make the best decisions about global application deployments. • All regions “ping” all other regions every 6 hours and store the average latency in DynamoDB. • Application consists of 2 parts: ◦ Front-end, single page Vue.js application ◦ Back-end, serverless API The app: AWS Inter-Region Latency Monitoring https://www.cloudping.co/ More details & code: https://github.com/mda590/cloudping.co
API and supporting services pose the most challenge for a manual deployment. ◦ Consists of disjointed parts (e.g. Lambda functions & API gateway). ◦ Specific dependencies on Amazon OS which must be built on AWS platform vs. built locally and pushed. • In order to reduce error and time to market, create an automated deployment pipeline for the API using no extra servers. • Pipeline should be automated and kick off anytime new code is pushed to ‘master’ branch.
to ‘master’ is detected. Code is pulled from GitHub. Dependencies are downloaded. Templates and packages are built. API and Lambda function are deployed via SAM templates. API is tested by calling several methods and ensuring they return expected results. Orchestrated via CodePipeline
Support directory - any code supporting the automated deployment process • buildspec.yml - CodeBuild build instructions • deploy.yml - SAM template defining Lambda API function and API gateway • lambda.py - Lambda function which acts as API backend. Used as input to SAM. • swagger-2.yml - Swagger definition for API methods and configuration. Used as input to SAM.
the Lambda deployment package, as well as the compiled SAM template. CodeBuild is configured to use the Amazon Linux w/ Python Docker container. The CodeBuild process outputs 3 artifacts, bundled together in a ZIP file: 1. Lambda.zip 2. Swagger-2.yml 3. Serverless-deploy.yml phases: install: commands: - pip install --upgrade pip - pip install numpy --target=./ pre_build: commands: - yum install zip -y build: commands: - zip -r lambda.zip lambda.py numpy*/ post_build: commands: - aws cloudformation package --template deploy.yml --output-template-file serverless-deploy.yml --s3-bucket <s3bucketforCF> artifacts: files: - lambda.zip - swagger-2.yml - serverless-deploy.yml Install: Retrieve and install the numpy package, which will be uploaded to Lambda. Pre-Build: Install the “zip” package to the container. This is used to package the Lambda artifact. Build: Create the lambda.zip artifact which will be used to create the Lambda function. Post Build: Upload the lambda artifact and the Swagger API definition to S3. Generate an updated SAM template with S3 paths. buildspec.yml
Automated deployment of a serverless function and API demo. Resources: CloudPingApiFunction: Type: 'AWS::Serverless::Function' Properties: Handler: lambda.lambda_handler Runtime: python2.7 CodeUri: s3://<bucket>/<zipId> Description: '' MemorySize: 256 Timeout: 30 Role: 'arn:aws:iam::506666621600:role/lambda_basic_execution' Events: AveragesHello: Type: Api Properties: RestApiId: !Ref CloudPingApiGateway Path: /hello Method: GET CloudPingApiGateway: Type: 'AWS::Serverless::Api' Properties: StageName: v1 DefinitionUri: s3://<bucket>/<swaggerId> Variables: LambdaFunctionName: !Ref CloudPingApiFunction LambdaRegion: !Ref 'AWS::Region' LambdaAcctId: !Ref 'AWS::AccountId' Outputs: ApiId: Description: Unique ID of the API endpoint created Value: !Ref CloudPingApiGateway serverless-deploy.yml Deployment consists of 2 parts. Part 1 Deployment of serverless-deploy.yml file, which was generated via SAM during the Build phase. Deployment is completed by creating and executing a CloudFormation Change Set. The CloudFormation process outputs the new API Gateway ID, which is used in the next part of deployment. Lambda function which will act as the API backend. For each API “resource” created, a corresponding event was created with the Lambda function. API Gateway deployment, with API specs defined in Swagger format. The new API ID is an Output of this template. It is used to create a Base Path Mapping.
Part 2 Configuration of the Custom Domain Name for the newly deployed API gateway. The Output from the CloudFormation script containing the API Gateway ID is passed into a Lambda function which takes the API Gateway ID and creates a new Base Path Mapping. This sets the deployed API Stage to use a Custom Domain Name. So rather than use a URL like this: https://7s6jzfopr6.execute-api.us-east-1.amazonaws.com/v1 The API is now accessible via: https://api-demo.cloudping.co def lambda_handler(event, context): jobId = event['CodePipeline.job']['id']; # S3 bucket and key are pulled from the data passed from the previous CodePipeline Action bucket = event['CodePipeline.job']['data'] ['inputArtifacts'][0]['location']['s3Location']['bucketName'] url = event['CodePipeline.job']['data'] ['inputArtifacts'][0]['location']['s3Location']['objectKey'] key = urllib.unquote_plus(url) s3_path = os.path.dirname(key) try: # Download the CloudFormation output zip # Open the zip file s3.download_file(bucket, key, '/tmp/cfoutput.zip') zfile = zipfile.ZipFile('/tmp/cfoutput.zip') namelist = zfile.namelist() data = zfile.read(namelist[0]) # Convert the CF output from string to JSON and grab the API ID data = json.loads(data) apiId = data['ApiId'] except Exception as e: ………(removed for size) try: # Create the API GW Base Path Mapping to the correct Custom Domain response = api.create_base_path_mapping( domainName='api-demo.cloudping.co', restApiId=apiId, stage='v1', ) putSuccess(jobId) except Exception as e: ………(removed for size) createApiGwPath.py Extract the output from CloudFormation to get the API ID Add a new Base Path Mapping for the API which was just created.
calls a Lambda function which executes 2 simple tests against the API. 1. Verify that: https://api-demo.cloudping.co/hello contains the text “Hello World!” 2. Verify that https://api-demo.cloudping.co/averages contains the text '"region": "ap-south-1"' Assuming these both are True, the CodePipeline job will be marked as successfully completed. If one of the above tests are false, the deployment will be marked as unsuccessful. def lambda_handler(event, context): # Get the current job ID for the CodePipeline job calling this function jobId = event['CodePipeline.job']['id'] status = True # Connect to Hello API test endpoint response = urllib2.urlopen('https://api-demo.cloudping.co/hello') html = response.read() expected_text = 'Hello World!' # Verify that the expected text was returned by the API call if(expected_text not in html): status = False # Connect to Cloudping Averages Data endpoint (calls DynamoDB) response = urllib2.urlopen('https://api-demo.cloudping.co/averages') html = response.read() expected_text = '"region": "ap-south-1"' # Verify that the expected text was returned by the API call if(expected_text not in html): status = False if(status == False): message = 'API did not respond or values being served are not expected.' print message putFailure(jobId, message) else: print 'API tests succeeded!' putSuccess(jobId) apiTest.py 1 2 Notify CodePipeline on whether the tests were successful or not.
oracle4engineer
shonansurvivors
naohiro_nakata
taka_aki
ra0kley
eida
kenta_hi
f6wbl6
kasada
naospon
puhitaku
rainerhahnekamp
ufuk
sonatard
holman
denniskardys
marktimemedia
paigeccino
eileencodes
jlugia
notwaldorf
marcelosomers
lara
geoffreycrofte
