CNN Digital Publishing Platform

Transcript

1. Digital Publishing Platform
   & AWS Lambda
   By Federico Cargnelutti
   Senior Software Engineer, CNN
   

   View Slide

2. AWS Lambda is an event-driven, serverless computing platform provided by AWS.
   Applications react to events, and Amazon runs the server and dynamically manages the
   allocation of resources.
   Node and Python are the leading languages for Lambda:
   ● Node: 67.6%
   ● Python: 26.4%
   ● Others: 6%
   (*) Source: epsagon.com
   AWS Lambda
   

   View Slide

3. Stellar uses the Pub/Sub pattern to notify other services when certain events take place.
   CMS Events:
   ● clay:createPage
   ● clay:publishLayout
   ● clay:publishPage
   ● clay:save
   ● clay:saveMeta
   ● clay:saveUser
   ● clay:schedulePage
   ● clay:unpublishPage
   ● clay:unschedulePag
   CMS Events
   

   View Slide

4. The amphora-event-bus
   package uses the Amazon
   SNS module to publish CMS
   Events to an SNS topic so that
   any number of distributed
   systems can subscribe to
   messages.
   Publisher
   

   View Slide

5. An SNS Topic (event bus) is used to achieve anonymity between publisher and
   subscriber. Lambdas subscribe to the topic to receive messages published to that topic.
   Event Bus
   

   View Slide

6. AWS SNS + SQS + Lambda
   ● Amazon SNS sends an Amazon SQS message to the subscribed queue.
   ● Amazon SQS notifies AWS Lambda (hey, wake up!)
   ● AWS Lambda polls the SQS queue and invokes the Lambda function
   synchronously with an event that contains queue messages.
   ● The Lambda function (or handler) reads messages in batches and invokes your
   function once for each batch.
   

   View Slide

7. SQS Message
   The Amazon SQS message will look similar to this:
   When your
   function
   successfully
   processes a batch,
   Lambda deletes its
   messages from the
   queue.
   

   View Slide

8. Lambda Hibernation
   AWS has the ability to launch Lambda functions, set them up as desired, hibernate
   them, and then bring them back to life when needed.
   The hibernation process stores the in-memory state of the instance, along with its
   private and elastic IP addresses, allowing it to pick up exactly where it left off.
   

   View Slide

9. Stellar 2.0 Serverless Architecture
   

   View Slide

10. Infrastructure (...gulp!)
    

    View Slide

11. CloudFormation Templates
    CloudFormation is an AWS tool for deploying infrastructure.
    AWS uses the CloudFormation template specification to define a Lambda function:
    ● Template sections
    ● Resources type
    ● Resource properties
    ● Data types
    

    View Slide

12. CloudFormation Templates
    You describe your desired infrastructure in YAML then submit your CloudFormation
    template for deployment:
    

    View Slide

13. Create Stack
    The aws-stack-builder.sh script automates the deployment of the Lambda stack
    using CloudFormation templates.
    ./aws-stack-builder.sh -c create-deploy -t ec2-compute-pipeline -b ec2-clay
    -p ps0001 -y -L all
    The script is responsible for:
    ● Creating an SQS Standard Queue
    ● Configuring the SQS queue to trigger an AWS Lambda function
    ● Creating an SNS topic
    ● Adding an SNS Subscription Filter Policy
    ● Subscribing the SQS queue to an SNS topic
    

    View Slide

14. To deploy only lambda functions:
    ./aws-stack-builder.sh -c deploy -y -k 4da396 -b lambdas-clay -L all
    Deploy Lambdas
    

    View Slide