Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CNN Digital Publishing Platform

CNN Digital Publishing Platform

Federico Cargnelutti

September 28, 2021
Tweet

More Decks by Federico Cargnelutti

Other Decks in Technology

Transcript

  1. Digital Publishing Platform & AWS Lambda By Federico Cargnelutti Senior

    Software Engineer, CNN
  2. AWS Lambda is an event-driven, serverless computing platform provided by

    AWS. Applications react to events, and Amazon runs the server and dynamically manages the allocation of resources. Node and Python are the leading languages for Lambda: • Node: 67.6% • Python: 26.4% • Others: 6% (*) Source: epsagon.com AWS Lambda
  3. Stellar uses the Pub/Sub pattern to notify other services when

    certain events take place. CMS Events: • clay:createPage • clay:publishLayout • clay:publishPage • clay:save • clay:saveMeta • clay:saveUser • clay:schedulePage • clay:unpublishPage • clay:unschedulePag CMS Events
  4. The amphora-event-bus package uses the Amazon SNS module to publish

    CMS Events to an SNS topic so that any number of distributed systems can subscribe to messages. Publisher
  5. An SNS Topic (event bus) is used to achieve anonymity

    between publisher and subscriber. Lambdas subscribe to the topic to receive messages published to that topic. Event Bus
  6. AWS SNS + SQS + Lambda • Amazon SNS sends

    an Amazon SQS message to the subscribed queue. • Amazon SQS notifies AWS Lambda (hey, wake up!) • AWS Lambda polls the SQS queue and invokes the Lambda function synchronously with an event that contains queue messages. • The Lambda function (or handler) reads messages in batches and invokes your function once for each batch.
  7. SQS Message The Amazon SQS message will look similar to

    this: When your function successfully processes a batch, Lambda deletes its messages from the queue.
  8. Lambda Hibernation AWS has the ability to launch Lambda functions,

    set them up as desired, hibernate them, and then bring them back to life when needed. The hibernation process stores the in-memory state of the instance, along with its private and elastic IP addresses, allowing it to pick up exactly where it left off.
  9. Stellar 2.0 Serverless Architecture

  10. Infrastructure (...gulp!)

  11. CloudFormation Templates CloudFormation is an AWS tool for deploying infrastructure.

    AWS uses the CloudFormation template specification to define a Lambda function: • Template sections • Resources type • Resource properties • Data types
  12. CloudFormation Templates You describe your desired infrastructure in YAML then

    submit your CloudFormation template for deployment:
  13. Create Stack The aws-stack-builder.sh script automates the deployment of the

    Lambda stack using CloudFormation templates. ./aws-stack-builder.sh -c create-deploy -t ec2-compute-pipeline -b ec2-clay -p ps0001 -y -L all The script is responsible for: • Creating an SQS Standard Queue • Configuring the SQS queue to trigger an AWS Lambda function • Creating an SNS topic • Adding an SNS Subscription Filter Policy • Subscribing the SQS queue to an SNS topic
  14. To deploy only lambda functions: ./aws-stack-builder.sh -c deploy -y -k

    4da396 -b lambdas-clay -L all Deploy Lambdas