Guide to creating an AWS Lambda
function in Python with DuckDB
nk(えぬけー)
Slide 2
Slide 2 text
Self-Introduction
Name:Nk(えぬけー)
Job Title:Digital Transformation Promotion?
(Excel Data Transfer Automation…just like that…)
Hobby:Playing guitar
X(Twitter):@NkEnuke
The article URL for today's presentation
is as follows↓
https://zenn.dev/amana/articles/7651ec03bb6c3e
Slide 3
Slide 3 text
Overview
1.Why AWS Lambda with DuckDB Now
2.Creating Lambda Layer(Including duckdb)
3.Uploading zipfile to Lambda Layer
4.Create and Execute the Lambda Function
Slide 4
Slide 4 text
Why AWS Lambda with DuckDB Now
I had heard about examples of using DuckDB with AWS
Lambda to perform SQL operations on S3.
(Instead of AWS Athena)
However, there is no article using Lambda function in Python
with duckdb.
Slide 5
Slide 5 text
#Premise
・The AWS Lambda function
has permissions
to access CSV data on S3.
・S3 has been set on Lambda
as a trigger.
・The Lambda settings are →
Slide 6
Slide 6 text
Creating Lambda Layer(Including duckdb)
1.Making directory on Local
mkdir python
2.pip install duckdb with this option
pip install -t python --platform manylinux2014_x86_64 --only-binary=:all: duckdb
3.Zip it for Lambda Layer
zip -r layer.zip python
Slide 7
Slide 7 text
Uploading zipfile to Lambda Layer
I made a
mistake…
have to set
x86_64
option
Slide 8
Slide 8 text
Create and Execute the Lambda Function
Slide 9
Slide 9 text
…And Done!
Slide 10
Slide 10 text
Summary
1.Instead of AWS Athena,we can use Lambda with
duckdb.(Lower price)
2.When you create Lambda Layer with duckdb,you have to set
some option on installing.
3.Lambda function need some code.(But 8 lines!)