Using the AWS SDK

Introduction to using the AWS SDK

Transcript

1. Using the AWS SDK
2. About me • Thorsten Höger • Cloud Consultant - Taimos GmbH • AWS Community Hero • twitter: @hoegertn • web: www.taimos.de
3. Agenda • What is an SDK? • Use Cases • Supported languages • Authentication • Anatomy of an API call • Examples • Pitfalls • Questions?
4. What is an SDK? • „A software development kit (SDK or devkit) is typically a set of software development tools that allows the creation of applications for a certain [...] development platform. “ - Wikipedia • Allows to call the AWS API without having to handle HTTP, Headers, Auth, etc. on your own.
5. Use Cases • Automation of repeated tasks • if CLI is not possible or case is too complicated for bash • running in AWS Lambda • platform independant • Using AWS managed services from you application • Storage - DynamoDB, S3, ... • Communication - SQS, SNS, Kinesis, SES, ... • AI/ML - Rekognition, Translate, ... • User management - Cognito, ... • Monitoring & Audit • Run security checks in your account
6. Supported languages • Java • Javascript / Typescript • .NET • PHP • Python • Ruby • Golang • C++
7. Authentication • Can be configured statically using AccessKey/SecretKey • should not be used for obvious reasons • Can use configuration of AWS CLI • Profiles, Temporary Credentials, SAML • No default support for source_profiles from CLI !! • MFA might be a bad idea for automation • Can use EC2 instance profiles / IAM Roles • Recommended when running on an EC2 box, Lambda, etc. • Additionally support for Environment variables • at least the keys are not part of the source code • à Examples
8. Authentication (e.g. Java) • public class DefaultAWSCredentialsProviderChain • provider chain that looks for credentials in this order: • Environment Variables • AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (RECOMMENDED since they are recognized by all the AWS SDKs and CLI except for .NET) • AWS_ACCESS_KEY and AWS_SECRET_KEY (only recognized by Java SDK) • Java System Properties - aws.accessKeyId and aws.secretKey • Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI • Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and security manager has permission to access the variable • Instance profile credentials delivered through the Amazon EC2 metadata service
9. Anatomy of an API call • Prepare your request object • similar fields to CLI arguments • Create API client object • provide authentication data anr region information • Call API using request object • like: response = client.method(request) • Parse response object • same structure as the CLI response • handle pagination here !!
10. Anatomy of an API call • Different way to call API using NodeJS • Callbacks • client.method(params, function(err, res) { doSomethingWith(res); }); • Promises • client.method(params).promise().then(res => doSomethingWith(res)); • client.method(params).promise().then(doSomethingWith); • Async/Await (ES2017 !not yet in AWS Lambda!) • let res = await client.method(params).promise();
11. Examples • Create client (simple) • Java client for EC2 • AmazonEC2 client = AmazonEC2ClientBuilder.defaultClient() • NodeJS client for DynamoDB • let dynamoClient = new AWS.DynamoDB.DocumentClient();
12. Examples • Create client with authentication • Java client with region and profile • AmazonEC2ClientBuilder.standard().withCredentials(new ProfileCredentialsProvider("myprofile")).withRegion(Regions.EU_CENTRAL_1). build()
13. Examples • Load configuration from SystemsManager Parameter Store • https://github.com/taimos/dvalin/blob/master/cloud/aws/src/main/java/de/ taimos/dvalin/cloud/aws/ParameterStorePropertyProvider.java • Encrypt and decrypt data using KMS • https://github.com/taimos/dvalin/blob/master/cloud/aws/src/main/java/de/ taimos/dvalin/cloud/aws/crypt/CryptoService.java
14. Live demo
15. Pitfalls • Proxy • most SDKs do not support proxies out-of-the-box • not even the HTTPS_PROXY env variable • Pagination • most list/describe calls have limits a do paginate • you have to implement the next call manually • Error handling • it is an internet API; many things can go wrong • Timeouts • Connections resets • API call limits
16. Questions?