Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using AWS CloudFront with S3 at SMARTSTUDY

Park Hyunwoo
February 16, 2013

Using AWS CloudFront with S3 at SMARTSTUDY

AWS Korean User Group #1 Seminar at 2013-01

Park Hyunwoo

February 16, 2013
Tweet

More Decks by Park Hyunwoo

Other Decks in Technology

Transcript

  1. What is S3 and CF? • S3 : Simple Storage

    Service ◦ Extreme durability. ◦ Multiple copies in selected region. ▪ vs EBS : Multiple copies in single available zone. • CF : CloudFront ◦ Content Delivery Network(CDN) service. ◦ Distribute from origin to AWS Edge Network ▪ http://aws.amazon.com/cloudfront/#details ▪ About 30 edges in global. (only in Earth, lol)
  2. Why choose these? • We serve large multimedia contents to

    global market in mobile application era. • We need redundant, fast and reasonable price content delivery service aims globally. • We have not enough human resource to maintain various system.
  3. S3 Bucket ( tokyo ) s3cmd Package ( encrypted )

    Thumbnails Video npk Transcoder Gnash FFmpeg qtfaststart Flash Movie
  4. How it works? • Copy to S3 ◦ ~100 files,

    just use Cyberduck or other GUI tool. ◦ Or, you have to use s3fs or s3cmd. ◦ If you need sync or invalidation, use s3cmd. ◦ We have ~30,000 files. Thus, s3cmd. ◦ We chose Tokyo bucket for faster uploading. • s3cmd ◦ Awesome! ◦ Works on S3 and CloudFront.
  5. How it works? (cont.) • s3cmd ◦ Can do almost

    everything as like as AWS console. ◦ Support INI-style configuration file. ◦ And, magical 'sync' command. (yeah!) • s3cmd sync ◦ Support glob-style exclude / include option. ◦ ... rexclude / rinclude for regular expression. ◦ Support automatic invalidation request. ◦ ... with rsync-like 'dry-run' option. (-n, --dry-run)
  6. • s3cmd sync ◦ Basically, sync based on MD5 information.

    ◦ (!) Over 15MB files, using only filesize. ◦ Our 30,000 files, 60GB = 8 min to dry-run. ( env : i7 2500K + SATA HDD to bucket in tokyo ) • s3cmd sync --cf-invalidate ◦ Call invalidation API automatically. ◦ (!) Simultaneous invalidation is up to 3. ◦ (!) Free for ~1,000 files per month. ◦ Check it up by s3cmd cfinvalinfo How it works? (cont.)
  7. * without transfer fee for deploying origin bucket to edges

    Bill for CloudFront Based on Total traffic • 10TB/month • $0.209/GB = $2,090* Bill for Domestic CDN Based on Peak traffic • 350Mbps/@peak • $6.00/Mbps = $2,100
  8. • Smaller ◦ Encode media with optimal options. ◦ Use

    dedicated option for each platform. ◦ Storage is cheaper than transfer fee! • Static ◦ Do not deploy volatile objects to CloudFront. ◦ Invalidation needs ~10min, often. ◦ We're still using Nginx for them. • Streamable ◦ Mobile networks are still slow. ◦ Do not make users to wait until finishing download. Tips
  9. • Gnash ◦ http://www.gnu.org/software/gnash/ • FFmpeg ◦ http://www.ffmpeg.org/ • qtfaststart

    ◦ https://github.com/danielgtaylor/qtfaststart • npk ◦ https://github.com/lqez/npk • s3cmd ◦ http://s3tools.org/ Link