Building Internet-scale Applications: The Beginning By @gaveen At #RefreshColombo 22 September 2011 

Gaveen Prabhasara twitter.com/gaveen SysAdmin DevOps Works at Thinkcube gaveen.owain.org github.com/gaveen [email protected] 

Standing on the Shoulders of Giants If I claim to be a wise man, well It surely means that I don't know “ “ - Kansas Carry On Wayward Son 

This is a 25 minute talk Which sounds like a collection of Random thoughts about Building Internet-scale Systems You probably already know 

Internet-scale What is it? Applications Does not always mean WebApps 

Internet-scale > Enterprise-scale Facebook, YouTube, Twitter, Gmail, Flikr Eg: Case: Facebook 750M+ active users 350M+ daily users 100B+ daily hits 30B+ monthly objects shared 2.5M+ websites integrated 130TB+ daily logs *Sources: http://goo.gl/zT1bL http://goo.gl/cdFPI Case: Akamai 95800+ servers 1000+ networks 70+ countries 15-30% WWW traffic *Source: http://goo.gl/yVmG9 

Culture Culture Culture How 

Know Your Trade Learn Acquire domain-specific knowledge Learn how people build similar services Learn about Technologies Stay updated RTFM / Use Search Engines ;) 

Get Right People Team Find balance in the team Know their strengths Nurture them Set a proper environment Don't step on others toes 

Go for the Best Solution Architecture & Design Have a solid architecture Find a good engineering design Don't over-engineer / over-complicate Avoid architecture degradation 

Right Tool for the Right Job Technology You favourite tool can't do everything. Face it 'Prejudice against' & 'religious for' isn't good for you Brand names aren't for worshiping Cloud isn't a magical silver bullet This is Spar... the Internet. It's big 

Do Things Right Process Use best practices Automate. Automate. Automate. Plan for scaling from the start Don't let the process be a burden Little more detail... 

Test All the Time or better, do Continuous Integration Little more detail... Testing isn't a QA job Let Devs take responsibility XP, TDD, BDD, Cucumber, etc. Don't push untested code Works-in-my-machine doesn't count Process 

Fight the Smell, Fight Anti-Patterns Little more detail... Re-factoring is good Code smell isn't something to behold Don't let the code age on you Don't let the architecture degrade Bad code is bad code Process 

Use Version Control Little more detail... Use a proper VCS Git, SVN, Hg, Bzr, etc. Helps automation No version control, no deploying Learn to use a VCS properly Process 

Use Configuration Management Little more detail... Infrastructure is Code, therefore Automate User a Configuration Management System Puppet, Chef, CFEngine, MCollective, etc. One-off work is (usually) a myth Managed Infra opposed to mucked-about infra DevOps Process 

Security isn't a Task Little more detail... You can't apply security later Both infra & code needs to be secure “Noboday wants to hack me” is a myth HTTPS isn't a fad # chmod 777 kills puppies SELinux / AppArmor isn't your enemy Process 

Scaling starts from the beginning Little more detail... Plan early. Start from basics. Balance load & plan for HA Use Caching Do stress / load testing Not every technology scales gracefully Use messaging, async jobs when needed Etc., etc. Process 

Monitor & Measure Little more detail... Capture important metrics Measure, analyze & improve Monitoring sucks. But still... Home grown a NMS aren't necessarily better Have notification alerting in place Process 

Q & A Anybody who wants ISO's of the following Linux distros, meet me with USB drives after the session. BackTrack, CentOS, Debian, Fedora, Hanthana, MeeGo, Mint, Ubuntu, Ubuntu Server