Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Development, Testing, Deployment and Scaling of your Python Web Application

Lakshman
September 28, 2011

Development, Testing, Deployment and Scaling of your Python Web Application

Talk at Pycon India '11

Lakshman

September 28, 2011
Tweet

More Decks by Lakshman

Other Decks in Technology

Transcript

  1. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Development, Testing, Deploying, Hosting, Monitoring of your Python Web App. Lakshman Prasad (@becomingGuru) September 18, 2011
  2. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Common non-Development Tasks • Distributed log storage and analysis • Backups and Snapshots • Graphing, Instrumentation and Monitoring • HTTP Caching, Memory Caching • Failover, Node addition/removal • Auto-scaling for cloud resources • Data Retention/Archival, • Data Model Changes, Database sharding • CDN Management • API Metering, Rate Limiting • Handling Multiple Environments, Multiple Versions, Rollbacks
  3. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites The Application ”Growth” Cycle • Develop • Test • Design Production Environment • Deploy • Monitor • Tune
  4. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers Monitoring Beta Invites
  5. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Jacob on Testing ”Code without tests is broken by design”
  6. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Unit Testing from django . u t i l s import u n i t t e s t from myapp . models import Animal class AnimalTestCase ( u n i t t e s t . TestCase ) : def setUp ( s e l f ) : s e l f . l i o n = Animal . o b j e c t s . c r e a t e (name=” l i o n ” , sound=” roar ” ) s e l f . cat = Animal . o b j e c t s . c r e a t e (name=” cat ” , sound=”meow” ) def testSpeaking ( s e l f ) : s e l f . a s s e r t E q u a l ( s e l f . l i o n . speak () , ’The l i o n says ” roar ” ’ ) s e l f . a s s e r t E q u a l ( s e l f . cat . speak () , ’The cat says ”meow” ’ )
  7. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Titus Brown on TDD and BDD ”I don’t do test-driven development; I do stupidity-driven testing. When I do something stupid, I write a test to make sure I don’t do it again.”
  8. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Feature Testing >>> from django . t e s t . c l i e n t import C l i e n t >>> c = C l i e n t () >>> response = c . post ( ’ / l o g i n / ’ , { ’ username ’ : ’ john ’ , ’ password ’ : ’ smith ’ }) >>> response . s t a t u s c o d e 200 >>> response = c . get ( ’ / customer / d e t a i l s / ’ ) >>> response . content ’ <!DOCTYPE html . . . ’
  9. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites In Browser Testing • Selenium • Twill
  10. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites An idempotent deployment • Automated - Fabric, Puppet, Capistrano, Buildout • Isolated - virtualenv, Buildout • Repeatable - pip, easyinstall • Dependency Management - Yum, Deb, pip
  11. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Seperate DB machine (with pooling)
  12. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Monitoring • Nagios • Monit • PingDom • PagerDuty, ServiceUptime, ...
  13. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Monitor Resources, generate pretty graphs
  14. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites For PyCon India attendees • epio invite code: pyconindia82731 • django-zoom priority invites!
  15. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites About Me • Active Djangonaut and active in Python world • Part of a few popular open source django applications github.com/becomingGuru, stackoverflow.com/users/55562 • Co-Authored an ebook ”django-design-patterns” • Architect and develop django applications at InMobi • Earlier, Consulting and Development via Agiliq Solutions • Developed several custom proprietory django applications • twitter.com/becomingGuru http://becomingguru.com
  16. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Resources So s t a r v i n g : https :// github . com/ a g i l i q /so−s t a r v i n g S c a l i n g : ” Cal Henderson : B u i l d i n g S c a l a b l e Web S i t e s ” H i g h s c a l a b i l i t y . com , Kitchensoap . com Performance : ” Steve Souders : Even Faster Websites ” Cloud Hosting : ”Ken Cochrane http :// kencochrane . net / blog /” ”Jacob Kaplan Moss : Django in the Real World”
  17. Introduction Development Testing Deploying Scaling and Performance Cloud Hosting Providers

    Monitoring Beta Invites Image Attributions http ://www. f l i c k r . com/ photos / t e j e d o r o d e l u z /3157690060/ http ://www. f l i c k r . com/ photos /23820645@N05/4287681570/ http ://www. f l i c k r . com/ photos / a i d a n j o n e s /3575000735/ http :// j a c o b i a n . org / http :// s a n j u a n c o l l e g e . edu/ l i b / images / p h i l o s o p h y b r a i n . jpg http ://www. f l i c k r . com/ photos /uhop /105062059/ http :// s3 . amazonaws . com/memebox/ uploads /136/ e x p o n e n t i a l g r a p h 2 . jpg http :// geekandpoke . typepad . com/ geekandpoke / images /2008/06/03/ s e x p l 1 8 . jpg http ://www. f l i c k r . com/ photos /go /253819/ http :// aroundthesphere . f i l e s . wordpress . com/2009/05/ swiss−army−k n i f e . jpg http ://www. f r e e f o t o . com/ images /41/04/41 04 9− − −Keep−Left web . jpg http ://www. f l i c k r . com/ photos / o r i n r o b e r t j o h n /114430223/