MongoDB UK 2012: MongoDB on Innovation Projects / Products

Norberto Leite, Software Engineer, Telefonica R&D
This talk is about the interest and adoption growth of a Research and Development company like Telefonica R&D. Going through the different phases that new technologies pass till they are accepted and adopt within the technological mindset of a company. I'll discuss those phases @Telefonica, the different steps that mongoDB is taking within the projects, the different problems that add to be dealt with towards the general consensus, the way project members start to see mongoDB, the corporate view towards this new technological solution and the way customers come along and start to trust on solutions that present mongoDB as part of the "package". Part of the talk is to give an insight of how 10gen has helped to develop a good understanding of the technological advantages, the training and support provided which leads to a better adoption of the product.

Transcript

1. 00 DISCLAIMER
2. 2 Disclaimer TELEFONICA TELEFONICA DIGITAL PRODUCT DEVELOPMENT AND INNOVATION INNOVATION WHAT? HOW? WHEN? 0.1 0.2
3. 0.1 TELEFONICA
4. 4 TELEFONICA DISCLAIMER 0.1 § Telefonica • One of the world’s biggest (ranked between 3 and 6 largest depending on the source). • A simple google search would give you some numbers › http://trak.in/tags/business/2010/03/31/top-best-telecom-companies-in-world/ › http://en.wikipedia.org/wiki/List_of_mobile_network_operators › http://www.creditloan.com/infographics/the-worlds-largest-telecom-providers/ § Telefonica Digital (http://www.telefonica.com/en/digital/html/home/) • Branch company dedicated to seizing opportunities within the digital world to deliver new growth to Telefonica § PDI (Project Development and Innovation) • Division under Telefonica Digitial
5. 0.2 INNOVATION
6. 6 INNOVATION DISCLAIMER 0.2 § What? • Let’s just ask google! › http://www.merriam-webster.com/dictionary/innovation ›  “introduction of something new; a new idea, method, device” › http://www.thefreedictionary.com/innovation ›  “something newly introduced, such as a new method or device” › http://en.wikipedia.org/wiki/Innovation ›  “Innovation is the creation of better or more effective products, processes, services, technologies, or ideas that are accepted by markets, governments, and society”
7. 7 INNOVATION DISCLAIMER § How? • By research? • By development? • By research and development? • … • How about by transforming your surroundings and making it better? 0.2
8. 8 INNOVATION DISCLAIMER § Quiz Contest • When to innovate? 1.  When things go well. 2.  When someone at the office comes up with a “brilliant” idea. 3.  When things go bad. 4.  When things go “S**t this thing does not scale!!!!”. •  Answer: ›  Who knows!?!? 0.2
9. 00 AGENDA
10. 10 Agenda MONGODB @ PDI HOW DOES IT START APROACH ADOPTION ACCEPTANCE MONGODB @ PRODUCTION ENVIRONMENT A VERY DIFFERENT STORY PROJECT VIA-CRUCIS SUITS SYSADMINS 10GEN RELATIONSHIP ACKNOLEDGEMENTS / LESSONS LEARN 01 02 03 04
11. 11 Agenda MONGODB @ PDI HOW DOES IT START APROACH ADOPTION ACCEPTANCE MONGODB @ PRODUCTION ENVIRONMENT A VERY DIFFERENT STORY PROJECT VIA-CRUCIS SUITS SYSADMINS 10GEN RELATIONSHIP ACKNOLEDGEMENTS / LESSONS LEARN 01 02 03 04
12. 01 MONGODB @ PDI
13. 13 HOW DOES IT START MONGODB @ PDI 01
14. 14 HOW DOES IT START MONGODB @ PDI 01 § PDI mission is to provide innovative services and products for the Telefonica group • This is our goal and reason of being § Innovation project usually have the following characteristics • Short development cycles • Fast prototyping • New technological experiments § Usually there is 10 ways to push new technology: • By people who understand binary • And by the ones who don’t
15. 15 HOW DOES IT START – PEOPLE THAT KNOW BINARY MONGODB @ PDI 01 § People eager to try new techs for the fun of it § Ground breakers and early adopters § Very smart and very determined § People that usually like to try first and think it about latter
16. 16 HOW DOES IT START – PEOPLE THAT DON’T MONGODB @ PDI 01 § People eager to push new projects § They want to be recognized for problem solvers and innovation leaders § They want fast results § Very perspective and persisting •  Usually they come up with very nice keynotes!!!
17. 17 HOW DOES IT START MONGODB @ PDI 01 § How does MongoDB fit all this stuff? • MongoDB has a lot of cool stuff • makes developers like • Managers love it • Sysadmin cry for it… › We will go into detail on all of this (I promise!)
18. 18 HOW DOES IT START MONGODB @ PDI 01 At PDI usually both types of people have a great sense of innovation, always looking forward to try new stuff. § Personalization Server is one of this early adopters that needed a consistent solution to a scale and flexibility problem. è
19. 19 APPROACH MONGODB @ PDI 01 @PDI both types of people pushed the envelope to have solutions that used MongoDB as a piece of the package. Even with different motivation mindsets mongodb as been seen by multiple roles within our company as an opportunity to solve some problems and to push a new piece of technology in which we believe can solve many problems.
20. 20 ADOPTION MONGODB @ PDI 01 § Given the nature of the problems that we have suffered across some projects the adoption of mongodb as part of the solution came naturally. • We run multiple variables • We run multiple alternatives (couchdb, redis, cassandra …) • We liked mongo better! § Solves some of the problems we have • But is not a silver bullet !!!!! • In some cases is “the solution” in others can be a disaster. § The “people that don’t understand binary” mongodb also see as ready for adoption across the company • 10gen offers support and training • Oracle is expensive J • They like mongo too!!
21. 21 Agenda MONGODB @ PDI HOW DOES IT START APROACH ADOPTION ACCEPTANCE MONGODB @ PRODUCTION ENVIRONMENT A VERY DIFFERENT STORY PROJECT VIA-CRUCIS SUITS SYSADMINS 10GEN RELATIONSHIP ACKNOLEDGEMENTS / LESSONS LEARN 01 02 03 04
22. 02 MONGODB @ PRODUCTION ENVIRONMENT
24. 24 A VERY DIFFERENT STORY MONGODB @ PRODUCTION ENVIRONMENT 02 § Innovation projects, as most software projects, have a lot of FUD associated with: • Customers think these projects solve everything • Sponsors sell them as they can solve everything • Systems integrators think we add problems instead of solving them § Since we integrate a lot new technology, sometimes complete new to most people the following always happens • All types of documentation must be provided • Everything must be “certified” by client • No one wants to be in charge of it • When something goes wrong you are the first to be blamed! › If the Eurozone is in crisis your are to be blamed!!!
25. 25 PROJECT VIA CRUCIS MONGODB @ PRODUCTION ENVIRONMENT 02
26. 26 PROJECT VIA CRUCIS MONGODB @ PRODUCTION ENVIRONMENT 02 § 1º Stage – Technology experiments • Find new technology that promises to solve your problem • Play a bit with it • Do a small demo › After this your manager will ask you to build a new version of the full project using this small demo in 2 weeks! • You present ½ of a functionality showing that is possible • Your manager doesn’t like but let’s you go along with it § 2º Stage – Oblivion • You start doing some “real work” to keep up with the rest of chores • Your manager does not even remember the name of that new thing your tried › BongoDB? SongoDB? Hahh Mongo!
27. 27 PROJECT VIA CRUCIS MONGODB @ PRODUCTION ENVIRONMENT 02 § 3º Stage – Oh S*** this doesn’t scale • Now everyone is saying that your ½ of a feature rocks and it solves everyone’s problems › Even the Eurozone crisis! • We try to make it a “full” product by bringing everyone on board § 4º Stage – Oblivion for the 2º time • The previous stage just made it clear that you have a big problem that needs to be solved • Since we have a roadmap your new version is going to be forgotten to attend higher priorities • Eventually the previous problem can be workaround.
28. 28 PROJECT VIA CRUCIS MONGODB @ PRODUCTION ENVIRONMENT 02 § 6º Stage – Go go go! • The client demands solution for the scale problem • They know you have been playing with some stuff that promises a lot • You have less than ½ of the team to deliver a full version based on that technology and you 3 months › This is basically asking you gently to give up! • Since you are brave, and you know binary, you take the challenge! § 7º - Push code • You know you are on the right track • You know that you are making progress • The team is happy • You are having fun!!!
29. 29 PROJECT VIA CRUCIS MONGODB @ PRODUCTION ENVIRONMENT 02 § 8º Stage – Mission accomplished! • 80% of the previous code has been ported to a new technology • You work in a new technology that no else in the company has experienced • You are a rock start › With a lot less pay, alcohol and parties, but still a rock star § 9º - Preprod / Integration • Someone finds bugs › For sure they messed up! • Someone starts complaining about some ISO norm 900000001111XXXX • You start to get some confusing mails › “…the software was installed correctly but it does not seem to do the expected…” • The request start to become crazy › “I want some expert from 10gen to fly to XZY to check my servers”
30. 30 PROJECT VIA CRUCIS MONGODB @ PRODUCTION ENVIRONMENT 02 § 10º Stage – Production • Bug fixing • Fast fixing § 11º Stage - Support • Support § 12º Stage - Feature Requests • If I could show you guys my roadmap! § 13º Stage - Updates § 14º Stage - Restart
31. 31 SUITS MONGODB @ PRODUCTION ENVIRONMENT 02 § Management is always looking for new features to push § They want to keep their customers happy • It’s their job! § They run on tight budgets and lots of expectations § They are not trying to make your life hard but they do it • Once again they don’t know binary § They need to be able to rely on new technology • This is one of the major benefits that MongoDB offers when compared with other solutions out there, support is very import for taking the decision on adopting a new technology § They do produce nice powerpoints
32. 32 SYSDAMINS MONGODB @ PRODUCTION ENVIRONMENT 02 § Always very skeptical • They’ve heard that story of this wonderful system that does everything by it self over and over and over again… § They are usually the last to know and the last to be consulted during the technological decision making § No one likes to be called at 2:00 AM to reboot some server / service § They have their routines, their python / perl / bash scripts that do magic § Are very averse of delegating their machines control § They are not known for their powerpoints skills
33. 33 Agenda MONGODB @ PDI HOW DOES IT START APROACH ADOPTION ACCEPTANCE MONGODB @ PRODUCTION ENVIRONMENT A VERY DIFFERENT STORY PROJECT VIA-CRUCIS SUITS SYSADMINS 10GEN RELATIONSHIP ACKNOLEDGEMENTS / LESSONS LEARN 01 02 03 04
34. 03 10GEN AND US
35. 35 TRAINING 10GEN RELATIONSHIP 03 § PDI and 10gen have a good relationship § We believe that mongodb is piece of technology that is shaping the database landscape § 10gen provided very good training sessions which strengths our engineers skills and provides more inside deep knowledge over mongodb capabilities and possibilities § We want to maintain this relationship by participating in events like this to help spread experience using this technology.
36. 36 SUPPORT 10GEN RELATIONSHIP 03 § Support has been great • Fast response • Proactive solutions • Deep dive into the systems that go way over mongodb “responsibilities” § On production environment support is vital to keep customers reassured § The feeling of partnership is very present • 10gen cares for the success of the projects that we are developing using mongodb which in turn produces confidence as a reliable product.
37. 37 Agenda MONGODB @ PDI HOW DOES IT START APROACH ADOPTION ACCEPTANCE MONGODB @ PRODUCTION ENVIRONMENT A VERY DIFFERENT STORY PROJECT VIA-CRUCIS SUITS SYSADMINS 10GEN RELATIONSHIP ACKNOLEDGEMENTS / LESSONS LEARN 01 02 03 04
38. 04 ACK
39. 39 NUMBERS, FACTS AND FIGURES ACKNOWLEDGMENTS 04 § Projects in-house using mongo are growing in number and maturity • 2 initial (early adopters based on prototypes) • After an year we have around 7 projects developed using mongodb • A significant number of co-workers trained ( • Mongodb is a key technology present in our techplan • These 2 early adopters are on the verge of becoming production projects › Measure of success.
40. 40 NUMBERS, FACTS AND FIGURES ACKNOWLEDGMENTS 04 § PS Numbers • Processing Speed: › Batch load of 25M users in 4hours › Oracle version estimate would take 17h->19h › 80% faster • Space: › Oracle version 250G › Mongo version 60G › 73% less space
41. 41 NUMBERS, FACTS AND FIGURES ACKNOWLEDGMENTS 04 § PS Numbers • Access Speed: › 200ms / request (Goal) › Oracle version – 40 requests / second avg (max: 80/s) › Mongo version – 300 requests / second avg (max: 600/s) › 80% throughput increase
42. 42 LESSONS LEARN ACKNOWLEDGMENTS 04 § Project via crucis was based in a real story • After 2 years of development of a solution based on oracle at the end did not responded to the demands. • New version using mongo toke 3 months to develop and to meat the performance demands. § Training for training is worth very little. • You need to get dirty! Get involved. • Participate and share among the co-workers
43. 43 LESSONS LEARN ACKNOWLEDGMENTS 04 § Project via crucis was based in a real story • After 2 years of development of a solution based on oracle that at the end did not responded to the demands. • New version using mongo toke 3 months to develop and to meat the performance demands. § Training for training is worth very little. • You need to get dirty! Get involved. • Participate and share among the co-workers § Breath deep when things don’t happen has expected
44. QUESTIONS?
45. MUITO OBRIGADO!!!