How to Create a Scalable System using Force.com, Heroku, Boomi and Iron.io

This presentation shows how and why to use queues and workers to decouple and make your application more scalable.

Transcript

1. Create a Massively Scalable System with Force.com and Heroku Travis Reeder, Iron.io, CTO and Co-founder
2. Safe harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
3. I am Travis Reeder CTO and Co-Founder of Iron.io Iron.io provides massively scalable and elastic cloud infrastructure services: IronWorker, IronMQ and IronCache. Building things to scale is our business.
4. Scalability Scalability is the ability of a system, network, or process, to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth. Source: Wikipedia
5. Systems integration Systems integration is the process of linking together different computing systems and software applications physically or functionally, to act as a coordinated whole. Source: Wikipedia
6. The example application Lead gen form. User fills out form, stores the info, posts to Salesforce as a lead and sends user an email. Easy right?
7. Easy? Not at scale  How do you keep your application responding fast at high load?  How do you deal with spikes? • Daily deal Site at 8am • You get TechCrunched  How do you deal with Rate Limits? • Rate limits at Salesforce – 5000 requests per day (208 per hour) – 5 concurrent requests
8. There is a simple solution Queues  Message Queues  Task Queues
9. Queues An amazing developer tool, great for:  Spikability  Keeping database load consistent  Decoupling parts of your application  Improving application performance • If it doesn’t need to be done while user is waiting, then don’t do it, put it on a queue  Integrations / communication between systems
10. Spikability If you do all your work in your application, you need servers... lots of servers.
11. Spikability without queues Need enough servers to handle the maximum traffic you might see.
12. Spikability with queues Queues soak up spikes with no performance or data loss.
13. Keeping database load consistent You decide how fast you want to process, not your traffic.
14. Decoupling / integrations Using queues to communicate between parts of your system is a good idea. Splitting your application into smaller parts can also be a good idea.  Loosely coupled  If one part fails, the others don’t  You can upgrade different parts separately  Maintain and find issues much easier  Oftentimes you have no control over it if it’s a third party
15. Architecture overview Heroku application Salesforce Boomi Email
16. The Heroku app Simple form. The only thing the controller does is queue up a worker task.
17. IronWorker task queue Create a worker and upload it once: ironworker.tasks.create(“lead_worker", config.merge(name: params[:name], email: params[:email], company: params[:company)) iron_worker upload lead_worker Then queue up tasks for it:
18. IronMQ message queue Very simple to use. One side pushes messages onto a queue (in our example, this is our lead gen app): The other side pulls them off (Boomi in this case): A message can be any arbitrary data that both sides understand. ironmq.queue(“lead”).post(msg.to_json) ironmq.queue(“lead”).get()
19. Boomi Boomi enables drag and drop integrations between SaaS and On-Premise applications via its out of the box connectors.
20. Scaling • On the front end, we’ve offloaded most of the work and there is no database to worry about, but if you need to scale it, Heroku makes it easy: • IronWorker and IronMQ are fully elastic so no scaling required. • Boomi and Salesforce, the main things to consider are rate limits and other limits. heroku ps:scale web=20
22. Let’s Try It! Grab your phone and go to: http://ironforce.herokuapp.com This demo is running on a single Heroku Dyno and it's ready to handle this room and then some. I promise I won’t spam you. But feel free to reply and contact me.
23. Travis Reeder CTO and co-founder Code can be found at: https://github.com/treeder/ironforce