Cloud versus physical cluster

This presentation attempts to generically examine the issues involved with
deciding cloud versus cluster. Every case will be different, it tries to
examine factors, architecture options, data scaling and make the point that
end of life costs should be added to the decision.

Links for further information and connecting

http://www.semtech-solutions.co.nz

http://www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ref=dp_byline_cont_book_1

https://nz.linkedin.com/pub/mike-frampton/20/630/385

Transcript

1. Cloud Versus Cluster • When creating distributed systems • How do you decide – cloud versus physical cluster ? • Or a hybrid system ? • What factors need to be considered ? • What open source systems might be used ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
2. Cloud Versus Cluster - Factors • How do you decide cloud versus cluster ? • What factors need to be considered ? – Cost – Security – Data volumes / velocity – Data peak / scaling – Other ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
3. Cloud Versus Cluster – Costs • What Costs do you need to consider ? – For public cloud – For private cluster – For hybrid • Are there tax implications ? • Are there costs benefits due to location ? • What about about end of life for your systems ? – Moving data – Mothballing systems – Any vendor lock in ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
4. Cloud Versus Cluster – Costs - Cluster • What are the costs / issues for a physical cluster ? – Hardware – Hardware failures – Energy + rental / location costs – Personnel – End of life costs – Data transmission – Networking – Tax and tax benefits – Other ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
5. Cloud Versus Cluster – Costs - Cloud • What are the costs / issues for a cloud based cluster ? – Depends on type used .. i.e. IaaS – Depends on data volumes – Consider data transmission – Personnel – some but less than cluster – End of life costs – Important ! • Costs to move data !! • Vendor lockin – Tax and tax benefits – Other ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
6. Cloud Versus Cluster – Security • How important is security to you ? • Is your public cloud secure ? – Can you guarantee that ? • Perhaps consider a private cloud • Or a hybrid public private cloud ? • How resilient / available is the resource ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
7. Cloud Versus Cluster – Data Volumes • The next two slides consider data volumes • What about resource scaling ? – Especially for the internet of things ( IoT ) – Potentially expect large demand peaks – Need to scale up as well as down • Also, scaling needs to follow resource demand – Dont want to be over scaled – Dont want to be under scaled www.semtech-solutions.co.nz info@semtech-solutions.co.nz
8. Peak Data Demand • The diagram below expresses the idea that – Periodic demand can be many times average ( i.e. IoT ) – This places pressure on the software stack for • Queueing, processing, storage www.semtech-solutions.co.nz info@semtech-solutions.co.nz
9. Project Resource Allocation Over Time • The diagram below expresses the idea that – Resource supply to a project will not meet demand exactly – There will be under and over supply – Unused or unavailable resources waste money www.semtech-solutions.co.nz info@semtech-solutions.co.nz
10. Cloud Versus Cluster – Architecture • What open sourced systems might be used ? • Mesos will help with – Resource sharing and management • Brooklyn will help with – Deployment modelling / monitoring – Will allow multiple cloud deploys • Cloud Stack will help with – Cloud creation / virtualisation • See other presentations here .. www.semtech-solutions.co.nz info@semtech-solutions.co.nz
11. Cloud Versus Cluster – Summary • Decide what factors are important to you i.e. – Is security more important than cost ? – Does IoT mean that auto scaling is required ? • Consider how you will deploy to cloud / cluster • Consider end of life • Create a cost model at start of project – To consider all cost factors – To justify your decision – Because every project instance will differ ! www.semtech-solutions.co.nz info@semtech-solutions.co.nz
12. Available Books • See “Big Data Made Easy” – Apress / Springer Jan 2015 • See “Mastering Apache Spark” – Packt Oct 2015 • Find the author on Amazon – www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ • Connect on LinkedIn – nz.linkedin.com/pub/mike-frampton/20/630/385 www.semtech-solutions.co.nz info@semtech-solutions.co.nz
13. Contact Us • Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz • Or connect on LinkedIn • We are always interested in – New technology – Opportunities – Technology based issues – Big data integration