Big data in the trenches

Big data in the trenches

Every minute the Internet produces a crazy amount of data. No matter if you work on social network, Internet of Things application, e-commerce site or online casino - you'll always end up with various sources and formats of data. Having data alone won't give you any advantage. Knowing what to do with it, will.
The situation when data is structured (even in a well known JSON format), either SQL or NoSQL is a dream for everyone who takes care of data. But often times you'll end up with very dynamic content, not very well structured or containing hard to parse items. You still need to live with this and act quickly in order to find the insight that is vital for your operations.
This presentation's purpose is not to praise Big Data or to promote its advantages and amazing business opportunities. We'll get our hands dirty touching upon entire path, from producing and capturing the data, to it's intermediate aggregation, through real-time data storages, stream processing to end up with distributed file system and batch processing over millions of records.
To support this with facts, we will talk about real life examples using PHP as front-facing layer, NoSQL databases, messaging queues and powerful map-reduce tools. You will hear many times about Couchbase, MongoDB, RabbitMQ, Kafka and Apache Spark, to name just a few. We'll also prove that SQL is not dead, and that it is entering into whole new era!


Wojciech Sznapka

October 02, 2016