• using databases • designing large-scale processing systems • integrate different data sources into Data Lake • knowledge of Hadoop ecosystem: HDFS, Spark, Hive, Kafka, Druid, etc. • data importing Data Engineer responsibilities and skills
it is deﬁnitely Artiﬁcial Intelligence. However, if it’s written in Python/R/Scala/whatever, it is probably Machine Learning. ML is just one of the attempts to achieve AI - the best we currently have, but surely not good enough to reach it at any point. Many forms of Government have been tried, and will be tried in this world of sin and woe. No one pretends that democracy is perfect or all-wise. Indeed it has been said that democracy is the worst form of Government except for all those other forms that have been tried from time to time.… Winston Churchill
many different tools which are used depending on the problem: - Kafka - events processing - HBase - key-value storage - Hive - SQL-like data storage - Spark - generic framework for distributed computing - and many more...
common choices when it comes to Data Science - R and Python. As we mostly have an experience with Python, there are some commonly used tools: - pandas - data manipulation - matplotlib, seaborn - data visualization - scikit-learn, Tensorﬂow, Keras - machine learning algorithms implementation
modern companies collects a lot of data which is not utilized, however it could and even should be. The myth is, we need to have lots of data to perform a modelling, but that’s not true. Actually even a small business may become a data driven organization, and Data Science shouldn’t be treated as a magical problem solver for all the issues we have.
aging society 2. process automation - e.g. replacing dangerous jobs with machines 3. ecommerce and sales - targeting customers 4. communication - chatbots, disabilities 5. funny images manipulation and memes generation