Slide 1

Slide 1 text

1 & Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering

Slide 2

Slide 2 text

2 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Today’s speakers

Slide 3

Slide 3 text

3 & 1. Introduction Agrico 2. Introduction Digital Power 3. What do Analytics Engineers do? 4. Agrico’s challenge 5. Our set-up, architecture & business context 6. What can you gain with Analytics Engineering? 7. Questions 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Outline What will we cover today?

Slide 4

Slide 4 text

4 International group Active all over the world Employer of choice ~300 colleagues Research driven we actively search for the best proprietary potato varieties Cooperative Our members ( ~1700) are both owners as well as suppliers 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Who we are

Slide 5

Slide 5 text

5 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering The challenges in today's world ….and how we cope with them Growing demand Growing conditions Technology Operational effort to optimize yields and add acreage R&D driven efforts to introduce new varieties ICT driven efforts to become data driven

Slide 6

Slide 6 text

6 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Our goal is to seek "good growth" ….but what does that mean? In volume In quality As an organization For our growers / members For the environment Good growth

Slide 7

Slide 7 text

7 Team Foundation & Subsidary Founding Locations 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering About Digital Power Amsterdam Den Bosch London 145 colleagues Digital Power B.V. founded in 2016

Slide 8

Slide 8 text

8 & Analytics Engineer 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What do Analytics Engineers do? Build data models Optimize transformations Manage data & documentation Technology Business Data sources Build infrastructure Optimize performance Manage data platform Data Engineer Data outputs Build reports Optimize insights Manage data products Data Analyst

Slide 9

Slide 9 text

9 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering The typical ERP1 conditions do not apply Our business and the processes applied are very different.... 1. ERP = Enterprise Resource Planning # The typical ERP system The situation at Agrico 1 Continuous process Season based 2 Article / product structure fixed Same physical product, different articles 3 Volumes / stocks known Volumes / stocks estimated 4 Prices known Prices determined at year end 5 Sales = delivery Volumes invoiced differ from volumes delivered ("Tarra")

Slide 10

Slide 10 text

10 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Agrico’s challenge Agrico process & specific data D365 entities Data transformation s Information requirements Generic ERP data Customized ERP data

Slide 11

Slide 11 text

11 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Our set-up, architecture & business context How do we address the challenge?

Slide 12

Slide 12 text

12 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Storing Processing engine Business logic & transformations Supporting tools & services Serving Costs control Monitoring Key vault Version control Business users bronze silver gold Landing zone Raw data Intermediate data models Final data models

Slide 13

Slide 13 text

13 Stakeholder Request Report Scope & Definitions Source Data Check Mock Tables & Report Bronze Layer Document Sources & Data Quality Checks Build Silver Models & Document Transformations Build Gold Models & Document Transformations Landing Zone Stakeholder Request Report Scope & Definitions Source Data Check Mock Tables & Report Bronze Layer Document Sources & Data Quality Checks Build Silver Models & Document Transformations Build Gold Models & Document Transformations Landing Zone 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering General Workflow

Slide 14

Slide 14 text

14 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What can you gain with Analytics Engineering? Improved Data Quality Reproducibility & Documentation Faster Time-to-Insight Collaboration & Communication

Slide 15

Slide 15 text

15 & Questions? You can also find us at booth 8! Analytics Engineers in their natural habitat Mark Kranenburg

Slide 16

Slide 16 text

16 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Developer Iga Developer Pim feature 01 feature 02 develop main tag 1.0 tag 1.1 tag 1.2 Databricks DEV warehouse Core Cloud Databricks ACC warehouse Databricks PRD warehouse feature N Data Model Repo PowerBI DEV PowerBI ACC PowerBI PRD Developer Daan Scheduled jobs

Slide 17

Slide 17 text

17 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What can you gain with Analytics Engineering? Improved Data Quality Reproducibility & Documentation Faster Time-to-Insight Collaboration & Communication Consistency Validation Cleansing Accountability Versioning Transparency Coherency Standards Efficiency Holistic Interdisciplinary Sound

Slide 18

Slide 18 text

18 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Improved Data Quality Consistency Validation Cleansing Stock?

Slide 19

Slide 19 text

19 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Faster Time-to-Insight Coherency Standards Efficiency

Slide 20

Slide 20 text

20 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Reproducibility & Documentation Accountability Versioning Transparency

Slide 21

Slide 21 text

21 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Collaboration & Communication Holistic Interdisciplinary Sound Data project Stakeholders Skillsets Roles Tools

Slide 22

Slide 22 text

22 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What can you gain with Analytics Engineering? Improved Data Quality Faster Time-to-Insight Reproducibility & Documentation Collaboration & Communication

Slide 23

Slide 23 text

23 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Questions? You can also find us at booth 8! Analytics Engineer in their natural habitat Connect with Iga Connect with Mark

Slide 24

Slide 24 text

24 & Questions? 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Analytics Engineer in their natural habitat

Slide 25

Slide 25 text

25 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering How to edit the footer • To change the footer date and description, go to "Insert" at the top left. • Then choose: "Header and footer." After clicking, a window will open with the different functions. • To add a presentation title, enter your own text under the heading: ''Footer‘’. • After you have made your choice click on: "Apply anywhere." Now a footer will appear on every slide in the PowerPoint Presentation. • If you uncheck the footer, the footer disappears completely.

Slide 26

Slide 26 text

26 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Style components

Slide 27

Slide 27 text

27 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Building Blocks model

Slide 28

Slide 28 text

28 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Icons Interactions Notifications Visualisations Tools & technology Shapes Other Expertise Services DA TWA DS DE CX Research Strategy Consultancy Projects Services Academy

Slide 29

Slide 29 text

29 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Illusrations

Slide 30

Slide 30 text

30 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Timeline 2012 Founding of Digital Power 2000 Title 2000 Title 2000 Title 2000 Title

Slide 31

Slide 31 text

31 Data projects Develop solutions, planning & budget Data academy Specialist training and creative sessions about data Data services Guaranteed continuation of your data operations Data consultancy Advice, interim management & hands-on work 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What we do

Slide 32

Slide 32 text

32 & Analytics Engineer 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What do Analytics Engineers do? Technology Business Data sources Data Engineer Data outputs Data Analyst

Slide 33

Slide 33 text

33 Data Strategy Research Customer experience Technical web analytics Data engineering Data science Data analytics 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Expertise

Slide 34

Slide 34 text

34 & • Climate change • Regulation (e.g. Green deal) • R&D driven efforts to introduce new varieties 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering The challenges in today's world ….and how we cope with them • Population growth • Demographic shifts • Consumption patterns • Operational effort to optimize yields and add acreage Growing demand Growing conditions • Precision agriculture • Connectivity through the chains • Data availability • ICT driven efforts to become data driven Technology

Slide 35

Slide 35 text

35 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Our goal is to seek "good growth" ….but what does that mean? In volume In quality As an organization For our growers / members For the environment Good growth

Slide 36

Slide 36 text

36 Team Foundation & Subsidary Founding Locations 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering About Digital Power Amsterdam Den Bosch London 145 colleagues Digital Power B.V. founded in 2016

Slide 37

Slide 37 text

37 & Analytics Engineer 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What do Analytics Engineers do? Build data models Optimize transformations Manage data & documentation Technology Business Data sources Build infrastructure Optimize performance Manage data platform Data Engineer Data outputs Build reports Optimize insights Manage data products Data Analyst

Slide 38

Slide 38 text

38 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering The typical ERP conditions do not apply Our business and the processes applied are very different.... The typical ERP system • Continuous process • Article / product structure • Volumes / stocks known • Prices known • Sales = delivery • Procurement first, sales follows • Can only deliver what is on stock The situation at Agrico • Season based – discontinued • Article structures flexible: same physical product, different articles • Volumes / stocks estimated • Prices determined at year end • Volumes invoiced differ from volumes delivered ("Tarra") • Sales (delivery) often determines procurement • Sale is often different than what is on stock

Slide 39

Slide 39 text

39 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Agrico’s challenge Agrico process & specific data D365 entities Data transformations Information requirements Generic ERP data Customized ERP data

Slide 40

Slide 40 text

40 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Our set-up, architecture & business context How do we address the challenge?

Slide 41

Slide 41 text

41 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Landing zone bronze silver gold Storing Processing engine Business logic & transformations Supporting tools & services Serving Costs control Monitoring Key vault Version control Business users

Slide 42

Slide 42 text

42 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Stakeholder Request Report Scope & Definitions Source Data Check Mock Tables & Report Bronze Layer Document Sources & Data Quality Checks Build Silver Models & Document Transformations Build Gold Models & Document Transformations Landing Zone General Workflow

Slide 43

Slide 43 text

43 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering What can you gain with Analytics Engineering? Improved Data Quality Faster Time-to-Insight Reproducibility & Documentation Collaboration & Communication

Slide 44

Slide 44 text

44 & Questions? You can also find us at booth 8! Analytics Engineers in their natural habitat

Slide 45

Slide 45 text

45 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Developer Iga Developer Pim feature 01 feature 02 develop main tag 1.0 tag 1.1 tag 1.2 Databricks DEV warehouse Core Cloud Databricks ACC warehouse Databricks PRD warehouse feature N Data Model Repo PowerBI DEV PowerBI ACC PowerBI PRD Developer Daan Scheduled jobs

Slide 46

Slide 46 text

46 & 12-9-2023 Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering Today’s speakers Iga Jarosz Analytics Engineer Mark Kranenburg CFO

Slide 47

Slide 47 text

47 & Data transformations without Data Engineering? How Agrico builds dashboards in no time with Analytics Engineering