Premday #3 - How Criteo operates SONiC network fabric

Criteo presents how it operates its SONiC network fabric at scale.

Transcript

1. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. How we operate our SONiC network fabric Spoiler: not by hand KEVIN PETREMANN STAFF SRE – NETWORK DC TEAM PREMDAY 2026
2. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. Our scale 1,700+ Community SONiC production switches 5 people in the team Manual processes stopped scaling a long time ago.
3. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. Our needs changed few years ago 3 From fire and forget to changes upon user request PAST TODAY TARGET Bootstrap ZTP for full configuration ZTP for interfaces and system (syslog, metadata...) ZTP for minimal configuration Day to day Update from time to time Same way for: • Pushing the full configuration • Maintaining the desired state (every 30min)
4. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. “ AFK pipeline 4 Open source, except the controller. Disclaimer: there are multiple valid automation options, the best choice depends on your needs SDN Controller API NetBox (CMDB) Data Aggregation API Salt Devices AFK
5. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. “ SDN Controller API 5 The entrypoint and abstraction layer. • Push: triggers changes e.g. auto-remediation, user requests • Pull: exposes real-time info e.g. health checks, network status, ramp-up readiness SDN Controller API NetBox (CMDB) Data Aggregation API Salt Devices AFK
6. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. “ NetBox - CMDB 6 Our source of truth: • IPAM + CMDB in one place • Network OS-agnostic data • Relational: no data duplication SDN Controller API NetBox (CMDB) Data Aggregation API Salt Devices AFK
7. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. “ Data Aggregation API 7 The validated view of the device: • Collects data from NetBox • Validates and builds full device config • Leverages YANG models (OpenConfig / IETF) • Network OS-agnostic SDN Controller API NetBox (CMDB) Data Aggregation API Salt Devices AFK
8. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. “ Salt 8 The only Network OS coupled layer • Consumes data from the Data Aggregation API • Renders it to device-specific config • Deploys to the device SDN Controller API NetBox (CMDB) Data Aggregation API Salt Devices AFK
9. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. It works well! Managing thousands of devices Change in data is deployed in less than 30min
10. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. Some caveats NetBox: Too slow at our scale IPAM touched by multiple teams = side effects Salt: Feedback loop: not fast/reliable enough to us Support declining
11. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. Vanilla OpenConfig Standard OpenConfig alone is not enough: Not all features are in the model (on purpose) We have added another model (IETF for SNMP, …) We have forked OpenConfig to add missing entries (delay open timer, aggregates, network, …)
12. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. The main pain point was supporting multiple Network OS Multi-OS tax: • Initially SONiC, and two proprietary NOS • Opposite implementations = nightmare to reconcile
13. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. “ Road to AFKv2 (maybe?) 13 Less components: • Move away from NetBox: Homemade for more control Hosting the data Generating and validating the device view (YANG) • Move away from Salt: Dedicated binary on the device SDN Controller API Network Source Of Truth Devices
14. PROPRIETARY & CONFIDENTIAL. COPYRIGHT © CRITEO 2026. ALL RIGHTS RESERVED. Thank you https://criteo.github.io/AFK/ 14