in Go ◦ Implements most syscalls like a PV hypervisor would – 2 components ◦ Sentry (main runsc+containerd interface) ◦ Gofer (runs more privileged system calls)
Blending Containers and Virtual Machines: A Study of Firecracker and gVisor. In 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’20), March 17, 2020, Lausanne, Switzerland. ACM, New York, NY, USA, 13 pages. – https://doi.org/10.1145/3381052.3381315 – Tl;dr in 2020 both ran more kernel code than plain runc
looked into gVisor escapes ◦ gVisor takes some setup, or using GKE ▪ Default GKE node pool doesn’t support it ◦ Sysbox is a quick `kubectl apply` on most clouds ▪ Nodes must have a sysbox-install=yes label ▪ Should look familiar on host nodes, eg
Color of Magic. This release highlights the open source magic that Kubernetes enables across the ecosystem. “It’s still magic even if you know how it’s done.” Sir Terry Pratchett’s 64 enhancements • Alpha: 24 • Beta: 20 • Stable: 18 • 2 deprecated
(IPPR) for VPA - Increase resource utilization and minimize costs by using non-disruptive Vertical Pod Autoscaling (VPA) for automatic workload rightsizing - You can control whether a container should be restarted when resizing by setting resizePolicy in the container specification. This allows fine-grained control based on resource type (CPU or memory). - NotRequired: (Default) Apply the resource change to the running container without restarting it. - RestartContainer: Restart the container to apply the new resource values.
or newer 2. Create a Pod 3. Resize the pod using 'patch': *- Public Preview; limitations apply; kubectl patch pod <pod-name> --subresource resize --patch \ '{"spec":{"containers":[{"name":"<container-name>", "resources":{"requests":{"cpu":"100"}, "limits":{"cpu":"100"}}}]}}' Official documentation: https://kubernetes.io/docs/tasks/configure-pod-container/resize-container-resources/
containers Sidecar containers are now Stable! - Sidecar containers used to enhance or to extend the functionality of the primary app container by providing additional services, or functionality such as logging, monitoring, security, or data synchronization, without directly altering the primary application code. - Use “restartPolicy” field to use init container as sidecar container. - Omitting the “restartPolicy” field means you want to create a pure init container.
- 90 Days of History - Disabling of Workflows - Private channels go away - Use slackdump to back up any DMs or private channel history (QR code below) We might move to Discord!!!
year $2T+ invoices managed on our platform per year 18M total US workers paid via QB payroll Intuit is leading the way in building an AI-native development platform using cloud native open source technology. We’re committed to building tools that scale and giving back to the open source community.
Processing Event Processing - Real World Examples Event-Driven Systems: The Backbone of Modern Technology Everywhere When customer places an order, an event triggers inventory, payment, and shipping processes IoT Data Processing IoT sensors trigger events when vibrations or temperatures exceed safe limits, initiating maintenance and safety protocols Real Time Analytics Events trigger continuous analysis and insights, like monitoring website traffic, orders etc Fraud Detection When a transaction is initiated, it triggers processes to identify and prevent fraudulent activities
Responsive Scalable & Flexible Event Driven Architecture Enables you to detect, process, and respond to events as they happen Reliable & Resilient 1 2 3 PROCESSING PROCESSING PROCESSING PROCESSING
Applications Learning Kafka etc is time consuming, requires writing a lot of boilerplate code to integrate and can be inefficient if parallel consumption isn’t implemented appropriately Boilerplate Code Scaling event-driven applications while maintaining reliability is challenging and costly. Managing infrastructure efficiently without overspending adds to the complexity Scaling Complexities Observability in event-driven applications is challenging, identifying where latencies are introduced whether in event consumption, or processing it requires deep visibility, making it difficult to diagnose issues Observability 8
and Secure Abstract Infrastructure Decouple Source/ Sinks Serverless for Event Processing “Enables developers to build applications faster by eliminating the need for them to manage infrastructure” “Decouple the business logic from the source and the destination” “Cost efficient, Elastic, No downtime upgrades, and Security patches”
Contribute • Explore GitHub for issues tagged with ‘good first issue’ or ‘help wanted’ • Pick one that excites you! Bring Numaflow to Work • Event-driven use cases are everywhere • Try out Numaflow in your projects and share your experience! Join the Community • Connect on Slack and be part of the conversation Rar Let’s shoot for the Stars! https://github.com/numaproj/numaflow Be part of the Community!
platform for running scalable and reliable event-driven applications Scalable and Cost efficient Automatically scales from 0 to X, handling backpressure, while being lightweight and cost-efficient. Capable of running on edge with a low resource footprint K8s native event processing K8s native lightweight event processing with fully featured streaming semantics Versatile and can seamlessly operate on the edge, on-prem or in the cloud Language agnostic framework SDKs in Java, Python, Golang, Rust. In-built source/sink connectors. Easy to write sources, functions and sinks
to Aerospace!! Event Processing Processing financial transactions and propagate to other capabilities Processing IoT Data Processing on both high and low-volume sensor data streams received from IoT systems Accelerator Chaining AI/ML pipeline with dynamic resource allocation (GPUs, FPGAs etc) Digital Signal Processing Detection, decoding, and demodulation of RF signals across edge devices and on cloud Fraud Detection Analyze crypto and blockchain-related transactions for fraud
indicator of performance/reliability Target/goal, set for SLI over specific window • % of successful Sync • Sync duration • number of bph (barks per hour) • success for 95% of Sync ops / 30 days • 95% of Sync completed <20s / 30 days • >=5 barks / hour
Informative about system health (CPU/memory) • Immediate alerts about critical production incidents • Historical data and patterns 🤷 YES, BUT… : • Reactive Response • Data Overload • Alert Fatigue • Are our customers happy?
your "error budget" • For each 9 we add to our SLO, we significantly increase the engineering effort required to maintain it. • Conscious Decisions: how much reliability tax we're willing to pay • Freedom to innovate when under budget: ◦ Budget depleted → focus on reliability ◦ Budget healthy → ship new features
user expectations and business requirements • Start conservative (lower targets) and adjust • Different component may need different targets • Adjust based on feedback
Commit changes → Git repository updated → ArgoCD detects changes → Applications synchronized → Deployment successful • "Developer View" : code committed → deployed" with black boxes Important developer touchpoints: • "Did my code deploy successfully?" • "How long until my changes are live?" • “Can I check status of deployment?”
dashboard • Use CI metrics to create SLIs, related to GitOps: ◦ Manifest generation performance • Advanced metrics: ◦ Leverage webhook notifications to calculate additional metrics
up alerts on error budget - early warning about issues • Revisit and refine SLI/SLO - it’s foundation for continuous improvements • IaC approach • Simple SLOs, but more info on dashboard
Multi-Tenancy in AWS EKS at the New York Times (David Grizzanti & Luke Philips) ◦ https://youtu.be/rro686bRIQU?si=1xOw1kSt3swdfU3V • The ArgoCD AppProject - What Is a Project and How to Power Your Multi-Tenant Security (Luke Philips & Serhiy Martynenko) ◦ https://youtu.be/x2WfwLSufCI?si=qMPoLpr75Of836rB • What We Learned Designing & Securing a Multi-Tenant Developer Platform at The New York Times (Ahmed Bebars & David Grizzanti) ◦ https://youtu.be/EniokAz-Plg?si=pnxqlC1Xd3XtCck8 • Automating Configuration and Permissions Testing for GitOps with OPA Conftest (Eve Ben Ezra & Michael Hume) ◦ https://youtu.be/VCX4UALQjeg?si=BdKkEE_3BeLEcVYg
cncfcanada
nagano
voginip
hashimoto_kei
expajp
syuchimu
shinpsan
hayashiyus884
enakai00
trycycle
michielstock
nikkihalliwell
afnizarnur
morganepeng
holman
letsgokoyo
iamctodd
zakiyama
destraynor
aleyda
skipperchong
swwweet
inesmontani
![Presenter: Jamon Camisso Email: [email protected] Location: Recursion Toronto Date: 2025-06-19](https://files.speakerdeck.com/presentations/d7ca4097bfe3494d91e8f3f91d7fdc8a/slide_0.jpg){kind=link}
