This website uses cookies
Read our Privacy policy and Terms of use for more information.
A live cohort for engineers running AI models on devices in the physical world.
Four sessions · Wednesdays 6 PM ET from November 4 · 15 seats · $950
Get notified when enrolment opens
Operating Edge-AI Inference in Production
A live cohort for engineers running AI models on devices in the physical world.
Four sessions · 15 seats · Founding cohort price
You run or are about to run AI inference on hardware outside a data center.
Cameras, sensors on a factory floor, devices in vehicles, boxes in buildings you don't control. You can deploy a model.
What you're less sure about is what happens on day 200, across 300 devices, when one site starts behaving differently from the rest.This is not an introduction to a specific device, TensorRT or machine learning.
It assumes you already know Linux, containers, and how to get a model running on a device.
If you've never deployed a model to edge hardware, start with NVIDIA's free material and come back.
Almost everything written about ML operations assumes a cluster you can reach.
Autoscaling, rolling updates, readiness probes, centralized logs, all of it assumes the machine is one API call away.At the edge, none of that holds.
Rollback is expensive, connectivity is optional, hardware fails physically, and a model can be perfectly healthy by every standard check while being substantially worse than the one it replaced.The material that does cover this is fragmented and almost always attached to a vendor's platform. This course is vendor-neutral and taught from production experience: what actually breaks, and what to do about it.
Design a staged model rollout across a device fleet, with automatic local rollback
Define what "healthy" means for a model in the field, and detect quality regression without ground truth
Instrument constrained devices to tell you what you need without saturating their network or storage
Choose edge hardware on evidence - throughput per watt, thermal headroom, cost per device at scale
Run multiple models on one accelerator without them destroying each other's latency
Diagnose a misbehaving device you cannot SSH into
Before we start About two hours of pre-work, sent when you enrol.
The edge hardware landscape: Jetson, Thor and the alternatives, and how to benchmark for your own workload instead of trusting a datasheet.
Model optimization basics: TensorRT engines, FP16 and INT8 quantization, and what accuracy you actually give up. Getting a model serving locally so you arrive ready to build on it.
Written material and curated links rather than video. This is the part that already exists for free, and live time is too valuable to spend on it. We recap the essentials in the first twenty minutes of Session 1, so nobody is left behind.
Session 1 - Shipping to a fleet Versioning the whole bundle rather than the weights. Provisioning and device identity.
OTA updates for models and containers: the competing patterns (A/B partitions, content-addressable images, container deltas) and what each costs you in storage, bandwidth and recovery.
Canary selection by diversity, not randomly. Staged rollouts, batch sizing, and devices that are simply unreachable. Building a change log you can debug against three weeks later.
Session 2 - Rollback and the physical world Local, automatic rollback: what should trigger it, what should not, and how to keep it idempotent when a device reboots mid-update.
Failure modes that do not exist in a data center: thermal throttling, power loss mid-write, intermittent and metered links, silent degradation.
Serving under real constraints: dynamic batching against hard latency limits, multiple models on one accelerator, MIG partitioning and when it helps. Why an engine compiled for one JetPack version is not valid on another.
Session 3 - Observability without access What to collect on constrained hardware, what to sample, what to drop, and why the usual agents are too heavy for the device.
Detecting inference-quality drift without labels: distribution shift on confidence and per-class rates, and the thresholds worth alerting on.
Device health, thermal behaviour and predicting hardware failure. Remote debugging with no shell. Capacity planning and cost per device. Running the post-incident review when you cannot reproduce the conditions.
Session 4 - Capstone and architecture review First half: deploy a model across a simulated fleet, push an update, detect a regression on a subset of devices, and roll back without touching the devices directly.
Second half: live review of attendees’ own fleet designs. Bring yours - this is the part you cannot get from documentation.
Four live sessions of three hours, one per week, on Zoom, plus about two hours of pre-work before the first session. Sessions are recorded and available afterwards. A shared channel runs for the duration of the cohort and stays open after it ends.Cohort is capped at 15 so there's room for your actual situation, not just the slides.
Hardware is optional. The labs run on a simulated fleet in containers on your own machine, so you can do everything without buying anything.
If you have edge hardware (such as a Jetson) available at work or your own device you can run the exercises on real silicon, and I'll cover the differences that only show up there.
Working knowledge of Linux and containers.
Starts Wednesday, November 4, 2026. Sessions Wednesdays at 6 PM ET on November 4, 11, 18 and December 2 (skipping Thanksgiving week).Founding cohort: $950. Smaller group, lower price, and more of my time per person than later cohorts will have. In exchange I'll ask for detailed feedback and, if it was worth your time, a testimonial.15 seats. Enrolment opens October 1st. Team rates available for three or more seats - email [email protected]
Daniel Gotlieb. I've spent over a decade building and operating Kubernetes and GPU infrastructure in production, and the last several years running AI inference on device fleets deployed in the physical world. I've taught Kubernetes, DevOps and automation to engineering teams since 2016.This is the course I wanted when I started running fleets and couldn't find.
Is this specific to NVIDIA hardware?
The examples use Jetson, but the operational patterns such as rollout, rollback, observability and fleet versioning apply to any edge accelerator.
We run on-prem rather than truly remote. Still relevant?
Yes, for most of it. If you can physically reach every device in ten minutes, the remote-debugging material matters less.
Can my company pay? Yes - invoices available. Email [email protected]
What if I can't make a session live? Everything is recorded, and questions can go in the channel. Most of the value is live, though please come if you can.