//Follow one frame

Smart camera

A camera with its own small computer, on the wall above a factory's loading dock. It watches the dock doors and the apron in front of them, runs a detection model on each frame, keeps a clip when a person steps into the marked zone in front of a door, and drops the rest. When someone asks, it shows the dock live.

Step 01 of 08, on the machine

Capture

A person walks into the marked zone in front of the empty dock door, beside a trailer at the next door. Light from the apron comes through the window and the lens onto the image sensor, which reads the whole picture at once, 40 times a second.

What it costs

92 MB of pixels a second: 1,920 × 1,200 pixels, 40 times a second, if each pixel is kept in one byte. That is about 740 Mbit/s, most of a gigabit link, before anything is compressed.

Derived from the Basler ace acA1920-40gc catalogue row

Why it is hard, and what is at work

Every frame is one exposure. With the sun behind the trailer, the person beside it is in shadow. At night the infrared lights reach only so far across the apron, and rain and fog throw their light back into the lens.

Parts working at this step

Lens, image sensor, and infrared lights
A fixed lens focuses the dock onto a 2.3 megapixel image sensor that reads every pixel at the same instant (a global shutter), so a moving forklift keeps its shape instead of leaning the way a line-by-line sensor draws it. Infrared lights round the lens light the apron at night, when the camera switches to black and white.
Housing, window, and sunshield
A sealed aluminium housing holds the camera, its computer, and its storage behind a flat glass window. A sunshield over the top keeps direct sun off the housing and the window, and rain off the glass.
Wall bracket
A steel arm holds the camera out from the wall above the dock doors, high enough to see over a trailer and the whole apron, tipped down toward it.

Step 02 of 08, on the machine

Detect

The processor scales the frame down and passes it to the detection model on the accelerator. The model returns a box round each thing it recognises, with a label and a score: here a trailer and a person.

What it costs

At most 25 ms for each frame if the model runs on all 40 frames a second. Detectors of this kind usually run at about 10 to 15 frames a second, so a person crossing quickly spends fewer frames in view.

Estimate

Why it is hard, and what is at work

A detector finds what looks like the pictures it learned from. A person half hidden by a trailer, a person pushing a cage that hides their legs, or a reflection on a wet apron each look different, and the model misses them or finds a person where there is none.

Parts working at this step

Processor and AI accelerator
A small computer runs the camera. It takes each frame from the sensor, scales it to the size the model expects, and runs a detection model on an accelerator card beside it. The model puts a box round each person, vehicle, or trailer it finds, with a score for how sure it is.

Step 03 of 08, on the machine

Decide

The camera's own software decides what to keep. A person inside the marked zone in front of a dock door is what the site asked to see, so the camera keeps a clip: a few seconds either side of this frame, taken from the recording of the last minute on its card. A frame with nobody in the zone is dropped.

What it costs

About 43 GB a day to send every frame as a 4 Mbit/s video. About 0.5 GB a day if the camera keeps 100 clips of 10 seconds at the same rate.

Estimate

Why it is hard, and what is at work

The camera cannot send every frame, and what it drops is gone. A loose rule fills the card and the link with forklifts; a tight one drops the near miss the site wanted to see. The site also decides what may leave it at all: the faces of its workers and the number plates of visiting trucks may have to be blurred or cut before a clip goes anywhere.

Parts working at this step

Processor and AI accelerator
A small computer runs the camera. It takes each frame from the sensor, scales it to the size the model expects, and runs a detection model on an accelerator card beside it. The model puts a box round each person, vehicle, or trailer it finds, with a score for how sure it is.
Clip storage
A memory card at the back of the board holds a recording of the last minute or so, and the clips the camera keeps, until they are sent or overwritten.

Step 04 of 08, on the machine

Stream

In the site office, someone asks to see the dock now. The camera's device agent has already dialled out to the server, so the camera needs no inbound port and the office needs no VPN (a private tunnel into the site network). The server sends the camera a command to open a live session. The camera publishes its video over WHIP, a standard way to send video over WebRTC, and the browser watches it over WHEP, its counterpart for watching. The video passes through the server, which sends it on to each person watching. The server records none of it: recording is the device agent's capture, at step 05.

What it costs

About 2 Mbit/s up the camera's link for as long as someone watches, for 640 × 480 pictures at 30 frames a second, the size, rate, and bitrate the solution's demo script sets when it encodes on the processor.

Estimate

Device to Cloud Flywheel solution: Cameras and vision devices

Why it is hard, and what is at work

Live video is the heaviest thing the camera sends, so it runs only while someone watches, and it shares the cable and the uplink with the clips. In the device agent, recording and live view each open the video source on their own. A network camera's stream can be opened twice; a camera on a USB port can be opened by only one program at a time, so recording and live view cannot both run on it.

Parts working at this step

PoE and network link
One Ethernet cable carries power to the camera (Power over Ethernet, PoE) and its data to the site network. A board at the back of the housing takes the power off the cable for the computer and the lights, and the cable runs along the arm into a junction box on the wall.
Processor and AI accelerator
A small computer runs the camera. It takes each frame from the sensor, scales it to the size the model expects, and runs a detection model on an accelerator card beside it. The model puts a box round each person, vehicle, or trailer it finds, with a score for how sure it is.

Where Device to Cloud Flywheel fits: Cameras and vision devices

Cameras in the field, fixed, on a machine, or a browser on a laptop. You want to see them live and pull frames back for training.

Demo

examples/browser-camera/demo.sh

Needs a running stack, a USB camera on the machine that runs it, Go, GStreamer, jq, curl, v4l2-ctl, and ffmpeg or GStreamer's H.264 decoder.

The demo script drives a real USB camera on the machine that runs it. It enrolls the camera as a device, advertises it as a media source, opens a live session, publishes the camera's video over WHIP, pulls it back over WHEP the way a viewer would, and saves one decoded frame as a picture. With --synthetic it publishes a GStreamer test pattern instead and says so on every line. It shows live view only: it records no segments and deploys no model. The solution catalogue lists only GStreamer for a synthetic camera as what the demo needs; this page follows the script. The solution page describes a browser page driven headlessly with a fake camera, and a smart-camera driver whose only transport is a mock; neither is what the script runs. The solution page also calls the server the recording point; in the code the server only relays live video, and recording happens on the device. The solution page also says a rollout pauses when a new detector does worse, a check the rollout engine does not make.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page, punctuation adjusted. What the demo needs is read from its script.

Step 05 of 08, across the fleet

Log

The kept clip waits on the camera's card. Across 1,000 cameras, clips pile up faster than a team can look at them. With capture turned on, Device to Cloud Flywheel's device agent records the camera's video in segments of about six seconds, keeps each on disk with the SHA-256 checksum of its bytes, and goes on recording while the link is down. It uploads every segment it records and has no keep-or-drop rule of its own. The choice made at step 03 lives in the video pipeline you give the agent, or in your own software. An operation you add can write kept clips to a file when the team, or a script of its own, calls it: the agent uploads the file and returns a reference in that command's result, outside the segment list.

What it costs

About 0.5 TB a day across 1,000 cameras if each keeps 100 clips of 10 seconds at about 4 Mbit/s. With capture uploading every segment of a 4 Mbit/s stream, it is about 43 TB a day.

Estimate

Device to Cloud Flywheel solution: Cameras and vision devices

The fleet view shows 1,000 cameras to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

A clip is only useful with the model version, the settings, and the time it was recorded under. A camera whose clock drifts files a near miss under the wrong minute. The code has a capture policy with a setting to upload only on an event, which the agent does not read.

Parts working at this step

Clip storage
A memory card at the back of the board holds a recording of the last minute or so, and the clips the camera keeps, until they are sent or overwritten.
Processor and AI accelerator
A small computer runs the camera. It takes each frame from the sensor, scales it to the size the model expects, and runs a detection model on an accelerator card beside it. The model puts a box round each person, vehicle, or trailer it finds, with a score for how sure it is.

Where Device to Cloud Flywheel fits: Cameras and vision devices

Cameras in the field, fixed, on a machine, or a browser on a laptop. You want to see them live and pull frames back for training.

Demo

examples/browser-camera/demo.sh

Needs a running stack, a USB camera on the machine that runs it, Go, GStreamer, jq, curl, v4l2-ctl, and ffmpeg or GStreamer's H.264 decoder.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page, punctuation adjusted. What the demo needs is read from its script.

Step 06 of 08, across the fleet

Return

Each segment goes up under the camera's own device credential: its bytes to the blob store first, then its times, size, and checksum to the server. A segment the server already holds from that camera, by checksum, is recorded once. The device agent dials out, so the camera's network needs no inbound port. The team lists a camera's segments on its Operations tab in the console, or through the API, and downloads the ones it wants with dcfctl blob download.

What it costs

About 7 minutes to send a day's 0.5 GB of clips if the camera gets about 10 Mbit/s of the site's uplink.

Estimate

Device to Cloud Flywheel solution: Cameras and vision devices

The fleet view shows 1,000 cameras to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

The camera shares the site's uplink with the office, so clips go up slowly, at night, or both. What leaves the site is the site's decision. Device to Cloud Flywheel moves the bytes it is given and blurs nothing. The reference camera driver takes named privacy zones as desired state, and its only transport is a mock, so blurring on a real camera is your own software's job.

Parts working at this step

PoE and network link
One Ethernet cable carries power to the camera (Power over Ethernet, PoE) and its data to the site network. A board at the back of the housing takes the power off the cable for the computer and the lights, and the cable runs along the arm into a junction box on the wall.

Where Device to Cloud Flywheel fits: Cameras and vision devices

Cameras in the field, fixed, on a machine, or a browser on a laptop. You want to see them live and pull frames back for training.

Demo

examples/browser-camera/demo.sh

Needs a running stack, a USB camera on the machine that runs it, Go, GStreamer, jq, curl, v4l2-ctl, and ffmpeg or GStreamer's H.264 decoder.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page, punctuation adjusted. What the demo needs is read from its script.

Step 07 of 08, across the fleet

Learn

The team sees that the detector at a newly added site misses people in dark overalls beside dark trailers. From that site's returned clips they label a few thousand frames and train a new detector with them. In Device to Cloud Flywheel, a segment comes back by its blob id, and a trained model goes up as a file in the blob store with its checksum. Device to Cloud Flywheel chooses no clips, labels nothing, and schedules no training.

What it costs

About 12 hours of labelling for 2,000 frames if each frame needs about 3 boxes and each box takes about 7 seconds.

Estimate

Device to Cloud Flywheel solution: Training a policy and shipping it to the fleet

A second product, Device to Cloud Engineer, is an engineer agent that writes the device code a machine like this runs. It is in development, and there is nothing to try yet.

The fleet view shows 1,000 cameras to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

A detector that sees the new site better can see an old site worse. The team has to test it on clips from every kind of site, by day and by night, and know which model and settings each clip was recorded with, or they compare pictures that differ in more than the one thing they changed.

Where Device to Cloud Flywheel fits: Training a policy and shipping it to the fleet

Field data coming back from machines, and a model you want to improve and redeploy without a bespoke pipeline each time. It runs across the other solutions rather than describing one kind of device.

Demo

embodiments/train/demo.sh

Needs a running stack, Go, uv, jq, and curl.

The demo collects data from a simulated excavator (a MuJoCo simulation), trains a policy on it, and ships it to a two-machine fleet. No camera is in it. The solution catalogue says the training and deploy halves do not depend on the hardware: a detector travels the same way, as a file in the blob store named in desired state by its checksum.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page, wording adjusted. What the demo needs is read from its script.

Step 08 of 08, across the fleet

Ship

The new detector goes out as desired state: a model file in the blob store, named by its id and SHA-256 checksum, with the pipeline and its zones. The reference smart-camera driver pulls the file, checks the checksum, and refuses a file that does not match. That driver runs against a mock camera, and the repository has no transport for a real one, so on a real camera your own code loads the model, the way the training demo's machine loads its policy. A rollout sends the change to one group first, here 4 of the 5 cameras at this one site, and releases the next group only when someone advances it and every camera in the groups before has the change in its desired state. It can be paused, resumed, and rolled back.

What it costs

About 10 GB to move if the detector is about 10 MB and goes to 1,000 cameras, over site uplinks shared with everything else.

Estimate

Device to Cloud Flywheel solution: Training a policy and shipping it to the fleet

The fleet view shows 1,000 cameras to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

A camera that swaps its model stops detecting while the new one loads, so 1,000 cameras updated at once go blind together. A staged rollout limits that to one group at a time, and a camera that keeps its old model running until the new one has loaded does not go blind at all. The solution page says a rollout pauses when a new detector does worse; the rollout engine has no such check, so whether the new detector is better at each site is the team's call, from its own measures. The rollout cuts every group to the same size, so groups of 4 cameras across 1,000 take 250 advances.

Parts working at this step

Processor and AI accelerator
A small computer runs the camera. It takes each frame from the sensor, scales it to the size the model expects, and runs a detection model on an accelerator card beside it. The model puts a box round each person, vehicle, or trailer it finds, with a score for how sure it is.
PoE and network link
One Ethernet cable carries power to the camera (Power over Ethernet, PoE) and its data to the site network. A board at the back of the housing takes the power off the cable for the computer and the lights, and the cable runs along the arm into a junction box on the wall.

Where Device to Cloud Flywheel fits: Training a policy and shipping it to the fleet

Field data coming back from machines, and a model you want to improve and redeploy without a bespoke pipeline each time. It runs across the other solutions rather than describing one kind of device.

Demo

embodiments/train/demo.sh

Needs a running stack, Go, uv, jq, and curl.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page, wording adjusted. What the demo needs is read from its script.

And back

Back to 01 Capture, with the new detector

The next person to cross the dock is captured and goes through the same steps, this time judged by the new detector. The clips it keeps from the new site show whether it now finds people in dark overalls beside a dark trailer.

On the machine

Across the fleet

And back

//Every part in the cycle

Each subsystem has its own way to fail.

The catalogue examples are representative parts taken from vendor datasheets and product pages, each linked to its source. They show the kind of component each subsystem uses and the numbers that matter for it. They are not a bill of materials, and nobody has checked that they work together.

Part	What it does	Why it is hard	Catalogue examples
Housing, window, and sunshieldWorking at 01 capture	What it does. A sealed aluminium housing holds the camera, its computer, and its storage behind a flat glass window. A sunshield over the top keeps direct sun off the housing and the window, and rain off the glass.	Why it is hard. A box on a sunny wall runs far hotter than the air round it, and the processor inside adds its own heat, which has to leave through the metal with no fan to clog. The window fogs on a cold morning and collects dust and spider webs, and a drop of water on the glass blurs the picture more than anything the model does.	Catalogue examples The catalogue has no enclosure row.
Lens, image sensor, and infrared lightsWorking at 01 capture	What it does. A fixed lens focuses the dock onto a 2.3 megapixel image sensor that reads every pixel at the same instant (a global shutter), so a moving forklift keeps its shape instead of leaning the way a line-by-line sensor draws it. Infrared lights round the lens light the apron at night, when the camera switches to black and white.	Why it is hard. The light at a dock door changes all day: sun on the apron, a dark trailer, headlights at night, and a low evening sun straight into the lens. A sensor like this one sets one exposure for the whole picture, so a person in shadow beside a sunlit trailer can come out too dark to detect. Under infrared, clothes and paint look different from the daytime colour pictures a model learned from.	Catalogue examples Basler ace acA1920-40gc resolution 1,920 × 1,200 · frame rate 40 Hz · sensor size 1/1.2 in The Basler ace is a machine-vision camera that takes its power over its network cable. It stands in here for the sensor: a smart camera puts a sensor of about this size and resolution behind its own lens, often one that combines two exposures of each frame to cope with bright and dark in one picture.
Processor and AI acceleratorWorking at 02 detect, 03 decide, 04 stream, 05 log, 08 ship	What it does. A small computer runs the camera. It takes each frame from the sensor, scales it to the size the model expects, and runs a detection model on an accelerator card beside it. The model puts a box round each person, vehicle, or trailer it finds, with a score for how sure it is.	Why it is hard. The model has to keep up with the camera inside the power the cable carries and the heat the housing sheds. A model trained on one site learns that site: its paint, its trailers, its light. At a new site it misses people in overalls of another colour, or finds a person in a stack of tyres.	Catalogue examples Hailo-8L M.2 AI throughput 13 trillion operations/s · length 80 mm · width 22 mm Raspberry Pi 5 (8 GB) core 4 · core clock 2,400 MHz · memory 8 GB A Raspberry Pi 5 with a Hailo-8L accelerator on an M.2 card is one way to build it. Commercial smart cameras put the processor and the accelerator in one chip.
Clip storageWorking at 03 decide, 05 log	What it does. A memory card at the back of the board holds a recording of the last minute or so, and the clips the camera keeps, until they are sent or overwritten.	Why it is hard. Memory cards wear out when they are written all day, and a card that fails loses the clip that mattered. When the link is down for a day and the card fills, the camera has to choose what to overwrite.	Catalogue examples The catalogue has no storage row.
PoE and network linkWorking at 04 stream, 06 return, 08 ship	What it does. One Ethernet cable carries power to the camera (Power over Ethernet, PoE) and its data to the site network. A board at the back of the housing takes the power off the cable for the computer and the lights, and the cable runs along the arm into a junction box on the wall.	Why it is hard. The cable sets the power budget: a PoE+ port delivers about 25 W at the camera, and the computer, the accelerator, the infrared lights, and a heater in winter all share it. The site network belongs to the site, whose IT team rarely opens an inbound port or gives a camera a public address. A site with no wired link sends over a cellular modem, which is paid for by the gigabyte.	Catalogue examples Sierra Wireless AirPrime MC7455 downlink rate max 300 Mbit/s · uplink rate max 50 Mbit/s · transmit power 23 dBm The catalogue has no PoE switch or injector row. The LTE (4G cellular) modem listed is what a camera uses where no cable reaches; its 50 Mbit/s uplink is a best case, shared with everyone else on the same cell.
Wall bracketWorking at 01 capture	What it does. A steel arm holds the camera out from the wall above the dock doors, high enough to see over a trailer and the whole apron, tipped down toward it.	Why it is hard. Wind and passing trucks shake a camera on an arm, and a shaking picture blurs and moves every box the model draws. A ladder knocks it, and every zone drawn on the picture is in the wrong place. At 5 m up, fixing it takes a lift and a closed dock door.	Catalogue examples The catalogue has no bracket or fastener row.

The drawing is illustrative, not a validated design. Costs marked estimate are estimates; the others are worked out from the catalogue part named under them.

//Where to start

The fleet half of this machine, as Device to Cloud Flywheel solutions.

Cameras and vision devices
Cameras in the field, fixed, on a machine, or a browser on a laptop. You want to see them live and pull frames back for training.
docs/solutions/cameras.md (opens when the code is published)
In the cycle at 04 stream, 05 log, 06 return. Quoted from the solution page, punctuation adjusted.
Training a policy and shipping it to the fleet
Field data coming back from machines, and a model you want to improve and redeploy without a bespoke pipeline each time. It runs across the other solutions rather than describing one kind of device.
docs/solutions/train-and-deploy.md (opens when the code is published)
In the cycle at 07 learn, 08 ship. Quoted from the solution page, wording adjusted.

Neither of the two solution pages has a section that separates measured results from claims, so nothing on this page is quoted as measured.

These pages live in the Device to Cloud Flywheel repository, which is private while the first release lands. Write to hello@devicetocloud.ai for access, and we will tell you when it opens.

//Where we help

Building one? Device to Cloud Flywheel is the software for the fleet half of this loop, from the first camera on.

Log, Return, Learn, and Ship: the steps after the machine works. The flywheel moves the data and the releases for every machine you have; what to train on and when to ship stay your team's call.

Request early access See the flywheel