//Follow one bucket

Autonomous excavator

A tracked excavator refitted to dig on its own. A lidar maps the pit, a computer plans each bucket and drives the machine's own hydraulics, and when the machine is stuck a person far away takes control over live video. The cab stays, so a person can still drive it from the seat.

Step 01 of 09, on the machine

See

The lidar on the mast sweeps the trench and the ground around it, and the computer builds a 3D map of where the soil is now. The bucket this cycle follows starts here, as a patch of soil on the trench floor.

What it costs

A new sweep every 100 ms, because the listed lidar turns 10 times a second. The arm keeps moving while each sweep is taken, so part of every sweep shows the arm, blurred, where the soil should be.

Derived from the Ouster OS1-64 catalogue row

Why it is hard, and what is at work

Each bucket changes the shape of the pit, dust rises on every pass, and the arm crosses the lidar's view on every cycle. The map of the trench floor is partly hidden and partly out of date on every pass.

Parts working at this step

Perception mast
A lidar on a mast over the cab, tilted down toward the digging, measures the pit and the ground around the machine in 3D, a second lidar watches behind it, and cameras beside the first see what a lidar cannot, such as a person in a high-visibility vest or a paint mark on the ground.

Step 02 of 09, on the machine

Plan

The computer compares the map with the design, the depth and width the job calls for, and plans the next cut: where the teeth go in, how deep they go, and how far the bucket drags before it curls.

What it costs

15 to 60 W for an embedded module like the one listed, depending on its power mode. More power buys a faster plan and more heat in a sealed box.

Derived from the NVIDIA Jetson AGX Orin 64GB catalogue row

Why it is hard, and what is at work

The plan can only guess at the soil. It has to leave room to stop short in hard ground and to fill up early in soft ground, and it must never cut below the design depth, because soil taken out past the design has to be put back and compacted.

Parts working at this step

Compute and CAN bus
A sealed computer behind the cab turns the lidar map into a plan for each bucket and sends valve commands over a CAN bus to the machine's own controller, which still drives the valves.

Step 03 of 09, on the machine

Dig

The valves send oil to the stick and bucket cylinders. The teeth go into the soil, the stick drags the bucket toward the machine, and the bucket curls up full. Pressure in the cylinder lines tells the computer how hard the soil pushes back.

What it costs

At most about 47 kW of oil power through one valve of the listed size, with both of its limits reached at once: 350 bar at 80 litres a minute.

Derived from the Bosch Rexroth 4WE6 directional spool valve catalogue row

Why it is hard, and what is at work

Soil is unknown until the bucket is in it. The hydraulics answer late and not in proportion to the command, so the computer corrects the cut as it goes, from the pressures and the joint angles, without stalling the engine or lifting the tracks.

Parts working at this step

Boom, stick, and bucket
The boom lifts, the stick reaches in and out, and the bucket curls, each on a steel pin. Together they reach several metres below the tracks and high enough to clear the side of a truck.
Hydraulic pump, valves, and cylinders
An engine-driven pump feeds a block of valves, and each valve sends oil to a cylinder or a motor: two cylinders lift the boom, one moves the stick, one curls the bucket, a motor turns the house, and two more drive the tracks.
Joint angle and pressure sensors
An angle sensor on each pin, tilt sensors on the boom and the stick, and pressure sensors on the cylinder lines tell the computer where the bucket is and how hard it is pushing.

Step 04 of 09, on the machine

Swing

The boom lifts the full bucket out of the trench, and the swing motor turns the house a quarter turn to the right, over the truck. The lidars watch the circle the counterweight and the bucket sweep through.

What it costs

About 8 s of a loading cycle of about 20 s goes to swinging out to the truck and back.

Estimate

Why it is hard, and what is at work

A person inside the swing circle can be caught between the counterweight and whatever stands behind it. The house carries tonnes of momentum, so it has to start braking well before the bucket reaches the truck, and it cannot stop hard without spilling the load.

Parts working at this step

Hydraulic pump, valves, and cylinders
An engine-driven pump feeds a block of valves, and each valve sends oil to a cylinder or a motor: two cylinders lift the boom, one moves the stick, one curls the bucket, a motor turns the house, and two more drive the tracks.
Stops and swing zone
Emergency stop buttons on the machine and a radio stop that a person on the site carries cut hydraulic power, and a warning beacon warns anyone nearby. The lidar watches the circle the house and the arm swing through.
Perception mast
A lidar on a mast over the cab, tilted down toward the digging, measures the pit and the ground around the machine in 3D, a second lidar watches behind it, and cameras beside the first see what a lidar cannot, such as a person in a high-visibility vest or a paint mark on the ground.

Step 05 of 09, on the machine

Dump

Over the truck, the bucket uncurls and the soil falls into the bed. The bucket this cycle follows is now part of a truckload.

What it costs

About 10 buckets to fill the truck, if the bucket holds about 1 m³ and the truck carries about 10 m³.

Estimate

Why it is hard, and what is at work

The bucket has to clear the truck's side, spread the load along the bed rather than drop it all in one place, and never pass over the truck's cab. Sticky soil stays in the bucket and has to be shaken out.

Parts working at this step

Boom, stick, and bucket
The boom lifts, the stick reaches in and out, and the bucket curls, each on a steel pin. Together they reach several metres below the tracks and high enough to clear the side of a truck.
Hydraulic pump, valves, and cylinders
An engine-driven pump feeds a block of valves, and each valve sends oil to a cylinder or a motor: two cylinders lift the boom, one moves the stick, one curls the bucket, a motor turns the house, and two more drive the tracks.

Step 06 of 09, on the machine

Take over

On the next pass the teeth stop against a buried pipe that no map showed. The machine holds still and calls a remote operator, who watches live video from a desk far away and moves each joint while holding down a deadman control. If the link drops, the machine is told to hold every joint still rather than move to a new position.

What it costs

At best 50 Mbit/s up for the video, the ceiling of the LTE modem listed, and only with carrier aggregation. That is shared with everything else the machine sends. A site at the edge of coverage gets far less, and the picture the operator steers by is the first thing to suffer.

Derived from the Sierra Wireless AirPrime MC7455 catalogue row

Device to Cloud Flywheel solution: Heavy equipment, simulated then real

Why it is hard, and what is at work

A person must be able to take over from far away without the machine ever moving on a lost or late link. Video over a cellular link lags and drops, so each command has to stay inside what that joint may safely do, and the machine has to stop on its own when commands stop arriving.

Parts working at this step

Radio link for take-over and video
A cellular modem and antennas on the hood connect the machine to the people who run it and to a remote operator, carrying live video out and joint commands in.
Compute and CAN bus
A sealed computer behind the cab turns the lidar map into a plan for each bucket and sends valve commands over a CAN bus to the machine's own controller, which still drives the valves.
Stops and swing zone
Emergency stop buttons on the machine and a radio stop that a person on the site carries cut hydraulic power, and a warning beacon warns anyone nearby. The lidar watches the circle the house and the arm swing through.

Where Device to Cloud Flywheel fits: Heavy equipment, simulated then real

Machines too expensive to experiment on. You want to train against a simulator and deploy the result to the real thing through the same path.

Demo

embodiments/excavator/demo.sh

Needs a running stack, uv for the MuJoCo sim.

The demo drives a MuJoCo model of an excavator arm. No physical excavator has been through this loop. The measured run covers collecting, training, and deploying a policy; the take-over by a remote operator is described on the solution page but is not part of that run.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page.

Step 07 of 09, across the fleet

Log

The machine keeps a record of this bucket: the joint angles and cylinder pressures through the dig, the lidar sweeps before and after, and the take-over video. Engine speed and hydraulic pressure come off the machine's own CAN bus. Across 1,000 machines, records like these pile up faster than any team can look at them.

What it costs

655,360 lidar points a second from the listed lidar on its own: 64 channels, 1,024 points a turn, 10 turns a second.

Derived from the Ouster OS1-64 catalogue row

Device to Cloud Flywheel solution: Photoreal simulation with Isaac

The fleet view shows 1,000 machines to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

The joint and pressure streams are small; the lidar sweeps and the video are not, and a site's link is weak. The machine writes each record to its own disk first, then uploads it as a file to a separate file store. The server that coordinates the machines handles only a short reference to each file, its id and checksum, and never the bytes.

Parts working at this step

Compute and CAN bus
A sealed computer behind the cab turns the lidar map into a plan for each bucket and sends valve commands over a CAN bus to the machine's own controller, which still drives the valves.
Joint angle and pressure sensors
An angle sensor on each pin, tilt sensors on the boom and the stick, and pressure sensors on the cylinder lines tell the computer where the bucket is and how hard it is pushing.

Where Device to Cloud Flywheel fits: Photoreal simulation with Isaac

An Isaac Sim scene and a need for training imagery that looks like what the machine will actually see.

Demo

embodiments/excavator-isaac/demo.sh

Needs Isaac Sim on an RTX host. NOT runnable on an ordinary machine; the demo says so and exits.

The demo renders a simulated excavator in Isaac Sim and needs an RTX host. The solution page's ingest steps run without Isaac: a stand-in program produces the joint stream and the CAN health signals, and they reach a live server. No physical excavator has sent either.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page.

Step 08 of 09, across the fleet

Learn

The fleet's records, this bucket among them, are the raw material for the next digging policy, the program that turns a plan into valve commands. People pick the runs worth learning from, the stalls and the take-overs among them, and send them to a training machine. Device to Cloud Flywheel moves each recorded run as a file checked by its checksum. When you issue the training command, it hands the job to your own training machine and brings the new policy back as a file. It doesn't choose the runs, and it doesn't start training on its own.

What it costs

About 1.5 million buckets a day across 1,000 machines, if each loads about 150 an hour over a ten-hour shift.

Estimate

Device to Cloud Flywheel solution: Heavy equipment, simulated then real

A second product, Device to Cloud Engineer, is an engineer agent that writes the device code a machine like this runs. It is in development, and there is nothing to try yet.

The fleet view shows 1,000 machines to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

The runs worth training on are rare: a new soil, a buried pipe, a take-over. Real machines are too expensive to experiment on, so most practice happens in a simulator, and a policy that digs well in simulated soil can still stall in real clay.

Where Device to Cloud Flywheel fits: Heavy equipment, simulated then real

Machines too expensive to experiment on. You want to train against a simulator and deploy the result to the real thing through the same path.

Demo

embodiments/excavator/demo.sh

Needs a running stack, uv for the MuJoCo sim.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page.

Step 09 of 09, across the fleet

Ship

The new policy goes out as a checkpoint file. The machine fetches it and refuses it if its SHA-256 checksum does not match. In a fleet this size, the team gives it to a first group of machines, compares their stalls and take-overs with the rest, and then advances the rollout or pauses it and rolls back. On this machine, the computer digs the next bucket with the new policy.

What it costs

About 20 GB of downloads for one release, if the policy file is about 20 MB and goes to 1,000 machines.

Estimate

Device to Cloud Flywheel solution: Heavy equipment, simulated then real

The fleet view shows 1,000 machines to show the problem at that size; Device to Cloud Flywheel has been run on fleets of single digits to low tens of devices.

Why it is hard, and what is at work

A policy that is better on average can be worse in one soil. Judging it needs a measure the machine does not grade for itself: the solution's own check reads where the simulated soil ended up instead of trusting the simulator's success counter. On a real site, the team decides what that measure is.

Parts working at this step

Compute and CAN bus
A sealed computer behind the cab turns the lidar map into a plan for each bucket and sends valve commands over a CAN bus to the machine's own controller, which still drives the valves.

Where Device to Cloud Flywheel fits: Heavy equipment, simulated then real

Machines too expensive to experiment on. You want to train against a simulator and deploy the result to the real thing through the same path.

Demo

embodiments/excavator/demo.sh

Needs a running stack, uv for the MuJoCo sim.

The repository is private while the first release lands, so this demo cannot be run from this page. Write to hello@devicetocloud.ai and we will tell you when it opens.

From the solution page.

And back

Back to 01 See, with the new model

The next bucket goes through the same steps. This time the computer digs it with the new policy, and what the machine records about it feeds the policy after that.

On the machine

Across the fleet

And back

//Every part in the cycle

Each subsystem has its own way to fail.

The catalogue examples are representative parts taken from vendor datasheets and product pages, each linked to its source. They show the kind of component each subsystem uses and the numbers that matter for it. They are not a bill of materials, and nobody has checked that they work together.

Part	What it does	Why it is hard	Catalogue examples
Tracks and slew ring	What it does. Two steel crawler tracks carry the machine over soft and broken ground. A slew ring on top of the track frame carries the upper structure, so the house, the cab, and the arm turn all the way round while the tracks stay put.	Why it is hard. Everything that turns with the house has to cross the slew ring: oil through a rotary joint, wiring through a slip ring. Next to the trench it is digging, the ground under the near track can give way, so the machine has to know where the edge is and keep its tracks back from it.	Catalogue examples Moog AC6438 through-bore slip ring through bore 0.5 in · circuit 24 · current per circuit 5 A The slip ring listed is an example of how wiring crosses a joint that turns without limit; the one on a machine this size is larger and sealed. The catalogue has no crawler track or slew bearing of this size.
Upper house, engine, and cab	What it does. The upper structure carries the diesel engine, the counterweight that balances a full bucket at reach, the fuel and hydraulic oil tanks, and the cab. The cab stays, so a person can still drive the machine from the seat.	Why it is hard. The engine sets how much hydraulic power there is, and it runs near full load for a whole shift in dust and heat. After the refit the machine has two drivers, a person in the seat and the computer, and it has to be clear at every moment which one is in control.	Catalogue examples The catalogue has no diesel engine, counterweight, or cab row.
Boom, stick, and bucketWorking at 03 dig, 05 dump	What it does. The boom lifts, the stick reaches in and out, and the bucket curls, each on a steel pin. Together they reach several metres below the tracks and high enough to clear the side of a truck.	Why it is hard. Soil is unknown until the bucket is in it. Clay, wet sand, roots, and buried rock each push back differently, and a motion that fills the bucket in one pass stalls it in the next. The pins wear, so the teeth end up centimetres from where the joint angles say they are.	Catalogue examples The catalogue has no boom, stick, or bucket row. They are welded steel, built by the machine's maker.
Hydraulic pump, valves, and cylindersWorking at 03 dig, 04 swing, 05 dump	What it does. An engine-driven pump feeds a block of valves, and each valve sends oil to a cylinder or a motor: two cylinders lift the boom, one moves the stick, one curls the bucket, a motor turns the house, and two more drive the tracks.	Why it is hard. Hydraulics are slow and nonlinear. A valve does nothing until its spool passes a dead band, flow changes with oil temperature and with how many motions share the pump, and a raised, loaded arm creeps down while its valve is closed, as oil leaks past the spool. A controller tuned on a cold morning overshoots by the afternoon.	Catalogue examples Bosch Rexroth 4WE6 directional spool valve max operating pressure 350 bar · coil voltage 24 V The valve listed is a small on-off directional valve, an example of the valve family. An excavator's main control valve is a larger block of proportional spools, usually moved by pilot pressure rather than directly by a solenoid.
Perception mastWorking at 01 see, 04 swing	What it does. A lidar on a mast over the cab, tilted down toward the digging, measures the pit and the ground around the machine in 3D, a second lidar watches behind it, and cameras beside the first see what a lidar cannot, such as a person in a high-visibility vest or a paint mark on the ground.	Why it is hard. The arm crosses the lidar's view on every cycle, dust rises from every bucket, and the ground changes shape each pass, so the map of the pit goes out of date almost as soon as it is made. The mast shakes with the engine and with every swing.	Catalogue examples Ouster OS1-64 range 80pct 170 m · range 10pct 90 m · range min 0.5 m Livox Mid-360 fov horizontal 360 ° · fov vertical 59 ° · range min 0.1 m Basler ace acA1920-40gc resolution 1,920 × 1,200 · frame rate 40 Hz · sensor size 1/1.2 in
Joint angle and pressure sensorsWorking at 03 dig, 07 log	What it does. An angle sensor on each pin, tilt sensors on the boom and the stick, and pressure sensors on the cylinder lines tell the computer where the bucket is and how hard it is pushing.	Why it is hard. The sensors sit where the digging is: on pins that flex, in mud, under blows from rock. Pressure is the only way the machine feels the soil, and it mixes the soil's resistance with the weight of the arm and of the load.	Catalogue examples ams-OSRAM AS5048A resolution 14 bit · resolution 0.0219 ° · fov 360 ° Bosch Sensortec BMI088 accel range max 24 g · gyro range max 2,000 °/s · supply voltage 2.4 to 3.6 V The angle and motion chips listed are examples of what sits inside such sensors; on a machine they come in sealed, rugged housings. The catalogue has no hydraulic pressure sensor row.
Compute and CAN busWorking at 02 plan, 06 take over, 07 log, 09 ship	What it does. A sealed computer behind the cab turns the lidar map into a plan for each bucket and sends valve commands over a CAN bus to the machine's own controller, which still drives the valves.	Why it is hard. The machine's own controller was built for a person's joysticks, and it may accept outside commands only through the maker's interface, or not at all. A plan has to leave margin for hydraulics that answer late, and the computer has to stop the machine if it falls behind.	Catalogue examples OnLogic Karbon 700-SE length 242 mm · width 240 mm · height 82 mm NVIDIA Jetson AGX Orin 64GB AI throughput 275 trillion operations/s · memory 64 GB · memory bandwidth 204.8 GB/s Texas Instruments TCAN3403-Q1 supply voltage 1.7 to 3.6 V · data rate max 8 Mbit/s · bus fault voltage 58 V The two computers are examples of a rugged vehicle computer and an embedded module for perception. The CAN transceiver is the chip that connects a computer to the bus.
Radio link for take-over and videoWorking at 06 take over	What it does. A cellular modem and antennas on the hood connect the machine to the people who run it and to a remote operator, carrying live video out and joint commands in.	Why it is hard. Coverage in a pit or behind a spoil heap comes and goes as the site changes. Video that stalls for half a second is long enough to hit something, so the operator has to see how old the picture is, and the machine has to stop on its own when the link drops.	Catalogue examples Sierra Wireless AirPrime MC7455 downlink rate max 300 Mbit/s · uplink rate max 50 Mbit/s · transmit power 23 dBm Quectel RM500Q-GL length 52 mm · width 30 mm · height 2.3 mm
Stops and swing zoneWorking at 04 swing, 06 take over	What it does. Emergency stop buttons on the machine and a radio stop that a person on the site carries cut hydraulic power, and a warning beacon warns anyone nearby. The lidar watches the circle the house and the arm swing through.	Why it is hard. A person inside the swing circle can be crushed between the counterweight and whatever stands behind it. Every stop has to cut hydraulic power in hardware without waiting on the computer, and a stopped machine has to hold the bucket where it is rather than let it drop.	Catalogue examples The catalogue has no safety-rated part. A real remote stop is a certified radio system wired to a safety relay that removes the pilot pressure the valves need to move.

The drawing is illustrative, not a validated design. Costs marked estimate are estimates; the others are worked out from the catalogue part named under them.

//Where to start

The fleet half of this machine, as Device to Cloud Flywheel solutions.

Heavy equipment, simulated then real
Machines too expensive to experiment on. You want to train against a simulator and deploy the result to the real thing through the same path.
Measured. e2e/uat_flywheel_showcase.sh runs the whole loop against a real dcf-server: sign up, enroll the excavator, collect-demo, enroll a trainer, train-now, load-policy, run-policy. It then fetches the deployed checkpoint's bytes back out of the blob store and grades those bytes, so the verdict is on what was actually deployed rather than on a local copy. On the run recorded in the local, git-ignored e2e/FLYWHEEL_ISAAC_LOG.md the script appends to (so it is not in a fresh clone), a policy cloned from clean demonstrations alone moved 4 of the 6 dirt bodies into the truck; the deployed policy, cloned from the same demonstrations plus disturbance-augmented ones, moved 6 of 6. A control trained on 9 clean demonstrations, matching the augmented set's count, averaged 4.5 of 6, so the gain came from the augmentation rather than from more data. A second organization's operator was refused on the device, its telemetry, its commands, and the checkpoint blob.
A claim. The machine in that run is a MuJoCo model. No physical excavator has been through this loop. The hardware capstone in tools/rc-excavator/ is written and has never been flashed, and docs/demo/GUIDE.md marks it pending-hardware. The photoreal Isaac scene is a separate high-fidelity half with its own gate; the trained policy has not been rendered inside it.
docs/solutions/heavy-equipment.md (opens when the code is published)
In the cycle at 06 take over, 08 learn, 09 ship. Quoted from the solution page, punctuation adjusted.
Photoreal simulation with Isaac
An Isaac Sim scene and a need for training imagery that looks like what the machine will actually see.
docs/solutions/sim-rendering.md (opens when the code is published)
In the cycle at 07 log.

These pages live in the Device to Cloud Flywheel repository, which is private while the first release lands. Write to hello@devicetocloud.ai for access, and we will tell you when it opens.

//Where we help

Building one? Device to Cloud Flywheel is the software for the fleet half of this loop, from the first machine on.

Log, Learn, and Ship: the steps after the machine works. The flywheel moves the data and the releases for every machine you have; what to train on and when to ship stay your team's call.

Request early access See the flywheel