A field guide to learned-policy robot infrastructure

Start with the loop, not the checkpoint

A learned robot policy is one component in a closed loop. The runtime reads an observation—usually camera images plus robot state—maps it into the model’s schema, asks the policy for one or more future actions, executes part of that prediction, and observes again.

Many modern policies predict an action chunk: a short sequence of future joint, pose, or gripper commands. The control loop consumes that sequence at a fixed rate. Deployment infrastructure owns everything around the neural network: synchronization, queues, action selection, safety checks, hardware dispatch, and evidence about what actually happened.

Deployment means recreating the training contract

The model learned under a specific set of assumptions. Two policies can both accept three images and a state vector, yet disagree about camera roles, RGB versus BGR, field order, units, prompt wording, timestamps, and whether an action is absolute or relative. A correctly shaped output tensor containing no NaN or infinity proves only that the code path ran.

Treat those assumptions as an executable runtime contract. It should declare observation keys and shapes, units, camera roles, control rate, action horizon and semantics, task vocabulary, preprocessing, and normalization assets. The dataset and training transforms are usually the most reliable evidence for that contract.

Make the plain policy work before making it smooth

First prove the shortest path: a fresh observation enters the adapter, the service returns finite actions, their meaning is correct, and one supervised command reaches the intended actuator. Jittery motion in the right direction is a useful baseline. Smooth motion from the wrong camera or into the wrong joint is not.

Do not begin with asynchronous scheduling, filters, splines, automatic retries, or relaxed limits. Each extra mechanism changes the evidence. Add them one at a time after the unmodified policy path works.

1. Verify camera identity, color, orientation, resolution, and cadence.
2. Verify state order and units against a physical movement.
3. Verify action order, units, and absolute/delta semantics.
4. Measure raw action size and inference latency.
5. Run a supervised, bounded hardware canary.

Separate replaceable computation from hardware ownership

The adapter converts robot observations into the policy schema. The policy service may then be replaced, restarted, or moved to another machine without changing the robot driver. Its output is still only a candidate: a router chooses among model, expert, and hold commands; a final boundary validates the selected command; exactly one process owns the hardware SDK and sends it.

This separation gives every failure a home. Schema errors belong at the adapter, model latency at the service boundary, arbitration at the router, and device state at the hardware owner.

Async inference is a timing problem before it is a blending problem

With synchronous execution, the robot may empty the current action chunk and stop while the next one is being generated. Asynchronous inference avoids that pause by requesting the next chunk early and letting control continue from the existing queue.

A policy may predict a horizon of H rows while the controller deliberately queues only an execution prefix E ≤ H. Trigger depth and queue accounting must use E, not the model's full horizon: an unused prediction tail is not latency budget unless it is actually eligible for execution.

The reply is based on an older observation. If inference took several control steps, its first rows now refer to time that has already passed. The runtime therefore records the execution counter at submission, measures how many rows were consumed while the request was in flight, adds any observation age, and removes that elapsed prefix from the reply.

Request timing should also come from measurements. Convert a recent high-percentile end-to-end latency into control steps, add a small scheduling margin, and request while at least that many executable rows remain. Re-measure when the model, accelerator, image transport, compiler, or network changes.

Preserve and transition are not latency estimates. Preserve deliberately keeps a few still-valid rows from the old plan after the reply arrives. Transition blends the following aligned rows. In no-blend mode the runtime still prefetches and drops stale rows; it simply switches directly to the new future plan.

Match the smoother to the motion artifact

“The robot is jerky” is an observation, not a diagnosis. Plot the raw command and feedback streams. Determine whether the discontinuity occurs inside a chunk, only at chunk boundaries, or because the requested motion exceeds what the controller can track. Then compare the same statistics with the demonstrations.

Low-pass filtering handles noise inside the chunk

The filter must carry state across ordinary chunk boundaries; resetting there recreates the seam it was meant to suppress. Reset it from fresh measured state after episode start, takeover, hold, or feedback loss. Apply it only to fields whose timing can tolerate lag—gripper events often should bypass it.

A fixed target is not permanently reduced: repeated updates converge toward it. The practical problem is that a finite action chunk keeps moving the target, so aggressive filtering may always trail the policy. Measure command-to-feedback lag instead of choosing α by appearance.

Splines solve a different problem

Use cubic, quintic, or minimum-jerk interpolation when a controller receives sparse waypoints with an explicit travel time—for example a preparation pose, homing move, or transition between dataset-sampled starts. A spline creates a dense position/velocity profile between known endpoints. It is usually the wrong tool for a rapidly replanned stream because each new chunk changes the endpoint before the previous curve finishes.

Rate and acceleration limits remain a final safety boundary, not a smoother. If they clip frequently, report the amount and fix the upstream action scale, cadence, or contract rather than quietly treating the clipped trajectory as normal.

elapsed-time alignment → optional seam blend → causal low-pass → rate/safety validation → sole hardware writer

Sample deployment conditions from the dataset

The first observation already conditions the whole rollout. A pose that looks symmetric or convenient to a human may be outside the starts seen during training. The same applies to camera placement, image rate, task wording, gripper range, and action cadence: they are parts of the learned input distribution, not setup decoration.

Select complete recorded states

Build the start library from successful training episodes. Restrict candidates to the early, pre-motion part of each episode; reject terminal states and pauses that occur after task progress; then cluster complete state vectors and choose real medoids. Do not assemble a pose from per-joint means or medians: every coordinate may look typical while their combination appeared in no demonstration.

For every selected start, retain its source episode, frame timestamp, full state vector, units, camera snapshot, task label, and reachability checks. Transition to it with a separately bounded motion primitive. If the robot cannot safely reach a sampled state, fix the transition or collect better coverage rather than inventing an unseen pose.

Use the data to set motion expectations

Before adding filters or changing rate limits, compute per-joint distributions of step size, velocity, acceleration, and gripper motion from the training episodes. Compare the raw policy output and human corrections using the same statistics at the same control rate. A runtime that produces much smaller steps will look timid; one that produces much larger steps will spend the rollout against safety limits.

DAgger should correct the policy without changing the task

In intervention-based DAgger, the policy visits states produced by its own mistakes, a human takes over when needed, and those corrections become new training data. The point is to add useful labels where the deployed policy actually fails—not to create a second, incompatible control domain.

Correction data is useful only when its timing and action meaning match the original training contract. A slower teleoperator, a different control rate, or smaller command increments can teach timid behavior even when every row is valid. Compare expert and training distributions for per-step change, velocity, gripper range, command cadence, and observation age before fine-tuning.

At takeover, align the expert device to the robot’s measured configuration and transfer relative motion from that reference. Sending the expert device’s unrelated absolute pose creates a jump; changing its gain or speed without checking the original dataset changes the label distribution.

Record lineage, then derive the training view

Keep the full raw episode immutable and label, at every control step, the model proposal, aligned expert proposal, selected source, safety-adjusted command, executed action, control mode, and takeover event. Training can then select expert-only windows, include the model-failure lead-in, or build a mixed view without guessing what happened.

Use a dispatch-assigned control-step identifier when possible. Nearest-timestamp joins can cross a takeover boundary or attach a validity signal to the wrong action. Missing or conflicting lineage should be quarantined, not guessed into a valid label.

Describe a deployment instead of hard-coding it

A model bundle contains the weights and the assets required to interpret them. A deployment descriptor is the small configuration file that binds that immutable bundle to a runtime contract: which server loads it, which graph connects it, which camera profile supplies observations, and which checks must pass. The launcher reads the descriptor; it should not contain a new model-specific branch for every checkpoint.

schema: deployment/v1
name: example-policy

bundle:
  path: models/<family>/<version>
  required: [checkpoint, contract.json]
  verifier: verify_bundle.py

policy:
  frequency_hz: 30
  action_horizon: 32
  execute_prefix: 16

observation:
  camera_profile: camera-1280x720-30.yml

runtime:
  server: managed
  graph: dual-arm-policy.yml

qualification:
  tests: [contract, adapter, service, no_write_launch]

For an already-supported backend, onboarding becomes configuration plus evidence rather than another if model == ... branch. The same descriptor drives download verification, runtime preparation, qualification, launch, and rollback.

Verify what was published and what arrived

• Publisher manifest: the file list, sizes, and hashes shipped with the bundle.
• Local manifest: the downloader’s structured receipt containing object paths, sizes, source metadata, and computed hashes.
• Local hashes: freshly computed content fingerprints, often also written as a flat SHA256SUMS file.

Cross-check all three before loading the model. A complete download can still be the wrong checkpoint, and a correctly named directory can still contain one corrupted file.

Debug the first broken boundary

Do not begin with the final visual symptom. Give one observation and one action a trace identifier, follow them through the system, and find the first boundary where the value, timestamp, or ownership no longer satisfies its contract.

1. What exact observation did the model receive?
2. What raw action chunk did it return?
3. What did timing and motion processing produce?
4. What did the router and final safety boundary accept?
5. What did the hardware receive and report back?

Most deployment arguments happen because the investigation jumps from what the model should have seen directly to what the robot appeared to do.