Lab 0: bring-up, safety, and the episode logger

Everything in this phase so far has been study; today you build. You have done this week before: racking a new node, flashing firmware, wiring monitoring, writing the runbook — the shift that turns hardware into infrastructure. Lab 0 is that shift for your robot station. The previous four lessons each handed you one layer: what the actuators do when commanded, a frame tree that makes where unambiguous, one clock that makes when unambiguous, and a transport with explicit QoS contracts. Today you assemble them into a station that is safe, resettable, and observable to the millisecond — before any learned policy touches it.

The deliverables double as this module's evidence: a hardware setup guide someone else can follow, the frame diagram, a timestamp schema, the baseline latency report — and, new today, a tested safety stack plus a working episode logger with one validated recording. Budget the hour like a bring-up shift: half on safety and reset, half on the logger and its validation.

Safety before autonomy: layers, not vigilance

The dangerous phase of lab robotics is not day one, when you watch every move. It is week three, when familiarity has replaced attention and a policy is generating commands no human previewed. You cannot fix that with vigilance; you fix it with layers. The stakes, honestly: a WidowX AI (0.77 m reach, 1.5 kg payload, 27 N·m shoulder-class joints, 100 N gripper) is no industrial arm, but it is a fast 4 kg machine that can bruise a hand, pinch a finger hard, sweep its own wrist camera into the table, or shove a fixture off the bench before you can react. Safety here protects fingers first, then hardware, calibration, and datasets. The layers, from lowest latency and least context to highest:

• Controller firmware limits — joint position range, velocity and effort ceilings, temperature shutdown, enforced in the arm's iNerve controller below your host. Millisecond latency, alive even when your host is dead. The driver exposes the limit configuration; set it explicitly.
• Command validator — one choke point every command passes through: reject NaN and out-of-range targets, clamp velocity and acceleration. Acts within one control tick.
• Stale-command watchdog — if the command stream ages past a timeout, trigger the safe-stop path. Catches dead or wedged upstream processes.
• Workspace box — a Cartesian bounding volume (say 500 × 400 × 350 mm above the table) checked against the commanded target's forward kinematics. Catches fresh, well-formed commands aimed somewhere terrible.
• Hardware e-stop — a physical button reachable without moving your feet, tested at the start of every session.
• Human procedure — never reach into the workspace while torque is enabled; a printed checklist by the arm.

Foundations: Time, Kinematics, and Control Modes

Before assembling the safety stack, we must rigorously define the physical and temporal primitives that the subsequent sections rely upon. The primary distinction is between the Control Loop and the Command Stream. The Control Loop is a local, deterministic process running at a fixed frequency (e.g., 100 Hz) that owns the direct connection to the robot's actuators. It is the heartbeat of the system. The Command Stream is an external, asynchronous input (from teleop or a policy) that may arrive at variable rates. The watchdog logic exists specifically to monitor the age of the Command Stream relative to the deterministic ticks of the Control Loop. If the stream stalls, the loop must act independently.

The second primitive is Forward Kinematics (FK). FK is the mathematical mapping from joint angles $q \in \mathbb{R}^n$ to the Cartesian position $x \in \mathbb{R}^3$ of the end-effector. The 'Workspace Box' safety layer operates in Cartesian space, but the robot is controlled in joint space. Therefore, every control tick requires solving FK to verify that the commanded joint target $q_{cmd}$ results in a Cartesian position $x_{cmd}$ within the safe bounds. This is a computational cost that must be accounted for in the loop's timing budget.

The third primitive is the distinction between Position Mode and Velocity Mode control. In Position Mode, the controller drives the joints toward a specific target angle $q_{target}$. If the command stream dies, the 'safe-stop' latches the last valid $q_{target}$, holding the arm in place. In Velocity Mode, the controller drives the joints at a specific rate $v$. If the stream dies, the arm continues moving at $v$ until the watchdog triggers a deceleration. This distinction is critical: staleness in Position Mode is generally benign (the arm stops), while staleness in Velocity Mode is dangerous (the arm runs away).

Finally, we define Jitter. Jitter is the standard deviation of the time interval between consecutive control loop iterations. In an ideal system, the interval is constant ($T_{ctrl}$). In practice, CPU contention, I/O blocking, or garbage collection can cause the interval to vary. High jitter destabilizes the control system and can cause the watchdog to miss a timeout if a tick is delayed. The exit gate requires that the 99th percentile of tick intervals remains within a strict budget (e.g., < 2x nominal).

Worked Example: Worst-Case Travel Distance

We now calculate the worst-case travel distance for a robot arm in Velocity Mode when the command stream fails. This calculation determines the maximum speed cap and watchdog timeout for a given safety margin. We assume the arm is moving at a constant velocity $v$ when the stream dies. The total travel distance $d_{worst}$ is the sum of the distance traveled during the detection phase and the distance traveled during the braking phase.

Given: $v = 0.3$ m/s, $T_{wd} = 0.2$ s, $T_{ctrl} = 0.01$ s, $a = 2$ m/s². The detection time is $T_{detect} = T_{wd} + T_{ctrl} = 0.21$ s. The distance during detection is $d_{detect} = v \times T_{detect} = 0.3 \times 0.21 = 0.063$ m. The braking distance is $d_{brake} = \frac{v^2}{2a} = \frac{0.09}{4} = 0.0225$ m. The total worst-case travel is $d_{worst} = 0.063 + 0.0225 = 0.0855$ m, or 85.5 mm. If the object being manipulated is smaller than 85.5 mm, the arm will overshoot it before stopping. To limit travel to 30 mm, we solve for $v$: $0.03 = v(0.21) + \frac{v^2}{4}$. This quadratic yields $v \approx 0.19$ m/s.

The watchdog deserves real engineering, because it guards the most common failure in a research stack: a Python process that is alive but silent — wedged on a lock, stalled in a GPU allocation, garbage-collecting at the wrong moment. Two design rules. First, the check runs in the control loop, never in the command callback: a dead publisher cannot invoke anything, so staleness must be polled by the process that owns the arm connection. Second, derive the timeout from a millimeter budget, in two steps. Detection: failure can strike just after a fresh command resets the age counter, and age is observed once per control tick, so detection takes up to $T_{wd}$ plus one control period $T_{ctrl}$. Braking: an arm moving at speed $v$ decelerating at $a$ travels $v^2/2a$. Add them:

Plug in typical numbers: $v = 0.3$ m/s, $T_{wd} = 200$ ms, a 100 Hz loop so $T_{ctrl} = 10$ ms, deceleration $2\ \text{m/s}^2$. Detection contributes $0.3 \times 0.21 = 63$ mm; braking adds $0.09 / 4 = 22.5$ mm; total roughly 85 mm — wider than the mug you are asking the arm to grasp. Invert it: to keep $d_{worst} \le 30$ mm with a 100 ms timeout, the quadratic gives $v \approx 0.19$ m/s. The millimeter budget sets both the watchdog timeout and the teleop speed cap. One caveat from the actuator lesson: in position mode a stale-but-fixed goal parks the arm at its last target, usually benign; in velocity mode staleness means runaway. The watchdog matters most for streaming control — exactly what teleop and VLA policies do.

import time

class StaleCommandWatchdog:
    """Polled by the 100 Hz control loop; fed by the command receiver."""

    def __init__(self, timeout_s=0.2, on_stale=None):
        self.timeout_s = timeout_s
        self.on_stale = on_stale        # triggers the safe-stop path
        self.last_cmd = time.monotonic()
        self.tripped = False

    def feed(self):
        """Call on every valid incoming command."""
        self.last_cmd = time.monotonic()
        self.tripped = False

    def check(self):
        """Call once per control tick. A dead publisher cannot call
        anything, so staleness must be polled, not event-driven."""
        age = time.monotonic() - self.last_cmd
        if age > self.timeout_s and not self.tripped:
            self.tripped = True
            self.on_stale(age)          # ramp to zero velocity, hold, latch fault

One rule binds the stack together: a safety path you have never triggered does not exist — the same truth as untested backups. On first bring-up, trip each layer deliberately: kill the teleop process mid-motion and watch the watchdog catch it; command a target outside the box and watch the validator reject it; press the e-stop under load. Each drill should produce a timestamped fault record — the first requirement for the logger below.

Deterministic reset: throughput is a research instrument

Here is the arithmetic nobody puts in papers. An episode — reset the scene, run the task, stop — is the unit of experiment, and its cost is episode time plus reset time. A teleoperated pick-and-place on a WidowX AI runs 30–45 s; a freehand reset — walk over, reposition by eye, drive home — runs 60–120 s. The reset, not the robot, is the bottleneck:

That 6.25 hours is the difference between a dataset collected in one afternoon and one spread across three sessions — during which lighting shifts, the camera mount gets bumped, and your teleop style drifts. Engineering the reset is engineering the dataset. Three tools. Fixtures: freehand placement lands within ±10–15 mm and ±10° of the intended pose; a taped outline tightens that to ±5 mm; a 3D-printed jig gets you to 1–2 mm. Marked poses: define named object start poses in the table frame and record which one each episode used. Reset scripts: one command that drives the arm through a fixed, collision-free joint-space path to $q_{home}$ and verifies arrival to within 0.01 rad per joint — never hand-drag the arm home: the WidowX AI's gravity-compensated float mode makes backdriving harmless, but a hand-placed home pose is irreproducible.

Reset determinism buys statistical power, not just speed. Comparing two checkpoints at roughly 60% versus 70% success already needs on the order of a hundred trials per arm. If the object start pose meanwhile wanders over a ±15 mm range, task difficulty varies run to run, outcome variance inflates, and a lucky draw of start poses can masquerade as a better policy. Log the start pose and reset type into every episode's metadata so no comparison is ever confounded silently.

Camera mounting: rigidity is calibration

The transforms lesson made $T_{\text{base,cam}}$ — the camera extrinsics — a first-class citizen of your frame tree. The uncomfortable physical fact: that transform is only as constant as the metal holding the camera. Every dataset bakes the extrinsics in implicitly; every pose estimate in Phase 02 multiplies through them explicitly. A mount that flexes changes the physical transform without changing a byte of your calibration file. The sensitivity is a one-line derivation: rotate the camera by a small angle $\delta\theta$ about its center, and a scene point at range $d$ projects as if the world shifted sideways by $d \sin\delta\theta \approx d\,\delta\theta$; through the pinhole model the image moves $f\,\delta\theta$ pixels for focal length $f$ in pixels.

Against your error floor: a decent checkerboard calibration leaves 0.2–0.5 px of reprojection residual, so a half-degree bump is 10–25× the floor — and unlike noise it is coherent; averaging removes none of it. Downstream, object pose estimates shift ~5 mm (half a gripper opening), and a dataset collected half before and half after the bump teaches a policy two conflicting geometries labeled identically.

Mounting practice follows from the math. No plastic goosenecks or friction ball heads on a long lever — they flex by degrees, not milliradians. Bolt the camera to aluminum extrusion or a machined bracket tied to the same rigid structure as the robot base, so a table knock moves both together and $T_{\text{base,cam}}$ survives. Strain-relieve the USB cable — a snag applies more torque than the mount's holding friction — and draw witness marks across every adjustable joint so displacement is visible at a glance. For the wrist-mounted D405, keep the arm-mount screws tight and witness-marked, with a cable service loop that survives full joint excursions; find the failing pose now, not mid-dataset.

The episode logger: Phase 01, serialized

An episode is the atom of everything that follows: a bounded recording of one attempt, reset to stop, sufficient to reconstruct what the robot saw, what it was told, what it actually did, and what went wrong. The logger is where all four lessons converge into one schema: capture-time stamps on a monotonic clock from the timing lesson; a frame-tree snapshot and calibration hash from the transforms lesson; command/acknowledgement semantics from the actuator lesson; an honest relationship with the transport from the ROS 2 lesson.

Why not just record rosbag2 files? Keep recording bags — they are the right tool for debugging transport. But a bag captures topics as transported: arrival-ordered, QoS-shaped, best-effort drops silently absent, no per-tick alignment between camera, state, and command streams. Training pipelines want the opposite: one tick-aligned, schema-versioned record per control step, readable without ROS installed. The episode logger writes that view at collection time, when alignment is cheap, instead of reconstructing it months later, when it is guesswork. Five parts:

• Episode metadata (once per episode): task name, operator, git SHA of the collection code, schema version, frame-tree snapshot, camera intrinsics and extrinsics with a calibration hash, object start pose and reset type, wall-clock start time for humans.
• Per-tick observations: capture timestamp (exposure midpoint, host-monotonic) and arrival timestamp for every camera frame, joint positions and velocities, gripper state.
• Commands: exactly what teleop or the policy requested, with send timestamp.
• Acknowledgements: what the arm controller actually accepted, echoed back in its ~500 Hz state stream, with receive timestamp.
• Faults: watchdog trips, validator rejections, e-stop events, dropped frames — timestamped, with a machine-readable cause.

episodes/2026-07-20/ep_000137/
  meta.json          # schema_version, task, operator, git_sha,
                     # calibration_hash, frame-tree snapshot,
                     # intrinsics/extrinsics, start pose, reset type
  ticks.jsonl        # one record per control tick, append-only
  frames/
    wrist_000000.jpg # filename encodes camera + tick index
    wrist_000001.jpg
    side_000000.jpg
    ...
  faults.jsonl       # one record per fault event
  end.json           # written only on clean shutdown: status, tick count

import json
import time
from pathlib import Path

class EpisodeLogger:
    SCHEMA_VERSION = 3

    def __init__(self, root, meta):
        self.dir = Path(root) / time.strftime("ep_%Y%m%d_%H%M%S")
        (self.dir / "frames").mkdir(parents=True)
        meta = dict(meta)
        meta["schema_version"] = self.SCHEMA_VERSION
        meta["t0_monotonic"] = time.monotonic()
        meta["t0_wall"] = time.time()   # human-facing only, never for math
        (self.dir / "meta.json").write_text(json.dumps(meta, indent=2))
        self.ticks = open(self.dir / "ticks.jsonl", "w")
        self.faults = open(self.dir / "faults.jsonl", "w")
        self.n = 0

    def log_tick(self, obs, cmd, ack):
        rec = {
            "tick": self.n,
            "t_capture": obs["t_capture"],   # exposure midpoint, monotonic
            "t_arrival": obs["t_arrival"],   # host receive time
            "q": obs["q"],                   # joint positions, rad, 6 floats
            "dq": obs["dq"],                 # joint velocities, rad/s
            "gripper": obs["gripper"],
            "cmd_q": cmd["q"],               # what the policy asked for
            "t_cmd": cmd["t_sent"],
            "ack_q": ack["q"],               # what the controller accepted
            "t_ack": ack["t_recv"],
        }
        self.ticks.write(json.dumps(rec) + "\n")
        self.n += 1

    def log_fault(self, kind, detail):
        rec = {"t": time.monotonic(), "kind": kind, "detail": detail}
        self.faults.write(json.dumps(rec) + "\n")
        self.faults.flush()                  # faults are never buffered

    def save_frame(self, cam, tick, jpeg_bytes):
        name = cam + "_" + str(tick).zfill(6) + ".jpg"
        (self.dir / "frames" / name).write_bytes(jpeg_bytes)

    def close(self, status):
        self.ticks.close()
        self.faults.close()
        done = {"status": status, "ticks": self.n}
        (self.dir / "end.json").write_text(json.dumps(done, indent=2))

Three design decisions carry the weight. Crash safety by construction: ticks and faults are append-only JSONL, and end.json is written only on clean shutdown — its absence marks a crashed episode that remains analyzable to the last line, exactly like a write-ahead log. The control loop never blocks on I/O: JPEG-encoding a 640×480 frame costs 5–15 ms, which would blow a 10 ms control tick; in production save_frame hands bytes to a separate writer process over a queue — the sketch shows the interface, not the concurrency. Schema versioning from record one: you will change this schema, and episode 40 from three weeks ago must stay loadable when you do. JSONL is deliberately boring; swap in Parquet once the schema stabilizes — the fields matter more than the format.

The exit gate: record, replay, reproduce

Now prove the station works end to end with a two-minute teleop recording. At 20 Hz that is 2,400 ticks; two RealSense-class cameras at 30 fps add 7,200 frames, roughly half a gigabyte of JPEG — a useful first calibration of storage burn (a serious collection day runs 50–100 GB). Make the recording exercise the whole station, not just the happy path:

• Start from a scripted reset, with the start pose recorded in metadata.
• Sweep the arm through most of the reachable workspace, at varying speeds.
• Include at least three gripper open/close events and one full pick-and-place.
• Trip the watchdog once on purpose (pause the teleop publisher mid-motion) so the episode contains a real fault record and a safe-stop.

Validation happens at two levels. Level one, data replay: load the episode on a machine with no ROS and no robot, render a video with joint traces and command-versus-ack overlays, and check invariants mechanically — capture timestamps strictly increasing, tick-period jitter within budget (p99 under twice nominal), every command paired with an ack or a fault, frame count matching tick count, metadata complete. Level two, open-loop re-execution: from the same scripted reset, feed the logged commands back at the logged cadence and measure how closely the arm retraces — with a deterministic reset, per-joint RMS deviation should land under 0.01 rad, and on an arm specced at 1 mm repeatability most of any larger residual is your reset procedure, not the arm. That residual is your station's repeatability noise floor; write it down, because every claim in the Phase 05 latency experiments must clear it to count as signal.

Then the gate itself, quoted from this module's contract: a second engineer can reproduce the setup and recover a synchronized episode using only your repository. No shoulder-surfing, no "oh, you also need to…" over chat — every such moment is a missing README line; fold it in and rerun. If no second engineer is handy, the honest substitutes are a clean account on a different machine, or yourself in two weeks, which is operationally the same person. This is the standard you already hold infrastructure to: if it only works on the machine where it was born, it does not work.

Where this goes next: the previous lesson, ROS 2 essentials: nodes, topics, QoS, and bags, gave you the transport this station runs on; this lab froze everything above it into a safe, reproducible instrument. Phase 02 opens with Forward kinematics from scratch — no coincidence that the workspace box you built today already needed FK to check a Cartesian bound. From here on, every model you derive and every policy you train runs on this station, and every claim rests on the episodes this logger records.