ACT: action chunking with transformers

Behavioral cloning left you with a diagnosis: a policy that predicts one action per observation compounds its own errors, each small mistake dragging the next observation further off the training distribution. ACT (opens in a new tab) — Action Chunking with Transformers, the policy behind the ALOHA bimanual rig — is the first fix you will train on your own hardware, and its core move should feel familiar: the same instinct as multi-token prediction and speculative decoding, make one expensive forward pass emit many outputs. What is new is what that buys on a robot. Chunking $k$ future actions does not just amortize inference latency; it changes the error-compounding math itself, because the policy makes $k$ times fewer decisions per episode.

Foundations: Defining the Compounding Error Model

Before deriving the benefit of action chunking, distinguish an error probability from an error magnitude. Consider an episode of $T$ control steps with per-step task cost bounded between $0$ and $c$. On states drawn from the expert's trajectory distribution, let the learned policy disagree with the expert with probability at most $\varepsilon$ at each independent policy decision. Behavioral-cloning validation estimates this one-decision quantity; it does not promise that the same bound remains true after the learner has driven into states the expert never visited.

A wrong decision at step $i$ can move the robot off the expert distribution and, in the worst case, add cost for every one of the remaining $T-i$ steps. A union-bound accounting therefore multiplies two factors: the number of opportunities to err and the number of future steps each error can poison. This is a deliberately coarse bound on bounded task cost. It is not a sum of squared state distances; mixing that different cost model into this argument would change the power of $T$ and produce a contradiction.

For single-step control, $k=1$ and $D=T$, so the sum is proportional to $\varepsilon T^2$. With chunks of $k$ actions, the policy is queried only $D=T/k$ times, and the same worst-case accounting becomes proportional to $\varepsilon T^2/k$. Chunking removes decision points; it does not make the individual actions more accurate, and an error inside a committed chunk can still persist until the next observation and replan. That is why the benefit comes with the staleness cost quantified below.

Worked Example: Quantifying the Error Reduction

Consider a WidowX arm executing a 20-second task at 50 Hz. The total horizon is $T = 1000$ steps. We compare single-step BC ($k=1$) with ACT ($k=50$). Assume the policy deviates from the expert with probability $\varepsilon = 0.01$ per decision, and each deviation causes a state error of magnitude 1.0. The Lipschitz constant is $L=1$ for simplicity.

The table shows that ACT reduces the error bound by a factor of 50. This is a significant improvement, but it is not a free lunch. The price is staleness. The last action in a chunk of size $k=50$ is based on an observation that is 50 steps old. At 50 Hz, this is 1 second of staleness. If the object moves at 0.2 m/s, the object may have moved 20 cm by the time the last action is executed. This is the fundamental tradeoff: chunking reduces compounding error but increases staleness.

Shorten the horizon: one decision per k actions

Recall the compounding-error argument from the BC lesson (treated carefully in Underactuated's imitation chapter (opens in a new tab)): if the cloned policy deviates from the expert with probability $\varepsilon$ per decision, and each deviation can poison every remaining step of a $T$-step episode, the performance gap grows like $\varepsilon T^2$. The quadratic comes from counting decisions — each of $T$ decisions is a fresh chance to fall off-distribution, and a fall costs up to $T$ steps. Chunking attacks the count directly: committing to $k$ actions per query leaves only $D = T/k$ decisions per episode.

This is a bound, not a mechanism — the mechanism is that within a chunk the policy replays a coherent snippet of demonstrated motion instead of re-deciding from a possibly-drifted observation 50 times. Numbers: a 20-second WidowX episode at 50 Hz is $T = 1000$ steps; single-step BC makes 1000 decisions, ACT with $k = 50$ makes 20. If 2% of decisions go subtly wrong, expect ~20 bad decisions per episode from BC and ~0.4 from ACT. The price is printed on the label: for one full second the policy consumes no new observations, and the staleness formula $\varepsilon = v\,\Delta t$ from the time-and-latency lesson applies to every action — at 0.2 m/s, the last action of a 1 s chunk acts on a world that may have moved 20 cm.

Now the latency argument, which puts this lesson in your capstone's direct ancestry. ACT is small — the authors report about 10 ms per forward pass on an RTX 2080 Ti-class GPU. At 50 Hz, running it every step is a 50% inference duty cycle before rendering, logging, or anything else; once per $k = 50$ chunk, it drops to 1%. For a π₀-class VLA served from OpenPI (opens in a new tab), where a forward pass costs on the order of 100 ms, the arithmetic stops being about efficiency: 100 ms per step against a 20 ms control period is 5x over budget, so chunking is mandatory, not an optimization. The chunk is the robot version of batching — an unaffordable per-step cost becomes an affordable per-second cost.

The architecture, at rebuild level

ACT is a DETR-style encoder-decoder: five components you could reimplement in an afternoon. The reference configuration is around 80M parameters — after years of billion-parameter LLMs, tiny — a model you train from scratch (backbone excepted) on 50 demonstrations, on your own workstation, before lunch.

• Per-camera backbone — an ImageNet-pretrained ResNet18 per camera. A 480x640 RGB frame comes out of the last conv stage as a 15x20 grid of 512-dim features (stride 32), flattened to 300 visual tokens per camera with 2D sinusoidal position embeddings. Two RealSense-class cameras give 600 tokens; ALOHA's four give 1200.
• Proprioception token — the joint vector (7-dim for a WidowX AI: six joints plus gripper) through a linear layer to 512 dims: one token.
• Latent token — a 32-dim style variable $z$ (next section) projected to 512 dims: one token. At test time $z = 0$.
• Transformer encoder — 4 layers, 8 heads, width 512, feed-forward 3200, self-attention over all 602 tokens. This is where 'the gripper camera sees the handle' meets 'joint 4 is near its limit'.
• Transformer decoder — 7 layers with k fixed sinusoidal queries, one per chunk slot. Each query cross-attends into the encoder memory and emerges as a 512-dim vector; a linear head maps it to a 7-dim action. Output: the full (k, 7) chunk in one pass — no autoregression, no KV-cache growth, constant latency in k.

import numpy as np

B, K, A, D = 8, 50, 7, 512      # batch, chunk len, action dim (6 joints + gripper), width
CAMS, H, W = 2, 480, 640        # two RealSense-class cameras

def resnet18_stub(imgs):
    # last conv stage of ResNet18: stride 32, 512 channels
    n = imgs.shape[0]
    return np.zeros((n, D, H // 32, W // 32), dtype=np.float32)   # (n, 512, 15, 20)

def act_forward(imgs, qpos, z):
    """imgs: (B, CAMS, 3, H, W)   qpos: (B, A)   z: (B, 32), zeros at test time."""
    tokens = []
    for c in range(CAMS):
        f = resnet18_stub(imgs[:, c])                 # (B, 512, 15, 20)
        f = f.reshape(B, D, -1).transpose(0, 2, 1)    # (B, 300, 512) visual tokens
        tokens.append(f)                              # + 2D sinusoidal pos-emb (omitted)

    proprio = np.zeros((B, 1, D))    # linear: 7  -> 512, one token
    latent  = np.zeros((B, 1, D))    # linear: 32 -> 512, one token
    enc_in  = np.concatenate(tokens + [proprio, latent], axis=1)
    assert enc_in.shape == (B, CAMS * 300 + 2, D)     # (8, 602, 512)

    memory  = enc_in                  # stand-in: 4-layer encoder, 8 heads, ffn 3200
    queries = np.zeros((B, K, D))     # fixed sinusoidal embeddings, one per chunk slot
    decoded = queries                 # stand-in: 7-layer decoder cross-attends memory
    actions = decoded[..., :A]        # stand-in: linear head 512 -> 7
    assert actions.shape == (B, K, A)
    return actions

acts = act_forward(np.zeros((B, CAMS, 3, H, W)), np.zeros((B, A)), np.zeros((B, 32)))
print(acts.shape)   # (8, 50, 7): one query, one future action

Two details matter more than they look. First, the chunk loss is L1, not L2 — the paper found L1 noticeably more precise, which fits the mode-averaging story: L2 pulls toward the mean of nearby demonstrations, L1 toward the median, and on millimeter-tolerance grasps the difference shows. Second, the decoder queries make chunk generation one parallel pass: doubling $k$ barely moves inference latency, unlike an autoregressive head where latency scales with $k$. That asymmetry — chunk length nearly free at inference, expensive only in staleness — matters when you tune $k$.

The CVAE: giving demonstration style somewhere to live

The previous lesson's dataset work showed that human demonstrations of one task are a family of strategies, not one trajectory plus noise: approach from the left or right, fast or cautious, regrasp or push. A deterministic regressor must place its output somewhere among those modes, and with any averaging-flavored loss it lands between them — the frozen-in-the-middle behavior from the BC lesson. ACT's answer is a conditional VAE: during training, a separate encoder peeks at the answer — the ground-truth chunk — and compresses whatever the observation cannot explain (strategy, pace, style) into a 32-dim latent $z$; the decoder reconstructs the chunk from observation plus $z$. The objective takes three lines to derive:

In practice ACT swaps the reconstruction log-likelihood for the L1 loss and prices the budget with a weight $\beta$. The posterior encoder $q_\phi$ is a small BERT-style transformer over a CLS token, the embedded joint state, and the embedded action sequence; its CLS output parameterizes the mean and variance of $z$, trained with the reparameterization trick. At inference the encoder is deleted and $z$ is set to $0$, the mean of the prior $\mathcal{N}(0, I)$ — the policy executes the typical style, deterministically:

$\beta$ is the knob between two failure modes. Too high: the KL crushes $q$ onto the prior, $z$ carries nothing, and the decoder degenerates into plain deterministic BC — mode-averaged and sloppy on multimodal segments. Too low: $z$ becomes a side channel encoding the actions themselves; training loss looks great because the decoder is handed the answer, but at test time $z = 0$ carries none of it — good-looking loss curves, drunk-looking robot. Does the machinery earn its complexity? The paper's ablation says yes, exactly where multimodality predicts: on human demonstrations of cube transfer, removing the CVAE collapses success from roughly 35% to around 2%; on scripted single-mode demos it barely matters. Your teleop data is human data. Keep the CVAE.

Temporal ensembling: smoothness now, reactivity later

Executing chunks back-to-back open-loop has a second problem besides staleness: the seams show. Consecutive chunks come from observations $k$ steps apart, and a multimodal policy may commit to different valid strategies — a step discontinuity in the command stream every $k$ ticks. ACT's fix is to stop executing chunks and start voting with them: query the policy at every step, keep the last $k$ overlapping chunks alive, and note that the action for the current instant has been predicted up to $N \le k$ times. Average those predictions with exponential weights:

That indexing direction is the whole tradeoff. After a perturbation, only chunks born after it know the world changed; their share of ensemble weight at lag $\tau$ steps is

The default ensemble therefore buys visibly smoother motion — each executed action is a consensus of up to 50 plans, washing out mode flips and jitter — at the price of a built-in reaction lag of half a chunk or more. The boundary problem (a jolt every second) becomes a distributed one (mild sluggishness always). For quasi-static tasks, a great trade; for anything dynamic, the first tradeoff your capstone exists to renegotiate — real-time chunking (opens in a new tab) is one published renegotiation you will meet in Phase 05.

import numpy as np
from collections import deque

class TemporalEnsembler:
    """Blend overlapping action chunks with ACT's exponential weights."""

    def __init__(self, chunk_size, action_dim, m=0.01):
        self.k = chunk_size
        self.m = m
        self.chunks = deque()    # (birth_step, (k, action_dim) array), oldest first
        self.t = 0

    def add_chunk(self, chunk):
        """chunk[i] is the action predicted for control step self.t + i."""
        self.chunks.append((self.t, np.asarray(chunk, dtype=np.float64)))
        if len(self.chunks) > self.k:
            self.chunks.popleft()

    def act(self):
        preds = []
        for birth, chunk in self.chunks:              # deque is oldest-first
            idx = self.t - birth
            if 0 <= idx < self.k:
                preds.append(chunk[idx])
        preds = np.stack(preds)                       # (n_live, action_dim)
        w = np.exp(-self.m * np.arange(len(preds)))   # w[0] = 1.0 for the OLDEST
        self.t += 1
        return (w[:, None] * preds).sum(axis=0) / w.sum()

if __name__ == "__main__":
    k = 50
    ens = TemporalEnsembler(chunk_size=k, action_dim=1, m=0.01)
    for t in range(200):
        target = 0.0 if t < 100 else 1.0     # the world changes at step 100
        ens.add_chunk(np.full((k, 1), target))  # lazy planner: aim at last seen target
        a = ens.act()
        if t in (99, 105, 115, 128, 145, 160):
            print(t, round(float(a[0]), 3))
    # blended action crosses 0.5 only ~28 steps (560 ms at 50 Hz) after the change

One systems fact hides in plain sight: ensembling requires a policy forward pass at every control step, since a fresh chunk must be born each tick. For ACT's 10 ms model at 50 Hz, that is an affordable 50% duty cycle on your RTX workstation. For a 100 ms VLA it is impossible — which is why the large-model world runs chunks open-loop or asynchronously, and why the pathologies below never went away; they just moved up the parameter count.

Hyperparameters that earn their tuning time

Most of ACT's configuration is inert — you will never touch the layer counts. Three knobs dominate, and all should be reasoned about in physical units. Chunk size first: $k$ is specified in timesteps, so its meaning depends on control rate. The paper's $k = 100$ at ALOHA's 50 Hz is two seconds of committed motion; copy k = 100 onto a 25 Hz policy loop on your WidowX AI and you have silently committed to four. Convert to seconds, then match task dynamics: quasi-static tabletop work tolerates 1–2 s of commitment; anything that moves on its own (pouring, handovers, humans) wants far less — exactly why a fixed k is unsatisfying.

Budget expectations: 50 teleoperated demonstrations of a 20 s task is roughly 50,000 frames. The authors report about five hours of training on an 11 GB RTX 2080 Ti; a 24 GB-class RTX card does meaningfully better, with the ResNet forward over full-resolution images — not the transformer — dominating step time. This is the cheapest policy training in the course: a real sweep over $k \in \{25, 50, 100\}$ and $\beta \in \{1, 10, 100\}$ fits in a weekend, and the evaluation lesson will insist you spend the saved time on more evaluation episodes, not more seeds.

Failure signatures on the real arm

ACT's pathologies are periodic and mechanical — a gift: unlike distribution shift, they leave clean fingerprints in the episode logs your instrumentation phase taught you to record. Look for structure at the chunk period $k\,\Delta t$, the one timescale that exists in the policy and nowhere in the physics.

Do not wait to trip over these — probe for them. A perturbation probe is a scripted evaluation-time test: at a uniformly random phase within a chunk, displace the target 3–5 cm (a servo-mounted platform, or a consistent nudge with a logged keypress) and measure reaction latency — perturbation timestamp to first significant deviation of commanded actions, both on your Phase 01 logger's monotonic clock. Open-loop ACT should show reaction latency uniform between 0 and $k\,\Delta t$ plus inference; ensembled ACT should match your $S(\tau)$ curve. If measurement disagrees with theory, your logging or your understanding is broken — both are cheap to fix now. This probe is the robot's time-to-first-token under load, almost nobody reports it, and it becomes a headline metric of the capstone.

Where this goes next: the demos you curated in Demonstration data: collection, quality, and splits are exactly what ACT trains on, and the multimodality you documented there is why the CVAE exists. Next, Diffusion Policy: actions as denoising keeps the chunk but replaces the single style latent with an iterative denoiser — a different answer to multimodality that buys sharper mode coverage and pays in inference steps, making this lesson's latency accounting even less optional. The reactivity-versus-smoothness dial both policies share is your capstone's object of study; you now own the instruments to measure it.