Where does a VLA decide what to do? Poking at the action interface of three robot policiesVLA 到底在哪里决定动作？拆解三个机器人策略的“动作接口”

VLAs are usually described as “a VLM plus an action head”. The VLM sees the cameras and reads the instruction; the head turns its hidden states into a chunk of motor commands. That picture hides the interesting question: what exactly crosses the boundary between the two? Is there a compact “action space” inside the VLM that the head reads, like a bottleneck? Is it geometric, like “the cup is 12 cm to the left”? Or is it something more like a motor plan?

I started this as a quick probing exercise and it slowly turned into seven phases of experiments, a few hundred figures and a lot of failed hypotheses. I do not think it is a paper: the setting is narrow, everything is open-loop, and the most useful outcome is a list of things that do not work. But I learned a lot from it, and the story is fun, so here it is. Internally I called the object I was hunting “A-Space” (the action space inside the model).

All three policies were fine-tuned on the RoboTwin bimanual simulation benchmark and share the same backbone: Qwen3-VL-4B, 36 decoder layers, hidden size 2560. Only the action head differs.

The dataset is 10 tasks × 30 trajectories × 10 timesteps = 3000 samples, each with three cameras (head, left wrist, right wrist) and a language instruction, and no proprioception. Splits are trajectory-disjoint, plus a held-out-task split. I hooked every layer, every action token, every cross-attention update (called ΔZ below) and every flow step, checked that all three models are deterministic under a fixed seed, and built a small attention debugger to look at things by eye.

Three tools carry the whole story:

• Linear probes ask: can a variable be read out of this hidden state?
• Interchange interventions ask: if I take the hidden state from a donor sample B and paste it into a receiver sample A, does A's action move toward B's? I measure this with the causal agreement ratio, CAR = ⟨a_patched − a_A, a_B − a_A⟩ / ‖a_B − a_A‖². CAR = 1 means the action fully became the donor's; 0 means nothing happened.
• Causal subspace search asks: what is the smallest k-dimensional subspace whose patch alone reproduces the full effect? I optimize an orthonormal basis U directly for CAR, with a rule that the edited states must stay on the data manifold (median out-of-distribution ratio ≤ 1.10), validation/test splits and multiple seeds.

Most claims were preregistered as hypothesis tests with Holm correction, so that I could not quietly drop the ones that failed. Many did.

The first phase was the classic one: train linear probes on every layer for task identity, current robot state and future motor targets (the end point and displacement of the upcoming action chunk).

• Task identity is trivially decodable (≈ 1.0) from layer 0. That is just the instruction words.
• The current robot state is already in the visual tokens at layer 0 (end-effector position R² ≈ 0.9). The arms are visible in the wrist cameras.
• Future-motor targets build up across depth. For OFT, the chunk-endpoint R² goes from about .61 at the embeddings to .86 around layer 32 and .94 at the head input. The last layer is always the best layer.
• Which arm the instruction refers to peaks in the middle layers (7–19) and then declines. Where the information concentrates is model-specific: in OFT text tokens beat image tokens, in PI image tokens win, GR00T is a tie.
• Each OFT action token on its own already holds almost the whole 32-step chunk. The concatenated final states of the expert heads are the best carriers overall (≈ .985) and transfer much better to a held-out task than the VLM states do (gripper .92–.94 vs .56–.71).
• Changing the flow step barely changes decodability. Remember this; it comes back in Act IV.

The obvious next step: take the probe directions for future motion (an 8-dimensional subspace I called A_motor), and check whether the model actually uses them. If I swap only those 8 dimensions from a donor into a receiver, does the receiver's action follow?

Almost not at all. At the head input, patching A_motor gives CAR ≈ 0.008, against 0.002 for a random 8-dimensional subspace. Its complement, i.e. everything the probe ignored, carries nearly all of the effect. Even at OFT's action tokens it only reaches 0.115, and inside the expert heads it is no better than random. Steering along probe directions moves the gripper monotonically, but by at most about 1.5 cm, and at the expert sites it sometimes moves it the wrong way.

The full-state patches tell a cleaner story about where the information flows:

• From layer 6 on, the full VLM state is sufficient (CAR = 1.0). Layer 0 is weak because Qwen3-VL's DeepStack re-injects the receiver's own visual features into later layers.
• In OFT, the image tokens stop mattering between layers 24 and 33: that is where the visual information gets copied into the 32 action tokens.
• OFT's action tokens are strikingly local: swapping token i changes only timestep i of the output (influence 1.52 on the diagonal vs 0.002 off it).
• In the flow heads, the cross-attention updates (ΔZ) act in a distributed, super-additive way across blocks.
• And the first hint of Act IV: replacing ΔZ only at the last flow step transfers .907 (PI) and .513 (GR00T). Replacing it at steps 0–2 transfers ≤ .03.

If probes find the wrong directions, search for the right ones directly. For each site I optimized an orthonormal basis U (k = 1 … 256) so that patching only the U-component of the donor state maximizes CAR on held-out pairs, while keeping the edited state on the data manifold. Then I asked how many directions are needed to recover 90% of the full-state effect (k₉₀).

Two controls kept me honest here. First, shuffled donors: if you patch a state from a random other sample, CAR is still about 0.5, because every donor differs from the receiver in a shared “move toward the typical state” direction. So 0.5, not 0, is the floor for donor-specific transfer. Second, I tried nonlinear bottlenecks instead of a linear U, and they added nothing.

So my definition had to change. A-Space is not one global subspace. It is a site-specific causal interface: crisp and tiny at OFT's action tokens, broader and non-unique in the late VLM, and inside the flow heads it is not a subspace at all, but a pathway that is only active at a particular time.

This is my favorite result. Both flow heads integrate four Euler steps from noise to action. I took the donor's cross-attention update from step s and inserted it into the receiver's run at step t, for all 4 × 4 combinations, with every other step running live.

• When it is inserted decides almost everything; where it came from decides almost nothing. Inserted at the last step, the donor's update transfers .91–.93 (PI) and .58–.64 (GR00T) whatever its source step. Inserted at steps 0–2 it transfers at most 3%.
• So the information that distinguishes donor from receiver is already present at step 0, but it is written into the action only at step 3.
• The sensitivity of the action to the head's internal state, ‖∂a/∂state‖, jumps 70× (GR00T) and 190× (PI) from step 2 to step 3.
• Patching only the final expert state at the last step gives CAR = 0.251 in both models, which equals the step size Δτ = 0.25. That is what an ideal rectified-flow head should do: the patch swaps exactly one step's worth of velocity.

Why the last step? Under the rectified-flow convention, the head's clean-action estimate at time τ is â = x_τ + (1 − τ)v. At the last step the remaining step size equals 1 − τ, so the output is the step-3 estimate. Anything the head predicts earlier is partly re-predicted away by later steps. Late commitment is therefore a property of this 4-step integrator, and both heads show it, even though they were trained separately.

Knowing where and when, I wanted to know what. My hypothesis was a “world → control” story: early layers encode absolute positions of objects in the world, and deeper layers convert them into end-effector-relative displacements that are easier to act on. To test it, I replayed seeds in the simulator and re-simulated scenes to get exact geometry labels: absolute target position, displacement from the end effector to the target, and so on.

The story mostly failed, in an informative way:

• Absolute target position is readable everywhere (≈ .98 at the head input). Relative displacement is never preferred over absolute in the full state at any depth. There is no world → control transition across depth.
• Inside OFT's 8-dimensional causal space, relative does beat absolute (+0.187, confirmed). But PCA and probe subspaces of the same size show the same preference, so this is not something special about the causal coordinates.
• The compression is motor first: going from the late VLM's 32-dim causal space to the action tokens' 8-dim one, absolute position is dropped and future action becomes the most enriched variable.
• The geometry-aligned part of the causal space behaves like a random part of the same size when patched. The causal coordinates look like a phase-conditioned motor code shared across tokens.
• Action is committed at the last flow step, but geometry is not specially committed there. And the end effector is outside the head camera in about 64% of frames, so the head camera cannot be the source of end-effector-relative geometry anyway.

In the end, 2 of 7 preregistered tests were confirmed.

Finally, the engineer in me asked: can any of this make a frozen VLA better or faster without training? I tried five ideas. No weights were changed, all hyperparameters were chosen on validation, and every number is on held-out test pairs. Everything was open-loop, so none of this is a task-success claim.

The honest summary: a representation that is mechanistically causal is not automatically a good handle for controlling inference. The one useful thing was the temporal finding, late commitment, not the subspaces themselves.

1. Decodability is cheap and causality is expensive, and they disagree. Probe directions were nearly orthogonal to the directions the model uses. If I had stopped after Act I, I would have written a confident and wrong story.
2. “Where” and “when” mattered more than “which directions”. The cleanest results were about sites (action tokens) and time (the last flow step), not about any particular basis.
3. Controls change the conclusions. Shuffled donors move the floor from 0 to 0.5. Without the on-manifold rule, the flow heads would have had a tidy 4–8-dim subspace that is really just an off-manifold artifact. Random and PCA baselines of the same size killed several “discoveries”.
4. Preregistering hurts, in a good way. Writing down tests before running them meant I had to report that 5 of 7 tests in the last phase failed.
5. Check the plumbing. A pooling artifact looked like a finding. “Layer 36” turned out to mean “output of decoder layer 35, before the final norm”, which is what the heads read. A verify run once silently truncated a file. All numbers here were re-checked against the result tables.

• One benchmark (RoboTwin, simulation), 10 tasks, 3000 samples, and one shared backbone. The three heads differ, but nothing here says how other backbones behave.
• Everything is open-loop: I measure how actions change, not whether the robot succeeds.
• Late commitment is tied to a 4-step Euler integrator. With more steps or a different solver, the picture may be smoother.
• Subspace coordinates outside OFT's action tokens are not unique across seeds, so I avoid interpreting individual directions.

If you work on VLA interpretability, or you have ideas for turning “late commitment” into something more useful, I would love to chat. My email is on the home page.