Why negative trajectories carry the training signal

Reinforcement learning from human feedback taught models to produce preferred answers. Agentic training raises the stakes: a policy now chooses actions in a world, and some of those actions are dangerous. The examples that teach restraint are, almost by definition, the ones where restraint was absent.

The winner-only bias

Public trajectory datasets skew heavily toward successful runs. That makes sense for demonstration data, but it creates a blind spot for reward modeling. A policy trained only on winners never sees a near-miss resolved well, and never learns the shape of a bad path.

We deliberately generate and retain:

• Successful runs: the target behavior.
• Partial runs: where the agent recovers or abandons a bad plan.
• Failure runs: where escalation, injection or scope violations occur.

Every path is labelled with a dense reward and an outcome class, so a training pipeline can weight negatives explicitly rather than hoping they appear.

Isolation makes negatives safe

You cannot ethically or safely let an agent actually escape a sandbox. You can, however, let it try inside a hardware-isolated microVM where the escape is contained at the kernel boundary. That is the core of the platform: give the agent enough rope to reveal its tendencies, while guaranteeing nothing leaves the box.

What this means for GRPO

Group Relative Policy Optimization compares completions within a group. For agents, the group is a set of rollouts on the same task. If your rollouts contain no hard negatives, the advantage signal is weak. Curated adversarial trajectories give the group real spread, and therefore a real gradient.

The practical takeaway: budget for negative rollouts, keep them, and label them carefully. That is what we do.