You never spend time
with your model and we can
ALL tell 👀

This mini-series is a collection of rants on RL from my POV. Pure, unfiltered, first-person opinions from someone who’s spent years deep in the trenches of pre-training, post-training/fine-tuning, inference time, and every layer of the stack for models from small distilled models (Pixel Real Tone base model) to frontier systems (Gemini + Nano Banana + Human Detection Models that powered Google Search, Waymo, Vertex AI).

I’ve eyeballed thousands of trajectories, judged parametric wins and losses until my eyes bled at 2AM, and sat through more “data” pitches than I care to count.

Who this is for:

• Startups post-training your own custom models for the first time
• Big companies considering building domain-specific agents
• Fresh grad researchers running your first real post-training loop (particularly for RL since that’s one of the main ways we do agentic post-training these days)
• RL researchers might probably also find this useful, but probably too rudimentary

This is not the consensus view of every big AI lab researcher. This is just me. But colleagues say I’m pretty good at spotting what’s going to work and what’s going to waste everyone’s time.

If your RL data keeps ending up in an abandoned directory, maybe start here 🤷.

Today’s Rant: Agentic Trajectory Eyeballing

You just fine-tuned an open-source model you found on X/Twitter for your customer-facing SaaS agent. Training loss looked reasonable, eval scores hit 81%. You ship it. Three days later, users are rage-quitting in the exact scenarios your product is supposed to handle. Half your team says retrain with more data. The other half says the evals were wrong. Nobody has opened a single trajectory.

This is what I mean when I say “you’ve never spent time with your model and we can ALL tell” 👀

There is no substitute for sitting down with your model’s actual traces—the thinking tokens, the tool calls, the outputs—and just reading them.

• Not skimming aggregate metrics.
• Not glancing at pass/fail rates on a dashboard.
• Actually reading what the model did, step by step, for a meaningful sample of tasks.

Ramble out loud and talk to yourself about why something does or does not make sense as a model behavior.

What I do Instead

Basically I wrote a guide about how I think about reviewing agentic trajectories.

Alright so here we go…

#01 What is a trajectory, really

A trajectory is a receipt for everything your model did on a task. Every decision, every tool call, every mistake. Not a chat log, but a complete decision record: every input your model saw, every intermediate output, every piece of reasoning it generated, from task start to final submission. Think tokens, API calls, drafted outputs, self-corrections. All of it, in sequence.

Your eval metrics only score the final output. That’s it. Your trajectory shows you how the model got there — whether it actually reasoned correctly, stumbled into the right answer by dumb luck, found a shortcut your rubric didn’t catch, or did something that you’ve genuinely never seen before. A model that passes your eval can still be doing something that will embarrass you in front of paying users. Your trajectories are where that shows up first.

#02 Sanity Check Your Harness

Whenever I’m debugging a Reinforcement Learning (RL) run I usually try and first figure out if my error is a harness problem or a training run problem. The best way to do this is to just sanity check my Harness. This is basically my first 3 things I look into when I do that (not exhaustive though~).

I’ll do a full deep dive on harness architecture in a future post, but here’s the TLDR is really check your harness before you jump to retraining. Your model learns from the trajectories it generates. If those have bad context, missing verifiers, or gameable rubrics, more training teaches the model to be confidently wrong. Read the traces before you rerun.

#03 The failure modes I look for first

After staring at enough of these, you start seeing the same failure modes over and over. Most trajectory failures fall into predictable buckets. Here’s how I break it down on first Eyeballing pass:

#04 Auriel’s Summarized 10-point eyeballing checklist for post training agentic models (mainly RL)

This is my final opinionated checklist I go through when I sit down with a batch of traces. For each item: what to look for, and whether it points to a harness problem, a training job problem, or could be either. For like ten trajectories it usually takes me around 90 minutes.

#05 Build your Agentic Eyeballing Trajectory Viewer in 30 minutes

Sometimes, to make it easier, I will also vibe-code myself a viewer tool — a tinder-style swipe-through interface — or use tools like hud.ai where I can step through each trajectory action by action. I look at my 10 point checklist above and I go through the trajectories, and from each Trajectory, I look at a single Trace and I begin critically thinking and reading through them. Here’s a sample prompt I use to vibe code a Coding Trajectory Viewer on the fly:

Takes around 30 minutes with your coding assistant of choice. Here’s a video sample of what it should look like and some screenshots of what I got when I vibe coded with Codex CLI and Claude Code in 1 shot:

Codex CLI

Claude Code

Once you have a viewer, the five questions I ask on every trace:

Do this enough and you build an intuition that no metric can replace. You start to feel when data is going to produce real learning versus when it’s going to be noise. That intuition is what makes someone effective at RL evaluation, and there is no shortcut to developing it other than putting in the hours with the actual traces and though this guide is a start it’s definitely very opinionated and 100% not exhaustive.

What you can do immediately

Thank you to my friends who edited this so many times over and over again with me and helped me take the first leap at posting on the internet: David Pantera, Daniel Kim, Jessica Li, Shagun, and Jay.