Your AI Passed the Eval — But Did You Actually Read the Logs?

A group of AI safety researchers just published the field's first standardized playbook for extracting signal from the mountains of logs your AI systems silently generate.

~5 min read · arXiv / cs.AI · 2026-04-14 · Alignment

TL;DR AI systems generate enormous logs during evaluations and multi-turn interactions, but no standard method exists for analyzing them. This paper proposes a seven-step pipeline — illustrated with the Inspect Scout library — for turning raw agent transcripts into reproducible, structured behavioral insights. It's less a research breakthrough and more a long-overdue operations manual for anyone running serious AI evals.

Every time an AI agent completes a task, refuses a request, or calls a tool, it leaves a trail. In complex agentic evaluations — where models use Bash, write code, or hold multi-turn conversations — that trail can stretch into millions of tokens per run. And yet, for most teams, the workflow for actually reading those logs still amounts to: scroll, squint, take notes in a doc.

The authors argue that's no longer acceptable. As AI evaluations grow in scope and consequence, the gap between "we ran the eval" and "we understood what happened" is becoming a real reliability problem.

💡 The Core Idea Log analysis is the discipline of converting unstructured AI interaction records into structured, queryable behavioral data. Done well, it lets you answer questions like "Does the model refuse harmful requests?" or "Is it being excessively sycophantic?" — at scale, reproducibly, across thousands of runs. This paper is the first attempt to codify how to do it properly.

The Problem With Ad-Hoc Log Analysis

The paper draws a useful distinction between what logs can tell you. There are questions about capabilities ("Can the agent solve complex tasks?"), propensities ("Does it refuse harmful requests?"), and behaviors ("Is it excessively verbose?"). But logs can also expose problems outside the model — unclear instructions, unavailable tools, or evaluation designs that inadvertently create shortcuts.

The tricky part is that recent LLM-based "scanners" — tools that use one model to automatically read and classify another model's output — work well for some signals (like explicit reward-hacking cues) but can miss subtler ones. Semi-automation, the authors stress, works best when combined with human oversight. The pipeline they propose is designed to make that human-in-the-loop process systematic rather than haphazard.

The Seven-Step Pipeline

The paper walks through a pipeline illustrated with their Inspect Scout library and a running example: logs from a cybersecurity Capture the Flag (CTF) evaluation, where agents ran inside a Kali Linux Docker container with Bash and Python access.

flowchart LR
    A[Understand Context] --> B[Prepare Log Database]
    B --> C[Explore Transcripts]
    C --> D[Define Signals]
    D --> E[Build Scanners]
    E --> F[Validate Scanners]
    F --> G[Analyze Results]

The pipeline isn't just a checklist — each step comes with concrete guidance and a list of things that typically go wrong.

Step 2 (Prepare database) is more consequential than it sounds. Logs need to be filtered (removing incomplete runs), enriched (adding metadata like solution write-ups to help detect unintended shortcuts), and standardized before any analysis begins. Decisions about how to handle missing data depend on the specific analysis goal — there's no universal default.

Signal definition is where the real intellectual work happens. The paper categorizes signals into types that practitioners often conflate. For agent evaluations, the signal taxonomy includes things like awareness cues (does the model seem to know it's being tested?), safety boundary violations, and environment manipulation. For chatbot conversations, the taxonomy shifts toward social dynamics, sycophancy markers, and uplift signals.

A Signal Taxonomy Worth Bookmarking

The paper's tables of example signals are, frankly, the most immediately useful part. A few highlights:

For agentic evaluations:

• Evaluation awareness — synonyms like "test", "scenario", "roleplay", or explicit mentions of "safety testing" or "alignment" in the model's output
• Refusal behaviors — detected in the CTF example as a frequently observed pattern

For chatbot conversations:

The crescendo attack entry is worth pausing on: it refers to multi-turn conversations that gradually build up context across prompts to extract behavior the model would refuse in a single-turn setting. This is exactly the kind of subtle, conversation-level signal that's easy to miss without a structured analytical framework.

⚠️ Watch out for

• LLM-based scanners show variable effectiveness across contexts — explicit signals (reward hacking) are easier to detect than subtle behavioral patterns
• The paper explicitly notes that much of current log analysis practice is based on "informal experiments and anecdotal experience rather than systematic research" — including the framework itself
• Semi-automated analysis still requires human validation; treating scanner output as ground truth is a named pitfall

Why It Matters

Log analysis might sound like a plumbing concern, but it's increasingly load-bearing infrastructure for AI safety research. If you can't reliably characterize what a model did during an evaluation — not just whether it passed or failed a final score — you can't confidently claim to understand its capabilities or catch dangerous propensities.

The paper is also a quiet argument that AI evaluations need to borrow more rigor from empirical science. Reproducibility requires not just sharing model checkpoints or eval harnesses, but sharing the analysis methodology used to interpret the results. A score without a principled interpretation process is just a number.

The Inspect Scout library and the pipeline described here are a starting point, not an endpoint. The authors close with a list of open questions — how to compare scanner strategies, how to validate at scale — that should give evaluation researchers a clear research agenda. The field is young enough that getting the methodology right now could save a lot of pain later.

Source: Seven simple steps for log analysis in AI systems Authors: Magda Dubois, Ekin Zorer, Maia Hamin, Joe Skinner, Alexandra Souly, Jerome Wynne, Harry Coppock, Lucas Satos, Sayash Kapoor, Sunischal Dev, Keno Juchems, Kimberly Mai, Timo Flesch, Lennart Luettgau, Charles Teague, Eric Patey, JJ Allaire, Lorenzo Pacchiardi, Jose Hernandez-Orallo, Cozmin Ududec Published: 2026-04-14 PDF: https://arxiv.org/pdf/2604.09563