When Your AI Planner Says "Trust Me" — These Researchers Are Forcing It to Explain Itself

Explainable AI planning for hybrid systems tackles one of the hardest transparency problems in autonomous AI: getting a planner that mixes discrete decisions with continuous dynamics to justify why it chose a plan — or why no plan exists at all.

~5 min read · arXiv / cs.AI · 2026-04-14 · Agentic

TL;DR This thesis introduces a framework for explainable AI planning (XAIP) in hybrid systems — environments where a planner must reason over both discrete actions (open valve, send robot left) and continuous physical dynamics (temperature curves, robot trajectories). It focuses on two hard problems: explaining why a planner chose a specific plan over alternatives, and explaining why some problems have no solution at all. The target domains are ones where opacity isn't just annoying — it's dangerous.

Most AI planning research assumes you trust the planner. Feed it a goal, receive a plan. But deploy that planner in a hospital, a power grid, or a self-driving car, and "trust me" stops being acceptable. Operators need to understand decisions. Regulators need audit trails. And when a planner says a task is impossible, someone needs to verify that claim isn't just a bug.

This thesis takes on that transparency problem in one of the trickiest planning settings: hybrid systems.

What Makes Hybrid Systems Hard

A hybrid system is one that combines discrete state transitions — the kind classical AI planners are comfortable with — with continuous dynamics governed by differential equations. Think of a warehouse robot that must decide which shelves to visit (discrete) while navigating physical space where its velocity and position evolve continuously (physics). Or a smart energy grid that issues switching commands (discrete) while managing power flows that obey circuit laws (continuous).

Classical planners assume a clean, symbolic world. Hybrid planners have to reason across both registers simultaneously, which makes planning harder — and makes explaining those plans significantly harder still.

💡 The Core Idea A planner working in a hybrid system can't just say "I chose action A before action B because A's preconditions are met first." The timing of continuous dynamics matters too. This thesis builds an explanation framework that accounts for both: why discrete actions were ordered as they were, and why the continuous state trajectories between them constrain what's even possible.

Two Kinds of Explanation

The thesis focuses on two distinct explanation problems, each with different audiences and stakes.

Explaining Plans: The Contrastive Question

The dominant approach here is contrastive explanation — answering questions of the form "why did you do this instead of that?" This mirrors how humans naturally interrogate decisions. You don't ask "explain your entire plan"; you ask "why did the robot go left instead of right?" or "why did the optimizer schedule maintenance now instead of later?"

The framework developed here handles a range of specific contrastive questions about hybrid plans:

Why was action A chosen over action B at this step?
Why does action X appear before time T in the plan?
Why does action X appear after time T?
Why does action X appear at all — what breaks if we remove it?
Why is the plan duration optimal (or is it)?
Why is the plan length what it is?

Each question type requires a different formal argument, because each probes a different aspect of how the planner's constraints — both discrete preconditions and continuous dynamics — shaped the solution.

sequenceDiagram
    participant User
    participant ExplanationFramework
    participant HybridPlanner

    User->>ExplanationFramework: Why action A before B?
    ExplanationFramework->>HybridPlanner: Constrain plan: force B before A
    HybridPlanner-->>ExplanationFramework: Infeasible / Suboptimal result
    ExplanationFramework-->>User: Because swapping A and B violates [constraint X]

This contrastive approach is powerful because it's falsifiable. Rather than generating a narrative explanation that might or might not be accurate, the framework actually re-solves the planning problem with the user's proposed alternative enforced — and reports what breaks. If swapping two actions makes the plan infeasible or longer, that's the explanation.

Explaining Unsolvability: The Harder Problem

The second, arguably more novel contribution is explaining why a plan doesn't exist.

When a planner returns "no solution found," there are two very different reasons this could happen: the problem is genuinely impossible given the constraints, or the planner failed to find a solution that does exist. In safety-critical domains, confusing these two is dangerous.

The thesis develops methods for proving the absence of a plan in hybrid systems — and crucially, making that proof human-interpretable. The approach uses a technique called δ-approximation, which handles the inherent difficulty that hybrid systems with continuous dynamics can be undecidable in general. By approximating the continuous dynamics to a bounded precision δ, the framework can produce decidable proofs of unsolvability that are still practically meaningful.

⚠️ Watch out for

Hybrid planning is formally undecidable in general — the δ-approximation approach trades exactness for decidability, so proofs of unsolvability are approximate, not absolute.

The thesis acknowledges limitations (Section 3.7), though specific details aren't available in the extracted content.

The framework targets planners that explicitly model continuous dynamics; purely symbolic planners or black-box neural planners are out of scope.

Why This Work Is Timely

Automated planning is no longer a research curiosity. It's embedded in warehouse logistics, hospital scheduling, autonomous vehicles, and energy grid management. As these systems become more capable, the opacity problem gets worse: more powerful planners make less intuitive decisions, and the stakes for misunderstanding those decisions rise.

Regulatory pressure is also building. The EU AI Act and similar frameworks are pushing toward mandatory explainability for high-risk AI applications — exactly the domains (healthcare, transport, critical infrastructure) where hybrid planning is deployed.

The particular focus on hybrid systems matters because most XAIP research has targeted simpler, purely discrete settings. Real-world autonomous systems almost universally involve continuous physics. A framework that only explains symbolic plans is of limited use when the hard part of the problem is the continuous dynamics coupling the discrete decisions together.

The robotic motion planning component of the thesis — a constraint-solving-based navigation algorithm for a lizard-inspired quadruped robot operating in unknown environments — grounds the abstract framework in a concrete physical system, connecting the theoretical XAIP contributions to an actual deployed robot.

The Broader Landscape

XAIP as a field is still young, and this thesis maps four major explanation strategies: contrastive explanation, model reconciliation (aligning the planner's world model with the user's mental model), logic-based proofs, and divide-and-conquer strategies that decompose complex explanations into simpler subproblems. The hybrid-systems focus is what distinguishes this work — most prior XAIP research lives in the cleaner world of classical, discrete planning.

For practitioners building autonomous systems that need to be auditable, the core takeaway is architectural: build explanation capability into the planning framework from the start, not as an afterthought. The contrastive approach in particular — re-solve with constraints enforced — is a pattern that generalizes beyond hybrid systems and is worth understanding regardless of your specific planning domain.

Source: Explainable Planning for Hybrid Systems Authors: Mir Md Sajid Sarwar Published: 2026-04-14 PDF: https://arxiv.org/pdf/2604.09578