Trustworthy Language from Structured Evidence: Explanations, Taxonomies, and Diagnostics

Abstract

Emerging AI systems must not only act correctly but also explain their behavior in verifiable natural language. This dissertation traces a research arc that begins with NavChat, a post-hoc framework for explaining Uncrewed Aerial Vehicle (UAV) navigation decisions made by deep reinforcement learning (DRL) policies. While NavChat’s large language model (LLM) rationales are often fluent and intuitive, they reveal systematic reasoning failures, such as mis-handling negative deltas, demonstrating that fluency alone is not evidence of faithfulness and raising a broader question: Do current models reliably perform operator-level reasoning over structured evidence, and how can we measure it? Motivated by the structured, episodic nature of UAV navigation traces, which can be represented as tabular sequences, we pivot to the Logical Table-to-Text (LT2T) paradigm, where tables provide a controllable sandbox for probing logical fidelity. First, we contribute a comprehensive LT2T survey that codifies core challenges, method taxonomies, and existing gaps, including weak evaluation metrics. We find that prior evidence for strong LLM reasoning is often based on aggregate automatic scores that obscure which logical operations models actually execute correctly and where verifiers disagree. Building on this finding, we design an operation-aware diagnostic framework for LLM-based LT2T that decomposes performance into four competencies: (1) logical form execution, (2) logical form conditioned generation, (3) logic type prediction, and (4) logical form free generation. This framework reveals a persistent meta-logical gap: models can produce plausible or even logically entailed statements while failing to identify the intended underlying operation, such as aggregation versus ordinal reasoning. This dissertation makes three contributions: (i) NavChat, an LLM-assisted method for rationalizing DRL navigation decisions and empirically characterizing explanation failure modes; (ii) a Logical Table-to-Text survey that systematizes methods and limitations for reasoning-grounded generation; and (iii) an operation-aware diagnostic framework that localizes model and evaluator failures by logical competency. Together, these contributions advance trustworthy language interfaces for structured AI by grounding explanations in model-relevant evidence and providing transparent diagnostics that clarify not only whether they fail, but also why they fail.