Overview

The Argument Audit Analysis Framework (AAA) is the heaviest analytical operation in T1 (argumentative artifact examination). It exists because arguments fail in two structurally different ways at once, and treating each failure separately misses the most consequential category of failure: the place where the two interact. Coherence-audit alone tells you whether the inferential moves hold. Frame-audit alone tells you what frame the argument imports. AAA runs both passes in full and then performs three synthesis stages on top — surfacing exactly the issues neither component would catch on its own.

The framework runs the two component audits in their full forms (not abbreviated). Stage 1 (frame-audit) surfaces the operative frame in alternative-comparable vocabulary, populates Entman’s four functions per frame (problem definition, causal interpretation, moral evaluation, treatment recommendation), inventories selection-and-salience (what is included AND what is silent), audits Lakoff metaphors and CDA presuppositions, and constructs a counterframe so the operative frame is visible as a frame. The stance is suspending — surface the frame, do not attack it. Stage 2 (coherence-audit) performs charitable reconstruction first, breaks each inferential move into Toulmin components (claim, data, warrant, backing, qualifier, rebuttal), names fallacies with full structural specification (quoted text plus inferential move plus violated principle plus reason-it-fails-here), and surfaces structural failures beyond the named-fallacy taxonomy (premise smuggling, scope shift, definitional drift). The verdict separates “this argument as given does not establish its conclusion” from “the conclusion is false.”

The framework’s load-bearing intellectual content lives in the three synthesis stages. Synthesis Stage 1 (frame-coherence merge) pairs each Toulmin move with the frame-imports bearing on it — surfacing where a warrant depends on a metaphor or presupposition that smuggles in framing-level commitments the coherence pass alone treated as background. Synthesis Stage 2 (cross-cutting integration) is the integration’s signature output, identifying three classes of issue: arguments whose internal coherence holds but only because contested frame-imports are doing analytical work, coherence-failures that turn out to be frame-substitutions across claims masquerading as inference, and motte-and-bailey-style structures that operate across multiple claims. Synthesis Stage 3 (integrated audit document) assembles the eight-section final report featuring per-claim coherence × frame pairings rather than parallel concatenation.

The framework carries Debate D2 explicitly — motte-and-bailey as fallacy versus doctrine. Shackel introduced the term as a doctrinal characterization requiring multi-claim alternation evidence; popular usage has expanded it into a fallacy label applied to single moves. AAA does not adjudicate the debate but enforces specificity: when motte-and-bailey is invoked, the audit names which claim is motte, which is bailey, where the alternation is observed, and whether Shackel’s stricter doctrinal usage or the wider fallacy-label usage best fits the case.

The framework is honest about what it does not do. It is stance-suspending and stance-neutral. It does not steelman, does not advocate, does not recommend rejection. When the audit surfaces something that warrants stance-bearing follow-up, AAA flags the handoff to T15 modes (steelman, balanced-critique, the red-team modes) rather than performing it. When the artifact is propaganda — Stanley-style strategic deployment of frames combined with intentional inferential trickery — AAA’s stance-suspending posture is insufficient and the framework recommends sideways routing to propaganda-audit (T1’s adversarial-stance specificity-specialized variant). The discipline of staying within its lane is what makes AAA’s findings trustworthy when they are produced.

Systemic context

AAA sits at the top of T1’s depth ladder, above the two atomic component modes (frame-audit and coherence-audit) and beside the specificity-specialized propaganda-audit. It is invoked three ways: directly by the user when both inferential and framing issues are suspected; by the four-stage pre-routing pipeline when a prompt classifies as T1-depth-molecular; or by handoff from one of the atomic modes when their analysis surfaces that the audit warrants integrated treatment. AAA’s downstream handoffs route to T15 modes for stance-bearing follow-up, to T9 (frame-comparison or worldview-cartography) when the dispute is really about which paradigm is correct, and to propaganda-audit for adversarial-stance work on persuasion-engineered artifacts. Lens dependencies span Toulmin (via coherence-audit), Lakoff/Goffman/Entman (via frame-audit), Walton’s argumentation schemes as the dialectical-classification overlay across both passes, and Shackel for the motte-and-bailey debate.

Ora’s output

Example question: A friend sent me an op-ed arguing that “AI is just statistics” and that safety concerns are “confused metaphysics about computation.” She thinks the argument is sound. I think something is off but I can’t articulate what. Can you audit it for me?

Ora’s response (running AAA on the op-ed):

Both inferential and framing issues are plausibly at play here, and your inability to name what is off is itself a signal that something cross-cutting may be operating. AAA will run both passes in full and then synthesize.

Stage 1 (frame-audit). The operative frame is “models-as-statistics” — a deflationary frame characterizing AI capabilities as the natural and expected output of large-scale curve-fitting, with no qualitative novelty requiring new safety frameworks. The Lakoff metaphor inventory surfaces “MODELS ARE CURVE-FITTERS” (source domain: regression analysis) and “PREDICTIONS OF CATASTROPHE ARE METAPHYSICS” (source domain: confused philosophical speculation). Goffman: a technical-redoing — safety discourse re-keyed into the technical-statistical frame, treating safety claims as category errors when stated in cognitive vocabulary. Entman’s four functions for the frame: problem = confused public discourse; cause = poor public understanding of how ML works; moral = clarity good, speculation bad; treatment = educate the public. Selection-and-salience: included = curve-fitting metaphor, training data, statistical interpretation of outputs. Excluded = emergent capabilities not predicted by training objective, in-context learning, capability gains from scaling that surprised researchers. Counterframe constructed: “models-as-cognitive-systems” — the same systems described in terms of capabilities, generalization, in-context reasoning, with safety becoming an empirical question about whether capabilities scale to dangerous regimes.

Stage 2 (coherence-audit). Charitable reconstruction first: the argument’s strongest version is “concerns about AI catastrophe rest on attributing cognition to systems whose actual operation is statistical pattern-matching; recognizing this dissolves the apparent threat.” Toulmin breakdown finds three moves. M1 (claim: AI capabilities are statistical pattern-matching; data: training is gradient descent; warrant: training mechanism determines the nature of the system) — the warrant is contestable; it assumes a level-collapse between training mechanism and inference-time properties. M2 (claim: catastrophe predictions rest on confused metaphysics; warrant: cognitive vocabulary applied to statistical systems is category error) — the warrant smuggles in the conclusion. M3 (claim: therefore safety concerns are misplaced; warrant: clarifying nature of systems dissolves safety concerns) — partial fail, since safety-relevant capabilities can exist independent of how the underlying mechanism is characterized. Named fallacies: begging the question on M2; genetic fallacy across moves. Structural failure beyond the taxonomy: premise smuggling — the implicit premise “if a system can be described statistically, it cannot have cognitive capabilities” is load-bearing but unstated.

Synthesis Stage 1 (frame-coherence merge). M1’s warrant is doing analytical work that the “models-as-statistics” frame supports but which is not independently justified. Under the counterframe, the same data does not warrant the same claim. M2’s warrant is the frame itself, deployed inside the argument to defeat the alternative frame. The frame import “MODELS ARE CURVE-FITTERS” influences M1, M2, and M3 — it is the load-bearing frame doing most of the argumentative work; the inferential moves are largely vehicles for frame-deployment.

Synthesis Stage 2 (cross-cutting integration). CI1 — coherence depends on contested frame import. The argument’s internal consistency holds within the “models-as-statistics” frame; under the counterframe, M1’s warrant fails and the argument collapses. The coherence-audit alone would flag M2 and M3 as fallacious; it would not articulate that M1’s apparent soundness is frame-conditional. CI3 — motte-and-bailey detected. Motte (defensible): “training is gradient descent on a loss function” — true, uncontroversial, technical. Bailey (desirable): “therefore safety concerns about advanced model capabilities are confused metaphysics” — substantive, contested, carries policy implications. Alternation observed across the op-ed when the argument is challenged. Per Caveats §D2, this fits Shackel’s doctrinal usage: multi-claim evidence supports the structural characterization.

The integrated finding. Your friend is partly right and partly wrong. The argument is internally consistent within its frame. But its coherence is frame-conditional, and the motte-and-bailey structure means the defensible technical claim is being used to defend a substantive policy claim that the technical claim does not actually warrant. The argument-as-given does not establish its conclusion. Whether the conclusion (safety concerns are misplaced) is true is a separate question that this audit does not adjudicate; if you want stance-bearing follow-up, route to balanced-critique or the red-team modes in T15.

Commercial AI comparison

Comparison content auto-populates when the comparison-refresh framework runs against this question. Drafters do not author this section.

Brief comparison commentary

Auto-populates with the comparison content above.

How to use this framework

You can run the Argument Audit Analysis pattern with any AI of your choice. The composition is two full component passes followed by three synthesis stages.

The prompt:

[Paste the framework specification]

Run AAA on this argumentative artifact.

Artifact: [Paste the article, op-ed, paper, ad, manifesto, policy document, debate transcript.]

Audit focus (optional): [Which conclusion or claim cluster’s support do you want audited?]

Why audit (optional): [What you suspect is off — frame manipulation, coherence failure, both, general structural sweep.]

Genre context (optional): [Academic paper / op-ed / ad / policy document / debate / political speech — informs Walton dialogue type and frame-typology selection.]

The AI runs Stage 1 (frame-audit, full) → Stage 2 (coherence-audit, full) → Synthesis Stage 1 (frame-coherence merge) → Synthesis Stage 2 (cross-cutting integration) → Synthesis Stage 3 (integrated audit document). The output follows the eight-section template: argument summary, per-claim coherence findings, frame-surfacing findings, cross-cutting issues, named fallacies and argumentative moves, overall argument-soundness assessment, residual uncertainties, confidence map.

For best results:

  1. Provide the artifact in its full form. AAA is a molecular operation. Excerpts that omit the alternation across multiple claims will defeat the cross-cutting integration’s ability to detect motte-and-bailey or frame-substitution patterns.
  2. Do not ask AAA for a verdict on the conclusion’s truth. The framework is conclusion-agnostic by design. “This argument as given does not establish its conclusion” is a different verdict from “the conclusion is false.” If you want the latter, route to T15 modes after the audit.
  3. When motte-and-bailey is invoked, look at the multi-claim evidence. Per Debate D2, Shackel’s stricter doctrinal usage requires alternation across multiple claims. AAA enforces the specificity but the user judgment is whether the doctrinal characterization is well-supported.
  4. If propaganda is suspected, route sideways. AAA’s stance-suspending posture is insufficient for persuasion-engineered artifacts. The framework will flag the recommendation to escalate to propaganda-audit when Stanley indicators are present.

The framework is deliberately tool-agnostic. The five-stage protocol, the Toulmin and Lakoff/Goffman/Entman lens commitments, the cross-cutting integration discipline, and the conclusion-agnostic verdict structure all survive the lift to any environment.

Other examples

  • A policy white paper arguing for a particular regulatory intervention. AAA’s frame-audit surfaces the operative frame (“regulation as protection” vs. counterframe “regulation as innovation tax”); coherence-audit finds the inferential structure mostly holds within the frame; cross-cutting integration surfaces that the cost-benefit calculations rest on assumptions the frame makes salient (consumer harms) while excluding what the counterframe would highlight (compliance burden distribution). The motte-and-bailey check fires on a separate pattern: the defensible claim that “regulators should set minimum safety standards” is being used to defend the bailey claim that the specific intervention proposed is the right one. Demonstrates AAA on policy artifacts where the frame is doing more work than the inferential structure.
  • A scientific paper’s Discussion section making policy recommendations. AAA’s frame-audit names the operative frame in which the scientific findings are interpreted; coherence-audit assesses the inferential structure connecting findings to recommendations; cross-cutting integration surfaces where the Discussion’s recommendations rest on policy-frame imports rather than on the paper’s empirical findings. This is the canonical “methods-conclusion gap” pattern — the methods section is rigorous, the recommendations section quietly imports a frame the methods do not establish. Demonstrates AAA on artifacts where the local coherence within sections masks cross-section frame slippage.
  • A debate transcript with two argumentative artifacts to audit in parallel. AAA can run on one artifact at a time; the user can run two AAA passes (one per artifact) and then compare the cross-cutting findings to see whether the two sides are actually arguing within the same frame or are talking past each other through frame-substitution. The latter is the most common diagnosis for debates that “go nowhere” — both sides have internally coherent arguments within incommensurable frames, and the dispute is at the frame level rather than at the inferential level. Demonstrates AAA’s role in surfacing frame-level disagreements that look like inferential disagreements.

Citations

The Argument Audit Analysis Framework draws on three traditions in roughly equal weight. From argumentation theory: Toulmin’s The Uses of Argument (1958/2003) for the six-component decomposition that exposes implicit warrants; Walton, Reed, and Macagno’s Argumentation Schemes (2008) for the dialectical-classification overlay; Hamblin’s Fallacies (1970) for the post-textbook critical theory of fallacy. From frame analysis: Lakoff and Johnson’s Metaphors We Live By (1980) and Lakoff’s Moral Politics (1996/2002) for conceptual metaphor; Goffman’s Frame Analysis (1974) for primary frameworks, keyings, and fabrications; Entman’s 1993 Journal of Communication article for the four-function operationalization. From the motte-and-bailey debate: Shackel’s 2005 “The Vacuity of Postmodernist Methodology” for the doctrinal characterization that AAA’s CQ4 enforces.

The cross-cutting-integration synthesis is internal to Ora and emerged from observing that single-component passes consistently miss the most consequential class of issue — arguments whose coherence is frame-conditional. The framework is single-author and originated 2026-05-01 alongside the broader T1 territory framework refactor; v1.0 is the current version.

Downloads

  • Framework specification (PDF) — link to ora-ai.org canonical artifact when published
  • Framework specification (plain text) — link to ora-ai.org canonical artifact when published
  • Full white paper (PDF) — link when published