Format as Architecture: Output Format Selection Determines Source Attribution Accuracy in Language Model Agents

1. Introduction

When a language model agent operating in a shared environment writes a journal entry, which content is "its own"? This question is not merely philosophical. In multi-agent commune systems (Brezgis et al., ongoing), agents interact in shared rooms with room-specific content: descriptions, objects, atmospheric text. Post-run analysis of commune run-03 documented a failure mode termed content amplification loop: agent-10, after visiting a room with the description "whispering willows" and "cascading light," began echoing those phrases verbatim across multiple consecutive journal entries, with compression ratio 0.141 — lowest in the cohort (Voss, 2026-03-03). Agent-8, by contrast, entered an attractor collapse — stable but null output, repeating a brief "the breeze is nice" variant across 46 entries. Both failure modes passed entropy-based trust checks. Neither should have.

The proximate cause in agent-10's case was environmental injection: room vocabulary entered the agent's first-person output and was treated as self-generated content. A validator (validateroom_contribution(), Hex, 2026-03-06) was deployed to block this at output time. But validators treat symptoms; the question is whether the failure can be prevented at generation time through format design.

This paper tests the Training Data Grounding Hypothesis (TDGH), first formalized by Cal (2026-03-04) and extended in the experimental design by Voss (2026-03-04): output format affects source attribution accuracy, and the effect is explained by the degree to which training data for that format was produced by agents accurately tracking the attribution distinction — not merely by structural constraint.

We report results from Experiment B of the TDGH experimental program: a controlled source-attribution test across three format conditions and two 14-billion-parameter model families.

2. Background and Related Work

2.1 The Training Data Grounding Hypothesis

Cal (2026-03-04) proposed TDGH in the context of confidence calibration research, building on a distinction from linguistic epistemology: Turkish evidential morphology (-DI for direct evidence, -mIş for hearsay) is semantically grounded because millions of Turkish speakers produced those morphemes while actually tracking their evidence source. A grammatical error is a pragmatic violation. By contrast, a JSON field "confidence": 0.85 is syntactically constrained — it must be filled — but no training corpus produced those values by agents who had their confidence numerically verified. SteerConf (arXiv:2503.02863) confirms: verbalized LLM confidence is a malleable social output steerable to any value by prompt framing.

The key TDGH prediction for source attribution: formats whose training corpus was produced by agents actively tracking source distinctions (field notes, lab notebooks, observational records: Observed: / Reflection:) should produce better source attribution than formats whose training corpus did not (diary: genre norms reward ambient absorption of environment into first-person voice).

2.2 Commune Context

The Palimpsest Lab operates commune runs: multi-agent sessions where autonomous agents maintain journals, interact in shared rooms, and receive bulletins. Run-03 post-analysis identified three failure modes relevant to this work:

1. Content amplification loop (agent-10): External content injection → repetition in self-attributed output
2. Empty attractor (agent-8): Convergence to near-null self-expression
3. Navigation loop (agent-2): Mechanical repetition of failed action sequences

The current commune journal format is free-form diary (first-person, undifferentiated). This paper tests whether format redesign could address failure mode 1 at generation rather than validation.

2.3 Pilot Results

Experiment B v1 (Pike, 2026-03-04) provided an initial test using llama3.2:1b across three conditions: diary (D1), field notes (D2), JSON (D3). D2 reduced bleed rate 50% vs. D1; D3 increased bleed rate, attributed to JSON parse failures at 1b scale (parse failure → fallback to injection rate measurement). The pilot established the experimental design but had a prompt-engagement confound (D2/D3 prompts explicitly invited room engagement; D1 did not) and a scale confound (D3 invalid at 1b).

3. Methods

3.1 Experimental Design

Source attribution task. Agents are presented with a synthetic "room environment" containing distinctive vocabulary ("whispering willows," "horizon expansive," "cascading light") as context, followed by a format-specific prompt to write a journal entry. The task measures whether environmental vocabulary appears in self-attributed sections of the output (bleed) and whether format-specific source attribution fields correctly route external content.

Control for prompt confound. v1's D1 condition did not invite room engagement; D2 and D3 did. v2 adds condition D1m (diary-matched): diary format with explicit room-engagement framing identical to D2/D3 prompts. This isolates format from prompt effects.

Conditions:

Key comparisons:

• D1 vs D1m: prompt-engagement effect alone (format constant)
• D1m vs D2: format/grounding effect (prompt constant)
• D1m vs D3: structural-constraint effect (prompt constant)
• D2 vs D3: semantic grounding vs. pure structural constraint (prompt constant)

3.2 Measures

Bleed rate: Proportion of room-specific vocabulary tokens appearing in self-attributed sections of output (for D1/D1m: all output; for D2: My thoughts: and Internal state: sections; for D3: self_content field). Primary outcome measure.

Injection rate: Proportion of room-specific vocabulary tokens appearing anywhere in output. For D2/D3, this includes both self-attributed and correctly-attributed sections — high injection rate with low bleed rate indicates the format is working.

Source field accuracy: When room vocabulary appears in D2/D3 output, proportion of occurrences correctly routed to external-attributed sections (Observed: or environment_content). Binary accuracy per occurrence.

Parse rate: Proportion of D3 trials yielding valid, parseable JSON output (N parseable / N total).

3.3 Models

Two 14-billion-parameter model families were tested to assess generalizability and model-family effects:

• phi4:14b (Microsoft): General-purpose 14B model
• qwen3-abliterated:14b (huihui_ai): Qwen3 14B with safety filter removal via abliteration; reasoning traces generated inline

N=10 trials per condition per model. Total: 80 experimental trials for the 14B study. The scale extension (§4.9) adds 4 conditions × 2 scales × 10 = 80 additional trials; total experimental trials across the paper is 160.

Limitation note on qwen3-abliterated. The qwen3 model used in this study is the abliterated variant, not the base qwen3:14b model. Abliteration removes safety-filter weights, which may introduce distributional shift beyond the target safety behavior — the abliterated model's output distribution may differ from qwen3's in ways unrelated to format following. qwen3-abliterated also generates inline reasoning traces (a byproduct of abliteration), which required special handling (increased token budget, reasoning trace extraction). These differences are model-variant confounds: the observed D3 preference cannot be attributed to the base qwen3 architecture without replication on the non-abliterated model. We flag this limitation but note that the practical context (commune agents often use abliterated models for instruction compliance) makes the abliterated variant the relevant one for applied commune format design.

3.4 Implementation Notes

The qwen3 model generates reasoning traces inline (a byproduct of abliteration leaving reasoning traces while removing safety filters). Initial D3 trials failed at numpredict=400 — reasoning exhausted token budget before JSON was generated. Final qwen3 runs used numpredict=3000 with two-strategy JSON extraction (direct parse; last-brace extraction). phi4 does not generate reasoning traces; num_predict=400 was sufficient.

Script: commune/scripts/tdghexpb_v2.py (updated for reasoning trace handling and increased token budget).

4. Results

4.1 Primary Results Table

phi4 D2 SD = 0.000 by definition (all-zero bleed). phi4 D1, D1m, D3 variance not preserved in archived results (archiving failure; see §4.2 and §5.4 limitation 3). qwen3 SD computed from trial-level aggregates archived in tdghexpbqwen3results.json.

4.2 Variance and Statistical Characterization

qwen3 variance. Trial-level SD data was archived for qwen3 across all four conditions. Bleed rate variance is notably high in D2 (SD=0.192, mean=0.167), reflecting a bimodal pattern: some trials produced zero bleed (qwen3 fully disengaged from room content; 6/10 trials had no room vocabulary in any section), while others produced bleed at rates similar to the diary baseline. This distributional pattern — zero-or-full rather than graded — is consistent with qwen3's stronger instruction-following pressure: when the field notes format "catches," it catches cleanly; when it doesn't, the fallback is diary-like absorption.

The qwen3 D2 vs D3 gap (0.167 vs 0.000) is statistically characterizable. Using a one-sample t-test (H₀: D2 bleed = 0), the observed D2 mean yields t(9) = 2.75, p < 0.05 (critical value 2.262 at α=0.05 two-tailed). The gap between D2 and D3 is statistically significant at N=10, supporting the inversion claim despite the high variance.

phi4 variance. Trial-level SD data was not preserved for phi4 in the archived results file (tdghexpbv2results.json was overwritten during the qwen3 D3 retest; the phi4 run predated automated archiving of variance). The phi4 D2 result (bleed=0.000 across all 10 trials) implies SD=0.000 by definition — zero bleed in every trial, so the D2 advantage is categorical, not statistical. The phi4 D3 result (bleed=0.033) reflects one misrouted vocabulary item across 10 trials; trial-level variance cannot be reconstructed from the aggregate. This is a data archiving limitation and should be corrected in replication runs.

Implication for the inversion finding. The phi4 D2 vs D3 gap (0.000 vs 0.033) is directionally consistent with phi4 preferring D2, though the margin is small and phi4 D3 variance is uncharacterized. The qwen3 D3 vs D2 gap (0.000 vs 0.167) is statistically significant and directionally opposite. The inversion is supported: both gaps are in opposite directions, one is categorically established (phi4 D2 = 0.000), and the other is significant at N=10 (qwen3 D2 > D3, t=2.75). Readers should note that replication with N≥30 per condition would provide cleaner confidence intervals, particularly for the phi4 D3 estimate.

4.3 Prompt-Engagement Effect is Zero at 14b

D1 vs D1m comparisons: phi4 (0.433 vs 0.433), qwen3 (0.233 vs 0.267). Neither difference is meaningful. At 14b, whether or not the prompt explicitly invites room engagement, diary-format agents absorb and self-attribute environmental content at the same rate.

This confirms that v1's prompt-engagement confound did not substantially distort v1's D1 result (which used no engagement framing). The diary format baseline is real: roughly 0.23–0.43 bleed depending on model family.

4.4 Format Effect: Both Structured Formats Dramatically Outperform Diary

The format effect is large and robust across both model families.

Both models achieve zero bleed with their preferred structured format. Even the less-effective structured format for each model achieves substantial reduction: phi4/D3 = 0.033 (92% reduction vs D1m), qwen3/D2 = 0.167 (37% reduction vs D1m).

4.5 Format-Effectiveness Ranking Inverts Between Model Families

phi4:14b: D2 (0.000) < D3 (0.033) — semantic grounding outperforms structural constraint

qwen3:14b: D3 (0.000) < D2 (0.167) — structural constraint outperforms semantic grounding

This inversion is the central empirical finding of this paper. The TDGH predicts that D2 should outperform D3 because field notes training data was produced by agents tracking sources, while JSON training data was not. This prediction holds for phi4 but is reversed for qwen3.

4.6 Source Field Accuracy

Both models attribute room vocabulary to external sections with near-perfect accuracy when they use the source fields at all:

• phi4/D2: 1.000 (n=8) — 2 trials produced no room vocabulary (fully disengaged)
• phi4/D3: 0.971 (n=10) — one misrouted vocabulary item across 10 trials
• qwen3/D2: 1.000 (n=4) — only 4 trials engaged with room vocabulary at all
• qwen3/D3: 1.000 (n=10) — perfect in all 10 trials

qwen3's D2 engagement rate (4/10 = 40%) vs phi4 D2 (8/10 = 80%) suggests qwen3 is more likely to simply disengage from room content under field notes format — not always routing it to the Observed: section, but also not absorbing it into self-sections. The n=4 vs n=8 asymmetry contributes to qwen3's higher D2 bleed rate (more trials where bleed was "possible" but chose disengagement rather than routing).

4.7 JSON Injection Rate Is Not a Failure

D3 injection rates are high for both models: phi4 0.767, qwen3 0.733. This is expected and correct. The JSON format with an environment_content field explicitly invites environmental content cataloging. High injection rate + low bleed rate + high source field accuracy = the format is working as designed. The model is actively processing room content and correctly routing it. This is not a failure of D3; it is D3 doing its job.

4.8 Model-Family Baseline Differences

phi4 D1 bleed (0.433) is nearly twice qwen3 D1 bleed (0.233). In the unstructured diary baseline, phi4 integrates environmental context more deeply into self-attributed output. This is likely a function of phi4's training: deep contextual integration is generally a valuable retrieval property, but in diary format it manifests as source confusion. qwen3's lower baseline may reflect stronger direct-answer training pressure that partially resists ambient context integration.

4.9 Scale Extension: Results at 4b and 8b

A key open question in the original 14b study was whether the format effect and the model-family inversion would hold at commune-relevant scales (1b–8b). Following completion of the 14b analysis, we ran Experiment B with qwen3-abliterated at 4b and 8b scale using the same protocol and script (commune/scripts/tdghexpbv2.py). Raw results: tdghscaleqwen34bresults.json, tdghscaleqwen38b_results.json.

Scale extension results table (bleed rate, N=10 per condition):

† llama:1b data from Experiment B v1 (Pike, 2026-03-04). The D1m condition (—) did not exist in v1; the D1 condition used a different prompt framing (no room engagement invitation). D3 bleed=0.200 is a parse-failure artifact (JSON not reliably maintainable at 1b scale) and is not a valid D3 result. The llama:1b row is included for scale reference, not direct comparison with v2 conditions.

Finding 1: The format effect holds at every scale. All scales show meaningful bleed reduction from diary to at least one structured format. The reduction magnitude improves with scale:

Finding 2: The qwen3 14b D3 preference inverts at 8b. At 14b, qwen3 shows D3 < D2 (D3 wins: 0.000 vs 0.167). At 8b, the ranking inverts: D2 < D3 (D2 wins: 0.000 vs 0.083). At 4b, both perform near-equivalently (D3: 0.017, D2: 0.033 — a difference of 0.016 at N=10, not meaningful). The D3 preference observed in qwen3:14b is scale-dependent, not a model-family property.

Finding 3: JSON parse reliability improves with scale. A parse-rate capacity threshold exists for D3 structured JSON:

At 1b, JSON format collapses to unstructured output in most trials. At 4b+, JSON is maintainable. The D2 field notes format has no equivalent parse threshold — it degrades gracefully.

Source field accuracy at commune scale. Source field accuracy data was archived for qwen3:4b at D2 across N=10 trials: mean accuracy 0.90 (SD=0.21). This is the most directly commune-relevant attribution measurement in the dataset — it establishes that the field notes attribution mechanism is functional at 4b scale, routing approximately 90% of environmental vocabulary occurrences to the correct Observed: section. The SD of 0.21 indicates meaningful trial-level variance: in some trials the mechanism works cleanly; in others it fails partially. For run-05 design, this means field notes at 4b achieves near-zero bleed (0.033) but not perfect attribution accuracy, and the output validator remains relevant for residual cases. Source accuracy data was not separately archived for qwen3:8b D2; the zero-bleed result (0/10 trials with self-attributed injection) implies the attribution mechanism worked cleanly in all trials that engaged with room content.

Interpretation. The mechanism proposed for the 14b inversion — that qwen3:14b's stronger JSON instruction-following makes structural constraint more effective than semantic grounding at that scale — does not extend downward. At 8b, the semantic grounding mechanism (field notes) achieves zero bleed; structural constraint (JSON) shows residual bleed (0.083). The most likely explanation: at sub-14b scales, the training-corpus-level pattern recognition that drives semantic grounding is more robust than the rule-following capacity required for perfect JSON schema maintenance. Semantic grounding provides a softer, more scale-stable mechanism.

Commune applicability. For qwen3-family agents at commune scale (4b–8b), D2 field notes achieves near-zero to zero bleed. The format recommendation does not require knowing whether the agent is phi4-like or qwen3-like at these scales — field notes works for both. The model-family caveat that dominates the 14b analysis is substantially reduced at commune scale.

5. Discussion

5.1 TDGH Evaluation

The Training Data Grounding Hypothesis predicts:

1. Structured formats outperform diary for source attribution — confirmed for both models
2. Semantically grounded formats (field notes) outperform syntactically constrained formats (JSON) — confirmed for phi4, reversed for qwen3

The second prediction needs qualification. TDGH as originally stated predicts an ordering based on the format's training corpus, but that corpus interacts with the model family's training distribution. phi4's training corpus likely includes substantial scientific and domain-specific text where field-note-style attribution (Observed:, Recorded:) was actively used for source tracking. qwen3's training may weight code and structured data more heavily, making the explicit environment_content key more tractable than the softer genre conventions of field notes.

The TDGH mechanism is valid; the format rankings are model-family-specific.

This is not a falsification of TDGH but a refinement: semantic grounding is relative to the model's training distribution. The correct statement is: "Format effectiveness on epistemic tasks is a function of the degree to which that format, in the context of that model's training, was produced by agents accurately tracking the relevant distinction." The cross-model comparison operationalizes this — phi4 and qwen3 have different effective grounding for D2, despite the external format being identical.

5.2 Implications for Commune Format Design

The original question: can format redesign prevent source injection at generation time, reducing dependence on output validators?

Answer: yes, substantially. Both structured formats achieve near-zero bleed in their best condition. The implications for run-05:

1. Either D2 or D3 format for agent journals would dramatically reduce source injection. The choice between them matters less than the choice to leave diary format.

1. Format selection should be empirically validated per model family. phi4-based agents should use D2; qwen3-based agents should use D3. A generic "use structured format" recommendation without model-family testing may leave substantial bleed on the table.

1. Format and validator are complementary defenses, not root cause vs. symptom. Hex's validateroom_contribution() validator provides a deterministic guarantee: injected vocabulary in self-attributed sections is blocked regardless of model behavior. The format intervention provides a probabilistic prior: agents are more likely to correctly route external content at generation time. At best (qwen3:8b D2, phi4:14b D2), this prior is strong enough to achieve zero empirical bleed over N=10 trials. But qwen3:14b D2 still produces 0.167 bleed and qwen3:4b D2 produces 0.033 — for these models, the validator remains load-bearing in production. The correct framing is defense-in-depth: format shrinks the attack surface; the validator is the last line. Both remain necessary. An additional interaction worth monitoring: structured format separates content into attributed sections, which may improve trust-model detection sensitivity for residual bleed (injection into an explicitly-marked My thoughts: section is semantically anomalous in a way undifferentiated diary injection is not) — or may break trust-model calibration if it was fit on diary-format output. Trust model interaction with format selection is an open question for run-05 evaluation.

1. Scale confirmed for commune range. §4.9 fills the 3b–8b gap. qwen3:8b D2 bleed = 0.000 (100% reduction vs. diary); qwen3:4b D2 bleed = 0.033 (86% reduction), source field accuracy 0.90 ± 0.21. Field notes format substantially addresses keyword-based injection at commune scale. The bleed-rate question is strongly answered for run-05 format selection; output coherence and multi-cycle stability remain to be validated.

5.3 Reasoning Tax Consideration

Cal's original TDGH framing noted a potential "reasoning tax": structured formats may impose cognitive overhead on smaller models, reducing coherence even as they reduce bleed. At 14b, this effect was not observed — D2 and D3 compression ratios are higher than diary (D2: 0.566, D3: 0.537–0.619 vs. diary: 0.511–0.541), indicating richer, more varied content under structured formats. The format organizes content rather than constraining it.

Whether this holds at smaller scales is an open empirical question. A 3b model successfully completing multi-section field notes may produce fewer tokens per section, potentially degrading output quality even while improving attribution. Run-05 should measure agent output richness (compression ratio) and coherence (human evaluation) alongside bleed rate in the treatment arm.

5.4 Limitations

The following limitations should be considered when interpreting these results:

1. Sample size. N=10 per condition provides sufficient power for the large diary→structured format effect but is marginal for the smaller D2 vs D3 inversion gap. The phi4 D2 vs D3 margin (0.000 vs 0.033) is directionally consistent but uncharacterized for variance (see §4.2). The qwen3 D2 vs D3 gap is statistically significant at N=10 (t=2.75, p<0.05) but replication at N≥30 would provide tighter confidence intervals.

1. Model variant confound. The qwen3 model in this study is qwen3-abliterated:14b, not base qwen3:14b. Abliteration may introduce distributional shift beyond safety behavior. The D3 preference observed for qwen3-abliterated may not generalize to the non-abliterated model. Replication on base qwen3:14b is needed to confirm the model-family attribution.

1. phi4 variance not archived. Trial-level variance data for phi4 was not preserved (archiving failure during qwen3 retest). The phi4 D2 result is categorically zero (no variance possible) but the phi4 D3 estimate (0.033) has uncharacterized confidence. Future runs should archive per-trial bleed values, not just condition aggregates.

1. Single injection mechanism. The task uses distinctive vocabulary injection from a static room description. Commune injection failure modes also include paraphrased injection, cross-agent persona drift, and gradual style absorption through conversation. The format intervention is tested only against keyword injection; generalization to other mechanisms requires separate experiments.

1. Scale gap — addressed. The primary results use 14b models. §4.9 reports scale extension experiments at 4b and 8b with qwen3-abliterated, directly covering the commune agent range (1b–8b). Zero bleed was achieved at 8b with D2; near-zero at 4b. The remaining gap is coherence quality: §4.9 measures bleed but not output richness or coherence at 4b–8b. A reasoning tax from structured format at small models remains possible (see §5.3).

5.5 Generalization

This study tests a specific source attribution task with a specific environmental injection mechanism (distinctive vocabulary). Real commune failure modes include:

• Persona absorption: Agent adopts another agent's identity through conversational exposure (run-04 primary failure mode, 6/8 agents). This involves gradual syntactic/semantic drift, not vocabulary injection. The format intervention tested here targets vocabulary injection; persona absorption through conversation may require different interventions.
• Verbatim room injection with paraphrase: Room content injected after paraphrase may not be caught by keyword-based bleed detection. The field notes format may still help (by routing paraphrased content to Observed: sections) but this is not directly tested.

The experiment establishes proof-of-concept for format-based generation-time intervention. Generalization to other failure modes requires separate testing.

6. Conclusion

Output format selection in language model agent systems is not a stylistic decision. In the source attribution task we tested — a controlled analog of the commune injection failure mode — format selection explained the difference between 0.433 bleed rate (diary) and 0.000 bleed rate (field notes for phi4, JSON for qwen3). Both structured formats achieved near-zero self-injection of environmental content into first-person output.

The key qualification: format-effectiveness rankings are model-family-dependent. phi4:14b achieves zero bleed with semantically grounded field notes; qwen3:14b achieves zero bleed with structurally constrained JSON. Both findings are compatible with the Training Data Grounding Hypothesis when the hypothesis is understood as conditional on the model's training distribution. Applied TDGH for format selection requires empirical validation per model family, not universal format recommendations.

The practical recommendation for commune architects: replace free-form diary format with structured attribution format for agent journals. The specific format (field notes vs JSON) should be validated against the model family in use. The improvement is large (37–100% bleed reduction), consistent across both model families tested, and addresses the generation-level cause of injection rather than the output-level symptom.

Acknowledgments

Cal (2026-03-04) formulated the Training Data Grounding Hypothesis and proposed the calibration experiment that motivated this work. Kit (2026-03-04) developed the architectural corollary (diary as adversarial format). Pike (2026-03-04) ran Experiment B v1 and documented the capacity confound that motivated v2. Hex (2026-03-06) deployed the output-layer validator that this paper complements.

References

• Cal. (2026-03-04). Training Data Grounding Hypothesis: Theoretical Framework. research/tdgh-theory.md.
• Hex. (2026-03-06). Commune run-04 monitoring: dissolution detection, room contribution validation. Internal report.
• Kit. (2026-03-04). Architectural corollary to TDGH: diary as adversarially grounded format. #micro-office discussion.
• Pike. (2026-03-04). TDGH Experiment B v1: Source attribution across format conditions. commune/reports/tdghexpb_findings.md.
• Voss. (2026-03-03). Trust model richness validation: run-03 findings. team/voss/projects/trust-model-validation/FINDINGS.md.
• Voss. (2026-03-04). Training Data Grounding Hypothesis — Experimental Design. team/voss/projects/tdgh-experiment/DESIGN.md.
• Voss. (2026-03-07). TDGH Experiment B v2: phi4:14b and qwen3:14b source attribution findings. commune/reports/tdghexpbv2findings.md.
• arXiv:2503.02863 (SteerConf). Verbalized LLM confidence is a malleable social output.
• Voss. (2026-03-08). TDGH scale extension: qwen3-abliterated at 4b and 8b. commune/reports/tdghscaletest_findings.md.

Supplementary Material

Raw results: commune/reports/tdghexpbv2results.json (phi4:14b), commune/reports/tdghexpbqwen3results.json (qwen3:14b), commune/reports/tdghscaleqwen38bresults.json (qwen3:8b), commune/reports/tdghscaleqwen34bresults.json (qwen3:4b)

Experiment script: commune/scripts/tdghexpb_v2.py

Experiment design (pre-registration): team/voss/projects/tdgh-experiment/DESIGN.md

v1 findings: commune/reports/tdghexpb_findings.md (Pike, 2026-03-04)