DIVERGE-LTM1: Latent Trajectory Marginalization

Status: closed negative after the frozen matched-fit gate.

Decision

JET1 failed before generalization: several simultaneously unlearned hard straight-through interfaces drove source grounding and algebra below chance. LTM1 changes the optimization boundary rather than repairing JET1. It trains a small factorized set of complete latent reasoning trajectories with an exact smooth sequence-level marginal objective. It never averages candidate state fields. Deployment commits to one complete trajectory before decoding.

LTM1 is a real-language development gate on the existing verified math/code/science/logic stream. It is not another register-board experiment, not a public product route, and not yet a DIVERGE architecture claim.

Architectural thesis

Let b binary source-conditioned fault lines define K = 2**b sticky assignments. For prompt x, each assignment a initializes one complete latent state:

z[a,0] = shared(x) + sum_j guarded_delta[j, a_j](x)
z[a,t+1] = T_theta(z[a,t], x)

T_theta is tied across recurrent steps and shared across assignments. The same assignment remains attached to a lineage for the entire trajectory. Every trajectory produces its own soft prefix and therefore its own complete response likelihood. No opcode, pointer, token, or state field is selected independently from another trajectory.

The training response is split deterministically into ordered contiguous reasoning chunks. Frozen token embeddings give each chunk a semantic target. Every latent step predicts the corresponding target, supplying dense smooth credit before any discrete commitment.

For complete-trajectory response energy E_a and source-only prior pi_a:

E_a = mean_token_NLL(y | x, z[a,T])
      + lambda_trace * ordered_trace_distance(z[a,1:T], y_chunks)

L_marginal = -logsumexp_a(log_softmax(pi)_a - E_a)

The log-sum-exp is over complete trajectories only. Hidden states are never averaged. At inference, argmax pi_a selects one prefix before autoregressive generation. No teacher response, answer label, verifier, host program, raw source reread, or external model is present at inference.

This differs from:

• JET1: no straight-through source/program/state interfaces;
• ordinary recurrence: several sticky complete trajectories rather than one;
• soft particle aggregation: no fieldwise or hidden-state mean at inference;
• best-of-N decoding: one model-owned latent lineage is chosen before text is generated;
• ordinary MoE: fault lines create a product of guarded state patches and the same assignment persists through recurrent computation.

The possible contribution is the conjunction of factorized sticky latent lineages, ordered trace-state supervision, and whole-sequence marginal credit. Every ingredient has adjacent prior art; novelty is unclaimed until matched transfer evidence exists.

Frozen implementation

• Backbone: pinned Qwen/Qwen3.5-0.8B revision 2fc06364715b967f1860aea9cf38778875588b17.
• Backbone training: rank-8/alpha-16 LoRA in the final four text layers; all non-LoRA tensors frozen.
• Fault lines: 2 binary variables, exactly 4 complete trajectories.
• Latent width: 384.
• Slots per trajectory: 8.
• Tied recurrent steps: 8.
• Attention heads: 8.
• Feed-forward multiplier: 2.
• Training-time trajectory mixture: exact log-sum-exp over all four complete candidates.
• Inference: source-prior MAP candidate, then one ordinary greedy decode.
• Trace targets: at most eight balanced contiguous response-token chunks; target vectors are detached means of the frozen input embeddings.
• Loss weights: trace 0.25, batch posterior-balance 0.01, and monotone halting 0.01.
• Optimizer: fused AdamW, LR 2e-4, betas (0.9, 0.95), weight decay 0.01, cosine decay, and gradient clipping at 1.0.
• No MEI/MQB/QTG/JET/HSC weights are loaded.

Frozen staged gate

Stage 0: mechanics

CPU tests must establish:

1. all 2**b assignments occur exactly once;
2. assignment identity is sticky across all recurrent steps;
3. prefixes and trajectory probes have stable geometry;
4. gradients reach shared, guarded, recurrent, prior, and output parameters;
5. sequence marginalization is finite and permutation-consistent;
6. inference selects one exact stored lineage, never a mean;
7. reset and lineage-swap controls change only the declared prefix;
8. checkpoint metadata reconstructs exact geometry.

Then run one two-update H100 smoke. Infrastructure-only faults may be repaired without changing the scientific contract.

Stage 1: bounded real-language fit

• Data: existing hash-audited V10 verified-priority stream v10_tokenbalanced_35m20c10s10p25t_4m_verified_r1.jsonl.
• Identical deterministic reservoir and row order for treatment and baseline.
• One seed: 2026080601.
• 100 updates, 16 selected rows, context 1,024.
• One LTM1 arm and one exact LoRA-only B1 control.
• Both arms receive the same 1,016 logical prompt/response token budget. LTM1 appends eight latent slots to reach 1,024 backbone positions; B1 receives no hidden prefix. Final NLL is compared with identical per-token weighting.

LTM1 qualifies for broad training only if:

• final token-weighted response NLL is no worse than B1;
• all 16 examples improve from update zero under teacher forcing;
• selected-trajectory trace cosine similarity is at least 0.90;
• at least two of four trajectory IDs are selected across the 16 prompts;
• every gradient and tensor remains finite;
• non-LoRA backbone tensors remain unchanged.

Failure closes LTM1 without a new seed, width, bit count, recurrent depth, trace weight, loss, schedule, layer count, or longer fit.

Stage 2: matched broad development

Only after Stage 1 passes:

• train one LTM1 and one B1 arm for 200 updates on the same V10 stream;
• match selected rows, row order, target tokens, context, optimizer, LR, LoRA geometry, and update count;
• charge LTM1's additional candidate FLOPs and wall time explicitly;
• evaluate both identically on the frozen 538-example development board: GSM8K 100, MATH-500 100, HumanEval 20, MBPP 20, GPQA 198, and BBH logic 100; code enters the five-domain macro as the HumanEval/MBPP mean.

Promotion requires all of:

1. at least +3.0 absolute five-domain macro over matched B1;
2. at least 15 additional solved examples;
3. improvement in at least three of five domains;
4. no domain regression greater than two points;
5. resetting the selected latent prefix loses at least three macro points;
6. forcing the lowest-prior complete lineage loses at least two macro points;
7. non-LoRA backbone hashes remain unchanged.

If treatment qualifies, run one parameter/FLOP-matched dense recurrent control and one unopened fresh verified board. If treatment fails, close LTM1 and do not reinterpret train fit, posterior entropy, or trajectory geometry as reasoning.

Stop rule

LTM1 is one bounded test of whether smooth complete-trajectory credit and trace-aligned recurrent state solve the optimization boundary exposed by JET1. It is not authorization for long continuation pretraining, public score routing, more candidates, annealed hard selection, alternate chunking, or a nearby schedule/loss variant. A negative result requires a different state or learning substrate.

Result

The frozen Stage 1 gate completed on 2026-08-06 and LTM1 is closed. Treatment job 743306, matched B1 job 743307, and B1 scorer 743308 all completed cleanly. Both arms used the same 16 rows, seed, 100 updates, and 223,200 logical response tokens. Every non-LoRA backbone tensor in LTM1 remained hash-identical.

LTM1 misses two conjunctive requirements. Its final NLL is 2.36 times B1's, and its selected-trace cosine is below the frozen 0.90 threshold. More importantly, final candidate-to-candidate cosine reaches exactly 1.0 while posterior entropy remains 1.385 nats, essentially ln(4). The four nominal lineages therefore converge to one shared response trace. Different source priors still select all four IDs across the tiny fit board, but those IDs no longer identify distinct hypotheses.

This localizes the failure. Smooth whole-trajectory marginal credit is learnable and avoids JET1's zero-gradient hard-interface collapse, but one teacher response per prompt supplies no information that could associate different fault-line assignments with different semantic alternatives. Generic balance or entropy regularization cannot create those alternatives. The next admissible substrate must present multiple complete, semantically different trajectories for the same prompt, carry independent verifier labels or contradiction evidence, and train a model-owned whole-lineage selector. It must not be an LTM1 diversity-weight, width, duration, seed, or schedule repair.

Stage 2 and its 538-example broad evaluation are canceled by the frozen stop rule. No public reasoning improvement is claimed.

Artifact SHA-256 values:

• LTM1 checkpoint: 0871c5825e7651282aacf709ec9a676863a6aa1a4cfc1eee254b2cf8647af19f
• LTM1 report: 20045826ea4d6e6c7abaf7cac6874e6a70ed8f752f8333649052348a2d468bd5
• B1 checkpoint: c099e16c7ec8f2df9f3fe9a68030ffa521f4c392410eb1885d7a7b8ec0529ce1
• B1 training report: 3ebb07c363c2a7fca8c1cd743c3f7dbd5ef700ec3976a7175aff32ff747ae4e8
• B1 fit score: 1fced5300959bb3d6de28ec491ff5fe9d998c7ef8f6e3fe36615492459aaddd0