Strategy — two tracks, and why

The Psi mission split cleanly into two complementary tracks. This repo is Track B.

Track A — the custom stack (Psi, the crown jewel)

A fully custom C++/CUDA/Metal SLM stack (own autograd, own GEMM kernels, no PyTorch), whose north star is capability-per-BIT and whose signature move is ternary (~1.58-bit) weights on a hand-written kernel. Its crown jewel is the sub-1M TinyStories record (smallest model to clear the TinyStories coherence bar). This is the novelty / craft track — nobody else hand-writes a ternary GEMM and trains a from-scratch model to a capability bar. Compute-light, laptop-native, differentiated.

Result so far (the crown jewel campaign): a sweep of 115K–1.2M models on the custom CUDA backend, graded on an LLM rubric — e.g. femto (115K) grammatical-but-drifts, nano (215K) close-on-grammar; the record candidates (small 354K / mid 574K) landing the smallest-that-clears result.

Track B — this repo (Shohin, the reasoning specialist, PyTorch)

The best sub-200M reasoning model of 2026 — verifiable reasoning (math / code / logic), English-only — built pragmatically in PyTorch to win an uncrowded benchmark race. Uses the levers the custom stack can't deliver quickly: short-CoT reasoning distillation + rejection sampling from a close ~1–2B teacher, a reasoning-tilted data diet, bf16 / multi-GPU throughput. See PLAN.md.

Why separate — and the hybrid endgame

Forcing the model onto the custom stack would cost ~8–14 person-weeks of infra (bf16/tensor-core GEMM, multi-GPU DP, trace-generation pipeline) before a competitive model trains, and fp32-no-tensor-cores means 5–15× less throughput. So: PyTorch for capability here, custom stack for craft there. The hybrid payoff: train + win the reasoner in PyTorch, then ternary-QAT the winner on the Psi C++ kernel — a world-class-per- bit small reasoner no framework ships. That aims the from-scratch investment at its actual edge (inference-time capability-per-bit), orthogonal to the benchmark race.

The pivot on this repo (recorded honestly)

Shohin started as a commonsense generalist (beat SmolLM2-135M on HellaSwag/PIQA/etc.). We reversed it to a verifiable-reasoning specialist after two realizations:

1. Reasoning is the right axis for a tiny model. Knowledge is storage-capped (~2 bits/param); reasoning is an algorithm and far less capacity-bound. Specializing where a 100M model can actually improve beats spreading thin across axes physics has already capped.
2. The niche is open. Sub-200M reasoning is a one-horse race (MobileLLM-R1-140M); sub-200M commonsense is a crowd.

The cost of the pivot, stated plainly: we trade commonsense breadth for reasoning depth (MobileLLM-R1's commonsense trailed SmolLM2). Commonsense drops to "no catastrophic regression." We don't get both at 130M.

Broader research findings that shaped this (from grounding the plan, mid-2026)

• Short CoT > long CoT for tiny students (learnability gap, 2502.12143) — the central lever; a 130M model can't represent long R1 traces anyway, so short-CoT is the recommended regime, not a compromise.
• RL is not the reasoning engine ≤200M — SFT/distillation beats RL at 950M (74.0 vs 57.0); verified RLVR floor ~0.5B. RL is optional polish. (Reverses our initial RL-first hunch.)
• Compact vocab > teacher's big vocab — because we trace-distill (no shared tokenizer needed), a ~32k English+math+code vocab frees ~60M params for reasoning instead of an embedding table. A concrete edge over MobileLLM-R1's 128k vocab.
• MoE — no (trades stored bits for FLOPs; dense optimal <500M). Knowledge — no (physics). Tool-use — a product feature, not a reasoning claim (never in the graded number).
• Looped / latent reasoning — the most interesting novelty bet (depth without params), unproven ≤200M; a stretch prototype, not plan-of-record.

Immediate next steps (see README.md)

1. Pin the reasoning eval harness (GSM8K, MATH-500, HumanEval, MBPP, a logic set); re-run MobileLLM-R1-140M.
2. Reasoning-tilted ~130M base (~100–150B tok, compact vocab) → a good reasoning substrate + working infra.
3. The decisive A/B: short-CoT vs long-CoT distillation from the ~1–2B teacher; attribute the lift.
4. In parallel: secure 8× H100 (PI email for highgpu, or cloud) — see COMPUTE.md; prototype looped-reasoning (stretch).