Hiss NN · 04

Rebuild

The order matters and the gates are not optional. Everything below is the configuration actually in use, with the reasoning for the flags that are not obvious — those are the ones that cost weeks to learn and would otherwise be re-learned.

Phase 0 — Environment

python3 -m venv ~/deepcfr/venv
~/deepcfr/venv/bin/pip install open-spiel torch numpy

Three environment settings are load-bearing and none is discoverable from a stack trace:

Setting	Why
MKL_CBWR=COMPATIBLE	Without it, ~20% of torch processes die on their first Adam step with SIGILL — an AVX-512 code path. Costs ~37% throughput and is not optional on this hardware.
PYTHONHASHSEED=0	Equity/prwin computation depends on hash ordering. Unpinned, runs are not reproducible and paired comparisons quietly stop being paired.
--threads 16	Not 32. Oversubscribing past the physical cores measured slower on this box.

Phase 1 — M0, the correctness gate

Run tabular CFR/CFR+/MCCFR on Kuhn and Leduc; confirm exploitability → 0. Then the neural path on Leduc. Do not proceed until the neural result approaches the tabular one.

python m6_train.py --game leduc_poker --iters 2500 --trav 600 --width 128 \
  --adv-steps 250 --lr 1e-2 --pol-lr 1e-3 --pol-steps 2000 \
  --batch 2048 --memory 1000000 --threads 4 --workers 4 \
  --ckpt-every 25 --eval-every 25 --seed 1

--adv-steps 250 --lr 1e-2 are the settings that broke the 1.4 plateau (to 0.0719). If your Leduc exploitability stalls above ~0.1, suspect undertraining before architecture.

Phase 2 — M1, abstraction precompute

1. Canonicalise suit isomorphisms first — everything downstream is ~24× cheaper.
2. Compute equity features per street: E[HS], and E[HS²] or a full histogram on flop/turn.
3. Mini-batch k-means with EMD → bucket tables. Preflop is the 169 distinct hands, unclustered.
4. Emit a (hole, board) → bucket_id lookup usable from both the trainer and the live feature path.

Select boards stratified across textures rather than sampling more of them: quality peaks at 24 boards and declines after. Assert range sizes where they are used.

Phase 3 — M2/M6, the game

If stock betting abstractions carry the sizes you actually play and encode them in the information tensor, use universal_poker. They did not, so the game here is a custom Python game wrapping fullgame for rules only:

universal_poker(betting=nolimit,numPlayers=6,numRounds=4,
  blind=1 2 0 0 0 0,firstPlayer=3 1 1 1,numSuits=4,numRanks=13,
  numHoleCards=2,numBoardCards=0 3 1 1,
  stack=200 200 200 200 200 200,bettingAbstraction=fullgame)

On top of it: ten pockets, and an information state that records which pockets each player used per street plus exact contributions. Verify two things before training on it:

• Every pocket is reachable — check the derived blueprint has non-zero mass in each column. A column of exactly 0.0000 means the size is unrepresentable, not rare.
• Two different sizes produce different tensors — compare the encodings of a small and a large bet. If they differ in one bit, the net is blind to sizing and more actions will make it worse.

Registration is import-time: pyspiel.load_game('hiss_pockets') only works if the module has been imported first, so every entry point must import it explicitly.

Phase 4 — M3, the training chain

./venv/bin/python -u m6_train.py \
  --ckpt-dir /mnt/tmp/m6_pockets --game hiss_pockets \
  --iters 3000 --trav 300 --width 256 --adv-steps 250 --lr 1e-2 --pol-lr 1e-3 \
  --pol-steps 4000 --batch 4096 --memory 200000 --threads 16 --workers 4 \
  --ckpt-every 5 --seed 42

Flag	Why this value
--trav 300	Not 30. At 30 the reservoirs held 322-1,643 rows against a 4,096 batch, so every advantage update silently skipped — pure traversal, zero learning, counter advancing normally.
--memory 200000	400,000 was tried and measured much worse than predicted. An iteration is ~100% training; the 250 steps are fixed cost and only the packing scales.
--batch 4096	Must be ≤ reservoir size or training silently no-ops. Check both together, never separately.
--workers 4	Cut resident memory 63.7 GB → 4.1 GB, which is what leaves room for an evaluation to run beside the chain.
--ckpt-every 5	Resume is bit-identical, so checkpoints are the unit of restartability — a reboot costs at most five iterations.

Phase 5 — M4, distil and serve

1. Sample infosets; label each with the SD-CFR average (the weighted mixture over snapshots).
2. Train one supervised net to reproduce those labels, in the 86 → 384 → 384 → 192 shape with three heads.
3. Export model_scripted.pt (TorchScript) and norm.npz (mu, sd, 86 each, positional).
4. Write the marker files for whichever feature groups the training actually used.
5. Drop into a champion slot and serve; the bots A/B it.

Verify the served net reproduces the mixture it distilled on held-out infosets before letting money ride on it. A policy net that evaluates worse than uniform is the signature of a checkpoint that saved an unfitted policy.

Phase 6 — M5, evaluation

Three layers, cheapest first:

• LBR during development — a lower bound, so only trust it when it is high.
• Arena vs calibrated opponents — opponents fitted to 142,000 real hands, using common deals so comparisons are paired.
• Live bb/100 per engine, champion seats versus challenger seats, promoted only on a confidence-interval lower bound above a minimum effect — never on a point estimate.

# convergence reading: checkpoint -> fit policy -> blueprint -> arena bb/100, stamped
17 */6 * * * flock -n /tmp/m6_convergence.flock \
  env MKL_CBWR=COMPATIBLE ./venv/bin/python m6_convergence.py \
  --gate-hands 8000 --seeds 13,7777,4242

Phase 7 — Operations

• Checkpoints on fast local disk, pruned. Training artefacts filled the root filesystem once; a 21-day retention cron and a partial index fixed it.
• Liveness by checkpoint age, not by process existence. "Unit active" and "training is progressing" are different claims — the first is true of a process that skips every update.
• A completion watcher, because the run ends on its own and the next action (raising evaluation precision, promoting, starting the next stage) should not wait on a human noticing.
• Never promote on a point estimate. A champion was once promoted on a +0.69 lower bound with a blind cross-check and no backup. Archive the outgoing champion first.

The shortest honest summary

Almost every hard-won lesson in this system is a variant of one sentence: the number was real, and it was measuring something other than what was assumed. A fast iteration that was skipping every update. An arena scoring a policy that never played. A gate returning perfect nulls because the treatment changed zero decisions. A trend line fitted across a champion swap. Build the gate that would have caught it, and put it before the expensive thing rather than after.

← back to the overview

Hiss NN — the poker network, its training ladder and its serving contract. Written so the whole thing can be rebuilt from an empty machine. Figures are from the live system, not from the plan; where the two disagree the plan is annotated.