Models¶
Each model is a package under
iron/applications/,
built on iron/lm/, the library every model shares (see
AGENTS.md). Each model's README says what is
specific to it and where to get its weights. The tests find a checkpoint at
$IRON_EXAMPLE_WEIGHTS_DIR/<checkpoint>, default /srv
(iron.lm.testing.weights_dir).
| Model | Application | Checkpoint | NPU1 (Phoenix, Hawk Point) | NPU2 (Strix, Strix Halo, Krackan) |
|---|---|---|---|---|
| Llama 3.2 1B, 3B | llama | llama3.2-1b, llama3.2-3b |
Yes, a dispatch per step | Yes |
| Granite 4.2 3B | granite | granite-4.2-3b |
Yes, a dispatch per step | Yes |
| Granite 4.0-H 1B | granite_hybrid | granite-4.0-h-1b |
Yes, a dispatch per step | Yes |
| Gemma 4 E2B: text decoder, vision and audio towers | gemma4 | gemma-4-e2b-it |
Text decoder, a dispatch per step; towers untested | Yes |
| EmbeddingGemma 2 | embedding_gemma2 | embeddinggemma-2 |
Yes, a dispatch per step; the widest images are refused | Yes |
| Gemma 4 E2B in FastFlowLM | gemma4_engine | $FLM_MODEL_PATH (the tests default to fastflowlm) |
No | Yes |
For the iron/lm models (all but the FastFlowLM engine), every step of
the model's graph runs on the NPU: embeddings, layers, the LM head, and for
the text decoders the next token's draw (the Sample operator). Gemma 4's
image resize and patching, and its log-mel spectrogram, run on the NPU too.
The host:
- turns text into token ids and back;
- decodes an image file and packs its pixels, and pads an audio clip (clips past 30 s are truncated);
- draws the sampler's random numbers ahead of generation;
- writes each call's per-call values;
- on NPU1, or with
--no-full-elf, starts each decode step and reads its token back, where NPU2 runs the whole loop on the device; - prepares weights once at load (RoPE tables, layouts).
The FastFlowLM engine runs IRON's operators inside FastFlowLM's own runtime, which keeps its tokenizer, sampling, serving, and the vision and audio towers' own operators.
NPU1 has no full-ELF dispatch, so each step is its own xclbin dispatch and the host starts each decode step on the token the last one drew; NPU2 runs the decode loop on the device. Each model's README says what its NPU1 tests cover.
Running, testing and tuning a text decoder¶
What the text decoders (Llama, Granite and Gemma 4 above) share:
- Setup:
requirements_examples.txt, as installing says. XRT maps every device buffer locked, and a model's weights and caches are banks of at most 3.5 GiB (iron.common.image.allocator.BANK), soulimit -lmust cover one bank. - Run:
python -m iron.applications.<application>.model <directory or repository id> [--prompt TEXT], the directory holdingconfig.json, the checkpoint and the tokenizer;-vlogs each compile;--helplists the options, and Llama's README explains them. - Test:
pytest tests/applications/<application>/(ortests/applications/decoders/ -k <application>for the decoders that share one test) loads the model once, records its time to first token and tokens per second (CI publishes them, and the nightly run fails on a regression; see Performance Checks), and checks accuracy against transformers' logits recorded ingolden.npzbeside the checkpoint, and determinism.pytest <test> --record-goldens -p no:xdistrecords missing logits where torch and transformers are installed; they are not IRON's requirements. - Tune:
python -m iron.lm.tune <application> --checkpoint <directory>times each operator at every setting of its tunables and rewrites the application'sprofiles/<device>.json; review it withgit diffand commit it.