Skip to content

IRON: Unlocking the Full Potential of NPUs

A close-to-metal Python API for AMD Ryzen™ AI NPUs.

IRON is an open-source & close-to-metal Python API enabling fast and efficient execution on AMD Ryzen™ AI NPUs. It relies on language bindings around the MLIR-AIE dialect.

🚀 Installation Set up the XDNA driver, XRT, and IRON itself on Ubuntu.

📖 Programming Guide Tiles, ObjectFifos, data movement, and building a design from scratch.

🤖 Models End-to-end language models running on the NPU, from Llama to Gemma 4.

📊 Benchmarks Operator throughput and latency, tracked over the project's history.

Key Features:

  • An operator library (GEMM, MHA, RMSNorm, RoPE, activations, ...) over mlir-aie's aie.iron, each operator one declared class
  • Graphs: a model's operators recorded once per input shape and compiled into one image, so intermediate values stay on the device
  • Autotuning of each operator's tunables into a per-device profile
  • End-to-end models on the NPU, from Llama 3.2 1B to a vision-language model (see Models)

Quick start

After installing IRON with its models extra, and given a Hugging Face token for meta-llama's gated repository:

from iron.applications.llama import Llama

model = Llama.load("meta-llama/Llama-3.2-1B")   # downloads, fits, compiles once
print(model.generate("The capital of France is", 8))   # greedy; pass an iron.lm.Sampler to sample

Llama.load also takes a local directory holding config.json, the checkpoint and tokenizer.json.

The IRON Python API for Ryzen™ AI NPUs is described in the following paper:

E. Hunhoff, J. Melber, K. Denolf, A. Bisca, S. Bayliss, S. Neuendorffer, J. Fifield, J. Lo, P. Vasireddy, P. James-Roxby, E. Keller. "Efficiency, Expressivity, and Extensibility in a Close-to-Metal NPU Programming Interface". In 33rd IEEE International Symposium On Field-Programmable Custom Computing Machines, May 2025.