GEN1 — onnxruntime-genai on riscv64

GEN1 — onnxruntime-genai on riscv64#

The riscv64 cross lane builds onnxruntime-genai from source. Nobody else does: upstream ships no riscv64 wheel in any version, runs no riscv64 CI (a full-tree grep of v0.15.2 finds zero riscv matches), and closed its one RISC-V field report as not-planned. This page is the engineering reference for that lane — what was changed, what each gate proves, and what to watch when the first real media-riscv64 build finally runs it.

See also — this page is the how it works. The other two halves: the validation watch was item A1, closed 2026-09-03 and archived in refactoring-backlog-archive-2026-09-03.md; refactoring-backlog-archive-2026-08-31.md and ../CHANGELOG.md (2026-08-31 closure window) are the history. The one-screen summary lives in linux-cross-builds.md.

Status: VALIDATED (2026-09-03)#

Wired ON 2026-08-31 by explicit user decision. Both open questions are now closed by measurement:

  • compiles — proven by the 2026-08-31 media-riscv64 stage (run r4).

  • generate() produces sane tokens — proven 2026-09-03; see Tier 4, measured.

The negative data point that motivated the doubt, onnxruntime-genai#594, is a RISC-V build (Phi-3 int4 on a LicheePi4A / XuanTie C910) that compiled, imported, and then emitted nonsense; it was closed as not-planned with no root cause. Tiers 1–3 of this lane’s smoke pass in exactly that state — see The smoke — which is why tier 4 exists at all.

The backlog carried this as GEN1 (“upstream ships none; IREE-style build plausible”, effort L) until the user asked for the lane to be on. The actual effort was far below that estimate, for the reason in the next section.

The lane at a glance#

File

Role in GEN1

linux/scripts/01-core/versions.env

Declares GENAI_ALLOW_RISCV64=true (a feature toggle, not a version pin)

linux/Dockerfile.media

Matching ARG GENAI_ALLOW_RISCV64=true → ENV on the onnxruntime builder stage; mounts linux/scripts/patches into the genai RUN

…/03-media/build/onnxruntime/build/60-build-genai.sh

Producer: escape hatch, patch apply, --use_guidance preflight, cross allowlist arm64|riscv64, cross-wheel ELF/EXT_SUFFIX assert

linux/scripts/patches/onnxruntime-genai/001-riscv64-target-platform.patch

The one upstream blocker, one hunk

…/03-media/runtime/verify-media-artifacts.sh (onnxruntime-genai)

Build-time artifact gate; reads the producer’s .gen1-lane-off marker

…/03-media/verify-genai-ort.sh

ORT provenance, in the same verify RUN: ORT_HOME in the top-level CMakeCache is a chain root, no _deps/ortlib-*/onnxruntime-*, no ORT archive, chain bytes under every chain ORT name, then the ORT build gate and its stamp. SKIP when the lane is off

…/03-media/runtime/validate-media-runtime.sh

LIB_DIRS root + unresolved-NEEDED scan over the genai lib dir

…/03-media/runtime/assemble-torch-app.sh

prune_conflicting_onnx_wheels — no longer deletes the CPU genai wheel

…/06-packaging/smoke-common.sh (smoke_genai_py)

The four-tier binding / generate() smoke payload

…/06-packaging/smoke-runtime-image.sh

check_genai_binding, _rt_versions_env_pin, the ARCH-PARITY table, the app-wheel floor

…/06-packaging/smoke-torch-venv.sh

The riscv64 genai pin policy (absence is now a FAILURE)

linux/scripts/tests/test-smoke-arch-parity.sh

Pins the new contract in both directions

Why the rest of the cross path needed no change#

The arm64 cross path added by GENAI-DRIFT (2026-08-24) was already fully arch-generic. It reads every architecture fact from platform.sh, and platform.sh already mapped riscv64 completely:

What the cross body reads

riscv64 value

CROSS_TARGET_TRIPLET

riscv64-linux-gnu

CROSS_TARGET_PROCESSOR (arch_cmake_system_processor_for)

riscv64

CROSS_RUST_TARGET

riscv64gc-unknown-linux-gnu

cross_wheel_platform_tag

linux_riscv64

arch_elf_machine_grep_for

RISC-V

cross_target_python_include_dir

target Python headers

Three further facts made the lane plausible without new machinery:

  • The ORT it links against is already real on riscv64. 30-build-native.sh has no arch skip, and verify-media-artifacts.sh onnxruntime-cpu hard-gates libonnxruntime.so* plus the C API header on every arch, so NATIVE_CPU_OUTPUT_DIR is populated before the genai stage runs.

  • No SIMD/intrinsic blocker in the v0.15.2 tree. The only _mm_ intrinsic is src/dml/dml_gpu_event.h (Windows/DML); the only __x86_64__ / __aarch64__ switch is src/telemetry/device_info.cpp, whose whole TU is telemetry-gated and --no_telemetry is passed on every arch; onnxruntime-extensions’ ocos_target_platform is MSVC-only; and dlib is pulled in with SOURCE_SUBDIR not_set, so its own SIMD detection never runs.

  • The cross allowlist is an explicit two-arm list, arm64|riscv64, not a blanket “any cross target”. Nothing else is validated, and an open allowlist would quietly have let a 386/powerpc target through.

Keep the two allowlists in lockstep. 60-build-genai.sh’s case "${ARCH}" in arm64|riscv64) and verify-media-artifacts.sh’s onnxruntime-genai arm are each other’s only cross-check. Change one, change the other — otherwise either the producer builds something nothing verifies, or the verifier demands something the producer never built.

The upstream patch: cmake/target_platform.cmake#

linux/scripts/patches/onnxruntime-genai/001-riscv64-target-platform.patch. One hunk, applied only when ARCH=riscv64.

Why configure FATAL_ERRORs on riscv64#

onnxruntime-genai v0.15.2’s cmake/target_platform.cmake, Linux branch (:53-64), matches only ^arm64 / ^aarch64 / ^(x86_64|amd64)$ / powerpc, and otherwise:

message(FATAL_ERROR "Unsupported architecture. CMAKE_SYSTEM_PROCESSOR: ${CMAKE_SYSTEM_PROCESSOR}")

It is reached from CMakeLists.txt:45 → cmake/global_variables.cmake:118 — i.e. during configure, before a single object is compiled. This lane passes CMAKE_SYSTEM_PROCESSOR=riscv64, so without the patch the stage dies in seconds.

The patch adds one elseif arm:

elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^riscv64.*")
  set(genai_target_platform "riscv64")

Why the added arm is inert everywhere else#

genai_target_platform is dead weight on this lane. A full-tree grep of v0.15.2 finds exactly three consumers, all unreachable here:

Consumer

Why it never fires

src/python/CMakeLists.txt:97

guarded by USE_WINML AND WIN32

src/java/CMakeLists.txt:172-174

ENABLE_JAVA=OFF

cmake/msvc_security.cmake:63

MSVC only

So the patch is semantically inert everywhere except at the FATAL_ERROR it removes.

The apply is gated on ARCH=riscv64 so amd64/arm64 never even invoke apply-patch.sh: their genai source tree stays byte-identical to upstream. Precedent for a riscv64-only patch applied exactly this way: linux/scripts/patches/libcamera/001-apps-add-libtiff-dependency.patch, from build-libcamera.sh.

How it was proven to apply — MEASURED#

Applied, and re-applied as a no-op, against a real clone of the pinned tag v0.15.2 (ed5f4e87). apply-patch.sh is idempotent by construction: it first tries a reverse-apply --check (already applied → SKIP), then a forward --check (→ APPLY), and errors loudly if neither works.

This is the only part of the patch story that is measured. That it makes the build succeed is not.

The rejected alternative: lying about CMAKE_SYSTEM_PROCESSOR#

Passing aarch64 for a riscv64 target would slip past the gate for the same reason the patch is inert (Rust_CARGO_TARGET is explicit, ocos_target_platform is MSVC-only). It was not shipped because it is:

  • invisible in build logs;

  • silently rotting on any upstream bump that starts reading the variable;

  • directly at odds with the arch asserts this repo depends on (assert_elf_arch, the EXT_SUFFIX check).

The patches bind-mount#

The genai RUN in Dockerfile.media gained --mount=type=bind,source=linux/scripts/patches,target=/opt/scripts/patches. apply-patch.sh itself already arrived via the whole-01-core mount; only the patch tree was missing. amd64/arm64 never read it. It does move that layer’s cache key — but so does any edit to 60-build-genai.sh, since the whole onnxruntime script dir is already mounted there. That dir is also mounted whole into the GPU ORT RUN, so an edit to 60-build-genai.sh re-keys the GPU ORT build on nvidia/amd too; that is why verify-genai-ort.sh sits at the 03-media root, mounted per file, and not beside it.

The producer hard-errors if the patch file is not found, naming the mount as the likely cause.

--use_guidance on riscv64#

Decision: keep it on. --use_guidance builds llguidance (pinned 94fa3912 in cmake/deps.txt) through Corrosion — a real cargo build for the target triple.

The evidence behind keeping it#

  • riscv64gc-unknown-linux-gnu is a rustc Tier-2-with-host-tools target, so rustup target add delivers a prebuilt std.

  • This repo does add it: 02-toolchain/install-rust.sh add_rust_target runs over for_each_cross_target --include-amd64 with CROSS_TARGETS, whose Dockerfile.toolchain default is amd64,arm64,riscv64.

  • The crate graph Corrosion actually imports (parser/Cargo.toml: toktrie, derivre, serde, anyhow, regex-syntax, indexmap, ahash) is pure Rust — no cc crate, no ring, no openssl-sys, no pyo3. Those enter the workspace Cargo.lock only via python_ext / sample_parser / toktrie_hf_*, which this import does not build. ahash falls back to its portable path without AES.

Dropping the flag would also be the wrong trade: guidance is what constrains generation, so a riscv64 wheel without it is a different wheel from the one amd64/arm64 ship — and the ARCH-PARITY table could not tell the difference, because it matches on the distribution name only.

The riscv64-only preflight#

The one way this can still bite is a missing rustup std for the triple, where Corrosion FATALs and kills a multi-hour stage. So the producer runs a riscv64-only preflight that drops --use_guidance on positive evidence of absence only:

  • target triple from CROSS_RUST_TARGET, else rust_target_triple_for_arch;

  • rustup target list --installed must run and produce output and not contain the triple → drop the flag with a loud warn;

  • a missing rustup, or a rustup that cannot list, leaves the flag on.

That is the same “only a positive answer flips the verdict” rule the runtime smokes use. On amd64/arm64 it is a literal no-op: GENAI_GUIDANCE_ARGS expands to --use_guidance unless the riscv64 branch fires.

It does not catch a link failure — only a missing std. See the watch list.

Corrosion cross wiring, for reference#

Two things in the cross define set exist for llguidance:

  • Rust_CARGO_TARGET=${CROSS_RUST_TARGET} — Corrosion passes an explicit --target to cargo, which would otherwise silently override the CARGO_BUILD_TARGET exported by setup_linux_cross_env. Corrosion also wires CARGO_TARGET_<T>_LINKER to CMAKE_C_COMPILER (the cross gcc) itself.

  • unset LIBRARY_PATH around the build.py invocation — the HOST-LINK LEAK found on the arm64 lane’s first live run (wave7b, 2026-08-24): setup_linux_cross_env exports LIBRARY_PATH with the TARGET libdirs, and the host cc honours it for -m64 links too, so cargo’s host build scripts (proc-macro2, anyhow, zerocopy, rustversion…) died with “aarch64 libgcc_s.so.1 is incompatible with elf64-x86-64” on every retry.

The cross-wheel gate in the producer#

On a cross build the wheel is the whole point of the stage, so a build.py that “succeeded” without emitting one must fail in 60-build-genai.sh — not resurface hours later at app assembly as a silent fallback to the app lock’s PyPI genai (the GENAI-DRIFT bug). There is deliberately no host pip wheel fallback: that would package an amd64 binding under a foreign platform tag.

The gate then proves the shipped bytes, not the build log:

  • every *.so / *.so.* inside the unpacked wheel goes through assert_elf_arch … "${ARCH}";

  • the pybind module must be named onnxruntime_genai${_GENAI_MODULE_EXT}, i.e. carry the target EXT_SUFFIX.

riscv64 flows through this unchanged, and that is the point. assert_elf_arch resolves riscv64 to the ELF machine string RISC-V through platform.sh arch_elf_machine_grep_for, so the existing gate proves the riscv64 bytes with no new code.

It is no longer the only net — verify-media-artifacts.sh’s onnxruntime-genai arm used to SKIP every non-arm64 cross target and now gates the riscv64 one too — but it is still the first and the most specific one.

The escape hatch: GENAI_ALLOW_RISCV64#

One knob turns the whole lane off and restores the pre-GEN1 behaviour exactly — 60-build-genai.sh creates its placeholder output tree and exits 0, and every downstream gate agrees. No code edit required.

It is deliberately riscv64-scoped. The pre-existing BUILD_GENAI switch is all-arch, so backing a riscv64 breakage out with that would also drop the shipped amd64/arm64 genai wheels.

Where it is declared, and how it reaches each consumer#

Consumer

How it gets there

Default when unset

versions.env

the declaration: GENAI_ALLOW_RISCV64=true

—

Dockerfile.media

ARG GENAI_ALLOW_RISCV64=true → ENV on the onnxruntime stage. The ARG literal must equal the versions.env value (verify-arg-consistency.sh enforces it)

—

60-build-genai.sh (producer)

reads the ENV

false — conservative, for standalone runs

verify-media-artifacts.sh (verifier)

reads the ENV and the marker file

false (fallback only)

smoke-torch-venv.sh (in-container)

passed in with -e by check_ml_version_pins

true — armed

smoke-runtime-image.sh (host)

_rt_versions_env_pin GENAI_ALLOW_RISCV64 reads versions.env host-side

empty = not asserted

The script-side conservative default follows the shape of ORT_ENABLE_WEBGPU / ORT_WEBGPU_ALLOW_CROSS: scripts default off for standalone runs, versions.env opts the orchestrated image builds in. Concretely, 60-build-genai.sh reads ${GENAI_ALLOW_RISCV64:-false} so that a bare bash 60-build-genai.sh against a riscv64 target cannot walk into this multi-hour build by accident (it was also still unvalidated when the default was chosen). The same reasoning is why GENAI_ALLOW_RISCV64 sits in verify-arg-consistency.sh’s SCRIPT_DEFAULT_DRIFT_ALLOW: the script default is meant to differ from the versions.env pin.

Why the ARG is promoted to ENV, not just read from versions.env. Two different RUN steps need the value — the producer and the verifier — and the verifier RUN (verify-media-artifacts.sh onnxruntime-genai) bind-mounts nothing but a tmpfs /tmp: no linux/scripts/01-core, so it cannot load_versions_env the file at all. A stage-level ENV is the only channel that reaches both. That is also why the ARG literal and the versions.env value must be kept equal by hand — verify-arg-consistency.sh is what catches them drifting.

This toggle is LINUX lane only: no Windows build consumes it (same as ORT_ENABLE_WEBGPU / ORT_WEBGPU_ALLOW_CROSS, and consistent with the repo-wide rule that the Windows lane carries its own separate backlog).

Two defects in this wiring were found in review and fixed the same day:

  1. It never reached the in-container smoke. nerdctl run inherits nothing from the host, and GENAI_ALLOW_RISCV64 is ARG/ENV only on the onnxruntime builder stage — the media final stage is FROM media-inputs, so the shipped image does not carry it. The baked smoke-torch-venv.sh therefore always saw the toggle unset, its riscv64 arm defaulted to “genai expected”, and the documented back-out still red the gate. check_ml_version_pins now forwards it explicitly with -e, reading the value out of versions.env host-side, so the smoke asserts the policy the build was configured with.

  2. Producer and verifier defaults disagreed in the failing direction (:-false vs :-true). With the ENV out of scope, the producer skipped and the verifier then hard-failed on the placeholder tree the producer had just created. Fixed by the marker, below.

The .gen1-lane-off marker#

When the producer takes the escape hatch it writes an empty file ${GENAI_OUTPUT_DIR}/.gen1-lane-off beside the placeholder tree, and the verifier prefers that file over re-deriving the policy.

The marker exists because the producer and the verifier run in different RUN steps, so re-deriving the same decision twice from an environment variable is a decision that can differ. Recording it as a file makes the verifier read what actually happened rather than what it thinks should have happened. The env check is kept only as a fallback for a tree written before the marker existed, and now uses the same default as the producer.

Hard requirement on the hatch: turning the lane off must not turn the build red. That is what all four toggle states are checked against.

The four toggle states#

Lane

Producer

Verifier

smoke-torch-venv

riscv64, true

cross-builds the wheel, asserts ELF + EXT_SUFFIX

gates for a real artifact

genai required

riscv64, false

placeholder tree + .gen1-lane-off, exit 0

SKIP (marker)

~~ not installed … policy, not drift

amd64/arm64, true

unchanged

unchanged

genai required (as before)

amd64/arm64, false

unchanged — the toggle is riscv64-scoped

unchanged

genai still required

test-smoke-arch-parity.sh pins the last row explicitly: the hatch must not silence amd64.

The smoke: smoke_genai_py#

Lives in smoke-common.sh beside the ONNX Add-graph fixture, for the same reason — one source of truth, injected as an env var so the check also gates images built before it existed. Driven by check_genai_binding in smoke-runtime-image.sh, which runs on every arch: riscv64 motivated it, but a host-suffixed or wrong-ELF binding is the same defect class on amd64/arm64.

Why an import check would not have been enough: issue #594 is a build that compiled and imported and then emitted nonsense. “The wheel is installed and imports” is precisely the state that bug was in — the same toothless class as the onnxruntime import check this file’s ONNX section replaced in wave 5.

Inputs#

Env var

Meaning

GENAI_EXPECT_VERSION

ONNXRUNTIME_GENAI_VERSION from versions.env; empty → version tier reports NOT ASSERTED

GENAI_EXPECT_ARCH

amd64 / arm64 / riscv64; empty → ELF tier reports NOT ASSERTED

GENAI_MODEL_DIR

optional real model directory — arms tier 4

GENAI_MAX_LENGTH

tier-4 max_length, default 16

The four tiers#

Tier

What it does

What it PROVES

1 — version

og.__version__, falling back to importlib.metadata

the installed genai equals the versions.env build pin

2 — ELF

reads 20 bytes of the loaded extension’s header (og.onnxruntime_genai.__file__) and compares e_machine

the bytes actually executing are TARGET-arch — read from the file header, not inferred from the wheel tag

3 — native code runs

pybind API objects exist; capability predicates (is_cuda_available/is_webgpu_available/is_qnn_available) execute; og.Tensor(numpy) round-trips a float32 buffer through OgaTensor; og.Config() on a non-model path is rejected by the library

native C++ code runs, dtype/shape mapping and the copy-back through the C API work, and the binding’s ABI matches

4 — generate()

encodes a prompt, generates, asserts on token content

that the model emits more than zero tokens and not one token repeated — the #594 nonsense signature

e_machine values used: amd64 62 (X86-64), arm64 183 (AArch64), riscv64 243 (RISC-V), 386 3. Read from the header rather than shelling out to readelf/file: neither is guaranteed in a runtime image, and the header is 20 bytes of well-specified wire format.

That header read is the only I/O in the whole program, and it is wrapped in a try/except OSError for a specific reason: every other step degrades to an UNPROVEN line or appends to fails, so an uncaught OSError here would be the one path that exits non-zero having printed no GENAI-BIND sentinel at all. check_genai_binding’s no-sentinel arm would then misdiagnose an unreadable extension file as “the generated program did not run” — on every arch. Caught, it becomes an ordinary DEFECT line.

Tier 3’s og.Config() check discriminates where the rejection came from: a TypeError/AttributeError means the binding refused it — a signature/ABI mismatch, a FAIL. Any other exception (or none) means native code ran and made the decision.

The program prints PROVEN / UNPROVEN / DEFECT lines itself, so a log reader never has to take the accounting on trust.

Tier 4 is UNARMED by default#

No model ships in these images — the smallest usable decoder is hundreds of MB, and the riscv64 lane is cross-built on an amd64 host, so it could not run one at build time anyway. Without GENAI_MODEL_DIR the tier is skipped and says so (GENAI-GEN SKIP:). Arm it by mounting a small model and setting GENAI_MODEL_DIR.

The app-wheel floor#

smoke-runtime-image.sh ratchets the orchestrant.smoke ok-count per arch. It may only ever be raised when an arch gains a component, never to turn a red run green.

arch

floor

last printed

why not tighter

amd64

15

15

at the observed value already

arm64

14

15 (2026-09-03)

one sample; 15 would leave zero slack

riscv64

13

14 (2026-09-03)

raised 12 → 13, keeping one wheel of slack

The per-arch counts only started being printed in the 2026-09-03 run, so every arch has a single observation. A floor set to exactly one observed count fails the next run if any single wheel is legitimately optional on that arch — which is why arm64 was left alone and riscv64 stopped a wheel short of what it printed. Raise arm64 to 15, and riscv64 to 14, once a second run confirms the count.

Tier 4, MEASURED 2026-09-03#

Run by hand against the shipped images, not by the build. The result is stronger than “sane tokens”: riscv64 greedy decoding is token-for-token identical to amd64.

amd64 (control)

riscv64 (qemu-user)

native ext e_machine

62 (X86-64)

243 (RISC-V)

first 8 new token ids

28, 198, 198, 57, 5248, 18948, 346, 2316

identical

decoded

,\n\nI'm glad you're enjoying the experience. I hope you find it as enjoyable as I

same prefix (shorter max_length)

verdict

GENAI-BIND OK rc=0

GENAI-BIND OK rc=0

Identical ids across two unrelated backends is a much tighter check than the smoke’s own degeneracy test (len(set(new)) == 1, the #594 signature): a kernel wrong enough to matter would have to be wrong identically on both. #594 does not reproduce on this lane.

Reproducing it#

The fixture is a real trained model, not a random one — see Why not a tiny random model.

# 1. fetch HuggingFaceTB/SmolLM2-135M-Instruct into ./smol-src (public, no token)
# 2. convert to GenAI format on amd64 (needs transformers, onnx, onnx_ir):
python -m onnxruntime_genai.models.builder -i ./smol-src -o ./smol-genai -p fp32 -e cpu
# 3. emit the smoke payload and run it on each arch:
bash -c 'source linux/scripts/06-packaging/smoke-common.sh; smoke_genai_py' > genai_smoke.py
nerdctl run --rm -v $PWD/smol-genai:/model:ro -v $PWD/genai_smoke.py:/smoke.py:ro \
  -e GENAI_MODEL_DIR=/model -e GENAI_MAX_LENGTH=16 -e GENAI_EXPECT_ARCH=riscv64 \
  --entrypoint /opt/venv/bin/python <image>:latest-riscv64 /smoke.py

The builder passes token=True to huggingface_hub, so -m <repo-id> fails with LocalTokenNotFoundError even for a public repo. Download the files yourself and use -i <dir>.

Why not a tiny random model#

The obvious fixture, hf-internal-testing/tiny-random-LlamaForCausalLM, is wrong twice over, and both were hit:

  1. hidden_size=16 / 9 heads → head_size=4, and ORT’s GroupQueryAttention requires head_size % 8 == 0. It fails on amd64 too: head_size must be a multiple of 8. Got head_size % 8 == 4.

  2. More subtly — an untrained model has near-uniform logits, so greedy decoding tends to emit one repeated token. That is precisely the #594 signature tier 4 flags as a DEFECT, so a random fixture would manufacture a false positive on every arch, riscv64 included.

Always run the amd64 control first. Here it is what identified the fixture as the broken part rather than the lane.

Installed-but-unimportable is a hard FAIL, not a SKIP#

The exit codes:

rc

Meaning

Driver’s verdict

0

ran and passed (GENAI-BIND OK:)

PASS

1

a real defect (GENAI-BIND FAIL:)

FAIL

3

genai legitimately absent (GENAI-BIND SKIP:)

SKIP, not a pass

4

SMOKE_GENAI_PY never crossed the container boundary (GENAI-BIND ABSENT:)

FAIL

An import failure has two very different causes and they must not share an exit code:

  • no wheel on this arch — benign; whether the wheel is supposed to be here is the ARCH-PARITY table’s judgement, not this smoke’s, so an image whose GenAI lane is legitimately off must not turn red twice;

  • the wheel IS installed but its native library will not load — a defect, and one that is invisible to every other gate. smoke-torch-venv’s installed_version() falls back to importlib.metadata when the import raises (so it happily reports the pinned version), and ARCH-PARITY only reads dist-info directory names.

Collapsing both into exit 3 made a broken riscv64 binding — an unresolved __atomic_* or NEEDED, exactly the risk this lane carries — report as a green SKIP. They are now distinguished by asking whether the distribution is present.

Exit status is not evidence#

check_genai_binding demands the GENAI-BIND sentinel in the output, not just rc=0. If SMOKE_GENAI_PY arrives empty, python - reads an empty program and exits 0. A non-OK sentinel with rc=0 is also an explicit FAIL: a non-OK verdict must never pass.

The version pin reaches the check through _rt_versions_env_pin, a small host-side versions.env reader hoisted out of check_clang_llvm_release’s LLVM_RELEASE lookup. Env wins, then the file, empty on any miss — and every caller must treat empty as “not asserted” rather than as a match. Its || true is load-bearing: under set -euo pipefail an absent key would abort the whole smoke with no summary.

Two adjacent defects fixed in the same window#

Both are risk-reducing and were done deliberately before any rebuild, so the rebuild is more likely to catch a real GEN1 failure than to ship one. Both are new gates on all three arches that have not been run against a real image.

The GenAI libraries were scanned by nothing#

validate-media-runtime.sh checked unresolved NEEDED only over ARTIFACTS (the gst / libcamera / ffmpeg binaries) plus the gst plugin dir; its LIB_DIRS sweep checks ELF machine only and is advisory. /usr/local/lib/onnxruntime-genai/lib appeared in neither list, so an unresolved NEEDED in libonnxruntime-genai*.so would have reached a shipped image unseen — precisely the riscv64 -latomic risk (GenAI’s CMake, unlike upstream ORT’s, has no libatomic probe).

Two changes:

  • the genai prefix joins LIB_DIRS, so it is a resolution root for other binaries and is covered by the advisory arch sweep;

  • the lib dir is walked by scan_plugin_directory — misnamed but generic (it walks a dir’s *.so and returns unresolved sonames), so it was reused rather than copied — feeding the existing ALL_MISSING repair/deny machinery.

Guarded by [ -d ], so it is a no-op when the lane is off and the placeholder tree has an empty lib/. amd64/arm64 are expected to resolve exactly as before, because genai’s own deps (libonnxruntime.so and friends) live in /usr/local/lib/onnxruntime-cpu/lib, already a LIB_DIRS root.

prune_conflicting_onnx_wheels deleted the wheel the same file needs#

On the default ONNX_PACKAGE=onnxruntime path, assemble-torch-app.sh ran rm -f /opt/wheels/*genai*.whl. That glob matches the CPU wheel onnxruntime_genai-<ver>-…whl this media lane builds on every arch — the very wheel build_uv_sync_args looks for ~60 lines later, and that ARCH-PARITY now asserts must be installed. Ordering confirmed: install_project_environment prunes (L441) before build_uv_sync_args looks (L218), so a successful rm would have resurrected the GENAI-DRIFT bug — no local wheel, silent fallback to the app lock’s PyPI genai.

It was inert only by accident: /opt/wheels was a read-only bind mount in Dockerfile.torch, so the rm failed and || true swallowed it. Making that mount rw would have broken all three arches at once — and it became rw on 2026-09-22 (fbc91813), so the narrowed glob below is now the only thing that keeps the CPU wheel.

Narrowed to the GPU-flavoured variants the arm actually means — it sits beside *_gpu-* and *_migraphx-* — i.e. *genai_cuda-*, *genai_rocm-*, *genai_directml-*. MEASURED: verified against a fixture that the CPU wheel and plain onnxruntime-* survive while *_gpu-* and *genai_cuda-* are removed. GEN1 turned this from dormant to self-contradictory.

What to watch on the first real riscv64 build#

In the order they can bite:

  1. The genai stage compiling at all under the GCC 16 riscv64 cross (onnxruntime-genai v0.15.2 + onnxruntime-extensions + dlib). Source-read only.

  2. cross_target_python_dev_ready returning true inside the onnxruntime media container. If it does not, the producer warns and exits 0 with an empty tree, and the verifier then FAILS the build — loudly and correctly, but hours in. (Same gate the ORT cross wheel uses in 30-build-native.sh: without target Python dev files the binding would compile against HOST headers.)

  3. llguidance / Corrosion actually LINKING for riscv64gc-unknown-linux-gnu. The preflight only catches a missing rustup std, not a link failure. Back-out if it bites: drop --use_guidance for riscv64 only.

  4. The pybind EXT_SUFFIX really being .cpython-314-riscv64-linux-gnu.so. Derived, not confirmed against a shipped riscv64 /opt/venv python the way the arm64 value was. It fails safe — the wheel assert turns a wrong suffix into a loud build error, never a silently-unimportable wheel — and the multiarch component is corroborated by verify-wheels.sh, which derives the same .cpython-XY-<target triplet>.so expectation from TARGET_ARCH and was written against a real riscv64 wheel that had leaked the HOST x86_64 suffix. Confirm cheaply by reading EXT_SUFFIX out of the shipped image.

  5. -latomic. GenAI’s CMake has no libatomic probe (upstream ORT does). Low probability; if it bites, the shared-library link dies on undefined __atomic_*. The new validate-media-runtime.sh scan is what would catch a surviving unresolved symbol at runtime.

  6. Then performance, not correctness — see below.

  7. Network flake in the cargo fetch. Deliberately left unfixed. It was an open item in the backlog’s section B at the time; the current refactoring-backlog.md no longer lists it. Note it if the riscv64 stage flakes.

Settled 2026-09-03: a real run printed 14, so the riscv64 floor moved 12 → 13 — one wheel of slack on a single observation. The rule for raising it further is in The app-wheel floor.

Performance: riscv64 MLAS is scalar#

Not a correctness issue, but it shapes expectations and timeouts. When GEN1 landed, ORT v1.29’s riscv64 MLAS fell back to the scalar reference kernels — including the int4 MatMulNBits kernel GenAI models want. The RVV kernels need onnxruntime_USE_RVV plus an -march=rv64gcv compile probe, and this repo set neither. The RVA23 baseline sets both now: the riscv64 compiler targets rv64gcv… and the ORT build passes onnxruntime_USE_RVV=ON (riscv64-rva23-baseline.md). The tier-4 run above predates it. Under qemu-user, generate() still needs a generous timeout.

Why ARCH-PARITY is deliberately red for existing riscv64 images#

The riscv64:onnxruntime_genai exemption in _parity_exempt is deleted. It used to read: “upstream ships no riscv64 wheel and closed its one RISC-V request as not-planned. Policy, not drift.” That was true of upstream; it is no longer true of this repo.

Consequences, all intended:

  • every currently-shipped riscv64 image now fails that assertion, and smoke-torch-venv reports XX onnxruntime-genai NOT INSTALLED;

  • an image built with GENAI_ALLOW_RISCV64 turned off also fails ARCH-PARITY — in that case the fix is to re-add the exemption arm together with the toggle.

Do not “fix” this by re-adding the exemption. The table’s contract is that every arm is a deletion candidate and may only encode a reason that is true today, never “not built yet” — otherwise it becomes a wish list. An absence nobody wrote down is drift, and drift must fail.

The parallel change in smoke-torch-venv.sh inverts the same policy. The old carve-out (STV_REQUIRE_GENAI, added 2026-08-11 after the assert flagged the documented absence as a pin failure) printed ~~ not installed (documented riscv64 skip) and passed no matter why the wheel was missing. Now the riscv64 arm keys off GENAI_ALLOW_RISCV64 instead: armed unless the lane was explicitly turned off. One knob for the whole feature on both the build and the smoke side, rather than a second smoke-only name nobody would think to set.

smoke-runtime-image.sh’s transitional pre-2026-08-12-wrapper elif branch is gone with it: keeping it would have printed the exact opposite story (“documented riscv64 exemption”) on the first genuine riscv64 miss.

The regression rows that pin this#

test-smoke-arch-parity.sh gained rows in both directions — the riscv64 arm of assert_pinned_versions had no case at all before, which is the blind spot that let a “documented skip” outlive the thing it documented:

Case

Asserts

_parity_exempt riscv64 onnxruntime_genai

now t_assert_fails (was t_assert_ok) — re-adding the exemption would silently un-gate the component GEN1 turned on

riscv64 image without genai

onnxruntime-genai NOT INSTALLED, FAILURES>=1

riscv64 image with the pinned genai

FAILURES=0

riscv64 + GENAI_ALLOW_RISCV64=false

FAILURES=0, and the message names the toggle

amd64 + GENAI_ALLOW_RISCV64=false

still FAILURES>=1 — the hatch is riscv64-scoped

_stv_drive grew a fourth parameter for one extra VAR=VALUE, because it drives the smoke under env -i and the toggle therefore has to be handed in explicitly.

Backing the lane out#

In increasing order of blast radius:

  1. Turn the lane off. Set GENAI_ALLOW_RISCV64=false in versions.env and the matching ARG default in Dockerfile.media — both, in the same commit; verify-arg-consistency.sh requires them to agree. Then re-add the riscv64:onnxruntime_genai arm to _parity_exempt (the text to restore is preserved as an NB comment at that spot), or ARCH-PARITY will still red. smoke-torch-venv follows the toggle automatically.

  2. Drop only --use_guidance for riscv64, if llguidance is the thing that breaks. The wheel then ships without guidance — a different wheel from amd64/arm64’s; record that fact, because no gate can see it.

  3. Do not use BUILD_GENAI=false to back out a riscv64 problem. It is the all-arch master switch and would drop the shipped amd64/arm64 genai wheels with it.

Removing the lane entirely would additionally mean deleting the patch, the patches mount, and the riscv64 arm of both cross allowlists — but there is no reason to prefer that over the toggle, which is verified to restore the pre-GEN1 behaviour exactly.

MEASURED vs REASONED#

Honest separation, as of 2026-09-03.

MEASURED

  • The patch applies, and re-applies as a no-op, against a real clone of v0.15.2 (ed5f4e87).

  • The arm64 cross lane really does produce a wheel — “Created wheel for onnxruntime-genai: onnxruntime_genai-0.15.2-cp314-cp314-linux_aarch64.whl”, media-arm64.log, 2026-08-27.

  • The arm64 EXT_SUFFIX was verified against the shipped arm64 image’s /opt/venv python.

  • The narrowed prune globs were verified against a fixture.

  • Static gates over the whole window: make lint clean (276 files), make preflight green, make test-linux-scripts 38 suites / 1001 assertions.

  • The source-reads of the v0.15.2 tree quoted above (the FATAL_ERROR branch, the three genai_target_platform consumers, the intrinsic survey, the llguidance crate graph) are observations of real source at the pinned tag.

MEASURED since — the 2026-08-31 r4 build and the 2026-09-03 bench runs

  • The riscv64 genai stage compiles (Built target onnxruntime-genai), and the wheel carries the right platform tag.

  • llguidance/Corrosion link for riscv64gc-unknown-linux-gnu — zero dropping --use_guidance in the run, so no silent parity divergence.

  • The riscv64 EXT_SUFFIX is .cpython-314-riscv64-linux-gnu.so, read off the shipped image by tier 2 (e_machine=243).

  • generate() produces sane tokens — in fact ids identical to amd64. See Tier 4, measured.

  • The validate-media-runtime.sh genai scan ran on all three arches in the 2026-09-03 chain. It resolved only libXv.so.1 / libwebpdemux.so.2, none of them genai; an independent ldd over the shipped riscv64 /usr/local/lib/onnxruntime-genai/lib shows 0 unresolved NEEDED.

  • The riscv64 app-wheel count printed 14; the floor moved 12 → 13.

REASONED, still never run

  • Whether -latomic is needed. The riscv64 link succeeded without it, which makes it unnecessary for this configuration — not that it never will be.

  • Tier 4 on real riscv64 silicon. The 2026-09-03 run was qemu-user, which emulates the ISA but not a physical core’s timing, errata or extension set. #594’s hardware was a XuanTie C910.