How Does a Vision Model Reach a Factory-Edge NPU? The ONNX→RKNN→RK3588 Route, Staged and Smoke-Tested

Every industrial vision project that survives the demo stage meets the same geography problem: the model was born on a workstation GPU, but it must live on a box bolted to a line — no cloud, no RTX, a hard cycle-time budget. In our stack the answer to that geography is Rockchip’s RK3588 edge SoC (6-TOPS NPU class), and this dispatch documents the route to it as it actually stands on the workstation: staged, smoke-tested at the container level, one mile from finished. The rubric as always: what it is → what it does → how to run it → what went wrong (or, honestly, what hasn’t been run yet).

The route, end to end

Four stations:

1. Train on the workstation. This is the anomalib rig from our previous field guide — PatchCore-class models on the 4090D.
2. Export to ONNX. The neutral interchange. The anomalib environment already carries onnx 1.19.1 (and openvino 2025.3.0, the parallel route to Intel edge hosts — ONNX keeps the model board-agnostic).
3. Convert and quantize to RKNN, in docker, on x86. Rockchip ships the toolkit as an Ubuntu 20.04 image; conversion plus INT8 quantization happens here, on the workstation, not on the target.
4. Cross-compile the application, deploy to the board. The aarch64 cross-compiler images build the host-side app; the board runs the NPU runtime.

What’s on disk

Three Rockchip images, pulled and verified (sizes from docker images):

Image	Size	Role
rk3588-rknn-dev	3.03 GB	RKNN toolkit: convert + quantize on x86_64
rk3588-cross-compiler	3.03 GB	aarch64 application cross-compile
rk3588-base-cross-compiler	798 MB	base layer for leaner build images

Total staging cost: about 6.86 GB. On 2026-08-30 we smoke-tested the dev image for the first time — no container from these images had ever been run (they sat staged since pull); a fresh container boots and reports Python 3.8.10 inside. The toolchain is real, executable, and was waiting for its model.

The mile, walked to its first station — and its wall

Since this guide’s first publication we walked the leg, and it split into a completed station and a documented wall:

Station one, done: the ONNX export. A resnet18 exported from the anomalib environment via torch.onnx.export(..., opset_version=12) — 46,733,662 bytes, random weights (the conversion case needs no trained model). The gotcha that cost one failed attempt: torch 2.9 flips the exporter default to the dynamo path, which demands onnxscript and is not installed here — dynamo=False pins the legacy exporter and works. Version churn again, at the very first station.

The wall: acquiring the converter itself. The RK3588 images carry the deployment-side SDK (/workspace/rknn_app_demo, rknpu2) — the conversion toolkit does not live in them; it distributes as a wheel from the project’s GitHub. From this network, six acquisition routes were tried and all failed, each with its own exact error: git clone over HTTPS dies on a GnuTLS termination; the GitHub Releases API answers but carries zero attached assets; the repository tree API lists no wheel files at all; raw.githubusercontent returns 404 for the path; the container cannot reach PyPI; and the domestic PyPI mirror has no such package. That inventory is the finding — the standard toolchain of China’s most popular edge NPU is, from an un-proxied mainland network, effectively airless. (The proxied route exists on this machine; we deliberately do not invoke it for a field guide that must reproduce clean.)

No .rknn artifact exists anywhere on disk — a machine-wide check confirms the conversion never happened here. Meanwhile the other edge route on this fleet actually ran: workstation-trained PatchCore running on a Mac mini’s CPU against a Basler camera — a reminder that “edge” is a requirement, and Apple Silicon was one valid answer while the NPU route waited on its toolkit.

The pitfalls (the ones you can schedule around)

Quantization accuracy is a task, not a setting. INT8 on a 6-TOPS NPU trades precision for speed by design; every converted model needs a validation pass against its pre-quantization self, and the drop is per-model news. Budget it.

Version triple-lock. Toolkit (docker tag), on-board runtime, and converted model must agree. The docker route makes the workstation side exactly reproducible; the board side is a discipline.

Two compilers, one app. The cross-compiler images build for aarch64 while the board runs its own runtime; drift between them surfaces as errors that look like anything but compiler drift. Keep a matched pair, note both versions in the deployment log.

Who this is for

Teams whose model works on the bench and now must survive a line: no network dependency, fixed latency, board-class power and price. If your roadmap is anomalib (or any vision model) → edge NPU, this is the map of the middle country, with the exact images named and the remaining leg marked in red.

Replication appendix

# The staged images (sizes as measured):
docker images | grep rk3588
# rk3588-rknn-dev    20.04   3.03GB
# rk3588-cross-compiler 20.04 3.03GB
# rk3588-base-cross-compiler 20.04 798MB

# Smoke test the conversion environment (first execution, 2026-08-30):
docker run --rm rk3588-rknn-dev:20.04 python3 -V
# Python 3.8.10

# The route's next mile (not yet run here):
# anomalib ckpt -> export ONNX (onnx 1.19.1 available in anomalib_env)
# -> docker run rk3588-rknn-dev ... rknn conversion + INT8 quantization
# -> cross-compile host app, deploy to RK3588, validate accuracy vs pre-quantization

Sources and method

First-party: the workstation’s docker image list, container smoke run, and the anomalib environment’s pip inventory (onnx 1.19.1, openvino 2025.3.0), inspected 2026-08-30. Third-party, checked: Rockchip’s rknn-toolkit2 repository (the converter the dev image packages) and onnx.ai for the interchange standard. Drafted with AI assistance under human editorial direction; the 6-TOPS NPU rating follows Rockchip’s public specification. Numbers from this staging are in the /data/ ledger.