RKNN — vision models on the NPU

RKNN is Rockchip's toolchain for running vision and general neural-network models on the NPU. This page is the end-to-end path: convert a trained model on a PC, then run inference on the board.

Supported chips: RK3588, RK3576, RK3568, RK3566. The RK3399/RK3288/T982 boards have no NPU — they can run models on CPU/GPU, but not through RKNN.

The three pieces

Rockchip splits the toolchain into parts. Knowing which runs where saves a lot of confusion:

Component	Runs on	Does
RKNN-Toolkit2	x86 Linux PC	Convert, quantize, simulate, evaluate
RKNN-Toolkit-Lite2	The board	Python inference (no conversion)
RKNN Runtime (librknnrt.so + C API)	The board	C/C++ inference

Conversion is PC-only. Lite2 on the board is inference-only — it cannot convert a model. Do the conversion once on a PC, deploy the .rknn result.

Step 1 — Install RKNN-Toolkit2 on a PC

Rockchip recommends x86 Ubuntu (20.04 or newer). Use a fresh Python environment; conda/miniforge avoids version clashes.

# On an x86 Linux PC — NOT on the board
git clone https://github.com/airockchip/rknn-toolkit2
cd rknn-toolkit2

# Install the wheel matching your Python version (from the packages/ dir)
pip install rknn-toolkit2 -i https://pypi.org/simple

The GitHub repo is the source of truth for the current version and the exact wheel names — check it rather than pinning an old version from memory.

Step 2 — Convert your model to .rknn

The converter takes PyTorch, ONNX, TensorFlow, TensorFlow Lite, Caffe or Darknet and emits a single .rknn file. Set the target to your board's chip.

convert.py — runs on the PC

from rknn.api import RKNN

rknn = RKNN(verbose=True)

# 1. Configure for the target chip. Use your board's SoC:
#    rk3588 | rk3576 | rk3568 | rk3566
rknn.config(
    mean_values=[[123.675, 116.28, 103.53]],
    std_values=[[58.395, 57.12, 57.375]],
    target_platform='rk3588',
)

# 2. Load the source model (ONNX shown; load_pytorch/load_tensorflow also exist)
rknn.load_onnx(model='yolo.onnx')

# 3. Build. do_quantization=True shrinks the model and speeds up the NPU;
#    supply a small set of representative images as the calibration dataset.
rknn.build(do_quantization=True, dataset='./dataset.txt')

# 4. Export the deployable file
rknn.export_rknn('yolo_rk3588.rknn')
rknn.release()

Quantization is the decision that matters

do_quantization=True converts the float model to INT8. On a 1 TOPS RK3568 this is usually not optional — it is what makes the model fit and run at speed. It costs a little accuracy.

The calibration dataset (dataset.txt, a list of image paths) is how the converter learns the range of your real inputs. Use images that look like what the device will actually see — not random samples. A bad calibration set is the most common cause of "the model works on my PC but is inaccurate on the board".

Step 3 — Copy the model to the board

scp yolo_rk3588.rknn  user@<board-ip>:/home/user/

Enable network ADB or SSH first — see Debug switches for turning network access on from your app, or use a serial console.

Step 4 — Run inference on the board (Python)

Install RKNN-Toolkit-Lite2 on the board (inference-only Python API), then:

infer.py — runs on the board

from rknnlite.api import RKNNLite

rknn = RKNNLite()
rknn.load_rknn('yolo_rk3588.rknn')

# NPU_CORE_AUTO lets the driver pick a core. On RK3588 you can pin to
# NPU_CORE_0 / _1 / _2, or _0_1_2 to run one big model across all three.
rknn.init_runtime(core_mask=RKNNLite.NPU_CORE_AUTO)

outputs = rknn.inference(inputs=[image])   # image: preprocessed numpy array
# ... post-process outputs (NMS, etc.) as your model requires ...

rknn.release()

For a C/C++ application, link librknnrt.so and use the RKNN Runtime C API instead — the model zoo below has complete C examples.

Verify the NPU is alive

If inference fails to start, check the driver on the board:

cat /sys/kernel/debug/rknpu/version    # driver version
sudo dmesg | grep "Initialized rknpu"  # confirms the NPU came up

A missing or too-old librknnrt.so is the usual culprit — the runtime library version should match the toolkit version you converted with. Push the matching librknnrt.so to /usr/lib on the board if they differ.

Don't start from scratch — use the model zoo

Rockchip's rknn_model_zoo has ready conversion scripts and pre/post-processing for common models (YOLO family, ResNet, mobilenet, and more) in both Python and C/C++. For a standard detection or classification model, start from the matching example and swap in your weights rather than writing conversion from zero.

Realistic expectations by chip

• RK3588 / RK3576 (6 TOPS) — real-time detection on live camera, multiple models or streams at once. This is where vision products live.
• RK3568 (~1 TOPS) — a single detection or classification stream. Good for a people counter, a face match, presence sensing running beside your app.
• RK3566 (~0.8 TOPS) — one light model. Benchmark before committing; if it is tight, prune/quantize harder or move up a chip.

Whatever the chip, benchmark your actual model on the actual board early. TOPS is a ceiling, not a promise for your specific network.

Related

• On-device AI overview — pick RKNN vs RKLLM
• RKLLM — for local language models (RK3588/RK3576)
• SoC reference — NPU capability per chip
• Debug switches — enable network access to copy models

rknn-toolkit2 evolves quickly — API details, wheel names and supported models change between releases. Verify against the current airockchip/rknn-toolkit2 repo.