Are you able to use NPU accel with immich?

[xoconoch]

Title. I went on a whole odyssey to try and get this working with NixOS a couple weeks ago with no success and I just stumbled up with your repo. Could you provide details on how you got it working? I am specially interested in the kernel version and the rknpu driver you're using

[YayaADev]

Hey. Im not able to use the NPU accel with immich directly. Immich uses regular ML frameworks like ONNX, OpenVINO. The Rockchip requires it to be in RKNN format. There is an open issue for support though.

For Immich im using rkmpp the (GPU Mali G610).

The Rknpu driver im using is RKNPU driver: v0.9.7 & kernel version is 6.1.75

Ran these commands

uname -r
6.1.75
➜  nixos-config git:(main) ✗ sudo cat /sys/kernel/debug/rknpu/version && sudo cat /sys/kernel/debug/rknpu/load

RKNPU driver: v0.9.7
NPU load:  Core0:  0%, Core1:  0%, Core2:  0%,

[xoconoch]

Would it be possible to update it to 0.9.8? My first guess would be compiling a more recent kernel

[YayaADev]

Updating wouldn't be easy. Im pulling from this github gnull/nixos-rk3588 they havent released a new version for the NPU.

[YayaADev]

Update,

I built the latest version of the friendlyelev kernel on nixOS and flashed it onto an SD card. I was able to boot and ssh onto the SD card in my board, confirmed it had the latest NPU driver 0.9.8.

I'll need to flash it onto my board for use and will upload the set up soon

[YayaADev]

I was able to successfully build the kernel from friendlyelecs branch that has the updated NPU version. im now on their latest kernel and NPU. I published a repo for others to be able to flash the image to their SD and install it on their cm3588 NAS. You can find it here https://github.com/YayaADev/nixos-friendlyelec-cm3588/tree/main

If you are already on nixOS already and would like to change your kernel to be using the branch from friendlyelec you can change the flake.nix like I did here https://github.com/YayaADev/nixos-config/blob/main/flake.nix#L7

[xoconoch]

Great, thank you for your input!

[YayaADev]

np, i also confirmed i am using the NPU for immich settings here https://github.com/YayaADev/nixos-config/blob/main/modules/services/immich.nix#L35 for reference