Thanks for your reply. A practical follow up: do you mount storage from the host or do you use virtual drives?

I use ZFS on my host and and file recovery is just so convenient when the services can write directly to it. I’m reluctant to give that up but assume a mount reduces isolation.

Title made me suspicious already.

As an alternative to mounting full /dev/dri, there are symlinks under /dev/dri/by-path/ which you can use. IIUC they are linked by the used PCI slot, hence should be as stable as the physical order of the cards.

I use this pattern in a compose file:

...
devices:
  - source: /dev/dri/by-path/pci-XXX-render
    target: /dev/dri/renderD128
    permissions: rwm

In case you are unaware, friends don’t let friends use ollama. The llama.cpp CLI has become pretty much equally easy to use, plus since you use Docker there’s no need to compile anything.

Edit: typos and missing words

There are tools like crontab.guru which helped me get started.

Welcome.

https://en.wikipedia.org/wiki/David_Heinemeier_Hansson#Controversy

The service starts as it should, right? The errors appear only on request, right? Any other sections in the compose logs that seem relevant?

You use podman-compose with the Podman backend?

And there are no volumes mounted, just to be sure?

No expert but don’t the logs look like the requests are coming through?

Yeah right? At least a layer of VPN on top.