Available unified memory in llama.cpp

[alex-mashin]

There is a bug (Misc. bug: UMA detection incorrectly limits available memory on AMD APUs with large TTM allocations) with reporting available shared memory for integrated GPUs (at least, AMD GPUs).

This bug is also present in llama.cpp models pulled with Docker Model Runner.

The workaround for this bug for an Ollama container that I use is:

  ollama:
    # Vulkan, rather than ROCM (image: ollama/ollama:rocm):
    image: ollama/ollama:latest
    environment:
      GGML_CUDA_ENABLE_UNIFIED_MEMORY: 1
      OLLAMA_VULKAN: 1
      OLLAMA_IGPU_ENABLE: 1

How can I similarly configure in docker-compose.yml a model run by Docker Model Runner?