Skip to content

RunCellpose crashes when using Podman with GPU in Mac #293

Open
Open
RunCellpose crashes when using Podman with GPU in Mac#293

Description

@emiglietta
emiglietta
opened on Sep 24, 2025

I'm trying to run Runcellpose on the images from the sample Cellprofiler pipeline using Podman on a M2 Mac, but it crashes when I want to use the GPU.

Running any version of Cellpose with or without GPU works fine when running from Python. I also have allocated as much memory as possible to Podman (~16GB).

V4 with GPU

Module crashes with the following error:

['/opt/podman/bin/podman', 'run', '--rm', '-v', '/Users/emigliet/Desktop/.cellprofiler_temp/feb751a2-1c11-4a5c-8163-82dd6c644b5c:/data', 'cellprofiler/runcellpose_with_pretrained:4.0.6', '--gpus', 'all', 'cellpose', '--verbose', '--dir', '/data/img', '--pretrained_model', 'cpsam', '--cellprob_threshold', '0.0', '--min_size', '15', '--anisotropy', '0.0', '--flow_threshold', '0.4', '--stitch_threshold', '0.0', '--exclude_on_edges']
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)

==========
== CUDA ==
==========

CUDA Version 11.8.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use the NVIDIA Container Toolkit to start this container with GPU support; see
   https://docs.nvidia.com/datacenter/cloud-native/ .

/opt/nvidia/nvidia_entrypoint.sh: line 67: exec: --: invalid option
exec: usage: exec [-cl] [-a name] [command [argument ...]] [redirection ...]
Failed to run module RunCellpose
Traceback (most recent call last):
  File "/Users/emigliet/Documents/GitHub/CP_DEV/CellProfiler-plugins/active_plugins/runcellpose.py", line 1011, in run
    cellpose_output = numpy.load(os.path.join(temp_img_dir, unique_name + "_seg.npy"), allow_pickle=True).item()
  File "/Users/emigliet/anaconda3/envs/cellposeSAM_cellprofiler/lib/python3.9/site-packages/numpy/lib/npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: '/Users/emigliet/Desktop/.cellprofiler_temp/feb751a2-1c11-4a5c-8163-82dd6c644b5c/img/feb751a2-1c11-4a5c-8163-82dd6c644b5c_seg.npy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/emigliet/anaconda3/envs/cellposeSAM_cellprofiler/lib/python3.9/site-packages/cellprofiler/gui/pipelinecontroller.py", line 3390, in do_step
    self.__pipeline.run_module(module, workspace_model)
  File "/Users/emigliet/anaconda3/envs/cellposeSAM_cellprofiler/lib/python3.9/site-packages/cellprofiler_core/pipeline/_pipeline.py", line 1349, in run_module
    module.run(workspace)
  File "/Users/emigliet/Documents/GitHub/CP_DEV/CellProfiler-plugins/active_plugins/runcellpose.py", line 1015, in run
    raise(FileNotFoundError("I'm sorry, Cellpose seems to have crashed and I'm not sure why, since it's in its own container."))
FileNotFoundError: I'm sorry, Cellpose seems to have crashed and I'm not sure why, since it's in its own container.
Using cp4 from python in the same environment works just fine using GPU :
Loading new model: cpsam
** TORCH MPS version installed and working. **
>>>> using GPU (MPS)
>>>> loading model /Users/emigliet/.cellpose/models/cpsam
Failed to set GPU memory share. Please check your PyTorch installation. Not setting per-process memory share.

V4 wit CPU

Run seems to freeze but just takes forever (with a very small 2D image), but eventually works:

['/opt/podman/bin/podman', 'run', '--rm', '-v', '/Users/emigliet/Documents/Proyectos FIL/Isa106/PICTO fotos para Esteban/MHCII-IBA1 9abril24.mdb/CP_output/.cellprofiler_temp/74ffc9db-c9b6-485c-95c5-68496d222a7e:/data', 'cellprofiler/runcellpose_with_pretrained:4.0.6', 'cellpose', '--verbose', '--dir', '/data/img', '--pretrained_model', 'cpsam', '--cellprob_threshold', '0.0', '--min_size', '15', '--anisotropy', '0.0', '--flow_threshold', '0.4', '--stitch_threshold', '0.0', '--exclude_on_edges']
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)

==========
== CUDA ==
==========

CUDA Version 11.8.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use the NVIDIA Container Toolkit to start this container with GPU support; see
   https://docs.nvidia.com/datacenter/cloud-native/ .



Welcome to CellposeSAM, cellpose v
cellpose version: 	4.0.6 
platform:       	linux 
python version: 	3.11.13 
torch version:  	2.7.1+cu118! The neural network component of
CPSAM is much larger than in previous versions and CPU excution is slow. 
We encourage users to use GPU/MPS if available. 


creating new log file
2025-09-24 21:25:41,008 [INFO] WRITING LOG OUTPUT TO /root/.cellpose/run.log
2025-09-24 21:25:41,010 [INFO] 
cellpose version: 	4.0.6 
platform:       	linux 
python version: 	3.11.13 
torch version:  	2.7.1+cu118
2025-09-24 21:25:41,011 [INFO] >>>> using CPU
2025-09-24 21:25:41,011 [INFO] >>>> using CPU
2025-09-24 21:25:41,011 [WARNING] the '--diam_mean' flag is deprecated in v4.0.1+ and no longer used
2025-09-24 21:25:41,026 [INFO] >>>> running cellpose on 1 images using all channels
2025-09-24 21:25:53,613 [INFO] >>>> loading model /root/.cellpose/models/cpsam
2025-09-24 21:25:57,953 [INFO] 0%|          | 0/1 [00:00<?, ?it/s]
[W924 21:25:58.430460378 NNPACK.cpp:57] Could not initialize NNPACK! Reason: Unsupported hardware.
[W924 22:02:25.092945292 NNPACK.cpp:57] Could not initialize NNPACK! Reason: Unsupported hardware.
[W924 22:02:27.676032493 NNPACK.cpp:57] Could not initialize NNPACK! Reason: Unsupported hardware.
[W924 22:02:30.128416575 NNPACK.cpp:57] Could not initialize NNPACK! Reason: Unsupported hardware.
2025-09-24 22:02:32,760 [INFO] 100%|##########| 1/1 [36:34<00:00, 2194.80s/it]
2025-09-24 22:02:32,763 [INFO] 100%|##########| 1/1 [36:34<00:00, 2194.81s/it]
2025-09-24 22:02:32,764 [INFO] >>>> completed in 2211.752 sec

V3 wit GPU

Produces the same error as V4

V3 with CPU

Runs fine

['/opt/podman/bin/podman', 'run', '--rm', '-v', '/Users/emigliet/Documents/Proyectos FIL/Isa106/PICTO fotos para Esteban/MHCII-IBA1 9abril24.mdb/CP_output/.cellprofiler_temp/40b2665a-c727-45fa-8b54-dc8b00b1d3e0:/data', 'cellprofiler/runcellpose_with_pretrained:3.1.1.2', 'cellpose', '--verbose', '--dir', '/data/img', '--pretrained_model', 'cyto3', '--chan', '0', '--chan2', '0', '--diameter', '30', '--cellprob_threshold', '0.0', '--min_size', '15', '--anisotropy', '0.0', '--flow_threshold', '0.4', '--stitch_threshold', '0.0']
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)

==========
== CUDA ==
==========

CUDA Version 11.8.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use the NVIDIA Container Toolkit to start this container with GPU support; see
   https://docs.nvidia.com/datacenter/cloud-native/ .

creating new log file
2025-09-24 22:07:08,464 [INFO] WRITING LOG OUTPUT TO /root/.cellpose/run.log
2025-09-24 22:07:08,467 [INFO] 
cellpose version: 	3.1.1.2 
platform:       	linux 
python version: 	3.11.13 
torch version:  	2.7.1+cu118
2025-09-24 22:07:08,467 [INFO] >>>> using CPU
2025-09-24 22:07:08,467 [INFO] >>>> using CPU
2025-09-24 22:07:08,490 [INFO] >>>> running cellpose on 1 images using chan_to_seg GRAY and chan (opt) NONE
2025-09-24 22:07:08,490 [INFO] >>>> using CPU
2025-09-24 22:07:08,490 [INFO] >>>> using CPU
2025-09-24 22:07:08,491 [INFO] >> cyto3 << model set to be used
2025-09-24 22:07:08,891 [INFO] >>>> loading model /root/.cellpose/models/cyto3
2025-09-24 22:07:09,606 [INFO] >>>> model diam_mean =  30.000 (ROIs rescaled to this size during training)
2025-09-24 22:07:09,607 [INFO] Downloading: "https://www.cellpose.org/models/size_cyto3.npy" to /root/.cellpose/models/size_cyto3.npy

100%|██████████| 3.54k/3.54k [00:00<00:00, 924kB/s]
2025-09-24 22:07:10,525 [INFO] >>>> using diameter 30.000 for all images
2025-09-24 22:07:10,527 [INFO] 0%|          | 0/1 [00:00<?, ?it/s]
2025-09-24 22:07:10,536 [INFO] channels set to [0, 0]
2025-09-24 22:07:10,536 [INFO] ~~~ FINDING MASKS ~~~
2025-09-24 22:08:07,691 [INFO] >>>> TOTAL TIME 57.15 sec
2025-09-24 22:08:07,738 [INFO] 100%|##########| 1/1 [00:57<00:00, 57.21s/it]
2025-09-24 22:08:07,739 [INFO] 100%|##########| 1/1 [00:57<00:00, 57.21s/it]
2025-09-24 22:08:07,739 [INFO] >>>> completed in 59.271 sec

Activity

  1. emiglietta commented on Sep 25, 2025

    @emiglietta
    emiglietta
    on Sep 25, 2025
    CollaboratorAuthor

    UPDATE: I checked my Podman Desktop configuration and Rosetta is enabled

    Image

  2. bethac07 commented on Sep 25, 2025

    @bethac07
    bethac07
    on Sep 25, 2025
    Member

    Do you have Docker Desktop and can check to see if the behavior is the same between each? Fine if no just checking.

    Seems like for Podman, we might need to think about making arm and non arm versions, but I suspect fewer people will be using Podman so that might affect the priority of doing that.

  3. emiglietta commented on Sep 25, 2025

    @emiglietta
    emiglietta
    on Sep 25, 2025
    CollaboratorAuthor

    I don't have Docker Desktop on my machine (since I don't routinely use it, it made no sense to pay for the license), that's why I wanted to try Podman, in case I need to try something on a container for a collaborator :)

  4. ShataDg commented on Oct 14, 2025

    @ShataDg
    ShataDg
    on Oct 14, 2025
    Collaborator

    I was running CellProfiler 4.2.8 with Cellpose 4 with the Docker option and with use GPU set to yes, and I received the same error. RunCellpose does work with use GPU set to no, but it takes quite a while. I have tried using the default values as well as increasing the values for GPU_memory_share, but it gives the same error.
    Image

    The following trace is CellProfiler 4.2.6 with Cellpose 4 (because I do not have CellProfiler 4.2.8 built from source)

    15:31:47: Debug: Adding duplicate image handler for 'Windows bitmap file'
15:31:47: Debug: Adding duplicate animation handler for '1' type
15:31:47: Debug: Adding duplicate animation handler for '2' type
Could not load runomnipose
Traceback (most recent call last):
  File "/Users/sdasgupt/anaconda3/envs/CP_SD/lib/python3.8/site-packages/cellprofiler_core/utilities/core/modules/__init__.py", line 71, in add_module
    m = __import__(mod, globals(), locals(), ["__all__"], 0)
  File "/Users/sdasgupt/CellProfiler-plugins/active_plugins/runomnipose.py", line 9, in <module>
    from cellpose_omni import models, io, core, plot
ModuleNotFoundError: No module named 'cellpose_omni'
Could not load runimagejscript
Traceback (most recent call last):
  File "/Users/sdasgupt/anaconda3/envs/CP_SD/lib/python3.8/site-packages/cellprofiler_core/utilities/core/modules/__init__.py", line 71, in add_module
    m = __import__(mod, globals(), locals(), ["__all__"], 0)
  File "/Users/sdasgupt/CellProfiler-plugins/active_plugins/runimagejscript.py", line 15, in <module>
    import cpij.bridge as ijbridge, cpij.server as ijserver
  File "/Users/sdasgupt/CellProfiler-plugins/active_plugins/cpij/bridge.py", line 3, in <module>
    import atexit, cpij.server as ijserver
  File "/Users/sdasgupt/CellProfiler-plugins/active_plugins/cpij/server.py", line 15, in <module>
    import jpype, imagej, multiprocessing, socket, threading, time
ModuleNotFoundError: No module named 'jpype'
could not load these modules: runomnipose,runimagejscript
['/usr/local/bin/docker', 'run', '--rm', '-v', '/Users/sdasgupt/Documents/.cellprofiler_temp/c5fd2ca6-5417-41f7-b8e5-4382418cfc6f:/data', 'cellprofiler/runcellpose_with_pretrained:4.0.6', '--gpus', 'all', 'cellpose', '--verbose', '--dir', '/data/img', '--pretrained_model', 'cpsam', '--cellprob_threshold', '0.0', '--min_size', '15', '--anisotropy', '0.0', '--flow_threshold', '0.4', '--stitch_threshold', '0.0', '--exclude_on_edges']
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

==========
== CUDA ==
==========

CUDA Version 11.8.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use the NVIDIA Container Toolkit to start this container with GPU support; see
   https://docs.nvidia.com/datacenter/cloud-native/ .

/opt/nvidia/nvidia_entrypoint.sh: line 67: exec: --: invalid option
exec: usage: exec [-cl] [-a name] [command [argument ...]] [redirection ...]
Failed to run module RunCellpose
Traceback (most recent call last):
  File "/Users/sdasgupt/CellProfiler-plugins/active_plugins/runcellpose.py", line 1011, in run
    cellpose_output = numpy.load(os.path.join(temp_img_dir, unique_name + "_seg.npy"), allow_pickle=True).item()
  File "/Users/sdasgupt/anaconda3/envs/CP_SD/lib/python3.8/site-packages/numpy/lib/npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: '/Users/sdasgupt/Documents/.cellprofiler_temp/c5fd2ca6-5417-41f7-b8e5-4382418cfc6f/img/c5fd2ca6-5417-41f7-b8e5-4382418cfc6f_seg.npy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sdasgupt/anaconda3/envs/CP_SD/lib/python3.8/site-packages/cellprofiler/gui/pipelinecontroller.py", line 3391, in do_step
    self.__pipeline.run_module(module, workspace_model)
  File "/Users/sdasgupt/anaconda3/envs/CP_SD/lib/python3.8/site-packages/cellprofiler_core/pipeline/_pipeline.py", line 1349, in run_module
    module.run(workspace)
  File "/Users/sdasgupt/CellProfiler-plugins/active_plugins/runcellpose.py", line 1015, in run
    raise(FileNotFoundError("I'm sorry, Cellpose seems to have crashed and I'm not sure why, since it's in its own container."))
FileNotFoundError: I'm sorry, Cellpose seems to have crashed and I'm not sure why, since it's in its own container.

  5. braymp commented on Jan 15, 2026

    @braymp
    braymp
    on Jan 15, 2026
    Contributor

    @ShataDg : I receive the same error on Windows/CP 4.2.8. According to this post, #276 (comment) it may not be possible to get Docker+GPU working (which makes me sad) but perhaps @ErinWeisbart has updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions