Memory leak when accessing magicgui as a method

[Antyos]

Describe the bug
I found this while trying to hot-reload a widget (which is probably a bad idea), but this seems like a bug regardless.

Creating an object with a @magicgui decorated method and then accessing that method via .show (even without calling) or .native will prevent the object from being garbage collected.

For example,

from magicgui import magicgui

class F:
    def __init__(self, x: int):
        self.x = x

    @magicgui()
    def func_gui(self, a: int = 1):
        print(a + self.x)

To Reproduce

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "magicgui",
#     "pyqt6==6.9.1",
# ]
#
# [tool.uv.sources]
# magicgui = { git = "https://github.com/pyapp-kit/magicgui" }
# ///
import gc

from magicgui import magic_factory, magicgui, widgets

class F(widgets.Container):
    def __init__(self, x: int):
        self.x = x
        self.func = self.func_fact()
        super().__init__()

    @magic_factory()
    def func_fact(self, a=1):
        print(a + self.x)

    @magicgui()
    def func_gui(self, a=1):
        print(a + self.x)


def count_instances(cls) -> list:
    return len([obj for obj in gc.get_objects() if isinstance(obj, cls)])


f = F(x=5)
f.func.show  # This is fine
f.func_gui.show  # [BAD] We don't have to call .show() to get the bug, just access
f.func_gui.native  # [BAD] This also breaks it

print(get_instances(F))
### 1

# f.func_gui.native.deleteLater()  # Ironically, calling this also produces the issue
# del f.func_gui  # This raises an AttributeError. Not sure if that's intended.
del f
gc.collect()

print(count_instances(F))
### 1

Expected behavior

Commenting out the f.func_gui lines will print:

1
0

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: Windows, Python 3.13.9
• backend: PyQt6==6.9.1
• magicgui version: from main branch

[Antyos]

I did a bit more digging, and I think the issue lies in these lines of the FunctionGui.__get__() method, specifically line 437.

magicgui/src/magicgui/widgets/_function_gui.py

Lines 434 to 440 in 2d985e7

	prior, self._param_options = (
	self._param_options,
	{
	p0: {"bind": obj},
	**self._param_options,
	},
	)

I understand why it's binding the object--so that a different instance of the widget can exist for each object, however, I think this is the cause of the leak. If we replace line 437 with a weakref, e.g.

 prior, self._param_options = ( 
     { 
-        p0: {"bind": obj}, 
+        p0: {"bind": weakref.proxy(obj)}, 
         **self._param_options, 
     }, 
 )

However, simply proxying the object may cause issues elsewhere. My example code, for instance, threw a ReferenceError when attempting to count the instances of F after deleting f. This other reference check, however, still confirms the leak before the fix and shows that it is resolved by the fix.

import weakref


f = F(x=5)
w = weakref.ref(f)
f.method_gui.show
del f
gc.collect()
print(f"Successfully collected: {w() is None}")

[Antyos]

I did a search for uses of FunctionGui._param_options to see if it switching to a weakref would break anything, but I can't find any uses of it all?

https://github.com/search?q=repo%3Apyapp-kit/magicgui%20_param_options&type=code

_param_options must be used somewhere, right? Otherwise, it can probably be replaced with a weakref without issue.

[Antyos]

I made a simple example of measuring the memory usage from adding 100 widgets to a Container, then clearing the container. The widgets were either a magic_factory, a plain widgets.Container, or a custom Container widget with an embedded magicgui method. Here are the results (code below).

Widget	Initial	Max	Cleared	Delta
magic_factory	0.004 MB	4.37 MB	0.369 MB	4.37 MB
widgets.Container	0.375 MB	3.72 MB	0.557 MB	3.34 MB
CustomContainerWidget	0.563 MB	8.41 MB	8.39 MB	7.85 MB

Note: Memory counter was not reset between runs

Code

import gc
import tracemalloc

import pandas as pd
from magicgui import magic_factory, magicgui, widgets  # noqa: F401


@magic_factory
def func_gui(a=1):
    print(a)


class CustomContainerWidget(widgets.Container):
    def __init__(self, x: int):
        self.x = x
        super().__init__(
            label=f"C(x={self.x})",
            widgets=[self.method_gui],
        )

    @magicgui()
    def method_gui(self, a=1):
        print(a + self.x)


def run_test(factory, count: int = 100):
    # tracemalloc.start()
    container = widgets.Container()
    init_memory = tracemalloc.get_traced_memory()[0]
    for i in range(count):
        container.append(factory())
    max_memory = tracemalloc.get_traced_memory()[0]
    container.clear()
    gc.collect()
    cleared_memory = tracemalloc.get_traced_memory()[0]
    print(
        f"Initial: {init_memory / 10**6:6.2f} MB  |  "
        f"Max: {max_memory / 10**6:6.2f} MB  |  "
        f"Post: {cleared_memory / 10**6:6.2f} MB  |  "
        f"(Δ: {(max_memory - init_memory) / 10**6:6.2f} MB)"
    )
    # tracemalloc.stop()
    return {
        "Initial": init_memory,
        "Max": max_memory,
        "Cleared": cleared_memory,
        "Delta": max_memory - init_memory,
    }


tracemalloc.start()
results = {}

print("Testing magic_factory...")
results["magic_factory"] = run_test(func_gui)
print("Testing widgets.Container...")
results["widgets.Container"] = run_test(
    lambda: widgets.Container(
        label="Container",
        widgets=[widgets.SpinBox(value=1)],
    )
)
print("Testing custom container Widget...")
results["CustomContainerWidget"] = run_test(lambda: CustomContainerWidget(x=1))

print("\nResults (in MB):\n")
print((pd.DataFrame(results) / 1e6).T.to_markdown())

Key takeaways:

• magic_factory and widgets.Container both have a small memory leak. Adding and removing a hundred for into a container, does increase the memory usage from the baseline. ("Cleared" > "Initial". This number increases if you increase the number of added widgets.)
• I didn't reset the memory counter between tests, so you'll notice that the memory occupied by the Container is not cleared after its scope is exited ("Cleared" value from one trial is the same as the "Initial" value of the next).
• Even with the supposed weakref.proxy fix mentioned above, adding and removing a hundred of the CustomContainerWidget does not decrease the memory usage ("Max"=="Cleared").

I will probably continue investigating this, but feedback is welcome.

A side discussion about FunctionGui._param_options that probably doesn't matter.

I found it--it looks like FunctionGui._param_options is used for generating the signature (which makes sense), and is then only stored for later use when creating copies.

magicgui/src/magicgui/widgets/_function_gui.py

Lines 172 to 179 in 2d985e7

	if param_options is None:
	param_options = {}
	elif not isinstance(param_options, dict):
	raise TypeError("'param_options' must be a dict of dicts")
	sig = magic_signature(
	function, gui_options=param_options, raise_on_unknown=raise_on_unknown
	)

magicgui/src/magicgui/widgets/_function_gui.py

Line 211 in 2d985e7

self._param_options = param_options

If that's the case, I think it's probably safe to use the weakref.proxy as noted above, since even if the class itself is deleted, the object will still keep a reference to the

[tlambert03]

Thanks for all your work here @Antyos, and your patience. I've been super busy with a cross atlantic move recently, but i have been following gratefully as you dig into this here. I will try to find the time to respond more thoroughly soon, but also feel free to submit a PR if/when you begin to establish your own opinion on how best to fix this

[Czaki]

@Antyos Could you check your reproducible script? It fails on the current main with ValueError: MagicFactory instance only accept keyword arguments

[Antyos]

@Antyos Could you check your reproducible script? It fails on the current main with ValueError: MagicFactory instance only accept keyword arguments

@Czaki I tried it again on the master branch and it seemed to work fine for me? Do you have more about the traceback? I assume the error is coming from the MagicFactory test. Maybe you can try running the other examples?

Using magicgui version: 0.10.3.dev4+g2f08fb3cd

Code again, just in case.

Run with uv run --active mgui_add_remove.py from the root directory of a local copy of the magicgui repo.

import gc
import tracemalloc

from magicgui import __version__ as mgui_version
from magicgui import magic_factory, magicgui, widgets  # noqa: F401


@magic_factory
def func_gui(a=1):
    print(a)


class CustomContainerWidget(widgets.Container):
    def __init__(self, x: int):
        self.x = x
        super().__init__(
            label=f"C(x={self.x})",
            widgets=[self.method_gui],
        )

    @magicgui()
    def method_gui(self, a=1):
        print(a + self.x)


def run_test(factory, count: int = 100):
    # tracemalloc.start()
    container = widgets.Container()
    init_memory = tracemalloc.get_traced_memory()[0]
    for i in range(count):
        container.append(factory())
    max_memory = tracemalloc.get_traced_memory()[0]
    container.clear()
    gc.collect()
    cleared_memory = tracemalloc.get_traced_memory()[0]
    print(
        f"Initial: {init_memory / 10**6:6.2f} MB  |  "
        f"Max: {max_memory / 10**6:6.2f} MB  |  "
        f"Post: {cleared_memory / 10**6:6.2f} MB  |  "
        f"(Δ: {(max_memory - init_memory) / 10**6:6.2f} MB)"
    )
    # tracemalloc.stop()
    return {
        "Initial": init_memory,
        "Max": max_memory,
        "Cleared": cleared_memory,
        "Delta": max_memory - init_memory,
    }


print(f"magicgui version: {mgui_version}")

tracemalloc.start()
results = {}

print("Testing func_gui...")
results["func_gui"] = run_test(func_gui)
gc.collect()
print("Testing widgets.Container...")
results["widgets.Container"] = run_test(
    lambda: widgets.Container(
        label="Container",
        widgets=[widgets.SpinBox(value=1)],
    )
)
gc.collect()
print("Testing custom container Widget...")
results["CustomContainerWidget"] = run_test(lambda: CustomContainerWidget(x=1))
gc.collect()

[Czaki]

Ok. Found. The error is

╰─$ python bug_reproduce/mgui_leak_memory.py 
Traceback (most recent call last):
  File "/home/czaki/Projects/magicgui/bug_reproduce/mgui_leak_memory.py", line 34, in <module>
    f = F(x=5)
  File "/home/czaki/Projects/magicgui/bug_reproduce/mgui_leak_memory.py", line 18, in __init__
    self.func = self.func_fact()
                ~~~~~~~~~~~~~~^^
  File "/home/czaki/Projects/magicgui/src/magicgui/type_map/_magicgui.py", line 91, in __call__
    raise ValueError("MagicFactory instance only accept keyword arguments")
ValueError: MagicFactory instance only accept keyword arguments                                                                                                                                                                                                                                                                                    /0.3s

But it is caused by running code on Python 3.14.

The second example works ok as it do not use the magic factory on the method.

[Antyos]

Ah, got it. Yes, please disregard that bit from the first code example (that issue is explored in #733).

The second code example is a much simpler case where I was still able to reproduce the issue. (Apologies for the confusion. It might not have been clear that I put code in a collapsed <details> section below the memory usage table in my earlier comment.)