Add safe context watcher API

[florentinl]

Adds PyContext and safe context-watcher bindings for GIL-enabled CPython 3.14:

• typed context-switch events
• RAII watcher registration
• panic- and error-safe callback trampoline

use pyo3::{context::ContextEvent, prelude::*};

fn context_changed(_py: Python<'_>, event: ContextEvent<'_, '_>) -> PyResult<()> {
    if let ContextEvent::Switched(context) = event {
        println!("current context: {context:?}");
    }
    Ok(())
}

fn main() -> PyResult<()> {
    Python::initialize();

    Python::attach(|py| {
        let _watcher = pyo3::register_context_watcher!(py, context_changed)?;
        py.run(
            c"import contextvars\ncontextvars.Context().run(lambda: None)",
            None,
            None,
        )
    })
}

Tested with unit tests, doctests, and Clippy on Python 3.14.

[chirizxc]

Should we mention this as well?

/// # Thread Safety
///
/// On GIL-enabled builds, callbacks are invoked synchronously. On free-threaded
/// builds (when this API is available), callbacks may be invoked concurrently.
/// Ensure your callback is thread-safe if needed.

[florentinl]

That's true, It should definitely be mentioned if we enable the API on free threaded builds.

The problem is that PyContext_AddWatcher and PyContext_ClearWatcher while available on free threaded are not callable concurrently and require external synchronization, which is not something we can guarantee from an extension.

I was thinking of making this API only available on GIL-enabled builds for this reason.

[bschoenmaeckers]

I think we can should requireSend + Sync + 'static on the provided closure.

[florentinl]

The API in this PR accepts a function path, which is coerced to a function pointer. Function pointers carry no captured state and are already Send + Sync + 'static, so there is nothing additional to enforce.

CPython’s watcher API does not expose a user-data pointer where we could store a closure, which is why I did not try to implement a closure registration api.

[florentinl]

@bschoenmaeckers can I ask you to give it another look ? Thanks

[davidhewitt]

The problem is that PyContext_AddWatcher and PyContext_ClearWatcher while available on free threaded are not callable concurrently and require external synchronization, which is not something we can guarantee from an extension.

Where do you conclude this? We should make a CPython issue for this if not already present.

[florentinl]

Absolutely agree, I raised an issue in CPython for this already: python/cpython#155619

[chirizxc]

LGTM, but im not a maintainer

[florentinl]

Still, thank you for taking a look.

[codspeed-hq]

Merging this PR will not alter performance

✅ 141 untouched benchmarks
⏩ 6 skipped benchmarks¹

---

_{Comparing florentinl:florentin.labelle/add-safe-wrappers-for-context-watching-api (68d00cc) with main (fbc8456)}

Footnotes

1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[davidhewitt]

I would quite like to avoid the macro for this API, it seems unfortunate. I understand why it's there, however. I'm thinking about what we can do as alternatives.

[davidhewitt]

I opened florentinl#1 which has a suggestion how I'd like this API to look.

Also placed a few other comments here.

[davidhewitt]

The problem is that PyContext_AddWatcher and PyContext_ClearWatcher while available on free threaded are not callable concurrently and require external synchronization, which is not something we can guarantee from an extension.

Where do you conclude this? We should make a CPython issue for this if not already present.

[florentinl]

Thank you a lot @davidhewitt for the thorough review and your design suggestion.

I updated the PR to remove the macro API in favor of the design you suggested and tried to address your other comments.