Document using limiting APIs for Python thread pools

README.md

@@ -8,19 +8,6 @@ Fine control of the underlying thread-pool size can be useful in
 workloads that involve nested parallelism so as to mitigate
 oversubscription issues.
 
-> **Important:** In its current state, `threadpoolctl` is only designed for
-> situations where BLAS and OpenMP are only called from the main Python thread.
-> Or, to be more accurate, `threadpoolctl` and BLAS/OpenMP APIs should only ever
-> called from the same, single Python thread. For example:
->
-> * When you're using it to configure a worker in a process pool, which then calls BLAS or OpenMP APIs directly in the main thread.
-> * A Jupyter notebook, where the BLAS or OpenMP APIs are being called from code running in the cell's main thread.
->
-> However, once you start calling BLAS or OpenMP APIs and `threadpoolctl` from
-> multiple different Python threads, the impact of the `threadpoolctl` limiting
-> APIs will be very inconsistent. For more details and a plan to fix this, see
-> https://github.com/joblib/threadpoolctl/issues/208
-
 ## Installation
 
 - For users, install the last published version from PyPI:
@@ -43,7 +30,7 @@ oversubscription issues.
   pytest
   ```
 
-## Usage
+## Usage: Introspection and debugging
 
 ### Command Line Interface
 
@@ -152,6 +139,22 @@ The state of these libraries is also accessible through the object oriented API:
 True
 ```
 
+## Usage when not using Python threads: Restricting Controlled Library Thread Pool Sizes
+
+There a two scenarios in which you might want to use `threadpoolctl`; each
+requires you to use different APIs.
+
+1. You do not expect to use any Python threads, so all the work will be started
+   directly from the main thread in the process. This is a simple case
+   where we can globally set thread limits.
+2. You will be parallelizing work using a Python thread pool, and your goal is
+   therefore to limit controlled libraries' thread pool sizes when
+   concurrently called from Python threads. This case is a bit more
+   complex to handle properly and requires a bit more verbose code.
+
+This section will cover the former case, and the latter is covered in the next
+usage section.
+
 ### Setting the Maximum Size of Thread-Pools
 
 Control the number of threads used by the underlying runtime libraries
@@ -184,11 +187,11 @@ however not act on libraries loaded after the instantiation of the
 ...     a_squared = a @ a
 ```
 
-### Restricting the limits to the scope of a function
+### Restricting the Limits to the Scope of a Function
 
 `threadpool_limits` and `ThreadpoolController` can also be used as decorators to set
 the maximum number of threads used by the supported libraries at a function level. The
-decorators are accessible through their `wrap` method:
+decorators are accessible through their `wrap` method.
 
 ```python
 >>> from threadpoolctl import ThreadpoolController, threadpool_limits
@@ -205,6 +208,120 @@ decorators are accessible through their `wrap` method:
 ...
 ```
 
+## Usage for Python threads: Restricting Controlled Library Thread Pool Sizes
+
+This section covers APIs to use when you will be using Python thread pools to parallelize work.
+
+### Setting the Maximum Size of Thread-Pools, When Python Thread Pools Are Used
+
+Limiting thread pool size in controlled libraries requires a two-step process.
+**Importantly, each Python worker thread must also call a method to limit
+controlled libraries in that thread.** With Python's
+`concurrent.futures.ThreadPoolExecutor`, you can do so by passing in an
+initializer function that will get called on thread startup.
+
+```python
+from threadpoolctl import threadpool_limits
+from concurrent.futures import ThreadPoolExecutor
+
+# This top-level limiter doesn't actually change the limits initially; it is
+# there to ensure the limits are reset _after_ the Python thread pool is done.
+# This is necessary because some underlying limiting APIs operate on a
+# process-wide basis.
+with threadpool_limits():
+    # Make sure each Python worker thread also calls threadpool_limits(). If
+    # you're using another thread pool class, you will need to do so some other
+    # way.
+    with ThreadPoolExecutor(4, initializer=lambda: threadpool_limits(limits=1)) as pool:
+        # ... run some BLAS-using code in the thread pool ...
+        pool.map(somefunc, someargs)
+```
+
+To prevent loading shared libraries repeatedly, you can reuse a
+`ThreadpoolController` object:
+
+```python
+from threadpoolctl import ThreadpoolController
+
+# This won't have any side-effects:
+CONTROLLER = ThreadpoolController()
+
+with (
+    CONTROLLER.limit(),
+    ThreadPoolExecutor(4, initializer=lambda: CONTROLLER.limit(limits=1)) as pool,
+):
+    # ... run some BLAS-using code in the thread pool ...
+    pool.map(somefunc, someargs)
+
+# Later...
+with (
+    CONTROLLER.limit(),
+    ThreadPoolExecutor(4, initializer=lambda: CONTROLLER.limit(limits=2)) as pool,
+):
+    # ... run some BLAS-using code in the thread pool ...
+    pool.map(somefunc, someargs)
+```
+
+You can also operate without a context manager:
+
+```python
+from threadpoolctl import ThreadpoolController
+
+CONTROLLER = ThreadpoolController():
+try:
+    limiter = controller.limit()
+    with ThreadPoolExecutor(
+            4, initializer=lambda: controller.limit(limits=1)) as pool:
+        # ... run some BLAS-using code in the thread pool ...
+        pool.map(somefunc, someargs)
+finally:
+    limiter.restore_original_limits()
+
+```
+
+### Switching Back And Forth Between Main Thread and Python Threads
+
+Unfortunately not all controlled libraries providing limiting APIs that are
+thread-specific. Limiting some libraries' thread pool sizes can therefore impact
+the whole process. This makes switching back and forth between running code that
+uses these libraries in Python threads and running it in the main thread a bit
+more complex: you need to set the limits each time you switch back and forth.
+
+Let's say your computer has 4 cores, and you're using some OpenMP API.
+
+```python
+POOL = ThreadPoolExecutor(4)
+CONTROLLER = ThreadpoolController()
+
+# 1. Run some work in a Python thread pool, which then runs in OpenMP.
+with CONTROLLER.limit(limits=1) as limiter:
+
+    def limit_then_do_work(*args, **kwargs):
+        # Set a limit on OpenMP in the current thread:
+        CONTROLLER.limit(limits=1)
+        # Do the actual work:
+        return do_real_work_with_openmp(*args, **kwargs)
+
+    results = POOL.map(limit_then_do_work, args)
+
+
+# 2. Run some work directly in main thread, using OpenMP.
+with CONTROLLER.limit(limits=1):
+    results2 = do_more_work_with_openmp(results)
+
+
+# 3. Do more work in a Python thread pool, this time with more parallelism:
+with CONTROLLER.limit(limits=2) as limiter:
+
+    def limit_then_do_work2(*args, **kwargs):
+        limiter.limit(limits=2)
+        return do_even_more_real_work_with_openmp(*args, **kwargs)
+
+    results3 = POOL.map(limit_then_do_work2, results2)
+```
+
+## Usage: Additional APIs and details
+
 ### Switching the FlexiBLAS backend
 
 `FlexiBLAS` is a BLAS wrapper for which the BLAS backend can be switched at runtime.
@@ -291,6 +408,7 @@ that this part of the API is experimental and subject to change without deprecat
 You can observe that the previously linked OpenBLAS shared object stays loaded by
 the Python program indefinitely, but FlexiBLAS itself no longer delegates BLAS calls
 to OpenBLAS as indicated by the `current_backend` attribute.
+
 ### Writing a custom library controller
 
 Currently, `threadpoolctl` has support for `OpenMP` and the main `BLAS` libraries.

pyproject.toml

@@ -27,3 +27,6 @@ homepage = "https://github.com/joblib/threadpoolctl"
 line-length = 88
 target_version = ['py39', 'py310', 'py311', 'py312', 'py313']
 preview = true
+
+[tool.ruff]
+line-length = 88