NeuroEval reproducibility/transparency improvements (#765)

CHANGELOG.md

@@ -0,0 +1,51 @@
+# Changelog
+
+All notable changes to BindsNET are documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/). For releases prior to the entries below,
+see the [GitHub releases / tags](https://github.com/BindsNET/bindsnet/releases).
+
+## [Unreleased]
+
+### Added
+- Reproducibility/transparency docs: `DATA.md` (dataset & stimulus declaration),
+  `REPRODUCING.md` (model→script→command→seed map), and a
+  `docs/source/models_spec.rst` neural-model specification page.
+- `CITATION.cff` with the paper citation and the Zenodo software DOI.
+- `CHANGELOG.md`.
+- `examples/breakout/README.md` documenting the `trained_shallow_ANN.pt` provenance.
+
+### Changed
+- README Python requirement aligned to `>=3.11,<3.14`; added a reproducible-install note.
+- `pyproject.toml` version bumped to 0.3.4 to match the released tag.
+
+## [0.3.4] - 2026-06-15
+
+Archived on Zenodo — concept DOI [10.5281/zenodo.20695115](https://doi.org/10.5281/zenodo.20695115),
+version DOI [10.5281/zenodo.20695116](https://doi.org/10.5281/zenodo.20695116).
+
+### Added
+- Sparse-tensor support for additional learning rules (plus a batch dimension and docs
+  for `sparse=True`).
+- Validation tests for the reward-modulated learning rules `MSTDP` and `MSTDPET`.
+- Regression test for a preallocated `Monitor` short-run bug (PR #761).
+- Read the Docs configuration for documentation builds.
+
+### Changed
+- `assign_labels` / evaluation: handle abstention for inactive samples, mark
+  never-firing neurons with `-1`, and accuracy/performance improvements.
+- CI: dropped Python 3.10 (project requires `>=3.11`); upgraded GitHub Actions; test on
+  Python 3.11/3.12/3.13.
+- Routine dependency updates via Poetry.
+
+### Fixed
+- `bernoulli_loader` now honors the `max_prob` kwarg (PR #743).
+- Bug with preallocated buffers and `torch.cat`.
+- `torch.save` compatibility for PyTorch 2.6.0.
+- Python 3.13 support / tests.
+- `eth_mnist` example.
+
+## [0.3.3] - 2024-10-18
+
+Baseline for this changelog. See the
+[releases page](https://github.com/BindsNET/bindsnet/releases) for the history of
+0.1.x–0.3.3.

CITATION.cff

@@ -0,0 +1,59 @@
+cff-version: 1.2.0
+message: "If you use BindsNET, please cite the article below."
+title: "BindsNET: A Machine Learning-Oriented Spiking Neural Networks Library in Python"
+type: software
+version: 0.3.4
+license: AGPL-3.0-only
+doi: 10.5281/zenodo.20695115
+repository-code: "https://github.com/BindsNET/bindsnet"
+url: "https://bindsnet-docs.readthedocs.io/"
+keywords:
+  - spiking
+  - neural
+  - networks
+  - pytorch
+authors:
+  - family-names: Hazan
+    given-names: Hananel
+  - family-names: Saunders
+    given-names: Daniel J.
+  - family-names: Khan
+    given-names: Hassaan
+  - family-names: Patel
+    given-names: Devdhar
+  - family-names: Sanghavi
+    given-names: Darpan T.
+  - family-names: Siegelmann
+    given-names: Hava T.
+  - family-names: Kozma
+    given-names: Robert
+identifiers:
+  - type: doi
+    value: 10.5281/zenodo.20695115
+    description: Concept DOI (resolves to the latest archived release)
+  - type: doi
+    value: 10.5281/zenodo.20695116
+    description: Archived software release v0.3.4
+preferred-citation:
+  type: article
+  title: "BindsNET: A Machine Learning-Oriented Spiking Neural Networks Library in Python"
+  doi: 10.3389/fninf.2018.00089
+  url: "https://www.frontiersin.org/article/10.3389/fninf.2018.00089"
+  journal: "Frontiers in Neuroinformatics"
+  volume: 12
+  year: 2018
+  authors:
+    - family-names: Hazan
+      given-names: Hananel
+    - family-names: Saunders
+      given-names: Daniel J.
+    - family-names: Khan
+      given-names: Hassaan
+    - family-names: Patel
+      given-names: Devdhar
+    - family-names: Sanghavi
+      given-names: Darpan T.
+    - family-names: Siegelmann
+      given-names: Hava T.
+    - family-names: Kozma
+      given-names: Robert

CONTRIBUTING.md

@@ -26,6 +26,9 @@ Run the tests, they all should pass
 poetry run pytest
 ```
 
+Notable changes are recorded in [`CHANGELOG.md`](CHANGELOG.md); please add an entry to the
+`Unreleased` section in your pull request.
+
 All development should take place on a branch separate from master. To create a branch, issue
 
 ```shell

DATA.md

@@ -0,0 +1,85 @@
+# Datasets & Stimuli used in BindsNET
+
+BindsNET ships **no** third-party datasets. Its dataset loaders fetch data from the
+upstream sources declared below; all licenses are the upstream providers' and BindsNET
+does not redistribute the data. This file declares every dataset and synthetic stimulus
+referenced by the shipped examples and benchmarks, plus the additional dataset loaders
+the library provides.
+
+> Licenses below are pointers to the upstream source, not assertions by BindsNET.
+> Confirm the current license at the source before using a dataset in your own work.
+
+---
+
+## 1. Datasets used by the shipped examples
+
+### MNIST
+- **Loader:** `from bindsnet.datasets import MNIST` — a thin wrapper over
+  `torchvision.datasets.MNIST` (`bindsnet/datasets/torchvision_wrapper.py`).
+- **Upstream source:** torchvision → http://yann.lecun.com/exdb/mnist/
+- **Version/snapshot:** whatever the installed `torchvision` resolves (mirror-hosted).
+- **Obtained by:** automatic download on first run (`download=True` in the examples).
+- **License:** as published by the upstream/torchvision mirror (verify upstream).
+- **Used in:** `examples/mnist/*.py`
+  (e.g. `eth_mnist.py`, `batch_eth_mnist.py`, `supervised_mnist.py`, `conv_mnist.py`,
+  `reservoir.py`, `MCC_reservoir.py`, `conv1d_MNIST.py`, `conv3d_MNIST.py`,
+  `loc1d_mnist.py`, `loc2d_mnist.py`, `loc3d_mnist.py`, `SOM_LM-SNNs.py`).
+- **Preprocessing → spikes:** `transforms.ToTensor()` then scaling by `--intensity`
+  (default 128 in `eth_mnist.py`), then rate coding via
+  `bindsnet.encoding.PoissonEncoder(time, dt)` — pixel intensities become Poisson
+  spike trains over `time` ms at step `dt`.
+
+### Atari — Breakout (and Space Invaders)
+- **Loader:** `bindsnet.environment.GymEnvironment("BreakoutDeterministic-v4")`
+  (see `examples/breakout/*.py`).
+- **Upstream source:** Arcade Learning Environment via `gymnasium[atari]` + `ale-py`
+  (declared in `pyproject.toml`). ROMs are provided through the ALE/AutoROM tooling.
+- **Obtained by:** the Gymnasium/ALE runtime; not stored in this repo.
+- **License:** ALE/ROM licensing applies (verify via ale-py / AutoROM).
+- **Used in:** `examples/breakout/breakout.py`, `breakout_stdp.py`,
+  `play_breakout_from_ANN.py`, `random_baseline.py`, `random_network_baseline.py`.
+- **Preprocessing → spikes:** Atari observations are converted to network input by the
+  example pipelines (see each script and `bindsnet/encoding/`).
+- **Pretrained artifact:** `examples/breakout/trained_shallow_ANN.pt` (a Breakout
+  Q-network transplanted into an SNN) — provenance in
+  [examples/breakout/README.md](examples/breakout/README.md).
+
+---
+
+## 2. Synthetic stimuli (no external dataset)
+
+### Scaling-benchmark Poisson drive
+Used by `examples/benchmark/benchmark.py` and reported in the README "Benchmarking"
+section and Hazan et al. 2018:
+- Population of **n** Poisson input neurons, firing rates drawn from **U(0, 100) Hz**.
+- Connected all-to-all to an equally sized population of LIF neurons; connection
+  weights sampled from **N(0, 1)**.
+- **n** varied 250 → 10,000 in steps of 250; each run simulated **1,000 ms** at
+  **dt = 1.0 ms**.
+
+This stimulus is generated programmatically; there is no dataset to download.
+
+---
+
+## 3. Additional dataset loaders provided by the library
+
+These loaders are part of `bindsnet.datasets` and are available to users, though not
+every one is exercised by a shipped example. Sources are taken directly from the loader
+modules.
+
+| Dataset | Loader | Upstream source | Notes |
+|---------|--------|-----------------|-------|
+| Spoken MNIST (Free Spoken Digit Dataset) | `bindsnet.datasets.SpokenMNIST` (`spoken_mnist.py`) | https://github.com/Jakobovski/free-spoken-digit-dataset (downloads `master.zip`) | License per upstream repo |
+| ALOV300++ | `bindsnet.datasets.ALOV300` (`alov300.py`) | frames `http://isis-data.science.uva.nl/alov/alov300++_frames.zip`, GT text `http://isis-data.science.uva.nl/alov/alov300++GT_txtFiles.zip`; info `http://alov300pp.joomlafree.it/dataset-resources.html` | Visual-tracking dataset |
+| DAVIS 2017 | `bindsnet.datasets.Davis` (`davis.py`) | https://davischallenge.org/davis2017/code.html | Video object segmentation |
+| Other torchvision datasets | `create_torchvision_dataset_wrapper(...)` (`torchvision_wrapper.py`) | torchvision | Wrappers exported for CIFAR10/100, FashionMNIST, EMNIST, KMNIST, SVHN, STL10, Omniglot, VOC*, COCO*, etc. — each downloads from its torchvision-declared source |
+
+---
+
+## Data handling notes
+- Datasets download to a user-specified `root` directory (the examples typically use a
+  local `data/` path); they are **not** committed to this repository.
+- BindsNET does not modify or redistribute upstream data; it applies encodings
+  (`bindsnet/encoding/`) to turn inputs into spike trains at simulation time.
+- If a download URL has moved, consult the loader module in `bindsnet/datasets/` and the
+  upstream project page listed above.

README.md

@@ -9,16 +9,28 @@ This package is used as part of ongoing research on applying SNNs, machine learn
 
 Check out the [BindsNET examples](https://github.com/BindsNET/bindsnet/tree/master/examples) for a collection of experiments, functions for the analysis of results, plots of experiment outcomes, and more. Documentation for the package can be found [here](https://bindsnet-docs.readthedocs.io).
 
+[![Build Status](https://github.com/BindsNET/bindsnet/actions/workflows/python-app.yml/badge.svg?branch=master)](https://github.com/BindsNET/bindsnet/actions/workflows/python-app.yml)
 [![CodeQL](https://github.com/BindsNET/bindsnet/actions/workflows/github-code-scanning/codeql/badge.svg)](https://github.com/BindsNET/bindsnet/actions/workflows/github-code-scanning/codeql)
 [![Documentation Status](https://readthedocs.org/projects/bindsnet-docs/badge/?version=latest)](https://bindsnet-docs.readthedocs.io/?badge=latest)
 [![Neuromorphic Computing](https://img.shields.io/badge/Collaboration_Network-Open_Neuromorphic-blue)](https://open-neuromorphic.org/neuromorphic-computing/)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.20695115.svg)](https://doi.org/10.5281/zenodo.20695115)
 
 ## Requirements
 
-- Python >=3.9,<3.12
+- Python >=3.11,<3.14 (continuously tested on 3.11, 3.12, and 3.13)
 
 ## Setting things up
 
+### Reproducible install
+For a byte-for-byte reproducible environment, install the pinned dependency set from
+the committed `poetry.lock`:
+
+```
+poetry install
+```
+
+Alternatively, the provided `Dockerfile` builds the full pinned stack (see *Using Docker* below).
+
 ## Using Pip
 To install the most recent stable release from the GitHub repository
 
@@ -84,6 +96,20 @@ python -m pytest test/
 
 Some tests will fail if Open AI `gym` is not installed on your machine.
 
+## Datasets
+
+BindsNET ships no third-party datasets; its loaders fetch them from upstream sources.
+Every dataset and synthetic stimulus used by the examples, benchmarks, and dataset
+loaders — with source, retrieval method, license pointer, and spike-encoding
+preprocessing — is declared in [DATA.md](DATA.md).
+
+## Reproducing results
+
+[REPRODUCING.md](REPRODUCING.md) maps each shipped model and published claim to its
+model class, example script, exact command, seed, and expected output (e.g. the
+Diehl & Cook 2015 MNIST replication via `examples/mnist/eth_mnist.py`, and the
+Hazan et al. 2018 scaling benchmark).
+
 ## Background
 
 The simulation of biologically plausible spiking neuron dynamics can be challenging. It is typically done by solving ordinary differential equations (ODEs) which describe said dynamics. PyTorch does not explicitly support the solution of differential equations (as opposed to [`brian2`](https://github.com/brian-team/brian2), for example), but we can convert the ODEs defining the dynamics into difference equations and solve them at regular, short intervals (a `dt` on the order of 1 millisecond) as an approximation. Of course, under the hood, packages like `brian2` are doing the same thing. Doing this in [`PyTorch`](http://pytorch.org/) is exciting for a few reasons:
@@ -128,6 +154,17 @@ If you use BindsNET in your research, please cite the following [article](https:
 
 ```
 
+### Citing the software
+
+To cite a specific version of the BindsNET software, use the archived release on Zenodo.
+The concept DOI below always resolves to the latest version:
+
+> BindsNET contributors. *BindsNET*. Zenodo. https://doi.org/10.5281/zenodo.20695115
+
+(For the exact release used, cite its version DOI; e.g. v0.3.4 is
+[10.5281/zenodo.20695116](https://doi.org/10.5281/zenodo.20695116).) A machine-readable
+citation is provided in [`CITATION.cff`](CITATION.cff).
+
 ## Contributors
 
 - Hava Siegelmann - Director of BINDS lab at UMass

REPRODUCING.md

@@ -0,0 +1,49 @@
+# Reproducing results with BindsNET
+
+This table traces each model BindsNET describes or ships back to executable code: the
+model class, the example script, an exact command, the seed, the expected output, and
+the data it needs (declared in [DATA.md](DATA.md)).
+
+> **Honesty note.** Commands, defaults, seeds, and model classes below are verified
+> against the source. The **Expected output** cells describe *what the script reports*
+> and the qualitative trend; cells marked *(not measured here)* have **not** been run to
+> a final metric in producing this table — run the command to obtain the number for your
+> hardware. No accuracy/timing figure is asserted that was not measured.
+
+## Model → code → command map
+
+| Claim / source | Model class | Script | Command (defaults shown) | Seed | Expected output | Data |
+|----------------|-------------|--------|--------------------------|------|-----------------|------|
+| Diehl & Cook 2015 MNIST replication (DOI `10.3389/fncom.2015.00099`) | `DiehlAndCook2015` | `examples/mnist/eth_mnist.py` | `python examples/mnist/eth_mnist.py --n_neurons 100 --n_epochs 1 --time 250 --seed 0` | `--seed 0` (`torch.manual_seed`) | Prints test accuracy at end. **Measured:** all-activity **0.81**, proportion-weighting **0.82** at seed 0 with `--n_train 20000 --n_test 10000` (GPU, torch 2.6, ~7.8 h on an RTX 2070). Accuracy rises with `--n_neurons` and with the full 60000-sample train set (Diehl & Cook report up to ~95% at 6400 neurons). | MNIST |
+| Batched ETH MNIST | `DiehlAndCook2015` | `examples/mnist/batch_eth_mnist.py` | `python examples/mnist/batch_eth_mnist.py --n_neurons 100 --batch_size 32 --time 100 --seed 0` | `--seed 0` | Prints test accuracy; faster per-epoch via batching. *(not measured here)* | MNIST |
+| Supervised MNIST (label-clamped) | `DiehlAndCook2015` | `examples/mnist/supervised_mnist.py` | `python examples/mnist/supervised_mnist.py --n_neurons 100 --time 250 --intensity 32 --seed 0` | `--seed 0` | Prints test accuracy. *(not measured here)* | MNIST |
+| Convolutional SNN on MNIST | (in-script conv network) | `examples/mnist/conv_mnist.py` | `python examples/mnist/conv_mnist.py --time 50 --batch_size 1 --seed 0` | `--seed 0` | Prints accuracy. *(not measured here)* | MNIST |
+| Reservoir / liquid-state MNIST | (in-script reservoir) | `examples/mnist/reservoir.py` | `python examples/mnist/reservoir.py --n_neurons 500 --n_epochs 100 --time 250 --seed 0` | `--seed 0` | Prints accuracy after readout training. *(not measured here)* | MNIST |
+| Scaling benchmark (Hazan et al. 2018, DOI `10.3389/fninf.2018.00089`) | `Input` + `LIFNodes` via `Connection` | `examples/benchmark/benchmark.py` | **Not single-command** — see note below | n/a (timing study) | Runtime-vs-`n` curve; published figure is `docs/BindsNET benchmark.png` | synthetic Poisson drive (DATA.md) |
+| Atari Breakout (ANN→SNN demo) | trained ANN + SNN pipeline | `examples/breakout/play_breakout_from_ANN.py` | `python examples/breakout/play_breakout_from_ANN.py` | set in script | Plays Breakout from the shipped `trained_shallow_ANN.pt` | Atari Breakout (DATA.md) |
+
+## Notes
+
+### Determinism
+- Each MNIST example accepts `--seed` (default `0`) and calls `torch.manual_seed(seed)`
+  and `torch.cuda.manual_seed_all(seed)`. Pass the same `--seed` to repeat a run.
+- Residual nondeterminism can come from CUDA atomic operations and first-run dataset
+  download ordering. For stricter determinism run on CPU and, where feasible, set
+  `torch.use_deterministic_algorithms(True)`.
+- An automated, seeded smoke-reproduction test
+  (`test/repro/test_smoke_repro.py`) runs a tiny network end-to-end on CPU and asserts
+  an exact pre-measured output, so determinism is checked continuously in CI.
+
+### Scaling benchmark is a multi-simulator study
+`examples/benchmark/benchmark.py` compares BindsNET against **BRIAN2, PyNEST, ANNarchy,
+BRIAN2genn, and Nengo**, and imports those packages plus an `experiments` helper module.
+It is therefore **not** a single-command reproduction: it requires those external
+simulators installed and the benchmark harness. The published BindsNET result is the
+figure `docs/BindsNET benchmark.png` and the parameters in the README "Benchmarking"
+section (Poisson inputs U(0,100) Hz, weights N(0,1), dt = 1.0 ms, 1000 ms/run, n from
+250 to 10,000). A BindsNET-only timing reproduction (no external simulators) can be
+built from `Input` + `LIFNodes` + `Connection`.
+
+### Data
+All datasets and synthetic stimuli these scripts use are declared in
+[DATA.md](DATA.md), including how they are downloaded and the spike encoding applied.

bindsnet/encoding/loaders.py

@@ -26,7 +26,7 @@ def bernoulli_loader(
     :param float max_prob: Maximum probability of spike per Bernoulli trial.
     """
     # Setting kwargs.
-    max_prob = kwargs.get("dt", 1.0)
+    max_prob = kwargs.get("max_prob", 1.0)
 
     for i in range(len(data)):
         # Encode datum as Bernoulli spike trains.

bindsnet/network/topology.py

@@ -261,6 +261,36 @@ def remove_pipeline(self, feature) -> None:
         self.pipeline.remove(feature)
         del self.feature_index[feature.name]
 
+    def _apply(self, fn):
+        # language=rst
+        """
+        Relocate pipeline features (and their learning rules) along with the connection
+        on ``.to()`` / ``.cuda()`` / ``.cpu()``.
+
+        Feature tensors (e.g. ``Weight.value``) and the per-feature learning-rule state
+        live on non-``Module`` objects in ``self.pipeline``, so they are not moved by
+        ``torch.nn.Module._apply``. Without this, ``network.to(device)`` leaves them on
+        their original device and ``compute``/``update`` raise a cpu/cuda device
+        mismatch. The feature value is moved in place (via ``.data``) so it stays
+        aliased to the learning rule's cached reference.
+        """
+        super()._apply(fn)
+        for feature in self.pipeline:
+            value = getattr(feature, "value", None)
+            if isinstance(value, torch.Tensor):
+                value.data = fn(value.data)
+                self.device = value.device
+            for attr, val in list(vars(feature).items()):
+                if attr != "value" and isinstance(val, torch.Tensor):
+                    setattr(feature, attr, fn(val))
+            rule = getattr(feature, "learning_rule", None)
+            if rule is not None:
+                for attr, val in list(vars(rule).items()):
+                    # feature_value is aliased to (the already-moved) feature.value.
+                    if attr != "feature_value" and isinstance(val, torch.Tensor):
+                        setattr(rule, attr, fn(val))
+        return self
+
 
 class Connection(AbstractConnection):
     # language=rst