Skip to content

Stabilize Harmony optimizer updates - #293

Open
fderop wants to merge 1 commit into
immunogenomics:masterfrom
fderop:fix/optimizer-reliability
Open

Stabilize Harmony optimizer updates#293
fderop wants to merge 1 commit into
immunogenomics:masterfrom
fderop:fix/optimizer-reliability

Conversation

@fderop

@fderop fderop commented Aug 31, 2026

Copy link
Copy Markdown

Problem

Harmony can report convergence after the outer objective increases. Direct float exponentiation can also produce nonfinite assignments and objectives for valid parameter values.

A run can then return finite corrected coordinates with invalid optimizer state. Results will change for affected runs that stopped early or produced nonfinite assignments.

Original code

Rcells = -dist_matcells;
Rcells.each_col() /= sigma;
Rcells = exp(Rcells);
Rcells = arma::normalise(Rcells, 1, 0);

if ((obj_old - obj_new) / abs(obj_old) < epsilon_harmony) {
  return(true);
}

The assignment code exponentiates unshifted float scores. If all cluster weights underflow for one cell, normalization divides by zero.

An objective increase makes the relative decrease negative. Every negative value is less than a positive epsilon, so the code reports convergence.

Evidence

The bundled cell_lines_small data contains 300 cells. A normal one-variable run used dataset, theta=1, sigma=0.02, and five clusters.

The old code stopped after three outer rounds when the objective increased:

648.9962 -> 652.6208

A float build also failed with donor labels, theta=1, sigma=0.01, and five clusters. Assignments and objectives became nonfinite, but corrected coordinates stayed finite.

Fix

Assignment scores now use double-precision log space. The code adds diversity terms before it subtracts each cell maximum and normalizes the scores.

Outer convergence now requires a finite, nonnegative relative decrease. A nonfinite optimizer state now raises a numerical error before correction or return.

The log-space assignment update contains the same multi-covariate product as #292.

Verification

The same dataset run now completes all ten requested outer rounds. It does not label the objective increase as convergence.

The donor run now keeps assignments, objectives, and coordinates finite. Donor and 10x chemistry cases also match the multiplicative assignment formula.

At sigma=0.1, the maximum assignment change is 1.17e-6. The maximum corrected-coordinate change is 1.15e-7.

The existing test suite passes. R CMD check --no-manual --timings completes with Status: OK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant