Skip to content

Improve integer CopySign with branchless bit-twiddling#130563

Open
tannergooding wants to merge 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-branchless-integer-copysign
Open

Improve integer CopySign with branchless bit-twiddling#130563
tannergooding wants to merge 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-branchless-integer-copysign

Conversation

@tannergooding

@tannergooding tannergooding commented Jul 11, 2026

Copy link
Copy Markdown
Member

Fixes #77579

The signed integer CopySign implementations used a data-dependent abs-then-negate sequence with two branches on the hot path. This replaces it with the standard branchless conditional negate:

signMask = (value ^ sign) >> (width - 1)   // all-ones iff the signs differ
result   = (value ^ signMask) - signMask   // (x ^ -1) - (-1) == -x; (x ^ 0) - 0 == x

which produces value with the sign of sign in a handful of ALU ops. A single predicted-not-taken branch is kept to preserve the OverflowException that CopySign(MinValue, non-negative) must still throw (the only case where result comes back negative for a non-negative sign).

Applied to sbyte, short, int, long, nint, and Int128. Int128 uses the same general form; a manual per-limb variant was measured and lowered identically, so the clearer form is used.

The floating-point and decimal CopySign paths already use bit/flag manipulation (or Vector128.ConditionalSelect), so they're unchanged.

Correctness of the transform (including the overflow-throw case) was verified exhaustively over the full 8-bit domain and, for Int128, against the width-generic reference over edge values plus randomized 128-bit inputs. Added CopySignTest coverage to the generic-math tests for each signed integer type.


Performance (array-loop over random inputs; ratio is new/old, lower is better):

Type arm64 (Apple M4) x64
int ~2.0x faster ~1.1x faster
long ~2.0x faster ~1.3x faster
Int128 ~1.7x faster roughly neutral (µarch-dependent)

The scalar types improve on every platform measured. Int128 is a solid win on arm64; on x64 it's roughly neutral and microarchitecture-dependent (measured a small gain on Intel Emerald Rapids and a small regression on AMD EPYC Turin), which is expected since x64 absorbs the original well-predicted branch cheaply and Int128 isn't hardware accelerated. A separate A/B run confirmed the general form matches the manual per-limb form and beats a compute-both-and-select alternative.

Note

This PR description was generated with the assistance of GitHub Copilot.

The signed integer CopySign implementations used a data-dependent
abs-then-negate sequence with two branches on the hot path. Replace it
with the standard branchless conditional negate:

    signMask = (value ^ sign) >> (width - 1)
    result   = (value ^ signMask) - signMask

which produces value with the sign of sign in a few ALU ops, keeping a
single predicted-not-taken branch to preserve the OverflowException that
CopySign(MinValue, non-negative) must still throw.

Int128 isn't hardware accelerated, so it operates on the lower/upper
halves directly and computes the sign mask once rather than materializing
the intermediate Int128 values the general form would produce.

Add CopySign coverage to the generic-math tests for each signed integer
type, including the MinValue overflow cases.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tannergooding

Copy link
Copy Markdown
Member Author

@EgorBot -linux_amd -osx_arm64

using System;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    private const int N = 1024;
    private int[] _i32 = new int[N], _i32Sign = new int[N];
    private long[] _i64 = new long[N], _i64Sign = new long[N];
    private Int128[] _i128 = new Int128[N], _i128Sign = new Int128[N];

    [GlobalSetup]
    public void Setup()
    {
        var r = new Random(42);
        for (int i = 0; i < N; i++)
        {
            // Non-negative magnitudes with a random sign; avoids MinValue so the overflow path never trips.
            _i32[i] = r.Next() * (r.Next(2) == 0 ? 1 : -1);
            _i32Sign[i] = r.Next() * (r.Next(2) == 0 ? 1 : -1);
            _i64[i] = (r.NextInt64() & long.MaxValue) * (r.Next(2) == 0 ? 1 : -1);
            _i64Sign[i] = (r.NextInt64() & long.MaxValue) * (r.Next(2) == 0 ? 1 : -1);
            _i128[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
            _i128Sign[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
        }
    }

    [Benchmark]
    public int Int32CopySign()
    {
        int acc = 0;
        int[] a = _i32, b = _i32Sign;
        for (int i = 0; i < a.Length; i++)
            acc += int.CopySign(a[i], b[i]);
        return acc;
    }

    [Benchmark]
    public long Int64CopySign()
    {
        long acc = 0;
        long[] a = _i64, b = _i64Sign;
        for (int i = 0; i < a.Length; i++)
            acc += long.CopySign(a[i], b[i]);
        return acc;
    }

    [Benchmark]
    public Int128 Int128CopySign()
    {
        Int128 acc = 0;
        Int128[] a = _i128, b = _i128Sign;
        for (int i = 0; i < a.Length; i++)
            acc += Int128.CopySign(a[i], b[i]);
        return acc;
    }
}

Note

This benchmark comment was generated with the assistance of GitHub Copilot.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the signed-integer CopySign implementations in System.Private.CoreLib to use a branchless conditional-negate pattern (with a preserved overflow-throw for the MinValue + non-negative sign case), and adds corresponding generic-math test coverage in System.Runtime.Tests.

Changes:

  • Replace CopySign implementations for sbyte, short, int, long, nint, and Int128 with a branchless sign-adjustment approach while preserving the OverflowException behavior for MinValue with non-negative sign.
  • Add CopySignTest coverage for each affected signed integer type in the generic-math tests.
  • Implement an Int128-specific half-limb version to avoid extra intermediate Int128 operations.
Show a summary per file
File Description
src/libraries/System.Private.CoreLib/src/System/SByte.cs Reworks sbyte.CopySign to compute sign mask + conditional negate, preserving overflow throw behavior.
src/libraries/System.Private.CoreLib/src/System/Int16.cs Reworks short.CopySign with the same branchless approach and overflow preservation.
src/libraries/System.Private.CoreLib/src/System/Int32.cs Reworks int.CopySign to the branchless mask/negate form with explicit overflow throw.
src/libraries/System.Private.CoreLib/src/System/Int64.cs Reworks long.CopySign similarly, using a 63-bit sign extraction.
src/libraries/System.Private.CoreLib/src/System/IntPtr.cs Reworks nint.CopySign using pointer-width sign extraction and preserved overflow behavior.
src/libraries/System.Private.CoreLib/src/System/Int128.cs Reworks Int128.CopySign using half-limb operations + single computed sign mask, with overflow preservation.
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/SByteTests.GenericMath.cs Adds CopySignTest coverage for sbyte.
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int16Tests.GenericMath.cs Adds CopySignTest coverage for short.
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int32Tests.GenericMath.cs Adds CopySignTest coverage for int.
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int64Tests.GenericMath.cs Adds CopySignTest coverage for long.
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.GenericMath.cs Adds CopySignTest coverage for nint, with 32-bit vs 64-bit expectations.
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int128Tests.GenericMath.cs Adds CopySignTest coverage for Int128, including overflow cases.

Copilot's findings

  • Files reviewed: 12/12 changed files
  • Comments generated: 0

@tannergooding

Copy link
Copy Markdown
Member Author

@EgorBot -linux_intel -osx_arm64

using System;
using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    private const int N = 1024;
    private Int128[] _v = new Int128[N], _s = new Int128[N];

    [GlobalSetup]
    public void Setup()
    {
        var r = new Random(42);
        for (int i = 0; i < N; i++)
        {
            // Non-negative magnitudes with random sign; avoids MinValue so no overflow path.
            _v[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
            _s[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
        }
    }

    [Benchmark(Baseline = true)]
    public Int128 Branchy()
    {
        Int128 acc = 0;
        var v = _v; var s = _s;
        for (int i = 0; i < v.Length; i++) acc += BranchyImpl(v[i], s[i]);
        return acc;
    }

    [Benchmark]
    public Int128 Limb()
    {
        Int128 acc = 0;
        var v = _v; var s = _s;
        for (int i = 0; i < v.Length; i++) acc += LimbImpl(v[i], s[i]);
        return acc;
    }

    [Benchmark]
    public Int128 Operator()
    {
        Int128 acc = 0;
        var v = _v; var s = _s;
        for (int i = 0; i < v.Length; i++) acc += OperatorImpl(v[i], s[i]);
        return acc;
    }

    [Benchmark]
    public Int128 Select()
    {
        Int128 acc = 0;
        var v = _v; var s = _s;
        for (int i = 0; i < v.Length; i++) acc += SelectImpl(v[i], s[i]);
        return acc;
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static Int128 BranchyImpl(Int128 value, Int128 sign)
    {
        Int128 absValue = value;
        if (Int128.IsNegative(absValue)) absValue = -absValue;
        if (Int128.IsPositive(sign))
        {
            if (Int128.IsNegative(absValue)) throw new OverflowException();
            return absValue;
        }
        return -absValue;
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static Int128 LimbImpl(Int128 value, Int128 sign)
    {
        ulong vl = (ulong)value, vu = (ulong)(value >> 64);
        ulong su = (ulong)(sign >> 64);
        ulong m = (ulong)((long)(vu ^ su) >> 63);
        ulong lo = vl ^ m, hi = vu ^ m;
        ulong rlo = lo - m;
        ulong borrow = (rlo > lo) ? 1UL : 0UL;
        ulong rhi = hi - m - borrow;
        if (((long)su >= 0) && ((long)rhi < 0)) throw new OverflowException();
        return new Int128(rhi, rlo);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static Int128 OperatorImpl(Int128 value, Int128 sign)
    {
        Int128 signMask = (value ^ sign) >> 127;
        Int128 result = (value ^ signMask) - signMask;
        if (Int128.IsPositive(sign) && Int128.IsNegative(result)) throw new OverflowException();
        return result;
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static Int128 SelectImpl(Int128 value, Int128 sign)
    {
        Int128 neg = -value;
        Int128 result = Int128.IsNegative(value ^ sign) ? neg : value;
        if (Int128.IsPositive(sign) && Int128.IsNegative(result)) throw new OverflowException();
        return result;
    }
}

Note

This benchmark comment was generated with the assistance of GitHub Copilot.

The manual limb-based borrow lowered identically to the general
(value ^ signMask) - signMask form in benchmarks, so use the clearer
form that mirrors the scalar integer types

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 11, 2026 23:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 12/12 changed files
  • Comments generated: 1

Comment thread src/libraries/System.Private.CoreLib/src/System/Int128.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CopySign is slow

2 participants