Improve integer CopySign with branchless bit-twiddling#130563
Open
tannergooding wants to merge 2 commits into
Open
Improve integer CopySign with branchless bit-twiddling#130563tannergooding wants to merge 2 commits into
tannergooding wants to merge 2 commits into
Conversation
The signed integer CopySign implementations used a data-dependent
abs-then-negate sequence with two branches on the hot path. Replace it
with the standard branchless conditional negate:
signMask = (value ^ sign) >> (width - 1)
result = (value ^ signMask) - signMask
which produces value with the sign of sign in a few ALU ops, keeping a
single predicted-not-taken branch to preserve the OverflowException that
CopySign(MinValue, non-negative) must still throw.
Int128 isn't hardware accelerated, so it operates on the lower/upper
halves directly and computes the sign mask once rather than materializing
the intermediate Int128 values the general form would produce.
Add CopySign coverage to the generic-math tests for each signed integer
type, including the MinValue overflow cases.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Member
Author
|
@EgorBot -linux_amd -osx_arm64 using System;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);
public class Bench
{
private const int N = 1024;
private int[] _i32 = new int[N], _i32Sign = new int[N];
private long[] _i64 = new long[N], _i64Sign = new long[N];
private Int128[] _i128 = new Int128[N], _i128Sign = new Int128[N];
[GlobalSetup]
public void Setup()
{
var r = new Random(42);
for (int i = 0; i < N; i++)
{
// Non-negative magnitudes with a random sign; avoids MinValue so the overflow path never trips.
_i32[i] = r.Next() * (r.Next(2) == 0 ? 1 : -1);
_i32Sign[i] = r.Next() * (r.Next(2) == 0 ? 1 : -1);
_i64[i] = (r.NextInt64() & long.MaxValue) * (r.Next(2) == 0 ? 1 : -1);
_i64Sign[i] = (r.NextInt64() & long.MaxValue) * (r.Next(2) == 0 ? 1 : -1);
_i128[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
_i128Sign[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
}
}
[Benchmark]
public int Int32CopySign()
{
int acc = 0;
int[] a = _i32, b = _i32Sign;
for (int i = 0; i < a.Length; i++)
acc += int.CopySign(a[i], b[i]);
return acc;
}
[Benchmark]
public long Int64CopySign()
{
long acc = 0;
long[] a = _i64, b = _i64Sign;
for (int i = 0; i < a.Length; i++)
acc += long.CopySign(a[i], b[i]);
return acc;
}
[Benchmark]
public Int128 Int128CopySign()
{
Int128 acc = 0;
Int128[] a = _i128, b = _i128Sign;
for (int i = 0; i < a.Length; i++)
acc += Int128.CopySign(a[i], b[i]);
return acc;
}
}Note This benchmark comment was generated with the assistance of GitHub Copilot. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the signed-integer CopySign implementations in System.Private.CoreLib to use a branchless conditional-negate pattern (with a preserved overflow-throw for the MinValue + non-negative sign case), and adds corresponding generic-math test coverage in System.Runtime.Tests.
Changes:
- Replace
CopySignimplementations forsbyte,short,int,long,nint, andInt128with a branchless sign-adjustment approach while preserving theOverflowExceptionbehavior forMinValuewith non-negative sign. - Add
CopySignTestcoverage for each affected signed integer type in the generic-math tests. - Implement an
Int128-specific half-limb version to avoid extra intermediateInt128operations.
Show a summary per file
| File | Description |
|---|---|
| src/libraries/System.Private.CoreLib/src/System/SByte.cs | Reworks sbyte.CopySign to compute sign mask + conditional negate, preserving overflow throw behavior. |
| src/libraries/System.Private.CoreLib/src/System/Int16.cs | Reworks short.CopySign with the same branchless approach and overflow preservation. |
| src/libraries/System.Private.CoreLib/src/System/Int32.cs | Reworks int.CopySign to the branchless mask/negate form with explicit overflow throw. |
| src/libraries/System.Private.CoreLib/src/System/Int64.cs | Reworks long.CopySign similarly, using a 63-bit sign extraction. |
| src/libraries/System.Private.CoreLib/src/System/IntPtr.cs | Reworks nint.CopySign using pointer-width sign extraction and preserved overflow behavior. |
| src/libraries/System.Private.CoreLib/src/System/Int128.cs | Reworks Int128.CopySign using half-limb operations + single computed sign mask, with overflow preservation. |
| src/libraries/System.Runtime/tests/System.Runtime.Tests/System/SByteTests.GenericMath.cs | Adds CopySignTest coverage for sbyte. |
| src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int16Tests.GenericMath.cs | Adds CopySignTest coverage for short. |
| src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int32Tests.GenericMath.cs | Adds CopySignTest coverage for int. |
| src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int64Tests.GenericMath.cs | Adds CopySignTest coverage for long. |
| src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.GenericMath.cs | Adds CopySignTest coverage for nint, with 32-bit vs 64-bit expectations. |
| src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Int128Tests.GenericMath.cs | Adds CopySignTest coverage for Int128, including overflow cases. |
Copilot's findings
- Files reviewed: 12/12 changed files
- Comments generated: 0
Member
Author
|
@EgorBot -linux_intel -osx_arm64 using System;
using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);
public class Bench
{
private const int N = 1024;
private Int128[] _v = new Int128[N], _s = new Int128[N];
[GlobalSetup]
public void Setup()
{
var r = new Random(42);
for (int i = 0; i < N; i++)
{
// Non-negative magnitudes with random sign; avoids MinValue so no overflow path.
_v[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
_s[i] = new Int128((ulong)r.NextInt64() >> 1, (ulong)r.NextInt64()) * (r.Next(2) == 0 ? 1 : -1);
}
}
[Benchmark(Baseline = true)]
public Int128 Branchy()
{
Int128 acc = 0;
var v = _v; var s = _s;
for (int i = 0; i < v.Length; i++) acc += BranchyImpl(v[i], s[i]);
return acc;
}
[Benchmark]
public Int128 Limb()
{
Int128 acc = 0;
var v = _v; var s = _s;
for (int i = 0; i < v.Length; i++) acc += LimbImpl(v[i], s[i]);
return acc;
}
[Benchmark]
public Int128 Operator()
{
Int128 acc = 0;
var v = _v; var s = _s;
for (int i = 0; i < v.Length; i++) acc += OperatorImpl(v[i], s[i]);
return acc;
}
[Benchmark]
public Int128 Select()
{
Int128 acc = 0;
var v = _v; var s = _s;
for (int i = 0; i < v.Length; i++) acc += SelectImpl(v[i], s[i]);
return acc;
}
[MethodImpl(MethodImplOptions.NoInlining)]
static Int128 BranchyImpl(Int128 value, Int128 sign)
{
Int128 absValue = value;
if (Int128.IsNegative(absValue)) absValue = -absValue;
if (Int128.IsPositive(sign))
{
if (Int128.IsNegative(absValue)) throw new OverflowException();
return absValue;
}
return -absValue;
}
[MethodImpl(MethodImplOptions.NoInlining)]
static Int128 LimbImpl(Int128 value, Int128 sign)
{
ulong vl = (ulong)value, vu = (ulong)(value >> 64);
ulong su = (ulong)(sign >> 64);
ulong m = (ulong)((long)(vu ^ su) >> 63);
ulong lo = vl ^ m, hi = vu ^ m;
ulong rlo = lo - m;
ulong borrow = (rlo > lo) ? 1UL : 0UL;
ulong rhi = hi - m - borrow;
if (((long)su >= 0) && ((long)rhi < 0)) throw new OverflowException();
return new Int128(rhi, rlo);
}
[MethodImpl(MethodImplOptions.NoInlining)]
static Int128 OperatorImpl(Int128 value, Int128 sign)
{
Int128 signMask = (value ^ sign) >> 127;
Int128 result = (value ^ signMask) - signMask;
if (Int128.IsPositive(sign) && Int128.IsNegative(result)) throw new OverflowException();
return result;
}
[MethodImpl(MethodImplOptions.NoInlining)]
static Int128 SelectImpl(Int128 value, Int128 sign)
{
Int128 neg = -value;
Int128 result = Int128.IsNegative(value ^ sign) ? neg : value;
if (Int128.IsPositive(sign) && Int128.IsNegative(result)) throw new OverflowException();
return result;
}
}Note This benchmark comment was generated with the assistance of GitHub Copilot. |
The manual limb-based borrow lowered identically to the general (value ^ signMask) - signMask form in benchmarks, so use the clearer form that mirrors the scalar integer types Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #77579
The signed integer
CopySignimplementations used a data-dependent abs-then-negate sequence with two branches on the hot path. This replaces it with the standard branchless conditional negate:which produces
valuewith the sign ofsignin a handful of ALU ops. A single predicted-not-taken branch is kept to preserve theOverflowExceptionthatCopySign(MinValue, non-negative)must still throw (the only case whereresultcomes back negative for a non-negativesign).Applied to
sbyte,short,int,long,nint, andInt128.Int128uses the same general form; a manual per-limb variant was measured and lowered identically, so the clearer form is used.The floating-point and
decimalCopySignpaths already use bit/flag manipulation (orVector128.ConditionalSelect), so they're unchanged.Correctness of the transform (including the overflow-throw case) was verified exhaustively over the full 8-bit domain and, for
Int128, against the width-generic reference over edge values plus randomized 128-bit inputs. AddedCopySignTestcoverage to the generic-math tests for each signed integer type.Performance (array-loop over random inputs; ratio is new/old, lower is better):
intlongInt128The scalar types improve on every platform measured.
Int128is a solid win on arm64; on x64 it's roughly neutral and microarchitecture-dependent (measured a small gain on Intel Emerald Rapids and a small regression on AMD EPYC Turin), which is expected since x64 absorbs the original well-predicted branch cheaply andInt128isn't hardware accelerated. A separate A/B run confirmed the general form matches the manual per-limb form and beats a compute-both-and-select alternative.Note
This PR description was generated with the assistance of GitHub Copilot.