IL Kernel Generator: Replace 500K+ lines of generated code with dynamic IL emission by Nucs · Pull Request #573 · SciSharp/NumSharp

Nucs · 2026-02-15T05:52:30Z

Summary

This PR implements the IL Kernel Generator, replacing NumSharp's ~500K+ lines of template-generated type-switch code with ~7K lines of dynamic IL emission using System.Reflection.Emit.

Closes #544 - [Core] Replace ~636K lines of generated math code with DynamicMethod IL emission
Closes #545 - [Core] SIMD-Optimized IL Emission (SIMD for contiguous arrays AND scalar broadcast)

Changes

Core Kernel Infrastructure (~7K lines)

File	Lines	Purpose
`ILKernelGenerator.cs`	4,800+	Main IL emission engine with SIMD support
`SimdKernels.cs`	626	SIMD vector operations (Vector256)
`ReductionKernel.cs`	377	Reduction operation definitions
`BinaryKernel.cs`	284	Binary operation enums & delegates

Dispatch Files

DefaultEngine.BinaryOp.cs - Binary ops (Add, Sub, Mul, Div, Mod)
DefaultEngine.UnaryOp.cs - 22 unary ops (Sin, Cos, Sqrt, Exp, etc.)
DefaultEngine.CompareOp.cs - Comparisons (==, !=, <, >, <=, >=)
DefaultEngine.BitwiseOp.cs - Bitwise AND/OR/XOR
DefaultEngine.ReductionOp.cs - Element-wise reductions

Files Deleted (73 total)

60 type-specific binary op files (Add, Sub, Mul, Div, Mod × 12 types)
13 type-specific comparison files (Equals × 12 types + dispatcher)

Net change: -498,481 lines (13,553 additions, 512,034 deletions)

SIMD Optimizations

Execution Path SIMD Status

Path	Description	IL SIMD	C# SIMD Fallback
SimdFull	Both arrays contiguous, same type	✅ Yes	✅ Yes
SimdScalarRight	Array + scalar (LHS type == Result type)	✅ Yes	✅ Yes
SimdScalarLeft	Scalar + array (RHS type == Result type)	✅ Yes	✅ Yes
SimdChunk	Inner-contiguous broadcast	❌ No (TODO)	✅ Yes (same-type)
General	Arbitrary strides	❌ No	❌ No

Note: Same-type operations (e.g., double + double) fall back to C# SimdKernels.cs which has full SIMD for SimdFull, SimdScalarRight/Left, and SimdChunk paths.

Scalar Broadcast Optimization

SIMD scalar operations hoist Vector256.Create(scalar) outside the loop:

// Before: scalar loop
for (int i = 0; i < n; i++)
    result[i] = lhs[i] + scalar;

// After: SIMD with hoisted broadcast
var scalarVec = Vector256.Create(scalar);  // hoisted!
for (; i <= n - 4; i += 4)
    (Vector256.Load(lhs + i) + scalarVec).Store(result + i);

Benchmark (10M elements):

Operation	Time
double + double_scalar	15.29 ms (baseline)
double + int_scalar	14.96 ms (IL SIMD ✓)
float + int_scalar	7.18 ms (IL SIMD ✓)

Bug Fixes Included

operator & and operator | - Were completely broken (returned null)
Log1p - Incorrectly using Log10 instead of Log
Sliced array × scalar - Incorrectly used SIMD path causing wrong indexing
Division type promotion - int/int now returns float64 (NumPy 2.x behavior)
Sign(NaN) - Now returns NaN instead of throwing ArithmeticException

Test Plan

All 2,597 tests pass (excluding OpenBugs category)
New test files: BattleProofTests, BinaryOpTests, UnaryOpTests, ComparisonOpTests, ReductionOpTests
Edge cases: NaN handling, empty arrays, sliced arrays, broadcast shapes, all 12 dtypes
SIMD correctness: verified with arrays of various sizes (including non-vector-aligned)

Architecture

Backends/Kernels/
├── ILKernelGenerator.cs    # IL emission engine with SIMD
├── BinaryKernel.cs         # Binary/Unary operation definitions
├── ReductionKernel.cs      # Reduction operation definitions
├── ScalarKernel.cs         # Scalar operation keys
├── SimdKernels.cs          # SIMD Vector256 operations (C# fallback)
└── KernelCache.cs          # Thread-safe kernel caching

Backends/Default/Math/
├── DefaultEngine.BinaryOp.cs
├── DefaultEngine.UnaryOp.cs
├── DefaultEngine.CompareOp.cs
├── DefaultEngine.BitwiseOp.cs
└── DefaultEngine.ReductionOp.cs

Performance

SIMD vectorization for contiguous arrays (Vector256) - all numeric types
SIMD scalar broadcast for mixed-type scalar operations (when array type == result type)
Strided path for sliced/broadcast arrays via coordinate iteration
Type promotion following NumPy 2.x semantics
Kernels are cached by (operation, input types, output type)

Future Work

IL SIMD for SimdChunk path (inner-contiguous broadcast)
AVX-512 / Vector512 support (when hardware adoption increases)
Vectorized type conversion for int + double_scalar cases

Fixed ArgumentOutOfRangeException when performing matrix multiplication on arrays with more than 2 dimensions (e.g., (3,1,2,2) @ (3,2,2)). Root causes: 1. Default.MatMul.cs: Loop count used `l.size` (total elements) instead of `iterShape.size` (number of matrix pairs to multiply) 2. UnmanagedStorage.Getters.cs: When indexing into broadcast arrays: - sliceSize incorrectly used parent's BufferSize for non-broadcast subshapes instead of the subshape's actual size - Shape offset was double-counted (once in GetSubshape, again because InternalArray.Slice already positioned at offset) The fix ensures: - Correct iteration count over batch dimensions - Proper sliceSize calculation based on subshape broadcast status - Shape offset reset to 0 after array slicing Verified against NumPy 2.4.2 output.

The tests incorrectly expected both arrays to have IsBroadcasted=True after np.broadcast_arrays(). Per NumPy semantics, only arrays that actually get broadcasted (have stride=0 for dimensions with size>1) should be flagged. When broadcasting (1,1,1) with (1,10,1): - Array 'a' (1,1,1→1,10,1): IsBroadcasted=True (strides become 0) - Array 'b' (1,10,1→1,10,1): IsBroadcasted=False (no change, no zero strides) NumSharp's behavior was correct; the test expectations were wrong.

When np.sum() or np.mean() is called with keepdims=True and no axis specified (element-wise reduction), the result should preserve all dimensions as size 1. Before: np.sum(arr_2d, keepdims=True).shape = (1) After: np.sum(arr_2d, keepdims=True).shape = (1, 1) Fixed in both ReduceAdd and ReduceMean by reshaping to an array of 1s with the same number of dimensions as the input, instead of just calling ExpandDimension(0) once. Verified against NumPy 2.4.2 behavior.

Extended the keepdims fix to all remaining reduction operations: - ReduceAMax (np.amax, np.max) - ReduceAMin (np.amin, np.min) - ReduceProduct (np.prod) - ReduceStd (np.std) - ReduceVar (np.var) Also fixed np.amax/np.amin API layer which ignored keepdims when axis=null. Added comprehensive parameterized test covering all reductions with multiple dtypes (Int32, Int64, Single, Double, Int16, Byte) to prevent regression. All 7 reduction functions now correctly preserve dimensions with keepdims=true, matching NumPy 2.x behavior.

This commit introduces a dynamic IL code generation system for NumSharp's element-wise operations, replacing hundreds of thousands of lines of template-generated type-switch code with ~7K lines of IL emission logic. Architecture: - ILKernelGenerator.cs: Main IL emission engine (~4.5K lines) - Generates typed kernels at runtime via System.Reflection.Emit - SIMD vectorization for contiguous float/double arrays (Vector256) - Strided path for sliced/broadcast arrays via coordinate iteration - BinaryKernel.cs: Binary operation definitions (Add, Sub, Mul, Div, Mod) - UnaryKernel.cs: Unary operations (22 ops: Sin, Cos, Sqrt, Exp, etc.) - ReductionKernel.cs: Element-wise reductions (Sum, Prod, Max, Min, etc.) - ScalarKernel.cs: Scalar operation keys (eliminates dynamic dispatch) Dispatch files (DefaultEngine.*.cs): - BinaryOp.cs: Binary operation dispatch with type promotion - UnaryOp.cs: Unary operation dispatch - BitwiseOp.cs: Bitwise AND/OR/XOR (fixes broken & and | operators) - CompareOp.cs: Comparison operations (==, !=, <, >, <=, >=) - ReductionOp.cs: Element-wise reduction dispatch Bug fixes included: 1. operator & and operator | were completely broken (returned null) 2. Default.Log1p was incorrectly using Log10 instead of Log 3. Sliced array × scalar incorrectly used SIMD path (wrong indexing) 4. Division type promotion: int/int now returns float64 (NumPy 2.x) 5. Sign(NaN) threw ArithmeticException, now returns NaN Files deleted: 73 type-specific generated files (~500K lines) - Add/*.cs, Subtract/*.cs, Multiply/*.cs, Divide/*.cs, Mod/*.cs (60 files) - Equals/*.cs (13 files) Files simplified: 22 unary operation files now single-line delegations Test results: 2597 tests pass (excluding 11 skipped, OpenBugs excluded)

Implement Vector256 SIMD operations for mixed-type scalar operations where the array type equals the result type (no per-element conversion needed). This optimizes operations like `double_array + int_scalar`. ## Changes - Add `EmitSimdScalarRightLoop()` for SIMD scalar right operand - Add `EmitSimdScalarLeftLoop()` for SIMD scalar left operand - Add `EmitVectorCreate()` helper for Vector256.Create(scalar) - Update `GenerateSimdScalarRightKernel()` to choose SIMD when eligible - Update `GenerateSimdScalarLeftKernel()` to choose SIMD when eligible ## SIMD Eligibility SIMD is used when: - ScalarRight: `LhsType == ResultType` (array needs no conversion) - ScalarLeft: `RhsType == ResultType` (array needs no conversion) - ResultType supports SIMD (float, double, int, long, etc.) - Operation has SIMD support (Add, Subtract, Multiply, Divide) ## Benchmark Results Array size: 10,000,000 elements Before (mixed-type used scalar loop): int + double_scalar: 19.09 ms After (SIMD when eligible): double + int_scalar: 14.96 ms [IL SIMD - matches baseline] float + int_scalar: 7.18 ms [IL SIMD - matches baseline] int + double_scalar: 15.84 ms [still scalar - needs conversion] ## Technical Details The SIMD scalar loop: 1. Loads scalar, converts to result type if needed 2. Broadcasts scalar to Vector256 using Vector256.Create() 3. SIMD loop: load array vector, perform vector op, store result 4. Tail loop handles remainder elements All 2597 tests pass.

Nucs added 5 commits February 14, 2026 13:40

Nucs added this to the NumPy 2.x Compliance milestone Feb 15, 2026

Nucs added bug Something isn't working core Internal engine: Shape, Storage, TensorEngine, iterators refactor Code cleanup without behavior change labels Feb 15, 2026

Nucs self-assigned this Feb 15, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IL Kernel Generator: Replace 500K+ lines of generated code with dynamic IL emission#573

IL Kernel Generator: Replace 500K+ lines of generated code with dynamic IL emission#573
Nucs wants to merge 6 commits intomasterfrom
ilkernel

Nucs commented Feb 15, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Nucs commented Feb 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Core Kernel Infrastructure (~7K lines)

Dispatch Files

Files Deleted (73 total)

SIMD Optimizations

Execution Path SIMD Status

Scalar Broadcast Optimization

Bug Fixes Included

Test Plan

Architecture

Performance

Future Work

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Nucs commented Feb 15, 2026 •

edited

Loading