Skip to content

fix: an empty string is no longer coerced to 0 in numeric contexts (HF-361) - #1768

Open
marcin-kordas-hoc wants to merge 4 commits into
developfrom
fix/hf-361-empty-string-coercion
Open

marcin-kordas-hoc wants to merge 4 commits into
developfrom
fix/hf-361-empty-string-coercion

Conversation

@marcin-kordas-hoc

@marcin-kordas-hoc marcin-kordas-hoc commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

What

An empty string ('') was unconditionally coerced to 0 in numeric contexts. This aligns HyperFormula with Excel, where "" is text and is never a number.

ArithmeticHelper.coerceNonDateScalarToMaybeNumber carried an arg === '' → return 0 early exit. Removing it lets '' fall through to normal string→number parsing, which correctly finds no number. Blank cells are a different internal value (EmptyValue, handled one branch above) and are untouched — evaluateNullToZero keeps working exactly as before.

Reported by a customer via support: COUNT("") returned 1 where Excel returns 0, and =""+0 returned 0 where Excel returns #VALUE!. The reporter had already verified in Excel Online that both cases error there, and had ruled out every config option as a workaround (there is none — the behaviour was hardcoded).

What changes, and what does not

The distinction that matters for anyone reading this to estimate impact: aggregation over a cell reference or a range does not change. SUM, AVERAGE, MIN, MAX and COUNT have always ignored text found in a reference, and they still do, because the aggregation path filters strings out (strictlyNumbers) before this coercion is reached. Every value below is measured on both engines; the Excel column is measured live through the Microsoft Graph API against Excel Online.

Changes — arithmetic and number-typed arguments, in both the literal and the reference form (A1 holds an empty string):

Formula Excel before after
=""+0 / =A1+1 #VALUE! 0 / 1 #VALUE!
=A1-1, =1-A1, =A1*2, =A1/1, =A1^2, =-A1, =A1% #VALUE! numbers #VALUE!
=ROUND(A1, 0), =ABS(A1), =INT("") #VALUE! 0 #VALUE!
=ACOT("") / =ACOT(A1) #VALUE! 1.5707963268 #VALUE!
=DATE(A1, 2, 3) #VALUE! 35 #VALUE!
=LN(""), =LOG10(""), =DATE("","","") #VALUE! #NUM! #VALUE!
=COT(""), =COTH("") #VALUE! #DIV/0! #VALUE!

Changes — an empty string written directly into an aggregation:

Formula Excel before after
=COUNT("") 0 1 0
=SUM(""), =AVERAGE(""), =MIN(""), =MAX(""), =PRODUCT("") #VALUE! 0 #VALUE!
=SUM(1, "") #VALUE! 1 #VALUE!
=SUMSQ(""), =MEDIAN(""), =STDEV("") #VALUE! 0 / #DIV/0! #VALUE!

Changes for the better — an empty string produced mid-formula. This is the most common real shape of the bug, and the strongest argument for the change: before, all of these were wrong; now all match Excel.

Formula Excel before after
=SUM(IF(TRUE,"",1)) #VALUE! 0 #VALUE!
=COUNT(IF(TRUE,"",1)) 0 1 0
=SUM(LEFT("abc",0)), =LEFT("abc",0)+1 #VALUE! 0 / 1 #VALUE!
=ROUND(IF(TRUE,"",1),0), =SUM(A1&""), =SUM(CONCATENATE("","")) #VALUE! numbers #VALUE!

Does not change — measured identical before and after, and matching Excel:

Formula Excel before and after
=SUM(A1), =MIN(A1), =MAX(A1), =COUNT(A1) 0 0
=AVERAGE(A1) #DIV/0! #DIV/0!
=COUNTA(A1) 1 1
=SUM(A1:C1) over 1, "", 2 3 3
=COUNT(A1:C1) over the same 2 2
=SUM({1,"",2}), =COUNT({1,"",2}) 3, 2 3, 2
a blank cell, either evaluateNullToZero setting unchanged
=COUNTIF(rng,""), =A1&"x", =A1="", =N(A1), =VALUE(A1) unchanged

One class that moves away from Excel

Where Excel evaluates an argument as an array containing only text, it ignores the text; before, HyperFormula happened to agree by coercing '' to 0:

Formula Excel before after
=SUM(IF(A1:A3>0,"",A1:A3)) 0 0 #VALUE!
=SUM({""}) 0 0 #VALUE!
=COUNT(IF({1,0},"",5)) 1 1 0

That agreement was coincidence, not support: HyperFormula's IF is not array-aware at all=ROWS(IF(A1:A3>0,"",A1:A3)) is 1, so it collapses the argument to a single value regardless. The idiom is already broken independently of this change: =SUM(IF(A1:A3>5,"",A1:A3)) over 1, 2, 3 is 6 in Excel and 1 here, before and after. What changes is only that the collapsed value stops being silently read as 0. Worth its own ticket; not a reason to hold this one.

Integration risk, and a design question worth answering first

setCellContents(address, '') produces a text cell holding '' (STRING, ISBLANK false), not a blank one — only null/undefined produce a blank. Verified against the Handsontable Formulas plugin on the published package:

  • Clearing cells with Delete is safecore.js's emptySelectedCells pushes null.
  • Paste is a real exposureSheetClip.parse('a\t\tc') yields ["a","","c"], copyPaste.js feeds that to populateFromArray, and formulas.js's syncChangeWithEngine forwards to setCellContents with only date-specific normalisation. Nothing maps '' back to null.

Scope of that exposure, though, is narrower than it first looks: for a pasted '' cell, SUM and COUNT are unaffected — only arithmetic and number-typed arguments break.

The design question: should a written '' parse as EmptyValue rather than as text? That would fix the reported bug and remove the paste exposure at the same time, at the cost of a different divergence (ISTEXT, COUNTA and LEN on such a cell). This PR takes the narrower route — fix the coercion, leave cell parsing alone — but the alternative deserves an explicit yes or no before a breaking change ships.

Verification

  • Paired tests: handsontable/hyperformula-tests#53, 27 new specs plus 8 assertions that pinned the old behaviour across coercions, function-ln, function-log10, function-cot, function-coth, function-acot and function-date (function-log changes an input, not an expectation).
  • Full private suite green: 503/503 suites, 6271 passed, 3 skipped.
  • Negative control: with this fix reverted, 16 of the 27 new specs fail — the spec detects the bug rather than passing vacuously.
  • Excel column measured live via MS Graph (Excel Online); probe scripts and raw JSON retained.
  • CI green, including browser-tests — the private suite runs under both Jest and Karma/Jasmine.

Definition of Done

  • Production code + JSDoc explaining why '' is deliberately not special-cased
  • Changelog entry under Changed as a breaking change
  • Migration guide section (docs/guide/migration-from-3.x-to-4.0.md)
  • Paired tests in hyperformula-tests
  • Green CI

Note: this targets 4.0 and adds docs/guide/migration-from-3.x-to-4.0.md, the same new file #1754 creates. Whichever lands first establishes it; the second resolves a mechanical conflict.

🤖 Generated with Claude Code


Note

High Risk
Breaking formula semantics across arithmetic and numeric functions; sheets that relied on "" as zero or store '' in cells will see different results, though range aggregations are largely unchanged.

Overview
This is a breaking 4.0 change that stops treating an empty string ("") as 0 in numeric contexts, matching Excel: arithmetic, number-typed function arguments, and literals passed directly into aggregations (e.g. =""+0, =ROUND(A1,0) with text "", =COUNT(""), =SUM(1,"")) now yield #VALUE! or updated counts instead of silently using zero. Blank cells (EmptyValue, null/undefined in the API) are unchanged, and aggregations over cell/range references still skip non-numeric text the same way as before.

ArithmeticHelper.coerceNonDateScalarToMaybeNumber drops the '' → 0 shortcut so empty text falls through normal parsing and callers surface #VALUE!. NumericAggregationPlugin.reduce unwraps parenthesized references so SUM((A1)) follows the reference path (ignore text) like Excel, not the literal-coercion path.

Documentation adds an Unreleased changelog entry, a 3.x → 4.0 migration guide (including setCellContents(..., null) vs ''), and a docs nav link.

Reviewed by Cursor Bugbot for commit a24a523. Bugbot is set up for automated code reviews on this repo. Configure here.

`ArithmeticHelper#coerceNonDateScalarToMaybeNumber` special-cased an
empty string ('') to coerce to 0, the same as a truly blank cell
(EmptyValue). This is a different case from EmptyValue -- a cell or
literal holding '' is text, not blank -- and it made HyperFormula
diverge from Excel in two ways:

- Arithmetic on '' returned 0 instead of #VALUE! (e.g. `=""+0`).
- COUNT("") returned 1 instead of 0, since the literal argument was
  coerced to 0 (an ExtendedNumber) before COUNT's own
  isExtendedNumber predicate ever saw it.

Root cause:
- Interpreter.ts `plusOp`/`minusOp`/etc. call
  `ArithmeticHelper#coerceScalarToNumberOrError`, which returned 0
  for '' before this fix.
- NumericAggregationPlugin#reduce (used by COUNT, SUM, AVERAGE,
  PRODUCT, MIN, MAX) calls the same coercion for a literal/expression
  argument (as opposed to a direct cell reference, which uses the
  aggregation's own predicate without numeric coercion) -- so
  COUNT("") went through the same 0-producing bug before
  isExtendedNumber(0) counted it.

This removes the `arg === ''` special case, letting '' fall through
to the normal string-to-number parsing (which correctly fails for
non-numeric text), matching how HyperFormula already treats any
other non-numeric string literal (e.g. `SUM("abc")` already returned
#VALUE!; `SUM("")` was the sole inconsistent exception).

Not affected (verified with tests): blank cells (EmptyValue) still
coerce to 0 in arithmetic and are still governed separately by
`evaluateNullToZero`; a '' criterion in COUNTIF/SUMIF-style functions
still matches only genuinely blank cells (Criterion.ts resolves that
case before any numeric coercion runs).

This is a breaking change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@qunabu

qunabu commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
hyperformula-docs a24a523 Sep 23 2026, 07:04 AM

The fix itself was already in place; this adds what a breaking change owes
the reader, and moves the tests where this repo keeps them.

- Changelog: moved the entry from Fixed to Changed and rewrote it in the
  "A **breaking change**:" form the other 4.0 entries use, with a migration
  guide link and the PR number instead of an internal task id.
- Migration guide: new docs/guide/migration-from-3.x-to-4.0.md (plus its
  sidebar entry) with a section on empty-string coercion. Every before/after
  value in it is measured, not assumed -- two rounds of probes across the
  unpatched and patched engine, plus the assertions the private suite already
  pinned. That caught two wrong claims: DATE("","","") returned #NUM! and not
  a computed date, and ACOT("") returned 1.5707963267949 -- a plausible number
  from an argument that was never numeric, which is the most useful example
  this change has.
- Removed test/hf-361-empty-string-coercion.spec.ts. DEV_DOCS.md:129 is
  explicit that interpreter specs do not live in the public repo, because a
  copy here runs twice and leaves develop red if only one of the paired PRs
  lands. Those tests move to the paired hyperformula-tests branch of the same
  name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown

Performance comparison of head (a24a523) vs base (c920375)

                                     testName |    base |    head | change
--------------------------------------------------------------------------
                                      Sheet A |  494.85 |  491.69 | -0.64%
                                      Sheet B |  163.06 |   158.8 | -2.61%
                                      Sheet T |  145.47 |   144.7 | -0.53%
                                Column ranges |  477.59 |  476.78 | -0.17%
                                Sorted lookup | 14323.2 | 14483.2 | +1.12%
Sheet A:  change value, add/remove row/column |   15.58 |   15.38 | -1.28%
 Sheet B: change value, add/remove row/column |  133.83 |  123.43 | -7.77%
                   Column ranges - add column |   152.5 |  144.33 | -5.36%
                Column ranges - without batch |  464.58 |  439.46 | -5.41%
                        Column ranges - batch |  119.26 |  112.27 | -5.86%

…n guide

Two independent reviews converged on the same defect: both documents implied
that a cell holding an empty string breaks any numeric formula. It does not.
Measured on both engines: SUM, AVERAGE, MIN, MAX and COUNT over a reference
or a range are byte-identical before and after -- they have always ignored
text found there, matching Excel -- because the aggregation path filters
strings out before this coercion is reached. Only arithmetic, number-typed
arguments, and an empty string written directly into an aggregation change.
A reader with =SUM(A1:A100) would have been told to expect a break that
cannot happen, in a document whose whole job is to predict breaks.

Also corrected or added, all measured rather than read off a test:

- =ACOT("") returned 1.5707963268, not 1.5707963267949. I had taken the
  digits from the private suite's toBeCloseTo(..., 10) assertion, which
  passes for both. Default config rounds to 10 significant digits.
- =DATE(A1, 2, 3) over a text empty string silently returned 35 -- a date in
  1900 -- which is the same "answered instead of failing" class as ACOT and a
  better example, because it is the reference form users actually write.
- Added the contexts the table omitted while claiming to be exhaustive:
  =""%, =SUM(1, ""), =ABS, =INT, =SUMSQ, =MEDIAN, =STDEV.
- Added the nested-expression class (=SUM(IF(TRUE,"",1)), =SUM(LEFT("abc",0))),
  where 3.x was wrong and 4.0 matches Excel, and the one class that moves away
  from Excel (an argument Excel evaluates as an array of only text), with the
  pre-existing non-array-aware IF that actually causes it.
- Fixed the code blocks against DOCS_CONTENT_GUIDE: the mandated
  { sheet, row, col } key order, a snippet that actually runs as written
  (verified), and an untagged fence for the before/after pair that is not
  JavaScript. Pointed N() and IFERROR at their real category anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@marcin-kordas-hoc
marcin-kordas-hoc marked this pull request as ready for review September 10, 2026 15:37

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c99427b. Configure here.

- v4.0
- empty string
- coercion
---

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant migration guide search tags

Low Severity

Tags empty string and coercion repeat the h2 Changes to empty string coercion, which search already indexes from headings. The multi-word empty string tag also matches empty and string separately, so generic queries can surface this migration page.

Fix in Cursor Fix in Web

Triggered by learned rule: VuePress tags frontmatter invariants

Reviewed by Cursor Bugbot for commit c99427b. Configure here.

@Tobiadefami Tobiadefami left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we preserve reference handling through parentheses? With B1 containing ="", =SUM((B1)) returns 0 in Excel Online and on base, but #VALUE! on this head.

A possible fix is to recursively look through parentheses when checking for CELL_REFERENCE in NumericAggregationPlugin.reduce(). I tested this temporarily: it fixes the regression while preserving rejection of literal empty strings. All 195 tests across 16 checked suites passed.

…eric aggregations

Reported in review on this PR: with B1 holding ="", SUM(B1) correctly returns
0 (aggregation over a reference ignores non-numeric text, unchanged by this
PR), but SUM((B1)) returned #VALUE!.

Root cause: NumericAggregationPlugin#reduce decides how to treat an argument
by checking arg.type against CELL_REFERENCE (and separately against
CELL_RANGE/COLUMN_RANGE/ROW_RANGE) to tell "this is a reference, ignore its
non-numeric text" apart from "this is a literal/computed value, coerce it to
a number." A parenthesized argument reaches the reducer as a PARENTHESIS node
wrapping the real expression, so neither check matched and (B1) fell through
to the same coercion "" now correctly fails for as a literal.

reduce() is the single shared implementation behind SUM, SUMSQ, COUNT,
COUNTA, AVERAGE, AVERAGEA, MIN, MAX, MINA, MAXA, PRODUCT, VAR.S/P, VARA/VARPA
and STDEV.S/P/A/PA (via doAverage/doCount/reduceAggregate(A)/doMax/doMin/
doSum/doProduct), so this was not SUM-specific: every one of those reproduced
the same #VALUE! regression through a single extra pair of parentheses.

Fix: unwrap PARENTHESIS nodes before either arg.type check, the same idiom
already used in FunctionPlugin#runFunctionWithReferenceArgument and
InformationPlugin. PARENTHESIS is a pure passthrough in
Interpreter#evaluateAst, so this only changes which branch classifies the
argument, not what evaluateAst(arg, state) itself returns.

Verified against Excel Online via MS Graph: a parenthesized reference or
range behaves identically to the bare one in every case tested (single and
double parentheses, SUM/COUNT/AVERAGE/MIN/MAX/PRODUCT/SUMSQ/STDEV, a range
containing an embedded empty string). Ranges wrapped in parentheses already
worked before this fix -- they take a different path in reduce() that checks
the evaluated value's runtime type (SimpleRangeValue) rather than the AST
node type -- so this fix does not change range handling, only single-cell
(and column/row) references.

Paired failing-then-passing tests: handsontable/hyperformula-tests, same
branch, "parentheses around a reference do not change numeric-aggregation
behavior" in empty-string-coercion.spec.ts. Negative control: reverting this
commit's source change while keeping those tests fails 5 of the 8 new specs
(the 3 that also pass without the fix are pins on already-correct behavior:
COUNT((A1)) and the two parenthesized-range cases).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@marcin-kordas-hoc

Copy link
Copy Markdown
Collaborator Author

@Tobiadefami reproduced and fixed — thanks for catching it, and sorry it sat this long.

SUM((B1)) was going through the same numeric-coercion path a literal gets, because (B1) reaches NumericAggregationPlugin#reduce as a PARENTHESIS AST node wrapping the CELL_REFERENCE, and the reducer's reference check only matched a bare CELL_REFERENCE node. Fixed by unwrapping PARENTHESIS before that check — the same pattern already used in FunctionPlugin#runFunctionWithReferenceArgument and InformationPlugin — so it's a one-line fix at a single shared choke point rather than a per-function patch.

That choke point (reduce()) is the shared implementation behind the whole numeric-aggregation family, not just SUM: COUNT, AVERAGE, MIN, MAX, PRODUCT, SUMSQ, STDEV.S/P (and their A-suffixed variants) all reproduced the identical #VALUE! regression through one extra pair of parentheses, and all are fixed by this single change. MIN/MAX weren't in your report but I checked them too since they share reduce() — same bug, same fix.

Verified against Excel Online (MS Graph, live session): parentheses are fully transparent to reference handling in every case I measured — single and double parens, the whole aggregation family, and a parenthesized range containing an embedded "". Your reading was exactly right.

Added 8 specs to the paired repo (empty-string-coercion.spec.ts), written and confirmed failing against the previous head before the fix existed, passing after, and reconfirmed failing with the source fix reverted and the tests kept (negative control). Local Jest: 503/503 suites, 6279 passing.

One caveat I want to be explicit about: I could not run the Karma/Jasmine browser leg locally — no headless browser in my environment — so I pushed rather than claim it green. CI is running it now on a24a523dd; please treat the browser result there as the real verdict, not my local run.

MEDIAN does not share this bug (different code path, no reference-detection special case at all) — flagging separately rather than folding it in here.

@codecov

codecov Bot commented Sep 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.32%. Comparing base (c920375) to head (a24a523).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #1768   +/-   ##
========================================
  Coverage    97.32%   97.32%           
========================================
  Files          195      195           
  Lines        15739    15739           
  Branches      3390     3460   +70     
========================================
  Hits         15318    15318           
+ Misses         421      413    -8     
- Partials         0        8    +8     
Files with missing lines Coverage Δ
src/interpreter/ArithmeticHelper.ts 98.66% <ø> (-0.01%) ⬇️
src/interpreter/plugin/NumericAggregationPlugin.ts 97.47% <100.00%> (+0.01%) ⬆️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants