Conversation
`erfc(z)` underflows for `z > ~26.5`, so `ln( erfc(z) )` returned `-Infinity` for `x - mu < c/1400` although the log-CDF is finite there. Use `ln(erfc(z)) = ln(erfcx(z)) - z^2` for `z >= 1`. The fixtures are now generated from the textbook formula in extended precision.
Coverage Report
The above coverage report was generated for the changes in this PR. |
0PrashantYadav0
left a comment
There was a problem hiding this comment.
Separate from this fix, the large-x tail still loses precision: logcdf( 1e40, 0.0, 1.0 ) returns 0 (exact ≈ -7.98e-21), and log1p( -erf( z ) ) gets it within ~1 ULP. Are you planning a follow-up for that?
| t.strictEqual( isAlmostSameValue( y, expected[ i ], 2 ), true, 'returns expected value' ); | ||
| } | ||
| t.end(); | ||
| }); |
There was a problem hiding this comment.
| }); | |
| }); | |
| tape( 'the function evaluates the logcdf for `x` very close to `mu`', function test( t ) { | |
| var y; | |
| y = logcdf( 1.0e-20, 0.0, 1.0 ); | |
| t.strictEqual( isAlmostSameValue( y, -5.0e19, 0 ), true, 'returns expected value' ); | |
| y = logcdf( 1.0e-200, 0.0, 1.0 ); | |
| t.strictEqual( isAlmostSameValue( y, -5.0e199, 0 ), true, 'returns expected value' ); | |
| t.end(); | |
| }); | |
| tape( 'if provided `x` equal to `mu`, the function returns `-infinity`', function test( t ) { | |
| var y = logcdf( 2.0, 2.0, 1.0 ); | |
| t.strictEqual( y, NINF, 'returns expected value' ); | |
| t.end(); | |
| }); |
near_mu.json stops at x-mu = 1e-15, so the gaps cited in the PR description and x == mu (which now goes through erfcx) aren't tested. Same in test.factory.js:238 and test.native.js:189 (with opts).
Co-authored-by: 0PrashantYadav0 <0PrashantYadav0@users.noreply.github.com>
|
@0PrashantYadav0 thanks. I added both tests in 55b4d8a, in On the large- |
|
LGTM but we still have to wait for others review. |
Description
This pull request:
erfcunderflow instats/base/dists/levy/logcdf(JS, factory and C).The function returned
ln( erfc( z ) )withz = sqrt( c / (2*(x-mu)) ).erfc(z)underflows to0forz > ~26.5, so forx - mu < c/1400the result is-Infinity, although the log-CDF there is finite (about-c / (2*(x-mu))):For
z >= 1it now returnsln( erfcx( z ) ) - w, wherew = c / (2*(x-mu))isz^2taken before the square root, so no rounding is added.erfcxis the scaled complementary error function,e^{z^2} erfc(z), which doesn't underflow. Belowz = 1the oldln( erfc( z ) )stays.Fixtures and tests:
runner.jlnow evaluates the textbook formulaln(erfc(sqrt(c/(2(x-mu)))))in 2048-bitBigFloat(withSpecialFunctions.erfc) instead ofDistributions.logcdf.REQUIREis updated, and the three fixtures are regenerated with the same input ranges.near_mu.jsonwithx - mulog-spaced over[1e-15, 1]. Smaller gaps pushexp(-w)beyond MPFR's exponent range.developagainst the exact values) and 2 ULP onnear_mu. The tolerances go from 0 to 3; the old 0 relied on the fixture using the same double-precision expression. Ondevelop, 802 of the 4,016 assertions intest.logcdf.jsfail, most of them returning-Infinity. With this change they all pass.Related Issues
None.
Questions
No.
Other
No.
Checklist
AI Assistance
If you answered "yes" above, how did you use AI assistance?
Disclosure
This PR was written primarily by Claude Code. I found the bug by evaluating every
stats/base/distslogcdf just above the lower end of its support.@stdlib-js/reviewers