Conversation
…y/quantile` `tan( PI*(p-0.5) )` rounds away the low-order bits of a `p` close to `0`: the lower tail is wrong by orders of magnitude for small `p`, and `p = 0` and `p = 1` return `-/+1.6e16` instead of `-/+Infinity`. Use `-1/tan(PI*p)` for `p < 0.25` and `1/tan(PI*(1-p))` for `p > 0.75`. The fixtures are now generated from the textbook formula in extended precision and the tests use ULP-based assertions.
Contributor
Coverage Report
The above coverage report was generated for the changes in this PR. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request:
stats/base/dists/cauchy/quantilefrompand1-p(JS, factory and C).The function computed
x0 + gamma*tan( PI*(p-0.5) ). Forpclose to0,p - 0.5rounds away the low-order bits ofp. The lower tail is then wrong by orders of magnitude, and the endpoints don't reach infinity:Since
tan(pi*(p - 0.5)) = -1/tan(pi*p), the function now uses:x0 - gamma/tan(PI*p)forp < 0.25,x0 + gamma/tan(PI*(1-p))forp > 0.75, where1-pis exact,p - 0.5is exact.Fixtures and tests:
runner.jlnow evaluates the textbook formula in 2048-bitBigFloatinstead ofDistributions.quantile, which has the samep - 0.5loss.Distributionsis dropped fromREQUIRE, and the three fixtures are regenerated with the same input ranges.There's a new
tails.json: 500 points withplog-spaced over[1e-300, 0.1]and 500 with1-plog-spaced over[1e-16, 0.1].x0is0there, to isolate the tail.The tests used an
EPS-relative delta (50EPS). They now useisAlmostSameValuewith the smallest passing tolerances, the same in JS and native:developlarge_gammapositive_mediannegative_mediantailsWhat's left in the first three comes from
x0 + gamma*tan(...)cancelling when the two terms nearly cancel, e.g.-9.28 + 9.29. That's the conditioning of the sum, not the tail. Ondevelop, 999 of the 4,015 assertions intest.quantile.jsfail. With this change they all pass.Related Issues
None. Same kind of fix as #15773 (
laplace/quantile).Questions
No.
Other
The comment in the old
large_gammatest about Julia's and stdlib'standisagreeing is gone. The fixtures no longer use Julia's double-precisiontan.Checklist
AI Assistance
If you answered "yes" above, how did you use AI assistance?
Disclosure
This PR was written primarily by Claude Code. I found the bug by checking
stats/base/distsquantile functions at tinyp.@stdlib-js/reviewers