You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
histogram: a nullable Float64 column holding a NaN gets no histogram (regression from #1069) #1077
A nullable Float64 column can hold a NaN that isn't pd.NA: arithmetic such as 0.0 / 0.0 produces one, and isna() is False for it. For such a column base_summary_stats (pd_stats_v2.py:131) returns min and max as pd.NA. Since #1069, histogram_series drops the NaN along with the infinities and returns numeric histogram_args, so histogram calls numeric_histogram with the pd.NA min, and the math.isfinite check #1069 added to fmt_num (histogram.py:21) raises TypeError: must be real number, not NAType. The column gets no histogram. Before #1069 the NaN reached np.quantile, histogram_args came back empty, and the column got a categorical histogram listing nan as a value.
importpandasaspdfrombuckaroo.customizations.pd_stats_v2importPD_ANALYSIS_V2frombuckaroo.pluggable_analysis_framework.stat_pipelineimportStatPipelinenum=pd.Series([0.0] + [float(i) foriinrange(1, 100)], dtype="Float64")
den=pd.Series([0.0] + [1.0] *99, dtype="Float64")
ser=num/den# row 0 is NaN, not pd.NAsd, errs=StatPipeline(PD_ANALYSIS_V2, unit_test=False).process_df(pd.DataFrame({"a": ser}))
print(sd["a"]["min"], sd["a"]["histogram"], [(e.stat_key, str(e.error)) foreinerrsife.stat_key=="histogram"])
# <NA> None [('histogram', 'must be real number, not NAType')]
mean, median and std fail on the same column (#1074).
Suggested fix
Treat a nullable column's NaN as missing when base_summary_stats takes min and max, so they span the numbers, and have fmt_num return a placeholder for a bound that isn't a real number instead of raising.
Context
Regression from #1069, found while probing histogram edge cases. Reproduces on main at e88ace9.
Probe script for this issue. Save it as probe_1077.py in a buckaroo checkout and run uv run python probe_1077.py. The output below is from main at e88ace9.
probe_1077.py
"""Since #1069, histogram_series drops a nullable Float64 column's NaN (not pd.NA) with the infinities and returnsnumeric histogram_args, but min/max are pd.NA and fmt_num's math.isfinite raises on them."""importlogginglogging.disable(logging.CRITICAL)
importwarningswarnings.filterwarnings('ignore')
importnumpyasnpimportpandasaspdfrombuckaroo.customizations.pd_stats_v2importPD_ANALYSIS_V2frombuckaroo.pluggable_analysis_framework.stat_pipelineimportStatPipelinenum=pd.Series([0.0] + [float(i) foriinrange(1, 100)], dtype='Float64')
den=pd.Series([0.0] + [1.0] *99, dtype='Float64')
ser=num/denprint('values', ser.head(3).tolist(), 'isna', ser.isna().head(3).tolist())
print('dropna().min() =', ser.dropna().min(), '| np.isfinite keeps', int(np.isfinite(ser.dropna()).sum()), 'of', len(ser))
sd, errs=StatPipeline(PD_ANALYSIS_V2, unit_test=False).process_df(pd.DataFrame({'a': ser}))
foreinerrs:
print('error', e.stat_key, type(e.error).__name__, str(e.error)[:120])
print('histogram_args empty:', sd['a']['histogram_args'] == {}, '| histogram:', sd['a']['histogram'])
values [nan, 1.0, 2.0] isna [False, False, False]
dropna().min() = <NA> | np.isfinite keeps 99 of 100
error mean TypeError float() argument must be a string or a real number, not 'NAType'
error std TypeError float() argument must be a string or a real number, not 'NAType'
error median TypeError float() argument must be a string or a real number, not 'NAType'
error histogram TypeError must be real number, not NAType
histogram_args empty: False | histogram: None
Activity
paddymul commentedon Oct 7, 2026
Probe script for this issue. Save it as
probe_1077.pyin a buckaroo checkout and runuv run python probe_1077.py. The output below is from main at e88ace9.probe_1077.py
Output at e88ace9