stats: length, null_count, min and max are computed on a 50,000-row sample above 1M cells #1050

Problem

DfStatsV2.get_operating_df and PlDfStatsV2.get_operating_df run every stat on df.sample(min(50_000, rows)) once rows * cols exceeds FAST_SUMMARY_WHEN_GREATER = 1_000_000 (buckaroo/pluggable_analysis_framework/df_stats_v2.py:62, :92; utils.py:9). That includes length, null_count, min and max, which a sample cannot answer. For a frame over 1M cells with more than 50,000 rows, length reads 50,000, null_count is the count inside the sample, and min and max are wrong in every column whose extreme row is not sampled. A frame over 1M cells with 50,000 rows or fewer is sampled as a full shuffled copy and stays exact. Nothing on the wire says the numbers describe a sample.

import numpy as np, pandas as pd, polars as pl
from buckaroo import BuckarooWidget
from buckaroo.polars_buckaroo import PolarsBuckarooWidget

data = np.random.default_rng(0).integers(0, 1_000_000_000, size=(200_000, 10))
data[100, :] = -5
data[150_000, :] = 2_000_000_000
df = pd.DataFrame(data, columns=list("abcdefghij"))
for w in (BuckarooWidget(df), PolarsBuckarooWidget(pl.from_pandas(df))):
    a = w.dataflow.summary_sd["a"]
    print(a["length"], a["min"], a["max"])
# pandas: 50000 10797 999988384
# polars: 50000 -5 999972934
# true:   200000 -5 2000000000

On pandas the planted -5 is missed in this run because the sample is unseeded, so the figure varies from run to run. Polars seeds it (seed=42) and happened to include that row.

Impact

length and the pinned min, max and null_count rows are wrong for most frames above about 1M cells and 50,000 rows, with no marker that they are sampled. Column width and the color_map rule read min and max.

Suggested fix

Compute length, null_count, min and max on the frame the stats class receives, and use the sample only for stats that need one (value_counts, mode, median, histogram). On /load that frame is already pre_limit-sampled (1,000,000 rows for the pandas and server samplers), so these stats are exact up to pre_limit. This changes visible numbers, so it needs a decision. A tier mechanism that does this and also lets a host cap the expensive stats by size is the direction in #911 and the stats ADR work (#1044).

Context

Found while sizing which stats to run on large frames. Reproduced on main at 992fdb3. Related: #911.