Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions paper/paper.bib
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ @book{fisher1935design
title={The Design of Experiments},
author={Fisher, Ronald A.},
year={1935},
publisher={Oliver and Boyd}
publisher={Oliver \& Boyd}
}

@ARTICLE{2020NumPy-Array,
Expand Down Expand Up @@ -129,7 +129,7 @@ @ARTICLE{2020SciPy-NMeth
author = {Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and
Haberland, Matt and Reddy, Tyler and Cournapeau, David and
Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and
Bright, Jonathan and van der Walt, St{\'e}fan J. and
Bright, Jonathan and {"van der Walt"}, St{\'e}fan J. and
Brett, Matthew and Wilson, Joshua and Millman, K. Jarrod and
Mayorov, Nikolay and Nelson, Andrew R. J. and Jones, Eric and
Kern, Robert and Larson, Eric and Carey, C J and
Expand All @@ -139,7 +139,7 @@ @ARTICLE{2020SciPy-NMeth
Harris, Charles R. and Archibald, Anne M. and
Ribeiro, Ant{\^o}nio H. and Pedregosa, Fabian and
van Mulbregt, Paul and {SciPy 1.0 Contributors}},
title = {{SciPy} 1.0: Fundamental Algorithms for Scientific Computing in Python},
title = {{SciPy} 1.0: Fundamental Algorithms for Scientific Computing in {Python}},
journal = {Nature Methods},
year = {2020},
volume = {17},
Expand All @@ -155,7 +155,7 @@ @misc{hillstrom2008
}

@article{finkelstein2012,
title={The Oregon Health Insurance Experiment: Evidence from the First Year},
title={The {Oregon} Health Insurance Experiment: Evidence from the First Year},
author={Finkelstein, Amy and Taubman, Sarah and Wright, Bill and Bernstein, Mira and Gruber, Jonathan and Newhouse, Joseph P. and Allen, Heidi and Baicker, Katherine and {Oregon Health Study Group}},
journal={The Quarterly Journal of Economics},
volume={127},
Expand All @@ -166,15 +166,16 @@ @article{finkelstein2012
}

@Manual{qte,
title = {qte: Quantile Treatment Effects},
title = {{qte}: Quantile Treatment Effects},
author = {Callaway, Brantly},
year = {2026},
note = {R package version 2.0.0},
url = {https://CRAN.R-project.org/package=qte}
url = {https://CRAN.R-project.org/package=qte},
doi = {10.32614/CRAN.package.qte}
}

@article{kobrosly2020causalcurve,
title={causal-curve: A Python Causal Inference Package to Estimate Causal Dose-Response Curves},
title={causal-curve: A {Python} Causal Inference Package to Estimate Causal Dose-Response Curves},
author={Kobrosly, Roni W.},
journal={Journal of Open Source Software},
volume={5},
Expand Down
12 changes: 6 additions & 6 deletions paper/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ bibliography: paper.bib

# Summary

`dte_adj` is a Python package for analyzing how an intervention, such as a marketing campaign, a medical treatment, or a policy change, affects the entire range of an outcome, not just its average. Practitioners running randomized experiments (RCTs, also known as A/B tests) can use it to answer questions such as "did the treatment help the users who were struggling the most?" or "did it move the whole distribution, or only shift the top end?", together with rigorous confidence bands around those answers.
`dte_adj` is a Python package for analyzing how an intervention, such as a marketing campaign, a medical treatment, or a policy change, affects the entire range of an outcome, not just its average. Practitioners running randomized experiments (RCTs, also known as A/B tests) can use it to answer questions such as "Did the treatment help the users who were struggling the most?" or "Did it move the whole distribution, or only shift the top end?", together with rigorous confidence bands around those answers.

More formally, `dte_adj` estimates distributional treatment effects (DTEs) in randomized experiments. Unlike traditional approaches that focus on average treatment effects, `dte_adj` enables researchers to analyze the full distributional impact of interventions across different outcome levels. The package implements machine learning-enhanced regression adjustment methods for variance reduction, supports multiple experimental designs including simple randomization, covariate-adaptive randomization, and settings with imperfect compliance, and provides a scikit-learn-like API [@scikit-learn] with comprehensive functionality for computing distribution functions, probability treatment effects, and quantile treatment effects with confidence intervals.
More formally, `dte_adj` estimates distributional treatment effects (DTEs) in randomized experiments. Unlike traditional approaches that focus on average treatment effects, `dte_adj` enables researchers to analyze the full distributional impact of interventions across different outcome levels. The package implements machine learning-enhanced regression adjustment methods for variance reduction, supports simple and covariate-adaptive randomization designs, accommodates settings with imperfect compliance, and provides a scikit-learn-like API [@scikit-learn] with comprehensive functionality for computing distribution functions, probability treatment effects, and quantile treatment effects with confidence intervals.

# Statement of Need

Expand All @@ -50,7 +50,7 @@ Several Python packages address causal inference, but none focus on distribution
- **EconML** [@econml]: Incorporates machine learning for heterogeneous treatment effect estimation (CATE) but does not address distributional effects.
- **causal-curve** [@kobrosly2020causalcurve]: Estimates dose-response curves but targets continuous treatments rather than distributional outcomes.

In the R ecosystem, packages like `qte` [@qte] provide quantile treatment effect estimation but lack machine learning integration for variance reduction. `dte_adj` uniquely combines: (1) distributional treatment effect estimation across the full outcome distribution, (2) machine learning-enhanced regression adjustment for precision gains, and (3) support for multiple experimental designs including covariate-adaptive randomization and imperfect compliance settings.
In the R ecosystem, packages like `qte` [@qte] provide quantile treatment effect estimation but lack machine learning integration for variance reduction. `dte_adj` uniquely combines (1) distributional treatment effect estimation across the full outcome distribution, (2) machine learning-enhanced regression adjustment for precision gains, and (3) support for multiple experimental designs, including covariate-adaptive randomization, as well as settings with imperfect compliance.

A standalone package is warranted because existing libraries are organized around scalar estimands (ATEs in `DoWhy`, heterogeneous CATEs in `EconML`), whereas distributional estimation requires a distinct set of primitives: distribution functions evaluated over grids of locations, interval probabilities, quantile inversion, pointwise and uniform confidence bands, and cross-fitted distributional regression for variance reduction. These do not map cleanly onto point-estimate abstractions, and a focused package also allows the implementation to track a rapidly evolving methodological literature on covariate-adaptive randomization, imperfect compliance, and multi-task learning for DTEs.

Expand All @@ -64,15 +64,15 @@ A standalone package is warranted because existing libraries are organized aroun

The stratified and non-stratified estimators share a common API with three methods: `predict_dte()` for distributional treatment effects, `predict_pte()` for probability treatment effects over intervals, and `predict_qte()` for quantile treatment effects. The local estimators expose `predict_ldte()` and `predict_lpte()` instead, for the imperfect-compliance setting. The adjusted estimators use K-fold cross-fitting to prevent overfitting and support both single-task and multi-task learning modes [@hirata2025efficientscalableestimationdistributional] for computational efficiency. Bootstrap methods provide confidence intervals with multiple variance estimation approaches.

The template method pattern is a natural fit here because every estimator shares the same outer algorithm (evaluate a distribution function on a grid, difference across treatment arms, and construct confidence bands) and differs only in how the conditional distribution is estimated, which depends on the experimental design and on whether a plug-in or a cross-fitted machine learning estimator is used. Defining the outer algorithm once in the base class keeps its statistical invariants in one place, whereas a strategy-based configuration would push that structure into runtime flags and obscure them. For the same reason, the estimators are exposed as distinct classes rather than a single configurable one: many flag combinations would be confusing (e.g., stratification without strata) or correspond to different estimands with different identifying assumptions, and separate classes make the required inputs explicit at construction time and let each estimator evolve independently.
The template method pattern is a natural fit here because every estimator shares the same outer algorithm (evaluate a distribution function on a grid, difference across treatment arms, and construct confidence bands) and differs only in how the conditional distribution is estimated, which depends on the experimental design and on whether a plug-in or a cross-fitted machine learning estimator is used. Defining the outer algorithm once in the base class keeps its statistical invariants in one place, whereas a strategy-based configuration would push that structure into runtime flags and obscure it. For the same reason, the estimators are exposed as distinct classes rather than a single configurable one: many flag combinations would be confusing (e.g., stratification without strata) or correspond to different estimands with different identifying assumptions, and separate classes make the required inputs explicit at construction time and let each estimator evolve independently.

![Distributional treatment effects for the Hillstrom email marketing dataset [@hillstrom2008], comparing Women's vs Men's email campaigns. The simple estimator (left, purple) and ML-adjusted estimator (right, green) show that adjustment substantially tightens confidence bands, demonstrating the variance reduction benefit of regression adjustment.](hillstorm_dte.png)
![Distributional treatment effects for the Hillstrom email marketing dataset [@hillstrom2008], comparing the women's and men's email campaigns. The simple estimator (left, purple) and ML-adjusted estimator (right, green) show that adjustment substantially tightens confidence bands, demonstrating the variance reduction benefit of regression adjustment.](hillstorm_dte.png)

![Local distributional treatment effects for emergency department costs in the Oregon Health Insurance Experiment [@finkelstein2012], estimated using `SimpleLocalDistributionEstimator` (left) and `AdjustedLocalDistributionEstimator` (right). Health insurance coverage shifts the distribution of ED costs, with ML adjustment again yielding narrower confidence intervals.](oregon_ldte_costs_comparison.png)

# Research Impact Statement

The methods implemented in `dte_adj` have been published across machine learning and econometrics venues: ICML 2024 [@byambadalai2024estimatingdistributionaltreatmenteffects], Econometric Reviews [@oka2025regression], ICML 2025 [@byambadalai2025efficientestimationdistributionaltreatment], and NeurIPS 2025 [@byambadalai2025imperfectcompliance]. The package has been applied in industry settings, including analyzing the distributional impact of content promotion on user engagement at ABEMA, a major video streaming platform [@yasui2026abema]. The documentation includes tutorials demonstrating applications to the Hillstrom email marketing dataset [@hillstrom2008] (Figure 1) and the Oregon Health Insurance Experiment [@finkelstein2012] (Figure 2), facilitating adoption by researchers in economics, marketing, and healthcare. End-to-end code for reproducing both figures is available in the online tutorials at <https://cyberagentailab.github.io/python-dte-adjustment/tutorials/>.
The methods implemented in `dte_adj` have been published across machine learning and econometrics venues: ICML 2024 [@byambadalai2024estimatingdistributionaltreatmenteffects], Econometric Reviews [@oka2025regression], ICML 2025 [@byambadalai2025efficientestimationdistributionaltreatment], and NeurIPS 2025 [@byambadalai2025imperfectcompliance]. The package has been applied in industry settings, including analyzing the distributional impact of content promotion on user engagement at ABEMA, a major video streaming platform [@yasui2026abema]. The documentation includes tutorials demonstrating applications to the Hillstrom email marketing dataset [@hillstrom2008, see Figure 1] and the Oregon Health Insurance Experiment [@finkelstein2012, see Figure 2], facilitating adoption by researchers in economics, marketing, and healthcare. End-to-end code for reproducing both figures is available in the online tutorials at <https://cyberagentailab.github.io/python-dte-adjustment/tutorials/>.

# AI Usage Disclosure

Expand Down
Loading