Assignments

Statistics 240, spring 2023

There will be approximately five problem sets, two coding projects involving open-source code contributions, and a longer term project involving data analysis.

The longer project involves re-analyzing data in a published paper that inappropriately used parametric methods or used inappropriate nonparametric methods, using appropriate nonparametric methods instead. Please start looking for a paper that interests you right away.

Schedule

All assignments are due at 11:59pm Pacific Time unless indicated otherwise.

Assignment	due date
Problem set 1: math background	1/30 (Monday)
Problem set 2: confidence bounds for the average treatment effect in binary experiments	2/12 (Sunday)
Problem set 3: permutation tests, rank-based tests, and simulating \(P\)-values	2/26 (Sunday)
Computational project 1: unit tests for permute	3/5 (Sunday)
Problem set 4: TBA	3/12 (Sunday)
Problem set 5: TBA	3/26 (Sunday)
Computational project 2: new functionality for permute	4/16 (Sunday)
Term project: TBA	5/5 (Friday)

1. Weeks 1-3: Review and inference about binary populations

   • If you have never logged into https://github.berkeley.edu, please do so asap so I can add you to the class organization.

   • If you do not already have an ORCID, sign up for one at https://orcid.org/.

   • review:

     • mathematical foundations

     • mathematical inequalities

     • discrete probability

   • Binomial and hypergeometric tests and confidence intervals

     • testing

     • duality between tests and confidence sets

     • confidence sets, tailoring the rejection region, bootstrap percentile CIs

     • binomial and hypergeometric simulations; comparison with the normal approximation

   • Randomized, controlled experiments

     • Fisher’s exact test

     • Causal inference, Neyman’s potential outcomes model, experiments with binary outcomes

     • Li, X. and P. Ding, 2016. Exact confidence intervals for the average causal effect on a binary outcome, Statistics in Medicine, 35, 6, 957-960. 10.1002/sim.6764

     • Aronow, P.M., H. Chang, and P. Lopatto, 2022?. Fast computation of exact confidence intervals for randomized experiments with binary outcomes. https://lopat.to/permutation.pdf

   • Inference about binary populations from stratified samples: Wright’s method, Wendell & Schmee’s method, and union-intersection tests using greedy optimization

   • The SPRT for Bernoulli trials

2. Weeks 4-6: Permutation tests

   • introduction to permutation tests

   • PRNGs and simulations

   • simulation and computational permutation tests

   • classical permutation tests based on ranks

   • tests of randomness and independence

   • Stark, P.B., and K. Ottoboni, 2018. Random Sampling: Practice Makes Imperfect.

   • Ivanova, A., S. Lederman, P.B. Stark, G. Sullivan, and B. Vaughn, 2022. Randomization tests in clinical trials with multiple imputation for handling missing data, Journal of Biopharmaceutical Statistics, 32, 441-449. https://doi.org/10.1080/10543406.2022.2080695

   • permutation tests based on the distance between the empirical and the null

   • combining tests

   • E-values

3. Weeks 7-8: Supermartingale-based tests

   • Reading:

     • martingales

     • nonnegative supermartingales and Ville’s inequality

     • examples: additive, multiplicative, likelihood ratios, prior-posterior ratio, betting

     • the SPRT

   • Inference from stratified samples

     • combining tests using \(E\)-values: multiplication and averaging

     • stratified inference using union-intersection tests

     • product supermartingales: ALPHA and Sweeter than SUITE

4. Weeks 9-10: Inference about bounded populations. - Kaplan, H., 1987. A Method of One-Sided Nonparametric Inference for the Mean of a Nonnegative Population, The Amer. Statistician, 41, 157-158. https://www.tandfonline.com/doi/abs/10.1080/00031305.1987.10475470?journalCode=utas20 - Stark, P.B., 2023. ALPHA: Audit that Learns from Previously Hand-Audited ballots, Ann. Appl. Stat., https://www.e-publications.org/ims/submission/AOAS/user/submissionFile/54812?confirm=3a9dc0d4 - Vovk, V., and R. Wang, 2021. \(E\)-values: Calibration, combination, and applications, http://alrw.net/e/02.pdf - Waudby-Smith, I. and A. Ramdas, 2022. Estimating means of bounded random variables by betting, https://arxiv.org/abs/2010.09686

5. Weeks 11-12: Resampling methods

6. Weeks 13-14: Density estimation and inference about densities

7. Weeks 15: Inverse problems and uncertainty quantification