Inference on quantiles: confidence intervals, p-values, and testing

Improved quantile inference for one- and two-sample (e.g., treatment vs. control) cases

Detailed documentation includes further explanation and examples; just type "help quantile_inf". The following briefly describes functionality as well as the theoretical foundations from the literature.

This code implements four different methods for quantile inference: Hutson (1999), bootstrap percentile-t, Kaplan (2011), and Kaplan and Goldman (2012). Papers with theoretical and simulation results for the third (and soon fourth) method can be found here: http://econ.ucsd.edu/~dkaplan/personalResearch.html

This Matlab code provides hypothesis testing (reject/don't), p-values, and confidence intervals, for one- and two-sample setups, and one- and two-sided inference.

When it is computable, the exact method based on fractional order statistics in Hutson (1999) always controls size and has significantly better power than other analytic or bootstrap methods in the one-sample case. Even for small samples, among simulations of 14 different types of distribution, empirical size for a nominal 5% test never exceeded 5.5%. The Kaplan and Goldman (2012) two-sample method is also based on fractional order statistics and has similarly good simulation results.

When a fractional order statistic method is not computable (further in the tails), the method of Kaplan (2011) or a bootstrap percentile-t is used. Extensive simulations (Section 6 of paper has some examples) have shown Kaplan (2011) to control size better than bootstrap ("BS") methods and the analytic Hall and Sheather (1988; "HS") method, while maintaining competitive power. This new method builds on the work of HS and Goh (2004) to improve testing properties via testing-optimal smoothing parameter selection using Edgeworth expansions and smoothing parameter dependent critical values. Simulations have not been done in the two-sample case when the sample are different in size, so bootstrap percentile-t is used there. In general, this particular bootstrap performs similarly to Kaplan (2011) in the two-sample case, though suffers size distortion in the one-sample case.

For example, if you have a univariate data sample and want to test if it is from a population with median (or whichever quantile/percentile) equal to zero, you can get a p-value, run this test at a 5% (or whatever value you input) level, and get a 95% (or whatever value you want) confidence interval for the median.

If you have two samples (say, treatment and control), you can test for equality at any quantile (percentile), either computing a p-value or testing at a specified level.

Again, detailed documentation includes further explanation and examples; just type "help quantile_inf"