File Exchange

image thumbnail


version (61.2 KB) by Andrew Penn
Two-sided bootstrap confidence intervals for small samples and samples with complex dependence structures


Updated 04 Aug 2020

GitHub view license on GitHub

Function File: ibootci

Bootstrap confidence intervals

ci = ibootci(nboot,bootfun,...) computes the 95% iterated (double) bootstrap confidence interval of the statistic computed by bootfun. nboot is a scalar, or vector of upto two positive integers indicating the number of replicate samples for the first and second bootstraps. bootfun is a function handle specified with @, or a string indicating the function name. The third and later input arguments are data (column vectors), that are used to create inputs to bootfun. ibootci creates each first level bootstrap by sampling from the rows of the column vector data arguments (which must be the same size). Nominal central coverage of two-sided intervals is calibrated to achieve second order accurate coverage by bootstrap iteration and interpolation. Linear interpolation of the empirical cumulative distribution function of bootstat is then used to construct two-sided confidence intervals. The resampling method used throughout is balanced resampling. Default values for the number of first and second bootstrap replicate sample sets in nboot are 5000 and 200 respectively.

ci = ibootci(nboot,{bootfun,...},...,'alpha',alpha) computes the iterated bootstrap confidence interval of the statistic defined by the function bootfun with coverage 100*(1-alpha)%, where alpha is a scalar value between 0 and 1. bootfun and the data that ibootci passes to it are contained in a single cell array. The default value of alpha is 0.05 corresponding to intervals with a coverage of 95% confidence.

ci = ibootci(nboot,{bootfun,...},...,'type',type) computes the bootstrap confidence interval of the statistic defined by the function bootfun. type is the confidence interval type, chosen from among the following:
'per' or 'percentile' - Percentile method.
'cper' - Bias-corrected percentile method.
'bca' - Bias-corrected and accelerated method (default).
'stud' or 'student' - Studentized (bootstrap-t) confidence interval.

ci = ibootci(nboot,{bootfun,...},...,'Weights',weights) specifies observation weights. weights must be a vector of non-negative numbers. The dimensions of weights must be equal to that of the non-scalar input arguments to bootfun. The weights are used as bootstrap sampling probabilities.

ci = ibootci(nboot,{bootfun,...},...,'Strata',strata) specifies a vector containing numeric identifiers of strata. The dimensions of strata must be equal to that of the non-scalar input arguments to bootfun. Bootstrap resampling is stratified so that every stratum is represented in each bootstrap test statistic.

ci = ibootci(nboot,{bootfun,...},...,'Clusters',clusters) specifies a vector containing numeric identifiers for clusters. Whereas strata are fixed, clusters are resampled. This is achieved by two-stage bootstrap resampling of residuals with shrinkage correction.

ci = ibootci(nboot,{bootfun,...},...,'Block',blocksize) specifies a positive integer (or 'auto') defining the block length for block bootstrapping dependent-data (e.g. time series). The algorithm uses circular, overlapping blocks. The double bootstrap resampling and calibration procedure makes interval coverage less sensitive block length. Block bootstrap can also be used in regression contexts by combining it with pairs bootstrap (i.e. by providing x and y vectors as data variables).

ci = ibootci(nboot,{bootfun,...},...,'smooth',bandwidth) applies additive random noise of the specified bandwidth to the bootstrap sample sets before evaluating bootfun.

ci = ibootci(nboot,{bootfun,...},...,'DEFF',state) estimates the design effect by resampling. State can be 'on' or 'off'.

[ci,bootstat] = ibootci(...) also returns the bootstrapped statistic computed for each of the bootstrap replicate samples sets. If only a single bootstrap is requested, bootstat will return a vector: each column of bootstat contains the result of applying bootfun to one replicate sample from the first bootstrap. If bootstrap iteration is requested, bootstat will return a cell array containing the statistics computed by bootfun in the first and second bootstrap. For the second boostrap, each column of bootstat contains the results of applying bootfun to each replicate sample from the second bootstrap for one replicate sample from the first bootstrap.

[ci,bootstat,S] = ibootci(...) also returns a structure containing the settings used in the bootstrap and the resulting statistics including the (double) bootstrap bias and standard error. Field descriptions are provided in the function help.

[ci,bootstat,S,calcurve] = ibootci(...) also returns calcurve, a vector of calibration values the same size as the first bootstrap replicate set.

[ci,bootstat,S,calcurve,bootidx] = ibootci(...) also returns bootidx, a matrix of indices from the first bootstrap.

Computations of confidence intervals can be accelerated by parallel processing.

ibootci has very similar usage to the function bootci. Please see function help for examples of function usage.

Cite As

Andrew Penn (2020). ibootci (, GitHub. Retrieved .

Comments and Ratings (6)


Andrew Penn

Hi Robert. The usage is very similar to bootci in the Matlab Statistics Toolbox. I have now added some example uses in the function help.


Looks great - could you please provide a simple example to illustrate correct use?


Corrected bug in double block bootstrap: Second block bootstrap resampling no longer samples across the block joints of the first bootstrap sample sets.

Fixed bug that prevented calculation of BCa intervals for Block bootstrap and made calculations of acceleration constant (for BCa intervals) erroneous for bootstrapping clustered data.

Added BCa intervals for all types of bootstrap sampling (ordinary, weighted, block, cluster etc.). Bug fix for cluster bootstrap resampling. Added parallel computing capabilities for Octave. Subfunctions moved into function files in iboot package.

Now includes bias-corrected percentile intervals

Added the option for smoothing the bootstrap to improve interval coverage for small smples. Removed BCa type intervals since these can be unreliable for small samples.

Parallel implementation of ibootci now works for all resampling options (with the only difference being that the first level bootstrap uses ordinary resampling with replacement instead of balanced resampling)

Added support for accelerating calculations by performing computations in parallel across CPUs. Made calculations of design effect optional.

Minor bug fix for DEFF calculation after Block and Cluster bootstrap.

Now includes a new optional input argument 'stderr' for provision of a function handle to calculate standard errors for Studentized intervals (instead of using bootstrap iteration). Also includes minor bug fixes.

Minor changes to syntax to provide backwards compatibility back to R2007A

For cluster resampling option, balanced resampling now occurs at the level of cluster means. Resampling now used to estimate design effect across the full range of input options. Minor bug fixes.

Wild bootstrap removed since it is now superseded by pairs bootstrap, which is implemented with double bootstrap and is compatible with block resampling. Corrected bootidx output argument for block resampling.

Now includes an option to perform Wild bootstrap, compatible with BCa intervals but not bootstrap iteration. Output structure includes autocorrelation results to examine the assumption of i.i.d.

Improved algorithm for block bootstrap. Now combines circular, overlapping blocks with the double bootstrap approach of Lee and Lai (2009); the calibration method makes interval coverage less sensitive to the choice of block length.

Added capability for block bootstrap of dependent data (e.g. time series)

Improved the coverage of cluster bootstrap for unbalanced cluster sizes.

Trimmed output structure down to providing only necessary information

Corrected calculation of ICC for output structure S

Now includes capability to perform two-stage nonparametric bootstrap sampling with shrinkage correction for clustered data

Made calculation of SSE components of stratified data (provided as descriptive statistics in output structure S) compatible with multivariate data. Now also returns the degrees of freedom.

Changed default interval type to BCa to be consistent with bootci. Corrected some comments and function help.

Added extra tags to the webpage

Now supports calculation confidence intervals using the bootstrap-t method. Standard errors of the bootstrap samples are determined by double bootstrap.

This new version can also perform stratified bootstrap resampling.

If bootstrap iteration is requested, bootstat returns: boostat statistics from both 1st and 2nd bootstrap; the double bootstrap bias and bias-corrected statistic in S; and, the double boostrap corrected standard error of the statistic in S.

Added a check for successful calculation of the acceleration constant for the BCa intervals

Corrected the check for when bootstrap confidence intervals hit the ends of the bootstrap distribution

Updated title and summary

Linked to Github

Moved to Github. Calibration and interval construction carried out by interpolating from the empirical distribution function. Minor performance improvement by avoiding a sort in the second bootstrap. Coverage confirmed with simulations.

Performance enhancement for the second bootstrap through alternative execution of randperm

Included error message when second bootstrap replicate size is insufficient for the desired alpha. Minor improvement to the calibration included. Changed output arguments: Now includes output structure (of settings and results) and calibration curve.

Added back a warning for when the resulting intervals hit the end(s) of the bootstrap distribution

Corrected typos (missing % signs) in the comments of the examples in the description

Updated image icon

Adjusted icon image dimensions

Adjusted icon image dimensions

Adjusted icon image size

Minor update to function description. Added image icon.


Swapped the order of output arguments to match Matlabs bootci


Set default bootstrap type to percentile.


Added the ability to use different types of intervals: percentile or BCa. The BCa type interval is the default.

Made minor changes to the description and to the function help

Corrected description

Added the capability to use bootidx input when calculating bootstrap confidence for multivariate data. Modified default values for nboot

Clarified some of the help/comments and included another example of it's usage for calculating confidence intervals for the median

Updated some comments

Modified default values for the number of bootstrap samples

Modified the implementation with weights based on simulation results

Updated the help comments

Fixed a bug preventing some function usage. Added back the ability to use functions (in the bootfun input argument) that do not output a row vector for matrix input argument(s) (e.g. 'corr' function).

Removed depreciated lines of code

Implemented appropriated weight allocation for the second bootstrap

Added interpolation also to the first bootstrap. Performance improvements made to the resampling algorithm. Added feature to provide weights to modify sampling probabilities.

Corrected a mistake presented in example 3 (in comment only). - Implementation modified for less memory footprint. Algorithm now uses balanced resampling. Coverage is calibrated using the interpolation method instead of the approximate Monte Carlo method. Also returns bias of the bootstrap.

Simplified title

Included example usage in the function help

Now less stringent data requirements. Data can be row vectors or column vectors

Now checks the dimensions of the data

MATLAB Release Compatibility
Created with R2019a
Compatible with R2007a to any release
Platform Compatibility
Windows macOS Linux