Learn more about Model-Based Calibration

Design Evaluation Tool

On this page…
Introducing the Design Evaluation Tool Table Options Design Matrix Full FX Matrix Model Terms Z2 Matrix Alias Matrix Z2.1 Matrix Regression Matrix Coefficient Information Standard Error Hat Matrix |X'X| Raw Residual Statistic Degrees of Freedom Table Design Evaluation Graphical Displays Export of Design Evaluation Information

Introducing the Design Evaluation Tool

The Design Evaluation tool is only available for linear models.

You can open the Design Evaluation tool from the Design Editor or from the Model Browser windows. From the Design Editor select Tools > Evaluate Designs and choose the design you want to evaluate. From the Model Browser global view, you can click the button.

In the Design Evaluation tool you can view all the information on correlations, covariance, confounding, and variance inflation factors (VIFs). You can investigate the effects of including or excluding model terms aided by this information (you must remove them in the Stepwise window). Interpretation is aided by color-coded values based on the magnitude of the numbers. You can specify changes to these criteria.

When you open the Design Evaluation tool, the default view is a table, as shown in the preceding example. You choose the elements to display from the list on the right. Click any of the items in the list described below to change the display. Some of the items have a choice of buttons that appear underneath the list box.

To see information about each display, click the toolbar button or select View > Matrix Information.

Back to Top

Table Options

You can apply color maps and filters to any items in a table view, and change the precision of the display.

To apply a color map or edit an existing one:

1. Select Options > Table > Colors. The Table Colors dialog box appears.

2. Select the check box Use a colormap for rendering matrix values.

3. Click the Define colormap button. The Colormap Editor dialog box appears, where you can choose how many levels to color map, and the colors and values to use to define the levels. Some tables have default color maps to aid analysis of the information, described below.

You can also use the Options > Table menu to change the precision (number of significant figures displayed) and to apply filters that remove specific values or values above or below a specific value from the display.

The status bar at bottom left displays whether color maps and filters are active in the current view.

When evaluating several designs, you can switch between them with the Next design toolbar button or the Design menu.

Back to Top

Design Matrix

Xn/Xc: design/actual factor test points matrix for the experiment, in natural or coded units. You can toggle between natural and coded units with the buttons on the right.

Back to Top

Full FX Matrix

Full model matrix, showing all possible terms in the model. You can include and exclude terms from the model here, by clicking on the green column headings. When you click one to remove a term, the column heading becomes red and the whole column is grayed.

The Full FX matrix is the same as the Jacobian for linear models if all terms included. Jacobian only includes 'in' terms. In the general case, jacobian is expressed:

(i, j) = df/dp_j (x_i)

In the case of linear models and RBFs this simplifies to:

(i,j) = j^th term evaluated at i^th data point = Jacobian matrix.

Back to Top

Model Terms

You can select terms for inclusion in or exclusion from the model here by clicking. You can toggle the button for each term by clicking. This changes the button from in (green) to out (red) and vice versa. You can then view the effect of these changes in the other displays.

Note   Removal of model terms only affects displays within the Design Evaluation tool. If you decide the proposed changes would be beneficial to your model, you must return to the Stepwise window and make the changes there to fit the new model.

Back to Top

Z2 Matrix

Z₂: Matrix of terms that have been removed from the model. If you haven't removed any terms, the main display is blank apart from the message "All terms are currently included in the model."

Back to Top

Alias Matrix

Like the Z₂ matrix, the alias matrix also displays terms that are not included in the model (and is therefore not available if all terms are included in the model). The purpose of the alias matrix is to show the pattern of confounding in the design.

A zero in a box indicates that the row term (currently included in the model) is not confounded with the column term (currently not in the model). A complete row of zeros indicates that the term in the model is not confounded with any of the terms excluded from the model. A column of zeros also indicates that the column term (currently not in the model) could be included (but at the cost of a reduction in the residual degrees of freedom).

A: the alias matrix is defined by the expression

Back to Top

Z2.1 Matrix

As this matrix also uses the terms not included in the model, it is not available if all terms are included.

Z_2.1 : Matrix defined by the expression

Back to Top

Regression Matrix

Regression matrix. Consists of terms included in the model. matrix where n is the number of test points in the design and p is the number of terms in the model.

Back to Top

Coefficient Information

When you select Coefficient information, six buttons appear below the list box. Covariance is displayed by default; click the buttons to select any of the others for display.

Covariance

Cov(b): variance-covariance matrix for the regression coefficient vector b.

Correlation

Corr(b): correlation matrix for the regression coefficient vector b.

By default Correlation has an active color map to aid analysis. Values below -0.9 are red, -0.9 to -0.7 are orange, -0.7 to 0.707 are black, 0.707 to 0.9 are orange, and greater than 0.9 are red. You can view and edit the color map using Options > Table > Colors.

Partial VIFs

Variance Inflation Factors (VIFs) are a measure of the nonorthogonality of the design with respect to the selected model. A fully orthogonal design has all VIFs equal to unity.

The Partial VIFs are calculated from the off-diagonal elements of Corr(b) as

for

Partial VIFs also has a default color map active (<1.2 black, >1.2<1.4 orange, >1.4 red). A filter is also applied, removing all values within 0.1 of 1. In regular designs such as Box-Behnken, many of the elements are exactly 1 and so need not be displayed; this plus the color coding makes it easier for you to see the important VIF values. You can always edit or remove color maps and filters.

Multiple VIFs

Measure of the nonorthogonality of the design. The Multiple VIFs are defined as the diagonal elements of Corr(b):

Multiple VIFs also has a default color map active (<8 black, 8><10 orange, >10 red). A filter is also applied, removing all values within 0.1 of 1. Once again this makes it easier to see values of interest.

2 Column Corr.

Corr(X); correlation for two columns of X.

Let W denote the matrix of w_ij values. Then the correlation matrix for the columns of X (excluding column 1) is Corr(X), defined as

Corr(X) = W'W

2 Column Correlation has the same default color map active as Correlation.

Single Term VIFs

Measure of the nonorthogonality of the design. The Single Term VIFs are defined as

for

Single term VIFs have a default color map active (<2 black, 2>red) and values within 0.1 of 1 are filtered out, to highlight values of interest.

Back to Top

Standard Error

: Standard error of the j^th coefficient relative to the RMSE.

Back to Top

Hat Matrix

Full Hat matrix

H: The Hat matrix.

H = QQ'

where Q results from a QR decomposition of X. Q is an orthonormal matrix and R is an matrix.

Leverage values

The leverage values are the terms on the leading diagonal of H (the Hat matrix). Leverage values have a color map active (<0.8 black, 0.8>orange<0.9, >0.9 red).

Back to Top

|X'X|

D; determinant of X'X.

D can be calculated from the QR decomposition of X as follows:

where p is the number of terms in the currently selected model.

This can be displayed in three forms:

Back to Top

Raw Residual Statistic

Covariance

Cov(e): Variance-covariance matrix for the residuals.

Cov(e) = (I-H)

Correlation

Corr(e) : Correlation matrix for the residuals.

Back to Top

Degrees of Freedom Table

To see the Degrees of Freedom table (and the information about each display), click the toolbar button or select View > Matrix Information.

Source	D.F.
Model	p
Residual	n-p
Replication	by calculation
Lack of fit	by calculation
Total	n

Replication is defined as follows:

Let there be n_j (>1) replications at the j^th replicated point. Then the degrees of freedom for replication are

and Lack of fit is given by n - p - degrees of freedom for replication.

Note: replication exists where two rows of X are identical. In regular designs the factor levels are clearly spaced and the concept of replication is unambiguous. However, in some situations spacing can be less clear, so a tolerance is imposed of 0.005 (coded units) in all factors. Points must fall within this tolerance to be considered replicated.

Back to Top

Design Evaluation Graphical Displays

The Design Evaluation tool has options for 1-D, 2-D, 3-D, and 4-D displays. You can switch to these by clicking the toolbar buttons or using the View menu.

Which displays are available depends on the information category selected in the list box. For the Design matrix, (with sufficient inputs) all options are available. For the Model terms, there are no display options.

You can edit the properties of all displays using the Options menu. You can configure the grid lines and background colors. In the 2-D image display you can click points in the image to see their values. All 3-D displays can be rotated as usual. You can edit all color map bars by double-clicking.

Back to Top

Export of Design Evaluation Information

All information displayed in the Design Evaluation tool can be exported to the workspace or to a .mat file using the radio buttons and Export button at the bottom right. You can enter a variable name in the edit box.

Back to Top

Prediction Error Variance Viewer

Data

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS