Using Validation Data

After you have created models, you can use the validation data.

To import and filter the validation data set,

1. Select the top project node in the Model Tree.

2. Double click the Data Object in the Data Sets pane. The Data Editor appears.

3. Click the Storage button in the toolbar.

   Click Store current filters and Store current test filters in the toolbar. This allows you to reuse these filters in another data object without having to recreate them.

4. In the Model Browser, click New Data in the toolbar.

5. Click the Open File icon in the toolbar to load data from a file.

   The Data Import Wizard appears to select a file.

6. Use the Browse button to find and select the DIVCP_Validation_DoE_Data.xls data file in the mbctraining folder. Double-click to load the file, and click Next.

7. The Data Import Wizard displays a summary screen. Click Finish to accept the data.

8. You need to define test groupings as before.

   1. Select Tools > Change Test Groupings (or use the toolbar button)

   2. In the Test Groupings dialog box, clear the check box One test/record.

   3. Locate and double-click GDOECT in the Variables list box. GDOECT appears in the left list.25 tests are defined. Close the Define Test Groupings dialog box.

9. To apply the same filters as before, click the Storage button in the toolbar.

   In the Storage dialog box, select the filter and test filter objects in turn and click Append stored object in the toolbar. This applies these filters to the current data object. Return to the Data Editor window and check the filters in the Filter List and Test Filter List View.

10. Close the Data Editor.

To attach the data set to your test plan for validation:

1. At the test plan level, select TestPlan > Validation Data. The Select Data for Validation wizard appears.

2. Select the validation data set (observe the number of tests), and click Next.

3. By default all tests are selected on the next screen, so click Finish to use all the tests to validate models in this test plan.

The validation data set appears in the right information pane for the test plan. Validation RMSE is automatically added to the summary statistics for comparison in the bottom list view of response models in the test plan.

You can now use the validation data to validate all models except response features. You can see validation statistics in the following places:

• Model List — Validation RMSE appears in the summary statistics in the lower list of models at the test plan, response and one-stage nodes

• At the local node view:

  • Pooled Statistics — Validation RMSE — The root mean squared error between the two-stage model and the validation data for all tests

  • Diagnostic Statistics > Local Diagnostics — Local model Validation RMSE for the currently selected test (if validation data is available for the current test—global variables must match)

• Summary Table — Validation RMSE for one-stage models

View validation plots in the following places:

• Plots of Validation residuals — For local models

• From any model node except response features, you can select Model > Evaluate > Validation Data to open the Model Evaluation window and investigate the model with the selected validation data.

  Check the model trends in the cross section view. Check the fit against the validation data. Is the fit acceptable? Do you need more data? Can you improve the fit with the existing data? Pay attention to the boundary model on the plots. Yellow areas are outside the boundary. Focus on the model trends only within the boundary model.

Selecting Global and Two-Stage Models

Exporting the Models

© 1984-2012- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS