Simulink.sdi.compareRuns

Compare data in two simulation runs

Description

example

diffResult = Simulink.sdi.compareRuns(runID1,runID2) compares the data in the runs that correspond to runID1 and runID2 and returns the result in the Simulink.sdi.DiffRunResult object diffResult. The comparison uses the Simulation Data Inspector comparison algorithm. For more information about the algorithm, see How the Simulation Data Inspector Compares Data.

example

diffResult = Simulink.sdi.compareRuns(runID1,runID2,Name,Value) compares the simulation runs that correspond to runID1 and runID2 using the options specified by one or more Name,Value pair arguments. For more information about how the options can affect the comparison, see How the Simulation Data Inspector Compares Data.

Examples

collapse all

This example shows how to compare runs of simulation data and then analyze and save the results using the Simulation Data Inspector programmatic interface.

Create Simulation Data

First, create simulation data by simulating a model that logs data. This example uses the ex_sldemo_absbrake model and analyzes the effect of changing the Desired relative slip value.

Load the model. Use the set_param function to specify an initial value for the relative slip and simulate the model.

load_system('ex_sldemo_absbrake')

set_param('ex_sldemo_absbrake/Desired relative slip','Value','0.24')
out_1 = sim('ex_sldemo_absbrake');

Use the set_param function to specify a different value for the relative slip and simulate the model again.

set_param('ex_sldemo_absbrake/Desired relative slip','Value','0.25')
out_2 = sim('ex_sldemo_absbrake');

Compare Runs Using Global Tolerance Values

First, use the Simulink.sdi.getAllRunIDs function to get the run IDs that correspond to the last two simulation runs.

runIDs = Simulink.sdi.getAllRunIDs;
runID1 = runIDs(end - 1);
runID2 = runIDs(end);

Now, use the Simulink.sdi.compareRuns function to compare the runs. Specify a global relative tolerance value of 0.2 and a global time tolerance value of 0.5.

runResult = Simulink.sdi.compareRuns(runID1,runID2,'reltol',0.2,'timetol',0.5);

Check the Summary property of the returned Simulink.sdi.DiffRunResult object.

runResult.Summary
ans = struct with fields:
       OutOfTolerance: 2
      WithinTolerance: 2
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

Two signal comparisons within the run were within tolerance, and two were out of tolerance.

Plot Comparison Results

You can use plots to analyze the comparison results. Access the signal result for the Ww signal from the DiffRunResult object that contains the comparison results using the getResultByIndex function. Check the Status property of the Simulink.sdi.DiffSignalResult object.

signalResult_Ww = getResultByIndex(runResult,1)
signalResult_Ww = 
  DiffSignalResult with properties:

             Name: 'yout.Ww'
           Status: OutOfTolerance
          AlignBy: 'Path'
        SignalID1: 120877
        SignalID2: 120923
    MaxDifference: 12.4878
            Sync1: [1x1 timeseries]
            Sync2: [1x1 timeseries]
             Diff: [1x1 timeseries]

signalResult_Ww.Status
ans = 
OutOfTolerance

The Ww signal comparison results are out of tolerance. Plot the difference signal to analyze the result.

figure(1)
plot(signalResult_Ww.Diff)

Save Comparison Results

You can save the comparison results to an MLDATX file to analyze later or to share with a colleague. Use the saveResult function to save the run data and comparison results.

saveResult(runResult,'desiredSlipResults')

The MLDATX file desiredSlipResults is created in the working directory. Use the Simulink.sdi.load function or the open function to view the results in the MLDATX file.

Using the Simulation Data Inspector programmatic interface, you can specify signal tolerance values to use in comparisons. This example uses the slexAircraftExample model and the Simulation Data Inspector to evaluate the effect of changing the time constant for the low-pass filter following the control input.

Configure the Model

Load the model and mark signals of interest for logging. This example logs data for the q and alpha signals.

load_system('slexAircraftExample')

Simulink.sdi.markSignalForStreaming('slexAircraftExample/Aircraft Dynamics Model',3,'on')
Simulink.sdi.markSignalForStreaming('slexAircraftExample/Aircraft Dynamics Model',4,'on')

Run Simulations

Run simulations with different low-pass filter time constants to generate results to compare. The slexAircraftExample model stores variables associated with the model in the model workspace. To modify the time constant value, access the model workspace and use the assignin function.

out1 = sim('slexAircraftExample');

modelWorkspace = get_param('slexAircraftExample','modelworkspace');
assignin(modelWorkspace,'Ts',1)

out2 = sim('slexAircraftExample');

Access and Compare Simulation Results

Access the simulation results using the Simulation Data Inspector programmatic interface. Each simulation creates a run in the Simulation Data Inspector with a unique run ID. You use the run IDs to compare the simulation results.

runIDs = Simulink.sdi.getAllRunIDs;
runIDTs1 = runIDs(end-1);
runIDTs2 = runIDs(end);

Use the Simulink.sdi.compareRuns function to compare the data from the simulations. Then inspect the Status property of the signal result to see whether the signals fell within the default tolerance of 0.

diffRun1 = Simulink.sdi.compareRuns(runIDTs1,runIDTs2);

sig1Result1 = getResultByIndex(diffRun1,1);
sig2Result1 = getResultByIndex(diffRun1,2);

sig1Result1.Status
ans = 
OutOfTolerance
sig2Result1.Status
ans = 
OutOfTolerance

Compare Runs with Signal Tolerances

By default, signals use 0 for all tolerance values, so the comparison returns out-of-tolerance results when the signals are not identical. To further analyze the effect of the time constant change, specify tolerance values for the signals. You can specify tolerances for a programmatic comparison using the properties of the Simulink.sdi.Signal objects in the runs you compare. The comparison uses the tolerances specified for the baseline Signal object. This example specifies a combination of time and absolute tolerances.

To specify tolerances, first access the Simulink.sdi.Signal objects that correspond to each signal in the runs you want to compare.

run1 = Simulink.sdi.getRun(runIDTs1);
sigID1 = getSignalIDByIndex(run1,1);
sigID2 = getSignalIDByIndex(run1,2);

sig1 = Simulink.sdi.getSignal(sigID1);
sig2 = Simulink.sdi.getSignal(sigID2);

Check the Name property to identify each Signal object.

sig1.Name
ans = 
'q, rad/sec'
sig2.Name
ans = 
'alpha, rad'

Specify an absolute tolerance of 0.1 and a time tolerance of 0.6 for the q signal using the AbsTol and TimeTol properties of the q signal object in the baseline run.

sig1.AbsTol = 0.1;
sig1.TimeTol = 0.6;

Specify an absolute tolerance of 0.2 and a time tolerance of 0.8 for the alpha signal using the AbsTol and TimeTol properties of the alpha signal object in the baseline run.

sig2.AbsTol = 0.2;
sig2.TimeTol = 0.8;

Compare the runs again and access the results.

diffRun2 = Simulink.sdi.compareRuns(runIDTs1,runIDTs2);
sig1Result2 = getResultByIndex(diffRun2,1);
sig2Result2 = getResultByIndex(diffRun2,2);

Check the Status property of each signal to determine whether the comparison results fell within the specified tolerances.

sig1Result2.Status
ans = 
WithinTolerance
sig2Result2.Status
ans = 
WithinTolerance

You can use the Simulink.sdi.compareRuns function to compare signal data and metadata, including data type and start and stop times. A single comparison may check for mismatches in one or more pieces of metadata. When you check for mismatches in signal metadata, the Summary property of the Simulink.sdi.DiffRunResult object may differ from a basic comparison because the Status property for a Simulink.sdi.DiffSignalResult object can indicate the metadata mismatch. You can configure comparisons using the Simulink.sdi.compareRuns function for imported data and for data logged from a simulation.

This example configures a comparison of runs created from workspace data three ways to show how the Summary of the DiffSignalResult object can provide specific information about signal mismatches.

Create Workspace Data

The Simulink.sdi.compareRuns function compares time series data. Create data for a sine wave to use as the baseline signal, using the timeseries format. Give the timeseries the name Wave Data.

time = 0:0.1:20;
sig1vals = sin(2*pi/5*time);
sig1_ts = timeseries(sig1vals,time);
sig1_ts.Name = 'Wave Data';

Create a second sine wave to compare against the baseline signal. Use a slightly different time vector and attenuate the signal so the two signals are not identical. Cast the signal data to the single data type. Also name this timeseries object Wave Data. The Simulation Data Inspector comparison algorithm will align these signals for comparison using the name.

time2 = 0:0.1:22;
sig2vals = single(0.98*sin(2*pi/5*time2));
sig2_ts = timeseries(sig2vals,time2);
sig2_ts.Name = 'Wave Data';

Create and Compare Runs in the Simulation Data Inspector

The Simulink.sdi.compareRuns function compares data contained in Simulink.sdi.Run objects. Use the Simulink.sdi.createRun function to create runs in the Simulation Data Inspector for the data. The Simulink.sdi.createRun function returns the run ID for each created run.

runID1 = Simulink.sdi.createRun('Baseline Run','vars',sig1_ts);
runID2 = Simulink.sdi.createRun('Compare to Run','vars',sig2_ts);

You can use the Simulink.sdi.compareRuns function to compare the runs. The comparison algorithm converts the signal data to the double data type and synchronizes the signal data before computing the difference signal.

basic_DRR = Simulink.sdi.compareRuns(runID1,runID2);

Check the Summary property of the returned Simulink.sdi.DiffRunResult object to see the result of the comparison.

basic_DRR.Summary
ans = struct with fields:
       OutOfTolerance: 1
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

The difference between the signals is out of tolerance.

Compare Runs and Check for Data Type Match

Depending on your system requirements, you may want the data types for signals you compare to match. You can use the Simulink.sdi.compareRuns function to configure the comparison algorithm to check for and report data type mismatches.

dataType_DRR = Simulink.sdi.compareRuns(runID1,runID2,'DataType','MustMatch');
dataType_DRR.Summary
ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 1
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

The result of the signal comparison is now DataTypeMismatch because the data for the baseline signal is double data type, while the data for the signal compared to the baseline is single data type.

Compare Runs and Check for Start and Stop Time Match

You can use the Simulink.sdi.compareRuns function to configure the comparison algorithm to check whether the aligned signals have the same start and stop times.

startStop_DRR = Simulink.sdi.compareRuns(runID1,runID2,'StartStop','MustMatch');
startStop_DRR.Summary
ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 1
          Unsupported: 0

The signal comparison result is now StartStopMismatch because the signals created in the workspace have different stop times.

This example shows how to compare runs using your desired criteria for aligning signals between runs.

Generate Simulation Data to Compare

Start by loading the slexAircraftExample system and marking signals of interest for logging.

load_system('slexAircraftExample')

Simulink.sdi.markSignalForStreaming('slexAircraftExample/Pilot',1,'on')
Simulink.sdi.markSignalForStreaming('slexAircraftExample/Aircraft Dynamics Model',... 
    4,'on')

This example runs two simulations of the slexAircraftExample model with different values for the variable Ts. Use get_param to access the model workspace for the slexAircraftExample model. Then, assign a value for the variable and run the first simulation.

modelWorkspace = get_param('slexAircraftExample','modelworkspace');
modelWorkspace.assignin('Ts',0.1);

out_1 = sim('slexAircraftExample');

Change the value of Ts and run the second simulation. Then,

modelWorkspace.assignin('Ts',0.2);

out_2 = sim('slexAircraftExample');
 

Use the Simulink.sdi.getAllRunIDs function to access the run IDs for the two simulation runs created when you simulated the slexAircraftExample model.

runIDs = Simulink.sdi.getAllRunIDs;
runID1 = runIDs(end-1);
runID2 = runIDs(end);

Define Alignment Criteria for the Comparison

Before running the comparison, define how you want the Simulation Data Inspector to align the signals between the runs. This example aligns signals by their name, then by their block path, and then by their Simulink identifier.

alignMethods = [Simulink.sdi.AlignType.SignalName
               Simulink.sdi.AlignType.BlockPath
               Simulink.sdi.AlignType.SID];

Compare the Runs with the Specified Alignment Criteria

Compare the simulation data in your two runs, using the alignment criteria you specified. The comparison uses a small time tolerance to account for the effect of differences in the step size used by the solver on the transition of the square wave input.

diffResults = Simulink.sdi.compareRuns(runID1,runID2,'align',alignMethods,...
    'timetol',0.005);

Check the Comparison Results for the Aligned Signals

You can use the getResultByIndex method to access the comparison results for the aligned signals in the runs you compared. You can use the Count property of the Simulink.sdi.DiffRunResult object to set up a for loop to check the Status property for each Simulink.sdi.DiffSignalResult object.

numComparisons = diffResults.count;

for k = 1:numComparisons
    resultAtIdx = getResultByIndex(diffResults,k);
    
    sigID1 = resultAtIdx.signalID1;
    sigID2 = resultAtIdx.signalID2;
    
    sig1 = Simulink.sdi.getSignal(sigID1);
    sig2 = Simulink.sdi.getSignal(sigID2);
    
    displayStr = 'Signals %s and %s: %s \n';
    fprintf(displayStr,sig1.Name,sig2.Name,resultAtIdx.Status);
end
Signals alpha, rad and alpha, rad: OutOfTolerance 
Signals Stick and Stick: WithinTolerance 

Input Arguments

collapse all

Numeric identifier for the baseline run in the comparison, specified as a run ID that corresponds to a run in the Simulation Data Inspector. The Simulation Data Inspector assigns run IDs when runs are created. You can get the run ID for a run by using the ID property of the Simulink.sdi.Run object, the Simulink.sdi.getAllRunIDs function, or the Simulink.sdi.getRunIDByIndex function.

Numeric identifier for the run to compare, specified as a run ID that corresponds to a run in the Simulation Data Inspector. The Simulation Data Inspector assigns run IDs when runs are created. You can get the run ID for a run by using the ID property of the Simulink.sdi.Run object, the Simulink.sdi.getAllRunIDs function, or the Simulink.sdi.getRunIDByIndex function.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: 'abstol',x,'align',alignOpts

Signal alignment options, specified as the comma-separated pair consisting of 'Align' and a string array or array of character vectors.

Array specifying alignment options to use for pairing signals from the runs being compared. The Simulation Data Inspector aligns signals first by the first element in the array, then by the second element in the array, and so on. For more information, see Signal Alignment.

ValueAligns By
Simulink.sdi.AlignType.BlockPathPath to the source block for the signal
Simulink.sdi.AlignType.SIDSimulink® identifier Simulink Identifiers
Simulink.sdi.AlignType.SignalNameSignal name
Simulink.sdi.AlignType.DataSourcePath of the variable in the MATLAB® workspace

Example: [Simulink.sdi.AlignType.SignalName,Simulink.sdi.AlignType.SID] specifies signal alignment by name and then by SID.

Positive-valued global absolute tolerance used for all signals in the run comparison, specified as the comma-separated pair consisting of 'AbsTol' and a scalar. For more information about how tolerances are used in comparisons, see Tolerance Specification.

Example: 0.5

Data Types: double

Positive-valued global relative tolerance used for all signals in the run comparison, specified as the comma-separated pair consisting of 'RelTol' and a scalar. The relative tolerance is expressed as a fractional multiplier. For example, 0.1 specifies a 10 percent tolerance. For more information about how the relative tolerance is applied in the Simulation Data Inspector, see Tolerance Specification.

Example: 0.1

Data Types: double

Positive-valued global time tolerance used for all signals in the run comparison, specified as the comma-separated pair consisting of 'TimeTol' and a scalar. Specify the time tolerance in units of seconds. For more information about tolerances in the Simulation Data Inspector, see Tolerance Specification.

Example: 0.2

Data Types: double

Specify the name-value pair 'DataType','MustMatch' when you want the comparison to be sensitive to data type mismatches in compared signals. When you specify this name-value pair, the algorithm compares the data types for aligned signals before synchronizing and comparing the signal data.

The Simulink.sdi.compareRuns function does not compare the data types of aligned signals unless you specify this name-value pair. The comparison algorithm can compare signals with different data types.

When signal data types do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to DataTypeMismatch.

When you specify that data types must match and configure the comparison to stop on the first mismatch, a data type mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Specify the name-value pair 'Time','MustMatch' when you want the comparison to be sensitive to mismatches in the time vectors of compared signals. When you specify this name-value pair, the algorithm compares the time vectors of aligned signals before synchronizing and comparing the signal data.

Comparisons are not sensitive to differences in signal time vectors unless you specify this name-value pair. For comparisons that are not sensitive to differences in the time vectors, the comparison algorithm synchronizes the signals prior to the comparison. For more information about how synchronization works, see How the Simulation Data Inspector Compares Data.

When the time vectors for signals do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to TimeMismatch.

When you specify that time vectors must match and configure the comparison to stop on the first mismatch, a time vector mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Specify the name-value pair 'StartStop','MustMatch' when you want the comparison to be sensitive to mismatches in signal start and stop times. When you specify this name-value pair, the algorithm compares the start and stop times for aligned signals before synchronizing and comparing the signal data.

When the start times and stop times do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to StartStopMismatch.

When you specify that start and stop times must match and configure the comparison to stop on the first mismatch, a start or stop time mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Whether the comparison stops without comparing remaining signals on the first detected mismatch, specified as the comma-separated pair consisting of 'StopOnFirstMismatch' and 'Metadata' or 'Any'. A stopped comparison may not compute results for all signals, and can return a mismatched result more quickly.

  • Metadata — A mismatch in metadata for aligned signals causes the comparison to stop. Metadata comparisons happen before comparing signal data.

    The Simulation Data Inspector always aligns signals and compares signal units. When you configure the comparison to stop on the first mismatch, an unaligned signal or mismatched units always causes the comparison to stop. You can specify additional name-value pairs to configure the comparison to check and stop on the first mismatch for additional metadata, such as signal data type, start and stop times, and time vectors.

  • Any — A mismatch in metadata or signal data for aligned signals causes the comparison to stop.

Output Arguments

collapse all

Comparison results, returned as a Simulink.sdi.DiffRunResult object.

Introduced in R2011b