commtest.ErrorRate

(To be removed) Create error rate test console

Compatibility

commtest.ErrorRate will be removed in a future release. Use comm.ErrorRate or bertool instead. For more information, see Compatibility Considerations.

Syntax

h = commtest.ErrorRate
h = commtest.ErrorRate(sys)
h = commtest.ErrorRate(sys,'PropertyName',PropertyValue,...)
h = commtest.ErrorRate('PropertyName',PropertyValue,...)

Description

h = commtest.ErrorRate returns an error rate test console, h. The error rate test console runs simulations of a system under test to obtain error rates.

h = commtest.ErrorRate(sys) returns an error rate test console, error rate test console, h, with each specified property set to the h, with an attached system under test, SYS.

h = commtest.ErrorRate(sys,'PropertyName',PropertyValue,...) returns an error rate test console, h, with an attached system under test, sys. Each specified property, 'PropertyName', is set to the specified value, PropertyValue.

h = commtest.ErrorRate('PropertyName',PropertyValue,...) returns an error rate test console, h, with each specified property 'PropertyName', set to the specified value, PropertyValue.

Properties

The error rate test console object has the properties in the following table. Setting any property resets the object. A property that is irrelevant is one that you can set, but its value does not affect measurements. Similarly, you cannot display irrelevant properties using the disp method. You can write to all properties, except for the ones explicitly noted otherwise.

PropertyDescription
Description'Error Rate Test Console'. Read-only.
SystemUnderTestNameSystem under test name. Read-only.
FrameLength

Specify the length of the transmission frame at each iteration. This property becomes relevant only when the system under test registers a valid test input.

  • If the system under test registers a NumTransmissions test input and calls its getInput method, the error rate test console returns the value stored in FrameLength. Using an internal data source, the system under test uses this value to generate a transmission frame of the specified length.

  • If the system under test registers a DiscreteRandomSource test input and calls its getInput method, the test console generates and returns a frame of symbols. The length of the frame of symbols matches the FrameLength property. This property defaults to 500.

IterationMode

Specify how the object determines simulation points.

  • If set to Combinatorial, the object performs simulations for all possible combinations of registered test parameter sweep values.

  • If set to Indexed, the object performs simulations for all indexed sweep value sets. The ith sweep value set consists of the ith element of every sweep value vector for each registered test parameter. All sweep value vectors must have equal length, except for values that are unit length.

Note that for the following sweep parameter settings:

  • Parameter1 = [a1 a2]

  • Parameter2 = [b1 b2]

  • Parameter3 = [c1]

In Indexed Mode, the test console performs simulations for the following sweep parameter sets:

(a1, b1, c1)

(a2, b2, c1)

In Combinatorial Mode, the test console performs simulations for the following sweep parameter sets:

(a1, b1, c1)

(a1, b2, c1)

(a2, b1, c1)

(a2, b2, c1)

SystemResetMode

Specify the stage of a simulation run at which the system resets.

  • Setting to Reset at new simulation point resets the system under test at the beginning of a new simulation point.

  • Setting to Reset at every iteration resets the system under test at every iteration.

SimulationLimitOption

Specify how to stop the simulation for each sweep parameter point.

  • If set to Number of transmissions the simulation for a sweep parameter point stops when the number of transmissions equals the value for MaxNumTransmissions.

    • Set TransmissionCountTestPoint to the name of the registered test point containing the transmission count you are comparing to MaxNumTransmissions.

  • If set to Number of errors the simulation for a sweep parameter point stops when the number of errors equals the value for MinNumErrors.

    • Set the ErrorCountTestPoint to the name of the registered test point containing the error count you are comparing to the MinNumErrors.

  • Setting to Number of errors or transmissions stops the simulation for a sweep parameter point when meeting one of two conditions.

    • The simulation stops when the number of transmissions equals the value for MaxNumTransmissions.

    • The simulation stops when obtaining the number of errors matching NumErrors.

  • Setting this property to Number of errors and transmissions stops the simulation for a sweep parameter point when meeting the following condition.

    • The simulation stops when the number of transmissions and the number errors have at least reached the values in MinNumTransmissions and MinNumErrors.

Set TransmissionCountTestPoint to the name of the registered test point that contains the transmission count you are comparing to the MaxNumTransmissions property.

To control the simulation length, set ErrorCountTestPoint to the name of the registered test point containing the error count you are comparing to MinNumErrors.

Call the info method of the error rate test console to see the valid registered test point names.

MaxNumTransmissions

Specify the maximum number of transmissions the object counts before stopping the simulation for a sweep parameter point. This property becomes relevant only when SimulationLimitOption is Number of transmissions or Number of errors or transmissions.

  • When setting SimulationLimitOption to Number of transmissions the simulation for each sweep parameter point stops when reaching the number of transmissions MaxNumTransmissions specifies.

  • Setting SimulationLimitOption to Number of errors or transmissions stops the simulation for each sweep parameter point for one of two conditions.

    • The simulation stops when completing the number of transmissions MaxNumTransmissions specifies.

    • The simulation stops when obtaining the number of errors MinNumErrors specifies.

The TransmissionCountTestPoint property supplies the name of a registered test point containing the count transmission type. Calling the info method of the error rate test console displays the valid registered test points. If this property contains registered test points, the test console runs iterations equal to the value for MaxNumTransmissions for each sweep parameter point. If this property has no registered test parameters, the test console runs the number of iterations equal to the value for MaxNumTransmissions and stops. The value defaults to 1000.

MinNumErrors

Specify the minimum number of errors the object counts before stopping the simulation for a sweep parameter point. This property becomes relevant only when setting the SimulationLimitOption to Number of errors or Number of errors or transmissions.

  • When setting SimulationLimitOption to Number of errors the simulation for each parameter point stops when reaching the number of errors you specify for the MinNumErrors property.

  • When setting the SimulationLimitOption property to Number of errors or transmissions the simulation for each sweep parameter point stops for one of two conditions.

    • The simulation stops when reaching the number of errors you specify for the MaxNumTransmissions property.

    • The simulation stops when reaching the number of errors you specify for the MinNumErrors property.

Specify the type of errors the error count uses by setting the ErrorCountTestPoint property to the name of a registered test point containing the count. Call the info method of the error rate test console to see the valid registered test point names. This value defaults to 100.

TransmissionCountTestPoint

Specify and register a test point containing the transmission count that controls the test console simulation stop mechanism. This property becomes relevant only when setting SimulationLimitOption to Number of transmissions, Number of errors or transmissions, or Number of errors and transmissions. In this scenario, if you register a test point, and TransmissionCountTestPoint equals Not set, the value of this property automatically updates to that of the registered test point name. Call the info method to see the valid test point names.

ErrorCountTestPoint

Specify and register the name of a test point containing the error count that controls the simulation stop mechanism. This property is only relevant when setting the SimulationLimitOption property to Number of errors, Number of errors or transmissions, or Number of errors and transmissions. In this scenario, if you register a test point, and ErrorCountTestPoint equals Not set, the value of this property automatically updates to that of the registered test point name. Call the info method to see the valid test point names.

Methods

The error rate test console object has the following methods:

run

Runs a simulation.

Runs the number of error rate simulations you specify for a system under test with a specified set of parameter values. If a Parallel Computing Toolbox™ license is available and a parpool is open, then the object distributes the iterations among the number of workers available.

getResults

Returns the simulation results.

r = getResults(h) returns the simulation results, r, for the test console, h. r is an object of the type you specify using testconsole.Results. It contains the simulation data for all the registered test points and methods to parse the data and plot it.

info

Returns a report of the current test console settings.

info(h) displays the current test console settings, such as registered test parameters and registered test points.

reset

Resets the error rate test console.

reset(h) resets test parameters and test probes and then clears all simulation results of test console, h.

attachSystem

Attaches a system to test console.

attachSystem(ho,sys) attaches a valid user-defined system, sys, to the test console, h.

detachSystem

Detaches the system from the test console.

detachSystem(h) detaches a system from the test console, h. This method also clears the registered test inputs, test parameters, test probes, and test points.

setTestParameterSweepValues

Sets test parameter sweep values.

setTestParameterSweepValues(h,name,sweep) specifies a set of sweep values, 'sweep', for the registered test parameter, 'name', in the test console, h. You only specify sweep values for registered test parameters. sweep must have values within the specified range of the test parameter. It can be a row vector of numeric values, or a cell array of char values. Display the valid ranges using the getTestParameterValidRanges method.

setTestParameterSweepValues(h,name1,sweep1,name2,sweep2...) simultaneously specifies sweep values for multiple registered test parameters.

getTestParameterSweepValues

Returns test parameter sweep values.

getTestParameterSweepValues(h,name) gets the sweep values currently specified for the registered test parameter, name, in the test console, h.

getTestParameterValidRanges

Returns the test parameter valid ranges.

getTestParameterValidRanges(h,name) gets the valid ranges for a registered test parameter, name, in the test console, h.

registerTestPoint

Registers a test point.

registerTestPoint(h, name, actprobe,expprobe) registers a new test point object, name, to the error rate test console, h. The test point must contain a pair of registered test probes, actprobe, and expprobe. actprobe contains actual data, and expprobe contains expected data. The object compares the data from these probes and obtains error rate values. The error rate calculation uses a default error rate calculator function that simply performs one-to-one comparisons of the data vectors available in the probes.

registerTestPoint(h, name, actprobe,expprobe, handle) adds the handle, handle, to a user-defined error calculation function that compares the data in the probes and then obtains error rate results.

The user-defined error calculation function must comply with the following syntax: [ecnt tcnt] = functionName(act, exp, udata) where

  • ecnt output corresponds to the error count

  • tcnt output is the number of transmissions used to obtain the error count

  • act and exp correspond to actual and expected data

The error rate test console sets the inputs to the data available in the pair of test point probes, actprobe, and expprobe.

udata is a data input that the system under test passes to the test console at run time, using the setUserData method. udata contains the data necessary to compute errors, such as delays and data buffers.

The error rate test console passes the data that the system under test logs to the error calculation functions for all the registered test points. Calling the info method returns the names of the registered test points and the error rate calculator functions associated with them. It also returns the names of the registered test probes.

unregisterTestPoint

Unregister a test point.

unregisterTestPoint(h,name) removes the test point, name, from the test console, h.

Examples

collapse all

The commtest.ErrorRate and testconsole.Results object packages will be removed in a future release. They can be used to perform parameter sweeps to analyze communication system performance. This example demonstrates a workflow that uses them and along with recommended alternate workflows.

Multiple Parameter Sweep and Parallel Run using commtest.ErrorRate

Obtain bit error rate and symbol error rate of an M-PSK system for different modulation orders and EbNo values. System under test is commtest.MPSKSystem.

% Create an M-ary PSK system
systemUnderTest = commtest.MPSKSystem;

% Instantiate an Error Rate Test Console and attach the system
errorRateTester = commtest.ErrorRate(systemUnderTest);
Warning: commtest.ErrorRate will be removed in the future. Use comm.ErrorRate or bertool instead. See <a href="matlab:helpview(fullfile(docroot, 'toolbox', 'comm', 'comm.map'), 'REMOVE_commtest_errorrate')">R2019b Communications Toolbox Release Notes</a> for more information.
errorRateTester.SimulationLimitOption = 'Number of errors or transmissions';
errorRateTester.MaxNumTransmissions = 1e5;

% Set sweep values for simulation test parameters
setTestParameterSweepValues(errorRateTester,'M',2.^[1 2 3 4],'EbNo',(-5:10))

% Register a test point
registerTestPoint(errorRateTester,'MPSK_BER','TxInputBits','RxOutputBits')

% Get information about the simulation settings
info(errorRateTester)
Test console name:           commtest.ErrorRate
System under test name:      commtest.MPSKSystem
Available test inputs:       NumTransmissions, RandomIntegerSource
Registered test inputs:      NumTransmissions
Registered test parameters:  EbNo, M
Registered test probes:      RxOutputBits, RxOutputSymbols, TxInputBits, TxInputSymbols
Registered test points:      MPSK_BER
Metric calculator functions: @commtest.ErrorRate.defaultErrorCalculator
Test metrics:                ErrorCount, TransmissionCount, ErrorRate
% Run the M-PSK simulations
run(errorRateTester)
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 12).
12 workers available for parallel computing. Simulations will be distributed among these workers. 
Running simulations...
% Get the results
mpskResults = getResults(errorRateTester);
Warning: testconsole.Results will be removed in the future. See <a href="matlab:helpview(fullfile(docroot, 'toolbox', 'comm', 'comm.map'), 'REMOVE_commtest_errorrate')">R2019b Communications Toolbox Release Notes</a> for more information.
% Get a semi-log scale plot of EbNo versus bit error rate for
% different values of modulation order M
mpskResults.TestParameter2 = 'M';
semilogy(mpskResults,'*-')

Multiple Parameter Sweep and Parallel Run using nested for loops and comm.ErrorRate

Run an error rate simulation over M=2.^(1:4) and EbNo=-5:10. Use comm.ErrorRate to collect both bit error rate (BER) and symbol error rate (SER) data. Run the simulations to collect a minimum of 100 symbol errors or for a maximum of 1e5 symbols.

% Set the M sweep values same as the commtest.ErrorRate object
getTestParameterSweepValues(errorRateTester,'M')
ans = 1×4

     2     4     8    16

MSweep = 2.^[1 2 3 4];
% Set EbNo sweep values same as the commtest.ErrorRate object
getTestParameterSweepValues(errorRateTester,'EbNo')
ans = 1×16

    -5    -4    -3    -2    -1     0     1     2     3     4     5     6     7     8     9    10

EbNoSweep = -5:10;
% Set minumum number of errors same as the commtest.ErrorRate object
errorRateTester.MinNumErrors
ans = 100
minNumErrors = 100;
% Set maximum number of transmissions same as the commtest.ErrorRate
% object. In this example a transmission is a symbol.
errorRateTester.MaxNumTransmissions
ans = 100000
MaxNumTransmissions = 1e5;
% Set frame length same as the commtest.ErrorRate object
errorRateTester.FrameLength
ans = 500
frameLength = 500;
% Find out if there is a parallel pool and how many workers are available
[licensePCT,~] = license('checkout','distrib_computing_toolbox');
if (licensePCT && ~isempty(ver('parallel')))
    p = gcp;
    if isempty(p)
        numWorkers = 1;
    else
        numWorkers = p.NumWorkers
    end
else
    numWorkers = 1;
end
numWorkers = 12
minNumErrorsPerWorker = minNumErrors/numWorkers;
maxNumSymbolsPerWorker = MaxNumTransmissions/numWorkers;

% Store results in an array, where first dimension is M and second
% dimension is EbNo. Initialize the vector with NaN values.
ser = nan(length(MSweep),length(EbNoSweep));
ber = nan(length(MSweep),length(EbNoSweep));

% First sweep is over M (modulation order)
for MIdx = 1:length(MSweep)
    M = MSweep(MIdx);
    bitsPerSymbol = log2(M);
    
    % Second sweep is over EbNo
    for EbNoIdx = 1:length(EbNoSweep)
        EbNo = EbNoSweep(EbNoIdx);
        
        SNR = EbNo+10*log10(bitsPerSymbol);
        
        numSymbolErrors = zeros(numWorkers,1);
        numBitErrors = zeros(numWorkers,1);
        numSymbols = zeros(numWorkers,1);
        
        parfor worker = 1:numWorkers
            symErrRate = comm.ErrorRate;
            bitErrRate = comm.ErrorRate;
            
            while (numSymbolErrors(worker) < minNumErrorsPerWorker) ...
                    || (numSymbols(worker) < maxNumSymbolsPerWorker)
                % Generate frameLength source outputs
                txMsg = randi([0 M-1],frameLength,1);
                
                % Modulate the data
                txOutput = pskmod(txMsg,M,0,'gray');
                % Pass data through an AWGN channel with current SNR value
                chnlOutput  = awgn(txOutput,SNR,'measured',[],'dB');
                % Demodulate the data
                rxOutput = pskdemod(chnlOutput,M,0,'gray');
                
                % Calculate number of symbol errors
                symErrVal = symErrRate(txMsg,rxOutput);
                numSymbolErrors(worker) = symErrVal(2);
                numSymbols(worker) = symErrVal(3);
                
                % Convert symbol streams to bit streams
                bTx = de2bi(txMsg,bitsPerSymbol,'left-msb')';
                bTx = bTx(:);
                bRx = de2bi(rxOutput,bitsPerSymbol,'left-msb')';
                bRx = bRx(:);
                
                % Calculate number of bit errors
                bitErrVal = bitErrRate(bTx,bRx);
                numBitErrors(worker) = bitErrVal(2);
            end
        end
        
        ber(MIdx,EbNoIdx) = sum(numBitErrors)/(sum(numSymbols)*bitsPerSymbol);
        ser(MIdx,EbNoIdx) = sum(numSymbolErrors)/sum(numSymbols);
    end
end
% Plot results
semilogy(EbNoSweep,ber,'*-')
grid on
title('MPSK BER')
xlabel('Eb/No')
ylabel('BER')
legendText = cell(length(MSweep),1);
for p=1:length(MSweep)
    legendText{p} = sprintf('M: %d',MSweep(p));
end
legend(legendText)

Multiple Variable Sweeps using BERTool

BERTool computes the BER as a function of signal-to-noise ratio. It analyzes performance either with Monte-Carlo simulations of MATLAB® functions and Simulink® models or with theoretical closed-form expressions for selected types of communication systems. The bertool function opens the BERTool. Here BERTool is configured to call the simulation defined in the function mpsksim included below.

function [ber,numBits] = mpsksim(EbNo,minNumErrs,maxNumBits)
% Import the Java class for BERTool, so that you will be able to stop the simulation using the "Stop" button on the BERTool.
import com.mathworks.toolbox.comm.BERTool;

frameLength = 500;

M = 16; % Can be 2, 4, 8, 16
bitsPerSymbol = log2(M);

maxNumSymbols = maxNumBits/bitsPerSymbol;

SNR = EbNo + 10*log10(bitsPerSymbol);

% Initialize variables related to exit criteria.
numBitErrors = 0;
numSymbols = 0;

while (numBitErrors < minNumErrs) || (numSymbols < maxNumSymbols)
  
  % Check if the user clicked the Stop button of BERTool.
  if (BERTool.getSimulationStop)
    break;
  end
  
  % Generate frameLength source outputs
  txMsg = randi([0 M-1],frameLength,1);
  numSymbols = numSymbols+frameLength;
  
  % Modulate the data
  txOutput = pskmod(txMsg,M,0,'gray');
  % Pass data through an AWGN channel with current SNR value
  chnlOutput  = awgn(txOutput,SNR,'measured',[],'dB');
  % Demodulate the data
  rxOutput = pskdemod(chnlOutput,M,0,'gray');
  
  % Convert symbol streams to bit streams
  bTx = de2bi(txMsg,bitsPerSymbol,'left-msb')';
  bTx = bTx(:);
  bRx = de2bi(rxOutput,bitsPerSymbol,'left-msb')';
  bRx = bRx(:);
  
  % Calculate number of bit errors
  numBitErrors = numBitErrors+sum(bTx~=bRx);
end

% Assign values to the output variables.
numBits = numSymbols*bitsPerSymbol;
ber = numBitErrors/numBits;

Configure BERTool as follows.

Set M=2 in the mpsksim function and click Run. Set the BER Data Set name to 'M=2'.

Display the BER curve for M=2.

Update the value for M in the mpsksim function, repeating this process for M = 4, 8, 16. You will see results similar to those below in the Bit Error Rate Analysis Tool window and the BER figure.

Parallel SNR Sweep using BERTool

Using parfor, run each simulation point in parallel by configuring your simulation function similar to the mpsksim_parfor function included below. Since parfor cannot work with the Java class for BERTool, you will not be able to stop the simulation using the Stop button.

function [ber,numBits] = mpsksim_parfor(EbNo,minNumErrs,maxNumBits)

% Find out if there is a parallel pool and how many workers are available
if license('test','Distrib_Computing_Toolbox')
  p = gcp;
  if isempty(p)
    numWorkers = 1;
  else
    numWorkers = p.NumWorkers;
  end
else
  numWorkers = 1;
end

M = 2;
bitsPerSymbol = log2(M);

maxNumSymbols = maxNumBits/bitsPerSymbol;

minNumErrorsPerWorker = minNumErrs/numWorkers;
maxNumSymbolsPerWorker = maxNumSymbols/numWorkers;
frameLength = 500;

SNR = EbNo + 10*log10(bitsPerSymbol);

% Initialize variables related to exit criteria.
numBitErrors = zeros(numWorkers,1);
numSymbols = zeros(numWorkers,1);

parfor worker = 1:numWorkers
  while (numBitErrors(worker) < minNumErrorsPerWorker) ...
      || (numSymbols(worker) < maxNumSymbolsPerWorker)
    
    % Generate frameLength source outputs
    txMsg = randi([0 M-1],frameLength,1);
    numSymbols(worker) = numSymbols(worker)+frameLength;
    
    % Modulate the data
    txOutput = pskmod(txMsg, M, 0, 'gray');
    % Pass data through an AWGN channel with current SNR value
    chnlOutput  = awgn(txOutput,SNR,'measured',[],'dB');
    % Demodulate the data
    rxOutput = pskdemod(chnlOutput,M,0,'gray');
    
    % Convert symbol streams to bit streams
    bTx = de2bi(txMsg,bitsPerSymbol,'left-msb')';
    bTx = bTx(:);
    bRx = de2bi(rxOutput,bitsPerSymbol,'left-msb')';
    bRx = bRx(:);
    
    % Calculate number of bit errors
    numBitErrors(worker) = numBitErrors(worker)+sum(bTx~=bRx);
  end
end

% Assign values to the output variables.
ber = sum(numBitErrors)/sum(numSymbols);
numBits = sum(numSymbols)*bitsPerSymbol;

Compatibility Considerations

expand all

Warns starting in R2019b

See Also

Objects

Functions

Introduced in R2009b