containers.Grid
The grid class is a fundamental part of passing test iterations from one testing stage to another in the tico Toolbox. The core implementation of tico is independent of this, however it simplifies handling envelopes for the user.
Conceptually, containers.Grid
relates to n-dimensional matrices like a MATLAB table
relates to a 2-dimensaional matrix. At its core, it captures high-dimensional data and annotates axis dimension names and iterators, like a table
has RowNames
and VariableNames
. A grid has three properties:
-
Data
captures the n-dimensional data matrix -
Iter
captures each dimension iterator values -
Dims
captures the names of each dimension -
User
captures any user-defined data
The key benefit of using the grid class lies in the vast amount of transformations it provides, most of which can be chained:
envelope1 = makegrid(true, {0:1000:10000, 0:50:200}, {'alt_ft', 'v_kts'});
envelope2 = makegrid([true, false], {["up", "dn"]}, {'gear'});
envelope3 = makegrid(true, {
'alt_ft', 0:1000:1000
'v_kts', 0:50:200
})
envelope1.union(envelope2, @or, false)
.filter(true) ...
.map(@produceEvidence) ...
.save("evidence.mat") ...
.map(@testRequirement) ...
.dense(tico.TestStatus.Missing) ...
.collapse("gear") ...
.save("results.mat");
The whole alphabetical list of grid operations is:
The grid class is inspired by Laravel Collections.
The following sections provide an overview. For detailed help, use help containers.Grid/funcname
.
Just like within a table
, you can subreference a grid by index or by label. The return value depends on your use of the operator. To index by numerical indices, use:
grid(2, 2:3, 1) % returns a sub-grid
grid{2, 2:3, 1} % returns a sub-matrix
You can also use the following functional syntax:
grid.slice(2, 2:3, 1) % returns a sub-grid
grid.slice(2, 2:3, 1).Data % returns a matrix
You can also index by key / value pair (the order does not matter):
grid("alt_ft", 1000, "gear", "up", "v_kts", 50:50:100)
You can also select individual iterators, providing a struct array:
iter = struct("alt_ft", 1000, "v_kts", 50, "gear", "up");
grid(iter)
You can select values via logical mask - the output might be a sparse grid:
mask = grid.Data == 42;
grid(mask)
You can select values via function handle evaluation (must return true or false):
grid(@(x) x == 42)
You can access a value and its iterator by linear index using at
:
data = grid.at(42)
[data, iter] = grid.at(42)
All these subreferencing operations are applicable to assignments as well.
Use ()
to insert a sub-grid and {}
to insert data.
grid{2, 2:3, 1} = repmat(true, [1, 2, 1])
grid{"alt_ft", 1000, "gear", "up", "v_kts", 100} = false
grid{struct("alt_ft", 0, "v_kts", 50, "gear", "up")} = true
grid{grid.Data == 42} = 43
grid{@(x) x == 42} = 43
You can reduce the size of the grid by filter()
ing certain values by function handle:
grid.filter(@(x) x == 42)
grid.filter(@not) % to search for 0
The result of the filter()
andoperation may or may not be sparse.
See [Sparse and Dense Grids] for more information.
The follow-up operations you can chain is not affected.
reject()
is the logical opposite of filter()
.
Both command achieve the same:
grid.reject(nan)
grid.reject(@isnan)
You can filter the grid by object property or struct field (all options are equivalent):
grid = makegrid(struct("Success", {true, false, true}), {1:3}, ["A"])
grid(".Success", true)
grid{".Success", true}
grid.slice(".Success", true)
grid.where(Success = true)
grid(@(v) v.Status == true)
The following operations transform the dimensionality of grids: collapse()
, extend()
, retain()
, sort()
, and permute()
.
You can reduce an <math-renderer class="js-inline-math" style="display: inline" data-static-url="https://github.githubassets.com/static" data-run-id="78a22efc66be840f280d828f802dbcda">$n$</math-renderer>-dimensional grid to a <math-renderer class="js-inline-math" style="display: inline" data-static-url="https://github.githubassets.com/static" data-run-id="78a22efc66be840f280d828f802dbcda">$k$</math-renderer>-dimensional grid using either collapse()
(to specify <math-renderer class="js-inline-math" style="display: inline" data-static-url="https://github.githubassets.com/static" data-run-id="78a22efc66be840f280d828f802dbcda">$n-k$</math-renderer> dimensions to remove) or retain()
(to specify all other <math-renderer class="js-inline-math" style="display: inline" data-static-url="https://github.githubassets.com/static" data-run-id="78a22efc66be840f280d828f802dbcda">$k$</math-renderer> dimensions). The first argument is the (list of) dimension(s) to collapse or retain, the second argument a reduction function that produces a scalar result from arrays or matrices of data:
results.collapse("gear", @join)
envelope.collapse("v_kts", @or)
results.retain(["alt_ft", "v_kts"], @join)
Using extend()
you can instead add new dimensions. Data from the previous sub-grid will be repeated along the axis, increasing the size of the hyperspace:
envelope.extend("gear", ["up", "dn"])
The function sort()
will reorder and permute iterators and dimension names to be alpanumerically increasing. Sorting grids does not modify the underlying data, and grids remain compatible for iterations.
sorted = grid.sort()
assert(iscompatible(sorted, grid))
You can also manually permute()
dimensions (though, not iterators):
grid = grid.permute(["v_kts", "gear", "alt_ft"])
The following operations transform the content of grids: map()
, except()
, only()
, and pluck()
.
You will use map()
to apply M:N mapping functions to grid data or iterators and capture the result in another grid, or multiple output grids:
grid = envelope.map(@produceEvidence)
grid = map(grid1, grid2, @evidenceWithTwoInputs)
[grid1, grid2] = envelope.map(@evidenceWithTwoOutputs)
[grid1, grid2] = map(grid3, grid4, @fcnWithTwoInputsAndOutputs)
map()
will automatically run in parallel if the grid was distributed()
before.
For any mapping, grid dimensions do not have to be identical. It is sufficient that both grids satisfy iscompatible()
.
When working with structure or object grids (i.e. grids containing struct
or objects as Data
), you can use except()
, only()
and pluck()
to work with fields and properties:
grid.except("result") % will remove a field
grid.only(["A", "B"]) % will keep two fields only
grid.pluck("Status") % extracts property "Status" from object matrix
The commands above are conceptually identical to, but faster than:
grid.map(@(s) rmfield(s, "result"))
grid.map(@(s) struct("A", s.A, "B", s.B))
grid.map(@(s) s.Status)
This section contains a list of operations that might be useful utilities:
-
assign()
replaces the whole grid (useful for method chaining). -
collect()
builds 1-dimensional grids from arrays. -
data()
returns the data of a grid. -
data(data)
writes data to the grid. -
each()
is likemap()
, but the mapping function has no outputs. -
find()
is likefilter()
, but returns the result data instead. -
first(@fcn)
returns a single element, if found based on function@fcn
. -
isempty()
returns true if and only if the grid contains no data. -
iter()
returns the iterator values of a grid as a struct array. -
iter(dim = iter)
writes a specific iterator to the grid. -
iter(iter)
writes iterator values to the grid. -
last(@fcn)
returns a single element, if found based on function@fcn
. -
loadgrid()
is the logical opposite tosavegrid()
. -
makegrid()
is a functional alias for the constructorcontainers.Grid
. -
pipe()
is for functional programming, to provide your own operation. -
save()
is for saving to mat file (you can continue chaining operations after this). -
savegrid()
is the functional counterpart tosave()
. -
struct()
removes the class interface from the grid data, so you can serialize it to a MAT file more easily. -
user()
returns the user data of a grid. -
user(key = value)
writes user data to the grid, assuminggrid.User
is a struct. -
user(user)
writes user data to the grid.
For more information, run help containers.Grid/funcname
.
The following operations provide insight into contents and structure of grids:
iscompatible()
will return true if and only if multiple grids have the same dimension names and iterator values. The order of values and dimensions does not matter:
tf = iscompatible(grid1, grid2)
contains()
will return true if and only if the queried value is contained in the grid data, or the function handle returns true for any grid point:
grid.contains(42)
grid.contains(@(answer) answer == 42)
every()
is similar to contains()
, but requires all grid points to contain the given value or fulfill the given function handle:
grid.every(42)
grid.every(@(answer) answer == 42)
size()
will return the number of iterator values in each dimension. This is slightly different from size(grid.Data)
, since size(grid)
does not truncate trailing ones:
size(grid) % might return [10, 5, 2, 1]
size(grid.Data) % might return [10, 5, 2]
ndims()
returns the number of named dimensions -- again this is different from ndims(grid.Data)
:
ndims(grid) % might return 4
ndims(grid.Data) % might return 3
Our grid joins are like a high-dimensional variation of relational database table joins. There are inner and outer joins. An inner join will result in the intersection of two rectangular hyperspaces. An outer join will result in the union space of two rectangular hyperspaces.
An example for an inner join:
>> grid_1 = makegrid(rand(3,3,3), {1:3, 1:3, 1:3}, ["a", "b", "c"]);
>> grid_2 = makegrid(rand(3,3,3), {1:3, 2:4, 1:3}, ["b", "c", "d"]);
>> joined = intersect(grid_1, grid_2, @mean)
joined =
2-dimensional Grid containing double with iterators:
b: [1, 2, 3]
c: [2, 3]
6 iterations total
The same example for an outer join:
>> joined = union(grid_1, grid_2, @mean, nan)
joined =
4-dimensional Grid containing double with iterators:
a: [1, 2, 3]
b: [1, 2, 3]
c: [1, 2, 3, 4]
d: [1, 2, 3]
36 iterations total
Either way, the first two arguments to intersect()
and union()
are the two grids to be joined. The third argument must be a pairwise function to produce a scalar value from each overlap. For outer joins, you also can specify missing values for the left and right grids, where they did not extend before.
Some operations like filter()
and reject()
can destroy the regularity of a grid. The result will be a sparse grid, or in other words, a point cloud. A sparse grid has no dimension names in .Dims
and only a single entry in .Iter
, which is a struct array with any possible iterations.
Sparse grids may also consume less memory if the sparsity is very high (e.g. 99%).
You can check, whether a grid issparse()
at any time.
To convert (dense) grids into sparse grids manually and vice versa, use the functions sparse()
and dense()
:
>> envelope = tico('polarion', 'grid', 'PAR-108').filter()
envelope =
22-dimensional sparse Grid containing logical with iterators:
...
1307 iterations total
>> envelope.dense(false) % must specify neutral element
envelope =
22-dimensional Grid containing logical with iterators:
...
4704 iterations total (1307 containing <true>)
You can use [Grid Content Transformations] on sparse grids as usual.
To parallelise work across nodes, you can distribute content of a grid to be processed on multiple workers. There are two aspects to this.
First option is to partition()
a grid into a number of subspaces equal to the number of workers. After processing, reassemble the hyperspace using union()
:
grids = cell(1, 4)
[grids{:}] = grid.partition()
parfor k = 1:numel(grids)
grids{k} = grids{k}.map(@workInParallel)
end
grid = union(grids{1}, grids(2:end))
Second option is to make a grid distributed()
, which transports the contents to each worker equally, then perform the mapping in parallel, then gather()
the distributed results back on the host:
grid = grid.distributed().map(@workInParallel).gather()
You can also distribute a grid on construction, which is similar to matrices:
grid = makegrid(data, iter, dims, 'distributed')
Using the partitioning approach, you have more fine control, which iterations are included with each subspace. However, this requires you to specify a mapping function that returns bin values 1-4 for each grid element. Read more in help containers.Grid/partition
.
This project is maintained by Florian Schwaiger.
Cite As
Schwaiger, Florian, et al. “Tico – a Toolbox to Author and Execute Large Parametrizable Test Suites in MATLAB.” AIAA SCITECH 2023 Forum, American Institute of Aeronautics and Astronautics, 2023, doi:10.2514/6.2023-1123.
MATLAB Release Compatibility
Platform Compatibility
Windows macOS LinuxTags
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!Discover Live Editor
Create scripts with code, output, and formatted text in a single executable document.
code
code/+containers/@Grid
code/+containers/@Grid/private
resources/scripts
test/perf
test/unit
test/data/iter
Version | Published | Release Notes | |
---|---|---|---|
2.3.0 | See release notes for this release on GitHub: https://github.com/fschwaiger/matlab-grid/releases/tag/v2.3.0 |
||
2.2.0 | See release notes for this release on GitHub: https://github.com/fschwaiger/matlab-grid/releases/tag/v2.2.0 |
||
2.1.0.0 | See release notes for this release on GitHub: https://github.com/fschwaiger/matlab-grid/releases/tag/v2.1.0 |
||
2.0.2.0 | See release notes for this release on GitHub: https://github.com/fschwaiger/matlab-grid/releases/tag/v2.0.2 |
||
2.0.1 | |||
2.0.0.0 | See release notes for this release on GitHub: https://github.com/fschwaiger/matlab-grid/releases/tag/v2.0.0 |