How to combine indexing?

Is there an elegant way of combining two indexings to select time points from a long time-series?

I have independent selection criteria :

1. marked events (index vector ind_Ev)
2. marked artifacted periods (ind_Art)

I would like to combine both to select the elements of a time series with 500000 elemtents, i.e. take the ind_Ev and exclude the ind_Art (ind_Ev & ~ind_Art), however, they can point to the same element, in which case it will be excluded.

Thank you so much,

Markus

Answer 1

[Sindar]

There may be a better way, but this should work:

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all ind_Ev elements
idx_Ev_not_Art(ind_Ev) = true;
% remove all ind_Art elements
idx_Ev_not_Art(ind_Art) = false;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

[Markus Gschwind]

Perfect thank you!

In case there are more conditions to combine (e.g. a special episodes to select from the whole data_Ev_not_Art), I could add a new selection AFTER that? However the new data_Ev_not_Art has a different indexing count and indexing doesnt work the same anymore! How should I integrate those?

[Sindar]

It'll require a little logic thinking, but the easiest way is to continue messing with the logical indexing array.

Let's say you go back and realize there's another set of data you want to include, regardless of whether it's labeled as an artifact, ind_always

to affect the original extraction:

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all ind_Ev elements
idx_Ev_not_Art(ind_Ev) = true;
% remove all ind_Art elements
idx_Ev_not_Art(ind_Art) = false;
% add all ind_always elements
idx_Ev_not_Art(ind_always) = true;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

or for a separate set:

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all ind_Ev elements
idx_Ev_not_Art(ind_Ev) = true;
% remove all ind_Art elements
idx_Ev_not_Art(ind_Art) = false;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

% add all ind_always elements
idx_Ev_notA_Al = idx_Ev_not_Art;
idx_Ev_notA_Al(ind_always) = true;
data_Ev_notA_Al = data(data_Ev_notA_Al);

or, maybe you have another condition that must be true, ind_period.

to affect the original extraction:

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all elements from ind_Ev AND ind_period [union for OR]
idx_Ev_not_Art( intesect(ind_Ev,ind_period) ) = true;
% remove all ind_Art elements
idx_Ev_not_Art(ind_Art) = false;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

or for a separate set:

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all ind_Ev elements
idx_Ev_not_Art(ind_Ev) = true;
% remove all ind_Art elements
idx_Ev_not_Art(ind_Art) = false;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

% build logical array from ind_period
idx_period = false(500000,1);
idx_period(ind_period) = true;
% find where both are true
idx_EvANDPer_not_Art = ( idx_Ev_not_Art & idx_period );
% extract from the original data set
data_EvANDPer_not_Art = data(idx_EvANDPer_not_Art);

[Markus Gschwind]

Thank you so much! That is brilliant!

only I guess that there is a typo in line

data_Ev_notA_Al = data(data_Ev_notA_Al);

should actually be

data_Ev_notA_Al = data(idx_Ev_notA_Al);

Thanks again and best wishes!

[Sindar]

yes, you are correct, copy-pasting gets me every time

[Markus Gschwind]

Actually, I needed to transform the indices to logicals in order to make it work.

And as long as I add several true conditions sequentially, the are additive.

So I will need to remove the artifacts at the end, right?

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all ind_Ev elements
idx_Ev_not_Art(logical(ind_Ev)) = true; 
% without the "logical" it gives the error : 
% Subscript indices must either be real positive integers or logicals.

% add all ind_period elements
idx_Ev_not_Art(logical(ind_period)) = true;

% remove all ind_Art elements
idx_Ev_not_Art(logical(ind_Art)) = false;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

[Sindar]

If you want to exclude artifacts, use the blocks "to affect the original extraction"

% set up a logical-indexing vector that selects no elements
idx_Ev_not_Art = false(500000,1);
% add all elements from ind_Ev OR ind_period [intesect for AND]
idx_Ev_not_Art( union(ind_Ev,ind_period) ) = true;
% remove all ind_Art elements
idx_Ev_not_Art(ind_Art) = false;
% extract those elements
data_Ev_not_Art = data(idx_Ev_not_Art);

That error suggests that your ind_Ev is not a set of indices (1,2,40,etc.). More likely than it being a numeric logical vector (0 1 1 0 0 1) is that something else has gone wrong. Can you print the first 10 elements and comment here?

ind_Ev(1:10)

[Markus Gschwind]

>> That error suggests that your ind_Ev is not a set of indices (1,2,40,etc.). More likely than it being a numeric logical vector (0 1 1 0 0 1) is that something else has gone wrong. Can you print the first 10 elements and comment here?

Yes, you were right! It was not [1,3,120,400 etc.] but [1 0 0 1 0 0 0 0 etc.]. When using the first it works without "logical", when using the second, I need "logical".

[Sindar]

Oh, I'd assumed you had indices, not logicals. There are faster ways in that case, since you can do the boolean logic directly:

idx_Ev = logical(ind_eV);
idx_period = logical(ind_period);
idx_Art = logical(ind_Art);

% data from either events, periods, or both. No artifacts
data_EvORPer_not_Art = data( (idx_Ev | idx_period) & ~idx_Art  );
% data from overlap of both events and periods. No artifacts
data_EvANDPer_not_Art = data( idx_Ev & idx_period & ~idx_Art  );
% data from events excluding artifacts, plus periods regardless
data_EvANDPer_not_Art = data( (idx_Ev & ~idx_Art) | idx_period   );