Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
simply filtering data

Subject: simply filtering data

From: Dave

Date: 28 Feb, 2009 14:15:04

Message: 1 of 6

hello, and thanks for you help in advance.

I have a large data set of over 17000 rows. i would like to filter this data set.

each column represents a reading from an analyser, and each reading has a data quality flag. I would like to know how to make a new data set based on multiple flags. The flags ate 0 (highest quality) to 4 (lowest quality). what I would like to be left with is a data set which is in the same order as the original but with the poor quality data removed. I only want the "0" flagged data, but knowing how to filter the data between flags and for multiple flag may help me understand the process.

regards

dave

Subject: simply filtering data

From: Dave

Date: 1 Mar, 2009 11:38:01

Message: 2 of 6

> I have a large data set of over 17000 rows. i would like to filter this data set.
>
> each column represents a reading from an analyser, and each reading has a data quality flag. I would like to know how to make a new data set based on multiple flags. The flags ate 0 (highest quality) to 4 (lowest quality). what I would like to be left with is a data set which is in the same order as the original but with the poor quality data removed. I only want the "0" flagged data, but knowing how to filter the data between flags and for multiple flag may help me understand the process.
>
> regards
>
> dave

Someone please help me with this, I've been trying to do this for two days and am still way behind square one. to clarify the first column is the data and the next is the flag, this pattern carries on for about 70 columns, however, i will only be using columns of the data.

thanks

Subject: simply filtering data

From: Matt

Date: 1 Mar, 2009 15:32:01

Message: 3 of 6

"Dave " <yogi_cave@yahoo.com> wrote in message <gobgt8$k2l$1@fred.mathworks.com>...
> hello, and thanks for you help in advance.
>
> I have a large data set of over 17000 rows. i would like to filter this data set.
>
> each column represents a reading from an analyser, and each reading has a data quality flag. I would like to know how to make a new data set based on multiple flags. The flags ate 0 (highest quality) to 4 (lowest quality). what I would like to be left with is a data set which is in the same order as the original but with the poor quality data removed. I only want the "0" flagged data, but knowing how to filter the data between flags and for multiple flag may help me understand the process.
>
> regards


Let D be your data set with 17000 rows and

newD=D(:,flags==0);

Subject: simply filtering data

From: Dave

Date: 1 Mar, 2009 18:08:01

Message: 4 of 6

thanks... I was obviously making it much more complicated than it actually is.

However, when I apply that piece of code i only get the first column of my original data. I really need all of the rows of data.

Subject: simply filtering data

From: Jos

Date: 1 Mar, 2009 18:28:02

Message: 5 of 6

"Dave " <yogi_cave@yahoo.com> wrote in message <goeiu1$in4$1@fred.mathworks.com>...
> thanks... I was obviously making it much more complicated than it actually is.
>
> However, when I apply that piece of code i only get the first column of my original data. I really need all of the rows of data.

Learn from this example

% some data
D = [1 2 3 ; 1 3 4 ; 2 5 4 ; 1 4 5 ; 3 5 6 ; 1 6 3 ; 2 5 2] ;

% compare all rows of first column to 1
q = D(:,1) == 1

% select all columns of rows where q is true
newD = D(q,:)

hth
Jos

Subject: simply filtering data

From: Dave

Date: 1 Mar, 2009 22:35:02

Message: 6 of 6

thanks! I'll give it ago tomorrow.

thank you both for helping a beginner like me, please keep up the good work!

dave

"Jos " <#10584@fileexchange.com> wrote in message <goek3i$5n2$1@fred.mathworks.com>...
> "Dave " <yogi_cave@yahoo.com> wrote in message <goeiu1$in4$1@fred.mathworks.com>...
> > thanks... I was obviously making it much more complicated than it actually is.
> >
> > However, when I apply that piece of code i only get the first column of my original data. I really need all of the rows of data.
>
> Learn from this example
>
> % some data
> D = [1 2 3 ; 1 3 4 ; 2 5 4 ; 1 4 5 ; 3 5 6 ; 1 6 3 ; 2 5 2] ;
>
> % compare all rows of first column to 1
> q = D(:,1) == 1
>
> % select all columns of rows where q is true
> newD = D(q,:)
>
> hth
> Jos

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us