Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
dealing with missing rows in the data (not NaN's)

Subject: dealing with missing rows in the data (not NaN's)

From: bekoi

Date: 1 Nov, 2010 17:08:04

Message: 1 of 4

Hi,
I'm still very "primitive" in matlab :D and I'm having a problem with my data analysis.
I have measurements every 10 minutes from 05.01.2010 00:00:00, until 10.06.2010 23:59:00.
So, the minutes column of my data looks like:
0
10
20
30
40
50
0
10 ....

My problem is that there are some rows missing in the data e.g: 0, 10, 30,40,50. And some times there are entire days missing or incomplete days. :(
So, my question is: how do I account for all the missing rows and replace them with rows of NaN's? I want to have the same dimensions I would have if all the minutes were there (22608 rows). At the moment I have 21659 rows.

Thanks for your help!

Subject: dealing with missing rows in the data (not NaN's)

From: James Tursa

Date: 1 Nov, 2010 17:26:04

Message: 2 of 4

"bekoi " <e5vnogue@hotmail.com> wrote in message <iams5k$all$1@fred.mathworks.com>...
> Hi,
> I'm still very "primitive" in matlab :D and I'm having a problem with my data analysis.
> I have measurements every 10 minutes from 05.01.2010 00:00:00, until 10.06.2010 23:59:00.
> So, the minutes column of my data looks like:
> 0
> 10
> 20
> 30
> 40
> 50
> 0
> 10 ....
>
> My problem is that there are some rows missing in the data e.g: 0, 10, 30,40,50. And some times there are entire days missing or incomplete days. :(
> So, my question is: how do I account for all the missing rows and replace them with rows of NaN's? I want to have the same dimensions I would have if all the minutes were there (22608 rows). At the moment I have 21659 rows.
>
> Thanks for your help!

Is there a way to create a 1-to-1 function to map the data you have into the larger matrix you want by using all of your time data? i.e., what does a typical row of your existing data look like, including all of the year,day,hour,minute,etc columns? The idea would be to preallocate a NaN array of the correct size that you want, fill in all of the date & time stuff in this array, then copy over your data based on the 1-to-1 function.

James Tursa

Subject: dealing with missing rows in the data (not NaN's)

From: dpb

Date: 1 Nov, 2010 17:22:55

Message: 3 of 4

bekoi wrote:
> Hi,
> I'm still very "primitive" in matlab :D and I'm having a problem with my
> data analysis.
> I have measurements every 10 minutes from 05.01.2010 00:00:00, until
> 10.06.2010 23:59:00.
...
> My problem is that there are some rows missing in the data e.g: 0, 10,
> 30,40,50. And some times there are entire days missing or incomplete
> days. :(
> So, my question is: how do I account for all the missing rows and
> replace them with rows of NaN's? I want to have the same dimensions I
> would have if all the minutes were there (22608 rows). At the moment I
> have 21659 rows.
>
> Thanks for your help!

Convert to serial times and then use

diff()

to find locations of differences greater than 10 min --> 10/(24*60)

--

Subject: dealing with missing rows in the data (not NaN's)

From: Miroslav Balda

Date: 1 Nov, 2010 17:40:08

Message: 4 of 4

"bekoi " <e5vnogue@hotmail.com> wrote in message <iams5k$all$1@fred.mathworks.com>...
> Hi,
> I'm still very "primitive" in matlab :D and I'm having a problem with my data analysis.
> I have measurements every 10 minutes from 05.01.2010 00:00:00, until 10.06.2010 23:59:00.
> So, the minutes column of my data looks like:
> 0
> 10
> 20
> 30
> 40
> 50
> 0
> 10 ....
>
> My problem is that there are some rows missing in the data e.g: 0, 10, 30,40,50. And some times there are entire days missing or incomplete days. :(
> So, my question is: how do I account for all the missing rows and replace them with rows of NaN's? I want to have the same dimensions I would have if all the minutes were there (22608 rows). At the moment I have 21659 rows.
>
> Thanks for your help!

Hi,
It is clear that the row starting with zero time anounces a new day. However, how youmark the empty day or days? If there is no additional information, you may not fill missing days with anything (NaNs).
Mira

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us