Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Can I extract data from file without importing?

Asked by kschau on 7 May 2013

Hey There!

I believe this is a simple answer, but I just cannot find it online anywhere.

I have a series of .dat files that I am utilizing that are substantial in size, of the order of about 800,000 numbers in a single column format.

What I need to do is consider the first .dat file in the series, extract three specific values within the single column, discard it and then move on to the next .dat file in the series, do the same, and so on...

I have no problem doing this using the importdata(blah blah.dat) method, however each time I do this I import the entire data set to my workspace (I have 600 .dat files in the series) and so it takes forever.

My questions is, can I simply "cherry pick" the three specific data points from the single column without importing the entire data set within the .dat file??

Or any comments as to how to reduce time while mass importing would be great.

Thanks in advance

-Kyle

2 Comments

per isakson on 8 May 2013

How many times in total will you need to access each specific dat-file?

kschau on 9 May 2013

Just once in general. I have 600 dat files per case and about 30 different cases to consider so I'm trying to avoid converting everything to binary.

kschau

Products

No products are associated with this question.

1 Answer

Answer by Image Analyst on 9 May 2013
Accepted answer

You can use fgetl() and then quit reading the file once you recognize that you have the 3 numbers that you need.

2 Comments

kschau on 9 May 2013

I have tried this but the length of each .dat file is so long (700,000 ish data points) that going line by line takes just as long. And the nature of the data is that one value will always be in the last 1/3 of the set. Thanks though.

Image Analyst on 9 May 2013

You can use fseek to home in on the likely starting position. That will speed it up.

Image Analyst

Contact us