Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Why is export(dataset, 'file', ...) so slow?

Subject: Why is export(dataset, 'file', ...) so slow?

From: kj

Date: 14 Jun, 2013 20:34:04

Message: 1 of 4







When I profiled a script I've been working on I was shocked to see
that a line of the form

    export(dset, 'file', outpath);

(where dset is a dataset object) takes a whopping 15 *minutes* to
execute. For writing out a file that's well under 20 MB? Really?

Admittedly, the dataset being saved is largeish (130K rows, 20
columns), but Python saves a comparable Pandas dataframe in a just
a few seconds.

What's going on here? Is there any way to speed this up?

Thanks in advance!

kj

Subject: Why is export(dataset, 'file', ...) so slow?

From: dpb

Date: 15 Jun, 2013 15:20:36

Message: 2 of 4

On 6/14/2013 3:34 PM, kj wrote:
> When I profiled a script I've been working on I was shocked to see
> that a line of the form
>
> export(dset, 'file', outpath);
>
> (where dset is a dataset object) takes a whopping 15 *minutes* to
> execute. For writing out a file that's well under 20 MB? Really?
>
> Admittedly, the dataset being saved is largeish (130K rows, 20
> columns), but Python saves a comparable Pandas dataframe in a just
> a few seconds.
>
> What's going on here? Is there any way to speed this up?
>
> Thanks in advance!
>

No ideas, but...seems like good query to "official" TMW support as if
not bug at least big performance hit.

--

Subject: Why is export(dataset, 'file', ...) so slow?

From: james bejon

Date: 15 Jun, 2013 16:13:10

Message: 3 of 4

Hmmm...I've had similar problems myself.

One thing to check may be that your computer's virus protection isn't getting in the way. (You should be able to monitor it while the write is going on.) I found on my computer that with every line of data I was writing, my virus protection was doing something. In the end (since it was a work computer and I couldn't switch the protection off), I coded up a routine that converted the whole dataset into a big character string and then dumped it onto the disk in one go.

Subject: Why is export(dataset, 'file', ...) so slow?

From: kj

Date: 17 Jun, 2013 14:26:11

Message: 4 of 4

In <kpi3ql$r4$1@newscl01ah.mathworks.com> "james bejon" <jamesbejon@yahoo.co.uk> writes:

>Hmmm...I've had similar problems myself.

>One thing to check may be that your computer's virus protection isn't getting in the way. (You should be able to monitor it while the write is going on.) I found on my computer that with every line of data I was writing, my virus protection was doing something. In the end (since it was a work computer and I couldn't switch the protection off), I coded up a routine that converted the whole dataset into a big character string and then dumped it onto the disk in one go.


Wow, many thanks for that tip!

kj

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us