Profile and debug load function

9 views (last 30 days)
Jan Kappen
Jan Kappen on 18 Feb 2024
Commented: Jan Kappen on 27 Feb 2024
I have a .mat file containing an object of a custom Matlab class that mainly consists of a big data table of type table, stored in one of the class' properties. The table contains ~20k rows and contains datetime, double, categorical arrays, and objects of class idnlarx, i.e. estimated nonlinear nlarx models.
The mat file is 520 MB in size (mat file v7 with enabled compression, v7.3 makes the size 10x) and takes between 240 and 290 seconds to load.
I'm trying to speed up the loading process which is clearly CPU bound. I think the problem is the idnlarx model which utilizes most of the space (> 1 GB when loaded to RAM and instantiated).
And I'd like to understand what takes so much time to load, but the profiler only shows this:
...
That means, 94% are self-time of the load function.
How do I find out what takes so long?
Thanks,
Jan

Answers (1)

Matt J
Matt J on 18 Feb 2024
Edited: Matt J on 18 Feb 2024
You could pursue your hypothesis by extracting the table from an object, then saving the idnlarx column to its own .mat file. Then, you could check how much time it takes to load back in. I suspect your hypothesis is correct, however, because I routinely load .mat files that are ~500 MB and, while it takes some time, it is nowhere near a minute long, let alone 4 minutes.
  8 Comments
Matt J
Matt J on 26 Feb 2024
Edited: Matt J on 26 Feb 2024
According to your original post, the idnlarx objects are members of your custom class. It is that class whose saveobj/loadobj method I am suggesting you customize, not the internal loadobj method of idnlarx. Transform the idnlarx members into something simpler and more compact, something which can be used to rebuild the original idnlarx memebrs later.
Jan Kappen
Jan Kappen on 27 Feb 2024
Understood. And that's where I struggle since I can't access the internals of idnlarx to rebuild the original idnlarx objects later. At least its constructor does not let me set all class properties to rebuild and I can't set them from "outside", because they are read only.
But I'll investigate, maybe there are other ways to serialize/deserialize.

Sign in to comment.

Products


Release

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!