Decreasing runtime on blfread or other suggestions for big datasets

I'm currently making a tool to read big CAN Loggings directly from a blf file. Since reading them takes a long time (95s), i tried filtering the Signals and only extracting the neccesary ones, but that did't brought the perfomance gain i was hoping for (down to 85s). Are there any possibilitys to speed up the computations made in blfread or is there an alternative? I already tried parfor with the different CAN Channels but that did't work either.

Thanks in Advance

Answer 1

[Hassaan]

• Use efficient libraries: Seek out the best library for BLF file reading, possibly a specialized one.
• Process in chunks: Read and process the file in smaller segments rather than all at once.
• Aggressive filtering: Apply stricter filters early in the process to minimize data load.
• Optimize file I/O: Use low-level disk reading functions for better performance.
• Explore parallel processing: Beyond parfor, consider separate MATLAB sessions or asynchronous operations for independent data channels.
• Convert data format: If feasible, store data in a format that is faster for MATLAB to read.
• Utilize hardware acceleration: Use GPU processing for computationally intensive tasks.
• MATLAB profiling: Identify slow code sections with the MATLAB profiler to focus your optimization efforts.
• Use compiled code: Implement slow MATLAB code in a compiled language like C++ via MEX files.
• Preprocess data: Convert BLF files to a more accessible format ahead of time.
• Read only what you need: If possible, load only the necessary metadata or headers.
• Vendor assistance: Contact the vendor for specialized advice on their BLF file format.
• Community forums: Leverage knowledge from MATLAB and automotive data analysis communities for additional tips.

------------------------------------------------------------------------------------------------------------------------------------------------

If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.