Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Creating a big Variable with splitting into numbered .mat-Files

Asked by Trier87 on 12 Jan 2013

Hello Guys, i need to create a "big" Variable but my Memory(4GB) is not big enough so i want now to split it into some numbered .mat-Files. With this code i would have the Variable Buff with 6.765.201 rows and 4 columns:

Buff = zeros(numel(1350:2:1450)*numel(-150:2:-50)*numel(825:2:925)*numel(1225:2:1325),4);
Ctr = 1;
for step1 = 1350:2:1450
    for step2=-150:2:-50
        for step3=825:2:925
            for step4=1225:2:1325    
                          Buff(Ctr, 1) = step1;
                          Buff(Ctr, 2) = step2;
                          Buff(Ctr, 3) = step3;
                          Buff(Ctr, 4) = step4;
                          Ctr = Ctr+1;   
              end
          end
      end
  end

But i want that matlab create Buff with 1 to 500.000 rows and 4 columns into saving_Buff_001.mat then the next 500.000 to 1.000.000 rows and 4 columns into saving_Buff_002.mat and 1.000.000 to 1.500.000 to saving_Buff_003 and so on...

My idea was something like this, but matlab creates first the big variable and then starts splitting

idx = mod(Ctr-1,500000)+1; 
if idx == 500000
       index = Ctr/500000; 
       save(sprintf('saving_Buff_%03d.mat',index),'Buff');
end

Hope somebody can help my, big thanks

2 Comments

per isakson on 12 Jan 2013

memmapfile might be a better alternative

Matt J on 12 Jan 2013

With this code i would have the Variable Buff with 6.765.201 rows and 4 columns.

A matrix of that size is only 206 MB. Should be no problem for a system with 4 GB RAM.

Trier87

Products

No products are associated with this question.

2 Answers

Answer by Matt J on 12 Jan 2013

In addition to memmapfile, there is also matfile, and you could put the whole thing in one file. No need to split into multiple files, as far as I can see, though I would recommend writing column-wise instead of row-wise

 matObj=matfile('myfile','Writable',true);
 for i=1:4
  matObj.Buff(:,i)=...;
 end

0 Comments

Matt J
Answer by Trier87 on 12 Jan 2013

Thx for the informations. This was only a example, my Real Variable has 215.000.0000 rows and 4 columns and thats a very large Variable so i need a pre initialization of the Variable an the Splitting. So any ideas how to solve this Problem. Thanks

2 Comments

José-Luis on 12 Jan 2013

Do you really need so large a variable in memory? Could you not just load what you need? Also, in the future, please don't place comments as answers as it might get confusing for those trying to join the discussion.

Jan Simon on 12 Jan 2013

@Trier87: Please do not post comments as answers.

Trier87

Contact us