Large dataset in loop and save results in a variable

I have a large dataset. where I have 25million rows, I will be using 1-250,000, 1-500,000 and so on. I wrote the code, but the vaiable in the dielectricconstant is not saving properly. I have the below code, I think its just need a little tweak.

clc
clear all

T= 298.5; % temperature
Kb= 1.38065E-23; %Boltzman's Constant
e0= 8.85418*10^(-12); % permittivity in vacuum
e=(1.603*(10^-19))^2; % charge of electron
vol = 35.627171   *   35.627171  *  35.627171*10^(-30);        
M = importdata('Trial-1-SPCE.dipole.Pw');
n = length(M);   % number of rows
%A = M(1:2:end,:);  % odd matrix--Dipole moment matrix

for n = 250000:250000:25000001
    DielectricConstant = [];
    for i = 1:100
        A = M(1:n,:);
        P = [A(:,3)+ A(:,4)+ A(:,5)];
        Q= (mean(P))^2; %ensemble average square
        P1= [A(:,3).^2+ A(:,4).^2+ A(:,5).^2];
        Q1= (mean(P1)); %mean of square of dipole moment
        MM= Q1-Q;
        scale = e*(10^(-20))/(3*e0*Kb*T*vol);
        DielectricConstant(i)= MM*scale;
    end
end

Answer 1

[Image Analyst]

Unfortunately you forgot to attach the data (Maybe it was too big > 5 MB). But try this:

clc
clear all
T= 298.5; % temperature
Kb= 1.38065E-23; %Boltzman's Constant
e0= 8.85418*10^(-12); % permittivity in vacuum
e=(1.603*(10^-19))^2; % charge of electron
vol = 35.627171   *   35.627171  *  35.627171*10^(-30);
M = importdata('Trial-1-SPCE.dipole.Pw');
n = length(M);   % number of rows
%A = M(1:2:end,:);  % odd matrix--Dipole moment matrix
all_n = 250000:250000:25000001
N = numel(all_n)
DielectricConstant = zeros(N, 100);
for k = 1 : numel(all_n)
    n = all_n(k);
    fprintf('On iteration %d of %d.\n', k, numel(all_n))
    DielectricConstant = [];
    for i = 1:100
        A = M(1:n,:);
        P = [A(:,3)+ A(:,4)+ A(:,5)];
        Q= (mean(P))^2; %ensemble average square
        P1= [A(:,3).^2+ A(:,4).^2+ A(:,5).^2];
        Q1= (mean(P1)); %mean of square of dipole moment
        MM= Q1-Q;
        scale = e*(10^(-20))/(3*e0*Kb*T*vol);
        DielectricConstant(k, i)= MM*scale;
    end
end
imshow(DielectricConstant, []);

If you have any more questions, then attach your data and code to read it in with the paperclip icon after you read this:

TUTORIAL: How to ask a question (on Answers) and get a fast answer

[Avik]

Thanks a lot for taking time to reply. This works for only the first 250,000 its doesn't update the matrix from [1:250,000], [1:500,000], [1:750000].....[1:25,000,000]. Sso the final matrix looks like this,

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

69.3336360696453 69.3336360696453 69.3336360696453 69.3336360696453 69.3336360696453 .

This 69.336.. value should change if it uses more and more data. The data file is large. More than 1GB. I am attching a small sample,

1 0.00400000000000000 -6.62582000000000 15.6498000000000 -6.95121000000000

2 0.00800000000000000 -7.59239000000000 15.3381000000000 -7.37722000000000

3 0.0120000000000000 -8.60858000000000 14.7438000000000 -7.06737000000000

4 0.0160000000000000 -9.53393000000000 13.8257000000000 -6.09994000000000

5 0.0200000000000000 -10.2651000000000 12.7887000000000 -4.72368000000000

6 0.0240000000000000 -10.7288000000000 12.0582000000000 -3.38682000000000

7 0.0280000000000000 -10.8928000000000 12.0443000000000 -2.56919000000000

8 0.0320000000000000 -10.8640000000000 12.9211000000000 -2.45252000000000

9 0.0360000000000000 -10.9157000000000 14.6122000000000 -2.79270000000000

10 0.0400000000000000 -11.2973000000000 16.7707000000000 -3.14338000000000

11 0.0440000000000000 -12.0715000000000 18.7813000000000 -3.17332000000000

12 0.0480000000000000 -13.0743000000000 20.0348000000000 -2.93442000000000

13 0.0520000000000000 -14.0118000000000 20.1603000000000 -2.74425000000000

14 0.0560000000000000 -14.6112000000000 19.0260000000000 -2.75518000000000

15 0.0600000000000000 -14.7091000000000 16.7647000000000 -2.85866000000000

16 0.0640000000000000 -14.2563000000000 13.8446000000000 -2.94562000000000

17 0.0680000000000000 -13.3389000000000 10.9588000000000 -3.09527000000000

18 0.0720000000000000 -12.3045000000000 8.80008000000000 -3.49126000000000

19 0.0760000000000000 -11.6915000000000 7.91536000000000 -4.20268000000000

20 0.0800000000000000 -11.8874000000000 8.58483000000000 -4.98463000000000