MATLAB Answers

Conversion of 3d array to 2d array in text file

20 views (last 30 days)
I have temperature data in text file. This text file consist on 3D array. Its complete description is as fellow
  • The first row, longitude, contains 20 values
  • The second row, latitude, contains 18 values
  • The third row, StdPressureLev, contains 24 values
  • From 4th row to onward its description is
Temperature_TqJ_A[x][y],value1, value2, …, valueN
  • X ranges from 0 to 23 —— this is the StdPressureLev index(pressure level index) which are ranging from 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1
  • Y ranges from 0 to 17 —— this is the Latitude index( ranging from 37.5, 36.5, 35.5, 34.5, 33.5, 32.5, 31.5, 30.5, 29.5, 28.5, 27.5, 26.5, 25.5, 24.5, 23.5, 22.5, 21.5)
  • The values for each row range from value1 to value 19 ——— these are the temperature values ordered by Longitude(ranging from 60.5, 61.5, 62.5, 63.5, 64.5, 65.5, 66.5, 67.5, 68.5, 69.5, 70.5, 71.5, 72.5, 73.5, 74.5, 75.5, 76.5, 77.5, 78.5) for a particular pressure and Latitude.Such form of 3D array repeated in a single text file!
My requirements: I want to form 24 different text files from this single text file, each ,based on pressure level(1-24 pressure level). Each text file in have 3 columns(first column consist on latitude, second consist on longitude and third column consist on temperature value at this lat, lon).
My Code With assistance of @Cedric Wannaz,once i prepared
pressures = [1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1] ;
% - Read source file.
fSpec = ['Temperature_TqJ_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
% - Extract/gather all x and gp values.
X = data{1} ;
GP = horzcat( data{3:end} ) ;
% - Build arrays of lon/lat which correspond to GP.
[lon, lat] = meshgrid( 60:79, 22:39 ) ;
% - Iterate through pressure IDs and process.
for x = 0 : 23
% Get relevant block of GP (the one thta corresponds to current p).
gp = GP(X==x,:) ;
% Build 3 columns data array.
data = [reshape(lat.',[],1), reshape(lon.',[],1), reshape(gp.',[],1)].';
% Verbose.
fprintf( 'Export for pressure ID %d -> p=%.1fhpa.\n', x, pressures(x+1) ) ;
% Export.
fId = fopen( sprintf( 'Output_%d.txt', x), 'w' ) ;
fprintf( fId, 'latitude\tlongitude\tGP_value\r\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\r\n', data(:) ) ;
fclose( fId ) ;
end
But now i am getting error, although its converting single text file(name data1)into 24 text files based on pressure level. But its first two column consist on randomly values of Lat, Lon while third column i do not sure either giving lat, or lon. I want to correct this in way, i describe above.
Text file have attached with this query. And a lot of thanks always for this assistance
Regards;

  4 Comments

Show 1 older comment
Walter Roberson
Walter Roberson on 17 Jan 2016
We thought you were going to show us the error message.
Muhammad Usman  Saleem
Muhammad Usman Saleem on 17 Jan 2016
Thanks @walter for your kind contributions..
Output of this code:
Export for pressure ID 0 -> p=1000.0hpa.
Export for pressure ID 1 -> p=925.0hpa.
Export for pressure ID 2 -> p=850.0hpa.
Export for pressure ID 3 -> p=700.0hpa.
Export for pressure ID 4 -> p=600.0hpa.
Export for pressure ID 5 -> p=500.0hpa.
Export for pressure ID 6 -> p=400.0hpa.
Export for pressure ID 7 -> p=300.0hpa.
Export for pressure ID 8 -> p=250.0hpa.
Export for pressure ID 9 -> p=200.0hpa.
Export for pressure ID 10 -> p=150.0hpa.
Export for pressure ID 11 -> p=100.0hpa.
Export for pressure ID 12 -> p=70.0hpa.
Export for pressure ID 13 -> p=50.0hpa.
Export for pressure ID 14 -> p=30.0hpa.
Export for pressure ID 15 -> p=20.0hpa.
Export for pressure ID 16 -> p=15.0hpa.
Export for pressure ID 17 -> p=10.0hpa.
Export for pressure ID 18 -> p=7.0hpa.
Export for pressure ID 19 -> p=5.0hpa.
Export for pressure ID 20 -> p=3.0hpa.
Export for pressure ID 21 -> p=2.0hpa.
Export for pressure ID 22 -> p=1.5hpa.
Export for pressure ID 23 -> p=1.0hpa.
This code is running fine. But its produced text files are wrong in column arrangements.
  1. first two columns are latitudes and longitudes( randomly spaced but not uniformly as i shown above)
  2. third column instead of temperature(Gp_value) is latitude and longitude value
I have attach one ext file data it generated.
Thanks for understanding my problem
Muhammad Usman  Saleem
Muhammad Usman Saleem on 18 Jan 2016
pressures = [1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, ...
100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1] ;
% - Read source file.
fSpec = ['GPHeight_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
These lines are not reading file content? Although its output is cell array of 1*22 cells. But why these cells are all empty?
Want to know that? why it not take data from my text file?
See its output in matlab
These are cell array of empty values.Why?

Sign in to comment.

Accepted Answer

Stephen Cobeldick
Stephen Cobeldick on 18 Jan 2016
Edited: Stephen Cobeldick on 18 Jan 2016
textscan returns empty arrays is because it could not match the provided format string with the actual data in the file. It cannot match the file data because you have changed the file format since Cedric Wannaz wrote that code. When you change the file, then the same formatting command will not work any more.
Cedric Wannaz defined a format string like this:
fSpec = ['Temperature_TqJ_A[%f][%f]', repmat( ', %f', 1, 20 )]
but the first line of your file looks like this:
Longitude, 60.5, 61.5, 62.5, 63.5, 64.5, 65.5, 66.5, 67.5, 68.5, 69.5, 70.5, 71.5, 72.5, 73.5, 74.5, 75.5, 76.5, 77.5, 78.5
This format string does not match the very first line of the file (the string literal is different, there are different number of numeric terms), so it stops looking and returns some empty arrays.
This correctly reads your sample data file:
% Read the data file:
fid = fopen('temp.txt','rt');
Longitude = cell2mat(textscan(fid,['%*s',repmat('%f',1,19)],1,'Delimiter',','));
Latitude = cell2mat(textscan(fid,['%*s',repmat('%f',1,17)],1,'Delimiter',','));
StdPress = cell2mat(textscan(fid,['%*s',repmat('%f',1,24)],1,'Delimiter',','));
C = textscan(fid,['%s',repmat('%f',1,19)],'Delimiter',',','CollectOutput',true);
fclose(fid);
% Parse indices of Pressure and Latitude into numeric (not used):
idx = cellfun(@(s)sscanf(s,'Temperature_TqJ_A[%f][%f]'),C{1},'UniformOutput',false);
idx = [idx{:}];
% Calculate first two columns:
[lat,lon] = meshgrid(Latitude,Longitude);
% Write output files:
for k = 0:23
idy = k==idx(1,:);
%mat = [lat(:),lon(:),reshape(C{2}(idy,:).',[],1)].'; % unsorted
mat = sortrows([lat(:),lon(:),reshape(C{2}(idy,:).',[],1)]).'; % sorted
fid = fopen(sprintf('Output_%02d.txt',k),'wt');
fprintf(fid,'latitude\tlongitude\tGP_value\n');
fprintf(fid,'%.3f\t%.3f\t%.3f\n',mat);
fclose(fid);
end
This code works for your sample data file, here:

  7 Comments

Show 4 older comments
Muhammad Usman  Saleem
Muhammad Usman Saleem on 18 Jan 2016
@Stephen I want to make ascending order of latitude column in text file. Please see my custom sorting of output_oo.text file generated from your kind code as below:
latitude longitude GP_value
21.5 60.5 299.391
21.5 61.5 299.477
21.5 62.5 299.758
21.5 63.5 299.734
21.5 64.5 299.758
21.5 65.5 299.914
21.5 66.5 300.18
21.5 67.5 300.242
21.5 68.5 300.258
21.5 69.5 302.25
21.5 70.5 306.391
21.5 71.5 306.406
21.5 72.5 303.68
21.5 73.5 306.461
21.5 74.5 -9999
21.5 75.5 -9999
21.5 76.5 -9999
21.5 77.5 -9999
21.5 78.5 -9999
22.5 60.5 299.93
22.5 61.5 299.656
22.5 62.5 299.727
22.5 63.5 299.641
22.5 64.5 299.742
22.5 65.5 299.867
22.5 66.5 300.078
22.5 67.5 300.117
22.5 68.5 300.289
22.5 69.5 304.094
22.5 70.5 306.977
22.5 71.5 308.594
22.5 72.5 306.656
22.5 73.5 307.523
22.5 74.5 -9999
22.5 75.5 -9999
22.5 76.5 -9999
22.5 77.5 -9999
22.5 78.5 -9999
23.5 60.5 299.656
23.5 61.5 299.727
23.5 62.5 299.711
23.5 63.5 299.648
23.5 64.5 299.711
23.5 65.5 299.852
23.5 66.5 299.914
23.5 67.5 300.305
23.5 68.5 303.195
23.5 69.5 306.258
23.5 70.5 306.82
23.5 71.5 307.609
23.5 72.5 309.156
23.5 73.5 308.734
23.5 74.5 -9999
23.5 75.5 -9999
23.5 76.5 -9999
23.5 77.5 -9999
23.5 78.5 -9999
24.5 60.5 299.844
24.5 61.5 300
24.5 62.5 299.859
24.5 63.5 299.625
24.5 64.5 299.641
24.5 65.5 299.578
24.5 66.5 300.203
24.5 67.5 303.711
24.5 68.5 305.914
24.5 69.5 307.492
24.5 70.5 307.438
24.5 71.5 308.398
24.5 72.5 310.008
24.5 73.5 -9999
24.5 74.5 -9999
24.5 75.5 -9999
24.5 76.5 -9999
24.5 77.5 -9999
24.5 78.5 -9999
25.5 60.5 303.547
25.5 61.5 306.422
25.5 62.5 304.625
25.5 63.5 303.328
25.5 64.5 302.305
25.5 65.5 302.172
25.5 66.5 303.25
25.5 67.5 306.539
25.5 68.5 307.703
25.5 69.5 308.805
25.5 70.5 310.133
25.5 71.5 309.609
25.5 72.5 307.953
25.5 73.5 -9999
25.5 74.5 -9999
25.5 75.5 -9999
25.5 76.5 -9999
25.5 77.5 -9999
25.5 78.5 -9999
26.5 60.5 -9999
26.5 61.5 -9999
26.5 62.5 308.125
26.5 63.5 -9999
26.5 64.5 -9999
26.5 65.5 -9999
26.5 66.5 307.391
26.5 67.5 307.688
26.5 68.5 308.258
26.5 69.5 309.523
26.5 70.5 309.266
26.5 71.5 -9999
26.5 72.5 -9999
26.5 73.5 -9999
26.5 74.5 -9999
26.5 75.5 -9999
26.5 76.5 -9999
26.5 77.5 -9999
26.5 78.5 -9999
27.5 60.5 -9999
27.5 61.5 -9999
27.5 62.5 -9999
27.5 63.5 -9999
27.5 64.5 -9999
27.5 65.5 -9999
27.5 66.5 -9999
27.5 67.5 306.766
27.5 68.5 306.695
27.5 69.5 308.453
27.5 70.5 307.703
27.5 71.5 307.312
27.5 72.5 -9999
27.5 73.5 -9999
27.5 74.5 -9999
27.5 75.5 -9999
27.5 76.5 -9999
27.5 77.5 -9999
27.5 78.5 -9999
28.5 60.5 -9999
28.5 61.5 -9999
28.5 62.5 -9999
28.5 63.5 -9999
28.5 64.5 -9999
28.5 65.5 -9999
28.5 66.5 -9999
28.5 67.5 307.469
28.5 68.5 306.102
28.5 69.5 306.445
28.5 70.5 306.586
28.5 71.5 306.695
28.5 72.5 306.688
28.5 73.5 -9999
28.5 74.5 -9999
28.5 75.5 -9999
28.5 76.5 -9999
28.5 77.5 -9999
28.5 78.5 -9999
29.5 60.5 -9999
29.5 61.5 -9999
29.5 62.5 -9999
29.5 63.5 -9999
29.5 64.5 -9999
29.5 65.5 -9999
29.5 66.5 -9999
29.5 67.5 305.539
29.5 68.5 -9999
29.5 69.5 304
29.5 70.5 305.016
29.5 71.5 304.875
29.5 72.5 305.062
29.5 73.5 -9999
29.5 74.5 -9999
29.5 75.5 -9999
29.5 76.5 -9999
29.5 77.5 -9999
29.5 78.5 -9999
30.5 60.5 -9999
30.5 61.5 -9999
30.5 62.5 -9999
30.5 63.5 -9999
30.5 64.5 -9999
30.5 65.5 -9999
30.5 66.5 -9999
30.5 67.5 -9999
30.5 68.5 -9999
30.5 69.5 -9999
30.5 70.5 302.688
30.5 71.5 302.188
30.5 72.5 -9999
30.5 73.5 -9999
30.5 74.5 -9999
30.5 75.5 -9999
30.5 76.5 -9999
30.5 77.5 -9999
30.5 78.5 -9999
31.5 60.5 -9999
31.5 61.5 -9999
31.5 62.5 -9999
31.5 63.5 -9999
31.5 64.5 -9999
31.5 65.5 -9999
31.5 66.5 -9999
31.5 67.5 -9999
31.5 68.5 -9999
31.5 69.5 -9999
31.5 70.5 -9999
31.5 71.5 -9999
31.5 72.5 -9999
31.5 73.5 -9999
31.5 74.5 -9999
31.5 75.5 -9999
31.5 76.5 -9999
31.5 77.5 -9999
31.5 78.5 -9999
32.5 60.5 -9999
32.5 61.5 -9999
32.5 62.5 -9999
32.5 63.5 -9999
32.5 64.5 -9999
32.5 65.5 -9999
32.5 66.5 -9999
32.5 67.5 -9999
32.5 68.5 -9999
32.5 69.5 -9999
32.5 70.5 -9999
32.5 71.5 -9999
32.5 72.5 -9999
32.5 73.5 -9999
32.5 74.5 -9999
32.5 75.5 -9999
32.5 76.5 -9999
32.5 77.5 -9999
32.5 78.5 -9999
33.5 60.5 -9999
33.5 61.5 -9999
33.5 62.5 -9999
33.5 63.5 -9999
33.5 64.5 -9999
33.5 65.5 -9999
33.5 66.5 -9999
33.5 67.5 -9999
33.5 68.5 -9999
33.5 69.5 -9999
33.5 70.5 -9999
33.5 71.5 -9999
33.5 72.5 -9999
33.5 73.5 -9999
33.5 74.5 -9999
33.5 75.5 -9999
33.5 76.5 -9999
33.5 77.5 -9999
33.5 78.5 -9999
34.5 60.5 -9999
34.5 61.5 -9999
34.5 62.5 -9999
34.5 63.5 -9999
34.5 64.5 -9999
34.5 65.5 -9999
34.5 66.5 -9999
34.5 67.5 -9999
34.5 68.5 -9999
34.5 69.5 -9999
34.5 70.5 -9999
34.5 71.5 -9999
34.5 72.5 -9999
34.5 73.5 -9999
34.5 74.5 -9999
34.5 75.5 -9999
34.5 76.5 -9999
34.5 77.5 -9999
34.5 78.5 -9999
35.5 60.5 -9999
35.5 61.5 -9999
35.5 62.5 -9999
35.5 63.5 -9999
35.5 64.5 -9999
35.5 65.5 -9999
35.5 66.5 -9999
35.5 67.5 -9999
35.5 68.5 -9999
35.5 69.5 -9999
35.5 70.5 -9999
35.5 71.5 -9999
35.5 72.5 -9999
35.5 73.5 -9999
35.5 74.5 -9999
35.5 75.5 -9999
35.5 76.5 -9999
35.5 77.5 -9999
35.5 78.5 -9999
36.5 60.5 -9999
36.5 61.5 -9999
36.5 62.5 -9999
36.5 63.5 -9999
36.5 64.5 -9999
36.5 65.5 -9999
36.5 66.5 -9999
36.5 67.5 -9999
36.5 68.5 -9999
36.5 69.5 -9999
36.5 70.5 -9999
36.5 71.5 -9999
36.5 72.5 -9999
36.5 73.5 -9999
36.5 74.5 -9999
36.5 75.5 -9999
36.5 76.5 -9999
36.5 77.5 -9999
36.5 78.5 -9999
37.5 60.5 298.062
37.5 61.5 296.523
37.5 62.5 -9999
37.5 63.5 -9999
37.5 64.5 -9999
37.5 65.5 -9999
37.5 66.5 -9999
37.5 67.5 -9999
37.5 68.5 -9999
37.5 69.5 -9999
37.5 70.5 -9999
37.5 71.5 -9999
37.5 72.5 -9999
37.5 73.5 -9999
37.5 74.5 -9999
37.5 75.5 -9999
37.5 76.5 -9999
37.5 77.5 -9999
37.5 78.5 -9999
I want to do such custom sorting to every file
Stephen Cobeldick
Stephen Cobeldick on 18 Jan 2016
This order is exactly what my code gives you.
Read my comment above and include the code that I gave that sorts the files by latitude:
mat = sortrows([lat(:),lon(:),reshape(C{2}(idy,:).',[],1)]).';
This line needs to replace the mat = ... line in my original answer (inside the loop). That is because this line sorts the data, exactly as you requested. That is why I wrote that line, to sort your data. And then tested it to make sure that it works. I also edited my answer to include this sorting, so you could copy the code from my answer and try it.
Here is a sample of how my code sorts your data:
latitude longitude GP_value
21.500 60.500 298.992
21.500 61.500 298.547
21.500 62.500 298.711
21.500 63.500 298.852
21.500 64.500 298.891
21.500 65.500 298.867
21.500 66.500 299.109
21.500 67.500 299.273
21.500 68.500 299.539
21.500 69.500 300.352
21.500 70.500 302.320
21.500 71.500 303.047
21.500 72.500 301.492
21.500 73.500 302.914
21.500 74.500 -9999.000
21.500 75.500 -9999.000
21.500 76.500 -9999.000
21.500 77.500 -9999.000
21.500 78.500 -9999.000
22.500 60.500 300.344
22.500 61.500 298.789
22.500 62.500 298.633
22.500 63.500 298.609
... lots here
37.500 74.500 -9999.000
37.500 75.500 -9999.000
37.500 76.500 -9999.000
37.500 77.500 -9999.000
37.500 78.500 -9999.000
Muhammad Usman  Saleem
Muhammad Usman Saleem on 19 Jan 2016
thank you so much for this kind assistance
Regards;
Muhammad Usman Saleem

Sign in to comment.

More Answers (1)

Thorsten
Thorsten on 18 Jan 2016
Edited: Thorsten on 18 Jan 2016
%%define some constants
filename = 'data1.txt';
NLongitude = 19;
NLatitude = 17;
NStdPressureLev = 24;
%%Process file
fid = fopen(filename);
line = fgets(fid); % first line is empty
% first read in values of Longitude, Latitude and StdPressureLev
line = fgets(fid); % should be Longitude
checktoken = 'Longitude';
assert(strcmp(line(1:length(checktoken)), checktoken), ...
[checktoken ' expected, but not found.'])
Longitude = sscanf(strrep(line(length(checktoken)+1:end), ',', ''), '%f');
assert(numel(Longitude)==NLongitude,...
['Wrong number of values for ' checktoken '.'])
line = fgets(fid); % should be Latitude
checktoken = 'Latitude';
assert(strcmp(line(1:length(checktoken)), checktoken), ...
[checktoken ' expected, but not found.'])
Latitude = sscanf(strrep(line(length(checktoken)+1:end), ',', ''), '%f');
assert(numel(Latitude) == NLatitude,...
['Wrong number of values for ' checktoken '.'])
line = fgets(fid); % should be StdPressureLev
checktoken = 'StdPressureLev';
assert(strcmp(line(1:length(checktoken)), checktoken), ...
[checktoken ' expected, but not found.'])
StdPressureLev = sscanf(strrep(line(length(checktoken)+1:end), ',', ''), '%f');
assert(numel(StdPressureLev) == NStdPressureLev,...
['Wrong number of values for ' checktoken '.'])
% read all numbers into a single 2D matrix of size
% (NStdPressureLev*NLatitude) x (2 + NLongitude)
format = ['Temperature_TqJ_A[%f][%f]', repmat(', %f', 1, NLongitude)];
T = cell2mat(textscan(fid, format, 'CollectOutput', true));
fclose(fid);
%%Reshape into 3D matrix NLatitude x NstdPressureLev x NLongitude
% of Temperature values
T = reshape(T(:, 3:end), [NLatitude, NStdPressureLev, NLongitude]);
%%Write files for each std pressure levels
% create columns for table of (Latitude Longitude Temperature)
[lat, lon] = meshgrid(Latitude, Longitude);
% cycle through all std pressure levels
for i = 1:3% NStdPressureLev
tem = squeeze(T(:,i,:))';
Tab_i = [lat(:) lon(:) tem(:)];
fid = fopen(sprintf('StdPressureLev_%d.txt', i), 'w');
fprintf(fid, '%%Std Pressure Level %f\n', StdPressureLev(i));
fprintf(fid, '%%Latitude Longitude Temperature\r\n');
fprintf(fid, '%.1f %.1f %.3f\n', Tab_i');
fclose(fid);
end

  4 Comments

Show 1 older comment
Thorsten
Thorsten on 5 Apr 2016
Hi Muhammad! Have you run my code on temp.txt? If you have further questions please start a new question. People do not look at answered questions. Best, T
Muhammad Usman  Saleem
Muhammad Usman Saleem on 5 Apr 2016
Dear @Thorsten, you may find it as new question here . Look at this please

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!