Conversion of 3d array to 2d array in text file

3 views (last 30 days)
I have temperature data in text file. This text file consist on 3D array. Its complete description is as fellow
  • The first row, longitude, contains 20 values
  • The second row, latitude, contains 18 values
  • The third row, StdPressureLev, contains 24 values
  • From 4th row to onward its description is
Temperature_TqJ_A[x][y],value1, value2, …, valueN
  • X ranges from 0 to 23 —— this is the StdPressureLev index(pressure level index) which are ranging from 1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1
  • Y ranges from 0 to 17 —— this is the Latitude index( ranging from 37.5, 36.5, 35.5, 34.5, 33.5, 32.5, 31.5, 30.5, 29.5, 28.5, 27.5, 26.5, 25.5, 24.5, 23.5, 22.5, 21.5)
  • The values for each row range from value1 to value 19 ——— these are the temperature values ordered by Longitude(ranging from 60.5, 61.5, 62.5, 63.5, 64.5, 65.5, 66.5, 67.5, 68.5, 69.5, 70.5, 71.5, 72.5, 73.5, 74.5, 75.5, 76.5, 77.5, 78.5) for a particular pressure and Latitude.Such form of 3D array repeated in a single text file!
My requirements: I want to form 24 different text files from this single text file, each ,based on pressure level(1-24 pressure level). Each text file in have 3 columns(first column consist on latitude, second consist on longitude and third column consist on temperature value at this lat, lon).
My Code With assistance of @Cedric Wannaz,once i prepared
pressures = [1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1] ;
% - Read source file.
fSpec = ['Temperature_TqJ_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
% - Extract/gather all x and gp values.
X = data{1} ;
GP = horzcat( data{3:end} ) ;
% - Build arrays of lon/lat which correspond to GP.
[lon, lat] = meshgrid( 60:79, 22:39 ) ;
% - Iterate through pressure IDs and process.
for x = 0 : 23
% Get relevant block of GP (the one thta corresponds to current p).
gp = GP(X==x,:) ;
% Build 3 columns data array.
data = [reshape(lat.',[],1), reshape(lon.',[],1), reshape(gp.',[],1)].';
% Verbose.
fprintf( 'Export for pressure ID %d -> p=%.1fhpa.\n', x, pressures(x+1) ) ;
% Export.
fId = fopen( sprintf( 'Output_%d.txt', x), 'w' ) ;
fprintf( fId, 'latitude\tlongitude\tGP_value\r\n' ) ;
fprintf( fId, '%.3f\t%.3f\t%.3f\r\n', data(:) ) ;
fclose( fId ) ;
end
But now i am getting error, although its converting single text file(name data1)into 24 text files based on pressure level. But its first two column consist on randomly values of Lat, Lon while third column i do not sure either giving lat, or lon. I want to correct this in way, i describe above.
Text file have attached with this query. And a lot of thanks always for this assistance
Regards;
  4 Comments
Muhammad Usman Saleem
Muhammad Usman Saleem on 17 Jan 2016
Thanks @walter for your kind contributions..
Output of this code:
Export for pressure ID 0 -> p=1000.0hpa.
Export for pressure ID 1 -> p=925.0hpa.
Export for pressure ID 2 -> p=850.0hpa.
Export for pressure ID 3 -> p=700.0hpa.
Export for pressure ID 4 -> p=600.0hpa.
Export for pressure ID 5 -> p=500.0hpa.
Export for pressure ID 6 -> p=400.0hpa.
Export for pressure ID 7 -> p=300.0hpa.
Export for pressure ID 8 -> p=250.0hpa.
Export for pressure ID 9 -> p=200.0hpa.
Export for pressure ID 10 -> p=150.0hpa.
Export for pressure ID 11 -> p=100.0hpa.
Export for pressure ID 12 -> p=70.0hpa.
Export for pressure ID 13 -> p=50.0hpa.
Export for pressure ID 14 -> p=30.0hpa.
Export for pressure ID 15 -> p=20.0hpa.
Export for pressure ID 16 -> p=15.0hpa.
Export for pressure ID 17 -> p=10.0hpa.
Export for pressure ID 18 -> p=7.0hpa.
Export for pressure ID 19 -> p=5.0hpa.
Export for pressure ID 20 -> p=3.0hpa.
Export for pressure ID 21 -> p=2.0hpa.
Export for pressure ID 22 -> p=1.5hpa.
Export for pressure ID 23 -> p=1.0hpa.
This code is running fine. But its produced text files are wrong in column arrangements.
  1. first two columns are latitudes and longitudes( randomly spaced but not uniformly as i shown above)
  2. third column instead of temperature(Gp_value) is latitude and longitude value
I have attach one ext file data it generated.
Thanks for understanding my problem
Muhammad Usman Saleem
Muhammad Usman Saleem on 18 Jan 2016
Edited: Muhammad Usman Saleem on 18 Jan 2016
pressures = [1000, 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, ...
100, 70, 50, 30, 20, 15, 10, 7, 5, 3, 2, 1.5, 1] ;
% - Read source file.
fSpec = ['GPHeight_A[%f][%f]', repmat( ', %f', 1, 20 )] ;
data = textscan( fileread( 'Data1.txt' ), fSpec ) ;
These lines are not reading file content? Although its output is cell array of 1*22 cells. But why these cells are all empty?
Want to know that? why it not take data from my text file?
See its output in matlab
These are cell array of empty values.Why?

Sign in to comment.

Accepted Answer

Stephen23
Stephen23 on 18 Jan 2016
Edited: Stephen23 on 18 Jan 2016
textscan returns empty arrays is because it could not match the provided format string with the actual data in the file. It cannot match the file data because you have changed the file format since Cedric Wannaz wrote that code. When you change the file, then the same formatting command will not work any more.
Cedric Wannaz defined a format string like this:
fSpec = ['Temperature_TqJ_A[%f][%f]', repmat( ', %f', 1, 20 )]
but the first line of your file looks like this:
Longitude, 60.5, 61.5, 62.5, 63.5, 64.5, 65.5, 66.5, 67.5, 68.5, 69.5, 70.5, 71.5, 72.5, 73.5, 74.5, 75.5, 76.5, 77.5, 78.5
This format string does not match the very first line of the file (the string literal is different, there are different number of numeric terms), so it stops looking and returns some empty arrays.
This correctly reads your sample data file:
% Read the data file:
fid = fopen('temp.txt','rt');
Longitude = cell2mat(textscan(fid,['%*s',repmat('%f',1,19)],1,'Delimiter',','));
Latitude = cell2mat(textscan(fid,['%*s',repmat('%f',1,17)],1,'Delimiter',','));
StdPress = cell2mat(textscan(fid,['%*s',repmat('%f',1,24)],1,'Delimiter',','));
C = textscan(fid,['%s',repmat('%f',1,19)],'Delimiter',',','CollectOutput',true);
fclose(fid);
% Parse indices of Pressure and Latitude into numeric (not used):
idx = cellfun(@(s)sscanf(s,'Temperature_TqJ_A[%f][%f]'),C{1},'UniformOutput',false);
idx = [idx{:}];
% Calculate first two columns:
[lat,lon] = meshgrid(Latitude,Longitude);
% Write output files:
for k = 0:23
idy = k==idx(1,:);
%mat = [lat(:),lon(:),reshape(C{2}(idy,:).',[],1)].'; % unsorted
mat = sortrows([lat(:),lon(:),reshape(C{2}(idy,:).',[],1)]).'; % sorted
fid = fopen(sprintf('Output_%02d.txt',k),'wt');
fprintf(fid,'latitude\tlongitude\tGP_value\n');
fprintf(fid,'%.3f\t%.3f\t%.3f\n',mat);
fclose(fid);
end
This code works for your sample data file, here:
  7 Comments
Stephen23
Stephen23 on 18 Jan 2016
Edited: Stephen23 on 18 Jan 2016
This order is exactly what my code gives you.
Read my comment above and include the code that I gave that sorts the files by latitude:
mat = sortrows([lat(:),lon(:),reshape(C{2}(idy,:).',[],1)]).';
This line needs to replace the mat = ... line in my original answer (inside the loop). That is because this line sorts the data, exactly as you requested. That is why I wrote that line, to sort your data. And then tested it to make sure that it works. I also edited my answer to include this sorting, so you could copy the code from my answer and try it.
Here is a sample of how my code sorts your data:
latitude longitude GP_value
21.500 60.500 298.992
21.500 61.500 298.547
21.500 62.500 298.711
21.500 63.500 298.852
21.500 64.500 298.891
21.500 65.500 298.867
21.500 66.500 299.109
21.500 67.500 299.273
21.500 68.500 299.539
21.500 69.500 300.352
21.500 70.500 302.320
21.500 71.500 303.047
21.500 72.500 301.492
21.500 73.500 302.914
21.500 74.500 -9999.000
21.500 75.500 -9999.000
21.500 76.500 -9999.000
21.500 77.500 -9999.000
21.500 78.500 -9999.000
22.500 60.500 300.344
22.500 61.500 298.789
22.500 62.500 298.633
22.500 63.500 298.609
... lots here
37.500 74.500 -9999.000
37.500 75.500 -9999.000
37.500 76.500 -9999.000
37.500 77.500 -9999.000
37.500 78.500 -9999.000
Muhammad Usman Saleem
Muhammad Usman Saleem on 19 Jan 2016
Edited: Muhammad Usman Saleem on 19 Jan 2016
thank you so much for this kind assistance
Regards;
Muhammad Usman Saleem

Sign in to comment.

More Answers (1)

Thorsten
Thorsten on 18 Jan 2016
Edited: Thorsten on 18 Jan 2016
%%define some constants
filename = 'data1.txt';
NLongitude = 19;
NLatitude = 17;
NStdPressureLev = 24;
%%Process file
fid = fopen(filename);
line = fgets(fid); % first line is empty
% first read in values of Longitude, Latitude and StdPressureLev
line = fgets(fid); % should be Longitude
checktoken = 'Longitude';
assert(strcmp(line(1:length(checktoken)), checktoken), ...
[checktoken ' expected, but not found.'])
Longitude = sscanf(strrep(line(length(checktoken)+1:end), ',', ''), '%f');
assert(numel(Longitude)==NLongitude,...
['Wrong number of values for ' checktoken '.'])
line = fgets(fid); % should be Latitude
checktoken = 'Latitude';
assert(strcmp(line(1:length(checktoken)), checktoken), ...
[checktoken ' expected, but not found.'])
Latitude = sscanf(strrep(line(length(checktoken)+1:end), ',', ''), '%f');
assert(numel(Latitude) == NLatitude,...
['Wrong number of values for ' checktoken '.'])
line = fgets(fid); % should be StdPressureLev
checktoken = 'StdPressureLev';
assert(strcmp(line(1:length(checktoken)), checktoken), ...
[checktoken ' expected, but not found.'])
StdPressureLev = sscanf(strrep(line(length(checktoken)+1:end), ',', ''), '%f');
assert(numel(StdPressureLev) == NStdPressureLev,...
['Wrong number of values for ' checktoken '.'])
% read all numbers into a single 2D matrix of size
% (NStdPressureLev*NLatitude) x (2 + NLongitude)
format = ['Temperature_TqJ_A[%f][%f]', repmat(', %f', 1, NLongitude)];
T = cell2mat(textscan(fid, format, 'CollectOutput', true));
fclose(fid);
%%Reshape into 3D matrix NLatitude x NstdPressureLev x NLongitude
% of Temperature values
T = reshape(T(:, 3:end), [NLatitude, NStdPressureLev, NLongitude]);
%%Write files for each std pressure levels
% create columns for table of (Latitude Longitude Temperature)
[lat, lon] = meshgrid(Latitude, Longitude);
% cycle through all std pressure levels
for i = 1:3% NStdPressureLev
tem = squeeze(T(:,i,:))';
Tab_i = [lat(:) lon(:) tem(:)];
fid = fopen(sprintf('StdPressureLev_%d.txt', i), 'w');
fprintf(fid, '%%Std Pressure Level %f\n', StdPressureLev(i));
fprintf(fid, '%%Latitude Longitude Temperature\r\n');
fprintf(fid, '%.1f %.1f %.3f\n', Tab_i');
fclose(fid);
end
  4 Comments
Thorsten
Thorsten on 5 Apr 2016
Hi Muhammad! Have you run my code on temp.txt? If you have further questions please start a new question. People do not look at answered questions. Best, T
Muhammad Usman Saleem
Muhammad Usman Saleem on 5 Apr 2016
Dear @Thorsten, you may find it as new question here . Look at this please

Sign in to comment.

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!