Convert sveral netCDF files to ASCII or CSV

HI all, I have 2000 files of netCDF (.nc). These files are for precipitation of my study area. I have the script which included below. I want to make this script to call all netCDF and output them as ASCII. Some netCDF files have attached too.. I really appreciate your help and time.

Thanks, Majid

[Jan]

What is your question? What have you tried so far? Which problems occur?

[KSSV]

netCDF is a good format..which are easy to access..why you want to change them to csv/ ascii?

[Majid Mohamod]

How are they easy to access, I want to combine them of each weather station. As result, I will have daily weather data which I'm going to use them as input in ArcSWAT to estimate the surface runoff. I want to make this script to call all netCDF and output them as ASCII.

[per isakson]

Have you verified that NetCDFreader together with NetCDF2ASCII work for one of your nc-file ?

The help says:

NetCDF2ASCII(header,PositionOfFile,PositionOfVariable)

How do one find the values for &nbsp PositionOfFile, PositionOfVariable ?

[per isakson]

The file you attached are from the FEX-submission: NetCDF reader and export to ASCII version 1.1 (2.17 KB) by Michael Pikridas. You should have informed us.

[Majid Mohamod]

I attached the license file, I thought as long as the license file is included, that would protect the copy right of the script producer.

[Majid Mohamod]

Please, "per isakson" Can you explain to me more how to do that. I'm beginner in this field. I tried different method as "GDAL", and I am still working on this issue. I appreciate your help if you can make this script work with multiple netCDF files!

[per isakson]

"GDAL" &nbsp is that GDAL - Geospatial Data Abstraction Library? What did you try?

"... work with multiple netCDF files!" &nbsp The first step must be to verify that the functions works as expected with one file. So far, I failed with one file. Did you succeed? Do you know that the FEX-submission is intended to work with your files?

[Majid Mohamod]

Yes, GDAL is Geospatial Data Abstraction Library.

The script is suppose to work with my files. The way how I run the script as the following: 1- I combined the two files " NetCDFreader + NetCDF2ASCII" as on script. 2- I changed the fpath to the folder path wher the netcdf files are located. 3. I run the script.

The result, the script didn't work!

I am worry that the way which I wrote the fpath is not correct. I don't know if there is a specific method to write the fpath?

As I understood from your comment, you wrote that the script didn't work with you too? Please, any suggestion to solve this problem?

Thanks, Majid

[per isakson]

• What makes you think that this FEX-submission can read and convert your files?
• "ArcSWAT" &nbsp is that The Soil and Water Assessment Tool (SWAT)?
• What makes you think that the FEX-submission will write a csv-file that is appropriate for ArcSWAT ?
• NetCDFreader requires that the extension is .nc. Yours is .nc4.

Answer 1

[KSSV]

Dear Majid

Reading nc files is not a deal in MATLAB. It is pretty straight forward and easy. If you would have tried to read the documentation, by this time you could have solved your problem. No files is required to read the ncfiles. Just you need to know only few functions: ncdisp, ncreaad and ncreadatt.

In the below code, I am reading the precipitation data of the six ncfiles which you gave in the attachment. I am reading the begin, end date and time also. The output is taken into cells.

files = dir('*.nc4') ;

nfiles = length(files) ;
P = cell(nfiles,1) ; % precipitation of all files
date = cell(nfiles,2) ;
time = cell(nfiles,2) ;
for i = 1:nfiles
    %%Read dat
    date{i,1} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.BeginDate')  ;
    date{i,2} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.EndDate')  ;
    %%Read time 
    time{i,1} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.BeginTime')  ;
    time{i,2} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.EndTime')  ;
    %%REad precipitation 
    P{i} = ncread(files(i).name,'precipitation') ;
end

Now you can write the above data into what ever format you want.

[Majid Mohamod]

I really appreciate your answer. Please, could you guide me how to output the precipitation data to ascii file or csv. Or there is away to combine all the netcdf files of each day to have a series of daily precipitation? Again thank you for helping me!

[KSSV]

I prefer to work with netCDF. See the below code, which reads all the above six ncfiles and writes them into single nc file.

files = dir('*.nc4') ;

nfiles = length(files) ;
P = cell(nfiles,1) ; % precipitation of all files
date = cell(nfiles,2) ;
time = cell(nfiles,2) ;
for i = 1:nfiles
    %%Read dat
    date{i,1} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.BeginDate')  ;
    date{i,2} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.EndDate')  ;
    %%Read time
    time{i,1} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.BeginTime')  ;
    time{i,2} = ncreadatt(files(i).name,'/','HDF5_GLOBAL.EndTime')  ;
    %%REad precipitation
    P{i} = ncread(files(i).name,'precipitation') ;
end

%%Write all the above data into nc-file
ncfile = 'merged.nc' ;
[nx,ny] = size(P{1}) ;
%
nccreate(ncfile,'time','Dimensions',{'time',1,Inf},'DeflateLevel',8) ;
%
nccreate(ncfile,'precipitation','Dimensions',{'lat',nx,'lon',ny,'time'},'DeflateLevel',8) ;
ncwriteatt(ncfile,'precipitation','units','mm');
ncwriteatt(ncfile,'precipitation','long_name','Daily accumulated precipitation (combined microwave-IR) estimate with gauge calibration over land');
ncwriteatt(ncfile,'precipitation','coordinates','lat lon');
ncwriteatt(ncfile,'precipitation','FillValue','-9999.9004');
ncwriteatt(ncfile,'precipitation','origname','precipitation');
ncwriteatt(ncfile,'precipitation','fullnamepath','/precipitation');
for i = 1:nfiles
    ncwrite(ncfile,'precipitation',P{i},[1,1,i]) ;
end
ncwriteatt(ncfile,'/','HDF5_GLOBAL.BeginDate',date{1,1}) ;
ncwriteatt(ncfile,'/','HDF5_GLOBAL.BeginTime',time{1,1}) ;
ncwriteatt(ncfile,'/','HDF5_GLOBAL.EndDate',date{nfiles,2}) ;
ncwriteatt(ncfile,'/','HDF5_GLOBAL.EndTime',time{1,2}) ;
ncwriteatt(ncfile,'/','creation_date',datestr(now));
ncdisp(ncfile)

[Majid Mohamod]

Thank you again for your offers. Any suggestion how to convert this file to csv, excel.. or ascii..? I prefer csv and excel format. The script you wrote for me is amazing, work perfectly. But please, can we finalized it by the conversion to the format which I mentioned to?

Thanks, Majid

[KSSV]

I have answered this in your new question. I was about to answer here..:)

[Majid Mohamod]

I am going to check it now. Thanks :)

Answer 2

[shobhit pipil]

data = [];

Files=dir('*.nc');

for k=1:length(Files)

FileNames=Files(k).name

pr=ncread(FileNames,'rainfall_amount');%name of netCDF file; 'pr' is the precip variable in the netCDF file

long=ncread(FileNames,'x'); %'longitude' is the longitude variable in the netCDF file

lat=ncread(FileNames,'y'); %'latitude' is the latitude variable in the netCDF file

for j= 716:746(lat); %1:length(lat);

for i= 328:368(long); %1:length(long);

v=pr(i,j,:); %read through all lat,longs in netCDF

outfile=sprintf('%d_%d_PCP.txt',lat(j),long(i));%name of outputfile; format is 'LATITUDE_LONGITUDE_PCP.txt'

fid=fopen(outfile,'w+');

data2=(v);

data2(isnan(data2)) = -99.0 ; %cleanup in case the data is a NaN

data = {data;data2};

end

end

end

%fprintf(fid, '18900101\n');%#write the climate data starting date to header of outfile. Change '20200101' to the starting date of your dataset. For example, Jan. 1, 1950 is 19500101

fprintf(fid,'%5.1f\n',data); %formatting the data for SWAT specifications

fclose(fid);

disp([outfile 'created'])%display created file

fclose all

[shobhit pipil]

Would anybody please reply to me. I am trying to extract the NetCDF pricipitation data into SWAT format. The current script is not able to write data from NetCDF to txt file from each iteration. Please help.

Answer 3

[javeed hamad]

HI every one!

I want to extract GHI (Global horizontal irradiation) data from NC file which include almost 3500 files for my study area, can someone please guide which script should use to extract multiple files simultaneously.

Thanks a lot, Javeed