MATLAB Answers

How to read netcdf files containing string arrays?

43 views (last 30 days)
I have a netCDF file which consists of a float array and a string array as verified by ncdump. In Matlab, I am able to load the float array, but I fail in loading the string array.
In particular, ncread produces an error:
ncread('string_test.nc','StringArray')
Error using netcdf.getVar (line 130)
12 is not a recognized netCDF datatype.
"12" is the code for NC_STRING and it seems like Matlab can not handle that.
The .nc file I'm trying to load can be downloaded from http://piotr.majdak.com/temp/netcdf/ where also more description is available.
I'm using Matlab 2011b - any ideas what I can do?

  0 Comments

Sign in to comment.

Accepted Answer

per isakson
per isakson on 5 Mar 2013
Edited: per isakson on 5 Mar 2013
I use R2012a.
ncdisp says: Datatype: UNSUPPORTED DATATYPE. See below. Why not accept that? What is the source of string_test.nc?
HDFView 2.9 reads string_test.nc as if it was an HDF5-file. So does Matlab's h5disp and h5read.
>> ncdisp( 'string_test.nc')
Source:
h:\m\cssm\string_test.nc
Format:
netcdf4
Dimensions:
test = 4
Variables:
FloatArray
Size: 4x1
Dimensions: test
Datatype: single
StringArray
Size: 4x1
Dimensions: test
Datatype: UNSUPPORTED DATATYPE
>> str = h5read( 'h:\m\cssm\string_test.nc', '/StringArray' )
str =
'NH10'
'CI03'
'HI17'
'NH11'
>>
>> h5disp( 'h:\m\cssm\string_test.nc', '/StringArray' )
HDF5 string_test.nc
Dataset 'StringArray'
Size: 4
MaxSize: 4
Datatype: H5T_STRING
String Length: variable
Padding: H5T_STR_NULLTERM
Character Set: H5T_CSET_ASCII
Character Type: H5T_C_S1
ChunkSize: []
Filters: none
Attributes:
'DIMENSION_LIST': H5T_VLEN

  3 Comments

Piotr Majdak
Piotr Majdak on 5 Mar 2013
Thanks for your response. I try to answer your questions, maybe some ideas will come up.
@ncdisp says: Datatype: UNSUPPORTED DATATYPE. that message is equivalent to the error I receive from ncread. I think that ncdisp uses the same implementation of netcdf as ncread.
@Why not accept that? netCDF supports strings in netCDF files. Since Matlab supports netCDF, it's hard to believe that Matlab's netcdf support is limited. And we really need string arrays support in our netcdf files.
@What is the source of string_test.nc? It is a simple convention file using string arrays. The CDL definition (srting_test.txt) can be found at http://piotr.majdak.com/temp/netcdf/. The netCDF file can be created by ncgen -b -k3 string_test.txt
@HDFView 2.9 reads string_test.nc as if it was an HDF5-file. HDF5 must be able to read the netcdf4 file because netcdf4 is based on HDF5.
@So does Matlab's h5disp and h5read: Glad to see that the HDF5-support in Matlab works as expected.
I see that there could be a work around for reading the problematic files by reading the problematic variables with HDF5. However, I also need a write support...
per isakson
per isakson on 5 Mar 2013
I think:
  • strings can be stored in more than one way (format).
  • Matlab is often slow to adapt to new versions of supported software.
  • When there is a high and a low-level set of functions in Matlab some functionality is typically missing in the high-level functions. I easily believe that Matlab's high-level support is limited. However, I often find it difficult to spot the critical statements (supported or not supported) in the documentation.
  • in this context one should avoid the word "must"
My guess:
  • ncread (R2012a) doesn't support variable length strings
Piotr Majdak
Piotr Majdak on 6 Mar 2013
I'm slowly coming to a similar conclusion. Thanks for the help.

Sign in to comment.

More Answers (3)

Maxime Hervo
Maxime Hervo on 31 Mar 2017
Is there any update for this issue ? I still have the same problem with MATLAB R2016b !

  0 Comments

Sign in to comment.


liu jibao
liu jibao on 31 Jan 2018
Is there any update for this issue ? I still have the same problem with MATLAB R2017b !

  2 Comments

Erik Quaeghebeur
Erik Quaeghebeur on 1 Mar 2018
It seems nobody bothered to report it as a bug, so it is not surprising nothing has changed.
I filed a bug report with Case Number 02978436.
Natalie Perlin
Natalie Perlin on 24 Jul 2019
Thank you for filing the report, I wonder if anything has been done since then!
I have the same problem with Matlab not being able to read string variables in R2017a, R2018b.

Sign in to comment.


Peter Kuma
Peter Kuma on 20 Aug 2018
Just to let others know, I have submitted a new bug report (since I could not find the one submitted by Erik). The answer was:
As you correctly pointed out, this is a current limitation of the
netCDF implementation. It is being actively investigated for
enhacement.
The workaround is to utilize the fact that netCDF4 formats are a
subset of the HDF5 format and we can utilize the HDF5 functionality
to read the data
Workaround:
Instead of ncdisp, ncread utilize h5disp, h5read
I hope the above workaround solves your issue. If you have any
further queries, please reach out to me.

  1 Comment

Anna Weeks
Anna Weeks on 21 Aug 2018
The ability to write a mapped array section of values into a netCDF variable (i.e the 'NF_PUT_VARM_ type' functionality) would be highly desirable.

Sign in to comment.