How do I access an Invalid-named variable from an exported MAT file that MATLAB will not recognize?

8 views (last 30 days)
Hi!
I am having a problem resolving how to read into MATLAB some exported data that I obtained from another program, which I have no control in how it names its files.
The acquisition software allows for a MATLAB export, and it contains two MATLAB files, one a time series, and the other the output data. Unfortunately, the output data contains an illegal MATLAB variable symbol (#) and cannot be read.
Is there anyway to change the illegal variable name to make it a valid one?
I copied and pasted below for convenience exactly the describe problem above from my MATLAB screen:
>> load 01_ExportData >> whos -file 01_ExportData Name Size Bytes Class Attributes
c001_Time 1x1200000 9600000 double
c002_Analog_Input_#4 1x1200000 9600000 double
>> c002_Analog_Input_#4 ??? c002_Analog_Input_#4 | Error: The input character is not valid in MATLAB statements or expressions.
>>
How would I change the c002_Analog_Input_#4 variable name to access it? Or just get around this problem! Remember, I cannot control the inclusion of this # symbol, since I have no say it out the program from which I exported my data names its files.
Thanks a bunch in advance!!! Jess Li

Answers (8)

Richard Thiessen
Richard Thiessen on 5 Oct 2019
Edited: Richard Thiessen on 6 Oct 2019
I had the same problem.
Things that don't work
Matlab itself refuses to load data from a .mat file if the variable name involved is invalid.
There are multiple ways to call the load() function
load("myfile.mat")
load("myfile.mat","variable name with spaces")
%obviously fails since the variable names in the file are invalid
%and can't be created in the workspace.
struct=load("myfile.mat")
%structs can't contain invalid variable names either
valid_varname=load("myfile.mat","variable name with spaces")
%this fails too, Matlab validates the name before retrieving it
%from the file
None of them work of course. Even if there's nothing technically stopping the last command from working Matlab still refuses to touch invalid variable names.
I tried using the LOADFIXNAMES library but that did not work for me. After installing the compiler (MinGW-w64) the extension compiled but crashed matlab as soon as I called it with my files.
What did work
Anaconda (A python programming language bundle) includes the scipy library. The scipy.io library has the loadmat() and savemat() functions which read and write matlab data files. Loadmat produces a data structure containing the (name,value) pairs for all variables in the file. The names can be changed and the new (name,value) pairs put into a similar structure and fed to savemat to produce a fixed output file.
Here's the code I used: (Note that this is python code not matlab code)
import scipy.io as sio
import pathlib
import string
def fix_name(name):
name="".join(c for c in name if c in string.ascii_letters+string.digits+"_")
if name[0] not in string.ascii_letters:#name must start with a letter
name="X"+name
name=name[:63]#names may not be longer than 63 chars
return name
ignore_names=["__header__","__version__","__globals__"]
def fix_struct(s):
out_struct=dict()
for name,value in s.items():
if name not in ignore_names:
out_struct[fix_name(name)]=value
return out_struct
def struct_needs_fixing(s):
for name in s.keys():
if name not in ignore_names:
if fix_name(name)!=name:
return True
return False
root=r"C:\someplace\folder_with_the_files"
root=pathlib.Path(root)
files=list(root.glob("*.mat"))#get all the .mat data files in the directory
Overwrite=False #overwrite files in place
for f in files:
new_f=f.parent/("fixed_"+f.name)#save the modified file with a prefix
if Overwrite:new_f=f
struct=sio.loadmat(str(f))
if struct_needs_fixing(struct):
print("fixing",str(f))
struct=fix_struct(struct)
sio.savemat(new_f,struct)
else:
print("skipping",str(f))
This removes all unallowed chars and prefixes names with "X" if they don't already start with a letter.
Anaconda itself takes about 20 minutes to install. The code above can be run from jupyter notebook which should be available in the start menu after installing anaconda. The file path can be copied from the windows file explorer (right-click on file)-->properties-->(popup window)-->location The above script will fix all the files in a given folder.

Jan
Jan on 28 Jan 2011
In Matlab 2009a this C-Mex file can copy the value of Variables from the workspace:
#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
char *Name;
mxArray *Var;
Name = mxArrayToString(prhs[0]);
Var = mexGetVariable("caller", Name);
if (Var != NULL) {
plhs[0] = mxDuplicateArray(Var);
} else {
plhs[0] = mxCreateDoubleMatrix(0, 0, mxREAL);
}
mxFree(Name);
return;
}
EDITED: "mxReal"->"mxREAL" Thanks Matt!
I've stored this as CopyData.c, compiled it and a call like
A = CopyData('c002_Analog_Input_#4')
gets the value of the variable with the ugly name. By the way: I've created the ugly variable using:
mexPutVariable("caller", "c002_Analog_Input_#4", Var)
Good luck, Jan
  4 Comments
Jan
Jan on 28 Jan 2011
Then try to ASSIGNIN a variable called '_ugly', '#' or ' '.
But to be exact, you are right: ASSIGNIN is worse, because users really use it for *allowed* ugly actions, while mexPutVariable is seldom used to perform *forbidden* actions, although it has the power to.

Sign in to comment.


James Tursa
James Tursa on 31 Oct 2013
  1 Comment
James Tursa
James Tursa on 8 Oct 2019
Unfortunately, TMW has changed the API to disallow mex routines from reading and fixing invalid names, so this submission is now obsolete.

Sign in to comment.


Matt Fig
Matt Fig on 26 Jan 2011
Give this a try. What is the extension of the data file?
X = load('01_ExportData') % May need the extension
D = struct2cell(X)
If that doesn't work, have a look at this thread:

Jan
Jan on 26 Jan 2011
Dynamic field names must not be valid Matlab symbols. So instead of calling a C-Mex, you can use this:
X = load('01_ExportData', '-mat')
Value = X.('c002_Analog_Input_#4');

Jessica
Jessica on 26 Jan 2011
Hi. First thank you for the response. I tried the above, but here is what I get when I do this:
>> x = load('01_ExportData' , '-mat') ??? Error using ==> load Invalid field name: 'c002_Analog_Input_#4'.
>>
I cannot even get to the next line of code to try. MATLAB just reports the invalid field name and will not recognize x as a variable to continue with the next line.
Argh! I'm not exactly a newbie with MATLAB but not an expert either, but this has had even a MATLAB TA I know stumped!
Am I doing something wrong here with the above, or is this some strange bottleneck with the pound-sign that cannot be resolved. I can use the original program that I exported for the analysis if all else fails, but using MATLAB for the stats is just far more superior!
Thank you again in advance for your help!
-Jess
  1 Comment
Jan
Jan on 26 Jan 2011
Which Matlab version do you use? As far as I can see Matlab has stopped the support of bad field names from LOAD after Matlab > 2009a. What a pitty. Do you have access to an <= 2009a version to convert the file?
Does load('01_ExportData.mat', 'c002_Analog_Input_#4') work?

Sign in to comment.


Jessica
Jessica on 27 Jan 2011
Hi. Yes, load('01_ExportData.mat', 'c002_Analog_Input_#4') does work. And I am using >2009a. But the funny thing is that even though I can load it with the load version you gave, it will not even recognize c002_Analog_Input_#4.
  1 Comment
Walter Roberson
Walter Roberson on 27 Jan 2011
Edited: Walter Roberson on 26 Nov 2020
Perhaps if you combined this with Jan's earlier suggestion, to form:
X = load('01_ExportData.mat', 'c002_Analog_Input_#4');
X.('c002_Analog_Input_#4')

Sign in to comment.


Jessica
Jessica on 28 Jan 2011
Hi Walter. Thanks again (and to everyone else). I am afraid that none of the code works unfortunately to fix the bad field name. I am a bit disappointed that MATLAB has no easy way to remedy this (which would seem to be an obvious feature to have) and puzzled that they would have gotten rid of loading bad field names after 2009a. At least you could have implemented some of the above code to change the variable name more readily.
Well, I guess my only alternative is to use the cruddy analysis routines in the main program I exported the data out from the beginning. :(
Thanks a bunch anyhows guys! I appreciate the posts. -Jess :)
  3 Comments
Jan
Jan on 29 Jan 2011
Thanks Matt! I'm still confused about the Answers mechanisms. In opposite to CSSM personal emails are helpful here sometimes.

Sign in to comment.

Categories

Find more on Scripts in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!