How do I access an Invalid-named variable from an exported MAT file that MATLAB will not recognize?
8 views (last 30 days)
Show older comments
Hi!
I am having a problem resolving how to read into MATLAB some exported data that I obtained from another program, which I have no control in how it names its files.
The acquisition software allows for a MATLAB export, and it contains two MATLAB files, one a time series, and the other the output data. Unfortunately, the output data contains an illegal MATLAB variable symbol (#) and cannot be read.
Is there anyway to change the illegal variable name to make it a valid one?
I copied and pasted below for convenience exactly the describe problem above from my MATLAB screen:
>> load 01_ExportData >> whos -file 01_ExportData Name Size Bytes Class Attributes
c001_Time 1x1200000 9600000 double
c002_Analog_Input_#4 1x1200000 9600000 double
>> c002_Analog_Input_#4 ??? c002_Analog_Input_#4 | Error: The input character is not valid in MATLAB statements or expressions.
>>
How would I change the c002_Analog_Input_#4 variable name to access it? Or just get around this problem! Remember, I cannot control the inclusion of this # symbol, since I have no say it out the program from which I exported my data names its files.
Thanks a bunch in advance!!! Jess Li
0 Comments
Answers (8)
Richard Thiessen
on 5 Oct 2019
Edited: Richard Thiessen
on 6 Oct 2019
I had the same problem.
Things that don't work
Matlab itself refuses to load data from a .mat file if the variable name involved is invalid.
There are multiple ways to call the load() function
load("myfile.mat")
load("myfile.mat","variable name with spaces")
%obviously fails since the variable names in the file are invalid
%and can't be created in the workspace.
struct=load("myfile.mat")
%structs can't contain invalid variable names either
valid_varname=load("myfile.mat","variable name with spaces")
%this fails too, Matlab validates the name before retrieving it
%from the file
None of them work of course. Even if there's nothing technically stopping the last command from working Matlab still refuses to touch invalid variable names.
I tried using the LOADFIXNAMES library but that did not work for me. After installing the compiler (MinGW-w64) the extension compiled but crashed matlab as soon as I called it with my files.
What did work
Anaconda (A python programming language bundle) includes the scipy library. The scipy.io library has the loadmat() and savemat() functions which read and write matlab data files. Loadmat produces a data structure containing the (name,value) pairs for all variables in the file. The names can be changed and the new (name,value) pairs put into a similar structure and fed to savemat to produce a fixed output file.
Here's the code I used: (Note that this is python code not matlab code)
import scipy.io as sio
import pathlib
import string
def fix_name(name):
name="".join(c for c in name if c in string.ascii_letters+string.digits+"_")
if name[0] not in string.ascii_letters:#name must start with a letter
name="X"+name
name=name[:63]#names may not be longer than 63 chars
return name
ignore_names=["__header__","__version__","__globals__"]
def fix_struct(s):
out_struct=dict()
for name,value in s.items():
if name not in ignore_names:
out_struct[fix_name(name)]=value
return out_struct
def struct_needs_fixing(s):
for name in s.keys():
if name not in ignore_names:
if fix_name(name)!=name:
return True
return False
root=r"C:\someplace\folder_with_the_files"
root=pathlib.Path(root)
files=list(root.glob("*.mat"))#get all the .mat data files in the directory
Overwrite=False #overwrite files in place
for f in files:
new_f=f.parent/("fixed_"+f.name)#save the modified file with a prefix
if Overwrite:new_f=f
struct=sio.loadmat(str(f))
if struct_needs_fixing(struct):
print("fixing",str(f))
struct=fix_struct(struct)
sio.savemat(new_f,struct)
else:
print("skipping",str(f))
This removes all unallowed chars and prefixes names with "X" if they don't already start with a letter.
Anaconda itself takes about 20 minutes to install. The code above can be run from jupyter notebook which should be available in the start menu after installing anaconda. The file path can be copied from the windows file explorer (right-click on file)-->properties-->(popup window)-->location The above script will fix all the files in a given folder.
1 Comment
S.R. Prakash
on 19 May 2021
This solution worked like a charm with my python3+numpy+scipy setup.
Thanks a bunch!!
Jan
on 28 Jan 2011
In Matlab 2009a this C-Mex file can copy the value of Variables from the workspace:
#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
char *Name;
mxArray *Var;
Name = mxArrayToString(prhs[0]);
Var = mexGetVariable("caller", Name);
if (Var != NULL) {
plhs[0] = mxDuplicateArray(Var);
} else {
plhs[0] = mxCreateDoubleMatrix(0, 0, mxREAL);
}
mxFree(Name);
return;
}
EDITED: "mxReal"->"mxREAL" Thanks Matt!
I've stored this as CopyData.c, compiled it and a call like
A = CopyData('c002_Analog_Input_#4')
gets the value of the variable with the ugly name. By the way: I've created the ugly variable using:
mexPutVariable("caller", "c002_Analog_Input_#4", Var)
Good luck, Jan
4 Comments
Jan
on 28 Jan 2011
Then try to ASSIGNIN a variable called '_ugly', '#' or ' '.
But to be exact, you are right: ASSIGNIN is worse, because users really use it for *allowed* ugly actions, while mexPutVariable is seldom used to perform *forbidden* actions, although it has the power to.
James Tursa
on 31 Oct 2013
You can try this FEX submission:
1 Comment
James Tursa
on 8 Oct 2019
Unfortunately, TMW has changed the API to disallow mex routines from reading and fixing invalid names, so this submission is now obsolete.
Matt Fig
on 26 Jan 2011
Give this a try. What is the extension of the data file?
X = load('01_ExportData') % May need the extension
D = struct2cell(X)
If that doesn't work, have a look at this thread:
1 Comment
Walter Roberson
on 26 Jan 2011
I agree, a small modification to Jame's mex code should work for this.
Jan
on 26 Jan 2011
Dynamic field names must not be valid Matlab symbols. So instead of calling a C-Mex, you can use this:
X = load('01_ExportData', '-mat')
Value = X.('c002_Analog_Input_#4');
Even the empty string is working! See: http://www.mathworks.com/matlabcentral/newsreader/view_thread/292602
2 Comments
Joel Duncan
on 26 Nov 2020
What about this situation I'm in?
X = load('PresTrace.15.mat')
Value = X.('c002_Analog_Input_#4')
% Error using load
% Invalid field name: '15PresTrace'
Walter Roberson
on 26 Nov 2020
Unfortunately in a later release the above option was disallowed.
Jessica
on 26 Jan 2011
1 Comment
Jan
on 26 Jan 2011
Which Matlab version do you use? As far as I can see Matlab has stopped the support of bad field names from LOAD after Matlab > 2009a. What a pitty. Do you have access to an <= 2009a version to convert the file?
Does load('01_ExportData.mat', 'c002_Analog_Input_#4') work?
Jessica
on 27 Jan 2011
1 Comment
Walter Roberson
on 27 Jan 2011
Edited: Walter Roberson
on 26 Nov 2020
Perhaps if you combined this with Jan's earlier suggestion, to form:
X = load('01_ExportData.mat', 'c002_Analog_Input_#4');
X.('c002_Analog_Input_#4')
See Also
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!