TUTORIAL: Why Variables Should Not Be Named Dynamically (eval)

Summary:

Dynamically accessing variable names can negatively impact the readability of your code and can cause it to run slower by preventing MATLAB from optimizing it as well as it could if you used alternate techniques. The most common alternative is to use simple and efficient indexing.

Explanation:

Sometimes beginners (and some self-taught professors) think it would be a good idea to dynamically create or access variable names, the variables are often named something like these:

• matrix1, matrix2, matrix3, matrix4, ...
• test_20kmh, test_50kmh, test_80kmh, ...
• nameA, nameB, nameC, nameD,...

Good reasons why dynamic variable names should be avoided (click the links to jump to the "answers" below):

• Slow
• Buggy
• Security Risk
• Difficult to Work With
• Obfuscated Code Intent
• Confuses Data with Code
• Code Helper Tools do not Work
• Magically Making Variables Appear in a Workspace is Risky

There are much better alternatives to accessing dynamic variable names:

• Indexing into Cell Array or ND-Array
• Non-scalar Structures (with Indexing)
• Dynamic Field-Names in a Structure
• load into a Structure, not into the Workspace
• save the Fields of a Scalar Structure
• Use a table or timetable Array
• Use more Efficient Ways to Pass Variables Between Workspaces

Note that avoiding eval (and assignin, etc.) is not some esoteric MATLAB restriction, it also applies to many other programming languages as well:

• Other Languages: do not use eval!
• PS: The eval function itself is not faulty!

MATLAB Documentation:

If you are not interested in reading the answers below then at least read MATLAB's own documentation on this topic Alternatives to the eval Function, which states "A frequent use of the eval function is to create sets of variables such as A1, A2, ..., An, but this approach does not use the array processing power of MATLAB and is not recommended. The preferred method is to store related data in a single array." Data in a single array can be accessed very efficiently using indexing.

Note that all of these problems and disadvantages also apply to functions load (without an output variable), assignin, evalin, and evalc, and the MATLAB documentation explicitly recommends to "Avoid functions such as eval, evalc, evalin, and feval(fname)".

The official MATLAB blogs explain why eval should be avoided, the better alternatives to eval, and clearly recommend against magically creating variables. Using eval comes out at position number one on this list of Top 10 MATLAB Code Practices That Make Me Cry. Experienced MATLAB users recommend avoiding using eval for trivial code, and have written extensively on this topic.

[Adam]

Now all we need is for Matlab Answers to have a small number of key sticky questions that remain at the top! Not that new users would notice it necessarily, but some would!

[Jan]

@Stephen: Thanks. I've voted for each of your answers. let me suggest to replace the tag "make me cry" by "make us cry". ;-)

[Jan]

This thread has 1100 views this month. Stephen, if perhaps 10% 0r 50% of the readers are encouraged to avoid eval and use one of the mentioned better solutions, this will have a measurable effect on the world-wide-average of the code quality. Thanks again.

[Walter Johnson]

Well I know little about the eval function and I will take it from you that it is something bad to use for this purpose. That said, being able to input the name of an existing file or object that you want to access or read in is not some wacko desire. This is pretty standard with programming languages. There may be reasons why MATLAB does not support this, but it is not a strange or clearly crazy thing to want to do.

[per isakson]

"being able to input the name of an existing file" that's no problem with Matlab.

filespec = 'c:\tmp\the_name_of_my_file.csv';
str = fileread( filespec );

and

fid = fopen( filespec', 'r' );
...

[Stephen23]

"...being able to input the name of an existing file..."

... is really easy and does not require magically accessing variable names.

"This is pretty standard with programming languages."

Indeed many languages support string evaluation... and yet interestingly experts and experienced users of most of those languages consider this to be an extremely inefficient and insecure practice that should be avoided. Perhaps you missed reading this thread, which gives links to some of those discussions:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_236129

"There may be reasons why MATLAB does not support this,.."

MATLAB certainly does support this. Why do you think it does not? In fact the main function that is used for this is currently mentioned 75 times on this thread (and also in the title itself), something that readers of this thread are unlikely miss.

"...but it is not a strange or clearly crazy thing to want to do"

I totally agree: I don't think that it is a strange or crazy thing to want to do. But it is important to note that wanting to do something is not at all synonymous with something being a good way to write code: there are certainly much simpler and more efficient methods of writing better code, which waste less time (coding time, debugging time, run time). The fact that magically accessing variables forces beginners to write slow, complex, buggy code, and is easily avoided by better, simpler code and/or data design, is not changed by how much beginners might want to do it.

We often get questions from beginners who want to calculate all permutations of a set of numbers, or want use some other highly inefficient or numerically unstable algorithm or want to use numerically unstable operations: just because they want to do these things does not mean that their approach is going to be efficient, or produce any sensible output, or even be tractable at all. In these cases we would show them (if at all possible) better ways of solving their problem, just like this thread does. The reality that some code is slower, buggier on average, harder to debug, and much less efficient is not changed by you want.

[Steven Lord]

Stephen, given how long this page has become, I recommend summarizing the discussion in a BLUF at the top of the question to avoid TL;DR syndrome, something along the lines of:

"Bottom line: Numbering variables can negatively impact the readability of your code and can cause it to run slower by preventing MATLAB from optimizing it as well as it could if you used alternate techniques. The most common alternatives are:

[bulleted list with a short description of no more than 3 common alternatives]

For more information, see the longer discussion on the rest of this page."

I know I could do it myself, but you've put a lot of effort into creating and updating this page.

[Ed Wheatcroft]

@Stephen23, I just wanted to say thanks for putting this extensive page together. I was about to use eval() for something because I thought there was no other way, but after reading this page from top to bottom I feel like a) I now understand why not to use eval() and b) I know the correct way to solve the issue I was having. I found the 'better alternaives' section especiclaly helpful, as a lot of the other posts about eval() seem to focus heavily on the reasons not to use it, but don't really teach you how to avoid it. Anyway, thanks for creating/maintaining the page :)

[DGM]

I find it surprising that we haven't found more use for the "make me cry" tag. I'll have to keep that in mind. :D

Answer 1

[Stephen23]

Some quotes from Steven Lord, responses to inappropriate use of eval and friends:

"DO NOT USE EVAL"

"This is why you SHOULD NOT "poof" variables into the workspace at runtime, whether via EVALIN, ASSIGNIN, EVAL, or LOAD."

"Do NOT use eval, evalc, or any other function in the eval family"

[Cris Luengo]

The 2nd link, to the newsreader, is dead. (Long live the newsreader!)

[Stephen23]

@Cris Lengo: you can still access that newsreader thread here:

https://groups.google.com/forum/#!searchin/comp.soft-sys.matlab/subject$3A%22behavior$20of$20assignin%22/comp.soft-sys.matlab/Y-BoQbPqqXQ/Cl7zpUyT5vIJ

The relevant text is:

MATLAB is not lying to you.

When you run your function, MATLAB needs to determine what each identifier

you use is as part of the process of parsing the function. At that time,

there's no indication in your code that debug should be a variable; however,

there is a function named debug. Therefore, MATLAB decides that the

instances of debug in the code should be calls to that function. When the

code is actually executed, a variable named debug is created, and WHICH

reflects that fact -- but at that point, it's too late for MATLAB to "change

its mind" and it tries to call the debug function on the last line. DEBUG

is a script file, though, and so you correctly receive an error.

This is why you SHOULD NOT "poof" variables into the workspace at runtime,

whether via EVALIN, ASSIGNIN, EVAL, or LOAD.

--

Steve Lord

[e-mail]

Answer 2

[Stephen23]

Slow

The MATLAB documentation Alternatives to the eval Function explains that code that uses eval is slower because "MATLAB® compiles code the first time you run it to enhance performance for future runs. However, because code in an eval statement can change at run time, it is not compiled".

MATLAB uses JIT acceleration tools that analyze code as it is being executed, and to optimize the code to run more efficiently. When eval is used the JIT optimizations are not effective, because every string has to get compiled and run again on every single iteration. This makes loops with eval very slow. This is also the reason why not just creating variables with dynamic variable names is slow, but accessing them is also slow.

https://www.mathworks.com/products/matlab/matlab-execution-engine.html

https://blogs.mathworks.com/loren/2016/02/12/run-code-faster-with-the-new-matlab-execution-engine/

https://blogs.mathworks.com/loren/2007/08/29/for-versus-while/ (in the comments)

https://www.mathworks.com/matlabcentral/answers/320839-how-bad-is-the-performance-of-run-time-introspection

https://www.mathworks.com/matlabcentral/answers/33284-processor-usage#answer_53297

https://www.mathworks.com/matlabcentral/answers/353349-how-to-make-a-2-files-mat-become-1#comment_478662

https://www.mathworks.com/matlabcentral/answers/105936-how-to-make-dynamic-variable-names-a1-a2-an-with-for-loop-using-eval-num2str#comment_598847

Even the eval hidden inside of str2num can slow down code:

https://www.mathworks.com/matlabcentral/answers/357420-how-to-convert-a-numeric-string-into-a-numeric-range

Answer 3

[Stephen23]

Security Risk

eval will evaluate any string at all, no matter what commands it contains. Does that sound secure to you? This string command might be malicious or simply a mistake, but it can do anything at all to your computer. Would you run code which could do anything at all to your computer, without knowing what it was about to do?

For some users the surprising answer is "yes please!".

https://www.mathworks.com/matlabcentral/answers/42752-assigning-input-string-as-variable-name#answer_52582

For example, try running this (taken from Jos' answer here):

eval(char('fkur*)Ykvj"GXCN"{qw"pgxgt"mpqy"yjcv"jcrrgpu0"Kv"eqwnf"jcxg"hqtocvvgf"{qwt"jctfftkxg"000)+'-2))

Did you really run it on your computer even though you had no idea what it would do? Every time code gets a user input and evaluates it gives that user the ability to run anything at all. Does that sound secure to you?

[Alexander Geldhof]

I'm late to the party, but what does this do?

For the same reasons you mentioned, I'm rather wary from entering this in my Matlab command line.

[Adam Danz]

Hint: the char(str-2) is a caesar-cipher that produces a command that is exectued by eval().

[Steven Lord]

Running the char command is safe, that will just create a char vector you can read. Remove the eval() around the char command and it won't execute the command stored in the char vector.

Answer 4

[Stephen23]

Difficult to Work With

Many beginners come here with questions that are basically some version of "I have lots of numbered variables but I cannot figure out how to do this simple operation...", or "my code is very slow/complex/buggy... how can I make it better?":

https://www.mathworks.com/matlabcentral/answers/322696-automatically-creating-an-array-of-variables-and-assigning-to-a-function-with-changing-number-of-out

https://www.mathworks.com/matlabcentral/answers/249044-code-fails-to-run-in-function-works-in-command-window-works-in-debug-evaluate-selection

https://www.mathworks.com/matlabcentral/answers/49756-use-eval-statement-to-create-matrices-with-variables

https://www.mathworks.com/matlabcentral/answers/341708-variable-names-in-loop

https://www.mathworks.com/matlabcentral/answers/346695-how-to-save-variables-to-an-array-using-a-for-loop

https://www.mathworks.com/matlabcentral/answers/356673-operation-on-variables-whose-name-contain-a-certain-string

https://www.mathworks.com/matlabcentral/answers/460207-concatenate-structures-select-structures-only-if-not-empty#answer_373564

Even advocates of eval get confused by it, fail to make it work properly, and can't even figure out why, as these two examples clearly show:

https://www.mathworks.com/matlabcentral/answers/158412-how-to-open-txt-files-with-variable-names#answer_154956

https://www.mathworks.com/matlabcentral/answers/272454-how-do-i-iterate-through-multiple-structures#answer_213034

Why can't they figure out why it does not work?:

• Totally obfuscated code due to indirect code evaluation.
• More complex than it needs to be.
• The code helper tools do not work.
• Syntax highlighting does not work.
• Static code checking does not work.
• No useful error messages, etc. etc.

Writing code is hard. Don't make it even harder by turning off the tools that check and help improve your code.

Answer 5

[Stephen23]

Buggy

Using eval makes it really hard to track down bugs, because it obfuscates the code and disables lots of code helper tools. Why would you even want to use a tool that makes it harder to debug and fix your code?

Here are some examples to illustrate how what should have been simple operations become very difficult to debug because of the choice to use eval:

https://www.mathworks.com/matlabcentral/answers/275439-help-with-code-problem

https://www.mathworks.com/matlabcentral/answers/158412-how-to-open-txt-files-with-variable-names#answer_154956

https://www.mathworks.com/matlabcentral/answers/272454-how-do-i-iterate-through-multiple-structures#answer_213034

https://www.mathworks.com/matlabcentral/answers/349843-getting-an-error-this-statement-is-incomplete

Code that generates variable names dynamically based on imported data or user inputs is also susceptible to the names reaching the name length limit:

https://www.mathworks.com/matlabcentral/answers/498489-clarification-about-using-only-the-first-namelengthmax-characters-of-the-name

This quote sums up debugging eval based code: "I've never even attempted to use it myself, but it seems it would create unreadable, undebuggable code. If you can't read it and can't fix it what good is it?" Note that eval's equally evil siblings evalc, evalin and assignin also make code slow and buggy:

https://www.mathworks.com/matlabcentral/answers/31963-functions-vs-script#answer_40489

https://www.mathworks.com/matlabcentral/answers/372206-use-of-evalc-with-num2str#answer_295604

Answer 6

[Stephen23]

Obfuscated Code Intent

What does this code do?:

x1 = [119,101,98,40,39,104,116,116,112,58,47,47,119,119,119];
x2 = [46,121,111,117,116,117,98,101,46,99,111,109,47,119,97];
x3 = [116,99,104,63,118,61,100,81,119,52,119,57,87,103,88];
x4 = [99,81,39,44,39,45,98,114,111,119,115,101,114,39,41];
eval(char([x1,x2,x3,x4]))

Unfortunately eval makes it easy to write code which is hard to understand: it is not clear what it does, or why. If you ran that code without knowing what it does, you should know that it could have deleted all of your data, or sent emails to everyone on your contact list, or downloaded anything at all from the internet, or worse...

Because eval easily hides the intent of the code many beginners end up writing code that is very hard to follow and understand. This makes the code buggy, and also harder to debug! See these examples:

https://www.mathworks.com/matlabcentral/answers/380551-what-is-the-function-of-sub2ind

Properly written code is clear and understandable. Clear and understandable code is easier to write, to bug-fix, and to maintain. Code is read more times than it is written, so never underestimate the importance of writing code that is clear and understandable: write code comments, write a help section, use consistent formatting and indentation, etc.

[Sean de Wolski]

I remember the newsreader thread you got this from!

[Walter Roberson]

You're never going to give up on this example, are you?

[Walter Roberson]

I just encountered someone using num2str() on a computed variable name, in order to have the effect of an eval() without using eval() directly in the code. This is, needless to say, obscure intent.

[Stephen23]

@Walter Roberson: should that example be STR2NUM? (which contains EVAL inside)

Answer 7

[Stephen23]

Code Helper Tools do not Work

The MATLAB editor contains many tools that advanced users continuously make use of, and beginners should particularly appreciate when learning MATLAB. However none of these tools work with code hidden inside eval:

• check the syntax as you type.
• shows warnings and errors on the RHS of the editor. Very very very very useful!
• Tab completion of function names, filenames, etc.
• variable highlighting shows all locations where a variable is used.
• f1 button to access function help.
• code analyzer prints reports on the code quality: errors, warnings, suggestions on how it could be improved.
• Text search or find-and-replace. Note that with magical variable names it is not even possible to do a simple search for a variable name, because if that variable name is generated dynamically then the name is not defined until the code is executed: how would you imagine searching for where a particular variable is used, if you cannot search for that variable? Ouch!
• text search in multiple files.
• automatically rename all instances of a variable name: very handy!
• and much more...

Note that these do not work when using eval, evalc, etc. to magically create or access variable names. Would you want to disable the tools that help you to write functioning code? Here are examples of how eval hides code errors and makes it hard to debug code:

https://www.mathworks.com/matlabcentral/answers/158412-how-to-open-txt-files-with-variable-names#comment_242698

https://www.mathworks.com/matlabcentral/answers/322696-automatically-creating-an-array-of-variables-and-assigning-to-a-function-with-changing-number-of-out

https://www.mathworks.com/matlabcentral/answers/374023-error-using-eval-function

https://www.mathworks.com/matlabcentral/answers/51946-systematic-do-not-use-global-don-t-use-eval#comment_107637

[Tom Hawkins]

On this topic, it would be great if the Code Analyzer and checkcode would actually flag a warning when eval etc. are used. Perhaps that would cut down the number of questions about them on here?

Answer 8

[Stephen23]

Alternative: Indexing into Cell Array or ND-Array

Oftentimes when a user wants to use eval they are trying to create numbered variables, which are effectively an index joined onto a name. It is usually better to turn that pseudo-index into a real index: MATLAB is fast and efficient when working with indices, and using indices will make code much much simpler than anything involving dynamic variable names:

https://www.mathworks.com/company/newsletters/articles/matrix-indexing-in-matlab.html

https://www.mathworks.com/help/matlab/math/matrix-indexing.html

Using ND-arrays is a particularly efficient way of handling data: many operations can be performed on complete arrays (known as code vectorization), and ND-arrays are easy to get data in and out of, and reduces the chance of bugs:

https://www.mathworks.com/matlabcentral/answers/409279-on-multiple-logical-test-failing-in-if-statement#answer_327952

Or simply put the data into the cells of a cell array:

https://www.mathworks.com/help/matlab/matlab_prog/access-data-in-a-cell-array.html

https://www.mathworks.com/help/matlab/matlab_prog/multilevel-indexing-to-access-parts-of-cells.html

And some real-world examples of where indexing is much simpler than eval:

https://www.mathworks.com/matlabcentral/answers/11981-help-with-find-possibly-inside-of-eval

https://www.mathworks.com/matlabcentral/answers/160310-dynamic-variables-in-loops

Answer 9

[Stephen23]

Alternative: Dynamic Field-Names in a Structure

dynamic fieldnames are easy to use, and do not require string evaluation.

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html

https://blogs.mathworks.com/loren/2005/12/13/use-dynamic-field-references/

Answer 10

[Stephen23]

Alternative: load into a Structure, not into the Workspace

The MATLAB documentation explains this in detail:

https://www.mathworks.com/help/matlab/import_export/troubleshooting-loading-variables-within-a-function.html

In almost all cases where data is imported programmatically (i.e. not just playing around in the command window) it is advisable to load data into an output argument (which is a structure if the file is a .mat file):

S = load(...);

The fields of the structure can be accessed directly, e.g:

S.X
S.Y

or by using dynamic fieldnames. Note that this is the inverse of saving the fields of a scalar structure.

It is important to note that (contrary to what some users seem to think) it is actually easier and much more robust to save and load data within a loops when the variable names in the .mat files do not change, as having to process different variable names in each file actually makes saving/loading the file data more complex, inefficient, and fragile.

Summary: when using a loop, keep all variable names the same!

Here are real-world examples of loading into variables:

https://www.mathworks.com/matlabcentral/answers/429680-how-to-load-a-mat-file-and-make-all-the-variables-from-that-mat-file-to-be-global-variables

https://www.mathworks.com/matlabcentral/answers/302359-access-variables-in-several-mat-files

https://www.mathworks.com/matlabcentral/answers/335253-mat-files-not-loading-correctly#answer_262994

https://www.mathworks.com/matlabcentral/answers/297146-loading-mat-file-into-a-variable

https://www.mathworks.com/matlabcentral/answers/28676-why-use-x-load-myfile-mat#answer_37821

https://www.mathworks.com/matlabcentral/answers/134091-variable-changes-value-after-using-load

https://www.mathworks.com/matlabcentral/answers/29579-decent-way-of-loading-and-saving-variables#answer_37932

https://www.mathworks.com/matlabcentral/answers/485484-using-load-in-a-test-methods-makes-the-test-pass#answer_397441

And finally Steven Lord's comment on load-ing straight into the workspace:

https://www.mathworks.com/matlabcentral/answers/349330-odd-behavior-with-a-simulink-reserved-name-in-a-function

EDIT: No longer active: https://www.mathworks.com/matlabcentral/newsreader/view_thread/244639#628669

Answer 11

[Stephen23]

Alternative: Non-Scalar Structure (with Indexing)

Using a non-scalar structure is much simpler than trying to access dynamic variable names. Here are some examples:

https://www.mathworks.com/matlabcentral/answers/79281-create-a-structure-name-based-on-the-string-of-a-variable-create-fieldnames-in-this-structure

https://www.mathworks.com/matlabcentral/answers/128912-concatenating-string-variable-names-and-passing-on-values-of-a-matrix-in-a-for-loop

A very neat example is using the output from DIR to store imported file data:

S = dir(..);
for k = 1:numel(S)
    S(k).data = readmatrix(S(k).name);
end

Answer 12

[Stephen23]

Other Languages: do not use eval!

In case you think that avoiding dynamic variable names is just some "weird MATLAB thing", here is the same discussion for some other programming languages, all advising "DO NOT create dynamic variable names":

https://blogs.msdn.microsoft.com/ericlippert/2003/11/01/eval-is-evil-part-one/

https://blogs.msdn.microsoft.com/ericlippert/2003/11/04/eval-is-evil-part-two/

http://nedbatchelder.com/blog/201112/keep_data_out_of_your_variable_names.html

http://www.techrepublic.com/blog/it-security/never-use-dynamic-variable-names/

http://stackoverflow.com/questions/11118486/python-list-as-variable-name

http://www.reddit.com/r/learnpython/comments/2a9o4s/why_is_it_a_bad_idea_to_create_dynamic_variables/

http://stackoverflow.com/questions/1479490/making-a-string-the-name-of-an-instance-object/1480247#1480247

http://stackoverflow.com/questions/25076883/creating-dynamically-named-variables-in-a-function-in-python-3-understanding-e

http://perl.plover.com/varvarname.html

https://24ways.org/2005/dont-be-eval

https://softwareengineering.stackexchange.com/questions/311507/why-are-eval-like-features-considered-evil-in-contrast-to-other-possibly-harmfu/311510

https://stackoverflow.com/questions/2571401/why-exactly-is-eval-evil

Some languages might use, require, or otherwise encourage dynamic variable names: if that is how they work efficiently, then so be it. But what is efficient in one language means nothing about the same approach in other languages... if you wish to use MATLAB efficiently, make your code easier to work with, and write in a way that other MATLAB users will appreciate, then that means learning how to use MATLAB features and tools:

https://www.mathworks.com/matlabcentral/answers/320774-looping-over-a-set-of-names-and-creating-a-set-of-variables

Answer 13

[Stephen23]

Magically Making Variables Appear in a Workspace is Risky

This leads to many subtle bugs that are extremely difficult to track down, if they are even noticed at all!

1) For a start variables of the same name will be overwritten without warning. Even just a spelling mistake or adding extra variables to a MAT file can change the behavior of your code, and because it depends on the data files that you are working with, can be very difficult to track down.

2) Importing multiple files in a loop can ruin your data: consider what will happen if your code processes a sequence of MAT files, which you think all contain the same variables. But one of them contains different variables (yeah, I know, your data files are perfect... sure). Consider what happens in badly-written, fragile code that simply LOADs directly into the workspace: it will happily process the data from the previously loaded file, without giving you any warning or notification that your data are now from the wrong file. Processing continues using the wrong data.

3) There is another serious yet subtle problem, which is caused by the MATLAB parser finding alternative functions/objects/... and calling those instead of using the magically-created variable: basically if the variable does not exist then the parser does its best to find something that matches where the name is called/used later... and it might just find something! For example:

https://www.mathworks.com/matlabcentral/answers/1914590-errors-with-imshow-function

https://www.mathworks.com/matlabcentral/answers/349330-odd-behavior-with-a-simulink-reserved-name-in-a-function

https://www.mathworks.com/matlabcentral/answers/28676-why-use-x-load-myfile-mat

https://www.mathworks.com/matlabcentral/answers/120518-trouble-with-accessing-variable-in-parfor-loop

https://www.mathworks.com/matlabcentral/answers/235119-can-you-explain-this-weird-behavior

https://www.mathworks.com/matlabcentral/newsreader/view_thread/244639#628669

https://www.mathworks.com/matlabcentral/answers/410758-function-call-doesn-t-work-in-script-works-in-command-window#answer_329197

https://www.mathworks.com/matlabcentral/answers/51946-systematic-do-not-use-global-don-t-use-eval#comment_107637

Or in some cases the parser might not find anything:

https://www.mathworks.com/matlabcentral/answers/1727880-difference-between-debug-parfor-loop-and-run-it

The solution is simple: do not magically "poof" variables into existence: Always load into a structure, and never create variable names dynamically.

[Jonathan]

You seem to imply that this is expected behaviour, but it indicates a (partially?) static name resolution instead of a dynamic one, which is really unexpected for an interpreted language. It is not the job of the parser to interpret the code; and surely if a variable is registered dynamically at runtime, it could overwrite whatever the parser decided previously. Granted, "poofing" variables into existence is a terrible idea, but this should not cause an inconsistent behaviour of name resolution. I think it would be fair to call this a bug, rather than a "gotcha".

[Stephen23]

@Jonathan:

"You seem to imply that this is expected behaviour"

I based my answer on various comments of TMW staff members (which I linked to): they are in a much better position than I to make such statements.

"...but it indicates a (partially?) static name resolution instead of a dynamic one, which is really unexpected for an interpreted language."

You assume that MATLAB is interpreted. MATLAB is not really a purely interpreted language: the JIT compiler analyses (static) code and finds ways to optimize it before running it. The TMW blogs state that "MATLAB provides the best of both worlds by compiling MATLAB code on-the-fly, or just-in-time.", distinguishing it from both compiled and interpreted code.

source: https://blogs.mathworks.com/loren/2016/02/12/run-code-faster-with-the-new-matlab-execution-engine/

Describing MATLAB as using "(partially?) static name resolution" is probably reasonable. Ask TMW for more details if you are interested.

"It is not the job of the parser to interpret the code''

Based on the explanations given by TMW staff members in various comments, blogs, and documentation, it appears that the JIT optimization does exactly that.

"if a variable is registered dynamically at runtime, it could overwrite whatever the parser decided previously"

While possible, that would likely be in direct conflict with the JIT parser. Essentially you would slow down (all?) code just to support bad code practices.

Read more:

https://www.mathworks.com/products/matlab/matlab-execution-engine.html

https://www.mathworks.com/matlabcentral/answers/430098-documentation-on-jit-compiler

[Guillaume]

Certainly, mathworks could introduce a construct to automatically resolve dynamic variable name, similar to the $$ construct that PHP uses. And indeed, from a performance point of view, on a purely interpreted language that would have little to no impact. However, as Stephen just pointed out, matlab is not purely interpreted (nowadays no decent programming language is) so anything that changes at runtime does have an impact on performance. If you look at the release notes over the year, you can clearly see that mathworks are constantly optimising variable resolution to take it out of the interpreted stage (hence why poofing variables with load can now fail where it worked before).

From a design point of view, dynamic variable names will always be a nightmare in most cases. PHP is not very well regarded for its security precisely because it allows these constructs.

When you start writing code to generate variable names, you're no longer writing code to process your data, you're writing code to generate the code that will process your data, and the increased complexity of this metaprogramming is always an added risk (of bugs, security issues, etc.)

[Jonathan]

Thanks to both of you for the very prompt responses! :)

I did not know that Matlab was not purely interpreted; knowing this clarifies the issue with dynamic names. Truth be told, I never use dynamic names in my code (because of some of the reasons listed in this thread), and would never recommend that anyone did. But I find edge-cases interesting.

Regarding potential impacts on performance, it seems to me that dynamic names can only ever occur due to an `eval`, `evalin` or `load` operation, is that right? I think it is fair to say that none of these functions should be used in a portion of code that is performance-critical. Though I am not one to give advice about language design, would it be possible to "re-parse" the code following such operations as they occur, in order to update the name resolution? This would badly affect performance when they are used, but would leave other situations unaffected, wouldn't it?

[Guillaume]

assignin and evalc also belong to that list. I can't think of any others right now.

The reparsing that you describe is most likely what was happening is past versions. But at least for load, it's clear that mathworks are moving away from that design and are not interested in supporting implicitly defined variables. I.e.

function myfunc()
    load data.mat   %creates a x variable
    disp(x(:));
end

no longer works (or will no longer work). See R2017a release notes.

I'm fine with that. If the optimiser has to detect whether or not it can optimise or must leave it later to reparse will have an impact on performance for both cases since it must do that detection.

[Walter Roberson]

There is currently some name re-resolution being done whenever the MATLAB path changes, including when you cd() -- which is a reason to avoid cd() in code.

In the past, I put some thought into the kinds of structures you would have to put in place in order to handle that situation efficiency . I did not follow it through, though; just some thought experiments.

Answer 14

[Stephen23]

Confuses Data with Code

The inclusion of data and meta-data within variable names (e.g. naming a variable with the user's input, the name of a test subject, or (very commonly) adding an index onto a variable name) is a subtle (but closely related) problem, and it should definitely be avoided. This quote from Image Analyst explains the problem succinctly: "When you start writing code to generate variable names, you're no longer writing code to process your data, you're writing code to generate the code that will process your data, and the increased complexity of this metaprogramming is always an added risk (of bugs, security issues, etc.)"

Read these discussions for an explanation of why it is a poor practice to put data and meta-data in variable names:

https://www.mathworks.com/matlabcentral/answers/56124-eval-is-evil-using-variables-created-dynamically-info-retrieval

https://www.mathworks.com/matlabcentral/answers/225435-save-variable-as-string-from-user-input

https://www.mathworks.com/matlabcentral/answers/12350-variable-name-length-exceeds-63-characters

In many cases that meta-data is just a de-facto index, i.e. a value that proscribes the order of the data. But in that case the de-facto index should be turned into a much more efficient real numeric index:

https://www.mathworks.com/matlabcentral/answers/325569-finding-the-means-of-a-large-number-of-individual-matrices#answer_255199

https://www.mathworks.com/matlabcentral/answers/8337-creating-a-matrix-with-many-numbered-variables

https://www.mathworks.com/matlabcentral/answers/240702-is-there-a-way-to-make-matlab-create-variables-of-any-number-on-its-own#answer_191569

[Samuel Gray]

...everything is data, the question is what do you do with it

[Walter Roberson]

@Samuel Gray

MATLAB supports targetting systems with Harvard architecture -- systems where the instructions and the data live in different address spaces, so the instructions are not data on such systems.

Answer 15

[Stephen23]

Alternative: Use more Efficient Ways to Pass Variables between Workspaces (applies to evalin, assignin, etc)

Use nested functions, or pass arguments, or use any of the other efficient ways to pass data between workspaces:

https://www.mathworks.com/help/matlab/matlab_prog/share-data-between-workspaces.html

https://www.mathworks.com/help/matlab/creating_guis/share-data-among-callbacks.html

https://www.mathworks.com/help/matlab/creating_guis/share-data-across-callbacks-in-app-designer.html

Answer 16

[Stephen23]

PS: eval is Not Faulty:

Some users apparently think that eval (and friends) must be faulty and should be removed from MATLAB altogether. They ask "if eval is so broken, why has it not been removed?"... but it is important to understand that the problem is caused by magically accessing variable names regardless of what tool or operation is used, and that eval (or assignin, or evalin, or load without an output argument, etc.) is simply being used inappropriately because there are much better methods available ( better in the sense faster, neater, simpler, more robust, etc). Read these discussions for good examples of this confusion:

https://www.mathworks.com/matlabcentral/answers/364851-assignin-and-eval-functions

https://www.mathworks.com/matlabcentral/answers/352237-is-there-the-more-elegant-way-to-do-this

It is important to note that any feature of a language can be used inefficiently or in an inappropriate way, not just eval, and this is not something that can be controlled by the language itself. For example, it is common that someone might solve something with slow loops and without preallocating the output arrays: this does not mean that for loops are "faulty" and need to be removed from MATLAB!

It is up to the programmer to write efficient code.

Answer 17

[Stephen23]

Alternative: save the Fields of a Scalar Structure

The save command has an option for saving the fields of a scalar structure as separate variables in a .mat file. For example, given a scalar structure:

S.A = 1;
S.B = [2,3];

this will save variables A and B in the .mat file:

save('myfile.mat','-struct','S')

This is the inverse function of loading into a structure. Some threads showing how this can be used:

https://www.mathworks.com/matlabcentral/answers/456803-how-to-allow-a-gui-user-to-defined-a-variable-name

https://www.mathworks.com/matlabcentral/answers/44175-saving-in-loop-using-eval-or-num2str

https://www.mathworks.com/matlabcentral/answers/314554-saving-function-variables-without-evil-eval#answer_245192

Answer 18

[Steven Lord]

Alternative: Use a table or timetable Array

table (introduced in release R2013b) and timetable (introduced in release R2016b) arrays allow you to store data with row and/or column names with which you can access the data. For example, if you create a table with variables named Age, Gender, Height, Weight, and Smoker and rows named with the last names of the patients:

load patients

patients = table(Age,Gender,Height,Weight,Smoker,...
    'RowNames',LastName);

you can ask for all the ages of the first five patients:

patients(1:5, 'Age')

or all the data for the patients with last names Smith or Jones:

patients({'Smith', 'Jones'}, :)

You can also add new variables to the table, either by hard-coding the name of the variable:

% Indicate if patients are greater than five and a half feet tall
patients.veryTall = patients.Height > 66

or using variable names stored in char or string variables. The code sample below creates new variables named over40 and under35 in the patients table using different indexing techniques.

newname1 = 'over40';
patients.(newname1) = patients.Age > 40;

newname2 = 'under35';
patients{:, newname2} = patients.Age < 35;

patients(1:10, :) % Show the first ten rows

The code sample below selects either Height or Weight and shows the selected variable for the fifth through tenth patients using dynamic names.

if rand > 0.5
    selectedVariable = 'Height';
else
    selectedVariable = 'Weight';
end
patients.(selectedVariable)(5:10)

See this documentation page for more information about techniques you can use to access and manipulate data in a table or timetable array. This documentation page contains information about accessing data in a timetable using the time information associated with the rows.

[Stephen23]

Simpler and more robust way to generate a table from that .mat file:

S = load('patients.mat');
T = struct2table(S,'RowNames',S.LastName);

Answer 19

[Econstudent]

You discuss at length why we shouldn't A, B or C and you also comment on how we could access certain objects.

Now, suppose we need to import a few time series -- but I can only import those series one at a time. The intention behind creating a sequence of variables inside a loop is often to store those time series in distinct object every time. That is, you want to assign the data to a different object every time and do it considerably more than once...

What other choice do you have besides creating objects within your loop?

[Stephen23]

"What other choice do you have besides creating objects within your loop?"

All of the choices that are explained above: cell arrays, structures, ND numeric arrays. And with newer MATLAB versions also tables, strings, datetime, etc. All of these allow you to "import those series one at a time", and use indexing (or fieldnames, etc) to put that data into one variable/object. Simple.

"The intention ... is often to store those time series in distinct object every time"

That is exactly what container arrays are for (e.g. cell array, structure arrays, etc.).

[Jan]

@Econstudent: Did you read this thread carefully? If your "sequence of variables" mean, that the variables are a sequence, they have a relation. Then this relation should be mapped in the code by collecting them in one array. An efficient and logic representation of the data is essential for good code, but it remains the art of programming.

[John Dzielski]

I hope I'm not going to get my head bit off here, but I have use case where I always use eval and I'm not sure of a better way. Suppose I have a directory with a bunch of files (usually there is a naming pattern, but let's assume not). They all contain the same type of csv data. If I load them, they are loaded into a variable name based on the file name. What I do is use eval to assign that data to a variable name 'data' (for example) and write my code so that it operates on the variable 'data'. Then I copy 'data' back to the original variable name using eval and save that usually as a mat file with a name based on the original filename. How do I do this without using eval?

[Rik]

The loading can be done by loading to a struct:

S=load(filename);
data=S.data;

The saving can be done with a struct as well:

S=struct('data',data);
save(filename,'-struct','S')

So again, no need for eval.

I personally prefer to use save along with this function:

function out=var2str(varargin),out=inputname(1);end

That way I will still have the code completion tools to help me with the variable names:

save(filename,'-struct',var2str(S))
%                               ^
%  This will be detected as usage of S, suppressing 'variable unused'
%  warnings and helping with mass renaming of variables.

[John Dzielski]

I'm maybe going to answer my own question here, but I can use readtable to load the csv data into a variable named 'data'. Anyway, I'd still like to know if my use of eval is still deprecated.

[Walter Roberson]

Pretty much all uses of eval() itself are deprecated.

There are some uses of evalin() that cannot be escaped from though.

[Stephen23]

"I'd still like to know if my use of eval is still deprecated"

Using LOAD to import textfile data was deprecated long ago in favor of much more versatile functions specifically designed for textfiles:

https://www.mathworks.com/help/matlab/import_export/ways-to-import-text-files.html

[Samuel Gray]

...is it not obvious the problem with creating N functions for a specific subset of the range of data that one function works with, while at the same time dropping support for the original function?

I may not want load and store to always involve mat files, but that doesn't mean that when I do or at least I find it to be an acceptable alternative to not having them work at all, I want my m-files to stop working because someone at Matlab decided that it was "bad code" and they would no longer support the way that it worked when I wrote the program.

Do you know how many companies have gone entirely out of business behind this logic?

You do realize that some clients are forced to pay vendors to support older versions of their tools that aren't "enhanced" with such logic in mind?

That's why you have 50 toolboxes and 5 million m-files (and 50 different file-types) in Matlab, right?

You honestly think that clients want to deal with this when they upgrade Matlab?

Someone has to spend real time and money to fix (and re-validate and re-secure) that old code.

What you have done is essentially take a 5 character string that did the job, and replaced it with a 50-character string.

To you it's a different function, a different concept, a different and better way of doing things.

To your clients it's just a new string of code which hopefully works at least as well as the old string.

It is fine that you've come up with an enhanced version of the function, that indeed may actually work better and be a positive change compared to the old function. That doesn't mean that we want you to break the old function.

[Samuel Gray]

any more than we want you to go into our code and randomly rename variables.

And don't get me started on using the word "should"

Anyone can tell someone else what should or should not be done, the point is that when code is writte?

That's how it IS being done, and a programmer should not have to worry about an IDE developer changing their code under them. There's your part and there's my part and I don't want the IDE developer rewriting my code by changing their IDE after the fact. If you want to deprecate your IDE features, that's one problem. But don't tell me today that this is how the IDE works and then change the way the IDE works tomorrow. Deprecation is merely a way of telling developers that this is how it works today but at any time of our choosing we're going to break it. And not deprecating a call is what?

That means that it will always work the exact same way? Of course not.

Ultimately programmers cannot develop code of any long-lasting value when IDE developers are free to make unilateral changes to the underlying IDE that negatively-affect program functionality and stability and then expect programmers to keep upgrading the IDE. At some point you just have to stop ugrading the products of such vendors and replace them, their products and their code, it's that simple. A very-good reason to stop upgrading the IDE is when they break existing code. Even the Matlab engineers who believe that this idea makes sense realize the difference between "deprecating" a function without breaking the functionality, and actually breaking the functionality!

And when the inevitable moment comes when the deprecation list itself is deprecated?

Go through Matlab and deprecate functions as much as you want. When they actually stop working the way that they used to work, from the callers' perspective? Then you will have a problem. Until then you just have an issue. When you start to convert your issues into client problems, your clients will start to convert their Matlab code into other scripting languages and Matlab sales will drop off a cliff. The question is why do clients have to make this obvious to you..and then make you actually care about it, short of not upgrading their licenses?

If you're trying to force your clients to switch to open-source or to use more expensive and proprietary, specialized tools, rather than a scripting language built on Java? you're doing a very good job of it. Dropping support for and changing the functionality of existing functions is a BAD IDEA that no one should ever indulge in. Entice programmers to use better functions, sure. But don't punish them for actually using the fuctions that you actually provide that do the job that they choose to use. You know who ultimately will pay the price for that? The client that pays the programmer to write the code and who needs John, or someone, to maintain it. They will have code that everyone swears worked just fine until Matlab X came out, and then they decided to upgrade to Matlab X. John will charge the customer to come back in and upgrade the code to work with Malab X, that's work for John. Or maybe John is long gone and they have to hire someone else to upgrade it, that's twice as much work for Johns' replacement. Who looks stupid in this? The guy who decided to upgrade Matlab. How often do you think that your clients are going to pay $1k per seat to upgrade Matlab just to stay current with Matlab when this is what happens? This is what we are going through right now, buying Matlab upgrades for workstations just because we have old code on them that doesn't run on the current Matlab that some of us program in right now.

It's either we program to 3 different versions of Matlab (R2018a-R2021b) or we upgrade them all at least to R2021b.

And at some point we are just going to jump out of Matlab altogether.

We will not develop in Matlab going forward, all of our development will be in either Qt or Python.

Both Qt and Python have open-source versions of their tools for clients who don't want to invest in a full-blown up to date toolkit. What you're doing isn't going to affect me technically nearly as much as it will affect my clients, which in the long run will affect my employment options until I compensate by shifting my development out of Matlab. Like any other vendor who does anything like this, you're just driving both developers and their clients away from your product.

It's a rule that every good developer already knows.

Don't break the code that you have working & deployed trying to develop the code that you want to have working & deployed.

That simply says that you don't want to have either the clients that you now have...or their references.

And if you're not going to actually break existing code then calling it "deprecated" is pointless.

That's the mentality that has programmers avoiding C++ and OOP like the plague despite its numerous benefits over C.

First and foremost, it's word-salad that literally was developed to obscure what the code is doing.

[Walter Roberson]

@Samuel Gray

"TL;DR: I wrote the code badly the first time, and now it is inconvenient to fix it. Vendors that permitted but discouraged the bad code should be required to support the bad code ever-after, for free."

Did I miss anything ?

[Walter Roberson]

@Samuel Gray

"If you're trying to force your clients to switch to open-source"

Ah, yes, open-source. Where you trade being dependent on one software manufacturer (which is, Yes, a potential problem) for being dependent on thousands of software manufacturers.

https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/

[Samuel Gray]

...meaning no disrespect but I'm sure that Matlab has not hired its own army of developers to develop all of the software that has been added to Matlab over the years. I can think of at least one 3rd-party app that has been integrated into Matlab that features promintently in our daily programming. Is the criteria that it isn't published on Git?

And still, is it better for us to depend on "Internet Collaboration" with "thousands of developers" or just depend on Matlab?

[Samuel Gray]

""TL;DR: I wrote the code badly the first time, and now it is inconvenient to fix it. Vendors that permitted but discouraged the bad code should be required to support the bad code ever-after, for free."

Did I miss anything ?"

No, you just got it entirely wrong, probably because you didn't bother to take the time to read and understand it :)

Do I really need to explain to you what you got wrong, very wrong, in this summary, or just pay you to go back and read it in detail and then comment? I thought that's what happened when I paid for my updated Matlab license, but I see that I was wrong.

I think that we all understand what "maintaining and supporting old code in the face of change" means.

The thing is, that's your busines-model. Your clients pay for support. Some clients expect that as part of the software purchase price (in fact often it's in the contract), some clients expect it just because they felt that they were nice enough to choose your software vs others, but in the end, as a developer to some degree you owe your clients some degree of software support, as a professional courtesy. I do not want to have to tell my customers that they cannot upgrade their version of Matlab because it will break my application. Now sure, that is going to happen at some point. But why make it happen more instead of less?

Does that attitude actually drive sales? Or hinder them?

Let me think of that the next time that I am up coding at 3am trying to get an AutomationServer function to work properly.

I guess that I can either find a better way to do what I'm trying to get it to do, or I can wait until tomorrow and call in to use my unlimited Matlab support options. Which would you prefer?

[Samuel Gray]

"Did you read this thread carefully? If your "sequence of variables" mean, that the variables are a sequence, they have a relation. Then this relation should be mapped in the code by collecting them in one array. An efficient and logic representation of the data is essential for good code, but it remains the art of programming"

My last comment on here, as I think this hits the point very well and I'm not a paid advisor.

Just a man with a foot on each side of this issue.

You take 1000 people who have never used Matlab before, 975 of them are just not going to be ready or willing to make the leap from writing a series of algebraic equations and putting them in a loop, to coding in matrices.

Especially with current hardware (I started coding in Matlab in the 80s on 8086s, when it was literally just a beefed-up version of DOS, it was a big improvement over writing my programs on punch-cards and more convenient than coding in C & Fortran in vi on a Dec100 terminal with an atteched ribbon-printer), even if it's 5 million times faster? And it will be even faster if they pre-allocate the arrays?

You've got to allow for the fact that your programmers, your customers, are not machines to which you can download a thousand page instruction manual on how to code it and update their AI systems to actually make good use of such information.

A lot of tech-guys forget this.

As you are all excellent programmers I realize that this may be difficult information for you to digest.

But most people do not think like programmers. Those of us who do...should remember that it's a learning-process.

The idea behind Matlab is to help people who are scientists and engineers first, even just STEM students in training...and programmers second.

If they were actually good programmers then they probably wouldn't use Matlab for all the reasons pointed-out by the OP.

That and the fact that it isn't free.

It is free for students. That's just one part of the problem.

They don't have to pay for it and their code only needs to run for a week or two.

They wil write "bad code" and spend hours longer than it would take you to write it.

But when they are done wtih it, it will work. Do you think they expect it to stop working simply because they run it under a newer version of Matlab?

We have code at our job that was developed by a bunch of Ph.D scientists over years of time & effort.

That code recently stopped working. They were like, "holy $hit, what did you do to it? It's been working fine for years!!!"

It wasn't what I did to it. But when I told them what was now wrong with it, and why it happened, they were like "well, that's why we hired you, so we didn't have to spend our time working on it anymore".

Those of us who make our living writing Matlab code? We need that code to run for years.

I am not joking with you about this. I know of at least one major defense contractor that simply threw away years of Matlab code because it stopped working after an upgrade and they had no program resources to apply to fixing the code. They did have some young Ph.Ds who they had hired to generate data to feed into the code. Those Ph.Ds were now responsible for testing each and every item coming off their producting line and certifying it for a Major Defense Client. They had to do all that work by hand. When they found time, as they found time, they slowly developed a bran new batch of test code to feed the data into. Do you realize the financial disincentive for their developing that code instead of fixing the old code? But their argument is "we'd have to go through the old code, and read and understand it and it's very likely that we'd end up rewriting major chunks of it anyway, and besides this way we get a lot of overtime".

Now, for a Major Defense Contractor? That might fly.

But you kow who ends-up paying for all that overtime?

We do. That is why we end up spending $3T per year on defense spending.

The commercial sector simply does not have the resources for that kind of nonsense, and in the end that is one reason why that program got killed-off despite an absolutely monstrous multi-decade investment in it, and in the end that meant a lot of lost jobs and lost revenue for the state, and in the end it was yet another black-eye for Matlab.

One of the major factors behind the rise of TestStand is that it is backwards-compatible with older test software from a variety of different vendors. Including Matlab. Even Labview can process Matlab code. You want to continue to be significant players in the test & measurement market, the business market, the scientific analysis markets going forward? Don't break your clients' code.

As support engineers, if you don't want to support your clients' "old code"?

I'm sure that will be noted somewhere and eventually you'll be moved to a more-appropriate position.

[Samuel Gray]

oh and this is more of a general warning for all professional programmers:

"Ah, yes, open-source. Where you trade being dependent on one software manufacturer (which is, Yes, a potential problem) for being dependent on thousands of software manufacturers."

...perhaps but you're only as dependent on them as you want to be.

If I use some OS code, that code implies the use of the GNU public license, or some such nonsense.

That's only if I make commercial use of the code.

If I'm just using that code to learn how to do what it does, or to act as part of a larger puzzle, that is not a problem.

It's not going to break because I change the compiler. It's highly unlikely to even be Matlab code in the first place.

It may be Python code, which might not even work at all, for the same reason.

But you know what? Out of a thousand Python programmers who have posted 10,000 pieces of Python code on Git?

I'm probably going to find one that actually works with the version of Python that I have, on my specific implementation of Python on my personal computer. OS has its issues, yes. But it has one very big and helpful thing going for it.

Keep looking and you'll eventually find what you need. For free.

And you're probably going to learn something useful in the process.

All that you have to worry about then is whether it remains compatible with your distro as you upgrade and enhance it.

And unlike with Matlab or a proprietary tool or OS, when an upgrade breaks code?

That fire is not likely to be looked-on favorably.

They are not going to say, "...yes! Now we get to buy a new house!"

That upgrade is not going ot make it into the distro without serious compatibility-testing.

And if it does break someones' code somewhere, that is likely to be fixed as quickly as possible.

And if not? You just find and use something else.

And if that's a real problem for you. You back-up your config before performing the upgrade.

Even Microsoft finally learned and admitted that lesson years ago.

For Matlab all that is required is to run parallel installs of Matlab and to change the COM server as appropriate, then validate your old Matlab code against the new version.

That can be done easily with Linux by installing a backup onto an external drive, or even to a VM disk and booting off it.

But I don't have to pay to upgrade my distro, and even MS learned not to upgrade their OS every 6 months and to obsolete out their tools years after releasing them when they are no longer relevant to the current market (VS Code, anyone?).

R2018a is 4 years old now. I can deal with this easily enough. 99.9% of the people using Matlab can't deal with this.

They will just hire someone to redo the entire project from scratch. The question is on what platform will it be developed.

You guys are betting the company that it will be done in Matlab and not in Python or some other popular and free tool.

And as such this could be the final nail in the coffin for my resistance to developing in Python.

Now I don't expect Python to be anywhere near as good or as broad as Matlab is in terms of performance and features.

No more than Linux is compared to Windows.

But it doesn't have to be as good or as fast. There's always C for that. And it only needs to be good enough.

That means no more $200 licenses of Win10 Pro. And no more $1k upgrades for Matlab and the toolkits.

And it means $3k less that you need to put into software to become an independent software contractor.

You can put that into your machine instead...and for $3k these days, you can buy a very-nice machine and lots of IoT toys to program. Ok Python might be slower than Matlab, but I'm running a machine that is more than fast enough to mask the loss of performance relative to Matlab and certainly way more than fast enough for most applications. Applications that can make money for you while you're not in the mood to write code...applications that you can't run on your work hardware...I'm just saying.

[Rik]

Given the fact that GNU Octave exists, you don't even need to be dependent on Matlab if you want to run m-code.

I'm a bit confused about what you are really saying here. Using load for text files has always been a shortcut as far as I can tell. Your comments sound as if comptentently written code is ceasing to work left right and center. I don't really see the evidence for that. Are you talking about the handfull of functions that have been deprecated? Functions like histc?

Most of your comments only apply to badly written code. Why should Mathworks care that is prone to breaking? Why should you?

A colleague of mine is currently working on a tool that has been developped in C around 2014. He has had to spend several months hunting down dependencies and updating code that no longer worked. If you don't package all of your code, it is prone to breaking in a few years. Matlab code from two decades ago will probably still run fine. There is excellent backwards and forwards compatibility in Matlab for well-written code. I don't see why your arguments would apply to well-written code.

I'm not saying open source doesn't have merrits, I'm just saying your arguments seem to apply to poorly written code in any language.

[Walter Roberson]

@Samuel Gray

"Do I really need to explain to you what you got wrong, very wrong, in this summary, or just pay you to go back and read it in detail and then comment? I thought that's what happened when I paid for my updated Matlab license, but I see that I was wrong."

You need to explain to me what I "got wrong, very wrong".

But first you need to look at any significant C program intended for multiple operating systems, and look at the sheer amount of #ifdef in the headers.

Question for you: Which vendor(s) have certified a Unix operating system for Intel x86 or x64 architecture within the last 15 years? (Since, after all, if you are working on a system that is not standards compliant, then you need to configure around non-standard behaviour...)

[Samuel Gray]

"But first you need to look at any significant C program intended for multiple operating systems, and look at the sheer amount of #ifdef in the headers. "

I think this is only a problem because we don't know exactly which #defines need to be included in the program, and what values they need to have, and where to get them.

So there's a lack of standardization in C/C++ You could always write programs in a "higher" language like Python, and let the developers of Python deal with those "C-level" problems. I'm sure that MS would help you with them if you'd just adopt VS 2019+ as your IDE, no problem.

I personally agree, C is one of the most frustrating languages to program in for that simple reason. C++ is like trying to program in C after taking acid. Let's make it harder by just throwing the code at a library full of header-files that are platform-specific. But part of that is because of Microsoft being Microsoft and walking into a part of people playing a game and changing the rules and changing all of the pieces and calling it the same game. They are only relevant to OS when they are in control. This is why you have entire segments of the x86 market that refuse to go anywhere near Windows.

But the BSD crowd is just as responsible for this for their simple failure to adhere to their own development guidelines and make makefiles that spell out for even the greenest C noob, exactly how to make and install their programs. And of course even that depends on the exact build environment. The one thing that MS is very good at is glomming onto OS without making OS-style mistakes. They do however often make MS-style mistakes and that is one reason why we still use Matlab. Without Matlab you'd have a bunch of little camps of people programming in different languages with little in common. And the great thing about having Matlab on Linux now is that hopefully you guys will abandon COM. And the Matlab features that rely on COM and likewise on Windows. Much of which could be written in VBA just as easily. COM is just a desperate attempt to duplicate functionality that's been in Unix for decades and now that Canonical has partnered with MS and has Ubuntu bundled with Windows, why would you even need COM in the first place? You can bundle Matlab with Ubuntu and there you go.

You just have to figure-out how to charge for Matlab ;)

(that's what the toolkits are for!)

...ultimately the BSD crowd is elitist and doesn't really care if someone finds it difficult to develop in their personal favorite flavor of Unix. That just means more work and more income for them. You want something easy to develop in, there's always Python, right? Programming in Python is a great idea until you find out that Python also is dependent on library-file versions. And certifying code? LOL

Answer 20

[John Dzielski]

I have a question about a specific use case for the eval command. It is one I use frequently, and I would like to understand if and why it is bad. I data sets from a piece of instrumentation where either the filename or the variable name often includes some identifying string and some sort of numbering. When I write analysis scripts, I will typically assume the data is stored in a variable called something like 'data'. I will use a command like eval(['data=',namedVariable]) to assigne the value to 'data' and then run the script. I will often use the reverse of the argument to copy the processed data back to the same variable and save it to a file. These scripts are often LiveScripts and the plot titles often derive from 'namedVariable', so a function call is not a useful solution here. What is wrong with doing this? (If anything).

[Jan]

This has been explained exhaustively in this thread already. Please take the time to read the question and the answers.

Creating variables dynamically impedes the debugging and the runtime massively. Instead of storing important information in the name of a variable, it is much clearer, safer and faster to use e.g. a struct and extra fields to carry the information.

[Cris Luengo]

You posted this as a comment last year, and got an answer from Rik: https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#comment_1681482

[John Dzielski]

This seems to link back to here. Is the link correct? I tried searching various ways, include "My MATLAB..." but only this and one other item appear. Thanks.

[Stephen23]

"What is wrong with doing this?"

You are forcing meta-data into the variable names. Your data design forces you into writing slow, complex, inefficient, obfuscated code that is buggy and difficult to debug. If you want to know why, read this thread.

"...so a function call is not a useful solution here"

What is a useful solution is storing meta-data as data in its own right.

[Rik]

The link should lead to a specific comment. Give it some time to load the entire page and jump to the correct location. Otherwise, search for var2str on this page.

On to your actual point: why would a function call not work? You can use even use the inputname function if you can't be bothered to supply the metadata as a char.

Instead of starting your comments and answers with a long story about why you don't think eval is that bad, why don't you directly ask for what the alternative is for your specific use case? That will save a lot of repetitive comments and answers. You can even post them as separate questions so they're easier to find for other people.

[Image Analyst]

Why do

eval(['data=',namedVariable])

when you can do the much simpler

data = namedVariable;

Any particular reason why you don't want to do the shorter, more straightforward way?

[John Dzielski]

Let me try to better describe the use case. It's my fault for not being clear. I am not arguing with the people that are giving advice, but I don't see the solution that eliminates the need to use eval (yet, and that may be due to my inability to see it). The post from Rik to my prior question (I don't think) addresses the issue. If there is a better way to do this, I want to understand what it is.

Suppose I have a workflow where I need to specify a directory path and a sequence of files that use a pattern to define a sequence of files based on that pattern. My implementation is something like:

filePath = '~/Folder/anotherFolder';
filePattern = 'pattern';
eval(['load ',filePath,'/',filePattern,'_ending1.mat']);
eval(['data  = ',filePattern]);
% ...  Run code written assuming variable name is data  ...
eval(['save ',filePath,'/',filePattern,'_ending2.mat']);

Rik's suggestion from 10 Aug '21 (I think) doesn't help because Rik assumes that the variable name in the mat-file is called 'data' and it is not. His suggestion was to do something like S=load(file); data = S.data;. I think I would still need to use eval to define the second statement. How can I do this w/o eval?

It is certainly possible to change the code to not use eval. That is absolutely clear. Changing 'pattern' though would require editing multiple lines in multiple files, while this implementation defines it only once. I would suggest that I am more likely to make an error if I don't use eval in this case. I understand the whole problem would change if the data (the data is a large struct) in the file was not based on the name of the file, but I wasn't the one who decided to do that and I'm stuck with it. The step of assigning my data to a variable called 'data' was an attempt to eliminate the use of eval at just about every step.

The reason I said I am not using a function is because this use case involves a live script. The two lines defining filePath and filePattern appear at the beginning of the live script. The script calls multiple m-files that produce plots and the idea is to go in after running the scipts and add comments on the plots so that results can be published. These m-files are all finding the data they need to process using filePath and filePattern, and lots of people have different opinions about what filePattern should be (another thing I have no control of).

[John Dzielski]

To Image Analyst:

I have read all of these posts you are steering me to. I have not found a way to do what needs to be done w/o using eval. The solution of simply doing

data = namedVariable;

does not work because the scripts do not know what the sequence of characters namedVariable is until the user is ready to run the script, and the exact sequence of characters is different in each script; although it follows a pattern. I did not create this situation, but I have to deal with it. I genually appreciate the time everyone has spent considering this. Thank you.

[Walter Roberson]

When I write analysis scripts, I will typically assume the data is stored in a variable called something like 'data'.

Make it a function that has a parameter named data

I will often use the reverse of the argument to copy the processed data back to the same variable and save it to a file.

tosave.(VariableName) = data;
save(filename, 'tosave', '-struct')

No need to write it into a variable named by VariableName to save it to that name inside the file.

[Steven Lord]

There is a MAT-file named census.mat included in MATLAB. Even if you don't know what variables the MAT-file contains you can still access the data without eval using dynamic field names.

theData = load('census.mat');
theVariables = fieldnames(theData);
for whichVariable = 1:numel(theVariables)
    name = theVariables{whichVariable};
    % Use dynamic field names to pull out each variable in turn
    data = theData.(name);
    fprintf("Variable %s from the MAT-file is of size %s.\n", name, ...
        mat2str(size(data)))
end

Variable cdate from the MAT-file is of size [21 1].
Variable pop from the MAT-file is of size [21 1].

You don't need to use eval with save either. Use the function form.

cd(tempdir)
for whichVariable = 1:numel(theVariables)
    name = theVariables{whichVariable};
    save(name + "_data.mat", '-struct', 'theData', name)
end
dir *.mat % Show the MAT-files

cdate_data.mat  pop_data.mat    

Each MAT-file created in that loop contains only one of the variables from census.mat, even though we loaded all into the struct array created by load.

whos -file cdate_data.mat

  Name        Size            Bytes  Class     Attributes

  cdate      21x1               168  double              

whos -file pop_data.mat

  Name       Size            Bytes  Class     Attributes

  pop       21x1               168  double              

[Samuel Gray]

"what is wrong with something" always depends on who you talk to.

all I will say about this is that I have graduated from using eval() to using evalin() and assignin()

...especially when developing alpha code, where I'm just trying to get the code to work, not worried so much about performance...I don't like to use globals and I don't like to ball-up a bunch of variables in structures and pass them between functions...it's much easier to just wrap function commands with sprintf and call them against the base workspace with an evalin, without having to keep track of what variables are in what workspace or mod my i/o structures.

In fact, you know, I could write a Matlab function to do the wrapping.

That's just slightly harder than creating a list of globals and putting it into the front of every function. (#include "listofglobals.txt"? A great idea!)

Which is the last thing that I would want to do. Global variables can be changed in any function, and any function workspace, in which they are declared global. With an evalin you're telling Matlab exactly which variables in which workspace you want to work with. Then when I have the code working the way that I want it to work, I might go back and clean up some of the evals. Maybe. Depending on how much they affect performance.

And you're only going to find that out by changing the code.

chers

[Stephen23]

"all I will say about this is that I have graduated from using eval() to using evalin() and assignin()"

So you are obfuscating the obfuscation behind more obfuscation functions?

That is not "graduating", that is just hiding the problem: your approach still uses dynamic variable names, just in a more complex, indirect, obfuscated way. You wrote screeds about unrelated topics: global variables and functions and workspaces, which indicates considerable confusion about what the problem really is.

"His suggestion was to do something like S=load(file); data = S.data;."

Because that is the best approach to using LOAD in code.

"I think I would still need to use eval to define the second statement"

Nope, quite wrong: https://blogs.mathworks.com/loren/2005/12/13/use-dynamic-field-references/

https://www.mathworks.com/help/matlab/matlab_prog/generate-field-names-from-variables.html

or alternatively: https://www.mathworks.com/help/matlab/ref/struct2cell.html

There, I just gave you two better approaches. You're welcome!

"If there is a better way to do this, I want to understand what it is."

As explained in this tutorial (recommended: reading it):

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_236125

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval#answer_236127

"I understand the whole problem would change if the data (the data is a large struct) in the file was not based on the name of the file,"

Correct.

Here is a solution that assumes that each Mat file contains exactly one variable:

F = fullfile(..);
C = struct2cell(load(F));
assert(isscalar(C),'Mat file must contain exactly one variable')
data = C{1};

As an aside, note that you should use FULLFILE rather than concatenating text together to construct the filepath.

[John Dzielski]

Except for having tried to implement it. I think I see the alternatives to each of the cases that I was using eval in. Some unfamiliar syntax that I'll need to read the documentation about, but thank you.

[Rik]

There isn't any shame in not knowing a specific syntax that avoids the need for eval. Your mistake is in repeating 'I don't think there is a way to avoid eval here'. As we have now demonstrated several times, there usually is.

Please post the next one as a separate question so others can find the solution as well. (and don't assume the only way is eval)