What frustrates you about MATLAB? #2
Rik
on 28 Jun 2020
Latest activity Reply by Mike Croucher
on 25 Jan 2024
Similar to what has happened with the wishlist threads (#1 #2 #3 #4 #5), the "what frustrates you about MATLAB" thread has become very large. This makes navigation difficult and increases page load times.
So here is the follow-up page.
What should you post where?
Next Gen threads (#1): features that would break compatibility with previous versions, but would be nice to have
@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)
312 Comments
Feel free to use the 'Ideas' Channel in the new MATLAB Central 'Discussions' section for the sort of conversation you were encouraging here Ideas - MATLAB Central Discussions (mathworks.com)
>> tAdd.IP=7;
To assign to or create a variable in a table, the number of rows must match the height of the table.
>>
No automagic expansion on table assignment...another need for b-ugly repmat() for no apparent reason...and there's no convenient shorthand 1D repmat() so have to remember to not forget the trailing ",1", too.
Unless you do remember there is repelem which I almost never do. But, it still doesn't quite fit the bill because you still have to transpose the result in which case may as well make it explicit with repmat() to start with.
Frustrating....
Actually your second example is more nearly the use case I had that prompted the complaint although I simplied it to just show the error. The use case of
t = table([1;2;3])
t=addvars(t,7,'After','Var1','NewVariableNames',{'NewVar'})
The specific case involved putting the new variable in a specific position within a table with a number of variables, not at the end that the t.NewVar(:) syntax does. While it does avoid the explicit vector creation, one then needs a second line to relocate it where needs to go.
I still think with the table class, both forms should be available...
With the table the height is known internally so it should be possible to simply expand the constant to the length needed behind the scenes, so "way back when" I tried to do that.
For some reason the analogous array syntax never struck me over the time the table has been in existence (and for the prior dataset object in the Stat TB before it got superceded (another rant over that, but that's another topic). I THINK that is probably owing to the specific text of the error message; it tells one what the problem is but it doesn't provide the hint as to another form for the expression; it implies one must assign the matching vector -- and with that particular LHS addressing, that is true.
On retrospect, it is apparent with the analogy that the (:) addressing would work, agreed; I admit I just didn't think of it Lo! those many years ago when first ran into it and had just used the klunky workaround by rote ever since, hadn't tried to find another syntax and for some reason it just didn't strike me to try...
By "native type auto-expansion", I think you mean like this:
x = [1;2;3]
x(:,2) = 7
The analogous tabular thing is this:
t = table([1;2;3])
t{:,2} = 7
But t.IP(:) = 7 is probably what I would choose.
Ah! I hadn't thought of using the explicit (:) addressing mode on the LHS, Peter, thanks...obviously, I guess, given the rant. :)
Thanks for pointing that out, while not as elegant as the native type auto-expansion (which I think is what most experienced MATLAB users would expect) it is definitely a significant improvement over the manual vector creation route.
This will do what you want
t = table([1;2;3])
t.IP(:) = 7
but I hear what you are saying.
My preference is able to have combinations returning simple standard array (with perhaps optional input argument to do so) not table.
@Bruno Luong thanks very much for sharing your example and experience. I'm happy to see that performance of combinations coupled with rowfun is close to that of your handwritten code and not orders of magnitude slower.
As a quick comment, you likely could write code that looks like this without the need for the wrapper:
FactorValuesCell = rowfun(@(varargin) cinematic(varargin{:}, PLenErrorArray1, LensErrorVariableNames) , ...
Tgammac, "OutputFormat", "cell", "SeparateInputs", false);
and perhaps that shaves off a bit of runtime.
All of this said, if you prefer your current approach and that works for you, then stick with it! That's part of the joy of MATLAB.
When we are thinking about the implementation of T{ROWS, NONSCALAR_COLUMNS} then there are a small number of conceptual routes:
- temp = [T.data{:}]; output = temp(ROWS, NONSCALAR_COLUMNS); -- that is, convert everything into one large array first, then extract the desired rows and columns from that array
- temp = [T.data(NONSCALAR_COLUMS){:}]; output = temp(ROWS,:); -- that is, extract the desired columns, convert everything in those columns to one large array, then extract the desired rows from that array
- temp = cellfun(@(M) M(ROWS), T.data, 'uniform', 0); temp = temp(NONSCALAR_COLUMNS0; output = [temp{:}]; -- that is, extrace the desired rows from every column, extract the desired subset of columns, then convert what results into one large array
- temp = cellfun(@(M) M(ROWS), T.data(NONSCALAR_COLUMNS), 'uniform', 0); output = [temp{:}]; -- that is, extrace the desired columns, extract the desired rows from those columns, then convert what results into one large array
However, considering there is one cell for each column, it never makes sense to bother converting the data in a column that is not going to be used, so we can rule out #1 and #3.
We are then left with the question of whether we extract everything in the desired columns into an array and extract the desired rows from that, or if we instead run a function on each cell to extract the desired rows and smoosh together the extracted rows into an array.
Under what circumstances is it going to be faster to convert everything into an array and extract rows from the array? Well, it would potentially be faster to do that if you are extracting all of the rows, if you specially detected ':' as the subscript and short-circuited doing the actual indexing. What about the case where the ':' index was already converted to 1:height -- what is the performance comparison between temp(1:end,:) compared to temp(:,:) ? The answer is that if you use (:,:) then MATLAB knows enough to not bother copying any data, but that if you use (1:end,:) or (1:size(temp,1),:) then MATLAB has to make a copy of the array. So if the code detected : as the first index then smashing everything into an array first and then not even bothering to index rows out of it could certainly be faster than cellfun() to extract only the wanted rows.
... but of course if you are going to bother to detect : as the subscript, T{:,NONSCALAR_COLUMNS} then it would be easy enough to skip the cellfun if you were doing the cellfun approach.
So when does the cost of running one anonymous function per cell to select rows, and smash the results together, exceed the cost of smashing everything together and selecting rows?
It would have to be a case such as having a lot of numeric variables that are already the same datatype, but with relatively few rows in the table. When you have relatively few variables, the cost of the cellfun becomes minor compared to the memory copying with large number of rows. When the variables are different data types, the cost of converting them to the same datatype goes up with the number of rows -- you would rather convert fewer rows.
My suspicion is that the cellfun approach would be faster most of the time (assuming that not all rows were being selected with a ':' index)
I ran a rowfun last night in which each variable needed to be processed . I ended up needing to approxiimately varfun(@(varargin) cellfun(@(M)AFunctionCall(M), varargin, 'uniform', 0), TABLE) ... which was not the most obvious of interfaces, and which many people would likely not think to do.
As Peter mentioned above, there is work being done at improving the performance of subscripting on tables and the use case of extracting a single row using {} is definitely a common use case that needs to be improved.
Walter regarding the two approaches you mentioned, I think both of them are valid, but both the implementation approaches yield better or worse results depending on the size of the tables you are working with, so the idea would be to implement it in a way that improves the performance for all kinds of tables. But as I said, this is something that Mathworks is actively looking into.
I run rowfun on my example of camera assembling and performance is good compare to my own combination array output.
My code:
rresolution = 4;
nlenses = 6;
gammac = AngleComb(nlenses, rresolution); % 4^6 x nlenses = 4096 x 6
ncomb = size(gammac,1);
tic
for i = 1:ncomb
gc = gammac(i,:);
% cinematic is process of simulation I cannot share
% PLenErrorArray1, LensErrorVariableNames is variables defined above
FactorValues = cinematic(gc, PLenErrorArray1, LensErrorVariableNames);
end
array_time = toc
function c = AngleComb(nlenses, rresolution)
% c = AngleComb(nlenses, rresolution)
% Generate an array of combinations of rotation angles (degree)
theta = (0:rresolution-1) * ((360) / rresolution);
c = cell(1, nlenses);
[c{:}] = ndgrid(theta);
c = cat(nlenses+1, c{end:-1:1});
c = reshape(c, [], nlenses);
end % AngleComb
I try the table access using combinations like this
theta = (0:rresolution-1) * ((360) / rresolution);
carg = repmat({theta}, 1, nlenses);
Tgammac = combinations(carg{:});
tic
FactorValuesCell = rowfun(@(varargin) cinematic_wrapper(PLenErrorArray1, LensErrorVariableNames, varargin{:}), ...
Tgammac, "OutputFormat", "cell");
table_time = toc
function FactorValues = cinematic_wrapper(PLenErrorArray1, LensErrorVariableNames, varargin)
gc = [varargin{:}];
FactorValues = cinematic(gc, PLenErrorArray1, LensErrorVariableNames);
end
On my PC
array_time is 0.080248 seconds.
table_time is 0.087576 seconds.
So it is very good.
I think finding the right way to use combinations efficiently by rowfun is ot very straighforward. The way rowfun accept a row as arguments list is fine but migh be not evident for peopla whos are not familiar with MATLAB. Same comment for wrapping using anonymous function if other input parameter are needed. In short finding the right path and coding is not evident. (It takes a week for several high level MATLAB users to figure that out).
On this specific camera code, I'll stay with my priginal ndgrid and for-loop. It is more readable and maintanable to me.
Walter, I think I misspoke. The correct info will follow soon.
(I think your cellfun command needs C(rows), not C{rows}.)
Thanks I'll do the simulation of camera assembly.
rowfun seems to be the way to go. This tip should be written in the documentation of combinations IMHO.
The whole point of combinations returning a table is the combinations --> rowfun workflow, as @Peter Perkins mentioned above.
For example, if you have a function that takes 6 inputs, and generates one output, you can sweep over all of them like this
f = @(a,b,c,d,e,f) cosd(a).*cosd(b).*cosd(c)+sind(d).*sind(e).*sind(f);
shift = 0:90:270;
T = combinations(10+shift, 20+shift, 30+shift, 40+shift, 50+shift, 60+shift);
tic;
z = rowfun(f, T, "OutputFormat", "uniform");
toc
tic;
w = zeros(height(T),1);
for k = 1:height(T)
onecomb = T{k,:};
w(k) = f(onecomb(1),onecomb(2),onecomb(3),onecomb(4),onecomb(5),onecomb(6));
end
toc
assert(isequal(z,w));
The expression with rowfun is more compact and more performant.
Of course, if you can get down to plain old doubles, you might be happier. Frequently, for the kinds of things people want to do, you have a variety of types running around, which makes even trying to pull out T{k,:} challenging since you may not at all get what you expect.
@Bruno Luong, if you would indulge us, I would appreciate seeing the timing of your smart phone camera example done with this kind of workflow.
If a row is what you need, T(i,:) is much faster than T{i,:}
? Is the implication that T(i,:).Variables would be expected to be faster than T{i,:} ??
Things are not as simple as "TMW should re-implement that part", for reasons that are deeper in the language
I have experimented with tables with mixed data types... but I am having difficulty finding any deep reason why output = T{rows, nonscalar_columns} should be treated internally as
temp1 = T.data(nonscalar_columns);
temp2 = [temp1{:}];
output = temp2(rows,:)
rather than as
temp1 = cellfun(@(C) C{rows}, T.data(nonscalar_columns), 'uniform', 0);
output = [temp1{:}];
The only thing I have come up with so far is that there is some weird semantics with tables containing tables... you can get it showing up like
var1 var2
1 x 10 table 1 x 10 table
1 x 10 table 1 x 10 table
1 x 10 table 1 x 10 table
but so far if you try to put tables with more than one row into the entries then you cannot seem to do that unless you make the variable into a cell. It is looking like the variable names all have to be the same within one column in this case... and doing various things collapses the rows-that-are-tables into a single multi-row table. Unfortunately between yesterday and today I have forgotten the steps that got me to that kind of table. I might have been playing around with cell2table() or struct2table()
@Peter Perkins I give you an example of a concrete "do something with onecomb" in a recent code I just worked with few weeks ago:
I work in simulating a performance of smart-phone camera assembling in a production line, and there are 6 lenses where I simulate eache lenses rotates by 90 degres. There are then 4^6 = 4096 combinations that I can (but not) generate with combinations function.
Then we have a neural network (NN) to evaluate the performace of each "onecomb " The NN last 1.7 microsecond for each combination. The 4096 combinations costs me only about 7 ms to simulate.
If I use table it will take 7 seconds to access the row by T{j,:}, 1000 times more than my simulation !!!
Furthermore from Walter's digging on code, as I understand each memory foot print of the statement
onecomb = T{j,:}
is like the whole extra content T{:,:}. When I think about using the table and then each row access manner the table is temporary rebuilt, I just scratch my head and ask myself: why not getting the T{:,:} at the first place and work directly from that.
This is again a though provoking from me.... ;-)
PS: Timing in code in script and function doesn't matter much at least in R2023b.
CRIMINY! I can't believe I didn't mention rowfun!
combsT = combinations(param1,param2);
[results1,results2] = rowfun(@simulationFun,combsT);
Wow, this subthread got real long real fast.
Walter (and Bruno), I was interpreting Bruno's "Or output as cell array" as meaning "Or output as cell array with one value in each cell". You are correct that "Or output as cell array with one column in each cell" would be equivalent to how table stores the data. (yes, struct shows you this. You guys know this, but for the record: DON'T RELY ON WHAT YOU FIND BY DOING THAT!!! THE INTERNALS ARE GUARANTEED TO CHANGE! GUARANTEED!) In any case, what would you then do with that cell array? Getting one "row" means using cellfun or a loop to dig into those columns. Not much fun. Getting a scalar struct of vectors, same deal. Tables let you slice both ways.
Bruno, this is a really tought-provoking topic. It will definitely give us things to think about. Things are not as simple as "TMW should re-implement that part", for reasons that are deeper in the language, but there are some things for us to think about to improve "extract one row from a table". So remember: "One thing is to not assume that things never get better."
Some more thoughts:
- If you are doing timings, run in a function, not a script, and not at the command line. I'm not even 100% up to date on all the optimizations the language can do in functions vs. scripts ("One thing is to not assume ...") , but functions are how you are actually using your code, so do the same for your timings.
- "% do something with onecomb ... " Ay, there's the rub. "Something" might be really fast, but for the output of combinations, especially for the parameter sweep use case, it's likely an expensive calculation that dwarfs any table subscripting. So far all the timings I've seen in this thread include "nothing" as "something", but maybe I missed it.
- For a parameter sweep, I can imagine having some simulation, and doing this:
combsT = combinations(param1,param2);
for i = 1:height(combsT)
[result1(i),result2(i)] = simulationFun(combsT.Param1(i),combsT.Params2(i));
end
which is almost certainly gonna be fast enough and is super readable.
- As I said, T.Variable and T.VariableName(i) are the fastest operations to extract data from a table. If a row is what you need, T(i,:) is much faster than T{i,:}, but of course if a numeric vector is what you need, then you'd need to turn t(i,:) into that.
- I mean, without knowing what's being done with each of those rows, this is somewhat of an abstract discussion. As Steve said,
combsT = combinations(param1,param2);
combsX = combsT.Variables;
% then a tight scalar loop
would be my suggestion in tha absence of more details.
OK now the debugger goes there.
I see now what it is damn slow when single row of a table extracting
TMW should re-implement that part.
extractData is only used if there are multiple variables being extracted at the same time. Your example happened to use a single variable.
Can you please run this
dbstop('in', fullfile(toolboxdir('matlab\datatypes\tabular\@tabular'), 'extractData.m'))
T=table(rand(10)); r=T{1,:}
does it stops?
I mean this:
%%
ThisCode % will run if you hit ctrl+enter (or cmd+enter)
%%
This is very handy during debugging, but can introduce unexpected behavior in edge cases.
I don't know what is section, I run from command line window.
Did you run from a section? Because in my experience breakpoints sometimes don't work as expected when I run code from a section.
Are you sure? I do
edit('C:\Program Files\MATLAB\R2023b\toolbox\matlab\datatypes\tabular\@tabular\extractData.m')
then put the break point at the first line of the function "vars = t.varDim.subs2inds(vars);" and run this
T=table(rand(10))
T{1,:}
it doesn't seem to stop at this file.
toolbox/matlab/datatypes/tabular/@tabular/braceReference.m has the code.
It uses toolbox/matlab/datatypes/tabular/@tabular/extractData.m to try to [ ] all of the selected cell columns together and then it extracts the desired row from the result.
... Definitely not the way I would have coded it.
Ah yeah but then data{1} migh not be the right class.The right class is the largest numerical (double) IMO.
Anyway this is not the point of my post, the point is I wonder how table row access is implemened so that is so slow.
No, at the point you initialize onecomb, j is not initialized.
Yeah it's betrter but
...'like',data{j}
rather than data{1}
onecomb = zeros(1,nvars);
That should probably be more like
onecomb = zeros(1,nvars,'like',data{1});
to avoid unnecessary class conversions.
@Peter Perkins "inefficient row access": you are correct that selecting one row of a table is not as fast as the same thing on a numeric array. Never will be. Much more going on. Is it faster than the equivalent operation on data that are stored as separate arrays? Maybe not at run time, but certainly it is faster at code compose time. Is it faster than the equivalent operation on a cell array? I forget,"
This code show it is VERY slow access table rows, and it is worse than accessing column cell. Do I miss a better method?
a=(1:3);
mctarget = 1e4;
p = ceil(log(mctarget)/log(length(a))); % == 9
%% All numercial data
c=repmat({a},1,p);
BigT=combinations(c{:});
mc=height(BigT);
tic
for k=1:mc
onecomb = BigT{k,:};
end
ttable = toc
% This will access to "internal" data of table, which has each column
% in a cell, and rebuild the all table rows
tic
StructFromBigT = struct(BigT); % Thanks Walter
data = StructFromBigT.data;
nvars = size(data,2); % == p
for k=1:mc
onecomb = zeros(1,nvars);
for j=1:nvars
onecomb(j) = data{j}(k);
end
end
tStructFromBigT = toc
In general struct() of an object reveals its properties, including its hidden properties. This is inherited from the days of schema.m when a MATLAB object literally was a struct that had been "blessed" to have a class name attached to it.
I have encountered a small number of objects that struct() could not be used on, but it works the great majority of the time.
T = combinations(uint8(1:50), ["r" "b" "g" "c" "y" "k" "h" "s" "v"]);
struct(T)
So yes, the data structure does have a cell array with one entry per variable. And there is a fair bit of overhead in code such as toolbox/matlab/datatypes/tabular/@tabular/dotReference.m . I cannot see any reason why using T.VARIABLE(index) would be faster than T{index,Offset} considering all of the checking going on.
@Walter Roberson "Combinations could reasonablly store each column in a separate cell entry. Each column is the same datatype."
I suspect that is actually pretty close to table internal storage. But no one in the TMW staffs is willing (allowed) to disclose it.
@Paul thanks for the link of the blog. I was not aware about it.
The argumenet "most design team prefers table" or bump into syntax confusion for optional argument is a little bit weak IMHO.
Sorry, when I said wasteful storage I compare with pure numerical array (uniform data to be combined) not mixing type. In that case any storage including table is wasteful. But I agree that the extra memory of table is negligible.
I cannot imagine user want to access to columns of the combinations table, most likely the ROWS of the combinations where all the combinations would be needed. Acessing row of table is PAINFULLY slow (sooo slow that the server cannot run my demo code bellow, I put the results run on my PC) and sometime it has a dangeruous trap in mixing data type (convert numerical data to string)
a=(1:3);
mctarget = 1e4;
p=ceil(log(mctarget)/log(length(a))); % == 9
%% All numercial data
c=repmat({a},1,p);
BigA=BrunoComb(c{:});
BigT=combinations(c{:});
mc=height(BigA);
size(BigT) % 19683 x 9
%% Time accessing rows, numerical types
tic
for k=1:mc
onecomb = BigA(k,:);
% do something with onecomb ...
end
tarray = toc % 0.0076
tic
for k=1:mc
onecomb = BigT{k,:};
end
ttable = toc % 7.6962
% So at the end we better do the conversion from table
% to standard array to work with, like this
tic
AfromT = BigT{:,:};
for k=1:mc
onecomb = AfromT(k,:);
end
tarrayfromT=toc % 0.0081
%% Mix data
c{end}=["apple" "orange" "cherry"]; % we mix string to numerical
BigA=BrunoComb(c{:});
BigT=combinations(c{:});
mc=height(BigA);
size(BigT)
%% Time accessing rows, numerical types
tic
for k=1:mc
onecomb = BigA(k,:);
end
tarray = toc % 0.0289
tic
for k=1:mc
onecomb = BigT{k,:}; % But this is NOT what user wants due to casting
end
ttable = toc % 264.8725 !!!!
% So at the end we better do the conversion from table
% to standard array to work with, like this
tic
AfromT = BigT{:,:}; % But this is NOT what user wants due to casting
for k=1:mc
onecomb = AfromT(k,:);
end
tarrayfromT=toc % 0.0245
%% Generate of combination output are standard array and not table
% for numerical data
function c=BrunoComb(varargin)
n = length(varargin);
c = cell(1,n);
[c{end:-1:1}] = ndgrid(varargin{end:-1:1});
isc=cellfun(@iscell,c);
if any(isc)
c(~isc) = cellfun(@num2cell, c(~isc), 'Unif', false);
else
iss=cellfun(@isstring,c);
if any(iss) && ~all(iss)
c(~isc) = cellfun(@num2cell, c(~isc), 'Unif', false);
end
end
c=cat(n+1,c{:});
c=reshape(c,[],n);
end
So what to do with combinations to overcome this?
I would guess many people use table but not fully aware about slow row accessing, and propertly convert the result of combinations to array.
"Combinations could reasonablly store each column in a separate cell entry. Each column is the same datatype."
Yes, we could have. I remember us discussing doing that. And I think everyone taking part in the discussion for this answer in this Answers post would be perfectly capable of operating on the output if it were a cell array each cell of which contains one column of the data that makes up the combinations.
But look at the people who are taking part in this discussion. The lowest "MATLAB Answers level" among participants as I type this is Level 7. [Peter and I are Staff, which doesn't necessarily say anything about our level of MATLAB knowledge, but trust me when I say we're both experts in MATLAB.]
In my experience, operating on tables (especially when you operate on them variable-wise, T.Var1 style) generally tends to be more accessible than cell arrays to newer users, people who might be say Level 4 or lower on Answers. [Yes, I know "MATLAB Answers level" isn't perfectly correlated with MATLAB skill level, but would you accept the premise that the correlation is likely positive?]
There are functions and classes in MATLAB that have a steeper learning curve and a higher point on the curve where you could say "I have learned this." Because of its purpose combinations isn't intended to be such a function. Ideally, I hope it's no harder (or not much harder) to learn than functions like sin or (basic uses of) plot. Think gentle slope rather than rock climbing wall. Did we succeed in making the combinations function useful, easy to learn, and easy to use? It hasn't been out for that long but I'd like to think so.
I'd also like to point out that even if you don't want to directly use a table array after calling combinations, converting a table all of whose variables are the same type (or types compatible for purposes of concatenation) into a homogeneous array is easy. It's just 10 characters more, using a syntax like that T.Var1 I mentioned above.
T = combinations(1:5, 6:10); % table
A = combinations(1:5, 6:10).Variables; % array
isequal(T.Var2, A(:, 2))
D = ["Huey", "Louie", "Dewey"];
S = combinations(D, D).Variables
Combinations could reasonablly store each column in a separate cell entry. Each column is the same datatype. Takes no more memory than a table would as a table internally uses cells to store the columns.
T = combinations(uint8(1:50), ["r" "b" "g" "c" "y" "k" "h" "s" "v"]);
C = {T.Var1, T.Var2};
whos T C
%timing to access one variable from each row
tic; for K = 1 : height(T); T{K,2}; end; toc
tic; for K = 1 : height(T); T.Var2(K); end; toc
tic; for K = 1 : height(T); C{2}(K); end; toc
%timing to access a row. Because of the string entry the following
%automatically convert the uint8 to string()
tic; for K = 1 : height(T); T{K,:}; end; toc
tic; for K = 1 : height(T); [T.Var1(K), T.Var2(K)]; end; toc
tic; for K = 1 : height(T); [C{1}(K), C{2}(K)]; end; toc
%but of course for wider arrays, writing the [] explicitly can become
%unworkable, so let us try some code that handles each cell in turn
tic; for K = 1 : height(T); arrayfun(@(IDX) string(C{IDX}(K)), 1:size(C,2)); end; toc
The repeated construction of anonymous functions with captured variables, and calls to those functions by the arrayfun, would be expected to be the slowest approach... but it is still more than twice as fast as the T{K,:} version.
Bruno, I'm not gonna claim your past experience is not real, but I'd like to address two of your comments:
1) "wasteful table storage/decoration": there's nothing at all wasteful about how tables store data. Your suggestion of a cell array as the output type of the combinations function in that example would not be great memory-wise compared to a table. Of course in that small example, memory doesn't matter. But a cell array would store every value in the output as a separate MATLAB array--that's how cell arrays work, they can hold anything in any cell, no homogeneity required. But it means a hundred-something bytes extra for each value. A table, OTOH, would store only two homogeneous arrays: a uint8 column and a string column.
T = combinations(uint8(1:50), ["r" "b" "g" "c" "y" "k" "h" "s" "v"])
T =
450×2 table
Var1 Var2
____ ____
1 "r"
1 "b"
[snip]
C = table2cell(T)
C =
450×2 cell array
{[ 1]} {["r"]}
{[ 1]} {["b"]}
[snip]
whos T C
Name Size Bytes Class Attributes
T 450x2 25145 table
C 450x2 160650 cell
T1 = T.Var1;
T2 = T.Var2;
whos T1 T2
Name Size Bytes Class Attributes
T1 450x1 450 uint8
T2 450x1 23496 string
25145/(450+23496)
ans =
1.0501
Not a lot of memory overhead to the table, and it disappears once you get beyond toy examples. A cell array with a few million rows? Good luck. Lemme reiterate that: memory-wise, a cell array is a terrible way to store tabular or homogeneous data. (Also terrible from a usability standpoint--what functions would you call on it? Not alot of things you can do with it.) Cell arrays absolutely have a use, but those ain't it.
What about compared to a numeric array?
X = rand(100000,2);
T = array2table(X);
whos X T
Name Size Bytes Class Attributes
T 100000x2 1601191 table
X 100000x2 1600000 double
Once you get past small data, memory overhead in the table is in the noise.
2) "inefficient row access": you are correct that selecting one row of a table is not as fast as the same thing on a numeric array. Never will be. Much more going on. Is it faster than the equivalent operation on data that are stored as separate arrays? Maybe not at run time, but certainly it is faster at code compose time. Is it faster than the equivalent operation on a cell array? I forget, but what you end up with from the cell row is not super helpful from a usability standpoint and c.f. my earlier point about memory.
Actually, I bet that you really mean "inefficient access of scalar elements" rather than "inefficient row access". But same idea.
Tables are best when you use vectorized operations. What to do about that if you can't write your code like that?
- One thing is to not assume that things never get better. Subscripting performance for tables, both reference and assignment, has gotten a lot faster over the last few years.
- Some subscripting syntaxes are faster than others. T.Var(1) is always gonna be faster than T{1,"Var1"}. Much more going on in the latter.
- Another thing is to not throw the baby out with the bath water. In a tight scalar loop, if accessing data in a table is a bottleneck, it is very often possible to hoist that part of the code into a function call, so the original loop becomes something like [t.Var1,T.Var2] = loopingFun(T.Var3,T.Var4). The rest of your code that holds data in a table stays the same.
3) You did not ask this, but performance of the new-in-R2023a(IIRC?) "math on homogeneous numeric data stored in a table" mostly compares favorably with math on core numeric array types, at least for not-toy-example cases.
In case you hadn't seen it and if interested, The new combinations function in MATLAB – for cartesian products and parameter sweeps blog post explains some of the design decisions that went into the combinations function.
I've used it extensively with the pro bono work I've done for local community college foundation financial records stored in a myriad of Excel spreadsheets and found it quite effective and not at all difficult to code with; in fact, the extra features built in with varfun and so on have been quite helpful and the variable addressing is no more complex than a struct.
Now, if one has monolithic arrays and numeric calculations galore, sure, it won't be the tool for such computational applications, but it has its place. Anything can be done at a lower level, but the convenience of some of the higher level features I've found very advantageous and speed/memory aren't issues with the size of these tables--although when compiled to a standalone app it does slow down quite noticeably, but I don't think that's the fault of the table itself but generally the compiler.
Exactly, I only use table when interact with end user, or readtable from csv file and then quickly convert it to lower level data.
Programming with Table is tedious, long, and slow. You guess it I'm not a big fan.
Maybe doesn't "need", but it surely is handy and convenient for user interactive interface.
Granted, it does bog down with really large tables.
Or output as cell array.
No serious programing needs the wasteful table storage/decoration and inefficient row access.
T = combinations(uint8([1 8 6]), ["red" "blue" "green"])
class(T.Var1)
The existing functionality does not require that the outputs are all the same data type, or even compatible data types. And that is useful. Still, it would probably be reasonable for there to be an option controlling the desired output format... maybe even the desired output class.
Having used it more extensively in the last number of months, my biggest frustrations still surround the App designer. It has improved massively since its early days, but some 'features' or lack of features are still immensely irritating for day to day usage, including (though I'm sure I forget some):
- Uneditable code - I understand why this is there, but Matlab is used by a lot of expert users as well as new users. It is ridiculous that we have no way to edit large blocks of code in an App Designer app. There ought to be an expert mode where you can simply edit anything you want, with all the risks that come with that in potentially making your app no longer work. I want to be able to edit the order of my properties and methods blocks - I always put my public ones at the top, private ones below, it's just my way of working - I don't want to enforce it on others, but I want the capability to do it myself. Also I always name my objects as 'obj' in every class I've written in over 10 years. Being forced to name it 'app' or 'comp' in un-editable function signatures is infuriating because I still just naturally use 'obj' in the code body without thinking and it wastes time fixing it.
- Public components - part of the same problem as the first point. I very rarely want all the components of my app to be public properties so that any external code can just mess with them as they choose. I want them private, and again, just having the option to do this is all I need. If other people want theirs public then fine, let them do that too, but at least give a choice to edit this.
- Save/debug behaviour - why does this behave differently to in the regular editor. It is so irritating and time-wasting when as a user of Matlab a way of working is ingrained in me from 17 years of working in the software and then this 'new' editor behaves differently. In the regular editor, when you are in debug mode you can edit a file and when you hit 'Save' it stops the debug and saves the file. I use this literally all the time when fixing bugs from a breakpoint. In App Designer you simply cannot save in Debug mode. You first have to click to stop debug and then save. Of course it's a small thing when you do it once, but when you are doing this 10s of times per day the inconsistency between the two editors seems totally un-necessary,.
I don't personally use any others these days. I used to program some using C++ and wxWidgets in the old days, but more recently only in Matlab. Previously within Matlab I would use GUIDE a long time ago and, more often, programmatic GUIs using java components and the GUI Layout Toolbox from File Exchange.
Edit Rik: added link to GUILT.
What other software/languages do you use to develop GUIs?
How slow getdata from the image acquisition toolbox is (at least for genicam cameras I have been working with). I had to drop matlab for our real time computer vision applications because of that.
Compare sldd files and publish a HMTL report using Matlab script:
I can compare .slx, .m, etc. files using the visdiff function and publish a report using the publish function like this:
compare = visdiff('File1.slx', 'File1_changed.slx');
publish(compare, 'format', 'html');
But I can´t do the same with .sldd files:
compare = visdiff('File1.sldd', 'File1_changed.sldd');
publish(compare, 'format', 'html');
Error using visdiff
Unable to compare 'C:\basic\File1.sldd' and 'C:\basic\File1_changed.sldd' without opening the Comparison Tool.
Why if I can generate an HTML report using the Comparision Tool in the same way, both for .slx and .sldd files?
I like matlab a lot, but the GUIDE can be an exercise in frustration with a lot of seemingly 'you can't get there from here' impediments. I've been using GUIDE since 2015 and if I had it to do over I would not have chosen it. Some simple things like displaying labels and titles for axes can be impossible no matter how many times you call the 'Visible' property. If one axes in a set of axes gets pushed back behind other axes using send to back, it can totally corrupt your layout even if it looks good in the fig file. Toolbar editing is a mess where it's very hard to create a user defined tool that is on an equal footing with the matlab built-in toolbar tools. Every once in a while you have to clear out the unnecessary callback functions auto generated in a 3000 line gui file. The handles object is it's own can of worms and can cause lots of frustration when not properly set and retrieved, a very weird set up for a global variable.
GUIDE was a great idea and could have been made to work much better than how it turned out.
If I knew then what I know now I wouldn't have used GUIDE, but they aren't paying me to change it, so I'm stuck.
For my next venture into a full blown signal processing app, I will take a serious look at Python, I've been doing a lot in Python lately, though not signal processing, and really like the language.
There is a reason I created this thread. GUIDE has been outdated for the better part of a decade (perhaps more). There is a reason Mathworks is removing it except for extremely ardent fans.
Python is only popular in the machine learning community because MATLAB isn’t free. Back before 2013 or so MATLAB was the standard programming language for machine learning work.Python would probably still be a fairly popular general programming language though. It was well on its way there before the scientific Python stack came about (and that’s a large part of the reason why the scientific Python stack was developed).
@Zoltán Csáti Exactly ... !!! MATLAB needs wrappers for many high-performance libraries developed during last years. TensorFlow is one good example from many others.
@John "Python is only popular in the machine learning community because MATLAB isn’t free." And because MATLAB does not have wrappers around the high-performance libraries that are written in C++ (e.g. TensorFlow), hence it does not scale well.
@Walter Roberson Are you sure, that significant part of Scientific Python development is realized by people in developing countries? From my point of view is total majority of SciPy developers from US and Europe academic community. And yes, in the whole FOSS community, which is dominantly not oriented on scientific computating, the situation is different ...
A lot of the Free Open Source Software (FOSS) community thrives by exploiting the labour of people in developing countries to work for little to no pay, in hopes that their contributions will be recognized by someone willing to hire them.
It would be nice if I could open a file in the editor as read-only, click somewhere to set the editor for read only after a file is open, and have a file open automatically as read only if opened due to a break point in the debugger.
There are at least two reasons not to set the matlab installation to read-only:
- license files for Windows are stored inside the installation directory, and if it were read-only then it would not be possible to activate windows without root access
- saving the matlab path writes into the installation directory by default and most people don't know to take the step to save the path elsewhere
I did not say that these are great reasons, and they still allow for the possibility of making most files read-only.
Unfortunately it is not rare for windows users to routinely run as administrator because they installed as administrator or because they are trying to get around the activation problem.
MacOS, the default for the most common license type writes the license into a user directory instead of into the installation directory, and I would have thought something similar could have been done for Windows...
The dbstop into a Matlab toolbox file is a big issue for me. As discussed here, a related issue is that when the debugger opens a file the focus goes to the editor, not the command window. The focus should be in the command window. Forturnately, I've never saved after inadvertently modifying a toolbox file.
Annnnnd now I just jinxed myself ...
Is there any reason to not set the Matlab installation to read-only at the OS level?
Omg, +1 on this. As an interpreted language where you just deploy source code out to production for execution, this gets messy. Especially because Matlab's standard library itself is shipped as M-code, and depending on how you deploy it (and sometimes MathWorks Tech Support will actually tell you to do this!) you end up with a Matlab installation whose *.m files are writable by users who are running it. And sometimes you do a dbstop if error and it brings you up into a Matlab function, not some of your own code. I've encountered cases where I've had a user trying to debug some code on a shared server, dbstop'ed into a core Matlab file, fat-fingered something, and of course when the Matlab Editor showed them a scary prompt that said "Do you want to save your changes before closing?" of course they hit yes, and they fuxxored the core Matlab installation for the whole server.
I've hosed my own Matlab installation before by accidentally saving a change to mdbstatus.m because that damn thing pops up all the time when you have dbstop if all error set, and I have Admin rights on my VWSes.
This isn't helped by the fact that the Matlab Activation Tool on Windows requires users running it to have local Administrator rights, even though it doesn't actually need it for any of the file modifications it does (from what MW TS has told me).
You could always write a wrapper:
function readonly(fn)
Q = matlab.desktop.editor.openDocument(which(fn));
Q.Editable = false;
end
I couldn't find a doc page for view(). Well, at least not for view() as used in this context.
Thanks for that information. Once I typed
help matlab.desktop.editor
I was able to navigate around the help. But I couldn't find a doc page for the API. I might have to start using that API approach, particularly when I open Matlab toolbox files. Still, it would be easier to just have something like
edit why.m -readonly
The first and second are possible using the Editor API. I'm not sure if the third is possible.
Q = matlab.desktop.editor.openDocument(which('why.m'));
Q.Editable = false;
Now try editing why.m. You will not be able to; MATLAB should make the error sound if you try.
Q.Editable = true;
Now you can edit it. [If you do, I recommend undoing the changes immediately afterward.] For more information:
help matlab.desktop.editor
methods(Q) % See what you can do with the Document
You could make a shortcut that toggles the Editable property on the active document and a function that includes the first two lines above to use instead of edit.
You can use the command "view" to open read-only if I recall correctly. However I do not know any way to toggle over to read only once open.
I think the function call overhead may frustrate some. Until last week, I was never bothered by the infamous function call overhead. I saw people complain about it before, but their test were never realistic to me.
I spent the last week refactoring a class for an inspection program. My goal was to split the methods into smaller methods first and then split the class into multiple classes second. After splitting methods into smaller methods, I realized that my program was significantly slower. Real world test showed that my program took 9x longer to complete an inspection on average. I was able to knock it down to 2x by optimizing the code. Let’s just say, I am second guessing refactoring going forward.
I know maintainability should supersede speed in most cases. This program is already in use. Decreasing the throughput would decrease the cost savings.
Quick Update: 2021a is defnitely faster. Great Job TMW.
Nevermind. It seems like I need to try for 2021a
Last fustration:MATLAB COMPLIER SDK implies MATLAB Advertisment.
If [blank] spends XX,XXX dollars extra to purchase the MATLAB Compiler SDK+ Required ADD ONS to export thier appliaction,[blank] should be able to replace the MATLAB icon with an icon of their choosing. Icons referes to the icons for figures and dialog boxes.
[blank] did not pay XX,XXX to advertise MATHWORKS
MATLAB makes large projects difficult to manage.
Reasons
- OOP - No multi class files. This combined with the other issues prevent me from using ver the "Single-responsiblity Principle".
- Namespaces- No within file name spaces. No within file imports
- File Search: As of 2021a the in file search and cross file search is still a joke. there is no continuous search. This adds a lot of time to development. I basically use the current folder panel to jump between files. Also the search bar should be a panel like in most IDEs.
- Tab Completion: Originally the lack of good tab completion for python and MATLAB was attributed to the dynamic nature of the languages. Now python has Tabnine and Kite. Matlab has nothing.
- Function Input Hints: Spotty.
- Function Documentation when you hover with mouse over function:
- No Syntax code highlighting: At least highlight the MATLAB functions.
- No intellisence equivalent.
- I could go on.
2. namespace, matlab does not use import, namespace is a kind of elegance, which is different from other languages, because the pursuit of "concise", but does not affect the duplication of function names, can be automatically identified!
For number 7, I am talking about an editor feature. I could care less about what the variable actually is at runtime. I don’t care if editor accidently highlights a variable named bwdist, sum, or size because it thinks it is an official MATLAB function. It is a good idea to know that you are overshadowing an official MATLAB function in your code.
There is an unofficial VS Code plugin that highlights MATLAB functions. I don’t understand the reason TMW cannot add this feature. https://marketplace.visualstudio.com/items?itemName=Gimly81.matlab. .
The SRP is about 1 responsibility per class not per file. The SRP does not discourage programing languages form having muliple classes per namespace or multiple classes per file.
If you have multiple classes in the same file, then you have more than one reason to modify the file, which violates the Single Responsibility Principle anyhow.
8. It looks to me that several Intellisense equivalents are included in MATLAB; https://docs.microsoft.com/en-us/visualstudio/ide/using-intellisense?view=vs-2022
7. Syntax highlighting.
MATLAB highlights keywords in blue, comments in green, character vectors in magenta-ish, global variables and shared variables in cyan-ish .
7. You cannot know whether a function is a "MATLAB function" (in the sense of the implemention being by Mathworks rather than your own or third-party) -- not until run-time.
MATLAB does not know that a name is an object method until the class is at least loaded, and cannot generally know what class an object will be until run-time. It is not clear to me that MATLAB should run through all classes and build all of the interface hints when the classes might well not be used in any thousand given sessions.
[Repost Per Recomendation]
MATLAB keeps reinventing the wheel and sometimes does a bad job . The #1 rule of programing is never reinvent the wheel.
- OOP: MATLAB OOP implementation is worse than all other well-known modern languages. Why? LabVIEW is actually worse but that is not a text based language.
- Live Editor: Users wanted a Jupyter notebook like interface for MATLAB. Instead of creating an official stable feature rich plugin for Jupyter notebook, TMW released the live editor. I don't know how TMW could checked off/ released that product in 2016. (They probably ignored tester feedback) You did not need the end users’ feedback to know that it was too slow. The input lag made me feel like I was VNC’d into a computer oversees. Also, the execution time/ for loops….. I kept trying it from 2016- 2018. I have not touched it since 2018b.
- Editor: I see a lot of complaints on the lack of modem IDE features. I agree. Before you create a new editor, I would recommend using an existing Opensource editor. I think it would be a good Idea to create a feature rich plugin for VSCODE.
Engineers always want to reinvent the wheel because it is more interesting, but it is not always worth effort. When TMW reinvents the wheel, it sometimes misses a lot of the qualities the made the original wheel great. This will overshadow the interesting features of the reinvented wheel.
MathWorks have now released a MATLAB Kernel for Jupyter. Details at Official MathWorks MATLAB kernel for Jupyter released » The MATLAB Blog - MATLAB & Simulink
I love Matlab. I has great optimization capabilities, sliding window functions (e.g., movmad.m), and most density functions I always require (e.g., ncfpdf.m). However, professionals in my field have largely moved to Python for TWO reasons:
- Matlab is comparatively slow and less fluid at downloading time series over an internet connection. For example, I often need to acquire geophysical time series from a repository called IRIS (https://ds.iris.edu/ds/nodes/dmc/software/downloads/irisfetch.m/). One tool exists (irisFetch.m), but it acquires and writes out data files at a relatively glacial pace.
- Plotting anything on a global map, like geophysicla data, is slow and error prone (e.g., geoplot.m).
As a geophysicist that often needs to rapidly download, write out, and plot seismic data, these two drawbacks do slow my progress. I hope Mathworks addresses these two issues soon--otherwise, I'll need to make the switch just to communictae with my colleagues.
Size of the ecosystem is not a useful measure if there is a lot of crap in the ecosystem -- pieces badly conceived or badly implemented, or which adds little value to existing versions.
What can be important is that the ecosystem is well organized, well tested, that it is easy to find and download relevant work, and that the versioning system is easy to use. A library of (say) 100 well-chosen tools might be more valuable by far than (say) 20000 badly organized weak contributions.
A scientist actually should make sure to develop robust methods. The only saving grace of science is methodology. I say this as a scientist myself.
Now, you shouldn't be spending a month to implement a boxplot, but spending time on developing a robust method is not wasted time. You must have a robust method, how else can you trust what you intend to publish?
@Walter Roberson What I meant is that if in a given field, there isn't a well-established ecosystem, you have to program everything from scratch. For large projects, this is not possible. For instance, a scientist cannot afford to spend months of his time to develop a robust method: he needs to focus on communicating new research ideas.
This network performance issue is pretty surprising to me: My understanding (and experience; I do a fair amount of ETL in Matlab, including NetCDF stuff) is that Matlab's web and networking stuff sits on top of bog-standard lower-level (C/C++ or Java) networking libraries, so the networking and attendant file I/O behavior of Matlab is pretty much the same as any other mainstream language, and performance is a function of your internet connection and internal networking and file server performance.
The specific irisfetch.m library you linked to looks like it's built on top of a custom "IRIS-WS" Java library, which is running inside Matlab's embedded JVM, and Matlab is just doing "scripting" calls on top of that Java library (just look at all those "java.*" references in that irisFetch.m!), and some Matlab urlread calls. This sounds more like an application-specific issue than a Matlab thing to me?
This is actually the sort of thing I like working on in my hobby/open-source spare time! If you want to specifically discuss irisFetch performance, open an Issue on their GitHub and tag me (@apjanke on GitHub), or drop by the Matlab Discord ( https://discord.com/invite/bBMbNCT ) and start a conversation.
Joshua, I’m not at all familiar with this data repository, and not really an expert at geophysical data. The examples in the irisfetch manual seem pretty simple, and I think maybe downloads that you are doing are much more complex. Can you give me an example of a typical download that you would do using irisfetch, and how you would plot those data? That would help to see the problems you are running into.
I have to disagree that the size of the ecosystem is what is most important.
For example if you were to look around, you would find a lot more people created emacs modules than are creating vi or vim routines. emacs by that definition has the larger ecosystem. But that doesn't mean that emacs should be used: for most people, emacs is much slower and way more complex (not in a good way).
Another example: the most popular kind of vehicle sold in North America is the SUV. Suppose that you are an inner-city courier. SUV has the largest ecosystem, so according to your logic you should use that. But in practice, the fastest inner-city couriers for small items are bicycle couriers: bicycles can be faster than cars in the inner city. (There used to be an annual race in London, bicycle vs black cab vs private car vs public transport; bicycles won most of the time.)
Use the right tool for the job. What matters most in practice is not how "good" a language is, rather how large its ecosystem is. For most tasks, Python has a wider user-base, that is why I switched from MATLAB two years ago.
While I'm now long retired from active consulting where it was a key element, even then the "red-haired stepchild" status of Fortran with mex support lacking many new features being carried along with the C counterparts and the limited compiler support plus the archaic F77 interace and use of non-Fortran-Standard features, etc., etc., etc., ... were very frustrating.
During the exercise with the recent Q? regarding debugging a Fortran mex file I discovered that the TMW-supplied MinGW download link for gcc quietly installs gfortran at the same time. Discovering that piqued my interest again and I've spent a day or so now exploring the impasse in trying to use gfortran for Fortran-only mex files or the difficulties encountered even to used mixed language mex functions.
This preamble leads me to ask -- if TMW® isn't going to take any action regarding supporting Fortran any more diligently than at the present (seemingly reluctantly) "just barely" level, how about a GNU project to build the needed infrastructure to support gfortran specifically, and Fortran generically, to a higher level?
To complete this task obviously would take getting a nondisclosure agreement in place and having access to the necessary internals to be able to resolve the details of the innards of mex, but without somesuch mechanism it appears Fortran support will, at best, remain in the dark ages and, at worst, just disappear when TMW finally decides to quit supporting it at all.
An addendum to the above proposal -- I subsequently have been able to create an XML file that mex will recognize and subsequently use the gfortran compiler -- the linking issue with Fortran object files to the provided libraries still exists, but I have some hope that with C interop this may also be solvable.
At this time I'm not yet familiar-enough with either gfortran nor C interop features to know exactly how to implement, but I think there is hope...
It would be far better to build the Fortran-linkable library entry points directly, of course, instead; that would be the end result of the proposed volunteer project if TMW® won't pick up the ball.
I presume it is simply that Fortran slipped markedly in relative popularity with the rise of C/C++ owing to the nearly 20-something years time lag from the release of the F77 Standard to F90/F95 ca 1997 as lacking in some features that changing practices and that potential new users were accustomed to being available.
With that loss of relative importance it appears TMW made commercial decisions to simply not dedicate resources beyond the minimal required towards Fortran support and that pattern has continued, but seems to also continue to be on a slowly declining path as well.
However, once the vendor cliques of that time frame dissolved a more broad-based represntation of users has had a major role in the Standards committee after F95, progress has been (by formal language standards measures, anyway) quite rapid with F2003, F2008 and F2018 is in draft with compilers already implementing some of the new features. Since adoption of a formal ISO Standard is an international affair, nothing moves extremely rapidly in the arena unlike a commercial vendor such as TMW with total control over what they choose to implement and on what time frame or open source languages/projects that have far less bureacracy in controlling the adoption of new features.
gcc and gfortran, of course, follow the relative pace of their respective standards bodies, other languages that may not have formal ISO standards are much more free for experimentation and also to make drastic shifts in course if a particular path chosen turns out to be detrimental going forward...Fortran has been exceedingly careful to maintain compatibility in as many areas as possible with only a few features having been formally declared obsolete and no longer supported (and most compilers have opted to retain those but require special switches in order to compile legacy code although use of such may prohibit using other specific modern features).
"... given that Matlab is basically built on top of Fortran, ..."
I was going to address that some initially but skipped over it in the end. Only the very first implementation was written in Fortran, even the initial commercial application was written in C as outlined in the installment of the paper Cleve Moler wrote for the ACM SIGPLAN History of Programming Languages Conference (HOPL-IV).
Cleve posted his paper in sections on his blog beginning at the beginning of the numeric software project that led to EISPAICK and LINPACK. The first of those is <Cleve HOPL2020 Paper Part I>. The description of the first MATLAB Fortran environment and the initial development of a commercial product don't come until the later installments <Historic MATLAB> describes the initial calculator in FORTRAN and the next installment the beginnings of the actual MATLAB language <pc-matlab-version-1-0>. As outlined there, the base code was written in C. (At least at that time, one would presume the internals were still the original FORTRAN implementations called from C; I have no idea and didn't see it ever mentioned about whether the libraries have been ported by TMW or not).
One can see one unfortunate side effect to this day in the input/output routines built on C stdio.h with fopen, fclose, fprintf and friends and all the warts that followed with them, particularly with the issues of fixed-width field data files that weren't really solved until very recently with the introduction of the import objects.
About the only user-visible homage to the FORTRAN roots is having kept the column-major storage order and the 1-based array indexing. With the latter, however, the ability in FORTRAN for arbitrary index base was not carried over even in the beginning owing to the shift away from having to declare variables to everything be automatically allocated and "everything is a double unless it is specifically declared/created otherwise" and that the creation/declaration happened only by a function call, not as declaration statements. Only in later releases did the option equivalent to KIND appear in such places as zeros.
In recent years, the move has been to more and more higher-level data contstructions and the introduction of OOP paradigms on top of the initial procedural-only implementation. These all have pushed the direction to C++ and the introduction of data structures beyond the simple intrinsic data types.
Even in its current state, the C interop features of the Fortran Standard don't address interop with C++; hence the difficulty in providing complete interaction from Fortran with out an intermediary layer.
It doe still, however, seem as though there should be no reason to not support GFortran to the same level as Intel or other commercial compiler(s) they should at some point decide to include.
I want to bump up the need for better GNU compiler support for FORTRAN on all platforms.
The symbolic toolbox seems to have a lot of power, but it is not user friendly. I don't understand a lot of the error messages, and the defaults for simplification are not good. I could go on and on. For school problems, this should be everyones favorite but it is not. For work problmes the same. I work for a company who pays several million per year in maintenace fees, and find myself going to the Wolfram Alfa for symbolic solutions. Please put some investment into this toolbox.
I understand that being a Mac user I'm in a niche position. But I find Mathworks® "preference" for Windows quite ludicrous. At the very least one would expect the Linux version to be feature equivalente with Windows, but that's not the case. And it's so much worse for macOS!
A lot of high end productivity features and toolboxes are just missing, specially for HDL coding. GPGPU being CUDA only—for so many years now—is begining to chafe, specially when you look ate the latest Apple Silicon (M1 Pro and M1 Max, at the time of this writing).
If you work with machine learning at any level, that's a big bummer!
The problem with OpenCL, I am told, is that it has too many optional features that are in fact not implemented or are implemented in different ways on different vendors and models. Mathworks would not have been able to provide a single OpenCL suite: it would have had to been customized by class of hardware implementation.
The OpenCL market share is just too small. You and might want the world to be different, but deep learning is mostly synonymous with CUDA. Mathworks is not setting a trend here, they're following it.
Your second point is a completely different topic: 'native support' is talking about compiled binaries that don't rely on the Rosetta 2 translation layer. That is very likely under development. It will come when it comes. If you want a better estimate you should contact Mathworks. They might be willing to tell you more under some form of NDA.
And why do you care which type of binary is used? I personally only care about the actual performance. From what I understood Rosetta 2 is pretty efficient, so there doesn't seem to be a major hurry in making the switch. But I'm not a mac user, so I only have an outsider's perspective.
OpenCL would have been a much better choice than CUDA as it supports both Nvidia, AMD and others. This was definately a mistake on Mathworks part.
I don't buy the "there isn't any acutal support provided by Apple for M1 hardware". How is it that other companies are able to release native support for their products and not Mathworks?
MATLAB will fall out of favor with Mac users if native M1 support isn't implemented with next release, perhaps that is better from a financial point of view for Mathworks, but it is sad all around.
As outlined by Walter on this thread, the lackluster GPU support (and machine/deep learning support) are not really the fault of Mathworks. Nvidia is really pushing people to use CUDA and providing a lot of support. This is also a question of chicken and egg: everybody uses CUDA, so all popular tools assume you have it. And because all popular tools assume CUDA for anything non-trivial, people are pushed to buy Nvidia hardware.
Apple and Nvidia haven't been on speaking terms for years now, so I wouldn't expect any support for CUDA any time soon on Mac OS. The other way around is also unlikely: because there isn't any market share, the changes of Mathworks biting the bullet (gambling on future market share) are slim to none.
The ARM processors Apple has designed do have excellent performance for many tasks, but since there isn't any actual support provided to hook into the hardware (at least that I'm aware of), it is difficult to design tools to take advantage of this presumed advantage.
@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)"
This kinda just says it all
Matlab has been on the market for over 25 years and this is their customer-feedback process?
How about this. Give us the source-code and we'll fix it ourselves.
@Adam Danz @Paul Hi Adam and Paul! I just started as doc staff in January and while I can see the comments submitted for a given page, I can't see who submitted them. So there's isn't currently a way to close that loop with you. But I'll flag this convo for the folks who work on processes to improve the documentation and hopefully they'll come up with a way to update a commenter. I will tell you that I view page comments as gold and sadly, most of my pages don't have ratings or comments. So please keep commenting! They are definitely getting read. (Most unique comment so far: "I am such a child")
Thanks Steven. I don't expect replies to the many documentation-feedback-submissions I add. It would just be nice to somehow communicate what happens with them. Otherwise I feel like the guy in Lost who continually pushes that button without knowing what it does.
The star ratings and the free text feedback go to the documentation staff directly. I know they read the feedback and action some or all of the suggestions / bug reports but I'm not sure what kind of acknowledgement (if any) they send to the people who submit the feedback.
I'll forward this feedback about the feedback process to a couple of the documentation staff members with whom I work.
My experience is the same. Unlike bug reports that yield a response within a few days and opportunity to have a discussion, in my experience feedback on documentation results in effectively no response. I stopped sending feedback on documentation a long time ago.
@Steven Lord, what happens when we use the documentation feedback? I must have submitted recommendations and supportive messages 100+ times using the documentation feedback system and I've recommended the feedback system to others but I have no idea what happens after an idea is submitted.
I also never know how to use the star ratings. I usually just select a rating just to get to the comment field.
MATLAB Answers is one way that users can provide feedback to MathWorks.
If you have questions or feedback on the documentation you can also use the "How useful was this information?" rating system at the end of each documentation page (which allows you to provide free text questions or feedback after you give a star rating.)
I really dislike the aggressive scalar expansion. I realize it is convenient for column-wise manipulation of matrices that are really tables of data, but it isn't worth it.
If I accidentally add a row and column, I used to get an error, but now I get a matrix. I think that is unlikely to be what anyone wants:
R2015b:
>> [1 2 3 4]+[5;6;7;8]
Error using +
Matrix dimensions must agree.
Now:
[1 2 3 4]+[5;6;7;8]
ans =
6 7 8 9
7 8 9 10
8 9 10 11
9 10 11 12
"The first time i was burned by row-plus-column, I added more input checking to a lot of my utiltiy functions that take either rows or columns. I don't think that means they had "programming faults" before."
You previously relied upon the inputs being compatible, or that MATLAB would produce an error that you expected the user to be able to decode in context. If not a "fault" then at the very least it was a "weakness".
In my own code, it is not uncommon for me to fail to validate that an input is a vector. However, I try to avoid making assumptions about whether it is a row vector or a column vector, or about whether two inputs have the same orientation, so I routinely code (:) and (:).' or reshape() calls (which has the side effect of having the code do something useful if non-vector was input.)
I find automatic binary singleton expansion very useful - lots of my code now uses it, not just with arithmetic operators but also with logical ones. (Such a pleasure to strip out the clumsy bsxfun calls!) For example, whenever you have a situation where you want to look at all the combinations of elements from two sets, it can reduce complexity in the code and provide an efficient solution. As it happens, I find it very natural and elegant, though I understand that others might not.
I agree with @Walter Roberson that a construction that was previously illegal (in that it would throw an error) is available to be put to work and made useful.
As someone coming from a MATLAB background, I was pleased to discover that Python also does the sensible thing.
Adding my two cents:
I am a firm believer that your functions should throw errors during input validation, not during runtime. There is of course the exception where a function is expected to run often and/or in a time-sensitive situation, in which case you could have the final function without input validation. (although I would still like a test suite to pick up these errors)
If your functions were throwing errors during @times instead of during input validation I would actually consider that risky programming. If row/col orientation checking is not part of your validation, then I would not consider it complete.
And as far as the clutter goes: I only see the call to my function, not the checking its doing. In general I don't even see that checking when editing my function, as I often factor it out to a separate input parsing function.
Well, I did qualify my statement accordingly.
FWIW, I still have to use bsxfun() for everything other than on the forum, since I still want stuff to work in older versions.
I think this is the crux of the matter:
" I'm not going to complain much that I get to use something that's more readable than a bunch of bsxfun() clutter. "
I agree the clutter of "bsxfun" or "repmat" makes it harder to see what is going on.
So, if you do a lot of row-to-column expansion in your typical work, you will like auto-expansion for the clutter reduction.
I don't do a lot of row-to-column expansion, so I dislike it for the added clutter of variable shape checking.
Those of you trying to suggest that those with other preferences are objectively wrong ought to check your assumptions.
I know a lot of people don't like it for their own reasons, but if opinions matter (and I'm sure mine isn't on the list of those that might), generalized array expansion is the single feature for which I tolerate how much worse R2019b is versus R2015b (in terms of memory usage & stability). I'm not going to complain much that I get to use something that's more readable than a bunch of bsxfun() clutter.
I bet it might confuse new students who never were forced to learn what operations require expansion to work, but I'm pretty sure there's no bump small enough that someone won't trip over it.
"You wrote, "I think that is unlikely to be what anyone wants:" and I showed you that it may well be exactly what is wanted."
Sure. That was an unfortunate bit of hyperbole on my part. I know perfectly well that the peope who added it wanted it.
You said "Relying on a programming language declaring an error to catch a programming fault has always been risky." Well, no. I do not find your single vs double quote example convincing.
The first time i was burned by row-plus-column, I added more input checking to a lot of my utiltiy functions that take either rows or columns. I don't think that means they had "programming faults" before.
That it is "fair game" is, of course true, particularly for a proprietary language like MATLAB that has no external Standards body such as Fortran, C, etc., that control their evolution.
That needing to approve Standards is a very time-consuming process and only allows for globally-accepted changes to be incorporated and so certainly constrains the speed and direction in which those languages evolve.
But, just because TMW can do something doesn't mean it is necessarily a good idea -- we have a multitude of examples of relatively recent introductions that have cluttered up the workspace and that are now deprecated but can't go away because once released in the wild, backwards compatibility requires they be retained.
It is, granted, a hard line to toe in a competitive environment.
You wrote, "I think that is unlikely to be what anyone wants:" and I showed you that it may well be exactly what is wanted.
MATLAB took something that was an error in all previous versions, and defined a meaning for it. That kind of expansion of a programming language is "fair game" in all programming language. Relying on a programming language declaring an error to catch a programming fault has always been risky.
The situation is no different than if you had previously relied upon MATLAB creating an error message if you used double quotes around a character vector, and were to be complaining now that double quotes have been given a defined behaviour that is different than single quotes, and wanted MATLAB to switch from using double quotes for strings into using (for example) #' and '#
#'abc'# + 1
on the grounds that you never used to accidentally use #' and '# instead of ' ' so you can live with #' '# but cannot live with
"abc" + 1
not either generating an error or else acting the same as
'abc' + 1
IMO, if TMW wanted to do this, they should have invented some other syntax similar to the "dot" notation so that the user retains control and gets expansion only where it is, as @Walter Roberson illustrates actually the wanted behavior and not otherwise.
I, too, have been burned a few times unexpectedly.
I understand the motivation. Whether you think it is worth the pain it causes elsewhere depends on what your typical work flow looks like (and maybe whether you came from a Python background).
I still forcefully object to it.
(1:4)+(10:10:40).'
That may well be exactly what the user wants.
I have deliberately used row + column for various computations. In particular, instead of doing
[X, Y] = ndgrid(1:4, 10:10:40)
Z = X.^2 + Y
I might well do
X = 1:4;
Y = (10:10:40).';
Z = X.^2 + Y
Calculating a function over a range of multiple variables is common, and happens easily if you are permitted to mix row and column vectors using implicit expansion: it saves building the coordinate matrices using meshgrid() or ndgrid()
I keep being reminded of this every time I accidentally destroy my own work or have to process someone else's crusty images.
If the settings used for imwrite() and print() for JPG output are 75% quality and 4:2:0 chroma subsampling, then JPG output is essentially useless for the typical image content they're asked to handle. Users who think they have an understanding of the compromises they're accepting when using JPG likely have no idea that they're opting for such abnormally low-fidelity settings.
Despite the integrity of my WAN connection, this isn't 1998 anymore. If options to use a reasonable set of output parameters can't be added, then the perilous nature of the defaults should be clearly documented. While Q can be set for imwrite() output, I don't know that it can when saving figures/axes. As far as I know, the chroma subsampling isn't documented anywhere and is not user-selectable.
Everyone expects JPG to be lossy,
In my experience, it is common for students to expect that JPEG is lossless.
On occasion, a student pretty much demands that mathematics work differently so that JPEG becomes a lossless compression...
Submit enhancement request(s)...
@Walter Roberson I tend to avoid it like a disease, just to avoid the chance that I absentmindedly save something important with the file extension selected from the prior figure. This is in direct conflict with my desire to make images for the forum small -- which is of extreme importance due to my bad connection. Even at that, I routinely run across plots (e.g. surf() or mesh()) which turn into garish pixel salads when saved as JPG. In other words, this stopped being appropriate even for web-only output sometime about 20 years ago.
In other contexts, I have little problem saving non-archival images in JPG format. GIMP's default is 90% quality with no subsampling (4:4:4). The difference in apparent quality is dramatic, and the chroma subsampling is an extreme contributor to that. Much of my casual time with MATLAB in the last decade has been using it as an augmentation of GIMP for post-processing digital paintings. How was I to know that using imwrite() with Q=90 wouldn't produce the level of quality I was used to accepting? MATLAB is a technical environment; quality probably matters more here than in GIMP, right?
By no means am I suggesting anyone desire to do technical image processing using lossy JPG, no matter the settings. That said, we both know the utter regularity with which users request help with trying to process images saved as JPG -- often unnecessary images of figures of images. People are going to use it anyway, whether it's simply because they don't care, or because they're conscientiously accepting an expectation that's not met by reality -- because they have no way to know otherwise.
TL;DR: Everyone expects JPG to be lossy, but undocumented defaults like this are sufficiently abnormal that I regard it as a noteworthy betrayal of those common expectations.
The baffling point to me is that jpg is the de facto standard in deep learning, even when actual values matter (e.g. in medical imaging). I don't think it should be, but this doesn't help.
I deal with this by only ever using .jpg if I am generating images to post here -- images intended to give the impression of what is going on without having to be the bit-for-bit final array.
... to clarify, I wouldn't use JPG for research work even if it had a different default subsampling or different default Quality. If I want lossless then I use a lossless image format, not jpg.
@Wolfgang Do you have a specific issue in mind? You can use the txt input if you want to standardize things. I have only had issues with unsupported installations (e.g. installing 6.5 (R13) on Windows 10).
I would agree with you when talking about portability (having the option of a completely USB-portable solution has advantages).
The worst of Matlab is the installation. For me it is each time a nightmare. I loose hours for nothing. It is the reason why I use Octave on simple tasks even so I own three licences for my reserach team. - And I am the lucky owner of a old licence of MathCAD. Always works and less troublesome.
Currently matlab lacks Tsai's calibration method integration, only Zhang Zhengyou's calibration method.
Currently I can only calibrate one image and cannot take multiple viewpoint images with a checkerboard grid, with non-co-planar 3D points and two-dim image points corresponding，What can I do?
- http://people.csail.mit.edu/bkph/articles/Tsai_Revisited.pdf Tsai, Roger Y. (1986)
The main page for "Graphics" https://www.mathworks.com/help/matlab/graphics.html has a link for "Graphics Objects" https://www.mathworks.com/help/matlab/graphics-objects.html that is not emphasized (or hidden either). The Graphics Objects page shows the hierarchy of graphics objects, and has a link for Graphics Objects Properties https://www.mathworks.com/help/matlab/graphics-object-properties.html . On that page under Top-Level Objects, there is a link to Axes Properties https://www.mathworks.com/help/matlab/ref/matlab.graphics.axis.axes-properties.html which contains information about Position Properties.
However, at the moment at least, I do not see a "User's Guide" that specifically talks through such matters.
For example, there is no link to that information from plot() .
For modifying the properties for an individual plot(), the link to the information used to be more obvious. Now it is why down under the description of the output arguments, https://www.mathworks.com/help/matlab/ref/plot.html#btzitot-h
I guess I'd ask for some examples of what you've tried to change that you couldn't find documented or how to do...to try to understand how to perhaps add some guidance.
I've not found it that hard for most things want commonly to do, then again, much of that comes I'm sure from 30 years with MATLAB by now, but the properties for either a window or an axes list everything that's there; sometimes there are convenience functions, knowing about those may be a little more iffy.
The frustrating thing for me is that there is the (relatively) recent trend towards adding new features by encapsulatiing or repackaging existing functionality in new objects and the penchant inside those to make things opaque or even black so one doesn't have access to pieces-parts that used to have. Just answered a Q? 20 minutes ago about how to turn off the Date display on a datetime axes that is doable, but it is a hidden property that one has to access to be able to do so, so it is undocumented and one has to know enough to know how to go "handle-diving" to find out what the property was called in order to access it. Why hide it? Sometimes, even, though, they've not exposed the needed guts at all and one does run into an entirely black hole.
Why are the plot settings so difficult to find. For example, if I want to change the size of a plot window, I have to scour their webpage to search through each of a hundred different ways that plots can be modified. Many don't seem to be formally documented (at least not that I can find). They are embedded in responses to the help center and I have to read 50 of those before I find what I'm looking for.
I've had to create my own help center just to record all the features related to plot so I don't have to search for it next time I need it.
Something as commonly used as the plot feature should be quick and easy to use and it's worth their time to clean up this mess of features related to plot.
Some time data comes as single, but text just rejects it
>> x=single(0);
>> y=single(0);
>> text(x,y,'Why single is not supported?')
Error using text
First two or three arguments must be numeric doubles.
I'm not certain offhand when this started working, but using single data as the coordinates in a text call works now.
z = single(0.5);
text(z, z, 'Hello!')
Support for single is intermittent. The documentation for text lists the supported types: double, categorical, datetime, and duration. There isn't any particular reason why singles are not accepted, but most functions will convert to double internally for numeric formats anyway. If I recall correctly sprintf and dec2hex don't do this anymore, but at the very least they used to. For some reason bwdist returns the output as a single, so there are some quirks with data type at times.
Support for non-doubles has gotten better over the years. In Matlab 6.5 (R13, June 2002) you could hardly do anything with the integer data types (not even plus() is defined). As far as I can tell you could only really cast back to double before you could do anything usefull. I have not been able to tell what the point of (u)int64 was in R13. Maybe a veteran user like Jan can educate me on this (I suspect you need a mex wrapper to get all the bits in an int64).
So many of the nits in graphics are just irritants/time-consuming to make things look decent --
- The inconsistency by using '%g" as default numeric format -- variable number digits is just bxxx-ugly,
- Can't turn off ticks on opposite side axes with multiple axes except by box off -- that then leaves no outline so have to draw it on boundary manually
- Still no builtin hatching patterns
- Only 8 named colors. Really, in 2021???
- Similar limitations for markers.
- Can't set xlim datetime on new axes unless first draw with a datetime. (The way to change is undocumented)
- Putting datetime and 'XAxisLocation','top' results in clipping year; doesn't adjust for needed added room
- And on and on and on...
Deeplearning Toolbox(R2020b)
other related-frustrate questions link(no answers):
It seems that Mathworks is not interested in many of the improvement questions raised by me, and I regret that there is still no answer!
MATLAB Answers is not an official support forum. You should not expect anyone from Mathworks to reply about tools that might or might not be planned.
The regular volunteers (most of whom are from outside Mathworks) do not happen to do much Deep Learning work. I know that I have never heard of most of the projects that you mention, so I have not had anything to contribute to your discussions.
Frustrating that arguments blocks can't be used in nested functions - just when you need them, and no obvious reason for this limitation.
Also - and I know this has been commented on elsewhere - having a nontransitive isequals is a high price to pay for making char and string more interchangeable. (Nontransitive as in isequal('a', "a") and isequal('a', 97) both return true, but isequal(97, "a") returns false.)
On the whole I welcome the relatively new string class and arguments blocks, but it isn't half fiddly changing over to them.
I have learned C before MATLAB and coming from C it's just natural that char is simply a numerical array like any other numerical array (granted MATLAB char is int16 and not int8 like C) .
I never see people having trouble with doing arithmetic with char array, you are in fact the first person who has expressed the danger of it. Either you know char array is int16 array named diffirently trougle class either you don't. If you don't then you don't try to do arithmetic with char array, if you do than it's just natural.
I simply dislike C++, object-like construction like string where the underlined structure and data are hidden from the programmer.
Yeah I have a colleague who use HEX2DEC and one day he asks me why his code cannot keep up with a data streaming from a device. Fortunately I could develop m code snipped above example to save him.
Now you know where I stand with.
The doc page for isequal is silent on how it handles inputs of different Fundamental MATLAB Classes and I suspect that some casual or new users would read that page and reasonably expect that
isequal('a',97)
to either return false or an error. Looking at that linked page, there is no reason to think that char and logical are included "under" numeric, even if that's how they are de facto handled in some situations. IMO, the documentation is lacking in general in explaining type conversions of inputs to functions and of operands for operators. Looking further down on that page, we see the phrase "Converts to/from numeric." However, that phrase applies to both "Characters and Strings," which as shown above isn't done consistently between those two fundamental classes. Is there ever a case where a string is converted to a numeric. Furthermore, the conversion for char isn't consistent across different operations:
>> ['a' 97]
ans =
'aa'
>> 'a' + 1
ans =
98
So for concatenation, char takes precedence (as documented but not easy to understand) but for addition numeric takes precedence (not explicitly documented as far as I can tell).
Do you think utility of the implicit conversion of char to numeric as in your code snippet is more than the cost of having to deal with the other issues that arise with that implicit conversion as discussed in this answer thread?
With regards to this particular example, I'm sure you're aware but in case other aren't: hex2dec and this blog post may be of interest.
"if TMW was starting from scratch based on what you know now, would the implicit cast from char to double be implemented in Matlab?"
None of TMW employee would answer this kind of question.
Personaly I'm happy to be able to code this
hex = '0x1F';
c = upper(hex(3:end));
b = c >= 'A' & c <= 'F';
c(b) = c(b) + (10 - 'A' + '0');
x = polyval(c-'0',16)
Try to do the same with string or explicit cast (and without base2dec()).
"MATLAB has long included character and logical under the category of "numeric"'
We all know that, however
isnumeric('a')
returns FALSE.
Another TMW logic that makes me scratch my head.
BTW even more "surprise''
>> s = sparse(97)
s =
(1,1) 97
>> isequal(s, 'a')
ans =
logical
1
MATLAB has long included character and logical under the category of "numeric" except where the documentation distinguishes.
To be clear, I have nothing against implicit type conversions in general. I'm not even sure I'm against Matlab's implicit conversion of char to double. But it's always looked a bit jarring to me and it can lead to what some may consider unexpected or illogical results as shown in this answer and comments.
Having said that, in 2019a
doc isequal % 2019a
doesn't really say what to expect for
isequal('a',97)
Maybe the doc explains elsewhere why that exression evaluates to true.
I haven't written a Pascal program in so long that I can't remember the type conversion rules! However, ending every statement with a ; proved to be a good habit to get into.
You know, there was experimentation with computer languages that supported absolutely no automatic data type conversions. 'a'+1 was strictly prohibitted in those languages, and so was
%{
integer I = 0
double J = 0.
I + J Attempt to mix datatypes, prohibitted!
%}
Because from theoretical point of view, if you somehow thing that 'a'+1 is confusing or a source of bugs, then adding an integer and a floating point number is confusing and a source of bugs too. If you think "Oh, but MATLAB does it right!" then you haven't thought about
uint8(17)/2
A = uint8(255)
A+1
Why aren't those 8.5 and 256, or 8 and 256 ? And why is
A = A + 1
uint8 datatype instead of being automatically "widened" to double on the right hand side, like in C and C++ -- and then because the type of the left side on an entire assignment is the same as the type of the right side, why isn't A left as double?
What's that, you don't expect datatypes to change when you do A = A + 1? But you have to expect that if you require that operations between integers and floating point "widen" like in C and C++.
Thus, if you have the goal of having consistency in operations without any surprises, then you have to prohibit operations between mixed data types, insisting on explicit datatype conversions every time... which can still lead to surprises if the programmer doesn't understand the implications of the datatypes they choose to convert to, but that's the programmer's fault, not the language's fault.
So as I mentioned, some "high level languages" with strict type checking were tried out. Tried in a big way, not just in a small study with a little focus group giving their opinion.
The result of those experiments? This: those languages are pretty much dead now. Or where they exist, it is in name only, with the strict type checking rules loosened in local hacks. When was the last time you programmed in Pascal?
ADA was designed with the goal of having strict denotation semantics, that programs written a particular way would always compute exactly the same thing on every implementation. You can see the justification for that as soon as you start thinking about implementing code for purposes such as decades-long space vehicle projects, or code for Fighters (aircraft) intended to last decades: You want to be able to be sure that you have no surprises in moving to replacement parts, as lives can literally depend on that.
ADA has remained pretty niche. And in practice they had to define conformance sections, dividing hardware and software response into areas that strict repeatibility was guaranteed, vs sections with permitted variance (within rules.)
Strict type checking just doesn't work with the way people program. Maybe they should program other ways, but they don't.
MATLAB was never defined to be a language for theoretical purity: it was designed to be a language for practical use by people who shouldn't have to know all those details in order to get useful results.
As Steven said in this comment, "we've learned a bunch about designing our software over the past couple decades." (I'm pretty sure I've seen similar statements in his other contributions), so I was curious about his views on this particular aspect of TMW software design.
I don't understand the problem of supporting 'a'+1. I'd prefer the flexibility rather than relying on double(). The programmer would have to be pretty poor to accidentally add characters to numbers and the user of a function would have to be very unaware of the expected inputs to enter characters rather than numbers and that could be caught by input validation.
> if TMW was starting from scratch based on what you know now,...
I doubt any individual could answer that question and the answer wouldn't matter, anyway. Matlab was started from scratch in the 70s based on research papers written in the 60s, the same decade the ASCII character encoding standard was created. Initially PC-Matlab was merely a matrix calculator and there were no programs and no graphics. That's certainly not how analytic programs are started from scratch today.
No, I don't want 'a' + 1 to be more like "a" + 1. I'm wondering if there is any real uitility in having 'a' + 1 do the implicit cast of a char to double, as opposed to having such an operation forced to be written as double('a') + 1. I'm intrigued that there is a substantial amount of code that relies on the implicit cast and would like to learn more about typical use cases. Presumably any code that works with the implicit cast will work with the explicit cast. On the other hand, there might be cases where the implicit cast can cause a problem to the unsuspecting user/coder, in which case the explicit cast is better anyway. Regardless, I realize the current behavior isn't going to change. But my questions was, if TMW was starting from scratch based on what you know now, would the implicit cast from char to double be implemented in Matlab?
Better in what sense? Memory efficiency, time performance, code cleanliness / understandability, etc.? I haven't measured but my gut instinct would say probably minimal if any difference in memory usage or time performance.
If you're considering suggesting that we should change 'a'+1 to behave more like "a"+1, you can suggest it but honestly I wouldn't hold my breath. There is a substantial amount of code written by users and by us here at MathWorks that likely depend upon the current behavior. To justify the cost of breaking backwards compatibility for a core operator that has behaved this way for multiple decades, I feel the benefits of doing so would have to be at least as substantial.
Is there a significant benefit to the implicit conversion of char to double? For example, is
'a'+1
better than
double('a')+1
char vectors and string arrays are not the same.
Re: comparison with double arrays, this is intentional because of differences in how the char and string constructors/converters behave when given a double array.
stringToDouble = double("a") % "a" is not the string representation of a number
doubleToString = string(97) % create the string representation of the number
charToDouble = double('a') % ASCII value
doubleToChar = char(97)
'a', "a", and 97 are in some sense "nice". What happens when we're not so nice?
stringToDouble2 = double("3.1416")
charToDouble2 = double('3.1416')
doubleToString2 = string(exp(1))
doubleToChar2 = char(exp(1)) % Non-printable character
backToDouble = double(doubleToChar2)
On a related note, char and string differ in the definition of the + operator.
'a' + 1 % Arithmetic on ASCII value
"a" + 1 % convert 1 to "1" and concatenate
When designing the string class we took what we'd learned over decades of users complaining about char, what users had told us they wanted to do with char but couldn't (or couldn't easily), and what users told us they liked about char. The two classes do not behave identically (if they did, what point would there have been to create the string class?) but IMO they each behave usefully.
I know that this char<->double thing goes way back and so it's a backwards compatiblity issue. I don't remember back far enough as to whether or not Matlab always provided an explicit conversion like
>> double('abc')
But from the time that the explicit conversion was introduced, I've never seen an advantage to not using it, and found one other inconsistency with using the implicit conversion.
There are various warts; this is another one that ought to generate bug report.
Got missed in regression testing, probably, although sometimes TMW comes back with real convoluted logic to justify an existing behavior; other times it comes down to "because we say so" if they don't want to fix something; on occasion it's been a compatibility argument. With string being so new, that one doesn't hold a lot of weight.
If TMW was starting over today, would they implement the implicit conversion of char to double?
They claim that string is more efficient than cell.
The rules they make has no firm logical foundation
>> isequal("a",{'a'})
ans =
logical
1
>> isequal('a',97)
ans =
logical
1
>> isequal({'a'},{97})
ans =
logical
1
>> isequal("a",{97}) % oh oh
ans =
logical
0
>> s= ["lorem","ipsum"];
>> s{1}
ans =
'lorem'
>> double(s{1})
ans =
108 111 114 101 109
>>
A string is just an encapsulation of a char() array. Before extract was introduced in R2020b (really late for a string class to have been introduced without), cell notation was the only way to get to the substring.
I see what you mean about the lack of transitivity between classes.
Character and numeric values have always been convertable but string and numberic conversion has never been supported. I think it makes sense to support char<-->string comparison. The char<-->numeric comparison has long been supported. Character arrays are a sequence of characters, just as a numeric array is a sequence of numbers. This is not true for string arrays. The string array ["lorem","ipsum"] is a 1x2 array and there is no numeric equivilant of "lorem" like there is between characters and their Unicode/ASCII equivilents.
Additionally, although isequal is fine to use with string comparison, stcmp/strcmpi are the better approaches and are consistent in their numeric comparison with char|string
strcmp('a', double('a'))
strcmp("a", double('a'))
Thanks Adam, but that's not my point. Yes, you can do that, but a basic function like isequals should mirror the properties of the mathematical operation that it models. If you can't trust an equals operation to be transitive, it's hard to know what you can trust, and it becomes necessary to do extra tests on everything you write, and to check even core functions against every possible combination of classes.
As for validateattributes and validatestring - I've never liked them at all I'm afraid. I think they're clumsy and I'm enjoying removing them as fast as I can in favour of arguments blocks. I'm sure there was a good reason for introducing arguments blocks into the language!
If the class can be char or string you can just wrap it in char() to ensure make the comparison work either way.
isequal(char('a'), 97)
isequal(char("a"), 97)
For nested functions there are other validation methods that are just as good (or even better since they are supported prior to 2019b). I like validateattributes & validatestring.
Here's my Matlab Gripes, 2020 Year End Edition.
This is coming from the perspective of someone who's primarily a software engineer, and came to Matlab from a Java/SQL/Perl background. I focus on building large-ish Matlab programs that integrate with other programming languagesl, data sources, and systems.
- import is kinda useless, because it's function-scoped instead of file-scoped or class-scoped.
- Database Toolbox returns empty result sets as [] instead of a 0-row table with the proper column structure, even if you requested table format.
- Database Toolbox tries to be too smart for its own good, with parsing your SQL and changing behavior based on it, doing probes for server-side structures, etc.
- datetime's exact semantics, precision, and storage costs are opaque and undocumented
- Some Matlab-supplied functions (I'm looking at you, datetime(strs) constructor!) use try/catch on the happy path, making dbstop if all error basically unusable.
- No generic user-defined-object display customization that works when you stick your objects inside struct, cell, or tables.
- No mutable Static properties on classes, only Constants.
- Mixed-mode arithmetic silently narrows floats to ints, instead of widening ints to floats.
- No good code browser in the Matlab Desktop, like basically every other IDE has.
- The Details widget in the Matlab Desktop shows the last file you clicked in Current Folder instead of the one that's focused in the Editor, and you can't sort its contents alphabetically. (MathWorks came so close to making this one Good, and yet...!)
- Matlab Production Server's management and monitoring tooling is pretty weak.
- If you call a Static method from another Static method on the same class, you have to fully package-and-class-qualify the name of that method.
- Subtly broken Java XML API setup in the included JVM.
On the whole, I'm just kind of bummed that there isn't more of an active Matlab developer or open source community. It's kind of lonely being a Matlab programmer.
But I don't want to sound too negative! Matlab has made some great improvements over the last few years, with declarative property constraints, arguments blocks, table arrays, the string, categorical, and datetime types, a much faster OOP execution engine, some of the new Editor features, indentation for jsonencode, jsonencode and jsondecode themselves for that matter, and so on.
So true. Strong back-compatibility is great for some users, especially businesses, because it means you can upgrade with minimal breakage. But it's also a lot of baggage when it comes to new development and innovation.
I hope that some day, MathWorks decides to make a Matlab X, where they're like, "Okay, we're gonna have one release where we take a bunch of big breaking changes in order to modernize the language", like with Python 3 or Perl 6.
"coming from the perspective of someone who's primarily a software engineer, and came to Matlab from a Java/SQL/Perl background..."
No disagreement with the list particularly although a lot of which I've never used so have no knowledge at all of.
Just a comment on context -- MATLAB and TMW have the great handicap in adding all the new features in that the basics of how the system works was developed 40 years ago and everything has been more-or-less had to have been hung onto the existing framework.
Much could be done in having the same functionality but in a more efficient fashion if one were to begin with a new base design structure.
The biggest frustration is the limits on supported mex compilers, especially for FORTRAN. Mathworks really painted themselves into a corner with exclusive support of Intel Fortran on Mac and Windows.
They should have stuck with GNU compilers or eleminate the restriction and let us decided which compiler works.
Also, why the slow and process of updaing GNU versions supported on Linux?
This problem is really terrible!
MEX support for Fortran is still extremelly important, at least from my point of view. So any kind of Fortran support ending for MEX is really nightmare.
But, the most important things are:
- MATLAB oficial support mainly the Intel Parallel Studio XE for (C/C++ and Fortran) on Windows platform. GNU compilers support on Windows should be main target for MATLAB (MinGW is now nearly obsolet ... latest version is from 2017??!!)
- slow updating process of updating GNU compilers version on Linux, which does not correspond to the current state of default GNU compilers version on supported Linux distributions
- and moreover, terrible situation with supported NVIDIA CUDA Toolkit versions (required by GPU coder and compiled CUDA objects). Matlab supported CUDA Toolkit does not corresponds to the required GCC versions on many recent LTS Linux distributions
So finaly, the TMW support always responds that external projects (GNU, NVIDIA CUDA, Intel Parallel Studio XE) is not under TMW control, but this situation is nearly unacceptable.
Amen, brother...particularly to "let us decided which compiler works"
And, they've not left the era of F77 yet, either.
I'd guess they would far prefer to just pull the Fortran support entirely and have been expecting to see it deprecated every release for last few years.
Sometimes things are on the edge of being documented or are almost a bug. An example:
It is relatively common knowledge under Matlab veterans (at least those I see posting on Answers) that the legacy syntax for cellfun is much faster than providing a function handle (which is faster than providing an anonymous function).
c=cell(20,30);c(randperm(end,end/4))={rand};%generate some data
timeit(@()lega(c));% the run feature might return an inconsistent timing, this should help
t=[timeit(@()lega(c)) timeit(@()hand(c)) timeit(@()anon(c))];
fprintf('%.2fms ',t*1000)
But it turns out the result is incorrect for strings. The reason is that the legacy syntax doesn't call overloaded methods (as the documentation explicitly states). The problem I have with this, is that this sounds like it would only apply to user-created classes, or to classes that are exclusive to a toolbox. It turns out that assuming fundamental classes will work, will burn you in the end:
% function handles are a fundamental class as well, but can't be empty
fundamental_classes={logical([]),string([]),char([]),int8([]), int16([]), int32([]), int64([]),uint8([]),uint16([]),uint32([]),uint64([]),single([]),[],table([]),timetable('Size',[0 0],'VariableTypes',{},'RowTimes',datetime([],[],[])),{},struct([])};
for n=1:numel(fundamental_classes)
if ~isequal(cellfun('isempty',fundamental_classes(n)),cellfun(@isempty,fundamental_classes(n)))
fprintf('For %ss, the legacy syntax doesn''t match the modern syntax.\n',class(fundamental_classes{n}))
end
end
%(local functions must appear at the end of the post for the run feature to work)
%call the functions in a loop to make them take longer
function lega(c),for n=1:100,cellfun('isempty' ,c);end,end
function hand(c),for n=1:100,cellfun(@isempty ,c);end,end
function anon(c),for n=1:100,cellfun(@(x)isempty(x),c);end,end
My main problem with this is not that you need to manually test things like this if you plan on taking advantage of arcane features. My problem is that Mathworks discourages optimizing your code for how Matlab works internally. When I read "this doesn't work for overloaded methods" I understand "though luck, you can't use this for custom classes, but vanilla Matlab classes are fine". In this case I should have read "dig deep in your Matlab install (e.g. with which) to try to figure out which data types use an overloaded method". Bug or feature? I think reasonable minds can disagree.
I found a similar issue where this page seems to suggest Matlab uses UTF-8 (or even the full 21 bits) to encode chars internally, while it actually uses UTF-16, as mentioned here. (on closer reading the first page could be construed to only apply to files and byte streams, but this lack of clarity is not usual for Matlab documentation)
I just checked, and R2010a doesn't have the C++ file (and R2007b does, I don't have access to any intermediate releases). I assume Yair checked the newest version at the time of posting the blog (R2009a), even if that release was only a month old. Perhaps a Mathworks engineer was reading that blog post and considered it leaking of source files, so it was no longer included in later releases.
I don't really understand why it was included in the first place anyway.
Yeah, I also recall them, was just checking the earliest release I still have installed...
Apparently they used to be there as pure C/C++ files, at least the one for ismembc, as evidenced by the post about it here. That post is from early 2009, so more than enough time to have it removed. I can't find the indicated path in an install of R2011a, so it may have been gone for a while. If you can convince an installer from back then to do its thing, you can still get the source file. I don't know anything about the legality of using such a source file.
Note that you can still call ismembc:
n=2e6; a=ceil(n*rand(n,1)); b=ceil(n*rand(n,1));
tic;ismembc(a,b);toc;
I can't find any .c source files in R2014b install subdirectories other than the mex samples code...
I've worked with 4.2c to 6.5.1 and 2009a. I cannot check this currently, but at least in 6.5 the source of many C-mex source files has been included: histc.c, ismembc.c, ...
For mxIsOpaque see e.g. https://www.mathworks.com/matlabcentral/answers/330929-how-to-access-matlab-string-data-in-mex-c
Thank you for digging into this. I'm not quite sure I fully grasp your test, but it is interesting to see another angle of this.
For the record, the customer service person I had contact with claimed this was a bug in the documentation, not the function. I still disagree, but it's their program.
Thanks Rik. I assume, that this is a bug.
Older Matlab version contained the C-MEX source cellfun.c . It is easy to insert a mxIsOpaque check to call @isempty instead of mxIsEmpty() for these variables. Then even {string([])} is recognized correctly as empty.
I've tried this and to my surprise a straight C-mex function checking mxIsEmpty or mxGetNumberOfElements()==0 is 2.5 times slower than the built-in cellfun even for standard data types. In both cases only one thread is used. So obviously the buit-in cellfun is not a simple C-Mex function anymore.
😂 Thank you
Most entertaining read of the day award goes to Rik!
COM Automation server doesn't support struct (Not even trying to convert the struct output to MWStructArray and throw error when necessary) ... What a shame ... otherwise would make C# perfect for us to use. Sigh ...
It frustrates me that else if is legal (R2018b)
%%
if length(c) > 1
d(end + 1,1) = 1;
else if length(c) == 1 %#ok<SEPEX>
if n > 0
ia = 1;
return
end
end
end
%%
a = 1;
if a == 1
b = 17;
else b = 17; %#ok<SEPEX>
end
Doc says: "Avoid adding a space after else within the elseif keyword (else if). The space creates a nested if statement that requires its own end keyword."
It may be by design, but it still frustrates me.
When is a comma or semicolon required as separator between "keyword" and "keyword"/"statement"?
%%
a = 1;
if a == 1 b = 17; %#ok<SEPEX>
else b = 18; %#ok<SEPEX>
end
%%
for jj = 1:2 a=jj^2; end
%%
while n<=3 n=n+1; end
I also misread Walter's intent...I'm in agreement with that comment entirely; which puts mine out of context with it.
Under some contexts it's not legal to separate commands by a space or sometimes a warning appears such as
I'm not arguing that else if should also be illegal since I prefer leaving it up to the user to understand the syntax but just wanted to point out the enforcement of rules in similar contexts. The addition of semicolons would avoid all of the errors and warnings above and IMO else; if is more readable to inexperienced users than else if and a warning message could point that out.
But are you going to actually enforce coding practice in the language?
It's still legal code even if not ideal practice.
elseif is not appropriate for the situation where the code would look like
else
if test
some code
end
final code within else
end
But how is it to be known for certain the space is a typo and user did not intend the nested if?
It frustrates me that
enumeration
events
methods
properties
are not reserved keywords (in R2018b).
The statement
methods = clazz.MethodList;
is legal!
Try this
%%
clearvars
%%
try
properties = 1;
catch me
keyboard
end
%%
try
methods = 1;
catch me
keyboard
end
%%
try
events = 1;
catch me
keyboard
end
%%
try
enumeration = 1;
catch me
keyboard
end
%%
whos
it displays
Name Size Bytes Class Attributes
enumeration 1x1 8 double
events 1x1 8 double
methods 1x1 8 double
properties 1x1 8 double
Yes, but
>> end = 17;
end = 17;
↑
Error: Illegal use of reserved keyword "end".
I like this behavior better.
Those keywords are only keywords when used inside a classdef file, this is true. They are also functions that help you gather information about the aspect of an object defined by using the corresponding keyword in the classdef file.
There is long precedent for having a keyword that's only a keyword in certain contexts.
x = 1:7;
y = x.^2;
y(end) % 49
for k = 1:numel(x)
fprintf("%d^2 is %d.\n", x(k), y(k))
end
One of those uses of end is hightlighted as a keyword. One isn't.
Also not in R2020b:
%%
clearvars
%%
try
properties = 1;
catch me
keyboard
end
%%
try
methods = 1;
catch me
keyboard
end
%%
try
events = 1;
catch me
keyboard
end
%%
try
enumeration = 1;
catch me
keyboard
end
%%
whos
MATLAB startup accelerator.
I just don't like my computer being polluted by some services installed in the task scheduler. I prefer it''s OFF by default. More and more people have SSD in the computer, the accelerator has no visible effect even when it's on.
The camorbit/camlight (camera toolbar) that makes the 3D graphic rotation turns forever when I swing with the mouse.
It's fun to see this gadget once or twice but if some calculation is going in parallel, this is NOT desirable that some animation takes a hand on the runing thread, and there is not obviousway to disable this feature beside overloading the camlight.m
I've created an enumeration class by use of the following command:
Simulink.defineIntEnumType('epb_parkbrakeswitch_SL_T','PB_SWITCH_NEUTRAL','PB_SWITCH_UP','PB_SWITCH_DOWN'},[0,1,2],'AddClassNameToEnumNames',true,'HeaderFile','Rte_Type.h','DataScope','Exported');
I have workspace items loaded that make use of this class. I want to save my workspace to a MAT file which will also include this enum class information. I don't currently see that this is possible. Our company's system team doesn't want separate M-FIles for each class but would like it all contained in one file. Currently I'm accomplishing this by have an M-File with this enumeration defintion at the top followed by the variable usages of it. This works, however some of the project's M-File sizes are getting pretty large and it takes awhile to save/load as an M-File. Wish there was a way for enum class defintions to be able to be saved in a MAT file, much faster to load and save.
The number one issue I have with Matlab is the way in which implicit expansion's syntax was implemented. Perhaps there's some incredibly subtle thinking behind why overloading linear algebra operators made sense, but I haven't seen it. It's not that I haven't agreed with the reasoning, it's that I haven't seen any reasoning at all. My recollection is discovering it when someone complained that it wasn't even in the release notes and that it is, in effect, a breaking[*] change.
Issues like https://www.mathworks.com/matlabcentral/answers/318495-deactivation-switch-for-implicit-expansion#comment_699667 show the ease with which the problem arises. It would have been far preferable to have an explicit operator for implicit expansion, such as `@`, e.g. @+, @-, etc...
[*] breaking because it breaks the rules of linear algebra, which is that `a + b` is only valid if both a and b have the same dimension.
Here are some other examples ...
size(vcol + v_unintended_row)
ndims(vcol + v_unintended_row)
max(svd(vcol + v_unintended_row))
Yes, because the function norm() is defined for both the vector and the matrix and always produce scalar output. In principle, every functions with this input/output specification are possible candidates for complains.
And the only example that I have seen is
norm(vcol + v_unintended_row)
People who makes such mistake (or the opposite) gets quite angry with MATLAB auto-expansion for some reason (well I guess they spend a big chunk of time before figure it out the silent bug).
Very well spoken ... so, there is practically no reason for any complains on "implicit expansion", because backward compatibility is not the real issue!!! People which does not like auto expansion can still use old constructs to keep backward compatibility.
Every known examples of the <R2016b MATLAB codes where auto expansion cause "problems" are buggy or not well written.
Just to be in the same line: Auto expansion breaks backward compatibility but NOT forward compatibility.
(Well written) Code written before 2016b continues to work.
The only issue people complain is some buggy code that operates on non-identical size arrays might no longer throw an error. But come on, Who care about compatibility of buggy code?
Star Strider specifically wrote,
"All I wrote was that I don’t use automatic explicit expansion in my FEX contributions, since there are many here who have MATLAB versions earlier than R2016b. Assuming automatic explicit expansion in a FEX contribution would not work for them, so I don’t use it. "
That was not a statement that old code broke when brought into R2016b: it is a statement that new code written with implicit expansion breaks when used with releases older than R2016b, and that Star Strider deliberately makes his code backwards compatible for earlier releases because in his experience there are still enough people using earlier releases to make the effort worth-while.
@Bruno Luong No ... I am asking for example of well coded MATLAB program (<R2016a), but on >=R2016a does not work, due to the "implicit expansion". Kenn Sebesta and StarRider mentioned here the fact, that "implicit expansion" break the older codes, so I would like to see them.
Yair reference is in the comments above, https://www.mathworks.com/matlabcentral/answers/556024-what-frustrates-you-about-matlab-2#comment_1047391
Why do you refer to Yair? The thread is started by Keen Sebesta.
Rik and Star Strider are talking about the situation where they are writing FEX contributions that they want to be usable by people who are using releases before R2016b. They are not saying that if you had valid R2016a code that because of implicit expansion it will fail in R2016b; they are not saying that any existing FEX contributions stopped working.
However, they would like to be able to turn off the R2016b+ implicit expansion feature so that when they are writing code they are given errors if they accidentally use implicit expansion: they want this to make it easier for them to write code that would work in older releases as well.
Yair, on the other hand, had a client that was counting on the error being generated, and starting having problems because the error was no longer being generated.
I would not have written code the way that Yair's client did, but the problem was real for them.
But any code using auto expansion is not backward compatible. Why you need an example for something that is so obvious?
@Rik I just asking for any representative example (< 2016a) where "implicit expansion" break the code, because some people here say it's very likely. That is all ... !!!
"in algebra a+b is only valid if a & b have identical dimensions. Likewise, A*b is only valid if the number of columns of A is identical to the rows of b."
It is not uncommon to see formulas in which people add a scalar constant to a matrix instead of building a matrix of the constant and adding that.
It is very very common to see formulas in which people multiply a scalar by a vector instead of building a diagonal matrix of appropriate size containing the constant on the diagonal and multiplying that.
MATLAB has supported scalar expansion from very very early on. I would suspect from the original design, but I have not had an opportunity use the original version.
MATLAB has never been a language designed as a linear algebra language specifically. It was instead designed to make some kinds of programming much easier. Purity of linear algebra was never a goal for it : ease of expression of programs was the goal.
Folks, please remain professional in this discussion.
I assure you we had quite a bit of discussion inside MathWorks about the potential of breaking existing code by making what once threw an error now work. I raised some of the concerns you (and many others) have raised myself. [Though your choice of @ as part of the operator wouldn't work, as Rik stated it is already used for defining function handles.]
@Rik, in algebra a+b is only valid if a & b have identical dimensions. Likewise, A*b is only valid if the number of columns of A is identical to the rows of b. On what basis do you disagree that implicit expansion overloading of + is unallowable by linalg properties?
Those are the rules of linear algebra you feel MATLAB should obey? MATLAB has never obeyed those rules, at least not strictly. Let's say for the sake of discussion that we were to introduce a way to stop expansion behavior and have MATLAB strictly abide by those rules you've cited. How long do you think it would take to fix your code? And how much code would break due to the increased memory required to create large constant matrices? Yes, it is likely you would need to fix your code. Let's take a look at a simple example with a hypothetical function to disable expansion.
>> disableExpansionBehavior; % this function does not exist
>> A = magic(4);
>> B = A + 1; % Error, A is size [4 4] and 1 is size [1 1]
>> C = 2 * A; % Error, 2 is size [1 1] and A is size [4 4]
>> D = ones(1e5);
>> E = D + 2; % Error, you need a 1e5-by-1e5 matrix of 2's
As for a concrete example of a function that would break (as written) without implicit expansion, the normalize function in MATLAB uses implicit expansion. Normalizing by 'zscore' (the default normalization method) subtracts the mean of an array A from the array then divides by (in the default case) the std.
F = magic(5)
M = mean(F)
S = std(F)
With implicit expansion you get (more or less, modulo some dim inputs and 'omitnan' to handle NaN values) what's in normalize.m.
N1 = (F-M)./S
N2 = normalize(F, 'zscore')
Without you'd need to use bsxfun or repmat as M and S are the same size but neither is the same size as F. If implicit expansion could be toggled on or off (or disabled entirely) normalize would have to test each time it was called whether implicit expansion was enabled and use a different code path (or temporarily enable it) if it wasn't.
What is your issue? Do you doubt people are using implicit expansion in real code? It is tricky to find an example, but I have posted code using it myself on this forum. Can you provide an example of a FEX submission that uses bsxfun? I would consider it a waste of time to search for an example. Why would that make his opinion irrelevant? Do you want me to post a minimal example to the FEX just so we have a link for you?
@StarRider Very constructive attitude! Please ignore this thread. You just spread irrelevant opinions without any reproducible examples.
Michal Kvasnicka —
Sigh. Find someone with MATLAB R2016a or earlier (there are still many), and tell them to run code written assuming automatic implicit expansion. Then deal with their frustration when it throws errors. That is your ‘example’!
Stopping here with this, since it is just not worth pursuing.
@StarRider No flame war at all!!! Just ask for examples, because lot of people mentioned this without any relevant examples.
Michal Kvasnicka — I wasn’t referring specifically to FEX contributions (although it would certainly apply to any FEX contributions that use automatic implicit expansion), only that reverting back to not supporting it would break a lot of code written using it. That’s obvious, so why the flame war?
@Star Strider You said: "However, it’s been a part of MATLAB for four years at this point, and reversing it now would likely break a lot of code."
So I ask for any example ... that is all!
@Rik you are completely wrong ... really :)
@Kenn, I don't see why you want to use strict definitions for one parameter, but not the other. I don't see why Matlab syntax should follow algebraic syntax requirements. It is useful if that is the case, but by no means required, as evidenced by .*.
I disagree with your statement that you need a dimension check every time control leaves your file. That is only the case if you don't trust the output. norm will not suddenly start outputting a different shape array on some release. Only if you rely on norm returning an error do you need checks, but my argument is that you shouldn't do that. Unless you're using undocumented features, every API change (using the term a bit loosely here) should be documented.
Any call to any function could not just change the dimension, but also the value (or trigger errors/warnings). If you have that little trust in a function you shouldn't just check the dimensions, but also the output, effectively re-implementing that function.
I can understand why people dislike this, I just don't understand your argument.
(@Michal, I see only 1 person seeming angry. If this thread/discussion frustrates you, feel free to ignore it and take a walk outside. I don't think implicit expansion is worth a high blood pressure 😉. Also, your last question sound like the start of a true Scotsman fallacy.)
Michal Kvasnicka —
All I wrote was that I don’t use automatic explicit expansion in my FEX contributions, since there are many here who have MATLAB versions earlier than R2016b. Assuming automatic explicit expansion in a FEX contribution would not work for them, so I don’t use it.
You are certainly welcome to assume that all MATLAB versions would be able to use automatic implicit expansion if you so desire, and write FEX contributions with that assumption. I don’t, and I won’t.
@Star Strider Give me ONE example from FEX, where "implicit expansion" broke the well coded MATLAB program!!! The MATLAB Coder is completely different story.
Tangentially, I discovered yesterday that the MATLAB Coder does not support automatic implicit expansion. (I do not have Coder, was helping someone who was having problems with ismember. A bsxfun call solved it, once I figured out what the problem was.)
I do not love automatic implicit expansion. However, it’s been a part of MATLAB for four years at this point, and reversing it now would likely break a lot of code. I have not and will not use it in my File Exchange contributions, for compatibility considerations, using bsxfun calls instead.
OK ... again the same angry discussion!!! Try to search in "MATLAB answers" for all posts on this topic. Still the same arguments, still the same counter-arguments. But "implicit expansion" is here several years yet and we all live with that without any serious troubles.
Add: In the 2016, when was the "implicit expansion" introduced (2016a) I was one from many who express really big complaints (very similar to Kenn Sebesta express now here). But more clever guys here convinced me, that any kind of special explicit operator (@ character, for example) is not a possible as effective and fully compatible solution. So, what ... ?!
Closing thought because I don't want this to devolve into an argument. This change is what frustrates me. It might not frustrate others, because they operate in a different way with different needs from Matlab.
Others may have different pain points, but since the topic at hand is what frustrates us most about Matlab this is where my colleagues and I stand. We're industry professionals and have had four years to grow accustomed to it. We still find that it provides no benefit (to us), and it came at an unacceptably high cost (to us).
I respect that others love this feature, but with all due respect this discussion isn't particularly useful and is a replay of prior discussions. We're not really adding anything new. The thing I'm still missing is why the syntax was done the way it was and I don't think we're going to find it out this way.
@Rik, in algebra a+b is only valid if a & b have identical dimensions. Likewise, A*b is only valid if the number of columns of A is identical to the rows of b. On what basis do you disagree that implicit expansion overloading of + is unallowable by linalg properties?
You rightly point out that .* is not valid algebra syntax and so it can be meant to mean anything convenient. No objections here to anything TMW does.
You might be right about readability, although I personally don't agree that there is a readability problem by knowing that the code author explicitly wanted implicit expansion, instead of wondering each time there is a +-*/ if the author made a mistake. No matter our guesses, it's never been discussed publicly and that's the rub.
"I don't get why you would need to sprinkle your code with dimension checks. You just need to do that at the start of your function, after which you are the one making sure there aren't any bugs in your code."
This is incorrect. You need to do a dimension check anytime control leaves the file. Any call to another function can potentially change the dimension, now or some time in the future. Previously, error trapping could have caught this immediately. Since 2016 it's a silent failure that can live on for years.
@Walter, with the exception of "misreading Yair", I agree and believe we're saying the same thing. Yair noticed it in 2016a Pre-release. At the time, it seems that TMW did not announce this change in any meangingul manner. Quoting him: "What I am concerned about is that I have not seen the new feature highlighted as a potential backward compatibility issue in the documentation or the release notes."
As such, I will stand by my claim that TMW introduced this without meaningful discussion with the community at-large, seemingly oblivious to the fact that there might be meaningful objections as well as useful suggestions for improvement.
I don't get why you would need to sprinkle your code with dimension checks. You just need to do that at the start of your function, after which you are the one making sure there aren't any bugs in your code. An invalid input error should (IMHO) occur at the start of your function during input validation, not in the middle. I do agree that it can be annoying if during debugging the error occurs far away from the buggy line of code (as Walter pointed out).
I disagree with your point that it violates algebra rules. The .* syntax isn't a valid algebra syntax. Is your problem with the fact that the meaning of a syntax inside Matlab changed? Or would it have been fine if implicit expansion had been there from day 1?
I'm not sure there is a convenient character that would achieve the same goal as the current system: readability. The @ character is a good candidate, but it may cause more confusion with function handles than it saves by avoiding bsxfun.
You are misreading Yair. Implicit expansion was clearly announced in r2016b -- I was one of the beta testers. What Yair was saying in his blog was that initially there was no explicit compatibility warning in the release notes that announced the feature.
The possibility for compatibility problems was obvious to me at the time. However I did not have any code that depended upon incompatibility generating an error to be automatically fixed up by try/catch, so for me the frustration was not around incompatibility but rather around my expectations that if I got the orientation wrong then the code would break pointing immediately to the problem; I was using the error to debug the code rather than relying on the error to switch my code flow to correct paths.
It was not mentioned in the release notes in the beginning. http://undocumentedmatlab.com/articles/afterthoughts-on-implicit-expansion gives a good description of the, at-the-time, issue with the changelog. Implicit Expansion first went live in 2016a Prerelease, but didn't go gold until 2016b. Moreover, initially TMW didn't draw much if any attention to the fact that this has breaking characteristics. They've since fixed that, but the oversight points out that they didn't really give much consideration to downstream effects.
I think we can conclude the feature roll-out was mishandled, and existing at-large users wer not given the opportunity to weigh in. Something was going on at the time, perhaps a large customer was applying pressure for this feature to be implemented in exactly this way. I don't know that we'll ever know.
What I do know is that it sure looks like the fix is as simple as using a separate operator for implicit expansion. Maybe there's a reason why this wasn't an option, but the easiest thing to believe is that a large customer wanted it this way and TMW doesn't want the public releations boondoggle of saying that out loud.
The implicit expansion booby-traps have been discussed fairly well, but I'll repeat the easiest example here. If you're using a built-in function, which means TMW specifically makes no future guarantees about the returned vector shape, something like norm(a-b) can get you in a ton of trouble. In a nutshell, you can feed Matlab an nx1 vector and get a 1xn vector in return. You'll get a facutally incorrect result from norm() when what you should get is an error.
And It's wholly impractical to sprinkle code with dimension checking every time a Matlab function is called. Especially because code like norm(a-filt(b) becomes impossible to sanity check.
In this context, it's always upsetting to see people push an onous onto the programmer. It's like suggesting that it's okay for a new C compiler to silently fail when there are typos in variable names. Code which violates the rules should fail to run, whether the rules ar strong typing (C/C++) or linear algebra. Changing the rules can have certain advantages, but it comes at a cost. In the case of implicit expansion why not have created a new operator so that there was no need to change the rules? This is the part I simply don't get, and it's the part that implicit expansion apologists conveniently sidestep.
In my case, the cost was dramatically scaling back Matlab usage and switching to Julia. I know this isn't pracitcal for everyone, and I know that I'm not a $$$ customer so I don't have any real power to fix this. I'm simply putting in my vote so that Matlab has useful public feedback.
R2016b release notes for MATLAB, first entry under the heading "Mathematics"
This topic (implicit expansion vs math correctness) was discused here many, many times. Some people were really very angry, but without any impact on TMW developers.
The final result always was: You have to learn to live with this specific feature. Any kind of implicit expansion disabling is very difficult to implement. You must change your programming habits.
So, now I always check very carefully my code to be correct, which is also quite a good practice in general. Moreover, implicit expansion gives to programmers significantly more pros than cons!
I believe I did see it mentioned in the release notes. It also warned that code that relied on an error being triggered by mismatching array dimensions would no longer work as intended. If you want to I can look it up again and find you a link.
Matlab should 'update' the new version instead of installing the new version. The reconfiguration is a nighmare because it's done only once a year or so. The extreme example is firefox, which updates once or twice a day. The users don't need to reconfigure anything.
At least matlab should provide a choice with every new version: new installtion or updating on existing version (and keep all configurations as is).
The extreme example is firefox, which updates once or twice a day. The users don't need to reconfigure anything.
Every time firefox installs a new Major version, it creates a new User Profile which it does not initialize from the old one. This messes up my careful configuration, and I have to spend the next while fixing it. It's a big nuisance. It does not warn you in advance it is going to happen, and it is not optional.
Additionally, it is often a good idea to keep old versions of Matlab around until you're sure whether the new release breaks anything. Treating it as an update wouldn't make sense.
The installer already grabs a lot of settings from the previous release (color scheme e.g.). I was also greeted by a popup from the Add-On Manager asking if I wanted to install the previously installed support packages.
I suppose that other settings (e.g. the path) could be ported as well.
I doubt they will fundamentally change this anywhere in the foreseable future, but what did you have in mind? There are already updates (which are the bugfixes), so renaming the releases (which introduce new features) to updates will be very confusing.
Livescript complaints
First off, they've come a long way since I tried them on release. But, there are still some issues:
Dropdown controls
- The "item value" box is tiny
- Keeping track of item label - item value pairs is unnecessarily complicated by the vertical arrangement (vs a table-type view)
- poor auto-updating of labels/values when the other is changed
- pasting code into the value box often separates lines unexpectedly (it seems to sometimes cut at mid-line whitespace)
output character limits
there seems to be no way to convince the livescript to print characters beyond a certain limit. For example,
for ind=1:10000
fprintf('Sample %d has been processed\n',ind)
end
and
for ind=1:10000
disp(['Sample ', int2str(ind), ' has been processed'])
end
both print only to
Sample 1971 has been processed
Sample...
Copying Output
- cannot select text in output
- copied text does not always match displayed text (e.g., copied cell arrays say "1×2 double" while output displays the numbers)
- unclear rules (moments ago, I learned that what's copied depends on where in the output box I right click)
Export to PDF
- can't read long code lines
- displays variables differently (e.g., doesn't truncate table rows)
- doesn't warn if code has been changed without re-running (so code and output don't match)
Minor gripes
- add "run from beginning to current cell"
- code prediction seems better than for normal scripts (even aside from the fact that variables are more often already evaluated when working with live scripts)
- not enough heading levels
- I'd prefer controls default to not running anything
- "unable to run code analysis... is an invalid Matlab file name."
- no way to collapse output (vs clearing)
- no way to fully un-collapse output (e.g., display full table instead of scrolling)
- figures are saved with "Visible" = off, so they don't open as expected
Yikes, I saw dropdown and then immediately thought of AppDesigner, sorry about this.
I'm also in 2020a. Are you in a live script or the app designer (I haven't used it)? I'm not aware of a programmatic way to control these live script control boxes.
Yes, newline separates items. Visually, this isn't helpful when value expressions wrap to new lines. Mechanically, this can be a problem when pasted text is interpreted as multiple lines (even when it is not)
Oh, you ment this box, i misunderstood.
If you want to change the Values,Items, it's best to use startupFcn callback, in which you specify the values. In this window, I think newline works (enter button), so values can be like this (my dropdown menu doesn't look the same though) What version do you use?
val1
asdf
probably*crazy%#"$!chars*!#=worktoo
If you mean, by changing the component name such as app.NameDropDown into another one, most of code (if not all) will change accordingly, at least in my case, I use R2020a.
Where do you see the grips? I tried dragging all over and couldn't get it to work
Keeping track... here, I only have two items and one value is a single line, so it's pretty easy to keep track of which value goes with which item. But, as you can see, the values don't obviously reflect the labels. So, what if I have 20 items? How do I find the right value to edit? Having it side-by-side with delimiters between lines would help immensely
I don't understand your third comment. My issue is that when I change a value or label, it often creates a new item instead of remaining connected to an existing one
Dropdown controls
- The "item value" box is tiny - You can increase it, click once to select the Dropdown, then once moreon the value box, you'll get these "grips" to increase it
- Keeping track of item label - item value pairs is unnecessarily complicated by the vertical arrangement (vs a table-type view) - not so clear what you mean by this
- poor auto-updating of labels/values when the other is changed - set your fields to get the property values, and rather change properties than fields if you will use those values
My only feeling while using Matlab is purely amazement. Given that it has such a long development time, I am amazed that it is such a poor execuse for an IDE/execution environment, and even amazed that people actually paid for this.
Come on, with all the money people have been paying you, at least make a decent UI with decent color scheme support. Even eclipse, which is free, supports this.
I don't feel the need to mention the lacking of the language and the standard libraries itself as it's likely a legacy thing that you guys either have to keep for legacy support or are too lazy to do anything about. Either way, I feel like I am writing code with both my hands and feets tied and only allowed to use my tongue to type.
I am forced to used this for my course (student license of course), nothing else. Never again.
As of release R2022a MATLAB Online now allows users to select a dark or light theme. In addition you can now change the text and background color in the Live Editor. See the first two items in the Environment section of the Release Notes for mroe information.
I creat this account to say this. I don't understand how Matlab (as a language, as an IDE) can be abnormal with that such large amount of money.
Eclipse has the resources of IBM behind it. (I know some of the key people who work on it.)
Expanding on Walter's comment, if you mean just for you see the "Keyboard Shortcuts Preferences" section on this documentation page.
The thing is that Ctrl-C and Ctrl-V for copy and paste is the standard for GUI apps everywhere
Not on Mac, which uses ⌘-C and ⌘-V
Can Matlab editor be set to Windows Default Set keybindings on Linux by default?
Do you mean just for you (or for all people running MATLAB off of your installation)? Or do you mean that you want Mathworks to change the default ?
If you mean that you want Mathworks to change the default, I would not expect that to happen. emacs binding has been standard for Unix since at least 1983.
Do you really think, that these threads are read by MTW people??? I am not sure!!!
TMW definitely choose the commercial way of its bussines ... so plenty of toolboxes with all whistles and bells, but all using the matlab engine which definitely needs significant development to catch current new trends in computer science.
Just a two examples (from many others):
1. ODEs ... try to compare matlab ODEs capabilities and performance with Julia's wonderful DifferentialEquations.jl. MATLAB shows really terrible performance and flexibility in this case.
2. VPA ... try to compare VPA and Advanpix MCT toolbox. Again, the MATLAB vpa functionality and performance are really terrible. Nice speed benchmark against MAtlab VPA, Julia, Wolfram Mathematica and several Python libraries (mpmath, flint+arb) for extended precision computations. The page is in Japanese but plots and tables are easy to understand.
3. and so on ... and so on ...
I think MATLAB is still a very unique and powerful tool for numerical calculations, but it is definitely losing strength and becoming a monster standing on clay feet.
ODEs are not part of the MATLAB engine.
Meanwhile, Mathworks has been working on improving the performance of the MATLAB engine.
I do think they should be improving the performance of the MuPAD engine, perhaps by adding multithreading (I do not know how much work the existing engine would take to make the routines thread-safe.)
Do you really think, that these threads are reading by MTW people??? I am not sure!!!
I know for a fact that these threads are read and considered by Mathworks employees.
scatteredInterpolant miss a 2 C^2 of methods supported by griddata 'cublic' and especially 'v4'.
Frustration/suggestion:
Can Matlab editor be set to Windows Default Set keybindings on Linux by default? The thing is that Ctrl-C and Ctrl-V for copy and paste is the standard for GUI apps everywhere even in Linux and the default Emacs Default Set keybindings breaks that.
Its even a FAQ now:
Interesting quote from the second link:
When installed on Linux, the MATLAB Editor defaults to using Emacs style keyboard shortcuts (copy, past, etc.). This might be difficult for those new to Linux.
I have used desktop Linux for 5-6 years and I still find Matlab using Emacs keybindings surprising in a bad way
Which do you prefer 'MultipleDelimsAsOne' or 'CollapseDelimiters'? Now and then I use the correct name in the wrong place.
I wish: a popup asking me whether I want the other. That would save me a bit of frustration and lower my blood pressure by several pascal.
Before I started using Python this year, I was a happy MATLAB user. I still like MATLAB because I am efficient in it, but here are some things I don't like about it. I will sometimes refer to Python because that's what I have experience with. I do not indend to start a flame war, especially because I see the merits in both languages.
- The community. There are some heavyweight MATLAB users (Walter Roberson, Image Analyst, John D'Errico, Jan, and I'm sure I missed many of you) who have immense knowledge and are also very active on MATLAB Answers. However, the forum is full of people whose posts do not show any research work. They expect complete solutions without showing any effort. When Python users raise issues on GitHub/Gitlab, etc., I don't have this feeling. Related: I also feel that the open-source nature of Python produces quality contributors.
- Lack of momentum. There are too many awesome projects that stop because the maintainer a) lost interest or b) left the university and has no access to MATLAB any more. Being closed-source really hampers the development.
- Ecosystem. I can only talk about areas I experienced, but Python (and now Julia) has such an extensive ecosystem and an active and responsive community, that one does not have to start reimplementing everything. MATLAB toolboxes sometimes provide a solution but university clusters often miss many toolboxes.
- Editor. The MATLAB Editor (the classical one, not the Live Editor) is useful but misses some features that others have. As if the MATLAB linter was available for other editors too!
- OOP. Reading the threads, people criticize the OOP syntax of MATLAB. To me, it is not a problem (Python also explicitly needs the instance as the first method parameter), but custom types are really slow in MATLAB. I would love to have (nearly) costless abstraction as in Julia or in C++; it happened to be that because of speed issues I had to sacrifice readability and turn to built-in types to achieve reasonable speed.
- Lack of efficient data structures. Complex problems require fast hash maps, linked lists and other data structures.
- Parallelism. The fact that open source tools support many levels of parallelism and MATLAB offers it in a toolbox is a big drawback. Moreover, launching parallel pools open new MATLAB workers that spend about 700 MB each! I didn't see any efficient massively parallel applications done by MATLAB.
- Invoking C code, let alone C++, is difficult with the MEX interface. The really powerful MEX commands (see on UndocumentedMATLAB) are not documented and change all the time. You also cannot debug MEX code with MATLAB Editor.
- To generate standalone application, you need an additional toolbox. Moreover, the MATLAB runtime is huge, so for a simple application, you have to download hundreds of megabytes. The startup time is also very slow.
- TMW's focus on shiny features and the toolboxes. The core numerical algorithms changed very little in the past 20 years. To me, MATLAB's focus should be on computational mathematics (as it originally was) and not polishing the IDE and other secondary issues. Yes, they are important, but supporting tensor operations, introducing state-of-the-art ODE solvers (as Julia's wonderful DifferentialEquations.jl) would be more useful for a scientist. I would just highlight git support. If someone has to solve complex git-related tasks, they will use a feature-rich editor (such as SmartGit) or will use the command line. What I want to point out is that TMW spends its workforce on implementing many things into MATLAB that simply cannot compete with specialized tools (diff tools, git support, etc.). If they concentrated on the core of MATLAB (things that are unrelated to the IDE), all people could profit from it. I saw the tendency that computational scientists use MATLAB less and less because they need the features that other languages' rich ecosystems provide. Honestly, I am deeply disappointed that TMW tries to satisfy novice engineers instead of providing powerful tools. You are losing hardcore programmers and scientists this way (cf. points 1-3). I follow the release notes and bitterly recognize that in the last decade, almost no improvement was made for the core. Just the superficial features... Sigh...
I didn't write about the merits of MATLAB (there are plenty) because the question was about the negative aspects. My heart breaks to see how MATLAB fails to catch up with the state-of-the-art of computational science.
On the other hand, maltlab is both a software and a language. At the level of software tools, it has to provide as much convenience and ease of use as possible for all kinds of users, and at the level of language features, it has to provide flexibility and efficiency for all kinds of users. So it cannot be compared to a language (python/julia), nor to a professional tool (git tools, vs IDE,...). matlab is the best combination between software and language, and I have never seen any tool in the open source community in the last 20 years that fits together so well.
Oops – I forgot the Matlab License prohibits benchmarking Matlab against competing products. (Section 3.4.) I don't think I'll be able to share those results after all.
Yep, soon as I write up the code to format the results; they're just sitting in an ugly pile of JSON right now...
I really hear you on the community and ecosystem issues. It seems like some of these wishlist/gripe issues could be addressed by third-party libraries, but it's not easy to author, share, or take dependencies on third-party libraries in Matlab like it is in some other languages. No package manager, no central package index (like PyPI or Maven Central), no virtual environments, no standard project/library layout or loading mechanism.
I know Toolboxes and Matlab Projects are making some strides in this area. Hope something comes of it.
For #5, you might be happy to know that the Matlab OOP execution engine got faster again in R2020b. In my benchmarking (https://github.com/janklab/matlab-bench), basic Matlab OOP operations like method calls and property access are now 2x or more faster than their Python equivalents.
I'm too old an fogey to begin to pick up anything new so I can't really comment on the Python or Julia parts, etc., but I am in whole-hearted agreement with essentially all 10 items but particularly #10 and as longtime Fortran practitioner the near total abandonment of Fortran for mex.
If find() operates along dimensions would make things a lot easier.
@Image Analyst "madhan, can you give a small example matrix and what you'd expect the output to be if you used find() in the two directions (vertical and horizontal"
To me pretty similar to FIND/IND2SUB, it returns according to the number of outputs at the caller
M = rand(2,3)>0.5
[linearidx] = find(M)
[row,col] = find(M)
% or more generic
A = rand(2,3,4);
[i] = ind2sub(size(A), 20); % A is considered as 1 x 24
[i,j] = ind2sub(size(A), 20); % A is considered as 2 x 12
[i,j,k] = ind2sub(size(A), 20); % A consider as 2 x 3 x 4
Similar to CELLFUN, ARRAYFUN that uses 'uniform' TRUE/FALSE input to let user decide.
In case of FIND, it can be even automatically decided: if N argument is provided output is an index ARRAY, otherwise it returns as CELL. To be precise one can imagine as the following:
For p-dimensional A of size (n1 x n2 x ... x nd x ... x np)
FIND(A, N, 'first'/'last', dim) returns numerical array (n1 x n2 x ... x N x ... np). The ouput result contains 0s in trailing in case there are less than N indexes found along the dimension d. I think 0s is better than NaN, since it would be similar to the way ISMEMBER, UNIQUE deal with non-existing element. It also doesn't have speed penalty as with NaN processing.
FIND(A, [], [], dim) returns cell-array (n1 x n2 x ... x 1 x ... np) where each cell element is the vector array of indices of suitable length.
User might eventually force CELL output even when N is provided by set 'Uniform' flag to FALSE.
madhan, can you give a small example matrix and what you'd expect the output to be if you used find() in the two directions (vertical and horizontal)?
Bruno, what would you propose the output should be for the case where the count is not 1? Taking into account that some searches along the dimension might find the full count and some might not.
For that matter, even if the count is 1 there is potential problem. For example,
B = [1 0 0; 0 1 0; 0 0 0]
then for column 1 and 2 the search would succeed but not column 3, so if a vector is to be returned then it would have to be the vector [1 2 []] . find() has no current semantics for returning something like [1 2 nan] for the case where the search does not find the full count.
How PERMUTE would help? For example what would you do for the workaround like this situation: I want find 1 element along first dimension of
>> B
B =
5×5 logical array
1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
>> dim = 1;
>> [~,loc] = max(B, [], dim) % This is my workaround
loc =
1 2 3 4 5
>> find(B, 1, 'first', dim) % Now
Error using find
Too many input arguments.
>> find(B, 1, 'first', dim) % would love to have this R2050a feature sooner
ans =
1 2 3 4 5
If this were to be changed, then it would be useful to also add a way to return the indices per-dimension (e.g., per-row, per-column), which should probably be done as a cell array even if the count is set to 1
This would be accompanied by an extension of sub2ind() to convert the cell arrays into linear indices.
For example it would be useful to be able to find the indices of the first (or last) element that met a test, per-row or per-column, and when you do, you probably want to be able to index using those values, but they are probably row or column relative, and the current methods to turn row or column relative indices per column (or per row) into linear indexing are a bit clumsy.
As a workaround you can use permute to reorder the dimensions, but I admit a built-in option would be nicer.
+1. Really frustrated not able to call FIND along a specific dimension(s).
FIND(..., 0) throw an error.
>> x=rand(1,10)>0.6
x =
1×10 logical array
1 0 1 1 1 0 0 0 1 1
>> find(x)
ans =
1 3 4 5 9 10
>> find(x,2)
ans =
1 3
>> find(x,1)
ans =
1
>> find(x,0)
Error using find
Second argument must be a positive scalar integer.
Why MATLAB prefers to throw an error rather than returns an empty array is a mystery to me.
Typically when N argument is from some calculation that can be 0 or not.
As example I just bumped into the ERROR when I do this code, I must redefine FIND that returns [] when N is 0 (see MYFIND()) as workaround, that is silly.
Beside that it make perfectly sence mathematically to returns [] for N == 0. The fact that FIND(...,0) must be [] is similar to these
- SUM([]) == 0,
- PROD([])==1,
- FACTORIAL(0) == 1,
- ZEROS(0) == []
which are all implemented as such.
What would be the use case for find(___,0)?
If would be greak MATLAB can do auto-expansion-like for array concatenation
>> A=1:4
A =
1 2 3 4
>> B=[0;
A]
B =
0 0 0 0
1 2 3 4
"Array" means numercial/cell/struct arrays, string and table
@Rik: Well, no it's quite easy. The rule is exactly like auto-expansion and bsxfun. The dimension with length==1 wil be expanded to match the other, so more complicated than this simple rule. I have done here BSXCAT.
My function is not too fast, cannot be invoked with bracket syntax, no benefit from jit accelerator. All that can be handled if TMW implements it.
I find that automatic zero expansion in particular is what I encounter the most often, with the occasional ones expansion and the occasional nan expansion. At the moment, I cannot bring to mind times that I used anything other than those as mass padding. I guess I must have used 255 at some point though.
I bet it would be very tricky to figure out a robust method to determine which value should be expanded, but this would be a very useful feature, especially if you are filling a large array with sub-arrays.
If colon operator supports an array input it would be nice:
A = 1:10;
A(:) : A(:)*100 % automatically converting each row to cell if number of elements differ
Yea Rik , exactly!
@Madhan, so you mean something like a shortcut for meshgrid followed by arrayfun? It would be a neat trick to drop on an unsuspecting new user on this forum ;)
Ah I see , it could raise many ambiguities. Or there could be an additional argument in the colon() like 'nonuniform' for example.
This would be an incompatible change. At present A:B is defined to be the vector A(1):B(1) and some people rely on that (especially people who use the result of size(variable) as the upper bound, not understanding that size(variable) returns a vector). If A:B were change to create a cell for non-scalar A or B, existing code would fail.
It would also lead to code that was not very intuitive. At the moment you can do things like 1 + (A:B) and that would not be legal if A or B were non-scalar since plus() with a cell argument is not legal.
The documentation. I hate to say that because I do think that TMW puts a lot of effort into the documentation, which makes it all the more frustrating when
a) I can't find something that I think should really be in there (and may be, but just not easily found)
b) When I can't systematically search through the contents to find the right page. I run into the issue all the time in the Simulink documentation, and the most frustrating think about this is that I distinctly recall that at some point in the past navigating the contents was simple and intuitive and quick.
Obviously, the documentation is easy to use when one just wants to look up a function or a block. So I'm really talking about those portions of the documentation that need to explain concepts.
As an example, I was just poking around in the Control System Toolbox looking for a discussion on specified vs. unspecified sample times for discrete time systems. How does that property affect results? What about connecting models with specified and unspecified sample times? When to use one vs the other? Thinks like that. Couldn't find it, though it seems to me that's pretty basic stuff for this toolbox.
And the change from help giving the text listing to the HTML doc -- which then has parts which end up with toolboxes with competing/overlying functionality/naming clashes which come up first instead of the base product so one then has to go find the version wanted.
Like what prompted this rant is that intersect just returned a graphics object version instead of the base product...
I couldn't agree more.
Currently, I try to understand what effect Enabled true/false has on the behavior of a AddOns Toolbox. I have both a sandbox and an installed AddOns of my toolbox on my desktop. What surprices are I in for?
So far I only learned that Enabled means enabled.
The year is 2020, and although Matlab may do a good job with autoparallelization, I find it frustrating that parfor, parsim etc. are still reserved to an extra toolbox. Parallel computing is just state of the art today.
Considering the competition, I think it would be a smart move to include at least some limited features of the Parallel Computing Toolbox into plain Matlab/Simulink. This could even promote the toolbox to plain Matlab users.
Agreed!
You're welcome. I think as an answer it would be better suited to a thread about what we're happy about regarding the capabilities of Matlab ;)
The year used to be 2021, and parfeval support was added to the base Matlab. So your wish has been granted.
From the release notes (R2021b):
Language and Programming
Run Code in the Background: Use parallel language to run code asynchronously
You can now run code in the background using backgroundPool. When you run code in the background, you can:
- Run other MATLAB code at the same time as a long running calculation
- Create more responsive user interfaces
Use the background pool with the following parallel language:
- parfeval and related functionality such as afterEach and afterAll
- parallel.pool.DataQueue and related functionality such as afterEach
- parallel.pool.PollableDataQueue and related functionality such as poll
Portable Parallel Code: Share parallel code and seamlessly run in parallel
You can now run parallel code even if you do not have Parallel Computing Toolbox™. When you run portable parallel code without Parallel Computing Toolbox, you run the code in serial on your machine. When you run this code with Parallel Computing Toolbox, you can automatically scale up and run the code in parallel on your local machine, on a remote cluster, or in the cloud.
The following parallel language features are available for prototyping:
When the debugger stops at a breakpoint, why does the focus and cursor go to the breakpoint, as opposed to the command line? I always start typing and corrupt the file being debugged. Is there any utility in having the cursor and control at the breakpoint?
Yes, yes it does! Thanks.
Question: I can see why a user would want the file opened with an indication of which break point is reached, but is there any utility in having the focus and the cursor at that point? Or is it just a thing that opening the file can't be decoupled from focus and cursor location?
Does deselecting the "Automatically open files when MATLAB reaches a breakpoint" preference in the Editor/Debugger Preferences do what you want?
Promote parameters to signals in Simulink models.
It should be possible through some kind of mask. At work, we are running through a problem where we need to change model parameters at run time, one solution is to use set_param, but that makes the simulation dreadfully slow.
The opposite (turning a signal to a parameter) is possible using constant blocks and masks.
I read the limitation comes from how parameters are used inside the models, but this functionality will help developers a lot.
Switching named parameter names for newer functions to not be compatible with old -- from the time of textscan on which goes way back, the parameter was 'headerlines' for the number of headerlines to skip. This followed w/ readtable at least; I forget about who else, but is well-established tradition.
Then up and come along the new readtimetable, readmatrix and friends and voila! it's now 'NumHeaderLines' . While perhaps could have been argued back in the beginning, once it was established, there's no reason to change just to change.
I will grant that some (I haven't tested them exhaustively, may be all) at least also recognize 'headerlines' as well -- that's a start.
But, it isn't reciprocal--the old guys weren't patched to accept the new parameter name so if one gets trained to use the new, then have typos for the previous functions.
Where are the user-interface design folks in the internal design reviews that let this happen??? Doesn't TMW have a specific part of the design review team to try to create and enforce consistency?
Stuff like <heatmap-axis-labels-printing-vertically> and <how-can-i-change-the-pspectrum-time-units>.
Introducing all these mostly opaque graphics objects that users can't get to to fix up for cases not foreseen by TMW.
bar is another abomination have complained about for 25+ years -- there are any number of threads on Answers about annotation for it that have answered over and over.
Not to mention hatching patterns and a much larger supply of prebuilt colors/patterns for automagic cycling...
And last, but not least, the butt-ugly "0" at the origin of the default axes instead of matching the rest of the formatting..yes, it comes that way automagically because of the default '%g' but it's still ugly as sin. :)
"bar is another abomination have complained about for 25+ years -- "
I just saw this complaint from earlier this year next door to the above--I will give TMW credit they did finally uncover the xData position so one can label bars without having to use hidden properties. This is a far cry from having a full set of properties settable for the bars including labels, location, etc., etc., that would be the (imo) proper implementation.
That the mind reading toolbox has never become available for general use? ;-)
Truly, that scares me just a bit if it ever truly became a reality, so not a real frustration.
So what frustrates me? As new capabilities are introduced into new releases, we find that the old tools stay around. This is necessary for backwards capability. And backwards capability is one of the things that makes MATLAB truly great, that I have nearly 35 year old code that still runs. But that leaves us with bloat, in the sense that we now have multiple tools with similar capability. New users find more stuff to learn. The result is a language that is not as clean, not as easy to use as I want.
For example, we have both strfind and findstr. If you read the help for findstr...
findstr is not recommended. Use CONTAINS or STRFIND instead.
But findstr remains in our universe. One day it might disappear. But when you want to use a code to do that, does anybody really remember which of findstr or strfind they SHOULD use? Which is the recommended code, which is deprecated?
I can offer other examples. We have delaunay, delaunayn, delaunayTriangulation, and triangulation. We even have alphaShape, which can be made to produce a delaunay triangulation if you know how to use it. But not all of those tools are truly compatible with each other. What are the differences? Sigh.
How about interpolation? There is the interp1/interp2/interp3/interpn family. But then we have griddedInterpolant. A different tool. Not compatible with the others. And griddedInterpolant requires an ndgrid call to use it, whereas the others rely on meshgrid. Sigh. While I prefer ndgrid to meshgrid because of the x/y inversion issue when you go to n dimensions that surely confuses people, but this difference will surely cause problems.
Likewise, for scattered interpolation, we have griddata. But then we have griddatan. We have scatteredInterpolant. And as confusing, there is no griddata3. griddata works in 2 or 3 dimensions. So they do similar things, but new users then need to learn about 3 tools.
There are other examples. For example, some toolboxes introduce capabilities that already exist in MATLAB. So if you have the correct toolbox, then you get a tool that replicates functionality that already exists. Yes, the new tool in that toolbox probably adds some new feature the author thought was useful. But again you have a source of confusion, especially a problem for newer users. An example of this is xcorr versus crosscorr. You get one from a toolbox.
Yes. I understand the reason for the various tools. But the result ends up as bloat. If I want to do X, then there should be a direct way to do X, ONE way. Not 10 ways, all of which have subtly different interfaces.
Is any of this really a major problem? Well, no. You learn how to use what solves your problem. And gradually, you learn about the alternatives.
I still love to use MATLAB. I just wish it were slightly more perfect.
One related thing that I noticed recently is some stuff with the documentation search:
Great: "not recommended; use histogram instead" is right there
not-so-great: histogram is not one of the top results
Is there a way to boost the listing of common & current functions?
I wasn't suggesting the main product remain stagnant but that TMW fork the development and let those with legacy apps that need that backward compatibility have a version for that purpose (basically only for that purpose altho I don't know I'd go so far as to prevent writing new stuff with it; maybe for noncommercial work only as part of license for new development.
Then the forked version could be the Phoenix... :)
But that is the problem. I truly want MATLAB to improve. Were it to stay stagnant would mean it would be now long forgotten, just another dead programming language, remembered fondly by some, and as a joke to others.
I just want it to rise fully formed from birth, perfect in every way. And THEN it can stay stagnant, as how can you improve on perfection? Hey, dreams usually are a bit silly.
Just a youngster, still... :)
That would have been the same year I went off on own and returned to the farm and began the transition to turning the consulting gig into the sideline. I'd've guessed you were @TMW before then.
I was pretty keen on picking up the dataset when it came out -- it had a lot of useful functionality for the type of data I was seeing a lot of at the time...
I was/am also pleased a similar functionality ended up in the base product where (imo) it should have been from the start instead of being introduced in the toolbox.
Just disappointed the direction taken to not have a more thorough beta test process in which such warts can be found and a final direction taken that obviates the duplication or that can't/couldn't find a way to clean the one up to be the latter.
Didn't mention the issue of namespace proliferation/pollution this also causes.
Surprised nobody's commented the fork idea -- even to say it's ludicrous! :)
19 years!!! Wow.
From my perspective:
Some of us individually, and absolutely MathWorks as an organization, has learned more about how to design and implement code over the lifetime of the organization and our tenures in the organization. Certainly the code I write now doesn't look the same as the code I wrote when I joined MathWorks nineteen years ago. I still (usually) recognize my own coding style, though I have surprised myself by asking "Who the heck wrote the code that way?" only to realize by doing some archaeology in our source control system the answer is "Some hack named Steve Lord." ;)
Some of those changes are related to the introduction of new tools and constructs. When I started, MATLAB didn't have a shipping unit testing framework, anonymous functions were still a couple years in the future, graphics functions returned numbers instead of handle objects, and in fact handle objects (and the classdef-based class system) hadn't been developed. [I was involved in reviewing the design for the classdef-based class system over a decade ago.]
Others represent us understanding that certain patterns and anti-patterns exist. [If we were to redesign the language now, personally I would suggest that eval end up deep in an internal package and/or have a much less discoverable name.]
As for the specific case of dataset and table ... that "more-or-less" in your description hides a lot of lessons we learned watching how people used dataset that led us to what we believe to be a better design for table.
Regarding your last point; I was amazed that TMW (have I already earned the nerd-status to be allowed to use this abreviation, or do I need to stick around for a few years longer?) introduced the boxchart function in R2020a. Why not move the boxplot function to base Matlab instead (along with the quantiles function, which seems to be the only part that didn't survive the move)? Now you have two functions that have the same job, but a subtly different syntax. And the second one returns 6 objects per box (without documentation). Great if you want to dive in and change the appearance, but a bit strange considering the OOP-direction of the past years. At least the doc for boxplot refers to boxchart.
About the legacy support: I really like that you can call cellfun with the char input which makes it run a lot faster. This is one of the few(?) exceptions where legacy code runs faster.
A=cell(10,12);for n=1:numel(A),A{n}=rand(randi(20,1,2));end
FSpec=['time elapsed: %.1f ' char(181) 's\n'];
fprintf(FSpec,10^6*timeit(@() cellfun('prodofsize',A)))
fprintf(FSpec,10^6*timeit(@() cellfun(@numel,A)))
fprintf(FSpec,10^6*timeit(@() cellfun(@(x) prod(size(x)),A)))
One cellfun pet-peeve: why didn't they use 'prodofsize' instead of 'numel'? Did they expect people to string this together like this?
dimcount=max(cellfun('ndims',A(:)));
out=ones(size(A));
for dim=1:dimcount
out=out.*cellfun('size',A,dim);
end
Or did the numel function not exist when they introduced cellfun?
Rabit hole time: this is no longer really documented, but this is apparently working Matlab syntax:
numel([],1,[2 4 6],1)
I hardly ever work with user-defined classes, but apparently this is the way Matlab used to determine the expected number of outputs for subsref. Since R2015b you're forced to use numArgumentsFromSubscript instead (I don't know what you would have to do if you want your class to work on both, you probably have to overload both).
Amen, brother!
And there's the Statistics dataset which was then introduced into base product more-or-less as table -- similar functionality but incompatible syntax.
The whole new bunch of replacement histogram and binning functions--also incompatible behavior.
And it goes on and on and on ...wonder just how much of the slower response is owing to all this extra baggage?
What would be the possibility of forking development to build a new product that keeps the best of the new and venerable but doesn't worry about the really, really old legacy stuff.
I'd give folks who have those apps a nonexpiring license for the present release but no new features; new sales and development goes with the svelte (and hopefully, much leaner/faster) release.
Then, be more disciplined in introducing new features into the wild before they're ready for prime time so don't have this duplication like with dataset when it wasn't quite what was hoped for.
The allowing toolboxes to have so many overall useful goodies that then get by public demand moved to base package but always, it seems, with twists, should also be greatly reduced.
See Also
Tags
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)