What frustrates you about MATLAB? #2
Rik
on 28 Jun 2020
Latest activity Reply by Mike Croucher
on 25 Jan 2024
Similar to what has happened with the wishlist threads (#1 #2 #3 #4 #5), the "what frustrates you about MATLAB" thread has become very large. This makes navigation difficult and increases page load times.
So here is the follow-up page.
What should you post where?
Next Gen threads (#1): features that would break compatibility with previous versions, but would be nice to have
@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)
312 Comments
Feel free to use the 'Ideas' Channel in the new MATLAB Central 'Discussions' section for the sort of conversation you were encouraging here Ideas - MATLAB Central Discussions (mathworks.com)
>> tAdd.IP=7;
To assign to or create a variable in a table, the number of rows must match the height of the table.
>>
No automagic expansion on table assignment...another need for b-ugly repmat() for no apparent reason...and there's no convenient shorthand 1D repmat() so have to remember to not forget the trailing ",1", too.
Unless you do remember there is repelem which I almost never do. But, it still doesn't quite fit the bill because you still have to transpose the result in which case may as well make it explicit with repmat() to start with.
Frustrating....
Actually your second example is more nearly the use case I had that prompted the complaint although I simplied it to just show the error. The use case of
t = table([1;2;3])
t=addvars(t,7,'After','Var1','NewVariableNames',{'NewVar'})
The specific case involved putting the new variable in a specific position within a table with a number of variables, not at the end that the t.NewVar(:) syntax does. While it does avoid the explicit vector creation, one then needs a second line to relocate it where needs to go.
I still think with the table class, both forms should be available...
With the table the height is known internally so it should be possible to simply expand the constant to the length needed behind the scenes, so "way back when" I tried to do that.
For some reason the analogous array syntax never struck me over the time the table has been in existence (and for the prior dataset object in the Stat TB before it got superceded (another rant over that, but that's another topic). I THINK that is probably owing to the specific text of the error message; it tells one what the problem is but it doesn't provide the hint as to another form for the expression; it implies one must assign the matching vector -- and with that particular LHS addressing, that is true.
On retrospect, it is apparent with the analogy that the (:) addressing would work, agreed; I admit I just didn't think of it Lo! those many years ago when first ran into it and had just used the klunky workaround by rote ever since, hadn't tried to find another syntax and for some reason it just didn't strike me to try...
By "native type auto-expansion", I think you mean like this:
x = [1;2;3]
x(:,2) = 7
The analogous tabular thing is this:
t = table([1;2;3])
t{:,2} = 7
But t.IP(:) = 7 is probably what I would choose.
Ah! I hadn't thought of using the explicit (:) addressing mode on the LHS, Peter, thanks...obviously, I guess, given the rant. :)
Thanks for pointing that out, while not as elegant as the native type auto-expansion (which I think is what most experienced MATLAB users would expect) it is definitely a significant improvement over the manual vector creation route.
This will do what you want
t = table([1;2;3])
t.IP(:) = 7
but I hear what you are saying.
My preference is able to have combinations returning simple standard array (with perhaps optional input argument to do so) not table.
@Bruno Luong thanks very much for sharing your example and experience. I'm happy to see that performance of combinations coupled with rowfun is close to that of your handwritten code and not orders of magnitude slower.
As a quick comment, you likely could write code that looks like this without the need for the wrapper:
FactorValuesCell = rowfun(@(varargin) cinematic(varargin{:}, PLenErrorArray1, LensErrorVariableNames) , ...
Tgammac, "OutputFormat", "cell", "SeparateInputs", false);
and perhaps that shaves off a bit of runtime.
All of this said, if you prefer your current approach and that works for you, then stick with it! That's part of the joy of MATLAB.
When we are thinking about the implementation of T{ROWS, NONSCALAR_COLUMNS} then there are a small number of conceptual routes:
- temp = [T.data{:}]; output = temp(ROWS, NONSCALAR_COLUMNS); -- that is, convert everything into one large array first, then extract the desired rows and columns from that array
- temp = [T.data(NONSCALAR_COLUMS){:}]; output = temp(ROWS,:); -- that is, extract the desired columns, convert everything in those columns to one large array, then extract the desired rows from that array
- temp = cellfun(@(M) M(ROWS), T.data, 'uniform', 0); temp = temp(NONSCALAR_COLUMNS0; output = [temp{:}]; -- that is, extrace the desired rows from every column, extract the desired subset of columns, then convert what results into one large array
- temp = cellfun(@(M) M(ROWS), T.data(NONSCALAR_COLUMNS), 'uniform', 0); output = [temp{:}]; -- that is, extrace the desired columns, extract the desired rows from those columns, then convert what results into one large array
However, considering there is one cell for each column, it never makes sense to bother converting the data in a column that is not going to be used, so we can rule out #1 and #3.
We are then left with the question of whether we extract everything in the desired columns into an array and extract the desired rows from that, or if we instead run a function on each cell to extract the desired rows and smoosh together the extracted rows into an array.
Under what circumstances is it going to be faster to convert everything into an array and extract rows from the array? Well, it would potentially be faster to do that if you are extracting all of the rows, if you specially detected ':' as the subscript and short-circuited doing the actual indexing. What about the case where the ':' index was already converted to 1:height -- what is the performance comparison between temp(1:end,:) compared to temp(:,:) ? The answer is that if you use (:,:) then MATLAB knows enough to not bother copying any data, but that if you use (1:end,:) or (1:size(temp,1),:) then MATLAB has to make a copy of the array. So if the code detected : as the first index then smashing everything into an array first and then not even bothering to index rows out of it could certainly be faster than cellfun() to extract only the wanted rows.
... but of course if you are going to bother to detect : as the subscript, T{:,NONSCALAR_COLUMNS} then it would be easy enough to skip the cellfun if you were doing the cellfun approach.
So when does the cost of running one anonymous function per cell to select rows, and smash the results together, exceed the cost of smashing everything together and selecting rows?
It would have to be a case such as having a lot of numeric variables that are already the same datatype, but with relatively few rows in the table. When you have relatively few variables, the cost of the cellfun becomes minor compared to the memory copying with large number of rows. When the variables are different data types, the cost of converting them to the same datatype goes up with the number of rows -- you would rather convert fewer rows.
My suspicion is that the cellfun approach would be faster most of the time (assuming that not all rows were being selected with a ':' index)
I ran a rowfun last night in which each variable needed to be processed . I ended up needing to approxiimately varfun(@(varargin) cellfun(@(M)AFunctionCall(M), varargin, 'uniform', 0), TABLE) ... which was not the most obvious of interfaces, and which many people would likely not think to do.
As Peter mentioned above, there is work being done at improving the performance of subscripting on tables and the use case of extracting a single row using {} is definitely a common use case that needs to be improved.
Walter regarding the two approaches you mentioned, I think both of them are valid, but both the implementation approaches yield better or worse results depending on the size of the tables you are working with, so the idea would be to implement it in a way that improves the performance for all kinds of tables. But as I said, this is something that Mathworks is actively looking into.
I run rowfun on my example of camera assembling and performance is good compare to my own combination array output.
My code:
rresolution = 4;
nlenses = 6;
gammac = AngleComb(nlenses, rresolution); % 4^6 x nlenses = 4096 x 6
ncomb = size(gammac,1);
tic
for i = 1:ncomb
gc = gammac(i,:);
% cinematic is process of simulation I cannot share
% PLenErrorArray1, LensErrorVariableNames is variables defined above
FactorValues = cinematic(gc, PLenErrorArray1, LensErrorVariableNames);
end
array_time = toc
function c = AngleComb(nlenses, rresolution)
% c = AngleComb(nlenses, rresolution)
% Generate an array of combinations of rotation angles (degree)
theta = (0:rresolution-1) * ((360) / rresolution);
c = cell(1, nlenses);
[c{:}] = ndgrid(theta);
c = cat(nlenses+1, c{end:-1:1});
c = reshape(c, [], nlenses);
end % AngleComb
I try the table access using combinations like this
theta = (0:rresolution-1) * ((360) / rresolution);
carg = repmat({theta}, 1, nlenses);
Tgammac = combinations(carg{:});
tic
FactorValuesCell = rowfun(@(varargin) cinematic_wrapper(PLenErrorArray1, LensErrorVariableNames, varargin{:}), ...
Tgammac, "OutputFormat", "cell");
table_time = toc
function FactorValues = cinematic_wrapper(PLenErrorArray1, LensErrorVariableNames, varargin)
gc = [varargin{:}];
FactorValues = cinematic(gc, PLenErrorArray1, LensErrorVariableNames);
end
On my PC
array_time is 0.080248 seconds.
table_time is 0.087576 seconds.
So it is very good.
I think finding the right way to use combinations efficiently by rowfun is ot very straighforward. The way rowfun accept a row as arguments list is fine but migh be not evident for peopla whos are not familiar with MATLAB. Same comment for wrapping using anonymous function if other input parameter are needed. In short finding the right path and coding is not evident. (It takes a week for several high level MATLAB users to figure that out).
On this specific camera code, I'll stay with my priginal ndgrid and for-loop. It is more readable and maintanable to me.
Walter, I think I misspoke. The correct info will follow soon.
(I think your cellfun command needs C(rows), not C{rows}.)
Thanks I'll do the simulation of camera assembly.
rowfun seems to be the way to go. This tip should be written in the documentation of combinations IMHO.
The whole point of combinations returning a table is the combinations --> rowfun workflow, as @Peter Perkins mentioned above.
For example, if you have a function that takes 6 inputs, and generates one output, you can sweep over all of them like this
f = @(a,b,c,d,e,f) cosd(a).*cosd(b).*cosd(c)+sind(d).*sind(e).*sind(f);
shift = 0:90:270;
T = combinations(10+shift, 20+shift, 30+shift, 40+shift, 50+shift, 60+shift);
tic;
z = rowfun(f, T, "OutputFormat", "uniform");
toc
tic;
w = zeros(height(T),1);
for k = 1:height(T)
onecomb = T{k,:};
w(k) = f(onecomb(1),onecomb(2),onecomb(3),onecomb(4),onecomb(5),onecomb(6));
end
toc
assert(isequal(z,w));
The expression with rowfun is more compact and more performant.
Of course, if you can get down to plain old doubles, you might be happier. Frequently, for the kinds of things people want to do, you have a variety of types running around, which makes even trying to pull out T{k,:} challenging since you may not at all get what you expect.
@Bruno Luong, if you would indulge us, I would appreciate seeing the timing of your smart phone camera example done with this kind of workflow.
If a row is what you need, T(i,:) is much faster than T{i,:}
? Is the implication that T(i,:).Variables would be expected to be faster than T{i,:} ??
Things are not as simple as "TMW should re-implement that part", for reasons that are deeper in the language
I have experimented with tables with mixed data types... but I am having difficulty finding any deep reason why output = T{rows, nonscalar_columns} should be treated internally as
temp1 = T.data(nonscalar_columns);
temp2 = [temp1{:}];
output = temp2(rows,:)
rather than as
temp1 = cellfun(@(C) C{rows}, T.data(nonscalar_columns), 'uniform', 0);
output = [temp1{:}];
The only thing I have come up with so far is that there is some weird semantics with tables containing tables... you can get it showing up like
var1 var2
1 x 10 table 1 x 10 table
1 x 10 table 1 x 10 table
1 x 10 table 1 x 10 table
but so far if you try to put tables with more than one row into the entries then you cannot seem to do that unless you make the variable into a cell. It is looking like the variable names all have to be the same within one column in this case... and doing various things collapses the rows-that-are-tables into a single multi-row table. Unfortunately between yesterday and today I have forgotten the steps that got me to that kind of table. I might have been playing around with cell2table() or struct2table()
@Peter Perkins I give you an example of a concrete "do something with onecomb" in a recent code I just worked with few weeks ago:
I work in simulating a performance of smart-phone camera assembling in a production line, and there are 6 lenses where I simulate eache lenses rotates by 90 degres. There are then 4^6 = 4096 combinations that I can (but not) generate with combinations function.
Then we have a neural network (NN) to evaluate the performace of each "onecomb " The NN last 1.7 microsecond for each combination. The 4096 combinations costs me only about 7 ms to simulate.
If I use table it will take 7 seconds to access the row by T{j,:}, 1000 times more than my simulation !!!
Furthermore from Walter's digging on code, as I understand each memory foot print of the statement
onecomb = T{j,:}
is like the whole extra content T{:,:}. When I think about using the table and then each row access manner the table is temporary rebuilt, I just scratch my head and ask myself: why not getting the T{:,:} at the first place and work directly from that.
This is again a though provoking from me.... ;-)
PS: Timing in code in script and function doesn't matter much at least in R2023b.
CRIMINY! I can't believe I didn't mention rowfun!
combsT = combinations(param1,param2);
[results1,results2] = rowfun(@simulationFun,combsT);
Wow, this subthread got real long real fast.
Walter (and Bruno), I was interpreting Bruno's "Or output as cell array" as meaning "Or output as cell array with one value in each cell". You are correct that "Or output as cell array with one column in each cell" would be equivalent to how table stores the data. (yes, struct shows you this. You guys know this, but for the record: DON'T RELY ON WHAT YOU FIND BY DOING THAT!!! THE INTERNALS ARE GUARANTEED TO CHANGE! GUARANTEED!) In any case, what would you then do with that cell array? Getting one "row" means using cellfun or a loop to dig into those columns. Not much fun. Getting a scalar struct of vectors, same deal. Tables let you slice both ways.
Bruno, this is a really tought-provoking topic. It will definitely give us things to think about. Things are not as simple as "TMW should re-implement that part", for reasons that are deeper in the language, but there are some things for us to think about to improve "extract one row from a table". So remember: "One thing is to not assume that things never get better."
Some more thoughts:
- If you are doing timings, run in a function, not a script, and not at the command line. I'm not even 100% up to date on all the optimizations the language can do in functions vs. scripts ("One thing is to not assume ...") , but functions are how you are actually using your code, so do the same for your timings.
- "% do something with onecomb ... " Ay, there's the rub. "Something" might be really fast, but for the output of combinations, especially for the parameter sweep use case, it's likely an expensive calculation that dwarfs any table subscripting. So far all the timings I've seen in this thread include "nothing" as "something", but maybe I missed it.
- For a parameter sweep, I can imagine having some simulation, and doing this:
combsT = combinations(param1,param2);
for i = 1:height(combsT)
[result1(i),result2(i)] = simulationFun(combsT.Param1(i),combsT.Params2(i));
end
which is almost certainly gonna be fast enough and is super readable.
- As I said, T.Variable and T.VariableName(i) are the fastest operations to extract data from a table. If a row is what you need, T(i,:) is much faster than T{i,:}, but of course if a numeric vector is what you need, then you'd need to turn t(i,:) into that.
- I mean, without knowing what's being done with each of those rows, this is somewhat of an abstract discussion. As Steve said,
combsT = combinations(param1,param2);
combsX = combsT.Variables;
% then a tight scalar loop
would be my suggestion in tha absence of more details.
OK now the debugger goes there.
I see now what it is damn slow when single row of a table extracting
TMW should re-implement that part.
extractData is only used if there are multiple variables being extracted at the same time. Your example happened to use a single variable.
Can you please run this
dbstop('in', fullfile(toolboxdir('matlab\datatypes\tabular\@tabular'), 'extractData.m'))
T=table(rand(10)); r=T{1,:}
does it stops?
I mean this:
%%
ThisCode % will run if you hit ctrl+enter (or cmd+enter)
%%
This is very handy during debugging, but can introduce unexpected behavior in edge cases.
I don't know what is section, I run from command line window.
Did you run from a section? Because in my experience breakpoints sometimes don't work as expected when I run code from a section.
Are you sure? I do
edit('C:\Program Files\MATLAB\R2023b\toolbox\matlab\datatypes\tabular\@tabular\extractData.m')
then put the break point at the first line of the function "vars = t.varDim.subs2inds(vars);" and run this
T=table(rand(10))
T{1,:}
it doesn't seem to stop at this file.
toolbox/matlab/datatypes/tabular/@tabular/braceReference.m has the code.
It uses toolbox/matlab/datatypes/tabular/@tabular/extractData.m to try to [ ] all of the selected cell columns together and then it extracts the desired row from the result.
... Definitely not the way I would have coded it.
Ah yeah but then data{1} migh not be the right class.The right class is the largest numerical (double) IMO.
Anyway this is not the point of my post, the point is I wonder how table row access is implemened so that is so slow.
No, at the point you initialize onecomb, j is not initialized.
Yeah it's betrter but
...'like',data{j}
rather than data{1}
onecomb = zeros(1,nvars);
That should probably be more like
onecomb = zeros(1,nvars,'like',data{1});
to avoid unnecessary class conversions.
@Peter Perkins "inefficient row access": you are correct that selecting one row of a table is not as fast as the same thing on a numeric array. Never will be. Much more going on. Is it faster than the equivalent operation on data that are stored as separate arrays? Maybe not at run time, but certainly it is faster at code compose time. Is it faster than the equivalent operation on a cell array? I forget,"
This code show it is VERY slow access table rows, and it is worse than accessing column cell. Do I miss a better method?
a=(1:3);
mctarget = 1e4;
p = ceil(log(mctarget)/log(length(a))); % == 9
%% All numercial data
c=repmat({a},1,p);
BigT=combinations(c{:});
mc=height(BigT);
tic
for k=1:mc
onecomb = BigT{k,:};
end
ttable = toc
% This will access to "internal" data of table, which has each column
% in a cell, and rebuild the all table rows
tic
StructFromBigT = struct(BigT); % Thanks Walter
data = StructFromBigT.data;
nvars = size(data,2); % == p
for k=1:mc
onecomb = zeros(1,nvars);
for j=1:nvars
onecomb(j) = data{j}(k);
end
end
tStructFromBigT = toc
In general struct() of an object reveals its properties, including its hidden properties. This is inherited from the days of schema.m when a MATLAB object literally was a struct that had been "blessed" to have a class name attached to it.
I have encountered a small number of objects that struct() could not be used on, but it works the great majority of the time.
T = combinations(uint8(1:50), ["r" "b" "g" "c" "y" "k" "h" "s" "v"]);
struct(T)
So yes, the data structure does have a cell array with one entry per variable. And there is a fair bit of overhead in code such as toolbox/matlab/datatypes/tabular/@tabular/dotReference.m . I cannot see any reason why using T.VARIABLE(index) would be faster than T{index,Offset} considering all of the checking going on.
@Walter Roberson "Combinations could reasonablly store each column in a separate cell entry. Each column is the same datatype."
I suspect that is actually pretty close to table internal storage. But no one in the TMW staffs is willing (allowed) to disclose it.
@Paul thanks for the link of the blog. I was not aware about it.
The argumenet "most design team prefers table" or bump into syntax confusion for optional argument is a little bit weak IMHO.
Sorry, when I said wasteful storage I compare with pure numerical array (uniform data to be combined) not mixing type. In that case any storage including table is wasteful. But I agree that the extra memory of table is negligible.
I cannot imagine user want to access to columns of the combinations table, most likely the ROWS of the combinations where all the combinations would be needed. Acessing row of table is PAINFULLY slow (sooo slow that the server cannot run my demo code bellow, I put the results run on my PC) and sometime it has a dangeruous trap in mixing data type (convert numerical data to string)
a=(1:3);
mctarget = 1e4;
p=ceil(log(mctarget)/log(length(a))); % == 9
%% All numercial data
c=repmat({a},1,p);
BigA=BrunoComb(c{:});
BigT=combinations(c{:});
mc=height(BigA);
size(BigT) % 19683 x 9
%% Time accessing rows, numerical types
tic
for k=1:mc
onecomb = BigA(k,:);
% do something with onecomb ...
end
tarray = toc % 0.0076
tic
for k=1:mc
onecomb = BigT{k,:};
end
ttable = toc % 7.6962
% So at the end we better do the conversion from table
% to standard array to work with, like this
tic
AfromT = BigT{:,:};
for k=1:mc
onecomb = AfromT(k,:);
end
tarrayfromT=toc % 0.0081
%% Mix data
c{end}=["apple" "orange" "cherry"]; % we mix string to numerical
BigA=BrunoComb(c{:});
BigT=combinations(c{:});
mc=height(BigA);
size(BigT)
%% Time accessing rows, numerical types
tic
for k=1:mc
onecomb = BigA(k,:);
end
tarray = toc % 0.0289
tic
for k=1:mc
onecomb = BigT{k,:}; % But this is NOT what user wants due to casting
end
ttable = toc % 264.8725 !!!!
% So at the end we better do the conversion from table
% to standard array to work with, like this
tic
AfromT = BigT{:,:}; % But this is NOT what user wants due to casting
for k=1:mc
onecomb = AfromT(k,:);
end
tarrayfromT=toc % 0.0245
%% Generate of combination output are standard array and not table
% for numerical data
function c=BrunoComb(varargin)
n = length(varargin);
c = cell(1,n);
[c{end:-1:1}] = ndgrid(varargin{end:-1:1});
isc=cellfun(@iscell,c);
if any(isc)
c(~isc) = cellfun(@num2cell, c(~isc), 'Unif', false);
else
iss=cellfun(@isstring,c);
if any(iss) && ~all(iss)
c(~isc) = cellfun(@num2cell, c(~isc), 'Unif', false);
end
end
c=cat(n+1,c{:});
c=reshape(c,[],n);
end
So what to do with combinations to overcome this?
I would guess many people use table but not fully aware about slow row accessing, and propertly convert the result of combinations to array.
"Combinations could reasonablly store each column in a separate cell entry. Each column is the same datatype."
Yes, we could have. I remember us discussing doing that. And I think everyone taking part in the discussion for this answer in this Answers post would be perfectly capable of operating on the output if it were a cell array each cell of which contains one column of the data that makes up the combinations.
But look at the people who are taking part in this discussion. The lowest "MATLAB Answers level" among participants as I type this is Level 7. [Peter and I are Staff, which doesn't necessarily say anything about our level of MATLAB knowledge, but trust me when I say we're both experts in MATLAB.]
In my experience, operating on tables (especially when you operate on them variable-wise, T.Var1 style) generally tends to be more accessible than cell arrays to newer users, people who might be say Level 4 or lower on Answers. [Yes, I know "MATLAB Answers level" isn't perfectly correlated with MATLAB skill level, but would you accept the premise that the correlation is likely positive?]
There are functions and classes in MATLAB that have a steeper learning curve and a higher point on the curve where you could say "I have learned this." Because of its purpose combinations isn't intended to be such a function. Ideally, I hope it's no harder (or not much harder) to learn than functions like sin or (basic uses of) plot. Think gentle slope rather than rock climbing wall. Did we succeed in making the combinations function useful, easy to learn, and easy to use? It hasn't been out for that long but I'd like to think so.
I'd also like to point out that even if you don't want to directly use a table array after calling combinations, converting a table all of whose variables are the same type (or types compatible for purposes of concatenation) into a homogeneous array is easy. It's just 10 characters more, using a syntax like that T.Var1 I mentioned above.
T = combinations(1:5, 6:10); % table
A = combinations(1:5, 6:10).Variables; % array
isequal(T.Var2, A(:, 2))
D = ["Huey", "Louie", "Dewey"];
S = combinations(D, D).Variables
Combinations could reasonablly store each column in a separate cell entry. Each column is the same datatype. Takes no more memory than a table would as a table internally uses cells to store the columns.
T = combinations(uint8(1:50), ["r" "b" "g" "c" "y" "k" "h" "s" "v"]);
C = {T.Var1, T.Var2};
whos T C
%timing to access one variable from each row
tic; for K = 1 : height(T); T{K,2}; end; toc
tic; for K = 1 : height(T); T.Var2(K); end; toc
tic; for K = 1 : height(T); C{2}(K); end; toc
%timing to access a row. Because of the string entry the following
%automatically convert the uint8 to string()
tic; for K = 1 : height(T); T{K,:}; end; toc
tic; for K = 1 : height(T); [T.Var1(K), T.Var2(K)]; end; toc
tic; for K = 1 : height(T); [C{1}(K), C{2}(K)]; end; toc
%but of course for wider arrays, writing the [] explicitly can become
%unworkable, so let us try some code that handles each cell in turn
tic; for K = 1 : height(T); arrayfun(@(IDX) string(C{IDX}(K)), 1:size(C,2)); end; toc
The repeated construction of anonymous functions with captured variables, and calls to those functions by the arrayfun, would be expected to be the slowest approach... but it is still more than twice as fast as the T{K,:} version.
Bruno, I'm not gonna claim your past experience is not real, but I'd like to address two of your comments:
1) "wasteful table storage/decoration": there's nothing at all wasteful about how tables store data. Your suggestion of a cell array as the output type of the combinations function in that example would not be great memory-wise compared to a table. Of course in that small example, memory doesn't matter. But a cell array would store every value in the output as a separate MATLAB array--that's how cell arrays work, they can hold anything in any cell, no homogeneity required. But it means a hundred-something bytes extra for each value. A table, OTOH, would store only two homogeneous arrays: a uint8 column and a string column.
T = combinations(uint8(1:50), ["r" "b" "g" "c" "y" "k" "h" "s" "v"])
T =
450×2 table
Var1 Var2
____ ____
1 "r"
1 "b"
[snip]
C = table2cell(T)
C =
450×2 cell array
{[ 1]} {["r"]}
{[ 1]} {["b"]}
[snip]
whos T C
Name Size Bytes Class Attributes
T 450x2 25145 table
C 450x2 160650 cell
T1 = T.Var1;
T2 = T.Var2;
whos T1 T2
Name Size Bytes Class Attributes
T1 450x1 450 uint8
T2 450x1 23496 string
25145/(450+23496)
ans =
1.0501
Not a lot of memory overhead to the table, and it disappears once you get beyond toy examples. A cell array with a few million rows? Good luck. Lemme reiterate that: memory-wise, a cell array is a terrible way to store tabular or homogeneous data. (Also terrible from a usability standpoint--what functions would you call on it? Not alot of things you can do with it.) Cell arrays absolutely have a use, but those ain't it.
What about compared to a numeric array?
X = rand(100000,2);
T = array2table(X);
whos X T
Name Size Bytes Class Attributes
T 100000x2 1601191 table
X 100000x2 1600000 double
Once you get past small data, memory overhead in the table is in the noise.
2) "inefficient row access": you are correct that selecting one row of a table is not as fast as the same thing on a numeric array. Never will be. Much more going on. Is it faster than the equivalent operation on data that are stored as separate arrays? Maybe not at run time, but certainly it is faster at code compose time. Is it faster than the equivalent operation on a cell array? I forget, but what you end up with from the cell row is not super helpful from a usability standpoint and c.f. my earlier point about memory.
Actually, I bet that you really mean "inefficient access of scalar elements" rather than "inefficient row access". But same idea.
Tables are best when you use vectorized operations. What to do about that if you can't write your code like that?
- One thing is to not assume that things never get better. Subscripting performance for tables, both reference and assignment, has gotten a lot faster over the last few years.
- Some subscripting syntaxes are faster than others. T.Var(1) is always gonna be faster than T{1,"Var1"}. Much more going on in the latter.
- Another thing is to not throw the baby out with the bath water. In a tight scalar loop, if accessing data in a table is a bottleneck, it is very often possible to hoist that part of the code into a function call, so the original loop becomes something like [t.Var1,T.Var2] = loopingFun(T.Var3,T.Var4). The rest of your code that holds data in a table stays the same.
3) You did not ask this, but performance of the new-in-R2023a(IIRC?) "math on homogeneous numeric data stored in a table" mostly compares favorably with math on core numeric array types, at least for not-toy-example cases.
In case you hadn't seen it and if interested, The new combinations function in MATLAB – for cartesian products and parameter sweeps blog post explains some of the design decisions that went into the combinations function.
I've used it extensively with the pro bono work I've done for local community college foundation financial records stored in a myriad of Excel spreadsheets and found it quite effective and not at all difficult to code with; in fact, the extra features built in with varfun and so on have been quite helpful and the variable addressing is no more complex than a struct.
Now, if one has monolithic arrays and numeric calculations galore, sure, it won't be the tool for such computational applications, but it has its place. Anything can be done at a lower level, but the convenience of some of the higher level features I've found very advantageous and speed/memory aren't issues with the size of these tables--although when compiled to a standalone app it does slow down quite noticeably, but I don't think that's the fault of the table itself but generally the compiler.
Exactly, I only use table when interact with end user, or readtable from csv file and then quickly convert it to lower level data.
Programming with Table is tedious, long, and slow. You guess it I'm not a big fan.
Maybe doesn't "need", but it surely is handy and convenient for user interactive interface.
Granted, it does bog down with really large tables.
Or output as cell array.
No serious programing needs the wasteful table storage/decoration and inefficient row access.
T = combinations(uint8([1 8 6]), ["red" "blue" "green"])
class(T.Var1)
The existing functionality does not require that the outputs are all the same data type, or even compatible data types. And that is useful. Still, it would probably be reasonable for there to be an option controlling the desired output format... maybe even the desired output class.
Having used it more extensively in the last number of months, my biggest frustrations still surround the App designer. It has improved massively since its early days, but some 'features' or lack of features are still immensely irritating for day to day usage, including (though I'm sure I forget some):
- Uneditable code - I understand why this is there, but Matlab is used by a lot of expert users as well as new users. It is ridiculous that we have no way to edit large blocks of code in an App Designer app. There ought to be an expert mode where you can simply edit anything you want, with all the risks that come with that in potentially making your app no longer work. I want to be able to edit the order of my properties and methods blocks - I always put my public ones at the top, private ones below, it's just my way of working - I don't want to enforce it on others, but I want the capability to do it myself. Also I always name my objects as 'obj' in every class I've written in over 10 years. Being forced to name it 'app' or 'comp' in un-editable function signatures is infuriating because I still just naturally use 'obj' in the code body without thinking and it wastes time fixing it.
- Public components - part of the same problem as the first point. I very rarely want all the components of my app to be public properties so that any external code can just mess with them as they choose. I want them private, and again, just having the option to do this is all I need. If other people want theirs public then fine, let them do that too, but at least give a choice to edit this.
- Save/debug behaviour - why does this behave differently to in the regular editor. It is so irritating and time-wasting when as a user of Matlab a way of working is ingrained in me from 17 years of working in the software and then this 'new' editor behaves differently. In the regular editor, when you are in debug mode you can edit a file and when you hit 'Save' it stops the debug and saves the file. I use this literally all the time when fixing bugs from a breakpoint. In App Designer you simply cannot save in Debug mode. You first have to click to stop debug and then save. Of course it's a small thing when you do it once, but when you are doing this 10s of times per day the inconsistency between the two editors seems totally un-necessary,.
I don't personally use any others these days. I used to program some using C++ and wxWidgets in the old days, but more recently only in Matlab. Previously within Matlab I would use GUIDE a long time ago and, more often, programmatic GUIs using java components and the GUI Layout Toolbox from File Exchange.
Edit Rik: added link to GUILT.
What other software/languages do you use to develop GUIs?
How slow getdata from the image acquisition toolbox is (at least for genicam cameras I have been working with). I had to drop matlab for our real time computer vision applications because of that.
Compare sldd files and publish a HMTL report using Matlab script:
I can compare .slx, .m, etc. files using the visdiff function and publish a report using the publish function like this:
compare = visdiff('File1.slx', 'File1_changed.slx');
publish(compare, 'format', 'html');
But I can´t do the same with .sldd files:
compare = visdiff('File1.sldd', 'File1_changed.sldd');
publish(compare, 'format', 'html');
Error using visdiff
Unable to compare 'C:\basic\File1.sldd' and 'C:\basic\File1_changed.sldd' without opening the Comparison Tool.
Why if I can generate an HTML report using the Comparision Tool in the same way, both for .slx and .sldd files?
I like matlab a lot, but the GUIDE can be an exercise in frustration with a lot of seemingly 'you can't get there from here' impediments. I've been using GUIDE since 2015 and if I had it to do over I would not have chosen it. Some simple things like displaying labels and titles for axes can be impossible no matter how many times you call the 'Visible' property. If one axes in a set of axes gets pushed back behind other axes using send to back, it can totally corrupt your layout even if it looks good in the fig file. Toolbar editing is a mess where it's very hard to create a user defined tool that is on an equal footing with the matlab built-in toolbar tools. Every once in a while you have to clear out the unnecessary callback functions auto generated in a 3000 line gui file. The handles object is it's own can of worms and can cause lots of frustration when not properly set and retrieved, a very weird set up for a global variable.
GUIDE was a great idea and could have been made to work much better than how it turned out.
If I knew then what I know now I wouldn't have used GUIDE, but they aren't paying me to change it, so I'm stuck.
For my next venture into a full blown signal processing app, I will take a serious look at Python, I've been doing a lot in Python lately, though not signal processing, and really like the language.
There is a reason I created this thread. GUIDE has been outdated for the better part of a decade (perhaps more). There is a reason Mathworks is removing it except for extremely ardent fans.
Python is only popular in the machine learning community because MATLAB isn’t free. Back before 2013 or so MATLAB was the standard programming language for machine learning work.Python would probably still be a fairly popular general programming language though. It was well on its way there before the scientific Python stack came about (and that’s a large part of the reason why the scientific Python stack was developed).
@Zoltán Csáti Exactly ... !!! MATLAB needs wrappers for many high-performance libraries developed during last years. TensorFlow is one good example from many others.
@John "Python is only popular in the machine learning community because MATLAB isn’t free." And because MATLAB does not have wrappers around the high-performance libraries that are written in C++ (e.g. TensorFlow), hence it does not scale well.
@Walter Roberson Are you sure, that significant part of Scientific Python development is realized by people in developing countries? From my point of view is total majority of SciPy developers from US and Europe academic community. And yes, in the whole FOSS community, which is dominantly not oriented on scientific computating, the situation is different ...
A lot of the Free Open Source Software (FOSS) community thrives by exploiting the labour of people in developing countries to work for little to no pay, in hopes that their contributions will be recognized by someone willing to hire them.
It would be nice if I could open a file in the editor as read-only, click somewhere to set the editor for read only after a file is open, and have a file open automatically as read only if opened due to a break point in the debugger.
There are at least two reasons not to set the matlab installation to read-only:
- license files for Windows are stored inside the installation directory, and if it were read-only then it would not be possible to activate windows without root access
- saving the matlab path writes into the installation directory by default and most people don't know to take the step to save the path elsewhere
I did not say that these are great reasons, and they still allow for the possibility of making most files read-only.
Unfortunately it is not rare for windows users to routinely run as administrator because they installed as administrator or because they are trying to get around the activation problem.
MacOS, the default for the most common license type writes the license into a user directory instead of into the installation directory, and I would have thought something similar could have been done for Windows...
The dbstop into a Matlab toolbox file is a big issue for me. As discussed here, a related issue is that when the debugger opens a file the focus goes to the editor, not the command window. The focus should be in the command window. Forturnately, I've never saved after inadvertently modifying a toolbox file.
Annnnnd now I just jinxed myself ...
Is there any reason to not set the Matlab installation to read-only at the OS level?
Omg, +1 on this. As an interpreted language where you just deploy source code out to production for execution, this gets messy. Especially because Matlab's standard library itself is shipped as M-code, and depending on how you deploy it (and sometimes MathWorks Tech Support will actually tell you to do this!) you end up with a Matlab installation whose *.m files are writable by users who are running it. And sometimes you do a dbstop if error and it brings you up into a Matlab function, not some of your own code. I've encountered cases where I've had a user trying to debug some code on a shared server, dbstop'ed into a core Matlab file, fat-fingered something, and of course when the Matlab Editor showed them a scary prompt that said "Do you want to save your changes before closing?" of course they hit yes, and they fuxxored the core Matlab installation for the whole server.
I've hosed my own Matlab installation before by accidentally saving a change to mdbstatus.m because that damn thing pops up all the time when you have dbstop if all error set, and I have Admin rights on my VWSes.
This isn't helped by the fact that the Matlab Activation Tool on Windows requires users running it to have local Administrator rights, even though it doesn't actually need it for any of the file modifications it does (from what MW TS has told me).
You could always write a wrapper:
function readonly(fn)
Q = matlab.desktop.editor.openDocument(which(fn));
Q.Editable = false;
end
I couldn't find a doc page for view(). Well, at least not for view() as used in this context.
Thanks for that information. Once I typed
help matlab.desktop.editor
I was able to navigate around the help. But I couldn't find a doc page for the API. I might have to start using that API approach, particularly when I open Matlab toolbox files. Still, it would be easier to just have something like
edit why.m -readonly
The first and second are possible using the Editor API. I'm not sure if the third is possible.
Q = matlab.desktop.editor.openDocument(which('why.m'));
Q.Editable = false;
Now try editing why.m. You will not be able to; MATLAB should make the error sound if you try.
Q.Editable = true;
Now you can edit it. [If you do, I recommend undoing the changes immediately afterward.] For more information:
help matlab.desktop.editor
methods(Q) % See what you can do with the Document
You could make a shortcut that toggles the Editable property on the active document and a function that includes the first two lines above to use instead of edit.
You can use the command "view" to open read-only if I recall correctly. However I do not know any way to toggle over to read only once open.
I think the function call overhead may frustrate some. Until last week, I was never bothered by the infamous function call overhead. I saw people complain about it before, but their test were never realistic to me.
I spent the last week refactoring a class for an inspection program. My goal was to split the methods into smaller methods first and then split the class into multiple classes second. After splitting methods into smaller methods, I realized that my program was significantly slower. Real world test showed that my program took 9x longer to complete an inspection on average. I was able to knock it down to 2x by optimizing the code. Let’s just say, I am second guessing refactoring going forward.
I know maintainability should supersede speed in most cases. This program is already in use. Decreasing the throughput would decrease the cost savings.
Quick Update: 2021a is defnitely faster. Great Job TMW.
Nevermind. It seems like I need to try for 2021a
Last fustration:MATLAB COMPLIER SDK implies MATLAB Advertisment.
If [blank] spends XX,XXX dollars extra to purchase the MATLAB Compiler SDK+ Required ADD ONS to export thier appliaction,[blank] should be able to replace the MATLAB icon with an icon of their choosing. Icons referes to the icons for figures and dialog boxes.
[blank] did not pay XX,XXX to advertise MATHWORKS
MATLAB makes large projects difficult to manage.
Reasons
- OOP - No multi class files. This combined with the other issues prevent me from using ver the "Single-responsiblity Principle".
- Namespaces- No within file name spaces. No within file imports
- File Search: As of 2021a the in file search and cross file search is still a joke. there is no continuous search. This adds a lot of time to development. I basically use the current folder panel to jump between files. Also the search bar should be a panel like in most IDEs.
- Tab Completion: Originally the lack of good tab completion for python and MATLAB was attributed to the dynamic nature of the languages. Now python has Tabnine and Kite. Matlab has nothing.
- Function Input Hints: Spotty.
- Function Documentation when you hover with mouse over function:
- No Syntax code highlighting: At least highlight the MATLAB functions.
- No intellisence equivalent.
- I could go on.
2. namespace, matlab does not use import, namespace is a kind of elegance, which is different from other languages, because the pursuit of "concise", but does not affect the duplication of function names, can be automatically identified!
For number 7, I am talking about an editor feature. I could care less about what the variable actually is at runtime. I don’t care if editor accidently highlights a variable named bwdist, sum, or size because it thinks it is an official MATLAB function. It is a good idea to know that you are overshadowing an official MATLAB function in your code.
There is an unofficial VS Code plugin that highlights MATLAB functions. I don’t understand the reason TMW cannot add this feature. https://marketplace.visualstudio.com/items?itemName=Gimly81.matlab. .
The SRP is about 1 responsibility per class not per file. The SRP does not discourage programing languages form having muliple classes per namespace or multiple classes per file.
If you have multiple classes in the same file, then you have more than one reason to modify the file, which violates the Single Responsibility Principle anyhow.
8. It looks to me that several Intellisense equivalents are included in MATLAB; https://docs.microsoft.com/en-us/visualstudio/ide/using-intellisense?view=vs-2022
7. Syntax highlighting.
MATLAB highlights keywords in blue, comments in green, character vectors in magenta-ish, global variables and shared variables in cyan-ish .
7. You cannot know whether a function is a "MATLAB function" (in the sense of the implemention being by Mathworks rather than your own or third-party) -- not until run-time.
MATLAB does not know that a name is an object method until the class is at least loaded, and cannot generally know what class an object will be until run-time. It is not clear to me that MATLAB should run through all classes and build all of the interface hints when the classes might well not be used in any thousand given sessions.
[Repost Per Recomendation]
MATLAB keeps reinventing the wheel and sometimes does a bad job . The #1 rule of programing is never reinvent the wheel.
- OOP: MATLAB OOP implementation is worse than all other well-known modern languages. Why? LabVIEW is actually worse but that is not a text based language.
- Live Editor: Users wanted a Jupyter notebook like interface for MATLAB. Instead of creating an official stable feature rich plugin for Jupyter notebook, TMW released the live editor. I don't know how TMW could checked off/ released that product in 2016. (They probably ignored tester feedback) You did not need the end users’ feedback to know that it was too slow. The input lag made me feel like I was VNC’d into a computer oversees. Also, the execution time/ for loops….. I kept trying it from 2016- 2018. I have not touched it since 2018b.
- Editor: I see a lot of complaints on the lack of modem IDE features. I agree. Before you create a new editor, I would recommend using an existing Opensource editor. I think it would be a good Idea to create a feature rich plugin for VSCODE.
Engineers always want to reinvent the wheel because it is more interesting, but it is not always worth effort. When TMW reinvents the wheel, it sometimes misses a lot of the qualities the made the original wheel great. This will overshadow the interesting features of the reinvented wheel.
MathWorks have now released a MATLAB Kernel for Jupyter. Details at Official MathWorks MATLAB kernel for Jupyter released » The MATLAB Blog - MATLAB & Simulink
I love Matlab. I has great optimization capabilities, sliding window functions (e.g., movmad.m), and most density functions I always require (e.g., ncfpdf.m). However, professionals in my field have largely moved to Python for TWO reasons:
- Matlab is comparatively slow and less fluid at downloading time series over an internet connection. For example, I often need to acquire geophysical time series from a repository called IRIS (https://ds.iris.edu/ds/nodes/dmc/software/downloads/irisfetch.m/). One tool exists (irisFetch.m), but it acquires and writes out data files at a relatively glacial pace.
- Plotting anything on a global map, like geophysicla data, is slow and error prone (e.g., geoplot.m).
As a geophysicist that often needs to rapidly download, write out, and plot seismic data, these two drawbacks do slow my progress. I hope Mathworks addresses these two issues soon--otherwise, I'll need to make the switch just to communictae with my colleagues.
Size of the ecosystem is not a useful measure if there is a lot of crap in the ecosystem -- pieces badly conceived or badly implemented, or which adds little value to existing versions.
What can be important is that the ecosystem is well organized, well tested, that it is easy to find and download relevant work, and that the versioning system is easy to use. A library of (say) 100 well-chosen tools might be more valuable by far than (say) 20000 badly organized weak contributions.
A scientist actually should make sure to develop robust methods. The only saving grace of science is methodology. I say this as a scientist myself.
Now, you shouldn't be spending a month to implement a boxplot, but spending time on developing a robust method is not wasted time. You must have a robust method, how else can you trust what you intend to publish?
@Walter Roberson What I meant is that if in a given field, there isn't a well-established ecosystem, you have to program everything from scratch. For large projects, this is not possible. For instance, a scientist cannot afford to spend months of his time to develop a robust method: he needs to focus on communicating new research ideas.
This network performance issue is pretty surprising to me: My understanding (and experience; I do a fair amount of ETL in Matlab, including NetCDF stuff) is that Matlab's web and networking stuff sits on top of bog-standard lower-level (C/C++ or Java) networking libraries, so the networking and attendant file I/O behavior of Matlab is pretty much the same as any other mainstream language, and performance is a function of your internet connection and internal networking and file server performance.
The specific irisfetch.m library you linked to looks like it's built on top of a custom "IRIS-WS" Java library, which is running inside Matlab's embedded JVM, and Matlab is just doing "scripting" calls on top of that Java library (just look at all those "java.*" references in that irisFetch.m!), and some Matlab urlread calls. This sounds more like an application-specific issue than a Matlab thing to me?
This is actually the sort of thing I like working on in my hobby/open-source spare time! If you want to specifically discuss irisFetch performance, open an Issue on their GitHub and tag me (@apjanke on GitHub), or drop by the Matlab Discord ( https://discord.com/invite/bBMbNCT ) and start a conversation.
Joshua, I’m not at all familiar with this data repository, and not really an expert at geophysical data. The examples in the irisfetch manual seem pretty simple, and I think maybe downloads that you are doing are much more complex. Can you give me an example of a typical download that you would do using irisfetch, and how you would plot those data? That would help to see the problems you are running into.
I have to disagree that the size of the ecosystem is what is most important.
For example if you were to look around, you would find a lot more people created emacs modules than are creating vi or vim routines. emacs by that definition has the larger ecosystem. But that doesn't mean that emacs should be used: for most people, emacs is much slower and way more complex (not in a good way).
Another example: the most popular kind of vehicle sold in North America is the SUV. Suppose that you are an inner-city courier. SUV has the largest ecosystem, so according to your logic you should use that. But in practice, the fastest inner-city couriers for small items are bicycle couriers: bicycles can be faster than cars in the inner city. (There used to be an annual race in London, bicycle vs black cab vs private car vs public transport; bicycles won most of the time.)
Use the right tool for the job. What matters most in practice is not how "good" a language is, rather how large its ecosystem is. For most tasks, Python has a wider user-base, that is why I switched from MATLAB two years ago.
While I'm now long retired from active consulting where it was a key element, even then the "red-haired stepchild" status of Fortran with mex support lacking many new features being carried along with the C counterparts and the limited compiler support plus the archaic F77 interace and use of non-Fortran-Standard features, etc., etc., etc., ... were very frustrating.
During the exercise with the recent Q? regarding debugging a Fortran mex file I discovered that the TMW-supplied MinGW download link for gcc quietly installs gfortran at the same time. Discovering that piqued my interest again and I've spent a day or so now exploring the impasse in trying to use gfortran for Fortran-only mex files or the difficulties encountered even to used mixed language mex functions.
This preamble leads me to ask -- if TMW® isn't going to take any action regarding supporting Fortran any more diligently than at the present (seemingly reluctantly) "just barely" level, how about a GNU project to build the needed infrastructure to support gfortran specifically, and Fortran generically, to a higher level?
To complete this task obviously would take getting a nondisclosure agreement in place and having access to the necessary internals to be able to resolve the details of the innards of mex, but without somesuch mechanism it appears Fortran support will, at best, remain in the dark ages and, at worst, just disappear when TMW finally decides to quit supporting it at all.
An addendum to the above proposal -- I subsequently have been able to create an XML file that mex will recognize and subsequently use the gfortran compiler -- the linking issue with Fortran object files to the provided libraries still exists, but I have some hope that with C interop this may also be solvable.
At this time I'm not yet familiar-enough with either gfortran nor C interop features to know exactly how to implement, but I think there is hope...
It would be far better to build the Fortran-linkable library entry points directly, of course, instead; that would be the end result of the proposed volunteer project if TMW® won't pick up the ball.
I presume it is simply that Fortran slipped markedly in relative popularity with the rise of C/C++ owing to the nearly 20-something years time lag from the release of the F77 Standard to F90/F95 ca 1997 as lacking in some features that changing practices and that potential new users were accustomed to being available.
With that loss of relative importance it appears TMW made commercial decisions to simply not dedicate resources beyond the minimal required towards Fortran support and that pattern has continued, but seems to also continue to be on a slowly declining path as well.
However, once the vendor cliques of that time frame dissolved a more broad-based represntation of users has had a major role in the Standards committee after F95, progress has been (by formal language standards measures, anyway) quite rapid with F2003, F2008 and F2018 is in draft with compilers already implementing some of the new features. Since adoption of a formal ISO Standard is an international affair, nothing moves extremely rapidly in the arena unlike a commercial vendor such as TMW with total control over what they choose to implement and on what time frame or open source languages/projects that have far less bureacracy in controlling the adoption of new features.
gcc and gfortran, of course, follow the relative pace of their respective standards bodies, other languages that may not have formal ISO standards are much more free for experimentation and also to make drastic shifts in course if a particular path chosen turns out to be detrimental going forward...Fortran has been exceedingly careful to maintain compatibility in as many areas as possible with only a few features having been formally declared obsolete and no longer supported (and most compilers have opted to retain those but require special switches in order to compile legacy code although use of such may prohibit using other specific modern features).
"... given that Matlab is basically built on top of Fortran, ..."
I was going to address that some initially but skipped over it in the end. Only the very first implementation was written in Fortran, even the initial commercial application was written in C as outlined in the installment of the paper Cleve Moler wrote for the ACM SIGPLAN History of Programming Languages Conference (HOPL-IV).
Cleve posted his paper in sections on his blog beginning at the beginning of the numeric software project that led to EISPAICK and LINPACK. The first of those is <Cleve HOPL2020 Paper Part I>. The description of the first MATLAB Fortran environment and the initial development of a commercial product don't come until the later installments <Historic MATLAB> describes the initial calculator in FORTRAN and the next installment the beginnings of the actual MATLAB language <pc-matlab-version-1-0>. As outlined there, the base code was written in C. (At least at that time, one would presume the internals were still the original FORTRAN implementations called from C; I have no idea and didn't see it ever mentioned about whether the libraries have been ported by TMW or not).
One can see one unfortunate side effect to this day in the input/output routines built on C stdio.h with fopen, fclose, fprintf and friends and all the warts that followed with them, particularly with the issues of fixed-width field data files that weren't really solved until very recently with the introduction of the import objects.
About the only user-visible homage to the FORTRAN roots is having kept the column-major storage order and the 1-based array indexing. With the latter, however, the ability in FORTRAN for arbitrary index base was not carried over even in the beginning owing to the shift away from having to declare variables to everything be automatically allocated and "everything is a double unless it is specifically declared/created otherwise" and that the creation/declaration happened only by a function call, not as declaration statements. Only in later releases did the option equivalent to KIND appear in such places as zeros.
In recent years, the move has been to more and more higher-level data contstructions and the introduction of OOP paradigms on top of the initial procedural-only implementation. These all have pushed the direction to C++ and the introduction of data structures beyond the simple intrinsic data types.
Even in its current state, the C interop features of the Fortran Standard don't address interop with C++; hence the difficulty in providing complete interaction from Fortran with out an intermediary layer.
It doe still, however, seem as though there should be no reason to not support GFortran to the same level as Intel or other commercial compiler(s) they should at some point decide to include.
I want to bump up the need for better GNU compiler support for FORTRAN on all platforms.
The symbolic toolbox seems to have a lot of power, but it is not user friendly. I don't understand a lot of the error messages, and the defaults for simplification are not good. I could go on and on. For school problems, this should be everyones favorite but it is not. For work problmes the same. I work for a company who pays several million per year in maintenace fees, and find myself going to the Wolfram Alfa for symbolic solutions. Please put some investment into this toolbox.
I understand that being a Mac user I'm in a niche position. But I find Mathworks® "preference" for Windows quite ludicrous. At the very least one would expect the Linux version to be feature equivalente with Windows, but that's not the case. And it's so much worse for macOS!
A lot of high end productivity features and toolboxes are just missing, specially for HDL coding. GPGPU being CUDA only—for so many years now—is begining to chafe, specially when you look ate the latest Apple Silicon (M1 Pro and M1 Max, at the time of this writing).
If you work with machine learning at any level, that's a big bummer!
The problem with OpenCL, I am told, is that it has too many optional features that are in fact not implemented or are implemented in different ways on different vendors and models. Mathworks would not have been able to provide a single OpenCL suite: it would have had to been customized by class of hardware implementation.
The OpenCL market share is just too small. You and might want the world to be different, but deep learning is mostly synonymous with CUDA. Mathworks is not setting a trend here, they're following it.
Your second point is a completely different topic: 'native support' is talking about compiled binaries that don't rely on the Rosetta 2 translation layer. That is very likely under development. It will come when it comes. If you want a better estimate you should contact Mathworks. They might be willing to tell you more under some form of NDA.
And why do you care which type of binary is used? I personally only care about the actual performance. From what I understood Rosetta 2 is pretty efficient, so there doesn't seem to be a major hurry in making the switch. But I'm not a mac user, so I only have an outsider's perspective.
OpenCL would have been a much better choice than CUDA as it supports both Nvidia, AMD and others. This was definately a mistake on Mathworks part.
I don't buy the "there isn't any acutal support provided by Apple for M1 hardware". How is it that other companies are able to release native support for their products and not Mathworks?
MATLAB will fall out of favor with Mac users if native M1 support isn't implemented with next release, perhaps that is better from a financial point of view for Mathworks, but it is sad all around.
As outlined by Walter on this thread, the lackluster GPU support (and machine/deep learning support) are not really the fault of Mathworks. Nvidia is really pushing people to use CUDA and providing a lot of support. This is also a question of chicken and egg: everybody uses CUDA, so all popular tools assume you have it. And because all popular tools assume CUDA for anything non-trivial, people are pushed to buy Nvidia hardware.
Apple and Nvidia haven't been on speaking terms for years now, so I wouldn't expect any support for CUDA any time soon on Mac OS. The other way around is also unlikely: because there isn't any market share, the changes of Mathworks biting the bullet (gambling on future market share) are slim to none.
The ARM processors Apple has designed do have excellent performance for many tasks, but since there isn't any actual support provided to hook into the hardware (at least that I'm aware of), it is difficult to design tools to take advantage of this presumed advantage.
@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)"
This kinda just says it all
Matlab has been on the market for over 25 years and this is their customer-feedback process?
How about this. Give us the source-code and we'll fix it ourselves.
@Adam Danz @Paul Hi Adam and Paul! I just started as doc staff in January and while I can see the comments submitted for a given page, I can't see who submitted them. So there's isn't currently a way to close that loop with you. But I'll flag this convo for the folks who work on processes to improve the documentation and hopefully they'll come up with a way to update a commenter. I will tell you that I view page comments as gold and sadly, most of my pages don't have ratings or comments. So please keep commenting! They are definitely getting read. (Most unique comment so far: "I am such a child")
Thanks Steven. I don't expect replies to the many documentation-feedback-submissions I add. It would just be nice to somehow communicate what happens with them. Otherwise I feel like the guy in Lost who continually pushes that button without knowing what it does.
The star ratings and the free text feedback go to the documentation staff directly. I know they read the feedback and action some or all of the suggestions / bug reports but I'm not sure what kind of acknowledgement (if any) they send to the people who submit the feedback.
I'll forward this feedback about the feedback process to a couple of the documentation staff members with whom I work.
My experience is the same. Unlike bug reports that yield a response within a few days and opportunity to have a discussion, in my experience feedback on documentation results in effectively no response. I stopped sending feedback on documentation a long time ago.
@Steven Lord, what happens when we use the documentation feedback? I must have submitted recommendations and supportive messages 100+ times using the documentation feedback system and I've recommended the feedback system to others but I have no idea what happens after an idea is submitted.
I also never know how to use the star ratings. I usually just select a rating just to get to the comment field.
MATLAB Answers is one way that users can provide feedback to MathWorks.
If you have questions or feedback on the documentation you can also use the "How useful was this information?" rating system at the end of each documentation page (which allows you to provide free text questions or feedback after you give a star rating.)
I really dislike the aggressive scalar expansion. I realize it is convenient for column-wise manipulation of matrices that are really tables of data, but it isn't worth it.
If I accidentally add a row and column, I used to get an error, but now I get a matrix. I think that is unlikely to be what anyone wants:
R2015b:
>> [1 2 3 4]+[5;6;7;8]
Error using +
Matrix dimensions must agree.
Now:
[1 2 3 4]+[5;6;7;8]
ans =
6 7 8 9
7 8 9 10
8 9 10 11
9 10 11 12
"The first time i was burned by row-plus-column, I added more input checking to a lot of my utiltiy functions that take either rows or columns. I don't think that means they had "programming faults" before."
You previously relied upon the inputs being compatible, or that MATLAB would produce an error that you expected the user to be able to decode in context. If not a "fault" then at the very least it was a "weakness".
In my own code, it is not uncommon for me to fail to validate that an input is a vector. However, I try to avoid making assumptions about whether it is a row vector or a column vector, or about whether two inputs have the same orientation, so I routinely code (:) and (:).' or reshape() calls (which has the side effect of having the code do something useful if non-vector was input.)
I find automatic binary singleton expansion very useful - lots of my code now uses it, not just with arithmetic operators but also with logical ones. (Such a pleasure to strip out the clumsy bsxfun calls!) For example, whenever you have a situation where you want to look at all the combinations of elements from two sets, it can reduce complexity in the code and provide an efficient solution. As it happens, I find it very natural and elegant, though I understand that others might not.
I agree with @Walter Roberson that a construction that was previously illegal (in that it would throw an error) is available to be put to work and made useful.
As someone coming from a MATLAB background, I was pleased to discover that Python also does the sensible thing.
Adding my two cents:
I am a firm believer that your functions should throw errors during input validation, not during runtime. There is of course the exception where a function is expected to run often and/or in a time-sensitive situation, in which case you could have the final function without input validation. (although I would still like a test suite to pick up these errors)
If your functions were throwing errors during @times instead of during input validation I would actually consider that risky programming. If row/col orientation checking is not part of your validation, then I would not consider it complete.
And as far as the clutter goes: I only see the call to my function, not the checking its doing. In general I don't even see that checking when editing my function, as I often factor it out to a separate input parsing function.
Well, I did qualify my statement accordingly.
FWIW, I still have to use bsxfun() for everything other than on the forum, since I still want stuff to work in older versions.
I think this is the crux of the matter:
" I'm not going to complain much that I get to use something that's more readable than a bunch of bsxfun() clutter. "
I agree the clutter of "bsxfun" or "repmat" makes it harder to see what is going on.
So, if you do a lot of row-to-column expansion in your typical work, you will like auto-expansion for the clutter reduction.
I don't do a lot of row-to-column expansion, so I dislike it for the added clutter of variable shape checking.
Those of you trying to suggest that those with other preferences are objectively wrong ought to check your assumptions.
I know a lot of people don't like it for their own reasons, but if opinions matter (and I'm sure mine isn't on the list of those that might), generalized array expansion is the single feature for which I tolerate how much worse R2019b is versus R2015b (in terms of memory usage & stability). I'm not going to complain much that I get to use something that's more readable than a bunch of bsxfun() clutter.
I bet it might confuse new students who never were forced to learn what operations require expansion to work, but I'm pretty sure there's no bump small enough that someone won't trip over it.
"You wrote, "I think that is unlikely to be what anyone wants:" and I showed you that it may well be exactly what is wanted."
Sure. That was an unfortunate bit of hyperbole on my part. I know perfectly well that the peope who added it wanted it.
You said "Relying on a programming language declaring an error to catch a programming fault has always been risky." Well, no. I do not find your single vs double quote example convincing.
The first time i was burned by row-plus-column, I added more input checking to a lot of my utiltiy functions that take either rows or columns. I don't think that means they had "programming faults" before.
That it is "fair game" is, of course true, particularly for a proprietary language like MATLAB that has no external Standards body such as Fortran, C, etc., that control their evolution.
That needing to approve Standards is a very time-consuming process and only allows for globally-accepted changes to be incorporated and so certainly constrains the speed and direction in which those languages evolve.
But, just because TMW can do something doesn't mean it is necessarily a good idea -- we have a multitude of examples of relatively recent introductions that have cluttered up the workspace and that are now deprecated but can't go away because once released in the wild, backwards compatibility requires they be retained.
It is, granted, a hard line to toe in a competitive environment.
You wrote, "I think that is unlikely to be what anyone wants:" and I showed you that it may well be exactly what is wanted.
MATLAB took something that was an error in all previous versions, and defined a meaning for it. That kind of expansion of a programming language is "fair game" in all programming language. Relying on a programming language declaring an error to catch a programming fault has always been risky.
The situation is no different than if you had previously relied upon MATLAB creating an error message if you used double quotes around a character vector, and were to be complaining now that double quotes have been given a defined behaviour that is different than single quotes, and wanted MATLAB to switch from using double quotes for strings into using (for example) #' and '#
#'abc'# + 1
on the grounds that you never used to accidentally use #' and '# instead of ' ' so you can live with #' '# but cannot live with
"abc" + 1
not either generating an error or else acting the same as
'abc' + 1
IMO, if TMW wanted to do this, they should have invented some other syntax similar to the "dot" notation so that the user retains control and gets expansion only where it is, as @Walter Roberson illustrates actually the wanted behavior and not otherwise.
I, too, have been burned a few times unexpectedly.
I understand the motivation. Whether you think it is worth the pain it causes elsewhere depends on what your typical work flow looks like (and maybe whether you came from a Python background).
I still forcefully object to it.
(1:4)+(10:10:40).'
That may well be exactly what the user wants.
I have deliberately used row + column for various computations. In particular, instead of doing
[X, Y] = ndgrid(1:4, 10:10:40)
Z = X.^2 + Y
I might well do
X = 1:4;
Y = (10:10:40).';
Z = X.^2 + Y
Calculating a function over a range of multiple variables is common, and happens easily if you are permitted to mix row and column vectors using implicit expansion: it saves building the coordinate matrices using meshgrid() or ndgrid()
I keep being reminded of this every time I accidentally destroy my own work or have to process someone else's crusty images.
If the settings used for imwrite() and print() for JPG output are 75% quality and 4:2:0 chroma subsampling, then JPG output is essentially useless for the typical image content they're asked to handle. Users who think they have an understanding of the compromises they're accepting when using JPG likely have no idea that they're opting for such abnormally low-fidelity settings.
Despite the integrity of my WAN connection, this isn't 1998 anymore. If options to use a reasonable set of output parameters can't be added, then the perilous nature of the defaults should be clearly documented. While Q can be set for imwrite() output, I don't know that it can when saving figures/axes. As far as I know, the chroma subsampling isn't documented anywhere and is not user-selectable.
Everyone expects JPG to be lossy,
In my experience, it is common for students to expect that JPEG is lossless.
On occasion, a student pretty much demands that mathematics work differently so that JPEG becomes a lossless compression...
Submit enhancement request(s)...
@Walter Roberson I tend to avoid it like a disease, just to avoid the chance that I absentmindedly save something important with the file extension selected from the prior figure. This is in direct conflict with my desire to make images for the forum small -- which is of extreme importance due to my bad connection. Even at that, I routinely run across plots (e.g. surf() or mesh()) which turn into garish pixel salads when saved as JPG. In other words, this stopped being appropriate even for web-only output sometime about 20 years ago.
In other contexts, I have little problem saving non-archival images in JPG format. GIMP's default is 90% quality with no subsampling (4:4:4). The difference in apparent quality is dramatic, and the chroma subsampling is an extreme contributor to that. Much of my casual time with MATLAB in the last decade has been using it as an augmentation of GIMP for post-processing digital paintings. How was I to know that using imwrite() with Q=90 wouldn't produce the level of quality I was used to accepting? MATLAB is a technical environment; quality probably matters more here than in GIMP, right?
By no means am I suggesting anyone desire to do technical image processing using lossy JPG, no matter the settings. That said, we both know the utter regularity with which users request help with trying to process images saved as JPG -- often unnecessary images of figures of images. People are going to use it anyway, whether it's simply because they don't care, or because they're conscientiously accepting an expectation that's not met by reality -- because they have no way to know otherwise.
TL;DR: Everyone expects JPG to be lossy, but undocumented defaults like this are sufficiently abnormal that I regard it as a noteworthy betrayal of those common expectations.
The baffling point to me is that jpg is the de facto standard in deep learning, even when actual values matter (e.g. in medical imaging). I don't think it should be, but this doesn't help.
I deal with this by only ever using .jpg if I am generating images to post here -- images intended to give the impression of what is going on without having to be the bit-for-bit final array.
... to clarify, I wouldn't use JPG for research work even if it had a different default subsampling or different default Quality. If I want lossless then I use a lossless image format, not jpg.
@Wolfgang Do you have a specific issue in mind? You can use the txt input if you want to standardize things. I have only had issues with unsupported installations (e.g. installing 6.5 (R13) on Windows 10).
I would agree with you when talking about portability (having the option of a completely USB-portable solution has advantages).
The worst of Matlab is the installation. For me it is each time a nightmare. I loose hours for nothing. It is the reason why I use Octave on simple tasks even so I own three licences for my reserach team. - And I am the lucky owner of a old licence of MathCAD. Always works and less troublesome.
Currently matlab lacks Tsai's calibration method integration, only Zhang Zhengyou's calibration method.
Currently I can only calibrate one image and cannot take multiple viewpoint images with a checkerboard grid, with non-co-planar 3D points and two-dim image points corresponding，What can I do?
- http://people.csail.mit.edu/bkph/articles/Tsai_Revisited.pdf Tsai, Roger Y. (1986)
See Also
Tags
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)