Thread Subject: New behaviour with nested functions hiding builtins

Dear all,

I installed today R2010a and to my great surprise one script I routinely used started giving incorrect results.
Long story short, I traced back the cause to be the presence of a nested function called "times" inside the script.

This breaks functionality of the "*" operator without any kind of error or warning, so all multiplications are wrong.
In previous Matlab versions this did not happen, meaning that the "times" subroutine was not hiding the operator "*".

I wanted to hear your opinion before submitting a bug report, because I believe this is an *extremely* dangerous change of behaviour that completely goes unnoticed!
Probably m-lint should give a big warning if a subroutine exists that hides a built-in?

Best regards,
Federico

"Federico Miorelli" <fmiorelli@REMOVEslbTHIS.com> wrote in message <hn5s85$hr4$1@fred.mathworks.com>...
> Dear all,
>
> I installed today R2010a and to my great surprise one script I routinely used started giving incorrect results.
> Long story short, I traced back the cause to be the presence of a nested function called "times" inside the script.
>
> This breaks functionality of the "*" operator without any kind of error or warning, so all multiplications are wrong.
> In previous Matlab versions this did not happen, meaning that the "times" subroutine was not hiding the operator "*".
>
> I wanted to hear your opinion before submitting a bug report, because I believe this is an *extremely* dangerous change of behaviour that completely goes unnoticed!
> Probably m-lint should give a big warning if a subroutine exists that hides a built-in?
=================

I'll reserve judgement until I see more of your code, but it sounds like a pretty serious bug to me.

I'd be curious to know what happens if you do a multiplication A*B where A and B are some user-defined class instead of doubles. There's no reason why a nested function, or even a non-nested function, would shadow a times() method at minimum unless that function has been placed in a directory called @double or @myClass or whatever.

When MATLAB sees an operation A*B, it's supposed to decides which times() method to apply based on class(A), class(B), and certain dispatching rules, and not simply to take the first times() method it finds in the search path. It looks like, here, MATLAB is skipping those dispatching rules altogether.

This appears to happen only in debug mode.
To reproduce, set a breakpoint at the beginning of the function below and continue when hit.
With the breakpoint it will always give 1234 as result, without it will give the correct answer.
This does not happen in Matlab 2009b and older.

----------------------------
%%%%%%%%%%%%%
function y = multiplyByTen(x)

myStruct.value = x; % put value inside a struct (see below)
y = 10 * myStruct.value; % use the built-in "*"

%%%%%%%%%%%%%
% Define a sub-function called like a built-in
function [out1,out2] = times(File,varargin)
% Make so that function will always return 1234 if called
if ~strcmp(date, 'FAIL')
    out1 = 1234;
    return;
end

% Let the parser think that this function works with a struct
out1 = File.x;
out2 = [];

"Federico Miorelli" <fmiorelli@REMOVEslbTHIS.com> wrote in message <hn5udg$ce5$1@fred.mathworks.com>...
> This appears to happen only in debug mode.
> To reproduce, set a breakpoint at the beginning of the function below and continue when hit.
> With the breakpoint it will always give 1234 as result, without it will give the correct answer.
> This does not happen in Matlab 2009b and older.
>
==================

I don't have R2010, so I can't reproduce it, but based on the example, it seems abundantly reportable as a bug.

Can I ask for more info here? Comment/question below.

(1) You mention that the script does not work when you upgrade to R2010a. I
don't know if you are runnning debug mode then. If not, I wonder what else
is broken. Does you code work correctly in normal mode?

(2) The discrepancy in and out of debug mode should be a bug. Here is are
more detail.

First, there are two functions, "mtimes" and "times". which can be written
as "*" and ".*" respectively. "times" is elementwise multiplication, while
"mtimes" is matrix multiply but if one of the input is a scalar, it is
essentially doing "times". For example,

10 .* [1 2 3]

is the same as

10*[1 2 3]

The question here is that whether "mtimes" should react to overloaded
"times" in this case. I can see argument from both sides. But for backward
compatibility, it is easy to argue that ignoring overloaded "times" is the
right thing to do.

Regards,
---Bob.

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message
news:hn5vj0$qm$1@fred.mathworks.com...
> "Federico Miorelli" <fmiorelli@REMOVEslbTHIS.com> wrote in message
> <hn5udg$ce5$1@fred.mathworks.com>...
>> This appears to happen only in debug mode.
>> To reproduce, set a breakpoint at the beginning of the function below and
>> continue when hit.
>> With the breakpoint it will always give 1234 as result, without it will
>> give the correct answer.
>> This does not happen in Matlab 2009b and older.
>>
> ==================
>
> I don't have R2010, so I can't reproduce it, but based on the example, it
> seems abundantly reportable as a bug.
>

I guess it's not a bug. Here's an excerpt from the MATLAB documentation on function precedence (R2009b) which describes this as intended behavior. If previous behavior wasn't consistent with this, I guess it was a bug....


Function Precedence Order

The function precedence order determines the precedence of one function over another based on the type of function and its location on the MATLAB path. MATLAB selects the correct function for a given context by applying the following function precedence rules in the order given here.

For items 3 through 7 in this list, the file MATLAB searches for can be any of four types: an M- or built-in file, preparsed M-file (P-Code), compiled C or Fortran file (MEX-file), or Simulink® model (MDL-file). See Multiple Implementation Types for more on this.

   1.

      Variable

      Before assuming that a name should match a function, MATLAB checks the current workspace to see if it matches a variable name. If MATLAB finds a match, it stops the search.
   2.

      Subfunction

      Subfunctions take precedence over all other M-file functions and overloaded methods that are on the path and have the same name. Even if the function is called with an argument of type matching that of an overloaded method, MATLAB uses the subfunction and ignores the overloaded method.

Matt,

While this is all true, consider this example, I use the function name
instead of .* to remove any confusion.

function y = test(x)
y = mtimes(10,x);
  %subfunction
  function z = times(a,b)
  z = a+b;

While times is overloaded in the local scope of test.m, the function mtimes
does not react to the overloaded times in R2009b or before. It make sense
because the example does not overloaded mtimes. In R2010a in debug mode, the
overloaded times is called and this is where the behaviour has changed. For
backward compatibility, it should consider a bug.

More importantly, I think the best way going forward is not to use a
function name that is the same as a built-in function, particularly one that
with an operator. Why? Consider the following, say this does what you want

function y = test(x)
y = 10*x;
  %subfunction
  function z = times(a,b)
  z = a+b;

Let generalize it in the following three ways and you would have broken your
code

function y = test(x)
y = 10.*x;
  %subfunction
  function z = times(a,b)
  z = a+b;

or

function y = test(x,w)
y = w.*x;
  %subfunction
  function z = times(a,b)
  z = a+b;

or

function y = test(x,w)
y = bsxfun(@times, w, x);
  %subfunction
  function z = times(a,b)
  z = a+b;

You can read the doc all day long. But it is so easy to make a mistake. It
is a genuine trap. So be careful.

---Bob.

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message
news:hn6dua$5hl$1@fred.mathworks.com...
>
> I guess it's not a bug. Here's an excerpt from the MATLAB documentation on
> function precedence (R2009b) which describes this as intended behavior. If
> previous behavior wasn't consistent with this, I guess it was a bug....
>
>
> Function Precedence Order
>
> The function precedence order determines the precedence of one function
> over another based on the type of function and its location on the MATLAB
> path. MATLAB selects the correct function for a given context by applying
> the following function precedence rules in the order given here.
>
> For items 3 through 7 in this list, the file MATLAB searches for can be
> any of four types: an M- or built-in file, preparsed M-file (P-Code),
> compiled C or Fortran file (MEX-file), or Simulink model (MDL-file). See
> Multiple Implementation Types for more on this.
>
> 1.
>
> Variable
>
> Before assuming that a name should match a function, MATLAB checks
> the current workspace to see if it matches a variable name. If MATLAB
> finds a match, it stops the search.
> 2.
>
> Subfunction
>
> Subfunctions take precedence over all other M-file functions and
> overloaded methods that are on the path and have the same name. Even if
> the function is called with an argument of type matching that of an
> overloaded method, MATLAB uses the subfunction and ignores the overloaded
> method.
>

"Bobby Cheng" <bcheng@mathworks.com> wrote:
> (1) You mention that the script does not work when you upgrade to R2010a. I
> don't know if you are runnning debug mode then. If not, I wonder what else
> is broken. Does you code work correctly in normal mode?

My code works OK in normal mode, the error happens only in debug mode.
This is where I would say it is a bug, or at least a VERY dangerous behaviour.

> In R2010a in debug mode, the overloaded times is called and this is where
> the behaviour has changed. For backward compatibility, it should consider a bug.
> More importantly, I think the best way going forward is not to use a
> function name that is the same as a built-in function, particularly one that
> with an operator.
> ...
> You can read the doc all day long. But it is so easy to make a mistake. It
> is a genuine trap. So be careful.

Couldn't agree more with everything you wrote.

Best regards,
Federico

Dear Federico!

> I installed today R2010a and to my great surprise one script I routinely used started giving incorrect results.

Just for clarification: Did the problem appear at first because you've started your script in debug mode?

Kind regards, Jan

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message
> Just for clarification: Did the problem appear at first because you've started your script in debug mode?

Hi Jan,
yes it did! It's just when I tried to write a little script to reproduce the bug that I noticed that it didn't happen in normal mode.
Understanding this drove me almost crazy by the way :)

Dear Federico!

> > Just for clarification: Did the problem appear at first because you've started your script in debug mode?

> yes it did!

Ok, thanks. So you've enabled the debugging mode for any other reason after updating to 2010A. Be happy that you've done this now and not in some month, because then it would not be you first idea, that this is related with the update.

> Understanding this drove me almost crazy by the way :)

I can imagine this. Drink a cup of coffee. It seems, that te one or other programmer of TMW did this also during testing the new version ;-)

Kind regards and thanks for locating this bug!!! Jan