MATLAB Central Newsreader - Matlab bugs

Matlab bugs

Jeremy Smith — Tue, 02 Sep 2008 19:35:04 -0400

One of my coworkers recently found a problem/bug in the
datenum function of Matlab 7.0. It was adjusting for
daylight savings time so we were left with some of our data
being shifted by an hour. This was fixed in later versions
of Matlab however nothing about daylight savings time was
mentioned in the documentation or revisions list of Matlab.

Where does a researcher have to go to find information about
changes that significantly impact code execution? How is
anyone suppose to check all of their old code to make sure
it works with newer versions of Matlab?

We need a tool which can check an m-file and give a nicely
formatted report on what changes have been made to all its
functions between two versions of Matlab. This includes
argument syntax, default settings, phasing out functions,
and speed. The researchers I work with are very close to
getting everyone on the same version of Matlab and declaring
an indefinite embargo on updating Matlab. We already work
with a researcher who has done this.

Re: Matlab bugs

Rodney Thomson — Wed, 03 Sep 2008 01:24:02 -0400

"Jeremy Smith" <smit1729@umn.NOSPAM.edu> wrote in message
<g9k4h7$957$1@fred.mathworks.com>...
> One of my coworkers recently found a problem/bug in the
> datenum function of Matlab 7.0. It was adjusting for
> daylight savings time so we were left with some of our data
> being shifted by an hour. This was fixed in later versions
> of Matlab however nothing about daylight savings time was
> mentioned in the documentation or revisions list of Matlab.
>
> Where does a researcher have to go to find information about
> changes that significantly impact code execution? How is
> anyone suppose to check all of their old code to make sure
> it works with newer versions of Matlab?
>
> We need a tool which can check an m-file and give a nicely
> formatted report on what changes have been made to all its
> functions between two versions of Matlab. This includes
> argument syntax, default settings, phasing out functions,
> and speed. The researchers I work with are very close to
> getting everyone on the same version of Matlab and declaring
> an indefinite embargo on updating Matlab. We already work
> with a researcher who has done this.

Updating a revision of MATLAB, GCC, Visual Studio, Perl,
Ruby, etc etc etc always comes at the risk of changing
behavior. You should never assume consistent behaviour,
particularly across major revision changes.

Before upgrading, check the release notes of all newer
releases (ie 2008 is here -
http://www.mathworks.com/access/helpdesk/help/base/relnotes/)

Rod

--
http://iheartmatlab.blogspot.com

Re: Matlab bugs

Jeremy Smith — Thu, 04 Sep 2008 13:41:03 -0400

The problem is that this change was not noted anywhere. I
searched the bug fixes and change notes before posting here.
I would be happy being able to search revisions. Where do I
find the revision history of a specific function or do I
have to hope the changes are obvious?

Re: Matlab bugs

dpb — Thu, 04 Sep 2008 14:09:32 -0400

Jeremy Smith wrote:
> The problem is that this change was not noted anywhere. I
> searched the bug fixes and change notes before posting here.
> I would be happy being able to search revisions. Where do I
> find the revision history of a specific function or do I
> have to hope the changes are obvious?

I'm sure it was noted somewhere internal to TMW, but that doesn't help
the end user unless those changes get reflected to some public document,
granted.

I've commented before that one weakness I see in Matlab documentation in
particular is that it doesn't have a formal syntax document or published
"internal Standard" for lack of better term. Hence the only
documentation on behavior is the help files which are written in large
part as informal "how to" and the actual code for accessible m-files.

This is a problem w/ any language/software product to more or less a
degree of course; it just seems more of an issue in my mind w/ the
proprietary products as opposed to languages that do have formal
Standards--there, at least, there is a public declaration of how the
specified portions of the language are to behave and it becomes one of
conformance to the Standard (or occasionally interpretation).

There is still a significant difference in a RAD-type tool such as
Matlab vis a vis a more general language in that despite the size of,
say, C and the rtl, there are so many more functions in a tool such as
Matlab the requirement for even more complete documentation would make a
significant additional burden on TMW. 'Tis not an enviable task ...

--

Re: Matlab bugs

someone — Thu, 04 Sep 2008 14:53:02 -0400

"Jeremy Smith" <smit1729@umn.NOSPAM.edu> wrote in message
<g9oohe$svo$1@fred.mathworks.com>...
> The problem is that this change was not noted anywhere.
I
> searched the bug fixes and change notes before posting
here.
> I would be happy being able to search revisions. Where
do I
> find the revision history of a specific function or do I
> have to hope the changes are obvious?

I know this general subject has come up before. As one who
sometimes has to write MATLAB code that is backward
compatible, it would be very helpful if the help or doc
file for a (new) function stated which version of MATLAB
introduced it. For example, (I already know the answer
through trial & error) what is the first relaese of MATLAB
that supports the sind function?

Re: Matlab bugs

Walter Roberson — Thu, 04 Sep 2008 18:54:21 -0400

Jeremy Smith wrote:

> Where does a researcher have to go to find information about
> changes that significantly impact code execution?

Mathworks keeps a user-accessible bug database since R14.
However, not all information is in the database, only those
bugs that someone feels "might" be of interest to users.
This differs noticeably from Cisco's approach (for one),
in which all bugs go in the same database but some bugs
might be -specifically- marked as being non-public
for security reasons (or if the bug report details would
expose customer information that the customer would prefer stayed
private.) Default private (Matlab) vs default public (Cisco).

The Mathworks bug database does not appear to offer a way
to generate reports about bugs present in particular releases
or about bugs solved in particular releases.

The Mathworks release notes for any Matlab release are
brief summarizes of new features, and often do not mention
much (if anything) about bug fixes. Cisco release notes,
on the other hand, include two sections: one of bugs
that were fixed in the release (with bug id and short short
summary in the release notes); the other of bugs that are known
to still be open in the release (again with information that
can be used to look up the bug details.)

> How is
> anyone suppose to check all of their old code to make sure
> it works with newer versions of Matlab?

How do you know that your code works with your existing
Matlab release? If you don't have unit tests or other similar
test methodologies, then you don't know whether a change
is a problem or an improvement.

I work in a research environment myself, so I know that
it is -very- common in research environments that one does
not actually -know- what the "right" answer for a program is.
You might feed in artificial or well-understood datasets and
observe the program behaviour, but unless you go through some
hefty mathematical proofs (which are often not possible) and
then do some hoary numerical analysis, you probably don't -know-
that your code is producing the "right" answer even for the
artificial datasets -- you might know that the answers are
"about right" for your test data, but once you introduce
real datasets that might have kinds of noise and data errors
you never thought of (or which you thought too improbable
to be worth spending time on) then chances are high that you
don't know whether a change is bad or not.

For example, suppose that Mathworks started deciding that
numerical accuracy was more important than speed for basic
operations such as summations, and so replaced the sum()
operator with one that took care to sort the values by
numeric significance in order to produce the highest resolution
answer it could. Unless you've already put that layer of
accuracy in your code, the answers produced by your code
would very likely be different in the new version. Which answers
would be the "right" ones? If Matlab pushed automatic parallization
into multiple cores and so ended up producing different answers for
sum() because the numeric cancellations fell at different points (on different
machines!) then which is the "right" answer?

If you already have complete test frameworks that somehow
have the "highest possible accuracy" answer coded in to them,
then when you get a new Matlab version, you just re-run your tests.
If you don't already have those rigid test frameworks, then you
probably don't really know that your existing code is "right" anyhow,
so knowing the details of every change wouldn't help you much.

> We need a tool which can check an m-file and give a nicely
> formatted report on what changes have been made to all its
> functions between two versions of Matlab. This includes
> argument syntax, default settings, phasing out functions,
> and speed.

So you want what in the software business would probably be
called a "conformance suite report". Conformance suite software
commonly runs from $100,000 to $250,000, per license (because
it is a lot of work to build the software and there are
surprisingly few people who care about complete accuracy
of their programs, so the work put in to the suite
has to be recaptured over a small market.) Even the C programming
language has only 2 or 3 vendors of compiler conformance suites
and one would -think- that with the wide variety of C compilers around
that a lot of people would be very concerned about accuracy of their
C compilers.

My personal opinion is that if you need the kind of tool that
you mention above, then you are writing in completely the
wrong programming language. Look at the documentation of most any
of the Matlab major routines, and notice that they contain *no*
discussion of numeric analysis, and notice that they contain
no options to allow you to increase accuracy at the expense of
program speed. The people who write the routines might -individually-
be quite concerned with accuracy, but Matlab as a product is not
and never has been. And just like you can never graft in security
into a program as an after-thought, you can never graft in accuracy
to a large tool such as Matlab as an after-thought.

A question: have you tested your routines with JIT (Just In Time)
optimization 'on' versus 'off'? Mathworks changes the details of JIT
in every release, so complex programs are likely to get different
answers in every release. If that is not a factor you have already
specifically taken into account, then the tool you describe would
be gilding the rotten lily.

I am not without sympathy for the difficulties of moving between
different releases, but I think that it would be sufficient for
Mathworks to beef up their release notes. On the other hand, presenting
the change notes for every routine would probably be far far too much
information to be readily absorbed.

--
Q = quotation(rand);
if isempty(Q); error('Quotation server filesystem problems')
else sprintf('%s',Q), end