Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Very slow performance for NaNs

Subject: Very slow performance for NaNs

From: Matthew Brett

Date: 17 Feb, 2003 11:31:31

Message: 1 of 11

Hi,


I have found a rather annoying system / matlab problem with NaNs and
was hoping that others had come across this and thought of a nice
solution.


The following little test program illustrates the problem. It shows
that multiplication by NaNs was hundreds of times slower than for
real numbers on a 2.5GHz pentium:


sz = 500;
A = ones(sz);
B = ones(sz);
tic
C = A*B;
t1 = toc;
A = A * NaN;
tic
C = A*B;
t2 = toc;
[tmp msg] = unix('uname -a');
fprintf('System: %s\nMatlab version: %s\nOnes: %8.4f; NaNs: %8.4f;
Propn: %8.4f\n',...
msg, version, t1, t2, t2/t1);


System: Linux ... 2.4.21pre4-5mdk #1 SMP
Sat Feb 8 22:27:52 CET 2003 i686 unknown unknown GNU/Linux
Matlab version: 6.5.0.180913a (R13)
Ones: 0.1447; NaNs: 108.7452; Propn: 751.7246


and the same was true of one Sun system (but not others):


System: SunOS navi 5.9 Generic_112233-03 sun4u sparc
SUNW,Sun-Fire-480R
Matlab version: 5.3.0.10183 (R11)
Ones: 0.3524; NaNs: 214.0200; Propn: 607.3470


The same range of figures seem to apply for different matlab versions
on the same platform.


Has anyone else noticed the same phenomenon? Is there any global
workaround


Best,


Matthew

Subject: Very slow performance for NaNs

From: Paul Fackler

Date: 17 Feb, 2003 15:38:06

Message: 2 of 11

This happens on my Intel Windows machine as well.
Paul

Matthew Brett wrote:

> Hi,
>
> I have found a rather annoying system / matlab problem with NaNs and
> was hoping that others had come across this and thought of a nice
> solution.
>
> The following little test program illustrates the problem. It shows
> that multiplication by NaNs was hundreds of times slower than for
> real numbers on a 2.5GHz pentium:
>
> sz = 500;
> A = ones(sz);
> B = ones(sz);
> tic
> C = A*B;
> t1 = toc;
> A = A * NaN;
> tic
> C = A*B;
> t2 = toc;
> [tmp msg] = unix('uname -a');
> fprintf('System: %s\nMatlab version: %s\nOnes: %8.4f; NaNs: %8.4f;
> Propn: %8.4f\n',...
> msg, version, t1, t2, t2/t1);
>
> System: Linux ... 2.4.21pre4-5mdk #1 SMP
> Sat Feb 8 22:27:52 CET 2003 i686 unknown unknown GNU/Linux
> Matlab version: 6.5.0.180913a (R13)
> Ones: 0.1447; NaNs: 108.7452; Propn: 751.7246
>
> and the same was true of one Sun system (but not others):
>
> System: SunOS navi 5.9 Generic_112233-03 sun4u sparc
> SUNW,Sun-Fire-480R
> Matlab version: 5.3.0.10183 (R11)
> Ones: 0.3524; NaNs: 214.0200; Propn: 607.3470
>
> The same range of figures seem to apply for different matlab versions
> on the same platform.
>
> Has anyone else noticed the same phenomenon? Is there any global
> workaround
>
> Best,
>
> Matthew

--
Paul L. Fackler
Department of Agricultural and Applied Economics
North Carolina State University
Raleigh NC 27695-8109
phone: 919-515-4535
fax: 919-515-1824
email: paul_fackler@ncsu.edu
web site: www4.ncsu.edu/~pfackler

Subject: Very slow performance for NaNs

From: Jim Maneval

Date: 17 Feb, 2003 18:51:23

Message: 3 of 11

Results for an old version


System: MacOS 9.1, 450 MHz
Matlab version 5.2.1.1421
Ones: 2.8687; NaNs: 2.7994;Propn: 0.9759


Maybe something has changed?


Matthew Brett wrote:
>
>
> Hi,
>
> I have found a rather annoying system / matlab problem with NaNs and
> was hoping that others had come across this and thought of a nice
> solution.

Subject: Very slow performance for NaNs

From: Penny Anderson

Date: 18 Feb, 2003 13:23:54

Message: 4 of 11

"Jim Maneval" <maneval@bucknell.edu> wrote in message
news:eeb967e.1@WebX.raydaftYaTP...
> Results for an old version
>
>
> System: MacOS 9.1, 450 MHz
> Matlab version 5.2.1.1421
> Ones: 2.8687; NaNs: 2.7994;Propn: 0.9759
>
>
> Maybe something has changed?
>
>
> Matthew Brett wrote:
> >
> >
> > Hi,
> >
> > I have found a rather annoying system / matlab problem with NaNs
and
> > was hoping that others had come across this and thought of a nice
> > solution.

We don't do anything more special to multiply NaNs than to multiply
finite values. We just use "*" like you would if you wrote a C, C++ or
FORTRAN program to do it.

This particular benchmark uses matrix multiply, which we turn around
and farm off to the BLAS routine DGEMM, but at the end of the day when
you get rid of the cache effects of this large (500-by-500) matrix,
it's just "*" and "+" on NaNs that's slow (slower on some machines
than others).

Penny.
--
Penny Anderson, The MathWorks, Inc.
penny@mathworks.com

Subject: Very slow performance for NaNs

From: Satrajit Ghosh

Date: 23 Feb, 2003 13:13:27

Message: 5 of 11

I just posted this as a question for TMW tech support. But there
seems to be a clear difference between Matlab running on Intel
processors vs AMD processors, independent of the operating system.
The following table shows what I mean.


The following piece of code executes differently on whether the
machine has an Intel or AMD processor (independent of processor
speed/operating system) and I have replicated it on almost every
machine I could get my hands on. For some reasons, matlab or the
underlying libraries [atlas/lapack]/processors deal with NaNs
differently on each type of CPU. As can be seen, if one deals with
large arrays containing NaNs, an Intel based Matlab process can
almost be ruled out. Any solution to the issue will be much
appreciated.


Thanks,


Satra


for i = 1:5,
a=rand(25*i);b=NaN*a;
tic;a*a;y(i,1)=toc;
tic;b*b;y(i,2)=toc;
end;


The following outputs are from Windows XP machines:
[Dual PIII 866 MHz, 1G of RAM]
y =
    0.0150 0
         0 0.0470
         0 0.1250
         0 0.2960
    0.0320 0.5460


[AMD 1GHz Athlon, 512MB of RAM]
y =
         0 0
         0 0
         0 0
         0 0.0100
    0.0100 0.0100


The following outputs are from Linux (based on Redhat 7.[2/3])
machines:
On Intel Processor [P4 Xeon 2.8, 6G of RAM]
y =
    0.0002 0.0111
    0.0002 0.0891
    0.0005 0.2999
    0.0012 0.7096
    0.0022 1.3835


On AMD [2200XP 2GB of RAM]
y =
    0.0181 0.0001
    0.0004 0.0002
    0.0016 0.0006
    0.0016 0.0018
    0.0030 0.0029

Subject: Very slow performance for NaNs

From: Satrajit Ghosh

Date: 23 Feb, 2003 15:17:30

Message: 6 of 11

To follow up the previous mail, the increase in time is directly
proportional to the number of operations involving NaNs.


*********** Multiplication ************
for i=1:20,a= rand(250);idx =
find(a(:)<=(i*0.05));b=a;b(idx)=NaN;tic;a*a;y(i,1)=toc;tic;a*b;y(i,
2)=toc;end;
>> y
    0.0179 5.1506
    0.0178 5.6244
    0.0178 5.9772
    0.0179 6.2807
    0.0178 6.5570
    0.0179 6.8373
    0.0178 7.1430
    0.0178 7.4105
    0.0177 7.6609
    0.0178 7.9829
    0.0178 8.2353
    0.0178 8.5397
    0.0178 8.7854
    0.0178 9.0894
    0.0178 9.3420
    0.0177 9.6318
    0.0178 9.9005
    0.0178 10.1729
    0.0177 10.4600
    0.0177 10.7336


************* Addition **************
>> for i=1:20,a= rand(250);idx =
find(a(:)<=(i*0.05));b=a;b(idx)=NaN;tic;a+a;y(i,1)=toc;tic;a+b;y(i,
2)=toc;end;
>> y
    0.0021 0.0023
    0.0014 0.0032
    0.0015 0.0043
    0.0015 0.0053
    0.0015 0.0063
    0.0015 0.0074
    0.0014 0.0084
    0.0014 0.0094
    0.0014 0.0104
    0.0014 0.0115
    0.0014 0.0125
    0.0014 0.0134
    0.0014 0.0145
    0.0014 0.0156
    0.0014 0.0166
    0.0014 0.0176
    0.0014 0.0186
    0.0014 0.0196
    0.0014 0.0207
    0.0014 0.0221

Subject: Very slow performance for NaNs

From: Matthew Brett

Date: 23 Feb, 2003 18:31:14

Message: 7 of 11

Penny Anderson wrote:


> We don't do anything more special to multiply NaNs than to multiply
> finite values. We just use "*" like you would if you wrote a C, C++
or
> FORTRAN program to do it.


Thanks for the reply. It looks as though the problem is due to the
processor, as you and Satra have suggested. I"ve appended a little
test program in C. The upshot is that some processors, and in
particular the pentium IV, are very slow at handling NaNs (about 40
times slower than for non NaN values), which can have a disabling
effect on matlab execution speed. I wonder if it would be worth the
mathworks distributing an NaN-avoiding routine for multiplication
where NaNs are likely to occur, like:


function c = nanmult(a,b)
realrow = ~any(isnan(a), 2);
realcol = ~any(isnan(b), 1);
c = ones(size(a, 1), size(b, 2));
c(:)=NaN;
c(realrow, realcol) = a(realrow,:) * b(:,realcol);


Thanks,


Matthew


# nan_time.c
#include <sys/time.h>
#include <sys/resource.h>
#include <stdio.h>


#ifndef RUSAGE_SELF
#define RUSAGE_SELF 0 /* calling process */
#endif


#define SZ 10000000


#define STIME(X) ((X.ru_utime.tv_sec + \
X.ru_stime.tv_sec) + \
                   (X.ru_utime.tv_usec + \
                    X.ru_stime.tv_usec) / 1.0E6)


int main( int argc, char *argv[ ]) {
  double i, mval;
  struct rusage t0, t1, t2;
  double a,b,c;
  char op;


  op = '*';
  getrusage(RUSAGE_SELF,&t0);
  mval = 1.0;
  for (i=0,c=1;i<SZ;i++) c*=mval;
  getrusage(RUSAGE_SELF,&t1);
  mval = 0.0 / 0.0;
  for (i=0,c=1;i<SZ;i++) c*=mval;
  getrusage(RUSAGE_SELF,&t2);
  a = STIME(t1) - STIME(t0);
  b = STIME(t2) - STIME(t1);
  c = b/a;
  printf("%c time %8.4f, %c NaN time %8.4f, proportion %8.4f\n",
op, a, op, b, c);


  op = '+';
  getrusage(RUSAGE_SELF,&t0);
  mval = 1.0;
  for (i=0,c=1;i<SZ;i++) c+=mval;
  getrusage(RUSAGE_SELF,&t1);
  mval = 0.0 / 0.0;
  for (i=0,c=1;i<SZ;i++) c+=mval;
  getrusage(RUSAGE_SELF,&t2);
  a = STIME(t1) - STIME(t0);
  b = STIME(t2) - STIME(t1);
  c = b/a;
  printf("%c time %8.4f, %c NaN time %8.4f, proportion %8.4f\n",
op, a, op, b, c);


  return(0);
}

Subject: Very slow performance for NaNs

From: Thomas R. Hoffend Jr.

Date: 24 Feb, 2003 11:40:12

Message: 8 of 11


I'm just curious. What are some applications that involve
multiplication of matrices with NaNs in them? In cases where I
have encountered NaNs such as when using griddata on measured
data, I perform some sorting of masking at some point to get rid
of the NaN's before doing a calculation. I try to avoid doing things
like taking a gradient of the gridded data with NaNs because this
can make the NaNs propagate inside the convex hull of my data.

Again, I'm just curious - and interested.

TRH

"Satrajit Ghosh" <satra@bu.edu> wrote in message
news:eeb967e.3@WebX.raydaftYaTP...
> I just posted this as a question for TMW tech support. But there
> seems to be a clear difference between Matlab running on Intel
> processors vs AMD processors, independent of the operating system.
> The following table shows what I mean.
>

(snip)

Subject: Very slow performance for NaNs

From: Penny Anderson

Date: 24 Feb, 2003 13:47:31

Message: 9 of 11

"Matthew Brett" <matthewbrett@yahoo.com> wrote in message
news:eeb967e.5@WebX.raydaftYaTP...
> Penny Anderson wrote:
>
>
> > We don't do anything more special to multiply NaNs than to
multiply
> > finite values. We just use "*" like you would if you wrote a C,
C++
> or
> > FORTRAN program to do it.
>
>
> Thanks for the reply. It looks as though the problem is due to the
> processor, as you and Satra have suggested. I"ve appended a little
> test program in C. The upshot is that some processors, and in
> particular the pentium IV, are very slow at handling NaNs (about 40
> times slower than for non NaN values), which can have a disabling
> effect on matlab execution speed. I wonder if it would be worth the
> mathworks distributing an NaN-avoiding routine for multiplication
> where NaNs are likely to occur, like:
>
>
> function c = nanmult(a,b)
> realrow = ~any(isnan(a), 2);
> realcol = ~any(isnan(b), 1);
> c = ones(size(a, 1), size(b, 2));
> c(:)=NaN;
> c(realrow, realcol) = a(realrow,:) * b(:,realcol);
>
>
> Thanks,
>
>
> Matthew

Matthew & Satra,

We're not too worried about picking up after individual hardware's
inefficiencies in this regard. We'd rather not penalize

- non-NaN computations
- hardware that handles NaNs efficiently (and this is a moving target
as new chips come out every year)

by checking for NaNs or special casing them.

Really and truly, if you have NaNs in your data, shouldn't that be a
sign to stop, take stock and massage your data? What is the meaning of
matrix multiply on a NaN-populated input anyway? I know NaNs have uses
as NULL markers in statistics and other applications, but as Thomas
Hoffend asks in this thread, what are you doing with these NaNs in
your data and what is their meaning in your algorithm?

Thanks,
  Penny.
--
Penny Anderson, The MathWorks, Inc.
penny@mathworks.com

Subject: Very slow performance for NaNs

From: borchers@rainbow.nmt.edu (Brian Borchers)

Date: 24 Feb, 2003 18:53:54

Message: 10 of 11

In addition to NaN's, many processors are extremely slow in doing
arithmetic on denormals and +-Inf's. There are situations in which
such computations are reasonable.
 
The report at
 
  http://www.sonic.net/~jddarcy/Research/fleckmrk.pdf

gives some information on processors of the mid 90's, but I haven't
seen anything more recent. I'd be interested in finding an up to
date comparison.

--
Brian Borchers borchers@nmt.edu
Department of Mathematics http://www.nmt.edu/~borchers/
New Mexico Tech Phone: 505-835-5813
Socorro, NM 87801 FAX: 505-835-5366

Subject: Very slow performance for NaNs

From: Satrajit Ghosh

Date: 24 Feb, 2003 14:28:30

Message: 11 of 11

Dear Penny,


Thanks for your reply.


I believe Matthew and I use a software package called SPM to perform
statistical analyses on functional MRI data. The statistics are
performed on time-series of 3D volumes or in short 4D data with one
dimension being time. NaNs are used to represent parts of the volume
that are outside the brain and/or the volume of interest and
therefore should not contribute to the statistics/statistical
clustering algorithms, etc.,.


It is true, one could rewrite various parts of the package where this
is a problem, but I believe the intention of the programmers was to
write it in a language that provided platform independence without
sacrificing performance [something that Matlab purports to do]. It
turns out that due to hardware differences this is not possible.


[However, I should point out that this is not stated on any
documentation from Mathworks and is directly related to scalability
of problems. When doing floating point computation, one has to take
into consideration all the different types of data the machinery is
going to handle and determine the limits and bottlenecks of these
across different platforms. And if there is a significant difference,
even if it is something that can't be fixed, it can be mentioned.]


Matthew has proposed a rather simple wrapper for NaN multiplication.
If a similar thing was implemented at the level of the numerical
operator in the interpreter, the only difference might be in the
overhead of determining:
1. existence of NaNs
2. architecture (possibly)
This could potentially be provided as an optional switch for Matlab
so that people can turn it on for special cases.


Take care,


Satra

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us