Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!x5g2000yqk.googlegroups.com!not-for-mail
From: Rune Allnor <allnor@tele.ntnu.no>
Newsgroups: comp.soft-sys.matlab
Subject: Re: An Interesting Real-World Problem...
Date: Thu, 2 Jul 2009 12:16:59 -0700 (PDT)
Organization: http://groups.google.com
Lines: 40
Message-ID: <51f3fb7d-7463-4f98-84d0-8ecdea30fa09@x5g2000yqk.googlegroups.com>
References: <h2iuip$nm2$1@fred.mathworks.com>
NNTP-Posting-Host: 77.17.142.219
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Trace: posting.google.com 1246562219 31975 127.0.0.1 (2 Jul 2009 19:16:59 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Thu, 2 Jul 2009 19:16:59 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: x5g2000yqk.googlegroups.com; posting-host=77.17.142.219; 
	posting-account=VAp5gAkAAAAmkCze5hvZtMeedpZWNthI
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; 
	Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 
	3.5.21022),gzip(gfe),gzip(gfe)
Xref: news.mathworks.com comp.soft-sys.matlab:552508


On 2 Jul, 20:34, "Geoffrey Akien" <geoff.ak...@gmail.com> wrote:
> I've been using MATLAB for around 6 months now, and only now have I needed to do something that is actually mathematically-related!  I'm a chemist, so high-level maths is not my strong point.
>
> I have some real-world data that I'd like to pick out the dramatic change in gradient of:
>
> http://www.flickr.com/photos/40058045@N07/3681613485/
> (wish I could include this in the post but never mind)
>
> Because the data is quite noisy (and not random) this is quite awkward.  In that picture there are about 5000 points.  You cannot smooth too much (in practice >300) without losing the information about the point at which the gradient changes (the smoothed data starts to cut the corner).  Even when the data is smoothed, the presence of non-random noise means that the "flat" part of the curve is not actually flat, and there are several stationary points (calculated thanks to STATIONARY from FEX) along that flat part.
>
> I've also tried fitting a least-squares linear regression using POLYFIT for each of the data points with ~500 points either side of it.  This was with the aim of picking out the part of the graph with the lowest R2 value - this I assumed would represent the fast gradient change.  However, this still does not pick out the right point, selecting a point about 0.1 hours before it should be, and I can't think why this is the case.
>
> Any other good suggestions?

Don't know about 'good', but at least a suggestion:

1) Use a 5-7 pt median filter to get rid of noise
2) Differentiate result
3) Integrate twice
4) Determine the threshold as where the cumulative
   sum reaches, say, 99% of max.

Or something like that.

Rune