Mann-Kendall Tau-b with Sen's Method (enhanced)

A non-parametric trend test including Sen's slope method using Mann-Kendall Tau-b technique

Description

The Mann-Kendall Tau-b non-parametric function computes a coefficient representing strength and direction of a trend for equally spaced data. While you do not need the Statistics Toolbox to compute Taub, you do need it to test for significance. This function will compute Tau-b (now tau-a as well), significance (and the various supporting statistics), and also a non-parametric slope method: Sens Method.

Note: Statistics Toolbox is only needed in this function to test for significance. Otherwise, there is nothing required beyond Matlab itself.

Enhancements include: now supports a seasonal kendall trend test recently submitted (sktt.m). Will allow for multiple observations in the same time index. Confidence intervals are added for Sens slope. Also a plot can be turned on/off.

Second enhancement: test for anomalies in trends, fixes, and notifies the user. Reference is in the comments of the function.

Given the speed of computers today, avoiding loops in Matlab is not as important for some computations as it has been in the past. Nevertheless, writing functions without loops is a form of elegance I think some people may appreciate. As such, this function is written without any loops. Example syntax to use the function is:
[taub tau h sig Z S sigma sen n senplot cilower ciupper] = ktaub(datain, alpha,wantplot)

• Required Products: Statistics Toolbox
• MATLAB release: MATLAB 7.7 (R2008b)

[Gerald Corzo]

 

[Till Ruprecht]

Dear Jeff Burkey,
thank you for providing a Mann-Kendall-algorithm based on Matlab.
A disadvantage of the algorithm obviously
is, that for large time series (i.e. 25.0000 units) the ram gets out of memory easily.
(In the example 25.000 x 25.000 x 8Byte (Double Precision) = 5 GB are required to store the replicated matrix). It would be great, having a alternative algorithm in
your method, taking into account the size
of the input data.

Greetings

[mostafa abd elhameed]

 

[Jeff Bukkey]

I would like to extend any and all CONSTRUCTIVE criticism, like Till Reprecht above. However, it is not very useful when someone submits a poor rating, and gives no comments. I?ve tried contacting Mr. Elhameed as to why he gave such a poor rating (and if possible address his concerns). I?ve received no response. In fact, Mr. Elhameed was kind enough to actually submit two ratings both with 1 star (what is the reason for that?). Mathworks was kind enough to remove one of his ratings since it seems a little disrespectful to submit the poorest rating twice, especially with no comments.
In response to Till Reprecht, I acknowledge the memory limitation in the source code with comments. My goal when submitting code to Mathworks is to provide an efficient method for performing computations. Given the way I maximized use of arrays and no circular logic, it comes at a cost of memory space. Some day when the need arises, I?ll submit a seasonal mann-kendall, heterogeneity, and maybe a few others to compliment the tau-b. But I really don?t plan on re-writing the code with loops.
Since I can get no response to Elhameed?s rating, I?m not sure it would be legitimate if I submitted this response with a rating, so I didn?t. I?m hoping other people will submit ratings (hopefully, 4~5 star) and comments, to offset Mr. Elhameed.
Regards,
Jeff (author).

[Ryan Galley]

Very simple and straightforward.
Easy to use.

[R Peterson]

I used part of it in an in-house implementation of the mann-kendall test. the calculation of the ties sum was ingenious.

[Domingo ALCARAZ-SEGURA]

It's great that it works without loops. Besides, now that W Vista allows to add RAM with flash-drives, the memory issue mentioned above does not seem such a problem. I'm excited about seeing the seasonal version of it.

[qassi tarawne]

I need to know how mann kendall trend can be done

[Jeff Burkey]

I would like to thank everyone that has commented. I'm close to the seasonal kendall, homogeneity, global trend, others. I'm in the process of validating the code for the statistics. - Jeff (author)

[Li Zhi]

There may be something wrong with your program, the calculated result of Zc and Sen's Slope are incorrect!

[Jeff Burkey]

I appologize. I did not see this comment until today (May 22, 2008). After conversing with Dr. Zhi, we agreed that the discrepancy was with the adjustments to S based on data set size. While my code adjusts S when n <= 10, it should adjust S when n >= 10. I also did note that many of the statistic packages out there adjust S regardless of dataset size (users choice at that point). This error does affect Z and p (at about the 3 significant digits or smaller). The correction has be uploaded. It should be live in a day or two. - Jeff (author)

[Jeff Burkey]

ok. Type-o on apologize.

[Hakan Moral]

 

[Hakan Moral]

Till Ruprecht([e-mail])
Is there an improvement in the code for the
large time series. if there is i will invaluably thank you for the improvement for a phd study of one of my friends i need this .
thanks in advance.

[Izi Krneki]

Is this the Kendall rank correlation? Because datain is Nx2. How do i compute Mann-Kendall trend test? Please help!

[Nicholas Cutler]

The program seems quite nice.
The latest version you download (4 July 2008) still doesn't have the correction to make s = S for small data sets.
I would like to make the taub calculation on a data set of length 8000. Unfortunately I keep getting the OUT OF MEMORY error. Any suggestions on this (besides the automatic suggestions Matlab gives - I've tried all of those).
One possibility - I've made a ktaub-a function which works quite quickly which is nice. I only have a handful of equal values for one of my 2 data sets - and when I perform the Matlab calculation [R,TIEADJ]=TIEDRANK(datain,1) I get

TIEADJ = 8 0 144

and this is also in a quick time.

Does anyone know how I can use these adjustment values for ties required in the
computation of Kendall's tau-b, as the Matlab documentation says? I cannot find any further documentation on it.

Thanks - and much appreciated, in advance.

Nick

[Jeff Burkey]

Nicholas,
When I was validating my code to other stat packages, I noticed they were returning S unadjusted. So while S is adjusted to s, the return in my matlab function is S unadjusted. If you prefer to return the adjusted, just change in the S to s in the function[...] = ktaub().
For your memory problem, I'm not sure what Matlab is suggesting, I've not had the luxury of having such a large data set to compute trends on. But maybe try casting (boxing, etc) the A# and B# matrices to single precision if you don't need double. What I'm not sure of is does Matlab require the subsequent function calls to work with double or will it box on the fly...don't know. You could pull the age old binary trick of scaling your doubles to long ints. Then scale back at the end tracking the precision needed.
Per your tied Rank function: I didn’t know about that one, but looking at the help text…can it be as simple as those are the three variables to adjust the denominator? If true, in my code those three would be: L1, t1a, t2a. Of course your S would be the adjusted S if necessary. If you want to compute your Z statistic you also need to then use the L1 variable and your ties.
Hopefully this helps, and without looking deeper into the tiedrank(x,1) function I can’t be more certain of what it does.
Regards,
Jeff.

[Ian Chien]

Could you give me a example how to build up the "datain" file used in sktt.m and ktaub.m, and Globaltrends.m in Matlab? THANKS.

[Jeff Burkey]

I've included a MAT file with example data to run the function. I hope this helps. I also added an example dataset file for ktaub, sktt, b17, globaltrends functions as well.

[Kristina]

 

[Kristina]

What are the units for Sen's slope for a time series with annual observations?

[Jeff Burkey]

Units for Sen's slope are what ever units you feed into it per time increment you feed into it. Example: cfs/year.

[Jorge]

Jeff,

Thanks for this. I am using it in my PhD research. I am a not quite familiar with matlab but I am getting there. I used the sktt code with monthly streamflow and rainfall data. Now I have new data with gaps. Would this affect the computations (n>10 in all cases and no bias)? I tried and got results. I also checked for bias in the raw data and it seems OK. Thanks for you help.

Cheers,

Jorge

[Jeff Burkey]

Jorge,
Sorry it took me so long to respond. I just saw this. In Mann-Kendall (ktaub), missing data versus not will make a difference, and the calculations account for missing data. However, for seasonal kendall (sktt), missing data are filled in with mid-ranks. So to answer your question, missing data versus not will make a difference, but missing data will not invalidate your results, unless you have a lot of missing data, but that's a different story then.

[Kazungu]

 

[Kazungu]

Jeff Burkey has giving us an excellently-written matlab-based software. Thanks for your comments which improves readability and ease of reference. Excellent and reliable resource.

[quang nguyen]

 

[Arvind Iyer]

The version of the code (ktaub.m) downloaded on July 20 2011, gives a parse error for the following lines.
[~,b] = unique(ro1);
[~,e] = unique(ro2);
How can this parse error be fixed?

[Jeff Burkey]

Arvind,
I'm guessing you're using an older version of matlab that does not accept the tilda '~'. Try replacing them with a variable.
- Jeff

[Jonathan Dyke]

Hi.. How might I expand this routine to incorporate a multivariate regression fit? i.e. 5 columns in X...

Thanks,
  Jon

[Antoniya]

Thanks Jeff! I find this function quite useful.