Code covered by the BSD License

### Highlights from Precision-Recall and ROC Curves

4.6
4.6 | 10 ratings Rate this file 142 Downloads (last 30 days) File Size: 4.14 KB File ID: #21528

# Precision-Recall and ROC Curves

### Stefan Schroedl (view profile)

23 Sep 2008 (Updated )

Calculate and plot P/R and ROC curves for binary classification tasks.

File Information
Description

Consider a binary classification task, and a real-valued predictor, where higher values denote more confidence that an instance is positive. By setting a fixed threshold on the output, we can trade-off recall (=true positive rate) versus false positive rate (resp. precision).

Depending on the relative class frequencies, ROC and P/R curves can highlight different properties; for details, see e.g., Davis & Goadrich, 'The Relationship Between Precision-Recall and ROC Curves', ICML 2006.

Acknowledgements

This file inspired Lynx Matlab Toolbox.

MATLAB release MATLAB 7.4 (R2007a)
Tags for This File   Please login to tag files.
Comments and Ratings (25)
21 Sep 2014 Shijia

### Shijia (view profile)

Hi Stefan,
Thanks a lot for sharing the codes. Can I ask one simple question?

As the ICML paper mentioned, in PR curve, recall values do not necessarily change linearly with precisions. Hence, the ROC curve is first constructed, and next, the PR curve is inferred from the ROC curve.

Could you please help confirm whether the provided codes do the similar stuff?

Thanks a lot
Shijia

29 May 2014 kiran paul

### kiran paul (view profile)

hi
can you help me, i am using FCM based CBIR.. i am stuck with the values of precision and recall using threshold could u plz help me with this!

29 May 2014 kiran paul

### kiran paul (view profile)

hi
can you help me, i am using FCM based CBIR.. i am stuck with the values of precision and recall using threshold could u plz help me with this!

Comment only
29 May 2013 Sbj

### Sbj (view profile)

Hi,
How do I implement these code to produce the PR curve or ROC curve comparing several segmentation algorithm to the ground truth

Comment only
07 Apr 2013 Yawar Rehman

### Yawar Rehman (view profile)

Hello Stefan,
can you help me out in it ... i am using SVM quadratic classifier; it returns class labels for test samples (i.e. 1(pos) or -1(neg)); how can i obtain score values for test samples to plot PR and ROC curves? Thanks a lot for the upload!

Comment only
07 Apr 2013 Yawar Rehman

### Yawar Rehman (view profile)

05 Aug 2012 Keikim Jap

04 Mar 2012 wx

### wx (view profile)

30 Dec 2011 Ahmed

### Ahmed (view profile)

05 Jul 2011 Saurabh Baghel

### Saurabh Baghel (view profile)

this code is intended for only binary classification tasks. what abt multiclass classification

Comment only
25 Jan 2011 Salha

### Salha (view profile)

Comment only
30 Nov 2010 Chris

### Chris (view profile)

Hi, I am interested in computing the F1-score for a precision-recall curve. The equation to do this is (2*precision*recall)/(precision+recall).

The outputs "prec" (precision) and "tpr" (recall), however, are vectors. So, if we take (precision' * recall) / (precision + recall), we will end up with a vector.

Shouldn't the F1-score be a scalar ranging from 0 to 1? Thanks for your help.

Warm Regards

Comment only
24 Nov 2010 krishna

### krishna (view profile)

i have 1 32x32 matrix in which classify the 32 clas ,all diagonal element shows correct classification while off diagonal element shows misclassification,how can i use this file for plotting precision and recall plot
krishna singh
singhkrishna5@gmail.com

Comment only
04 Nov 2010 Segun Oshin

### Segun Oshin (view profile)

Hi,

I am trying to obtain the Area Under the Precision-Recall curve. In a previous answer, you stated that your separately submitted aucroc.m would be able to estimate this, but this appears to only measure the area under ROC Curves. Since Precision-Recall curves are different, how can I determine the area under them from an AUROC? Or are you aware of any other methods of measure the Area under P-R curves?

Kind regards

08 Oct 2010 Vladislavs

### Vladislavs (view profile)

It seems that your function requires statistics toolbox. There is a function "quantile" which is found only in the statistics toolbox. It would be nice to use alternative or a free equivalent. Thanks for a function!

07 May 2010 Stefan Schroedl

### Stefan Schroedl (view profile)

Hi Zeehasham,

precision-recall curves are useful for classifiers that output a score (e.g., the higher, the more likely to be in the positive class) - if the classifier only gives you a class label, you won't get a graph, only a single precision/recall point.

Given such a classifier, for any threshold ("thresh"), you can assign examples as positive if the score exceeds it, otherwise as negative. So, for each threshold, the return values of the procedure give the corresponding precision ("prec"), true-positive rate ("tpr"), and false-positive ("fpr") rate, in that order.

Hope that helps, good luck!

Comment only
04 May 2010 zeehasham rasheed

### zeehasham rasheed (view profile)

Hey Stefan
I am using a binary classifier and want to ask you few questions

Can you tell me whats inside "prec" variable as it displays two rows. Is the first row is precision and second row is recall ?

What is "thresh" variable?

Also can you briefly explain prec/recall graph.

Your code is really helpful. Good Work !

17 Mar 2010 Stefan Schroedl

### Stefan Schroedl (view profile)

Hi Ashwin,
the link is the same as the old one.
you can use any classifier to produce the scores, the script is independent of that.

Comment only
17 Mar 2010 Ashiwn Kumar

### Ashiwn Kumar (view profile)

Hi Stefan ,
can u provide the link for the new version that u have uploaded.

Can i use baseyian classifier in this code??

Pls reply,so that i can complete my project

Comment only
17 Mar 2010 Stefan Schroedl

### Stefan Schroedl (view profile)

Farooq,
I just uploaded a new version with better option descriptions, hope that makes it more usable.
- 'count' (now called 'instanceCount') can be used if there are multiple instances with the same score. This would denote the number of instances, and 'target' the number of positive class members among those.
- area under the curve can be computed more efficiently with my 'auroc' submission.
Thanks!

Comment only
03 Nov 2009 Skynet

### Skynet (view profile)

It seems that perfcurve does this now.

Comment only
17 Apr 2009 Farooq Azam

### Farooq Azam (view profile)

Very useful code. Thanks.
But I have two questions/comments:
- what is the role of 'count'? an example would help.
- how one would calculate the area under the curve?

Thanks

Comment only
18 Dec 2008 Stefan Schroedl

### Stefan Schroedl (view profile)

The prediction itself is not part of this function, it only evaluates the output taken from an exteral predictor.

Comment only
08 Dec 2008 Xian Chen

### Xian Chen (view profile)

good. thanks a lot.
one question: which algorithm have you used to calculate? bayes? SVM?

Comment only
12 Oct 2008 David Chiang

Thanks a lot!