Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
ROC curve of target and source images

Subject: ROC curve of target and source images

From: Carl S.

Date: 25 Feb, 2013 12:06:08

Message: 1 of 11

Hi,

I need matlab codes to plot a roc curve of target and source images

(I have searched but the files in FileExchange are not clear (not get image files as input to give the plotted curve) for my purpose)

Subject: ROC curve of target and source images

From: Carl S.

Date: 25 Feb, 2013 19:19:09

Message: 2 of 11

"Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> Hi,
>
> I need matlab codes to plot a roc curve of target and source images
>
> (I have searched but the files in FileExchange are not clear (not get image files as input to give the plotted curve) for my purpose)

Hi,
I have seen the following function;
[tpr,fpr,thresholds] = roc(Targets, Outputs)

I think this function gives the parameters to draw ROC curves, such as true positive, false negative... But, how to obtain the input parameters from png file images ?

Subject: ROC curve of target and source images

From: Carl S.

Date: 26 Feb, 2013 11:50:07

Message: 3 of 11

"Carl S." wrote in message <kggdfd$51j$1@newscl01ah.mathworks.com>...
> "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> > Hi,
> >
> > I need matlab codes to plot a roc curve of target and source images
> >
> > (I have searched but the files in FileExchange are not clear (not get image files as input to give the plotted curve) for my purpose)
>
> Hi,
> I have seen the following function;
> [tpr,fpr,thresholds] = roc(Targets, Outputs)
>
> I think this function gives the parameters to draw ROC curves, such as true positive, false negative... But, how to obtain the input parameters from png file images ?

I have performed the following process:
 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients
 
image1 0.8313
image2 0.9157
image3 0.9328
image4 0.7790
image5 0.8547
image6 0.7026

2) I have chosen 0.7 as the threshold value, and wrote the following codes
scores = [0.8313 0.9157 0.9328 0.7790 0.8547 0.7026]
 labels = [1 1 1 0 1 0] % <= Here, ones corresponds to positive (acceptable coefficient, zero corresponds to negative (not acceptable)
 ROC(scores,labels);
 
The ROC function gives the TP and FP values and also draw the ROC curve. But the result is not good, although there is only two image that have zero label. So, the ROC curve is not correct. I still need a help, and it is urgent please :(
 
function
[Tps, Fps] = ROC(scores, labels)
%% Sort Labels and Scores by Scores
sl = [scores; labels];
[d1 d2] = sort(sl(1,:));
sorted_sl = sl(:,d2);
s_scores = sorted_sl(1,:);
s_labels = round(sorted_sl(2,:));
%% Constants
counts = histc(s_labels, unique(s_labels));
Tps = zeros(1, size(s_labels,2) + 1);
Fps = zeros(1, size(s_labels,2) + 1);
negCount = counts(1);
posCount = counts(2);
%% Shift threshold to find the ROC
for
thresIdx = 1:(size(s_scores,2)+1)
% for each Threshold Index
tpCount = 0;
fpCount = 0;
for i = [1:size(s_scores,2)]
if (i >= thresIdx) % We think it is positive
if (s_labels(i) == 1) % Right!
tpCount = tpCount + 1;
else % Wrong!
fpCount = fpCount + 1;
end
end
end
Tps(thresIdx) = tpCount/posCount;
Fps(thresIdx) = fpCount/negCount;
end
%% Draw the Curve
% Sort [Tps;Fps]
x = Tps;
y = Fps;
% Interplotion to draw spline line
count = 100;
dist = (x(1) - x(size(x,2)))/100;
xx = [x(1):-dist:x(size(x,2))];
% In order to get the interpolations, we remove all the unique numbers
[d1 d2] = unique(x);
uni_x = x(1,d2);
uni_y = y(1,d2);
yy = spline(uni_x,uni_y,xx);
% No value should exceed 1
yy = min(yy, 1);
plot(x,y,'x',xx,yy);

Subject: ROC curve of target and source images

From: Alan B

Date: 27 Feb, 2013 00:41:08

Message: 4 of 11

 > "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> I have performed the following process:
> 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients

ROC curves are more generally applicable than just your application, so it is not clear what you expect your function to do. If you want help, you might want to try explaining what you are trying to do in more detail.

Subject: ROC curve of target and source images

From: Carl S.

Date: 27 Feb, 2013 03:25:07

Message: 5 of 11

"Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgjkn4$b5j$1@newscl01ah.mathworks.com>...
> > "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> > I have performed the following process:
> > 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients
>
> ROC curves are more generally applicable than just your application, so it is not clear what you expect your function to do. If you want help, you might want to try explaining what you are trying to do in more detail.

Thanks Alan,
I would like to compare different segmentation algorithms. I have obtained segmented images from these algorithms. I have manually segmented images. I want to draw ROC curve. To draw ROC curve, I need some values such as positive true, negative true values. When I searched matlab functions to obtain these values, the functions ask Label and Scores. But I dont know how can I decide the labels and scores. Think that you have segmented brain image. How can you decide ?

Shortly, I have automatically segmented images obtained from different algorithms. And manually segmented reference images. Now, I need an application that gets these images as input and draws ROC curves.
Help me pls

Subject: ROC curve of target and source images

From: Alan B

Date: 1 Mar, 2013 17:03:08

Message: 6 of 11

"Carl S." wrote in message <kgjuaj$6d3$1@newscl01ah.mathworks.com>...
> "Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgjkn4$b5j$1@newscl01ah.mathworks.com>...
> > > "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> > > I have performed the following process:
> > > 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients
> >
> > ROC curves are more generally applicable than just your application, so it is not clear what you expect your function to do. If you want help, you might want to try explaining what you are trying to do in more detail.
>
> Thanks Alan,
> I would like to compare different segmentation algorithms. I have obtained segmented images from these algorithms. I have manually segmented images. I want to draw ROC curve. To draw ROC curve, I need some values such as positive true, negative true values. When I searched matlab functions to obtain these values, the functions ask Label and Scores. But I dont know how can I decide the labels and scores. Think that you have segmented brain image. How can you decide ?
>
> Shortly, I have automatically segmented images obtained from different algorithms. And manually segmented reference images. Now, I need an application that gets these images as input and draws ROC curves.
> Help me pls

This isn't a matlab-specific question; there are several different ways you might do that. If you can explain exactly what algorithm you are trying to implement, then you can expect an answer here. If you aren't sure what algorithm is appropriate to use for your problem, then your teacher/boss/reference paper is a better place to get that information.

Subject: ROC curve of target and source images

From: Carl S.

Date: 2 Mar, 2013 18:56:05

Message: 7 of 11

"Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgqn0c$8pv$1@newscl01ah.mathworks.com>...
> "Carl S." wrote in message <kgjuaj$6d3$1@newscl01ah.mathworks.com>...
> > "Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgjkn4$b5j$1@newscl01ah.mathworks.com>...
> > > > "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> > > > I have performed the following process:
> > > > 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients
> > >
> > > ROC curves are more generally applicable than just your application, so it is not clear what you expect your function to do. If you want help, you might want to try explaining what you are trying to do in more detail.
> >
> > Thanks Alan,
> > I would like to compare different segmentation algorithms. I have obtained segmented images from these algorithms. I have manually segmented images. I want to draw ROC curve. To draw ROC curve, I need some values such as positive true, negative true values. When I searched matlab functions to obtain these values, the functions ask Label and Scores. But I dont know how can I decide the labels and scores. Think that you have segmented brain image. How can you decide ?
> >
> > Shortly, I have automatically segmented images obtained from different algorithms. And manually segmented reference images. Now, I need an application that gets these images as input and draws ROC curves.
> > Help me pls
>
> This isn't a matlab-specific question; there are several different ways you might do that. If you can explain exactly what algorithm you are trying to implement, then you can expect an answer here. If you aren't sure what algorithm is appropriate to use for your problem, then your teacher/boss/reference paper is a better place to get that information.

Thanks Alan,
I have implemented an algorithm to draw ROC curves and to obtain accuracy value by using grayscale test and target images. But I think, the results are not ture since it give %90 accuracy when the test image seems very different. Could you please help me where the mistake is in the following codes:

TargetMask = zeros(size(TargetImg(:,:,1)));
TargetMask = TargetImg(:,:,1)>0;
NotTargetMask = logical(1-TargetMask);

TestMask = zeros(size(TestImg));
TestMask= TestImg>0;

FPimg = TestMask.*NotTargetMask
FPcounts = sum(FPimg);
FP = sum(FPimg(:))

TPimg = TestMask.*TargetMask
TPcounts = sum(TPimg);
TP = sum(TPimg(:))

FNimg = TargetMask-TPimg
FNcounts = sum(FNimg);
FN = sum(FNimg(:))

 TNimg = 1- FPimg
 TNcounts = sum(TNimg);
 TN =sum(TNimg(:))
 
 FP_TNcounts = FPcounts + TNcounts;
 TP_FNcounts = FNcounts + TPcounts;

 FP_TNcounts(find(FP_TNcounts==0))=1e-10; % to solve divide by zero error
 TP_FNcounts(find(TP_FNcounts==0))=1e-10; % to solve divide by zero error
  
FPrate(:,:,i) = FPcounts./FP_TNcounts; % 1-specifisity
TPrate(:,:,i) = TPcounts./TP_FNcounts; % sensitivity

Subject: ROC curve of target and source images

From: Carl S.

Date: 4 Mar, 2013 10:06:05

Message: 8 of 11

"Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgqn0c$8pv$1@newscl01ah.mathworks.com>...
> "Carl S." wrote in message <kgjuaj$6d3$1@newscl01ah.mathworks.com>...
> > "Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgjkn4$b5j$1@newscl01ah.mathworks.com>...
> > > > "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> > > > I have performed the following process:
> > > > 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients
> > >
> > > ROC curves are more generally applicable than just your application, so it is not clear what you expect your function to do. If you want help, you might want to try explaining what you are trying to do in more detail.
> >
> > Thanks Alan,
> > I would like to compare different segmentation algorithms. I have obtained segmented images from these algorithms. I have manually segmented images. I want to draw ROC curve. To draw ROC curve, I need some values such as positive true, negative true values. When I searched matlab functions to obtain these values, the functions ask Label and Scores. But I dont know how can I decide the labels and scores. Think that you have segmented brain image. How can you decide ?
> >
> > Shortly, I have automatically segmented images obtained from different algorithms. And manually segmented reference images. Now, I need an application that gets these images as input and draws ROC curves.
> > Help me pls
>
> This isn't a matlab-specific question; there are several different ways you might do that. If you can explain exactly what algorithm you are trying to implement, then you can expect an answer here. If you aren't sure what algorithm is appropriate to use for your problem, then your teacher/boss/reference paper is a better place to get that information.

I have updated the codes but still the results are unexpected :((
Could you PLEASE run the following codes for your images and inform me whether you get correct results or not. I need your help

clear all; close all;

TargetImg=imread('GRAYSCALEref.png');
Test(:,:,1)=imread('GRAYSCALEtest1.png');
Test(:,:,2)=imread('GRAYSCALEtest2.png');

[u v]=size(TargetImg);
TPrate=zeros(1, u, 2); FPrate=zeros(1, u, 2);

for i=1:2

TestImg=Test(:,:,i);

TargetMask = zeros(size(TargetImg(:,:,1)));
TargetMask = TargetImg(:,:,1)>0;
NotTargetMask = logical(1-TargetMask);

TestMask = zeros(size(TestImg));
TestMask= TestImg>0;

FPimg = TestMask.*NotTargetMask
FP = sum(FPimg(:))

TPimg = TestMask.*TargetMask
TP = sum(TPimg(:))

FNimg = TargetMask-TPimg
FN = sum(FNimg(:))

 TNimg = 1- FPimg
 TN =sum(TNimg(:))

 TPrate(1,:,i)=TP/(TP+FN);
 FPrate(1,:,i)=FP/(FP+TN);
 
Sensitivity(i)= TP/(TP+FN)*100
Specificity(i)= TN/(TN+FP)*100
Accuracy(i)=(TP+TN)/(TP+TN+FN+FP)*100

end
plot(sort(FPrate(:,:,1)), sort(TPrate(:,:,1)),'-y','LineWidth',3);
plot(sort(FPrate(:,:,2)), sort(TPrate(:,:,2)),'-g','LineWidth',3);
axis([0:1,0:1]);

Subject: ROC curve of target and source images

From: S C.Carl

Date: 4 Mar, 2013 17:32:09

Message: 9 of 11

"S C.Carl" wrote in message <kh1rmd$9n8$1@newscl01ah.mathworks.com>...
> "Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgqn0c$8pv$1@newscl01ah.mathworks.com>...
> > "Carl S." wrote in message <kgjuaj$6d3$1@newscl01ah.mathworks.com>...
> > > "Alan B" <monguin61REM@OVETHIS.yahoo.com> wrote in message <kgjkn4$b5j$1@newscl01ah.mathworks.com>...
> > > > > "Carl S." wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> > > > > I have performed the following process:
> > > > > 1) I have applied Dice similarity measurement for automatically segmented images and obtained the following similarity coefficients
> > > >
> > > > ROC curves are more generally applicable than just your application, so it is not clear what you expect your function to do. If you want help, you might want to try explaining what you are trying to do in more detail.
> > >
> > > Thanks Alan,
> > > I would like to compare different segmentation algorithms. I have obtained segmented images from these algorithms. I have manually segmented images. I want to draw ROC curve. To draw ROC curve, I need some values such as positive true, negative true values. When I searched matlab functions to obtain these values, the functions ask Label and Scores. But I dont know how can I decide the labels and scores. Think that you have segmented brain image. How can you decide ?
> > >
> > > Shortly, I have automatically segmented images obtained from different algorithms. And manually segmented reference images. Now, I need an application that gets these images as input and draws ROC curves.
> > > Help me pls
> >
> > This isn't a matlab-specific question; there are several different ways you might do that. If you can explain exactly what algorithm you are trying to implement, then you can expect an answer here. If you aren't sure what algorithm is appropriate to use for your problem, then your teacher/boss/reference paper is a better place to get that information.
>
> I have updated the codes but still the results are unexpected :((
> Could you PLEASE run the following codes for your images and inform me whether you get correct results or not. I need your help
>
> clear all; close all;
>
> TargetImg=imread('GRAYSCALEref.png');
> Test(:,:,1)=imread('GRAYSCALEtest1.png');
> Test(:,:,2)=imread('GRAYSCALEtest2.png');
>
> [u v]=size(TargetImg);
> TPrate=zeros(1, u, 2); FPrate=zeros(1, u, 2);
>
> for i=1:2
>
> TestImg=Test(:,:,i);
>
> TargetMask = zeros(size(TargetImg(:,:,1)));
> TargetMask = TargetImg(:,:,1)>0;
> NotTargetMask = logical(1-TargetMask);
>
> TestMask = zeros(size(TestImg));
> TestMask= TestImg>0;
>
> FPimg = TestMask.*NotTargetMask
> FP = sum(FPimg(:))
>
> TPimg = TestMask.*TargetMask
> TP = sum(TPimg(:))
>
> FNimg = TargetMask-TPimg
> FN = sum(FNimg(:))
>
> TNimg = 1- FPimg
> TN =sum(TNimg(:))
>
> TPrate(1,:,i)=TP/(TP+FN);
> FPrate(1,:,i)=FP/(FP+TN);
>
> Sensitivity(i)= TP/(TP+FN)*100
> Specificity(i)= TN/(TN+FP)*100
> Accuracy(i)=(TP+TN)/(TP+TN+FN+FP)*100
>
> end
> plot(sort(FPrate(:,:,1)), sort(TPrate(:,:,1)),'-y','LineWidth',3);
> plot(sort(FPrate(:,:,2)), sort(TPrate(:,:,2)),'-g','LineWidth',3);
> axis([0:1,0:1]);

The values from my target and test images , which have the same size (256,256), from the above codes are :

TP = 1371
FN = 205
FP = 12171
TN = 51789

As you see, FP is high (means that classifier indicates some pixels as foreground but they are actually background). Therefore, I guess that the accuracy is lower than about 50%, but the formula, Accuracy=(TP+TN)/(TP+TN+FN+FP)*100 gives the result as 81.11.
Similarly, sensitivity and specificity values are 86.99 and 80.97, which are too high for this test image

According to these metrics the classifier is successful. But the foreground on the test image seems very different from the foreground on the target image. I mean the classifier is not successful but the results do not say the same thing. WHY ? Please correct me
if I wrong (Note: I used zero as the threshold value to separate the foreground(grayscale) and background that is already zero)

My other question is that the following codes
> TPrate(1,:,i)=TP/(TP+FN);
> FPrate(1,:,i)=FP/(FP+TN);
give a point. I would like to see the curve. How to draw the curve ?

Subject: ROC curve of target and source images

From: Ashraful Alam

Date: 24 Sep, 2013 06:32:06

Message: 10 of 11

Can true positive and true negative measure applied in COLOR image. That means, i segment an color MRI image by fuzzy clustering method, now if i want to measure the true positive and true negative number of pixel, how can i do it? pls help......
thanksation
Ashraful Alam

Subject: ROC curve of target and source images

From: Anitha

Date: 14 Mar, 2014 10:46:08

Message: 11 of 11

"S C.Carl" wrote in message <kgfk3g$9i5$1@newscl01ah.mathworks.com>...
> Hi,
>
> I need matlab codes to plot a roc curve of target and source images
>
> (I have searched but the files in FileExchange are not clear (not get image files as input to give the plotted curve) for my purpose)

Hi,

Did u get the solution for drawing ROC curve between Ground truth and the segmented image

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us