Color segmentation of skin markers: HSV or Lab? How to choose thresholds?

I am pretty new to color image processing and now face the task of detecting colored skin markers like in the image below. In the cursory reading I've done, it seems like HSV and Lab are the color spaces of choice for distinguishing features based on color, but the reasoning why one would choose one over the other is not clear to me. My attempts so far have used HSV and I'm reasonably happy with the degree of segmentation accuracy I'm getting in the segmentation map to the right. It was generated using the function colorseghsv() below

imagesc(colorseghsv(A)); axis image off

However I'm wondering if Lab might be more appropriate or robust in this scenario for any reason. In this application, I will have a good deal of control over the lighting and color content of the image. For example, I've deliberately painted the markers in fluorescent colors to enhance them and that can always be made the case for all subjects.

I'm also wondering if there are more systematic ways of choosing thresholds for binning the color space coordinates. In my code below, the thresholds were chosen largely through a combination of trial and error and manually inspecting the HSV components of markers in a test image.

function segmap=colorseghsv(rgbImage)

hsv=rgb2hsv(rgbImage);

      colors={'green','rose','blue','yellow','green'};
      BW=false;
      for i=1:length(colors)
        BW=BW|markerbin(colors{i},hsv);
      end    

BW = imclose(BW,ones(5));

for ii=3:-1:1

   Q=regionprops(BW,rgbImage(:,:,ii),'MeanIntensity');
     cmap(:,ii)=vertcat(Q.MeanIntensity)/255;

end

segmap=label2rgb(bwlabel(BW),cmap,'k');

function bw=markerbin(label,hsv)

    vthresh=.9; 

   H=hsv(:,:,1);
   S=hsv(:,:,2);
   V=hsv(:,:,3);

     Vbin=V>=vthresh;

   switch label

       case 'blue'

           Hbin=abs(H-.55)<=.1;
            Sbin=S>=.5;

       case 'yellow'

           Hbin=abs(H-.16)<=.1;
            Sbin=S>=.5;

       case 'green'

           Hbin=abs(H-.325)<=.1;
           Sbin=S>=.5;

       case 'orange'    

           Hbin=abs(H-.125)<=.025;
           Sbin=S>=.5; 

       case 'rose' 

           Hbin=abs(H-.85)<=.1; 
           Sbin=S>=.5;

   end

    bw=Hbin&Sbin&Vbin;

Answer 1

[Image Analyst]

Matt, welcome to the interesting field of image analysis - I think you'll find it fun and interesting to work with pictures. HSV is kind of like LAB color space except that you're using cylindrical coordinates instead of cartesian coordinates. So like in any regular 3D space, you can specify x,y,z or if you'd prefer, you can use radius, angle, and height. With either method you can specify a point/coordinate in the 3D space. It's the same with LAB and HSV (well almost but that's good enough for now).

So, cartesian space, LAB space, is good for thresholding or carving out rectangular boxes. Cylindrical space, HSV space, is good if you need to carve out a pie-shaped sector from the color space. It would be good if you could download ImageJ and use the Color inspector 3D ( http://rsb.info.nih.gov/ij/plugins/color-inspector.html) and install it so you can see your color image's gamut in the different color spaces so you can see which space is best to threshold in. Maybe later tonight I can capture a screenshot of your image and upload it.

[Matt J]

Thanks, IA. I've attached a .mat file with the RGB image. I guess there's no MATLAB equivalent of the Color Inspector?

[Image Analyst]

Looking at the Color Inspector Gamut:

It appears that it would be best, or simplest, to segment this in HSV space. It looks like your colored dots could all be found just by looking at an S value > 0.5. Or you could improve it and go to lower S in the blue and green. So if you create a binary image by thresholding the S image and then masking it by the hue image. See how the gamut extends out in the orange and yellow areas? Well you could take pixels with saturation greater than, say 0.25 as long as their hue angle is not in the 0-90 sector. If it is you'd want to use a higher saturation threshold in that sector. You could improve it even more if you include the V channel. Take a look at the gamut from a different viewpoint:

You can see that each of the "pure" colors is in a different V range. Magenta and Orange have pretty high V, yellow and green have medium to high V, and blue has low to high V. So you could have 3 masks in the V channel to more precisely carve out the colors you want out of the gamut, and thus segment out the colors you want from the image.

I tried to write my own in MATLAB but it pales in comparison to the one for ImageJ. It's attached, for what it's worth. Someone at the Mathworks knows of the power of this tool and said he may take a crack at it for a File Exchange submission but there's no guarantees.

I hope that illustrates the situation and strategy better for you. Good luck.

By the way, if you want to run the Color Inspector directly from MATLAB, make a new toolbar shortcut and paste this code in and follow the instructions:

% Windows installation instructions:
% First, download and install ImageJ from here:
% http://rsb.info.nih.gov/ij/download.html
% Then download the 3D Color Inspector plug-in for ImageJ here:
% http://rsb.info.nih.gov/ij/plugins/color-inspector.html
% and save it here:
% C:/Program Files/ImageJ/plugins/Color
%
% Now we need to enable the plug-in to work with MATLAB.
% First open your classpath file:
% C:\Program Files\MATLAB\R2013b\toolbox\local\classpath.txt
% for editing.  Then add the following lines as new lines to your classpath file:
% C:/Program Files/ImageJ/ij.jar
% C:/Program Files/ImageJ/plugins/Color/Color_Inspector_3D.jar

clc;  % Clear the MATLAB command window.
% User must browse for the file they want to open.
% Set up a starting/initial folder that they will start browsing from.
startingFolder = pwd; % Change to something specific if you want.
if ~exist(startingFolder, 'dir')
  startingFolder = pwd;
end
% Bring up the "Open File" dialog box.
[baseFileName, folder] = uigetfile({'*.jpg;*.tif;*.png;*.gif;*.bmp','Image Files (jpg,tif,png,bmp,gif)';...
'*.*','All Files (*.*)' },'Specify image file',  startingFolder);

if baseFileName ~= 0
  % If they didn't click Cancel, build the full filename:
  fullFileName = fullfile(folder, baseFileName)
  % Now pass that into the 3D Color Inspector plugin, and you're done.
  color_inspector(fullFileName)
end

[Matt J]

Thanks! ImageJ seems to be a handy thing to have.

Lowering the S threshold for green definitely helped and lowering the V threshold for blue definitely helped. Beyond that, changing thresholds started to pick up unwanted pixels. I guess the image I gave you wasn't complete enough as a training data set...

[Matt J]

I assume you applied ImageJ to a downsampling of my original image? When I do my own scatter plot below, I see much denser sampling,

[Image Analyst]

I clicked on the image to display it, then right clicked and saved it to disk. Then I launched imageJ, read in the image, dragged out a box over just the half of the image containing the face, and launched the color inspector 3D plugin. I didn't downsample anything. I also wonder why the background on your image is white while mine is gray. Here is what it looks like when I first launch it. Are you sure you ran the same plugin? Are you sure you dragged a box over your original image and not over your segmented image (which would kind of defeat the benefit)?

[Matt J]

I didn't use ImageJ at all. I used my own MATLAB-implemented knock off below. I guess ImageJ's cropping tool included some sort of downsampling, then?

function colorinspector3d(rgbImage,colorspace)
%UNDER CONSTRUCTION

if nargin<2, colorspace='hsv'; end

switch colorspace

      case 'hsv'

          I=rgb2hsv(rgbImage);

           th=I(:,:,1);
           r=I(:,:,2);
           z=I(:,:,3)*2*pi;

           [x,y,z]=pol2cart(th(:),r(:),z(:));

      otherwise

          error 'Not implemented yet'

end

rgbData=reshape(double(rgbImage),[],3)/255;

scatter3(x,y,z,[],rgbData,'*','HitTest','off','Interruptible','off');

[Sean de Wolski]

Matt, if you're current on SMS, you might like something in the current prerelease.

[Matt J]

What "something" do you mean, Sean?

[Sean de Wolski]

Look at the release notes.

[Matt J]

I had a mistake in my colorinspector3d() routine. I was treating V as the polar coordinate instead of H. My plots should really look like below. Interesting, though... blue and magenta were much more sharply resolved in what I had before. Seems like I discovered a useful new color space :)

[Matt J]

Look at the release notes.

@Sean,

The release notes here

http://www.mathworks.com/help/matlab/release-notes.html

don't include the R2014a pre-release. Not sure where I should be looking...

[Sean de Wolski]

I think you have to install it though there might be a way to find it through your MathWorks account.

Have a great weekend.

Answer 2

[Roewie Miggs Urbano]

what is the formula used in color segmentation? thanks and God Bless

[Image Analyst]

Thresholding, often. Could also be other things - see the code for rgb2ind.