Comparing images using detect features

Hello.

Just to point it out right away; im fairly new to the whole image analyzing so this might be a dumb questio, but here it goes.

Lets say i have a picture of a painting. The picture of the painting is clear, good lighting and from a straight angle. Then i have a picture of a room, the lightning and the angle might be diffrent from the origninal picture. Then i want the program to compare the images and find the location of the paintning in the room.

I have searched for a tutorial/example of something simular for a few days and came across one that worked pretty well with the image that they had used, but once i tried with an image of my own it couldn't find enough simular features.

This is the code i've used:

    signImage=rgb2gray(imread(sign));
    figure(1)
    imshow(signImage)
    title('Image of sign')

    findSignImage=rgb2gray(imread(findsign));
    figure(2)
    imshow(findSignImage)
    title('Image of the place to find the sign')

    signPoints=detectSURFFeatures(signImage);
    findPoints=detectSURFFeatures(findSignImage);

    figure(3)
    imshow(signImage)
    title('100 strongest features from sign image')
    hold on
    plot(selectStrongest(signPoints,100))

    figure(4)
    imshow(findSignImage)
    title('300 strongest features from the place to find the sign')
    hold on
    plot(selectStrongest(findPoints,300))

    [signFeatures, signPoints] = extractFeatures(signImage, signPoints);
    [findFeatures, findPoints] = extractFeatures(findSignImage, findPoints);

    picPairs=matchFeatures(signFeatures, findFeatures);

    matchedSignPoints = signPoints(picPairs(:,1),:);
    matchedFindPoints = findPoints(picPairs(:,2),:);
    figure(5)
    showMatchedFeatures(signImage, findSignImage,matchedSignPoints, matchedFindPoints, 'montage')
    title('Matched points both images')

    [tform, inlierSignPoints, inlierFindPoints] = estimateGeometricTransform(matchedSignPoints, matchedFindPoints, 'affine');
    figure(6)
    showMatchedFeatures(signImage, findSignImage, inlierSignPoints, inlierFindPoints, 'montage')
    title('matched points of interest')

    signPolygon=[1,1; size(signImage,2), 1; size(signImage,2), size(signImage,1); 1, size(signImage,1); 1,1];
    newSignPolygon=transformPointsForward(tform, signPolygon);

    figure(7)
    imshow(findSignImage)
    hold on
    line(newSignPolygon(:,1), newSignPolygon(:,2), 'color', 'y');
    title('Detected sign')

Where "sign.png" is the picture of the paiting and "findSign.png" is the picture of the room.

I assume that detectSURFFeatures might not be optimal for this kind of program, but as i said im fairly new to the whole thing and haven't really understood all of the diffrent features that are possible to exctract from an image.

Any tips on how to make this more flexible? Any other features that might be worth taking a look at? Any help at all would be appreciated.

Thanks, and sorry if it got a bit messy.

Answer 1

[Image Analyst]

The Computer Vision System Toolbox can help with this. See http://www.mathworks.com/products/computer-vision/features.html#feature-detection%2C-extraction%2C-and-matching. And I know SURF has been successfully used in CBIR (Content Based Image Retrieval) to do things like "Find me all photos that have a picture frame in the." - in fact I saw this exact question used in a demo of CBIR in a course I took at an SPIE meeting. You might look up CBIR - they are making tremendous advances rapidly in that field.

If you have an exact template, you can use normalized cross correlation, and I've attached a demo for you.

[Anton]

Thanks a lot for the fast response, i will most likely use some of those functions.

Lets say i have 300 photos containing an item from diffrent angles and diffrent lighting and such, and i want to crop out the part that i find interesting. What i find most interesting is based on a reference picture which i therefore want to compare. Apparently SURF cant identify enough common points of interest for the objects i've been trying my code on and i suppose i have to change the feature im approching. But would it be possible to combine several detectXXXfeatures and let them combine their result to find the most interesting points? Or is there an "easier" way to go about the problem?

[Image Analyst]

If surf doesn't work, then more traditional, standard image segmentation methods may work. It depends on how easy it is to determine those characteristics for the images. If you want rectangles, then maybe you can try corner() in the Image Processing Toolbox, for example.