Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
thermal image object identification

Subject: thermal image object identification

From: Bruno

Date: 23 Jul, 2009 11:39:02

Message: 1 of 4

Greetings,

three images were posted under "thermal image object identification" title, file exchange area, which are:

post_24bits.png: original screenshot, thermal image
post_gray.png: gray converted image
post_Canny.png: edge detection result image file

Problem description:
Given a thermal image (post_24bits.png as an example), objects should be identified and closed boundaries (egdes) be associated with each one. In this example, the wires, post, transformer and such should be recognised and their edges be delimited.
It should be noted that these images will be extracted from cam feeds, and the view angle, height, day time, as well as weather conditions are all subject to change. It points, as far as I can see, to a 3d->2d shape recognition problem, with scale, rotation, and light variations.

What has been tried:
I've been doing some rough experiments with what Matlab offers in its libraries, such as:
imread / NTSC 24bit to gray conversion / edge (Canny) => giving the post_Canny.png result
imread / graytresh / im2bw / imfill / bwlabel / regionprops => some info about the image

Results:
Regionprops give nice results when using circles, stars and triangles in a solid background color, that is, as a very basic, getting started, simulation. With practical images, no usefull results till now.

Comments:
I've been surprised that each step in this process has many doctoral thesis covering them. After reading some nice, yet technically challenging papers (Mikolajczyk, Meltzer and others), I am figuring out this is far from trivial work.
Edge detection, descriptors extraction, morphology information and cross correlation are methods that could lead to usefull results, but I'm afraid only built-in functions will not handle all of this...

What is asked for:
I'm hoping to get feedback and ideas from people that have already worked with thermal images, and sucessfully recognised some shapes in this kind of hostile environment. I'm supposing, incorrectly maybe, that a conversion to gray would not be advised here, because there is edge and shape informations in these thermal images' RGB color space.

Thanks for the time,

Bruno Marchesi.
bruno.marchesi%gmail.com

Subject: thermal image object identification

From: Máday Péter

Date: 23 Jul, 2009 23:47:24

Message: 2 of 4

Hi

I might be wrong, but the RGB of the thermal image is just a
visualization tool for the actual temperature values, so that they are
more easily understandable (the homogenous domains are better
identifyable) e.g. they have nothing to do with the shape informations
you wanted to extract. As the color image is just produced by
transforming the acquired measurement data by such a colormap.

e.g. the colormap command in MATLAB serves the same purpose.

Peter

Subject: thermal image object identification

From: ImageAnalyst

Date: 24 Jul, 2009 02:15:07

Message: 3 of 4

On Jul 23, 7:39 am, "Bruno " <bruno.march...@gmail.com> wrote:
> Greetings,
>
> three images were posted under "thermal image object identification" title, file exchange area, which are:
>
> post_24bits.png: original screenshot, thermal image
> post_gray.png: gray converted image
> post_Canny.png: edge detection result image file
>
> Problem description:
> Given a thermal image (post_24bits.png as an example), objects should be identified and closed boundaries (egdes) be associated with each one. In this example, the wires, post, transformer and such should be recognised and their edges be delimited.
> It should be noted that these images will be extracted from cam feeds, and the view angle, height, day time, as well as weather conditions are all subject to change. It points, as far as I can see, to a 3d->2d shape recognition problem, with scale, rotation, and light variations.
>
> What has been tried:
> I've been doing some rough experiments with what Matlab offers in its libraries, such as:
> imread / NTSC 24bit to gray conversion / edge (Canny) => giving the post_Canny.png result
> imread / graytresh / im2bw / imfill / bwlabel / regionprops => some info about the image
>
> Results:
> Regionprops give nice results when using circles, stars and triangles in a solid background color, that is, as a very basic, getting started, simulation. With practical images, no usefull results till now.
>
> Comments:
> I've been surprised that each step in this process has many doctoral thesis covering them. After reading some nice, yet technically challenging papers (Mikolajczyk, Meltzer and others), I am figuring out this is far from trivial work.
> Edge detection, descriptors extraction, morphology information and cross correlation are methods that could lead to usefull results, but I'm afraid only built-in functions will not handle all of this...
>
> What is asked for:
> I'm hoping to get feedback and ideas from people that have already worked with thermal images, and sucessfully recognised some shapes in this kind of hostile environment. I'm supposing, incorrectly maybe, that a conversion to gray would not be advised here, because there is edge and shape informations in these thermal images' RGB color space.
>
> Thanks for the time,
>
> Bruno Marchesi.
> bruno.marchesi%gmail.com

--------------------------------------------------------------------------------------------------------
Bruno:
I've analyzed thermal images for years. I searched the file exchange
for your name:
http://www.mathworks.com/matlabcentral/fileexchange/?term=Bruno+Marchesi
and nothing showed up. Maybe they removed it since the file exchange
is really meant for code examples, not images. Or maybe it's there
but I can't find it. Can you make it easy for us by just posting the
link to your images? Maybe you can post on flickr.com or something
like that.

Peter's right - a thermal image is monochrome and if yours is color,
then some pseudocolor lookup table (colormap) has been applied. It is
important that you get the images without this colormap. The colormap
will just make it a lot more difficult.

It will be very difficult to give heuristic definitions of regions
unless the regions are either uniform temperature, or else well
outlined by a nice edge. Even then you'll need additional
information. Even in visible it would be hard. Imagine taking a
visible light photo of your computer's motherboard and then trying to
(with code) automatically identify every little part as to whether it
is a capacitor, resistor, CPU, I/O chip, metal trace, clock crystal,
etc. Not so easy.
Regards,
ImageAnalyst

Subject: thermal image object identification

From: Bruno

Date: 24 Jul, 2009 12:33:01

Message: 4 of 4

About the image files:
After one day they really didn't show up at file exchange, so please find them here:
http://picasaweb.google.com.br/bruno.marchesi/Thermal#

Peter:
Yes, you're right and that should be straightforward since the camera sensor didn't even capture visible spectrum. The one we're using goes from 2-13(micro)m. I didn't take the time to think about it before, but the colors are just a representation scheme indeed, thanks.

ImageAnalyst:
You said:
"It will be very difficult to give heuristic definitions of regions
unless the regions are either uniform temperature, or else well
outlined by a nice edge. Even then you'll need additional
information. Even in visible it would be hard. Imagine taking a
visible light photo of your computer's motherboard and then trying to
(with code) automatically identify every little part as to whether it
is a capacitor, resistor, CPU, I/O chip, metal trace, clock crystal,
etc. Not so easy."

Reply:
If you have some time, please take a look to the 24bit image, I believe we have workable edges, haven't we? I won't go so far to call them nice, but it appears, from my inexperience, that it is feasible... Our team is considering to start just with the transformer, then expanding to other elements.
Since the technicians will collect the feeds with a camera mounted on a vehicle, and it will be just passing in front the posts etc, we could have different angles, viewpoints, to match. I was thinking to build, instead of one object model, an array of them, with various viewpoints of the same object.
When we manage to write some code with scale invariant, edge description capabilities, this array of models could be used to find a match in the captured frame. That's the best approach I could think so far...


Thanks a lot for the replies.

Bruno Marchesi.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us