Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
ethnicity identification from facial images

Subject: ethnicity identification from facial images

From: Sai Krishna Popuri

Date: 8 Nov, 2010 17:24:05

Message: 1 of 16

Hi,
As part of my final project, I am working on a recognition system for age, gender and ethnicity. I am a newbie to image processing. I have gone through some research papers which suggest Linear Discriminant Analysis, some suggest using a local color histogram and skin color segmentation, some also suggest Gabor Filter banks + Adaboost learning + SVM classifier. I am not asking for a cut paste run code, but, can anyone please throw some light on what technique I should use and any suggestions on how to implement it

Subject: ethnicity identification from facial images

From: Jan Simon

Date: 8 Nov, 2010 17:51:04

Message: 2 of 16

Dear Sai,

> As part of my final project, I am working on a recognition system for age, gender and ethnicity.

I cannot imagine what this project could be useful to.
Can you explain the advantages or the purpose of an automatic discrimination of ethnicity?
Do you expect that you can decide for a specific gender in 100% of the cases at all?

Looking in the history, I get a bad bad feeling, whenever people try to sort human individuals into distinct groups. But I'm curious to learn more.

Kind regards, Jan

Subject: ethnicity identification from facial images

From: Sai Krishna Popuri

Date: 8 Nov, 2010 18:12:06

Message: 3 of 16

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <ib9da7$e1g$1@fred.mathworks.com>...
> Dear Sai,
>
> > As part of my final project, I am working on a recognition system for age, gender and ethnicity.
>
> I cannot imagine what this project could be useful to.
> Can you explain the advantages or the purpose of an automatic discrimination of ethnicity?
> Do you expect that you can decide for a specific gender in 100% of the cases at all?

Dear Jan,
            I am not a kind of person who would discriminate between people just because they are black,white, brown or whatever the color be. I am just working on a project which would finally be integrated into a LCD panel which would help deliver targeted advertisements. The camera atop the LCD analyses the live video or takes periodical snapshots to deliver targeted ads. I am initially trying to implement the system to classify still images. :)
Research has been done/going on in this specific area. I am not the one who started it

Sai Krishna

Subject: ethnicity identification from facial images

From: ImageAnalyst

Date: 8 Nov, 2010 18:12:15

Message: 4 of 16

Sai Krishna Popuri:
I suggest you change to a simpler project. This is far too advanced
for a newbie to image processing, or even someone with over 30 years
of image processing experience. Heck, even most humans would have a
high rate of failure for this, let alone a computer algorithm. Get
something you can successfully accomplish as your "final project."

Subject: ethnicity identification from facial images

From: Sai Krishna Popuri

Date: 8 Nov, 2010 18:34:04

Message: 5 of 16

@ImageAnalyst
Thanks for your advice
The whole idea of a Camera integrated in a LCD panel is slightly far fetched.. I agree.. I am only trying to implement the classification on still images. Processing Video is presently out of question. Do you think it is too hard to atleast identify any two of these three parameters age, gender and ethnicity. I've gone through few research papers which outline a system for Gender classification(Rowley, Baluja). It didn't strike me to be too hard. I have done a introductory course in Image Processing and have some background in Signal Processing as well
No, I am not targeting 100% accuracy, I know even 90% is very hard for ethnicity.
Once again, any help for the gender detection would be much appreciated
Sai Krishna

Subject: ethnicity identification from facial images

From: Walter Roberson

Date: 8 Nov, 2010 18:41:23

Message: 6 of 16

On 10-11-08 11:51 AM, Jan Simon wrote:
> Dear Sai,
>
>> As part of my final project, I am working on a recognition system for
>> age, gender and ethnicity.
>
> I cannot imagine what this project could be useful to.
> Can you explain the advantages or the purpose of an automatic
> discrimination of ethnicity?

Other than the obvious uses for recognizing / tracking individuals in crowds
for police investigations or for detecting known criminals or "terrorists" ??

There are commercial uses. For example, a chain of stores would derive
commercial advantage from analyzing the age / gender / ethnicity of the
passers-by (pedestrians, bus, cars) of each location, in order to derive what
kinds of wares to stock and (by way of other studies) what kind of advertising
to use to lure those passers-by in to the store.

Subject: ethnicity identification from facial images

From: Sean

Date: 8 Nov, 2010 18:50:07

Message: 7 of 16

"Sai Krishna Popuri" <saikrishna89@gmail.com> wrote in message <ib9fqs$s9g$1@fred.mathworks.com>...
> @ImageAnalyst
> Thanks for your advice
> The whole idea of a Camera integrated in a LCD panel is slightly far fetched.. I agree.. I am only trying to implement the classification on still images. Processing Video is presently out of question. Do you think it is too hard to atleast identify any two of these three parameters age, gender and ethnicity. I've gone through few research papers which outline a system for Gender classification(Rowley, Baluja). It didn't strike me to be too hard. I have done a introductory course in Image Processing and have some background in Signal Processing as well
> No, I am not targeting 100% accuracy, I know even 90% is very hard for ethnicity.
> Once again, any help for the gender detection would be much appreciated
> Sai Krishna

You have a very hard project.

I have trouble determining if someone is male/female in person much less in a picture. I can't imagine teaching a computer to tell the difference. The same thing with ethnicity. Do you actually mean "ethnicity" or do you mean race? Ethnicity is the historical and cultural background of a person. Thus not only do you need to identify their race, gender, age; you need to identify them! With approximately 6 billion people on the planet and most of them not photographed from a similar angle and lighting to your reference image, I would say this is virtually an impossible task. Well maybe Google could find a way to do it, but they're special :)

If you mean race, then your problem is not impossible but still difficult. Will the images be all of the same lighting? That would make a huge difference. Anyway, I would recommend finding a new project.

Subject: ethnicity identification from facial images

From: ImageAnalyst

Date: 8 Nov, 2010 19:28:54

Message: 8 of 16

Sai, I know you said you have some papers to get you started. Here
are the rest:

http://iris.usc.edu/Vision-Notes/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics

Virtually all papers ever published on the topic are listed here.
Obviously I haven't read them all, but you're welcome to, particularly
the overview/review/survey papers. I think about all you can hope for
is some very general classification based on the mean skin color,
especially for a student project. Like dark skin = african or indian
desnent, light skin = European, etc. You may also want to take hair
color into account. But I think your accuracy will be very low unless
you have a very small training set and test set of images.

Subject: ethnicity identification from facial images

From: Walter Roberson

Date: 8 Nov, 2010 20:40:31

Message: 9 of 16

On 10-11-08 12:12 PM, Sai Krishna Popuri wrote:

> I am just
> working on a project which would finally be integrated into a LCD panel
> which would help deliver targeted advertisements. The camera atop the
> LCD analyses the live video or takes periodical snapshots to deliver
> targeted ads.

Ads, especially targeted ads, tend to change quickly. The LCD panel would thus
be likely to have a network connection on it, in order to download the ads
themselves and the criteria for whom they should be shown to (and, of course,
bug fixes and security fixes.) Once you have that combination, there would
inevitably be pressure from security people to at least monitor the video
feed, or to use the algorithms to track or recognize people -- for example to
recognize known shoplifters if nothing else.

Subject: ethnicity identification from facial images

From: Jan Simon

Date: 8 Nov, 2010 21:47:04

Message: 10 of 16

Dear Walter,

> >> As part of my final project, I am working on a recognition system for
> >> age, gender and ethnicity.
> >
> > I cannot imagine what this project could be useful to.
 
> Other than the obvious uses for recognizing / tracking individuals in crowds
> for police investigations or for detecting known criminals or "terrorists" ??
> There are commercial uses.

You hit the point. You set the term "terrorists" in quotes.
I've thought twice before I've chosen the formulation, that I cannot imagine a use. I claim, that the "obvious use" is not "useful" in a strong scientific or ethical definition.

In my personal opinion, automatic discrimination between ethnical groups implies a discrimination against humen ever. White commercials for white pedestrians?!
But I admit that this opinion does not directly concern Matlab and is in consequence off topic in CSSM. On the other hand, I believe that programmers are responsible for the application of their programs also.

Anyhow, I did not want to offend or accuse the OP! I did unterstand, that it was not his/her (sorry, I cannot identify the gender of "Sai Krishna Popuri") decision to start this project. Age/gender/ethnic specific commercials are not one of the most demanding problems in the wide field of racism.

Kind regards, Jan

Subject: ethnicity identification from facial images

From: Sai Krishna Popuri

Date: 8 Nov, 2010 22:27:04

Message: 11 of 16

@Imageanalyst many many thanks again !!

Please lets not take this too seriously.... I hope this idea of targeted ads may have originated in some board room for cutting down the expenditures involved in marketing and advertising. they want to make their marketing specific and their ads too so they could have a better (money spent on ads) to (increase in sales ratio).

Leaving ethnicity alone, I dont see any harm in doing age and gender classification ??
Any help on that would be much appreciated
@Jan I am a male (Never thought i would have to post this on a public forum lol..)

Subject: ethnicity identification from facial images

From: ImageAnalyst

Date: 8 Nov, 2010 22:44:21

Message: 12 of 16

On Nov 8, 3:40 pm, Walter Roberson <rober...@hushmail.com> wrote:
> there would
> inevitably be pressure from security people to at least monitor the video
> feed, or to use the algorithms to track or recognize people -- for example to
> recognize known shoplifters if nothing else.
-----------------------------------------------------------------
They have this already in some areas. I saw several years ago (on the
Discovery or Learning Channel) how the casinos in Las Vegas have
cameras that do face recognition to identify known cheats. They even
do a passable job at recognizing disguises (beards, hats, glasses),
and bring up a list of images of possible known cheats from the
database that might match the person in question. Then the operator
can make a decision whether to send security personnel down to watch
the person more closely. It was surprising how sophisticated the
casino security is. I also heard about face scanning at the Super
Bowl where they would scan the crowd looking for known "wanted"
persons.

Prof Thomas Huang from the University of Illinois (Urbana) has done
some nice work in age estimation:

http://www.upi.com/Science_News/2008/09/25/Age-estimation-software-is-created/UPI-72841222350660/

Subject: ethnicity identification from facial images

From: Jan Simon

Date: 9 Nov, 2010 01:02:03

Message: 13 of 16

Dear Sai,

> Leaving ethnicity alone, I dont see any harm in doing age and gender classification ??

So let's start with CLASS 1 software with a well age detection:
It will advertise toys and candy if I'm walking along the LCD screen together with my 4 year old son. I will tell him:
"Son, this is the temptation. And now lets excercise to resist it."

Now the smarter CLASS 2 implementation, which tries to consider the sex:
All the the presented toys will be pink, because my son decided to let his hair grow in the winter. I will tell him: "Don't be sad. There are some cars also inside te shop."

The well trained CLASS 3 software suggests to visit a barber.

Let's omit the intermediate steps and come directly to the CLASS 10 advertiser program: The new German identity cards contain an RFID chip. Therefore the sophisticated LCD-panel recognizes, that I haven't follow the suggestion to go to the barber shop the last three times. It gives up this strategy and suggests me to go to the next pharmacy, because I haven't bought new drugs for my gall bladder for 8 weeks -- and "increasing the sales ratio" includes a simple calculation: 42 Euro for Fenipentol or 12 Euro for a Lego airplane - forget the child...

So where are we currently:
- Guess, to which time of the day the commercials for sweeties are shown in the tv?
- Where do you find overwhelming cupboards containing candy in the big stores?
- Why are the advertisments for beer optimized for a target audience in the age of 13 to 19? 60% of the 15 year old boys can identify all beer brands from the sound jingles in the tv commercials.
- Currently advertisments on posters (placards?) for alcoholic drinks and cigaretts are not allowed near to school buildings. But of course the producers can and will by-pass this law by launching advertisments optimized to reach teenagers using age-detection systems like in your project: A teenager has more life time to buy a specific brand of cigaretts!

"Increasing the sales ratio" reminds me to an article about hybrid cars: Their price can be reduced by 700 Euro, because they do not need advertising: the people buy them voluntarily. And the other way around: If I buy a conventional car, I have to pay additional 700 Euro for letting the manufacturer tell me, that I want to buy it.

I do see a certain potential to harm in an automatic age and gender classification. But I do not take it too serious, because I'm not paid for saving the world. Lukily you got some more useful help from others, which have more knowledge about face recognition, and less opinions.

Sorry for being absolutely off topic, Jan

Subject: ethnicity identification from facial images

From: Sai Krishna Popuri

Date: 10 Nov, 2010 17:11:04

Message: 14 of 16

@ImageAnalyst

I have been working on the same problem of ethnicity, My professor suggested me to use a local color histogram to classify the images,
So, I have a few frontal face images of people of one race. I separately computed the histograms for R,G & B planes. I have normalized them to have unit area (divided by the sum in each case). I am using images of size 36*36. I, then concatenated these three histograms into one histogram having a bin size of 3*256.
I am told by my professor to use the K Nearest Neighbor classifier. I now have only 40 frontal images in my database. So, I kept 30 for training and 10 for testing (I wasn't able to download standard databases). I basically have no idea as to how to use a KNN classifier on a histogram that i have!! please throw some light on this. I have scaled down my lofty ambitions of using an advanced classifier like SVM :). Can u also explain to me what training actually means and how it is done.

P.S On a completely unrelated note, if you have any good implementation of a face detector using Viola jones algorithm, can u please provide me a link !

Subject: ethnicity identification from facial images

From: ImageAnalyst

Date: 10 Nov, 2010 18:03:54

Message: 15 of 16

Try a simple color classifier like this demo:
http://www.mathworks.com/products/demos/image/color_seg_k/ipexhistology.html
Have, say, 3 or 4 classes for the types of races you hope to identify
(Caucasian, African, Latino, Asian, etc.). Each race would have a
certain color gamut that gets identified as a certain numbered race,
(e.g. 1, 2, 3, 4, etc.)

Subject: ethnicity identification from facial images

From: Sai Krishna Popuri

Date: 16 Nov, 2010 13:17:04

Message: 16 of 16

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <445e5698-e871-4a7c-b510-febb3a36b5cc@g2g2000vbe.googlegroups.com>...
> Try a simple color classifier like this demo:
> http://www.mathworks.com/products/demos/image/color_seg_k/ipexhistology.html
> Have, say, 3 or 4 classes for the types of races you hope to identify
> (Caucasian, African, Latino, Asian, etc.). Each race would have a
> certain color gamut that gets identified as a certain numbered race,
> (e.g. 1, 2, 3, 4, etc.)

@ ImageAnalyst
Your inputs have been really valuable.. A big thank you for all the time you have put in.
Coming to the problem of classifiers, I borrowed images from the color feret database and then Extracted faces out of them using a viola jones implementation. The faces detected were equal in length and width. I again scaled all the faces to the resolution 96*96 pixels.
I then plotted the color histograms in R,G and B planes seperately, normalised them and concatenated them to form a single column matrix of 150*1 size (I used 50 bins each for R,G&B). Now, I made a training matrix of 60 images by using the above column matrix (used the inbuilt knnclassify of matlab). Now I have a test set of 30 images, 10 of each race i.e. 1. brown complexion 2.black complexion 3.white complexion. I am able to get a 70% accuracy in case of each race on an average. the accuracy is only 60% in black complexion.
I am thinking of using a SVM classifier on the same test set. Would you recommend using RGB or changing to a different color space? Lot of images in color feret are flushed due to high exposure. Any ideas of a filter that i can use for removing that unnecessary whiteness of the image ?

thanks,

Sai Krishna Popuri

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us