Multivariate Gaussian Mixture Model Optimization by Cross Entropy

Stochastic multi-extremum optimization.

Description

Fit a multivariate gaussian mixture by a cross-entropy method. Cross-Entropy is a powerfull tool to achieve stochastic multi-extremum optimization.

Please visit http://iew3.technion.ac.il/CE/ for more informations

i) Please compile mex-files by the mexme_ce_gauss.m (if compiler is not setup, run mex -setup before.

ii) Run the program demo test_ce_mvgm.m

• MATLAB release: MATLAB 6.5 (R13)
• Other requirements: A C compiler

[Michael Boedigheimer]

Includes an optional animated display of the current solution as it converges, which is fun.
One nice feature is that it can estimate the number of guassian mixtures that are present. Given the correct starting number, it did better than the EM method I used (at least for the single challenging case I looked at).
Offers a lot of control of the starting and boundary conditions, which may be important in some applications.
Mostly written in c, with windows dll. means its less portable. Even so, it is still slow.

[George Uameres]

What fun!
nice way to learn about gaussian mixture models.

[feng shigang]

 

[feng shigang]

 

[Omid Aghazadeh]

Perhaps a revision is required for higher verions of matlab. Could not run the code on R2009b. Passing a 3XK matrix as input, I get
??? Index exceeds matrix dimensions.

Error in ==> ce_mvgm at 314
    Rmax(indice1) = Ra(indice1)/9 + 2*Ra(indice1)/9;
....
Ra is of dimension 1X3 and indice1 has indexes up to 9.

[Omid Aghazadeh]

Solved that one. (Needed to compile mex files!)
Now, I get NaNs!
t = 2, current sol = NaN, global sol = Inf, d = 0
t = 3, current sol = NaN, global sol = Inf, d = 0
t = 4, current sol = NaN, global sol = Inf, d = 0

[Sebastien Paris]

I think randnt is not well compiled. Did you had the correct option

mex -DranSHR3 randnt.c

or

mex -DranKISS randnt.c

?
 

[Geoffrey Lewen]

mex files compile without error and test_ce_mvgm runs without error but hangs, requiring force-kill of Matlab 2010a to recover. Using debugger, so far, sample_gaussian_mixture.m is the routine hanging....

[Sebastien Paris]

Geoffrey, please give you exact plateform, i.e., OS, CPU, Compiler and so on ... Thanks you

[Geoffrey Lewen]

Microsoft Windows XP Version 5.1 (Build 2600: SP2)
Intel Core(TM)2 Duo CPU T7300@2.00GHz
2.97GB RAM
MATLAB Version 7.10.0.499 (R2010a)
Java VM Version: Java 1.6.0_12-b04
Sun Microsystems Inc. Java HotSpot(TM) Client VM mixed mode

* MATLAB Compiler Version 4.13 (R2010a

Thank You!

[Sebastien Paris]

and your C compiler ? LCC, MSVC ?

(mex -setup)

[Geoffrey Lewen]

Sorry---compiler is LCC.

[Anton Andriyenko]

Very nice piece of software!
Have you tried to run it on Linux? Something seems to be wrong and I am not sure whether it is the compiler or something else. Basically sample_mvgm doesn't seem to work properly. The simple call
Z=sample_mvgm;
generates numbers in the range -10^10 to 10^10. Have you experienced something like that? Also the optimization runs much much slower (ca. 3x) than on Windows. According to the Matlab-profiler the bottleneck is in the last line of sample_gaussian_mixture when dirirnd is called.

I am running ubuntu 10.04 64bit on a i7 quad core
tried Matlab 2009b and 2010a
tried gcc-4.1 and gcc-4.4
always same behavior

Thanks for you help!

[Anton Andriyenko]

 

[Sebastien Paris]

Hello,

Yes, it's seems to have a bug on Linux 64 bits with GCC ... it's a question long int definition between 32 & 64 I think. On windows, it's working either on 32 or 64... I don't have a Linux 64 close to me to debug ...

[Anton Andriyenko]

Thanks for the reply. So, changing "unsigned long" to "int" in all files seems to have solved both problems. Probably not the cleanest solution but it works for now :-)