View License

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Highlights from
Fuzzy Entropy and Mutual Information

5.0 | 4 ratings Rate this file 32 Downloads (last 30 days) File Size: 3.88 KB File ID: #31888 Version: 1.3

Fuzzy Entropy and Mutual Information


Rami Khushaba (view profile)


21 Jun 2011 (Updated )

An implementation of the theory of fuzzy entropy and fuzzy mutual information.

| Watch this File

File Information

Nowadays there are heaps of articles on the theory of fuzzy entropy and fuzzy mutual information. However, there is a clear significant lack for a Matlab implementation of these concepts. Based on numerous requests from students and researchers, I have prepared this code to simplify such concepts and give a tool that you can try directly. Of course, you may find heaps of different methods by which you may enhance the functionality of the code, so please feel free to inform me and the rest of any such updates overhere. Kindly, if you use this code then cite either of the following papers:

[1] R. N. Khushaba, A. Al-Jumaily, and A. Al-Ani, “Novel Feature Extraction Method based on Fuzzy Entropy and Wavelet Packet Transform for Myoelectric Control”, 7th International Symposium on Communications and Information Technologies ISCIT2007, Sydney, Australia, pp. 352 – 357.

[2] R. N. Khushaba, S. Kodagoa, S. Lal, and G. Dissanayake, “Driver Drowsiness Classification Using Fuzzy Wavelet Packet Based Feature Extraction Algorithm”, IEEE Transaction on Biomedical Engineering, vol. 58, no. 1, pp. 121-131, 2011.

[3] Ahmed Al-Ani, Rami N. Khushaba, "A Population Based Feature Subset Selection Algorithm Guided by Fuzzy Feature Dependency", AMLTA 2012, CCIS 322, pp. 430–438, 2012.

P.S: Let me know about bugs, if any.

Dr. Rami N. Khushaba

MATLAB release MATLAB 7.11 (R2010b)
Other requirements It should be efficient and fast enough, give it a try and let me know.
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (15)
14 Jan 2016 HONG WANG

Good~But it is a little time consuming

19 Nov 2015 Md . Asaduzzaman Asad

in function [fea] = mrmr_mid_d(d, f, K,I_Cx,I_xx) what would be the value of I_Cx and I_xx ?

Comment only
16 Aug 2015 mosab

mosab (view profile)

Hi every body
I'm working in feature selection with BPSO using Mutual information (By Hanchuan Peng April 16, 2003)
and i have faced problem with code
Subscript indices must either be real positive integers or logicals.

Error in BPSO (line 47)
can any one help me pleas..

the files in:

thanks advanced

Comment only
07 Jun 2014 md

md (view profile)


thank you for sharing it! i have a question i tried your fuzzy code with a signal with gaussian nose and always return de same value

i'm wondering if is possible to read “Novel Feature Extraction Method based on Fuzzy Entropy and Wavelet Packet Transform for Myoelectric Control” for understand how to enter the input signal ..i was looking in internet but i cant find it!

Comment only
06 Jan 2014 Rami Khushaba

Rami Khushaba (view profile)

Hi ZoidBerg

Just make sure that your class label, this is the last column in data, is actually organized as 1 2 3 4 etc..., i.e., the label haves consecutive numbering and in that case you can ignore or comment the grp2idx function.


Comment only
06 Dec 2013 Zoidberg

Thanks, but does it require Statistics Toolbox?
I get "Undefined function 'grp2idx' for input arguments of type 'double'".

Comment only
14 Sep 2013 Rami Khushaba

Rami Khushaba (view profile)


For those interested in mutual information based feature selection, you can use this code with the well-known MRMR feature selection algorithm from:

You will have to adjust the fuzzification parameter m to reflect better performance. You will also have to modify the original function by Hanchuan Peng into the following:


function [fea] = mrmr_mid_d(d, f, K,I_Cx,I_xx)
% MID scheme according to MRMR
% Original By Hanchuan Peng
% April 16, 2003
% Modification By Rami Khushaba
% 13/09/2013

nd = size(d,2);
nc = size(d,1);
t = I_Cx;
[tmp, idxs] = sort(t,'descend');
fea_base = idxs(1:K);
fea(1) = idxs(1);
KMAX = min(1000,nd); %500
idxleft = idxs(2:KMAX);
if bdisp==1,
fprintf('k=1 cost_time=(N/A) cur_fea=%d #left_cand=%d\n', ...
fea(k), length(idxleft));

for k=2:K,
ncand = length(idxleft);
curlastfea = length(fea);
for i=1:ncand,
t_mi(i) = I_Cx(idxleft(i));
mi_array(idxleft(i),curlastfea) = I_xx(fea(curlastfea), idxleft(i));
c_mi(i) = mean(mi_array(idxleft(i), :));
[tmp, fea(k)] = max(t_mi(1:ncand) - c_mi(1:ncand));
tmpidx = fea(k); fea(k) = idxleft(tmpidx); idxleft(tmpidx) = [];
if bdisp==1,
fprintf('k=%d cost_time=%5.4f cur_fea=%d #left_cand=%d\n', ...
k, cputime-t1, fea(k), length(idxleft));

Post any question related to fuzzy MI and I will try to reply as soon as possible.


Comment only
15 Jan 2013 Qasem

Qasem (view profile)

10 Apr 2012 Rami Khushaba

Rami Khushaba (view profile)

You welcome.

Comment only
04 Apr 2012 Fereshteh Yousefi

many many thanks for your sharing.

11 Aug 2011 Liu Guo

thank you very much!I got it

Comment only
04 Jul 2011 Rami Khushaba

Rami Khushaba (view profile)

Liu, an example is included in the code.

Comment only
04 Jul 2011 Rami Khushaba

Rami Khushaba (view profile)

Thanks Ali.

Comment only
02 Jul 2011 Liu Guo

Thank you for sharing it.In fact i want to download 'Fuzzy Mutual Information and Fuzzy Entropy '.How use it?

Comment only
23 Jun 2011 Ali Ali

Thank you very much Dr. Rami for sharing this great code

29 Aug 2011 1.1

The estimation of I_Cxx and I_xx have been updated into a better version now. Old versions are also included but commented for future use. If you find any bugs kindly let me know.

12 Mar 2012 1.2

I_xx estimation corrected with an example showing the values of I_xx and H_x when we have two completely redundant features.

13 Sep 2013 1.3

The estimation is more accurate now, you can even plug it with the well-known MRMR feature selection and try it instead of the C++ MI toolbox that comes with MRMR algorithm.

Contact us