different-order entropy coding
5 views (last 30 days)
Show older comments
Dear all, I am doing a project about compressing text files using different-order entropies. I am done with the calculating 1-order entropy P(xn), 2-order entropy (P(xn|xn-1)) and 3-order entropy (P(xn|xn-1,xn-2)). Here are my entropies 4.2019, 3.2661 and 2.379, respectively for a sample input text from Smaple text . In the second step, I wrote another code to encode 1-order entropy using Huffman Coding. It works and Decoding is giving me the same input. However, I am little bit confused for the 2-order and 3-order entropy. After calculating entropy, what is the next step? Here are my symbols for the 2-order entropy.
,
-
.
:
;
A
B
E
F
G
H
I
L
M
O
S
T
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
,
,,
,-
,.
,:
,;
,A
,B
,E
,F
,G
,H
,I
,L
,M
,O
,S
,T
,a
,b
,c
,d
,e
,f
,g
,h
,i
,j
,k
,l
,m
,n
,o
,p
,q
,r
,s
,t
,u
,v
,w
,x
,y
-
-,
--
-.
-:
-;
-A
-B
-E
-F
-G
-H
-I
-L
-M
-O
-S
-T
-a
-b
-c
-d
-e
-f
-g
-h
-i
-j
-k
-l
-m
-n
-o
-p
-q
-r
-s
-t
-u
-v
-w
-x
-y
.
.,
.-
..
.:
.;
.A
.B
.E
.F
.G
.H
.I
.L
.M
.O
.S
.T
.a
.b
.c
.d
.e
.f
.g
.h
.i
.j
.k
.l
.m
.n
.o
.p
.q
.r
.s
.t
.u
.v
.w
.x
.y
:
:,
:-
:.
::
:;
:A
:B
:E
:F
:G
:H
:I
:L
:M
:O
:S
:T
:a
:b
:c
:d
:e
:f
:g
:h
:i
:j
:k
:l
:m
:n
:o
:p
:q
:r
:s
:t
:u
:v
:w
:x
:y
;
;,
;-
;.
;:
;;
;A
;B
;E
;F
;G
;H
;I
;L
;M
;O
;S
;T
;a
;b
;c
;d
;e
;f
;g
;h
;i
;j
;k
;l
;m
;n
;o
;p
;q
;r
;s
;t
;u
;v
;w
;x
;y
A
A,
A-
A.
A:
A;
AA
AB
AE
AF
AG
AH
AI
AL
AM
AO
AS
AT
Aa
Ab
Ac
Ad
Ae
Af
Ag
Ah
Ai
Aj
Ak
Al
Am
An
Ao
Ap
Aq
Ar
As
At
Au
Av
Aw
Ax
Ay
B
B,
B-
B.
B:
B;
BA
BB
BE
BF
BG
BH
BI
BL
BM
BO
BS
BT
Ba
Bb
Bc
Bd
Be
Bf
Bg
Bh
Bi
Bj
Bk
Bl
Bm
Bn
Bo
Bp
Bq
Br
Bs
Bt
Bu
Bv
Bw
Bx
By
E
E,
E-
E.
E:
E;
EA
EB
EE
EF
EG
EH
EI
EL
EM
EO
ES
ET
Ea
Eb
Ec
Ed
Ee
Ef
Eg
Eh
Ei
Ej
Ek
El
Em
En
Eo
Ep
Eq
Er
Es
Et
Eu
Ev
Ew
Ex
Ey
F
F,
F-
F.
F:
F;
FA
FB
FE
FF
FG
FH
FI
FL
FM
FO
FS
FT
Fa
Fb
Fc
Fd
Fe
Ff
Fg
Fh
Fi
Fj
Fk
Fl
Fm
Fn
Fo
Fp
Fq
Fr
Fs
Ft
Fu
Fv
Fw
Fx
Fy
G
G,
G-
G.
G:
G;
GA
GB
GE
GF
GG
GH
GI
GL
GM
GO
GS
GT
Ga
Gb
Gc
Gd
Ge
Gf
Gg
Gh
Gi
Gj
Gk
Gl
Gm
Gn
Go
Gp
Gq
Gr
Gs
Gt
Gu
Gv
Gw
Gx
Gy
H
H,
H-
H.
H:
H;
HA
HB
HE
HF
HG
HH
HI
HL
HM
HO
HS
HT
Ha
Hb
Hc
Hd
He
Hf
Hg
Hh
Hi
Hj
Hk
Hl
Hm
Hn
Ho
Hp
Hq
Hr
Hs
Ht
Hu
Hv
Hw
Hx
Hy
I
I,
I-
I.
I:
I;
IA
IB
IE
IF
IG
IH
II
IL
IM
IO
IS
IT
Ia
Ib
Ic
Id
Ie
If
Ig
Ih
Ii
Ij
Ik
Il
Im
In
Io
Ip
Iq
Ir
Is
It
Iu
Iv
Iw
Ix
Iy
L
L,
L-
L.
L:
L;
LA
LB
LE
LF
LG
LH
LI
LL
LM
LO
LS
LT
La
Lb
Lc
Ld
Le
Lf
Lg
Lh
Li
Lj
Lk
Ll
Lm
Ln
Lo
Lp
Lq
Lr
Ls
Lt
Lu
Lv
Lw
Lx
Ly
M
M,
M-
M.
M:
M;
MA
MB
ME
MF
MG
MH
MI
ML
MM
MO
MS
MT
Ma
Mb
Mc
Md
Me
Mf
Mg
Mh
Mi
Mj
Mk
Ml
Mm
Mn
Mo
Mp
Mq
Mr
Ms
Mt
Mu
Mv
Mw
Mx
My
O
O,
O-
O.
O:
O;
OA
OB
OE
OF
OG
OH
OI
OL
OM
OO
OS
OT
Oa
Ob
Oc
Od
Oe
Of
Og
Oh
Oi
Oj
Ok
Ol
Om
On
Oo
Op
Oq
Or
Os
Ot
Ou
Ov
Ow
Ox
Oy
S
S,
S-
S.
S:
S;
SA
SB
SE
SF
SG
SH
SI
SL
SM
SO
SS
ST
Sa
Sb
Sc
Sd
Se
Sf
Sg
Sh
Si
Sj
Sk
Sl
Sm
Sn
So
Sp
Sq
Sr
Ss
St
Su
Sv
Sw
Sx
Sy
T
T,
T-
T.
T:
T;
TA
TB
TE
TF
TG
TH
TI
TL
TM
TO
TS
TT
Ta
Tb
Tc
Td
Te
Tf
Tg
Th
Ti
Tj
Tk
Tl
Tm
Tn
To
Tp
Tq
Tr
Ts
Tt
Tu
Tv
Tw
Tx
Ty
a
a,
a-
a.
a:
a;
aA
aB
aE
aF
aG
aH
aI
aL
aM
aO
aS
aT
aa
ab
ac
ad
ae
af
ag
ah
ai
aj
ak
al
am
an
ao
ap
aq
ar
as
at
au
av
aw
ax
ay
b
b,
b-
b.
b:
b;
bA
bB
bE
bF
bG
bH
bI
bL
bM
bO
bS
bT
ba
bb
bc
bd
be
bf
bg
bh
bi
bj
bk
bl
bm
bn
bo
bp
bq
br
bs
bt
bu
bv
bw
bx
by
c
c,
c-
c.
c:
c;
cA
cB
cE
cF
cG
cH
cI
cL
cM
cO
cS
cT
ca
cb
cc
cd
ce
cf
cg
ch
ci
cj
ck
cl
cm
cn
co
cp
cq
cr
cs
ct
cu
cv
cw
cx
cy
d
d,
d-
d.
d:
d;
dA
dB
dE
dF
dG
dH
dI
dL
dM
dO
dS
dT
da
db
dc
dd
de
df
dg
dh
di
dj
dk
dl
dm
dn
do
dp
dq
dr
ds
dt
du
dv
dw
dx
dy
e
e,
e-
e.
e:
e;
eA
eB
eE
eF
eG
eH
eI
eL
eM
eO
eS
eT
ea
eb
ec
ed
ee
ef
eg
eh
ei
ej
ek
el
em
en
eo
ep
eq
er
es
et
eu
ev
ew
ex
ey
f
f,
f-
f.
f:
f;
fA
fB
fE
fF
fG
fH
fI
fL
fM
fO
fS
fT
fa
fb
fc
fd
fe
ff
fg
fh
fi
fj
fk
fl
fm
fn
fo
fp
fq
fr
fs
ft
fu
fv
fw
fx
fy
g
g,
g-
g.
g:
g;
gA
gB
gE
gF
gG
gH
gI
gL
gM
gO
gS
gT
ga
gb
gc
gd
ge
gf
gg
gh
gi
gj
gk
gl
gm
gn
go
gp
gq
gr
gs
gt
gu
gv
gw
gx
gy
h
h,
h-
h.
h:
h;
hA
hB
hE
hF
hG
hH
hI
hL
hM
hO
hS
hT
ha
hb
hc
hd
he
hf
hg
hh
hi
hj
hk
hl
hm
hn
ho
hp
hq
hr
hs
ht
hu
hv
hw
hx
hy
i
i,
i-
i.
i:
i;
iA
iB
iE
iF
iG
iH
iI
iL
iM
iO
iS
iT
ia
ib
ic
id
ie
if
ig
ih
ii
ij
ik
il
im
in
io
ip
iq
ir
is
it
iu
iv
iw
ix
iy
j
j,
j-
j.
j:
j;
jA
jB
jE
jF
jG
jH
jI
jL
jM
jO
jS
jT
ja
jb
jc
jd
je
jf
jg
jh
ji
jj
jk
jl
jm
jn
jo
jp
jq
jr
js
jt
ju
jv
jw
jx
jy
k
k,
k-
k.
k:
k;
kA
kB
kE
kF
kG
kH
kI
kL
kM
kO
kS
kT
ka
kb
kc
kd
ke
kf
kg
kh
ki
kj
kk
kl
km
kn
ko
kp
kq
kr
ks
kt
ku
kv
kw
kx
ky
l
l,
l-
l.
l:
l;
lA
lB
lE
lF
lG
lH
lI
lL
lM
lO
lS
lT
la
lb
lc
ld
le
lf
lg
lh
li
lj
lk
ll
lm
ln
lo
lp
lq
lr
ls
lt
lu
lv
lw
lx
ly
m
m,
m-
m.
m:
m;
mA
mB
mE
mF
mG
mH
mI
mL
mM
mO
mS
mT
ma
mb
mc
md
me
mf
mg
mh
mi
mj
mk
ml
mm
mn
mo
mp
mq
mr
ms
mt
mu
mv
mw
mx
my
n
n,
n-
n.
n:
n;
nA
nB
nE
nF
nG
nH
nI
nL
nM
nO
nS
nT
na
nb
nc
nd
ne
nf
ng
nh
ni
nj
nk
nl
nm
nn
no
np
nq
nr
ns
nt
nu
nv
nw
nx
ny
o
o,
o-
o.
o:
o;
oA
oB
oE
oF
oG
oH
oI
oL
oM
oO
oS
oT
oa
ob
oc
od
oe
of
og
oh
oi
oj
ok
ol
om
on
oo
op
oq
or
os
ot
ou
ov
ow
ox
oy
p
p,
p-
p.
p:
p;
pA
pB
pE
pF
pG
pH
pI
pL
pM
pO
pS
pT
pa
pb
pc
pd
pe
pf
pg
ph
pi
pj
pk
pl
pm
pn
po
pp
pq
pr
ps
pt
pu
pv
pw
px
py
q
q,
q-
q.
q:
q;
qA
qB
qE
qF
qG
qH
qI
qL
qM
qO
qS
qT
qa
qb
qc
qd
qe
qf
qg
qh
qi
qj
qk
ql
qm
qn
qo
qp
qq
qr
qs
qt
qu
qv
qw
qx
qy
r
r,
r-
r.
r:
r;
rA
rB
rE
rF
rG
rH
rI
rL
rM
rO
rS
rT
ra
rb
rc
rd
re
rf
rg
rh
ri
rj
rk
rl
rm
rn
ro
rp
rq
rr
rs
rt
ru
rv
rw
rx
ry
s
s,
s-
s.
s:
s;
sA
sB
sE
sF
sG
sH
sI
sL
sM
sO
sS
sT
sa
sb
sc
sd
se
sf
sg
sh
si
sj
sk
sl
sm
sn
so
sp
sq
sr
ss
st
su
sv
sw
sx
sy
t
t,
t-
t.
t:
t;
tA
tB
tE
tF
tG
tH
tI
tL
tM
tO
tS
tT
ta
tb
tc
td
te
tf
tg
th
ti
tj
tk
tl
tm
tn
to
tp
tq
tr
ts
tt
tu
tv
tw
tx
ty
u
u,
u-
u.
u:
u;
uA
uB
uE
uF
uG
uH
uI
uL
uM
uO
uS
uT
ua
ub
uc
ud
ue
uf
ug
uh
ui
uj
uk
ul
um
un
uo
up
uq
ur
us
ut
uu
uv
uw
ux
uy
v
v,
v-
v.
v:
v;
vA
vB
vE
vF
vG
vH
vI
vL
vM
vO
vS
vT
va
vb
vc
vd
ve
vf
vg
vh
vi
vj
vk
vl
vm
vn
vo
vp
vq
vr
vs
vt
vu
vv
vw
vx
vy
w
w,
w-
w.
w:
w;
wA
wB
wE
wF
wG
wH
wI
wL
wM
wO
wS
wT
wa
wb
wc
wd
we
wf
wg
wh
wi
wj
wk
wl
wm
wn
wo
wp
wq
wr
ws
wt
wu
wv
ww
wx
wy
x
x,
x-
x.
x:
x;
xA
xB
xE
xF
xG
xH
xI
xL
xM
xO
xS
xT
xa
xb
xc
xd
xe
xf
xg
xh
xi
xj
xk
xl
xm
xn
xo
xp
xq
xr
xs
xt
xu
xv
xw
xx
xy
y
y,
y-
y.
y:
y;
yA
yB
yE
yF
yG
yH
yI
yL
yM
yO
yS
yT
ya
yb
yc
yd
ye
yf
yg
yh
yi
yj
yk
yl
ym
yn
yo
yp
yq
yr
ys
yt
yu
yv
yw
yx
yy
.
Thank you
0 Comments
Answers (0)
See Also
Categories
Find more on Denoising and Compression in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!