Thread Subject:

Guess the data encoding!

Hi everybody, I'm trying to reverse engineering an instrument at work (University). Using a serial spy cable I've figured out what its sending. At regular intervals the device broadcasts some information (shown in hexadecimal here), with the corresponding values the software spits out:

00 76 64 04 00 DE == 3.590E-004
00 8C 61 0C 00 F9 == 1.012E-001

(I can give out more values if anyone is interested.)

It seems like either the first set of 00s are signify the start if the data, or perhaps the last two bytes are a checksum of some sort so that the number is single precision. I've been messing around with typecast for a while now and can't seem to get anywhere.

In the mean time I'm trying to get hold of the author of the software, but right now I can't for the life of me figure out what encoding the software is using.

Any suggestions?

Dear Geoffrey!

> Hi everybody, I'm trying to reverse engineering an instrument at work (University). Using a serial spy cable I've figured out what its sending. At regular intervals the device broadcasts some information (shown in hexadecimal here), with the corresponding values the software spits out:
>
> 00 76 64 04 00 DE == 3.590E-004
> 00 8C 61 0C 00 F9 == 1.012E-001
>
> (I can give out more values if anyone is interested.)

Please give us some more numbers.
Jan

"Jan Simon" <matlab.THIS_YEAR@nMINUSsimon.de> wrote in message <hdcji0$j04$1@fred.mathworks.com>...
> Dear Geoffrey!
>
> > Hi everybody, I'm trying to reverse engineering an instrument at work (University). Using a serial spy cable I've figured out what its sending. At regular intervals the device broadcasts some information (shown in hexadecimal here), with the corresponding values the software spits out:
> >
> > 00 76 64 04 00 DE == 3.590E-004
> > 00 8C 61 0C 00 F9 == 1.012E-001
> >
> > (I can give out more values if anyone is interested.)
>
> Please give us some more numbers.
> Jan

Firstly, those numbers I gave you were wrong, since I had some extra processing turned on in the software. Secondly, I've done some digging and I've figured out how the software encodes integers, but not floating points.

When it sends commands to change the settings an example is:

(all numbers in hexadecimal)

10000: 82 00 10 27 B9
100: 82 00 64 00 E6
10: 82 00 0A 00 8C

The first byte is the command to write a new parameter, the second byte says which parameter to change, and the final byte is a checksum. So 10000 in hex is 2710, which gets sent in the "wrong" order. But that's integers.

When the software is collecting data, the device broadcasts 6-byte lumps:

There's an option to change the data to linear or not and I have no idea which is the native data format.

00 51 38 04 00 8D == 3.178528E-0004 (roughly -34.5 dB - not clear if it arrives as logarithmic or not)
00 BC 47 04 00 07 == 3.316628E-0004
00 0C 64 0C 00 7C == 1.019e-1

The 1st and 5th bytes are always 00. I have a suspicion that the first byte signifies the start of transmission, and that perhaps the last byte is a checksum? I've tried playing around with typecast(uint8(data(2:5)), 'single') with and without byte swapping but I can't seem to arrive at a sensible answer.

Thanks

Dear Geoffrey!

> 00 51 38 04 00 8D == 3.178528E-0004 (roughly -34.5 dB - not clear if it arrives as logarithmic or not)
> 00 BC 47 04 00 07 == 3.316628E-0004
> 00 0C 64 0C 00 7C == 1.019e-1

Dechiffering a code with 32 bits with just 3 examples is a kind of random guessing. Please give me more data.
Jan

I hope this lot helps, I've given you two sets at different levels so you might be able to get something out of it. I'm not sure if the program receives it in logarithmic or linear so there's a set for each (sorry can't get common data sets for that).

Thanks


Set 1 (LED on)

Intensity/dB Data
-34.96496 00 62 39 04 00 9F
-34.91826 00 48 3D 04 00 89
-34.92692 00 8F 3C 04 00 CF
-34.99225 00 1B 37 04 00 56
-34.95064 00 94 3A 04 00 D2
-34.87488 00 E7 40 04 00 2B
-35.02823 00 1A 34 04 00 52
-34.97919 00 32 38 04 00 6E
-35.02529 00 59 34 04 00 91
-35.01391 00 4C 35 04 00 85
-34.95083 00 90 3A 04 00 CE
-34.87460 00 ED 40 04 00 31
-34.91756 00 57 3D 04 00 98
-34.88213 00 4C 40 04 00 90

Intensity Data
3.205632E-0004 00 65 3B 04 00 A4
3.221248E-0004 00 28 3D 04 00 69
3.288833E-0004 00 AF 44 04 00 F7
3.178254E-0004 00 49 38 04 00 85
3.234709E-0004 00 AB 3E 04 00 ED
3.214348E-0004 00 61 3C 04 00 A1
3.242072E-0004 00 7E 3F 04 00 C1
3.149922E-0004 00 0A 35 04 00 43
3.231469E-0004 00 4E 3E 04 00 90
3.222498E-0004 00 4C 3D 04 00 8D
3.165778E-0004 00 DC 36 04 00 16


Set 2 (LED off)

Intensity/dB Data
-16.81825 00 23 24 0A 00 51
-16.80014 00 A6 25 0A 00 D5
-16.80355 00 5D 25 0A 00 8C
-16.80056 00 9D 25 0A 00 CC
-16.80098 00 94 25 0A 00 C3
-16.79494 00 15 26 0A 00 45
-16.79335 00 37 26 0A 00 67
-16.79434 00 22 26 0A 00 52
-16.78652 00 C9 26 0A 00 F9
-16.78858 00 9D 26 0A 00 CD
-16.78165 00 31 27 0A 00 62
-16.78774 00 AF 26 0A 00 DF
-16.78839 00 A1 26 0A 00 D1
-16.78357 00 08 27 0A 00 39
-16.77936 00 62 27 0A 00 93
-16.78039 00 4C 27 0A 00 7D
-16.78338 00 0C 27 0A 00 3D
-16.78582 00 D8 26 0A 00 08
-16.77623 00 A5 27 0A 00 D6

Intensity Data
2.113343E-0002 00 CF 29 0A 00 02
2.115189E-0002 00 20 2A 0A 00 54
2.115439E-0002 00 2B 2A 0A 00 5F
2.120963E-0002 00 1D 2B 0A 00 52
2.122014E-0002 00 4B 2B 0A 00 80
2.124989E-0002 00 CD 2B 0A 00 02
2.129459E-0002 00 90 2C 0A 00 C6
2.133501E-0002 00 40 2D 0A 00 77
2.137182E-0002 00 E0 2D 0A 00 17
2.143617E-0002 00 F7 2E 0A 00 2F

Geoffrey Akien <geoff.akien@gmail.com> wrote:
> I hope this lot helps, I've given you two sets at different levels so you
might be able to get something out of it. I'm not sure if the program
receives it in logarithmic or linear so there's a set for each (sorry can't
get common data sets for that).
>

The extra data helps a lot. It looks to me as though the data
corresponding to the intensities is contained in the first four bytes, but
they are arranged from least to most significant byte, so turn them around.

h = [
'04396200'
'043D4800'
'043C8F00'
'04371B00'
'043A9400'
'0440E700'
'04341A00'
'04383200'
'04345900'
'04354C00'
'043A9000'
'0440ED00'
'043D5700'
'04404C00'
]

y = [
-34.96496
-34.91826
-34.92692
-34.99225
-34.95064
-34.87488
-35.02823
-34.97919
-35.02529
-35.01391
-34.95083
-34.87460
-34.91756
-34.88213
]

x = hex2dec(h)
plot(x,y,'+-')


--
Dr Tristram J. Scott
Energy Consultant

Dear Tristram, dear Geoffrey!

> The extra data helps a lot. It looks to me as though the data
> corresponding to the intensities is contained in the first four bytes, but
> they are arranged from least to most significant byte, so turn them around.
>
> h = [
> '04396200'
> '043D4800'
> '043C8F00'
> '04371B00'
> '043A9400'
> '0440E700'
> '04341A00'
> '04383200'
> '04345900'
> '04354C00'
> '043A9000'
> '0440ED00'
> '043D5700'
> '04404C00'
> ]
>
> y = [
> -34.96496
> -34.91826
> -34.92692
> -34.99225
> -34.95064
> -34.87488
> -35.02823
> -34.97919
> -35.02529
> -35.01391
> -34.95083
> -34.87460
> -34.91756
> -34.88213
> ]
>
> x = hex2dec(h)
> plot(x,y,'+-')

Well done, Tristram!
y = hex2dec(x) * 1.82800636878895e-007 - 47.9199538652254
This is not exactly what I expected.
Jan

That's fantastic! I didn't think of doing it that way, I'll remember that technique for next time. Thanks Tristram!

Geoffrey Akien <geoff.akien@gmail.com> wrote:
> That's fantastic! I didn't think of doing it that way, I'll remember
> that technique for next time. Thanks Tristram!

The pattern was there in the larger data set, with the fourth column
looking pretty much the same within a group of values.

The last two digits look to be a simple checksum. They are the last eight
bits of the total of all the rest of the line.

00 D8 26 0A 00 08

D8 + 26 + 0A = 108.



--
Dr Tristram J. Scott
Energy Consultant