Code covered by the BSD License  

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Highlights from
DataHash

5.0
5.0 | 22 ratings Rate this file 66 Downloads (last 30 days) File Size: 6.66 KB File ID: #31272 Version: 1.5

DataHash

by

Jan Simon (view profile)

 

01 May 2011 (Updated )

MD5 or SHA hash for array, struct, cell or file

| Watch this File

File Information
Description

DATAHASH - Hash for Matlab array, struct, cell or file
Hash = DataHash(Data, Opt)
  Data: Array of built-in types (U)INT8/16/32/64, SINGLE, DOUBLE (real or complex)
        CHAR, LOGICAL, CELL, STRUCT (scalar or array, nested), function_handle.
      
  Opt: Options struct:
        Opt.Method: 'SHA-1', 'SHA-256', 'SHA-384', 'SHA-512', 'MD2', 'MD5'.
        Opt.Format: Output format: 'hex', 'HEX', 'double', 'uint8', 'base64'
        Opt.Input:
             'array': The contents, type and size of the input [Data] are
                    considered for the creation of the hash. Nested CELLs
                    and STRUCT arrays are parsed recursively. Empty arrays of
                    different type reply different hashs.
             'file': [Data] is treated as file name and the hash is calculated
                    for the files contents.
             'bin': [Data] is a numerical, LOGICAL or CHAR array. Only the
                    binary contents of the array is considered, such that
                    e.g. empty arrays of different type reply the same hash.
             'ascii': Same as 'bin', but only the 8-bit ASCII part of the 16-bit
                    Matlab CHARs is considered.
  Hash: String or numeric vector.
EXAMPLES:
Default: MD5, hex:
  DataHash([]) % 7de5637fd217d0e44e0082f4d79b3e73
SHA-1, Base64:
  S.a = uint8([]);
  S.b = {{1:10}, struct('q', uint64(415))};
  Opt.Format = 'base64';
  Opt.Method = 'SHA-1';
  DataHash(S, Opt) % ZMe4eUAp0G9TDrvSW0/Qc0gQ9/A
Comparison with standard hash programs:
    Opt.Method = 'SHA-256';
    Opt.Input = 'ascii'; % Important! Matlab CHARs have 16 bits
    DataHash('abc', Opt)
Feel free to ask me about a version for Matlab 6.5.

Michael Kleder's "Compute Hash" works similar, but does not accept structs, cells or files:
  http://www.mathworks.com/matlabcentral/fileexchange/8944
"GetMD5" is 2 to 100 times faster, but it replies only MD5 hashes and a C-compiler is required:
  http://www.mathworks.com/matlabcentral/fileexchange/25921

Tested: Matlab 7.7, 7.8, 7.13, 8.6, WinXP, Win7/64, Java: 1.3, 1.6, 1.7

Bugreports and enhancement requests are welcome.

Acknowledgements

Typecast And Typecastx C Mex Functions, Get Md5, and Serialize/Deserialize inspired this file.

This file inspired Cache Results, Cachedcall, Lynx Matlab Toolbox, and Java Md5.

MATLAB release MATLAB 8.6 (R2015b)
MATLAB Search Path
/
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (41)
18 Jun 2016 Jan Simon

Jan Simon (view profile)

@Christian: Thanks you to finding this bug. It is fixed now.

Comment only
14 Jun 2016 Christian

Hey this is a great function! I've run into a problem though. I found if the file is empty (size of 0 bytes), it throws an error:

Error using java.security.MessageDigest$Delegate/update
A Non-scalar value was passed for a scalar argument

Error in DataHash (line 231)
Engine.update(Data);

Perhaps it should return whatever the hash is for a empty stream or an empty string?

05 Jun 2016 Ondrej Tichacek

Awesome function! It would be great if other encodings (such as base32 and base36) were supported as well. I implemented base32 as an extension of yours base64 (though not 100% sure if correctly) but have no clue how to get base36.

15 Oct 2015 Jan Simon

Jan Simon (view profile)

@Uilke: Wow, this is a bug in the documentation. 240f... is the expected output. The results have been changed in the version submitted on 30-Mar-2015, but I forgot to update the help text accordingly.

Comment only
15 Oct 2015 Uilke Stelwagen

Tried datahash([]) on Win7 64-bits with ML 7.13 (R2011b) and 8.6 (R2015b) and with both get 240f5f01f052bd89f38da2165dcf25c7 instead of the in the help mentioned 7de5637fd217d0e44e0082f4d79b3e73
Any ideas why?

Comment only
18 Sep 2015 Matt Raum

@Matt Raum: I found a workaround using the undocumented serialization function getByteStreamFromArray

DataHash(getByteStreamFromArray(MyClassObj))

gets me what I need.

Comment only
18 Sep 2015 Matt Raum

Jan -- Great work on this, very handy! I'd like to be able to use this to hash an instance of a custom class but I'm getting the error below:

Warning: Type of variable not considered: MyClass
> In DataHash>CoreHash_ at 336
In DataHash at 253

Can you help?

18 Sep 2015 Matt Raum  
13 Sep 2015 Jan Simon

Jan Simon (view profile)

@Ivan Cordon: The effect is explained in NOTES in the help section and was discussed at 26 May 2012:
To get the same result as in the online generator, only the binary contents of the data must be considered, not the Matlab class, and only the 8-bit ASCII part of the 16-bit Matlab CHAR:
Opt.Method='SHA-256'; Opt.Input='bin';
DataHash(uint8('ivan'), Opt)
Then you get cd0b9452... also.

Or a little bit shorter with the newest version:
Opt.Method='SHA-256'; Opt.Input='ascii';
DataHash('ivan', Opt)

Comment only
05 Sep 2015 Ivan Cordon

Dear Jan,

The program works for me but I seem to be getting a wrong result. I inputting the Opt struct with these parameters:

Method: 'SHA-256'

and calling the function as follows:

Hash = DataHash('ivan', Opt)

the result i am getting is:

53607f405e61c5b4bd7796d5ef1ce7cc123559094123c314726c68aac1e21dab

whereas checking on the different online generators (http://www.xorbin.com/tools/sha256-hash-calculator) i get this:

cd0b9452fc376fc4c35a60087b366f70d883fc901524daf1f122fbd319384f6a

Do you know why I am getting this difference?

Thanks!!

13 Aug 2015 Jan Simon

Jan Simon (view profile)

@Haitam: Do you want to obtain the clear text from the hash value? This is possible only if the input has not more bits than the hash value. For longer messages the result is not unique. And even for short messages the hashing algorithms are not designed for a reverse analysis. It is possible e.g. by a brute force attack, but this problem is not covered by my submission.

Comment only
01 Aug 2015 Haitham

Dear simon , the code is working great , i would like to know how i can recompute the hash value?
The data hash generated 558f68181d2b0c9d57d41ce7aa36b71d9
i would like to get the original value before hashing it.
thanks a lot...

Comment only
17 Jun 2015 Jan Simon

Jan Simon (view profile)

@Haitham: Look at the examples for a valid method to call this function. The error message tells you that you did not provide input arguments, or too many of them. Check the command or contact me under the address given in the help section (THISYEAR is "2010", sorry, a typo).

Comment only
17 Jun 2015 Haitham

Dear jan
thanks for the file , but i am getting error while running the code it self.

Error using DataHash>Error_L (line 466)
*** DataHash: 1 or 2 inputs required.

Error in DataHash (line 177)
Error_L('BadNInput', '1 or 2 inputs required.');

Comment only
12 Jun 2015 Alberto Gutierrez  
13 Apr 2015 Daniel Golden

Great submission, Jan. Consider putting the code on GitHub so that users can more easily comment and contribute.

16 Mar 2015 Jan Simon

Jan Simon (view profile)

Unfortunately there are some collisions (same hash for different data). E.g.: The scalar 0 and the empty double array [1 x 1 x 0] get the same hash. I'm going to publish a new version, which fixes these bugs, but produce other hashs in consequence. In addition sparse arrays are considered.

The submission CalcMD5 is updated also, such that (nested) cells and structs are considered also. Because the overhead of calling Java is omitted, this is dramatically faster than DataHash, I get a speedup factor > 100 for nested structs.

Comment only
12 Mar 2015 Aslak Grinsted

Aslak Grinsted (view profile)

 
11 Nov 2014 Noam Greenboim

Well written, good running time

13 Sep 2014 Giorgio

Well written!

04 Jul 2014 José Crespo Barrios  
07 May 2014 Andreas Bonelli  
22 Jan 2014 Christos Mitsakos  
16 Jan 2014 Clemens  
09 Jul 2013 Sergio Zlotnik  
13 Feb 2013 Nath

Nath (view profile)

function handle refering to struct containing the function will create infinite loop. Is there any workaround ?

Exemple:

d= dynamicprops();
addprop(d,'f');
d.f= @(varargin) struct2cell(d);
DataHash(d.f) % infinite loop

Comment only
09 Feb 2013 Arindam Bose

Arindam Bose (view profile)

Hi, Do anyone have any idea, how to decrypt an MD5 code?

01 Nov 2012 Igor

Igor (view profile)

 
31 Oct 2012 David

David (view profile)

Hi, I'm just wondering if this function works for p-files? Line 144-147 tells the function to stop if the file is not an m-file. When I comment these lines out, it seems to provide the correct hash for the p-file.

02 Aug 2012 Jan Simon

Jan Simon (view profile)

Thanks, Jan, a good idea. I will include this in the next version.

Comment only
24 Jul 2012 Jan Achterhold

Great function! Just as a little improvement, I added support for Java and MATLAB objects by calling their hashCode function. Just insert

elseif (isobject(Data) || isjava(Data))....
&& ismethod(Data, 'hashCode')
Engine = CoreHash(Data.hashCode, Engine);

into the CoreHash() and CoreHash_() functions (in the main if branches).

Regards, Jan

26 May 2012 Jan Simon

Jan Simon (view profile)

The examples use 8-bit CHAR strings, while Matlab uses 16 bits per CHAR. In addition DataHash includes information about the type and dimension of the input, as described in the help text. Because e.g. Mido asked for a version, which considers the actual data only last September, I've included this feature last year. Note the conversion to 8-bits:
Opt.Input = 'bin'; Opt.Method = 'MD5';
DataHash(uint8('The quick brown fox jumps over the lazy dog.'), Opt)
% >> e4d909c290d0fb1ca068ffaddf22cbd0
as expected.
But DataHash fails for the empty string and binary input! Thanks for finding this bug, it will be fixed soon.

Comment only
23 May 2012 Kotya Karapetyan

It would be nice to be able to check the returned values against some public results. For example Wikipedia mentions MD5("") = d41d8cd98f00b204e9800998ecf8427e and MD5("The quick brown fox jumps over the lazy dog.")
= e4d909c290d0fb1ca068ffaddf22cbd0, and I don't know how to get the same from DataHash to check it.

Comment only
26 Apr 2012 Jan Simon

Jan Simon (view profile)

@Oyvind: I have posted an MD5-calculator also as MEX-function. But it does not allow the direct accumulation of the hash for nested cells or structs. But you could do a XORing of the partial hashes. But this will not be trivial.

Comment only
24 Apr 2012 Oyvind

Oyvind (view profile)

Hi,
Is it possible to run Datahash without Java (I am stuck with the limitation of excel 2003 using a compiled version of datahash tha requires JVM).
Oyvind

Comment only
19 Sep 2011 Jan Simon

Jan Simon (view profile)

@Mido: Now you can use the Opt.Input='bin' method to create the hash for the raw data.

Comment only
07 Sep 2011 Jan Simon

Jan Simon (view profile)

@Mido: DataHash considers the class and dimensions of the inputs, otherwise UINT8([0,0]) and UINT16(0) would have the same hash.
I'm going to add the option, that only the contents of the input is considered. But even then the MD5 of 'I am not happy' will not be '59b4...', because Matlab uses 16 bits for a CHAR - you look for UINT8('I am not happy').
You can find a fast Md5 tool here: http://www.mathworks.com/matlabcentral/fileexchange/25921-calcmd5

Comment only
04 Sep 2011 Mido Mido

It does not work with me!!
When i tried:

Opt.Format = 'HEX';
Opt.Method = 'MD5';
DataHash('I am not happy', Opt)

it gives me : "7C23124A8F69D72A65C0E86A4B9075CF"
although the correct is:
"59b469ea3ffbe72cf4983facf13cbe1f"

The same also for SHA-1!!

Could any one help?

Comment only
20 Jul 2011 Martin

Martin (view profile)

 
09 Jun 2011 Aaron

Aaron (view profile)

Extremely well made function.
Very easy to use.
Works correctly.

03 May 2011 Francois Rongère

Very useful for my simulations parameters sets !

Thank you Jan,

Regards,

François.

Updates
12 Sep 2011 1.2

Binary mode add to consider only the contents of the data.

27 Jun 2012 1.3

Accept empty input for binary mode now.

30 Mar 2015 1.4

Fixed bugs: Strings and empty array for "binary" mode. For "array" mode [] and zeros(1,1,0) had the same hash before.

15 Oct 2015 1.4

In the version 30-Mar-2015 the results have been changed, but the help section has not been adjusted. This is fixed now.
Structs arrays are processed much faster now and the checksum differs from earlier versions.

29 Feb 2016 1.5

No need to use TYPECASTX in modern Matlab versions. Hashes have changed for 'Array' mode to reply the same output as GetMD5. Speed improvements for STRUCT arrays.

18 Jun 2016 1.5

Bugfix: Failed for empty files.

Contact us