Code covered by the BSD License  

Highlights from
SparsePack

Be the first to rate this file! 11 Downloads (last 30 days) File Size: 11.2 KB File ID: #25939
image thumbnail

SparsePack

by

 

24 Nov 2009 (Updated )

Packs a sparse matrix into a single uint8 array, reducing memory usage.

| Watch this File

File Information
Description

This package is particularly useful if you have a lot of relatively small sparse matrices in memory which are not frequently used. By packing the matrices, the memory usage is reduced. As a next step, the result can be compressed with GZIP (as with CompressLib: http://www.mathworks.com/matlabcentral/fileexchange/25656-compression-routines).

Reduction of the size is obtained in 2 ways:
1) By using only the number of bytes required for the sparse pointers. By default, 64-bit Matlab uses 64-bit pointers (8 byte). This is overkill if the dimensions of your matrix are small (e.g. <65000).

2) By offering a different numeric format. Matlab can only store doubles as sparse, while a single or int8 may provide enough precision.

Formats supported are logical/int8/uint8/int16/uint16/int32/uint32/single/double/pattern. 'pattern' is a special format which only stores the sparsity pattern (when unpacked, every non-zero entry will be 1). In contrast, the 'logical' type also remembers if an entry is 0 or not.

Special tags are 'keep' and 'cat'. By default, SparsePack only packs the number of used nonzeros (taken from the index). By specifying 'keep', all entries are stored (see spalloc for more information). 'cat' simply concatenates the vectors without any memory reduction.

Rounding is done the way Matlab does: 139.4 is rounded to 127 in int8 type.
The code is extensively tested with Matlab 2009b on 64-bit Linux, but also compiles and runs well on 32-bit Linux and 32-bit Windows. You can even exchange a packed sparse matrix between 64-bit and 32-bit, and a nice error is given if the unpacked matrix does not fit on the 32-bit system.

The code probably also depends on the little endianess of the byte ordering. I have no big endian hardware to test/optimize it on.

SparsePack is fast and a compression rate of 50% is easily met.

Compile:
 mex -largeArrayDims SparsePack.c
 mex -largeArrayDims SparseUnPack.c

Test:
 testSparsePack

Use (see screenshot):
 B = SparsePack(sparse(A), 'single', 'keep');
 C = SparseUnPack(B);

Acknowledgements

Compression Routines inspired this file.

MATLAB release MATLAB 7.9 (R2009b)
Other requirements gcc, lcc, icc, vc, 32-bit, 64-bit, little-endian
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Updates
25 Nov 2009

Decided to round the values like Matlab's round to prevent unexpected differences.

Contact us