Code covered by the BSD License  

Highlights from
sharedmatrix

4.63636

4.6 | 12 ratings Rate this file 37 Downloads (last 30 days) File Size: 39.3 KB File ID: #28572
image thumbnail

sharedmatrix

by

 

27 Aug 2010 (Updated )

SHAREDMATRIX Allows any Matlab object to be shared between Matlab sessions (w/o using file I/O).

| Watch this File

File Information
Description

SHAREDMATRIX Allows any Matlab object (e.g., struct, nd-cell, nd-matrix, sparse matrix) to be shared between multiple Matlab sessions without resorting to file I/O. The Matlab sessions must have access to the same shared memory resources, i.e., the processes are on the same physical system. This program uses shared memory functions specified by POSIX and in doing so avoids disk I/O for sharing. The program should work on any Linux variant but was only tested on Ubuntu.

*UPDATE: Thanks to contributor Andrew Smith, we now support Windows through the Boost InterProcess library. The windows version has not been tested by the first author.

% For example, assuming you have data X:

shmkey=12345;
sharedmatrix('clone',shmkey,X);
clear X;
spmd(8)
    X=sharedmatrix('attach',shmkey);
    % do something with X
    sharedmatrix('detach',shmkey,X);
end
sharedmatrix('free',shmkey);

For complete description please see my blog post:
http://smlv.cc.gatech.edu/2010/08/27/shared-memory-in-matlab/

Acknowledgements

Inplace Array: A Semi Pointer Package For Matlab inspired this file.

This file inspired Semaphore, Dm Utils (Data Mining Utils), and Mx Array Definition.

MATLAB release MATLAB 7.11 (R2010b)
Other requirements * Linux: POSIX shared memory (sys/shm.h) required. * Windows: Boost InterProcess required. After R2010b, array access inlining no longer supported (${MATLABROOT}/extern/include/matrix.h). Hence <R2010b must use -D"COMPLIANTMODE", otherwise no flag.
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (60)
02 May 2014 Bin

Joshua, Is it possible to apply your method to implement a shared vector between a c++ executable and Matlab, instead of two Matlab processes?

thank you very much

25 Nov 2013 Kevin Stone

dominik, sharedmatrix doesn't support class objects. Only numeric (dense/sparse/complex are all supported), logical, and character matrices, as well as cell arrays and structs consisting of those types of matrices.

25 Nov 2013 dominik

hey, this submission, if it supports sharing griddedInterpolants, seems to be what i neeed.

But being new to mex files i failed to compile it on win64 matlab 2012a and 2013b. The compiled files of Kevin did work for me, but when i used them i couldnt share a variable of the griddedInterpolant class (while for a matrix it worked). In contrast http://smlv.cc.gatech.edu/2010/08/27/shared-memory-in-matlab/ seems to indicate any matlab object should be shareable.

Can anyone help with the compilation?
Here is whow i attempted to compile the files:

- installed sdk 7.1
- ran mex -setup
- downloaded and unzipped the latest boost
- adjusted the file SharedMemory_install for the boost path
- ran SharedMemory_install

i get a lot of errors, the last lines are " c:\users\dthaler\desktop\win\SharedMemory.hpp(85) : see declaration of 'mxArray_tag'
SharedMemory.cpp(586) : error C2227: left of '->data' must point to class/struct/union/generic type
SharedMemory.cpp(586) : error C2228: left of '.number_array' must have class/struct/union
SharedMemory.cpp(586) : error C2228: left of '.pimag_data' must have class/struct/union
SharedMemory.cpp(699) : warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data
SharedMemory.cpp(920) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data

C:\APPS\MATLAB\R2012A\BIN\MEX.PL: Error: Compile of 'SharedMemory.cpp' failed. "

- adding the code mentioned below "struct mxArray_tag { ..." to SharedMemory.hpp didnt help (different error: "LINK : fatal error LNK1104: cannot open file 'libboost_date_time-vc100-mt-1_55.lib' ")

- neither did adding the mex option -D"COMPLIANTMODE" (same error)

25 Nov 2013 dominik

i forgot: i use windows 64 bit and therefore the windows version of shared matrix.

17 Jun 2013 Kevin Stone

This function was exactly what I've been looking for! I work with large matrices that only need read access, and having each worker store the whole thing wastes a ridiculous amount of memory.

However, I did notice a few bugs in the windows implementation. One was with alignment -- I have a lot of SSE mex code that needs 16 byte aligned arrays -- setting align_size in the header wasn't working as intended. Another was the handling of empty matrices and uninitialized cells in cell arrays. Also, to keep matlab from crashing (due to detach not being called) on ctrl-c or when an error occurs I wrapped the SharedMemory calls in handle classes, and made use of James Tursa's mxGetPropertyPtr submission.

The modified code/scripts are at:

http://bengal.missouri.edu/~kes25c/SharedMemory-Windows.zip

It includes precompiled binaries for windows 64 (made with Visual Studio 2010 sp1 and Boost 1.53). Maybe it will help someone else.

14 Jun 2013 ag  
13 Jun 2013 James Tursa

FYI, the published mxArray_tag that you are using is incorrect for later versions of MATLAB. The last item you list is:

size_t reserved6[3];

I know this simply reflects what TMW has actually put in their header files, but it is incorrect for later versions of MATLAB. I verified this by actually looking at the starting addresses of a couple of mxArray variables that happened to be directly next to each other in memory. The last item is not really there. Not sure it makes a differenc to what you are doing, but on later versions of MATLAB you would need to do this in order to avoid accessing memory beyond the actual struct:

size_t reserved6[2];

12 Jun 2013 Austin Abrams

I'm having some trouble with cloning arrays above a particular size:

>> shmkey = 12345;
>> shmsiz = sharedmatrix('clone',shmkey,rand(524000,1));
>> sharedmatrix('free', shmkey);
>> shmsiz = sharedmatrix('clone',shmkey,rand(525000,1));
??? Error using ==> sharedmatrix
Unable to create shared memory segment.

If it's helpful:

$cat /etc/sysctl.conf
kern.sysv.shmmax=33554432
kern.sysv.shmmin=1
kern.sysv.shmmni=256
kern.sysv.shmseg=64
kern.sysv.shmall=8192

>> m525000 = rand(525000,1);
>> m524000 = rand(524000,1);
>> whos m*
Name Size Bytes Class Attributes

m524000 524000x1 4192000 double
m525000 525000x1 4200000 double

Any thoughts?

01 Jun 2013 Nneka Richards

Hi,

I am getting an error:

Unable to detach shared memory.

What might be the cause of this error?

Thank you

11 Mar 2013 Ole

Didn't I check these stars?

11 Mar 2013 Ole

Just awesome!

I agree with Evan that the installation process is a bit rough for a compiler newbie (took me half a day to get this running). Among other things, I had to manually add code for the definition of a certain "mxArray_tag" from an "undocumented matlab" site (thanks to James Tursa for pointing this out!).

But once it is compiled it does a great job! If I detach from shared data after manipulating it, matlab does not crash. Perfect, big THANX!

Especially to Andrew Smith for porting this program to windows (thank you)!

Btw, I intend to use this programm for letting two matlab programs send output to the same parallel interface. This will (hopefully) be realized by a third program reading its instructions from shared memory. Looking forwart to testing this!

04 Jan 2013 Evan

Wow... difficult to get installed due to missing libraries, missing code, but works like a charm:

Excellent for local parallel computing (parfor) when you're broadcasting a large variable. Shared Memory allows you to skip this step, thereby greatly speeding up your code.

31 Dec 2012 Evan

Jesus Christ! Am I the only one having bad installation issues? Fucking A!

25 May 2012 Peter Costa

Nice Nice but
it works only between matlab of same version...
for example it does not work between matlab 2010a x64 and 2010a x32

14 Jan 2012 E Akbas  
14 Jan 2012 E Akbas

Detach doesn't seem to work for this variable:

x = cell(2,1);
x{2} = [1];

A related question: I see that you deepcopy an mxArray to the shared memory by going through all the elements of the array. Is it not possible to copy the mxArray as a whole block using calls like memcpy and mxDuplicateArray?

14 Jan 2012 E Akbas

I'm getting "Unable to detach shared memory." errors. What are potential reasons for this?

01 Nov 2011 Joshua Dillon

Email me. jvdillon at gmail

01 Nov 2011 oscar escarcega

So Joshua! Do u have Andrew´s email?

31 Oct 2011 oscar escarcega

Ok, I am a Linux user too =), but i need to used your app in a windows machine =(. Do you have the contact of Andrew Smith? I could ask for some support directly!

31 Oct 2011 Joshua Dillon

Hi Oscar. Sorry I do not. I don't even have access to a windows machine so it is unlikely that I will be able to obtain this information even in the foreseeable future. If you figure it out though, please do share it with us! :)

31 Oct 2011 oscar escarcega

Hello Joshua! Do you have some examples of the Andrew Smith´s windows version working the server on cpp and the client on matlab? is It possible?

19 Jul 2011 Joshua Dillon

Hey Dan--thanks for the feedback and letting us know about this trick--this is very useful! (I believe this is also confirmed by Przemyslaw's observations.)

On the next update I will add this example to the comments.

Thanks again for the feedback Dan and everyone else. Hopefully we can keep improving shared memory support!

Speaking of which--to any willing to take on the task... There should be a way to make sharedmatrix work better by having it follow and update the reference-to list in the mxArray data structure. This would make sharedmatrix variables behave more like "regular" variables. This is probably the currently most useful feature enhancement. Please email me if you're interested in helping!

19 Jul 2011 dan

Joshua, thank you for this routine. It's a godsend.

When writing to a shared variable inside a parfor loop I found I couldn't use shared_x=y I had to use shared_x(1:end)=y(1:end). Like this:

sharedmatrix('clone', x_key, x); clear x;
parfor 1:10
local_x = sharedmatrix('attach', x_key);
local_y = rand(size(local_x));

% THIS DOES NOT WORK
local_x = local_y;

% but this does
local_x(1:end) = local_y(1:end);

end; %end parfor

I assume this has to do with the mysteries of MATLAB copy-on-write. Anyway just wanted to post this important tip.

ps. This is Matlab r2010b on Linux x64.

05 Jul 2011 Joshua Dillon

Hi Przemyslaw. You are exactly correct in your observation and I'm surprised someone didn't complain about this sooner! :) In fact, this issue affected me when I was working with a 16gb cell array.

I designed it this way because I *somehow* didn't think of this simple and elegant solution! (I think because I was so focused on avoiding disk usage.) What I will do for simplicity and generality is have a killclone directive which dumps the data to disk, frees it, and loads it into shm.

I will play with alloc idea too. I avoided this orginally because I was concerned that there are too many matlab internals to support--however I could at lease provide a two-dimensional double container.

I will commit these changes as soon as I have a free moment! (Which could be a >month as I am trying to graduate.)

I do want to note that for the meantime, the current program is still acceptable. Letting the system do the swapping is effectively the same, just that it will involve one extra read from disk that isn't necessary.

I was able to load a 16gb cell into a 24gb machine (sharedmatrix would need 32gb) by letting it thrash for 20min or so. I then left the matrix in memory for the next two weeks!.

29 Jun 2011 Przemyslaw

Hi Joshua. I really appreciate Your piece of code. I plan to use it in near future but i see one prospective drawback that could be potentially simply to overcome (i hope at least).
I mean if you want to work on really large amount of data that there is a moment when you need to store in memory two copies of it.

C = rand(13000,13000);
shmkey=1234567;
shmsiz=sharedmatrix('clone',shmkey,C);% double memory usage
clear C; %until this moment

Is it possible to add a directive like 'alloc' or something in this way to allocate a matrix in a SHM?
To fill it with data from file (f.e) so one wouldn't have two copies of the data - so swapping wouldn't be necessary.
so it could work like this

shmkey=1234567;
[C,shmsiz]=sharedmatrix('alloc',shmkey,[13000 13000],'double');
C(:,:) = rand(13000,13000);

05 Jun 2011 Joshua Dillon

Hi Guy. As far as either of us could tell, Windows does not natively have shared memory. As I am not a Windows user though, I could be wrong.

I had hoped that we could design a platform independent version, however even within Boost the shared memory API is Windows specific.

05 Jun 2011 guy katz

I have a question:
when And Andrew implemented the SharedMemory function why he use boost and did not use the shared memory function of windows?

01 Jun 2011 guy katz  
31 May 2011 Binlin Wu

Josh, thank you for your emails. The code is working now for vectors. It's not critical if it works for scalar or not.

This is a great program and will be very helpful.

26 May 2011 Joshua Dillon

Submitted new version. Turns out you cannot share a scalar (Alen Wu) and I committed the "open_or_create" change (Guy Katz).

07 May 2011 Joshua Dillon

Thanks Guy for your feedback and correction--I will make the change and upload the fix.

07 May 2011 guy katz

I am sorry my mistake there is nothing wrong with the code. It is work fine.
Although there is one thing in the part of the "clone" change from
pSegment = new windows_shared_memory(create_only, segment_name, read_write, sm_size);
to
pSegment = new windows_shared_memory(open_or_create, segment_name, read_write, sm_size);
and it will be more easy to work with the code.
tanhx

07 May 2011 guy katz

I use the windows version of sharedmatrix
the SharedMemory. It seems that SharedMemory still crash when I try to attach multidimensional array.
Can some one help me.
I use the last version of SharedMemory.
Any way it is still grate function.
thank's ahead

15 Apr 2011 Joshua Dillon

Just a heads up--I introduced a regression bug which is now solved. It should only affect sparse matrices, and probably only a cell array of sparse matrices at that.

09 Apr 2011 Joshua Dillon

Dan--thanks for exploring this. Understanding memory statistics in *nix is indeed very difficult and I am certainly no expert! Be assured though, that the object is being shared and not copied despite top's reporting (writing to a shared matrix also verifies this).

I have uploaded a version which incorporates Andrew Smith's code for structs (should be available on Apr. 11). I also made several bug fixes and code cleaning. I hope this has not introduced any new bugs--unfortunately I have not had time to do extensive testing so please let me know if something doesn't work right!

I did not update the help info. This version really does support every Matlab object! Note however, that write support doesn't always work on cells or structs--only a simple matrix has guaranteed read/write support between processes. Every other object is only guaranteed to have read support.

26 Mar 2011 dan

Further tests reveal... yes, top includes shared memory in its report of each process's RES and VIRT usage.

Joshua this may be worth a mention in the documentation.

25 Mar 2011 dan

Another VERY grateful user; thank you for this contribution!

But in my tests and monitoring using top it seems that each parfor process creates a local copy of the shared global variable. Does 'top' include shared memory when reporting each process' RAM use?

Here's my test:

matlabpool('open');
C = rand(13000,13000);
shmkey=1234567;
shmsiz=sharedmatrix('clone',shmkey,C);
clear C;

parfor i = 1:5, MyTest(shmkey, i); end;

sharedmatrix('free',shmkey);

And MyTest.m():

function MyTest( shmkey, threadID )

pC = sharedmatrix('attach',shmkey);

%% This operation should not require a local copy of pC since it should be read-only.
result = nnz(pC);
result = result * threadID;
whos('pC'); % reports 1.3gb
disp(result);

sharedmatrix('detach',shmkey,pC);
clear pC;

end

When I run this test and monitor it using top, it shows each worker MATLAB process momentarily using >1gb of RAM.

Is this an artifact of 'top' -- does top report process memory inclusive of shared memory it's accessing? Or does nnz(x) actually create a local copy of x in each worker?

18 Feb 2011 Joshua Dillon

Thanks to the awesome efforts of my new colleague Andrew Smith, we now support structs and multidimensional objects. Through Boost::Interprocess it is also platform independent! Thanks Andrew!

Stay tuned for the new release!

21 Dec 2010 Min

As mentioned in previous posts and the included documentation, carefully detaching and free'ing the sharememory keys is essential because any slight sloppiness with managing your shared memory will absolutely punish you and crash Matlab most ungracefully.

That said, this function rocks! My previous multi-core version of my Matlab script shared very large 2D complex arrays by writing and reading ".mat" files. Sure it worked, but I could hear my disk RAID totally getting hammered and a good amount of time spent with file I/O. With the new version, which shares much (but not all) of the information via the sharedmatrix function, I saw my memory usage shrank to about half as much. Pretty cool, really.

Again, there are some particular limitations in its implementation. For instance, attaching a key to something other than a single variable, such as an array of structs or a cell array caused Matlab to crash while trying to detach the key. Even so, I found the function incredibly useful.

20 Dec 2010 Will

This function fills an important gap in Matlab's parallel toolbox. It's common to have a large (usually read-only) dataset on which you would like to do some operations in parallel, but you would like to avoid 2 bottlenecks:

1) Having to read the data off disk (slow).

2) Having to store the same local copy on each worker (waste of memory).

I had an application which was severely RAM limited and could only use half of the cores on my local cluster, due to the need to keep common data files in memory on each worker. This function solved that problem by allowing each worker access to the same shared copy of the data. It works well and is fast.

Some implementation issues which might save you time:

1) You will need to make the changes Joshua mentioned above to sharedmatrix.c.

2) On OSX, the default maximum of shared memory is set to 4MB at boot time. You can easily increase this by creating a config file. See this page for instructions: http://www.spy-hill.com/help/apple/SharedMemory.html

3) Be careful not to clear attached shared variables in your code. As noted in the documentation, this will cause an error.

20 Dec 2010 Joshua Dillon

As of Matlab 2010b, Mathworks has apparently dropped support of inline access to the mxArray--a feature fundamental to the functionality of this mex file. (Thanks Will for alerting me to this problem.)

The following code snippet should fix 2010b compilation errors; paste it just before the line that reads "typedef struct mxArray_tag mxArrayHack;" in sharedmatrix.c.

struct mxArray_tag {
void *reserved;
int reserved1[2];
void *reserved2;
size_t number_of_dims;
unsigned int reserved3;
struct {
unsigned int flag0 : 1;
unsigned int flag1 : 1;
unsigned int flag2 : 1;
unsigned int flag3 : 1;
unsigned int flag4 : 1;
unsigned int flag5 : 1;
unsigned int flag6 : 1;
unsigned int flag7 : 1;
unsigned int flag7a: 1;
unsigned int flag8 : 1;
unsigned int flag9 : 1;
unsigned int flag10 : 1;
unsigned int flag11 : 4;
unsigned int flag12 : 8;
unsigned int flag13 : 8;
} flags;
size_t reserved4[2];
union {
struct {
void *pdata;
void *pimag_data;
void *reserved5;
size_t reserved6[3];
} number_array;
} data;
};

19 Dec 2010 Will

Hi Joshua,

I'm trying to compile this on OSX 10.6 (Snow Leopard) with R2010b. gcc complains about the following error:

"error: dereferencing pointer to incomplete type"

in reference to

((mxArrayHack*)*A)->data.number_array.pdata = pr;

Any idea what might be causing this? I'm not a C programmer by any stretch. Thanks in advance if you are able to come up with anything.

16 Dec 2010 Min  
30 Sep 2010 Joshua Dillon

Yes, you are correct. I think that cygwin will be needed to provide shared memory functionality.

It would be possible to port this to boost::interprocess, but we should first try to compile this work using Cygwin as this keeps the code the same.

30 Sep 2010 Sebastiaan

No idea, but that should be more about (system) libraries than compiler, right? I am sorry, but I do not have a windows version of Matlab, nor do I have windows.

30 Sep 2010 Joshua Dillon

Hi Sebastiaan. Does LCC support shared memory operations? Are you running windows and if so, were you able to use this program?

30 Sep 2010 Sebastiaan

For Windows, do you need Cygwin? Matlab comes with the LCC compiler in Windows. Pretty bare-bone, but a lot of people will be happy if it compiles with LCC I guess.

30 Sep 2010 Joshua Dillon

Hi Alexander.
1) Yes! The data exists outside of Matlab hence you don't even need Matlab to access it. And of course it never uses the disk (unless the system swaps it out of course).
2) Let's work together to try to compile this with cygwin. I only have Linux but I looked into it and I think its possible. You can find my email on my website, under contact.

Cheers,
Josh

30 Sep 2010 Alexander

Thanks for your effort.
1. Is the shared matrix stored for subsequent MATLAB sessions (when MATLAB quits and restarts).
2. If yes, I would highly appreciate having a Windows version.

15 Sep 2010 Joshua Dillon

New version submitted; added ipcs interface whosshared.m.

09 Sep 2010 Joshua Dillon

New version submitted; corrects mishandling of sparse logical matrices.

01 Sep 2010 Joshua Dillon

If there is interest for a Windows version, I can easily port this code to use boost::interprocess. This would make it platform independent. Please respond if this is something you need.

31 Aug 2010 Joshua Dillon

Thanks for the tip, although I doubt it will be that helpful. The problem is that my program necessarily bypasses the Mex API and directly manipulates the mxArray data members. To do this kind of manipulation for the Matlab "struct" object requires reverse engineering its mxArray specification. Unfortunately this is not specified by matrix.h (cf. "struct mxArray_tag"). Although I do have an idea how to do this, it requires hard-coding a new C-struct, rather than just typedef'ing the one specified by matrix.h which decreases the chances that this program will work on older versions of Matlab.

I will still look into your suggestion though. I appreciate the help!

31 Aug 2010 Sebastiaan

Thanks Joshua. The forced re-read I did is a result of array manipulation outside Matlab's scope I use myself (e.g. call Twice(A) to double all elements of A without copying memory, but Matlab does not know then that A has changed, so it does not update pointers to the same data).

Maybe this will give you ideas on how to copy structures:
http://www.mathworks.com/matlabcentral/fileexchange/25656-compression-routines

This package has routines which copies any Matlab structure (except sparse matrices) to a byte array, and reconstructs it back.

31 Aug 2010 Joshua Dillon

By the way, Sebastiaan, you might be able to assign a struct member a shared matrix. If the copy-on-write mechanism works the way I hope it will, then the shared memory linkages this tool constructs *should* be preserved.

Having said that though, it is just as likely that it won't work. I considered supporting structs but cell arrays were hard enough and believe this covers most people's needs (or they could adapt their code if necessary). I will however put this on the TODO list for version 2.

One more issue of confusion I just realized from your second question: you do not need to force Matlab to reread memory. The shared object points to exactly the same piece of memory in your system. When you change one matrix you are not "updating" the other matrix--it is literally the same data.

However, you can do local "manipulations," by design. For example, the parent can have matrix A shared and the child can be A' (transposed) yet they will still have the same shared data. The only part that is truly shared is the data itself. This was done to maximize the "sharing potential," if that makes sense.

31 Aug 2010 Joshua Dillon

Hi Sebastiaan. Thanks for the corrections to the description--it has been updated.

The matrix that is cloned is not shared. You should clear the original to avoid confusion. To rework your example:

Parent:
A = [1 1];
shmkey=1;
sharedmatrix('clone', shmkey, A);
% at this point A is a totally local copy
clear A; % no need to have the original in memory
A = sharedmatrix('attach',shmkey); % now A exists as shared
% do something on client
% A is now [2 1]

Client:
shmkey = 1;
A = sharedmatrix('attach', shmkey);
A(1) = 2;

Thanks again for the question and correction!

31 Aug 2010 Sebastiaan

Very nice tool to share between different Matlab sessions. I have no use for it now since I only work with structures, but I will keep this in mind.

1) There is a bug in your description on the FEX:
X=sharedmatrix('detach',shmkey);
should be:
sharedmatrix('detach',shmkey, X);

2) I do not fully understand how it works. If I share a matrix, and change a value in the attached session, the matrix does not get updated in the parent session (also when forcing Matlab to update the matrix):
Parent:
A = [1 1];
sharedmatrix('clone', 1, A);

Client:
A = sharedmatrix('attach', 1);
A(1) = 2;

Parent, force Matlab to re-read memory:
A(2) = 2;
disp(A)
[1 2]

Why is A not updated in the parent session? It is in other client sessions.

30 Aug 2010 Joshua Dillon

Fixed version is now available.

30 Aug 2010 Joshua Dillon

Note: this first version mishandles empty cell array members and has a typo in install_shared_matrix.m. The fix is uploaded and pending review, but can also be obtained from the blog description (see above for link).

Updates
30 Aug 2010

1) fixed empty member bug
2) fixed install_sharedmatrix.m
3) updated/clarified documentation

31 Aug 2010

updated description

31 Aug 2010

updated description

09 Sep 2010

Corrected mishandling of sparse logical matrices.

15 Sep 2010

Added ipcs interface: whosshared.m

09 Apr 2011

Thanks to contributor Andrew Smith, now support structs.

Various other minor bug fixes.

14 Apr 2011

Fixed regression bug: incorrect calculation of sparse matrix size (nzmax).

26 May 2011

- Sharing of scalars is now explicitly not supported (Alen Wu).
- Windows version clone open_or_create (Guy Katz).

08 Jun 2011

No change to code. Updated icon only.

Contact us