Automatic Rebase

Partially solution for the problem with the discrepance between available and Matlab-used memory

Automatic Rebase
================================================================================

Problem description
--------------------------------------------------------------------------------
It is a well known Matlab problem that only small matrixes can be managed in relation to the given RAM capacity. That's because their size is terminated by the size of the coherent, defragmented RAM ranges.

Problem solution
--------------------------------------------------------------------------------
The RAM is fragmented by loaded program modules, primarily DLLs. They all have a favourite position, the so called "base-address". The compiler default is 0x10000000 and all modules with this address collide. Then the operating system makes a time-consuming, not quite effective calculation of alternative addresses which results in the mentioned memory management problem. Once loaded the DLLs can not be shifted, so the base-address must be changed before loading. This is called "to rebase" and it is unfortunately sometimes not without problems, because some DLLs are just expected to be in a certain position.

Functional description
--------------------------------------------------------------------------------
There is a couple of rebase tools, but "rebase.exe" in Microsoft's Visual Studio seems to be the best. It is a command line tool without batch processing.

So Automatic Rebase generates a batch file which tries to automatically rebase all DLLs given by the input to a specified position with a first fit algorithm. This input is the output of the Matlab extension "dumpmemmex".

Because of the problem mentioned above you should use the backup-option. It makes the batch file copy each rebased DLL. DLLs which raise problems can be set on the blacklist, so they will not be rebased any longer.

Automatic Rebase is implemented in PHP, so it can be offered as a web service.

Functional problems
--------------------------------------------------------------------------------
Automatic Rebase does not handle other program modules such as executables. This has not been tested. Furthermore Automatic Rebase will not run as web service as it is, because it e.g. does not save blacklist, basebase, etc.

Usage instruction
--------------------------------------------------------------------------------
- Type a suitable base adress from the second dumpmemmex column into the "Basebase" input (default is 0x00000000).
- Copy the output of dumpmemmex and paste it into the "Input" textarea.
- Fill the "Blacklist" textarea line by line with problematical DLLs (optional).
- It is highly recommended to use the backup option: click on the "with Backup" chechbox (optional).
- Click on the "Generate Batch" button.
- Copy the content of the "Output" textarea in a new "*.bat" file or right click on "[last Batch]" and "Save target as...".
- Execute the batch file in the same directory in which the rebase.exe is located.

- Repeat all this for a better result.
- Force a rollback with the backed up DLLs if problems occur.

!!! Be warned, this is highly experimental. Use it on your own risk. !!!

Related Documents
--------------------------------------------------------------------------------
Need for Rebasing a DLL: http://www.codeproject.com/dll/RebaseDll.asp
Technical Solution Number 1-19IT9: http://www.mathworks.com/support/solutions/data/1-19IT9.html?solution=1-19IT9
1106 - Memory Management Guide: http://www.mathworks.com/support/tech-notes/1100/1106.html?BB=1
Handling Large Data Sets with MATLAB 7: http://www.mathworks.com/company/newsletters/digest/nov04/newfeatures.html
Microsoft Visual Studio Developer Center: http://msdn.microsoft.com/vstudio/

© by Johannes Kissel, Forschungszentrum Karlsruhe / IPE