Function precedence in multiple nested Matlab projects
3 views (last 30 days)
Show older comments
Matlab projects can add other projects as references. I started to use this to modularize my Matlab libraries. In fact, every git repo belongs to one Matlab package/namespace and gets consumed by an application. I started to use buildinspace/peru: a generic package manager, for including other people's code in your projects (github.com) for source code based dependency management (no, I do not want to use submodules).
That means, it ends up with a folder structure like
myApp
- src
-- myApp.m % needs my.namespace.lib1.m and my.othernamespace.lib2
- myAppPrj.prj % references lib-ml-lib1.prj and lib-ml-lib2.prj, does not set anything manually to the path
- resources % ML project resources
- lib
-- lib-ml-lib1
--- lib-ml-lib1.prj % adds src folder of lib1 to path
--- resources % ML project resources lib1
--- src/
---- +my/+othernamespace/lib1.m
-- lib-ml-lib2
--- lib-ml-lib2.prj % adds src folder of lib2 to path
--- resources % ML project resources lib2
--- src/
---- +my/+namespace/lib2.m
I hope the structure is clear; myApp.m depends on lib1.m and lib2.m which are functions or classes in nested packages.
This worked fine until lib1 and lib2 started to depend on lib3, but in potentially different versions. peru successfully copies the requested versions of lib3 into the lib folders in lib1 and lib2, but it seems to be random which version is chosen and I don't understand what's going on.
New situation:
myApp
- src
-- myApp.m % needs my.namespace.lib1.m and my.othernamespace.lib2
- myAppPrj.prj % references lib-ml-lib1.prj and lib-ml-lib2.prj, does not set anything manually to the path
- resources % ML project resources
- lib
-- lib-ml-lib1
--- lib-ml-lib1.prj % adds src folder of lib1 to path
--- resources % ML project resources lib1
--- src/
---- +my/+othernamespace/lib1.m
--- lib
---- lib-ml-lib3 % lib 3 in version 1.x
----- lib-ml-lib3.prj % adds src folder of lib3 to path
----- src/
------ +my/+otherothernamespace/lib3.m
-- lib-ml-lib2
--- lib-ml-lib2.prj % adds src folder of lib2 to path
--- resources % ML project resources lib2
--- src/
---- +my/+namespace/lib2.m
--- lib
---- lib-ml-lib3 % lib 3 in version 2.x
----- lib-ml-lib3.prj % adds src folder of lib3 to path
----- src/
------ +my/+otherothernamespace/lib3.m
Some questions
- Is the first found function in the pathtool used or the last one?
- how do referenced projects work, does every project have its own path or does everything go into a global one?
- how to explain that which("my.otherothernamespace.lib3", "-all") from the myApp (which does not directly depend on lib3) level finds the lib in the lib-ml-lib2 folder, i.e. in version 2.x but the profiler shows that lib-ml-lib2 uses the lib3 in lib-ml-lib1 folder (!), i.e. in version 1.x although its local lib folder contains the lib3 in version 2.x?
- What whould you do to solve that? It's not possible to avoid nested libs if I want to do proper dependency control
Thanks!
Jan
0 Comments
Answers (1)
Rishi
on 28 Feb 2024
Hi Jan,
I understand from your question that you want explanations regarding function precedence in MATLAB search path. Below are the answer to your questions:
1. The first file or function found in the pathtool is used. To verify this, you can use the 'which' function. In the MATLAB Command Winodw, type the following:
which filename
This returns the file which will be used. You can learn more about it from the given documentation:
2. Referenced projects in MATLAB add their paths to the global MATLAB path. This path is shared across the entire MATLAB environment, which means that all projects contribute to and use the same global path.
3. The 'which' function, when used with '-all' flag, prints out all of the paths of all item with the requested name. Hence, the below line will show all instances of the function 'my.otherothernamespace.lib3' currently on path:
which("my.otherothernamespace.lib3", "-all")
More about it can be read from the below documentation:
The Profiler shows that 'lib-ml-lib2' uses the 'lib3' in 'lib-ml-lib1' folder because this version of the file is higher up on the MATLAB path than the 'lib-ml-lib2' folder.
4. Here are a couple of possible solutions to this:
Version Control within Namespaces: Ensure that each version of lib3 is placed in a different namespace corresponding to its version. This way, lib1 and lib2 can explicitly reference the correct version of lib3.
Project Initialization Scripts: Write custom initialization scripts for each project that set up the path correctly by removing any conflicting paths and then adding the required version of lib3. This ensures that the correct version is used during the project's execution. You can do this using the functions 'addpath' and 'rmpath'. You can learn more about these functions from the below documentations:
Hope this helps!
3 Comments
Rishi
on 28 Feb 2024
Edited: Rishi
on 28 Feb 2024
You can create a wrapper function in which you can pass in the namespace as an argument, and then use the 'eval' function to evaluate the function. You can learn more about the 'eval' function from the below documentation:
I have implemented the code below:
function output = libraryWrapper(version,inputArg1,inputArg2)
evalString = ['my.', version, '.lib3(', num2str(inputArg1),',', num2str(inputArg2), ');'];
output = eval(evalString);
end
My folders are as follows:
Main
- Sub1
-- +my
--- +ns
---- lib3.m
- Sub2
-- +my
--- +ns1
---- lib3.m
In this sample implementation, the 'lib3' function in 'ns' adds two numbers, whereas 'lib3' in 'ns1' subtracts them.
I have attached the output for both of the cases.
While publishing your libraries as MATLAB toolbox will directly solve your issue as toolboxes are added to the global path, but it can be helpful in packaging, versioning and ease of distribution.
See Also
Categories
Find more on Source Control in Projects in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!