Matlab Projects: how to properly reference dependencies with certain version number

Hi all,

we're trying to find out the optimal workflow with Matlab Projects and came to a couple of roadblocks:

• Is there a way to reference a toolbox directly?
• Can its version number be specified?
• Can we specify the path of a toolbox (i.e. having a toolbox/package "storage", where toolboxes are released to)
• Can we reference projects from a GIT/SVN path (+revision obviously)

In fact what we would like to achieve is:

• Moving away from global namespaces (packages) without proper revision information to Matlab Projects, tracked by GIT.
• Share these projects as toolboxes
• Compile a standalone exe from that (Matlab Compiler)

Suppose we have a big application "The App". Right now, The App is part of a global package/namespace hierachy, which is stored on a fileshare everybody has access to. We want to separate The App as a Matlab Project, tracked by Git. Obviously, there are dependencies that do not directly belong to The App (like file helper, generic plotting tools and so on). These tools could be moved into one or several Matlab Projects as well. How do we reference these properly? As far as I've seen, only relative and absolute references to other Matlab Projects are possible. But since every Matlab Project is its own Git repo (which makes sense), the working copies may be "everywhere", i.e. it's very hard to retain a proper directory structure for either relative or absolute paths on all developer machines. How do you solve that?

The next step would be to share these (coupled) Matlab Projects as toolboxes (which shall reference each other again), to ensure not every Matlab user has to open the Matlab Projects first to have all the tools on the path. How can we cross-creference (custom made) toolboxes in a certain version?

You see, we're looking for a "real" package + dependency manager such as nuget (https://www.nuget.org/). There you have packages with a version number that can reference other required packages, again with a certain range of compatible versions. And the best - all dependent packages are automatically installed.

How can we achieve that with Matlab? My coworkers are very surprised that this is not a standard feature in Matlab.

Thank you!

[Andy Campbell]

Hi Jan,

This is a great question and the workflow you are working towards is completely rational and a good approach. Are you using any particular binary repository manager at your company? Something like artifactory or nexus?

Andy

[Jan Kappen]

Dear Andy,

what a cool coincidence, I just commented your great blog article about packaging.

However, no, we are not using a binary repository manager yet and although there are plans this won't happen in the mid term time range.

I also believe that we have to start a bit earlier in the devops chain - although many of the described problems above are about compiled artefacts, the issues already occur in development phase, on source code level. I.e. in python we have pip which can fetch dependencies directly from either a registry (I would love to setup a similar package registry which can be browsed locally in our company) or from an arbitrary git location. And it installs/registers the downloaded stuff right away so that it can be found properly.

With the current Matlab Projects, the user must know 1) where to find the repo for the Project, 2) where to find the dependent/referenced Projects and 3) how to organize the local folder structure, since Matlab Projects can only reference absolute or relative file system paths. I fear this is really not practically usable for all my colleagues.

Do you (or anybody else) have some ideas to push me into the right direction? I already contacted the technical support but they referred to GIT subprojects which is absolutely not the correct way, since the VCS is not responsible for dependency management.

Thank you!

Br, Jan

[Andy Campbell]

Hey Jan,

Sorry for the delay, and I'll look back at the blog comment because it seems I am probably late to answer that too!

I definitely have some thoughts about maybe some things you can do to help set up what hyou would like. If you aren't using a binary repository manager like artifactory, did you have any plans as to how you would set up this regostry you describer for sharing toolboxes with your company? THat might help me figure out a good recommmendation.

Also, this seems like a good thing to blog about.

Thanks,

Andy

[Patrick Braun]

@Andy Campbell Hi Andy and Jan,

If I might slide into this topic, since you have mentioned Artifactory in particular:

Do you have good references/further reading on best-practices for managing MATLAB-apps using the artifactory?

Our current obstacle is that we are uploading new versions of our tools "by hand" via the artifactoy-UI.

We have an open question concerning our problems with setting up scripts that do the compiling and uploading via webwrite, especially the proper encoding of mlappinstall-files.

We haven't found to much on MATLAB/artifactory workflows, so furthere reference is very much appreciated!

Thanks

[Jan Kappen]

Sorry @Patrick Braun I have not. It's still an open topic and it's frustrating.

[Jan Kappen]

@Andy Campbell

we have Artifactory now, but that does not really solve the primary problem. There's no way to relate packages/projects/toolboxes by their version.

I have a GUI that needs a database backend. That exists in two major versions, and both must be maintained. There's no package manager where I could say "GUI depends on v1.0.1" in that branch and on "v2.2.3" on this branch (in our CI pipeline). I have to clone that by hand, relying on the git tags.

Is there something automated coming?

[Jan Kappen]

See also Dependency management & package manager like pip in ML - MATLAB Answers - MATLAB Central (mathworks.com)