Meson pattern: Monorepo, which supports individual subproject builds

June 5, 2023 from Phillip Johnston

As mentioned in the article, Develop in a Monorepo and spread out on independent repositories Develop in a Monorepo and spread out on independent repositoriesWe have summarized many of our repos in a Monorepo setup, but we will continue to distribute changes to the existing independent repository.

To support this model, we have the following requirements for the Build the system:

• We have to be able to build the Monorepo as a whole.
• We have to be able to set up sub -projects within the Monorepo individually.

There are some challenges to make this smoothly Our preferred construction system, Meson. In this article I will describe the challenges, my solution and possible alternatives.

Table of contents:

1. The subproject challenge
2. Structural support
3. Symlinks probably also work here too
4. OMG just use CMAKE
5. References

The subproject challenge

In “distributed mode”, our various repository are connected to each other via Meson’s subprojekt system. Each repository contains a sentence of Wrap files This describes which external dependencies are required and how you get. When you configure the project, Meson picks up the necessary external dependencies.

Since the existing projects are already working through the subproject mechanism, the obvious route is to the entire source code in the subprojects/ Directory (Meson’s standard storage location for storing Wrap files and sub -project artifacts).

There is only one problem: Meson demands that sub -projects be kept in a flat directory structure. Absolutely NO Organization is allowed. If you try to call up a sub -project that is stored in a subdirectory inside subprojects/You see a sandpit injury error or a wrap error (e.g. “error: directory key must be a name and no path”).

We have more than 45 repositories in this Monorepo, and this number will increase over time. A flat structure is not appealing.

Structural support

If a pure subproject approach is excluded, we have to rethink how builds are structured.

The basic requirement is that we have to deal with every sub -project project() Definition and access only to the various build goals provided by the sub -project. A naive idea would be to have the top level of a sub -project meson.build File contain the project() Definition and the standard -Build infrastructure. We could have a special folder (e.g., e.g. build/) that would contain a meson.build File that processes the subproject-specific build setup. The monorepo building of the top level could then include <subproject>/build/meson.buildevade project() Call the infrastructure setup and double. The independent build would also contain this file.

There is only one problem with this approach: Meson’s subdir() You can only functions descend A tree. You can use relative paths (e.g., e.g. subdir('../src') Throws a mistake).

Apart from that, the core idea is still solid. If we create in Monorepo mode, we will handle the top list of the sub-project and instead directly to the <subproject>/src/ And <subproject>/test/ Lists to achieve the relevant goals.

In detail here is the pattern that we have come up with:

• The top monorepo meson.build File must specify:
  • Monorepo project() Call
  • Access to the required building infrastructure
  • Access to external subprojekect dependencies that are required, any Unterproject in the entire Monorepo tree
• The sub -project meson.build File must specify:
  • Sub -project project() Call
  • Access to the required building infrastructure
  • Access to dependencies that are relevant for this specific sub -project.
• The sub -project meson.build File must not Define all build goals or dependencies that are exported by the sub -project.
  • Instead, all goals in the sub -project must be defined src/Present test/And docs/ Directory (or others that are relevant).
  • As usual, the sub -project meson.build The file will contain these directories about the directories subdir() Calls.
• The Monorepo meson.build Files bypass the subproject root directories. Instead, they are included directly:
  • subdir('<subproject>/src')
  • `Subdir (‘/test’)
  • subdir('<subproject>/docs')

At the time of the publication of this article, the pattern was proven with the EMBVM-Core Project in Monorepo mode. We are now working on the restructuring of the build rules for the other projects.

Symlinks probably also work here too

Like in Dealing with double files in a monorepoWe already rely on Symlinks to eliminate duplication and to rationalize Monorepo work. The sub -project challenge could probably also be addressed by Symlinks.

The source code could be organized properly outside subprojects/ Folder while becoming a flat structure within the subproject folder.

I didn’t try that when I first developed another idea (and I like it more). But I think that it is worth mentioning as a simple solution to the problem with minimal necessary rework.

OMG just use CMAKE

When I talked to other developers about the challenges described above, I am asked regularly why I don’t leave Meson and use something like CMAKE that is not in my way.

The reason is that I love Meson so much. I have used so many build systems over the years, including CMAKE (finally we Sell ​​a CMake course). Meson, not CMAKE, that is closely agreed with my own opinions on how Build systems should work. One point of the interior is not enough to send me somewhere else, especially since there are simple problems that I can use.

References

Related