Dealing with signed commits when creating and sharing a Monorepo

January 26, 2023 from Phillip Johnston

A large roadblock on the way to the destination of Development in a monorepo and distribution on independent repositories was created with signed commits. This article reports the problems with which I was confronted, options for treating the problem and my thoughts on this matter. This is still an open examination and I try my ideas and results. I will finally update the article with the final solution to this problem.

Table of contents:

1. background
2. Options to rewrite the story with signed commits
3. First class support in GIT and Filter repo
4. Save and create signatures in commit messages and create it again
5. References

background

The Shopsys/Monorepo tools Repository that I used as the basis for my tools clearly found that it did not work with signed commits. I thought little about it when I started my Monorepo efforts because I didn’t think I used signed commits at all.

I successfully merged my repository together and later (after the move filter-repo) I have successfully shared the story for individual projects. Then I came across a problem with some projects, this gap and push routine worked well. But with others the split process reproduced the same StoryBut every commit had another commit. So when I tried to press, I am (rightly) rejected by the server due to different stories (rightly).

It confused me that the split process for some repositors, but did not work for others. When I analyzed the rewritten history tree after the split process, I found that all proper fortress values ​​were restored and saved. The initial commit had another hash, which led to a “Hash misery”. After a deeper look, this was the case with all problematic repositors.

Why was the initial commitment different? I used signed commits without realizing it. When I created the repository, I had Github Create helpful README And LICENSE Files. And it turns out Ui. Therefore, not only initial commits are problematic, but every combination and changes that are committed directly from the user interface will also break.

Here is the right initial committee object:

(printf"commit %s\0" $(git --no-replace-objects cat-file commit HEAD | wc -c); git cat-file commit HEAD)
commit 661tree f374393bc68231c9107ceeca3550076938407d1a
author Phillip Johnston  1659540832 -0700
committer GitHub  1659540832 -0700
gpgsig -----BEGIN PGP SIGNATURE-----

 wsBcBAABCAAQBQJi6pVgCRBK7hj4Ov3rIwAAQIAIAAZ8wJS81BJuPXWjh63gEk/j
 mP/29+QN8KtJzUzlBJHyUv499DXCITup7DO4b6ww+E1lGDBeNOnv4ZCz2juzn9Jx
 1ySHtLtNsOHcWtwlZ6WLXKS3fXTIMAAYqM4zF2pe5qsegKKn7P24kJipROd9JTX+
 EEFGBZ2fLrHoyhr+YQOBggtapeW5b7eWtVCPueYxdkgbsSQNhV4bpGKdxWUelnG8
 Fj1KSISK84OK5UDc85KQaeUdo2B/DdRSjUzz8/3k5cWo+1hOEC9R0YqJTfgxzizp
 34przZncDJhbQ0lCHkRWIz/IagRwT/wVFVOaQlEF5kkp9XTONggG+vZnBNhDLTE=
 =x4Rx
 -----END PGP SIGNATURE-----


Initial commit%

After a rewriting, these signature information is missing:

(printf"commit %s\0" $(git --no-replace-objects cat-file commit HEAD | wc -c); git cat-file commit HEAD)
commit 188tree f374393bc68231c9107ceeca3550076938407d1a
author Phillip Johnston  1659540832 -0700
committer GitHub  1659540832 -0700

Initial commit%

Options to rewrite the story with signed commits

I researched a few days and hacked and tried to advance this problem. I confirmed that filter-repo or filter-branch actively strip signature information. The reasoning makes sense: we change history and therefore the signatures are invalid because they apply to the original commit. In addition to in our application, we specifically create the signed commit in its original form, and so the content of the signature would be valid with the recovered commit.

From the perspective of the Monorepo, it is not at all about signed commits. It would be okay to pull them off. If we have signed in the Monorepo Commits that have been made via the Github user interface, it is also okay that the signatures are deducted and pushed into the distributed repositorys that do not need to know from them.

However, we need these original signed commits to obtain the distribution of the distribution in the distributed repository. Without signatures, we left some unwanted options:

• Force Push to the Distributed Repositories to reset the story and remove the signed commits
  • This has the advantage of being a unique event, so that in the future we can successfully exceed the sales repository in the future
  • This would break all LIT-specific links in articles on the website, for which a significant investment would be required to remedy it
  • Many of these repository have users, and this would lead to significant problems:
    • Submodule would break, since working commits would no longer be valid
    • Dependency management systems on which a certain commit would break because these commits would no longer be valid
    • Users with the checked -out repositories would have to reset a hard to the main Branch on the server to get updates
• Wear new changes in the original course in the distribution repository
  • We can drive new commits without drama and preserve the signed commits via the original repository
  • All tags are withdrawn from the commits – in the process losing valuable version information and making it impossible to consistently pint the dependency versions
• Reduce the “Force Push Problem” problem by receiving the original branch and committed
  • As a result, existing commit references for sub -module and dependency management tools are valid
  • This would be necessary to avoid that existing links are broken from the website.
  • This increases the size of the repository and makes the story more confusing

I would not hesitate to force a single repository. However, this problem occurs in most of our repository. This makes it an unpleasant way and I want to avoid one.

First class support in GIT and Filter repo

In my research I found a link In the filter repo problem for this problem To a Set of patches for gits fast-export And fast-import Tools This would add more sophisticated options for handling signed commits – one of them would be to conclude the signature literally. I could handle it because I am not really interested in the signature for these commits in Monorepo. The only problem is that these patches have been idle since April 2021 and Elijah Newren confirmed to me that the efforts were dropped.

The first steps in this way must be checked whether this approach would work for me:

1. Fork gitUse the existing patches and create the tool
2. Fork git-filter-repoAdjust the tool to use the new arguments (and add all other necessary support).
3. Make sure that the changes actually do what I want – determine
4. Find out if I can create a method git-filter-repo That would enable me to get signatures without introducing invalid signatures in the Monorepo

At the time of the publication of this article, I successfully created a ProOF-of-Concept with steps 1-4. I can bring stories from several repositories into a Monorepo, force the signatures to be obtained (which are then invalid in Monorepo) and recreate the original commit -hashes during the gap process. I have successfully completed a Proof-of-Concept for step 4 by creating a command line switch that postpones the fixed reversal signatures into a commit message, which can then be restored at a later date. This is described in more detail in the next section.

From here I have three ways forward:

• Stand up for the changes and get them together gitThen git-filter-repo
  • This would enable me to leave my fork after everyone has changed and are released
  • I have to make additional changes to the git Code to support filter-repoExpectations as noted in this topic comment
  • I have to make my necessary changes filter-repo so that they are tasty for general consumption
• Continue to use my fork versions on my build server (and maybe my local development machine)
  • Only the build server would need the fork tools so that it can automatically run the Split operations
  • It is possible that I landed changes in git And git-filter-repoBut not in a way that I use to eliminate my fork (s)
• Decide that it is not worth continuing and choosing a different way forward
  • The email-based patch and feedback river could be a nightmare that interrupts me and prompted me to give up the effort

Ideally, there would be a practicable long-term solution for this problem-it was common enough for people who go a similar path on which I am. This currently seems the most achievable path and the one I actively follow.

Save and create signatures in commit messages and create it again

In The shopsy spend this problemOne of the ShopsS team members states:

I assume that it is theoretically possible to save all data via the signature during the construction of the Monorepo into the CommitMSG and then use this data in order to create exactly the same signed commit during the division. I haven’t examined it yet, but it should be possible to save all raw data of the comite encoded in the commit MSG (It is possible to save 100 MIB there) and restore during the division of the exact commit …

This is a fascinating option. Admittedly, I have absolutely no idea how I can achieve it without changing git or git-filter-repo. Before my storms in git Modifications, I spent a few hours to examine this path, but made no progress.

However, this idea inspired the exploration point 4 in the previous section, which I have now successfully prototyped. Force The preservation of festive signatures is not ideal because these signatures in the rewritten Monorepo history are invalid. So I created a command line flag for git-filter-repo This would enable the storage of original signature information in the commit message. When signatures are moved to a commit message, I name the rename gpgsig Prefix too original_gpgsigAnd it makes it clear that this is an original signature for a rewritten commit and b) these values ​​are easy to find in the future. I then created an additional command line switch that can be used during the gap process. This switch is looking for permanent reports that contain original_gpgsigExtracts from the commit message (restoration of the original value) and the proper restoration of the signature in the committee object.

This path means that I can safely carry out a complete Monorepo reunification process while I maintain signatures. I don’t know filter-repo. But at least I know that I have a practical path forward with a fork filter-repo.

Outdated path

Before my focus on adding the necessary support git I had another idea. It just seemed to be on the surface, but I didn’t invest much time to find out how I can achieve it.

• Create a repository assignment of signed commit-hashes for shared and non-signed hashes
• Copy the content of the signed commit to the right place
• Write down the tree to have a parent of the signed version, not the non -signed version

References

Related