GitHub Backend

lorry was designed with Gitlab as a backend in mind. As such, it is not compiled with GitHub support by default; to use GitHub as your mirroring host, you should compile with the github feature flag set. The GitHub backend has some differences in behaviour to the default Gitlab backend, due to GitHub not behaving the same way as Gitlab and thus being unable to support the same operations.

The first of these is that GitHub does not support arbitrary namespacing. This makes the prefix field somewhat dangeous. If you wish to mirror into the lorry user account’s personal namespace, the prefix field should be set to the lorry user account’s username. If you wish to mirror into an Organisation’s namespace prefix should be set to the name of the organisation to mirror into.

Also, GitHub scans commits for leaked secret tokens. This means that repositories containing what looks like a secret token will be rejected, stopping mirroring. If this occurs, you should disable secret scanning on the lorry user account and in the repository you are mirroring into.

Deployment as a GitHub App

Instead of operating as bot user, lorry can instead operate as a GitHub App. This means that the usual username and private-token fields are no longer respected. Instead your should supply the app field in your downstream configuration, which takes the form of:

app = { app-id = 2662472, private-key-file = "../lorry-app.private-key.pem" }

Where the app-id field is the App ID shown in the GitHub App dashboard, and private-key-file is a filepath ponting to the private key file that could downloaded from the dhashboard when you created the App. There is no way to supply the private key as a literal or an environment variable.

This requires that the App be installed in all namespaces that lorry is expected to mirror into. The App must be given the Administration: Read and Write, and Contents: Read and Write permissions.

Similairly, one can authenticate as an app when pulling in the confgit repo. The same configuration as above applies, except that the top-level field is called github-app. Again, once must make sure the app is installed in the confgit repo.

Manually testing the backend

Unfortuanetely, there currently is no automated test suite to test that the GitHub backend works, mostly due to the fact that we cannot spin up a local instance of GitHub. Instead, developers have to manually test the backend. To test this, you will need to create an Organisation to mirror into.To test PAT authenitcation, you will need a GitHub account for lorry to authenticate as, and to test GitHub App authentication, you will need an App to use. This App needs to be installed into the Organisation, with the permissions described above. Finally, you will need a confgit repository that mirrors some git repo, pointed at the Organisation. When testing PAT authentication, the confgit should also point at the lorry user’s personal namespace.

Then, run lorry2 locally with the required config for the authentication mechanism you are testing. Within 2 runs, you should see the repositories mirrored into the Organisation. If you are testing PAT authentication, then you should also see the repositoires mirrored into the lorry2 user’s account. You should then delete the repositories once you’re done, so the next round of testing excercises repository creation.

Note that the GitHub backend has two limitations that affect which repositories you use for testing. First, the GitHub backend does not attempt to set up LFS, so you should avoid mirroring a repository that uses it. Secondly, as mentioned earlier, GitHub automatically scans commits for content that looks like secret tokens, and rejects pushes containing them. If your mirrored repository contains such data (for example, this very repository, in the test cases for censoring such tokens from logs), the test will fail. Either choose an upstream that contains nothing of the sort, exclude branches containing such content with the ref_patterns field, or disable secret scanning. The latter option is limited in that you can only set it on the repo- or Organisation- level, which means you will have to manually go into the settings of the repository created in the user namespace while the test is running for the mirror to succeed.