[arch-dev-public] GitLab switchover and SSO migration

[Lukas Fleischer]

On Sun, 17 May 2020 at 14:39:25, Sven-Hendrik Haase via arch-dev-public wrote:
> As some of you know, we've been toying with two ideas for a while:
> Arch-wide centralized user management as well as using GitLab to
> consolidate some of our current services. The overall goal is manifold.
> In no particular order, the goals are to
> - make Arch more contributor friendly
> - provide more modern tools for ourselves
> - enable more automation
> - make Arch services more secure
> - make team management activities less error-prone and more streamlined

Great! Thanks a lot for working on this!

> After looking at various solutions, we eventually settled on Keycloak
> since it seemed like a modern, well-maintained, and secure piece of
> software. It allows us to enable logins for services via OpenID Connect
> and SAML which is likely the best coverage we could hope for. It also
> allows us to connect other social login providers such as github.com and
> gitlab.com and it supports Recaptcha, 2FA, and WebAuthn out of the box.
> The idea is to eventually transition all our online service as well as
> SSH keys to Keycloak to ease on-/offboarding and make it less error-prone.

Is 2FA going to be opt-in? Will it be mandatory for members of the
staff? You mentioned increased security before but having a single
password also increases the attack surface in case a password is stolen.

> 1) Transition as many staff-only services to Keycloak as possible. We
> looked at our current services and put up a table on the wiki that shows
> support per service [2].
> Some of the services that we operate are deprecated or have
> functionality that is also provided by GitLab. In our current
> understanding, this concerns Flyspray, Kanboard, and Patchwork. Of
> those, Kanboard and some of the Flyspray projects will be our first
> targets to transition to GitLab. We'll continue using Flyspray for the
> time being for package bugs but will discontinue its use for all
> non-packaging bugs. The reason for this is that how we manage our bugs
> for packages is somewhat intertwined with the svn2git migration which is
> yet to be done and might dictate a different repo structure than what we
> would come up with currently and we don't want to block on this. This
> was also discussed in a recent DevOps meeting [3].

When saying "svn2git migration", are you referring to the already
existing svntogit repositories on git.archlinux.org or to the migration
of our main package VCS to Git?

Could you elaborate on how svn2git migration and Flyspray are
intertwined? I could not find any details in [3].

> 2) We'd like to get rid of our own cgit instance at git.archlinux.org
> and transition our git hosting into GitLab. AUR git access will stay as
> it is due to its special shell magic.

You may already be aware of this but I'd like to clarify, especially
since the DevOps meeting notes [3] mention that "git.archlinux.org
likely needs to be kept for svn2git and the AUR": the cgit instance for
AUR packages is entirely separate from the cgit instance running
git.archlinux.org --- shutting down git.archlinux.org should not impact
aur.archlinux.org in any way.

> We tried hard to come up with a good source of users to import into
> Keycloak so that we could seed that database with a solid user base but
> sadly it appears that there is no trusted source of users that we can
> rely on. Potential candidates were the wiki, BBS, AUR but we ruled them
> all out in their current state as none of them have always had email
> verification and so we can't trust those emails to be the sole source of
> truth. In order to still allow users to keep their old contributions in
> cases where they can prove their identity via email, we'll build a new
> small web application that allows them to connect their new Keycloak
> identity to their other identities. For now, we seeded the Keycloak
> database with the only known-good source of trusted emails: Staff from
> Archweb.

Email addresses of aurweb accounts have to be confirmed (and accounts
without verified email addresses are not usable and can be filtered out
by a simple SQL query). Old accounts have been purged in ~2014 and, to
the best of my knowledge, there should not be any active accounts left
that did not go through the email verification process.

> On more goal we had is automatic github.com mirroring in some fashion.
> We looked at creating a two-way github.com <-> GitLab mirror but that
> setup can break easily in the case of force pushes and race conditions
> and also would have us looking at both places for pull requests. It
> seems simpler to us for the time being to have one-way mirroring from
> our GitLab to github.com only and then allow github.com users to easily
> collaborate on our GitLab via github.com social login. It's a little bit
> more hassle for the users than collaborating directly on github.com but
> it's a lot less hassle for us so it's perhaps the best compromise.

One argument for users to prefer GitHub is that many already have an
account there. Ceasing GitHub support might have an impact on engagement
and make it less likely for one-time contributors to submit a simple
patch. Maybe that's a sacrifice we're willing to make, though.

On a related note, will this impact projects that prefer email patch
submissions in any way (except that they can now opt-in to GitLab too)?

Best,
Lukas

> [0] https://gitlab.com/gitlab-com/marketing/community-relations/gitlab-oss
> [1]
> https://gitlab.com/gitlab-com/marketing/community-relations/gitlab-oss/-/commit/9c40905b9ceef41628d9a3ebbf06060060bc1384
> [2] https://wiki.archlinux.org/index.php/DeveloperWiki:SSOMigration
> [3]
> https://wiki.archlinux.org/index.php/DeveloperWiki:DevopsMeetings/2020-05-06