[aur-general] AUR4, git, subtrees ELI5?

[Lukas Fleischer]

On Fri, 12 Jun 2015 at 01:42:41, Eli Schwartz wrote:
> [...]
> That still requires packagers to go through several new hoops.
> [...]

Which ones? Can't think of any apart from generating the key and adding
it to the web interface. It literally takes ~1 minute. You need to do
the same thing after creating an account on a Git hosting platform like
GitHub.

> And I can see the benefit of allowing tarball uploads for the same reason
> that the AUR3 included a web interface for adding new packages (rather than
> having a hard requirement for using a helper).
> e.g. uploading changes from another computer while traveling.
> 
> What happens if the package already exists? Say, the packager switches to a
> new computer.
> 

What's the issue here? `git clone` will clone the existing repository.
Might want to use some options such that it doesn't actually check out
HEAD. Or have another command to pull changes before submitting.

If you work on two machines in parallel, you will get a merge conflict
or a denied push, sure. But that's an inherent issue. When using the
tarball submission process you would overwrite changes from the other
machine which is way worse than something telling you that there is
something wrong.

> Does the tool also setup an ssh key and add that key to the packager's
> account?
> 

No. It also does not register a new AUR account or setup your Internet
connection. It submits packages to the AUR. As I said before, generating
and adding the key is a tiny one-time process and automating it doesn't
pay off.

> [...]
> I would like to see a method for submitting a tarball, having the aurweb
> checkout the package's master branch, overwrite the index with the tarball
> contents,  and commit that with a generic message.
> (e.g. "upgpkg: ${pkgname} ${pkgver}-${pkgrel}").
> 
> Aside from the need to actually spend time implementing that, is there any
> reason not to?
> [...]

Just to name a few issues:

* We would have to reintroduce the Archive::Tar library which we had
  several issues with in the past and were very happy to get rid of.

* You need to be very careful when extracting tarballs. It is quite easy
  to build ZIP bombs. This is one of the reasons we only extracted the
  PKGBUILD (and no other files) when the AUR submissions still required
  uploading source tarballs. What you suggest would require extracting
  everything, though.

* We would have to create checkouts for the Git repositories that are
  submitted via tarballs. We put a lot of time into making the storage
  as space efficient as possible, using a shared object storage with
  gitnamespaces. We can currently store the ~10000 packages uploaded so
  far with <40MB disk usage. Creating checkouts would mean that this
  increases by a factor of ~20. Creating and destroying checkouts on the
  fly is also quite ugly and something I would like to avoid.

* The AUR web interface is written in PHP, the Git backend is written in
  Python. We would either have to duplicate all the sanity checks or
  create some weird interface between the tools that involves reading
  and converting error messages from the standard file descriptors.

Also, I do not understand all the fuss about Git. We don't expect people
to be a Git expert. There are a lot of tutorials and there are detailed
explanations in the Arch wiki. You should be able to submit a package by
only copy-pasting snippets from the wiki without even understanding what
is going on.

If people really refuse to learn the five most basic Git commands, they
better don't maintain any AUR packages. Maintaining packages means that
you are willing to understand basic packaging and VCS tools. Tracking
down issues with a package often involves using the upstream VCS and
nowadays, most projects use Git.

It is expected (and intended) that some (hopefully not too many) AUR
package maintainers back out due to the new system. It is also intended
that a huge number of packages is not imported into the new AUR and
anything that automatically transfers packages is undesirable. We only
want users to resubmit their stuff if they want to take care of it in
the future. Think of it as a huge AUR cleanup.

Regards,
Lukas