Arch Linux public upload server
Hey all! As some of you might already know, I maintain an upload server project called `rustypaste`: https://archlinux.org/packages/extra/x86_64/rustypaste/ I’ve been using it daily for a couple of years now and have been quite happy with it. I used to provide a public instance for free use, but due to some issues with the provider, I had to shut it down: https://github.com/orhun/rustypaste/issues/326 The other day, I started wondering why Arch Linux shouldn't support a similar public service, like https://paste.ubuntu.com, but powered by rustypaste. I believe we could configure rustypaste to allow public uploads with a retention period and necessary security measures in place. I was going to bring this up with the DevOps team, but I thought it might be better to get general opinions on the matter first. 🐻 Cheers, Orhun
On 2024-09-01 11:56:30 (+0300), Orhun Parmaksiz wrote:
The other day, I started wondering why Arch Linux shouldn't support a similar public service, like https://paste.ubuntu.com, but powered by rustypaste. I believe we could configure rustypaste to allow public uploads with a retention period and necessary security measures in place.
Sounds like a great idea! I'm all for it! Best, David -- https://sleepmap.de
On Sun, Sep 01, 2024 at 11:56:30AM GMT, Orhun Parmaksiz wrote:
Hey all!
As some of you might already know, I maintain an upload server project called `rustypaste`: https://archlinux.org/packages/extra/x86_64/rustypaste/
I’ve been using it daily for a couple of years now and have been quite happy with it.
I used to provide a public instance for free use, but due to some issues with the provider, I had to shut it down: https://github.com/orhun/rustypaste/issues/326
The other day, I started wondering why Arch Linux shouldn't support a similar public service, like https://paste.ubuntu.com, but powered by rustypaste. I believe we could configure rustypaste to allow public uploads with a retention period and necessary security measures in place.
I think it's important to specify how the moderation capabilities are. Public paste services are *bounds* to be abused and can distribute malicious files and illegal files. This needs to be dealt with and should probably not cause too much friction. Other then the above issue I'm positive. -- Morten Linderud PGP: 9C02FF419FECBE16
Sep 1, 2024 11:49:46 Morten Linderud <foxboron@archlinux.org>:
I think it's important to specify how the moderation capabilities are. Public paste services are *bounds* to be abused and can distribute malicious files and illegal files. This needs to be dealt with and should probably not cause too much friction.
This is also my main concern, both in terms of users as well as putting more burden on devops to comply with official complaint requests that come in through the hosting provider. So if we want to see something like this, I strongly recommend we put in some measures. I'd like to see a proper moderation tool, as well as a native way to report a violation, so it's easy to moderate. On top, I'd like to see this being connected to our keycloak via OIDC as an identity provider, much like Ubuntu One does, not allowing arbitrary unauthenticated use. Massively limit the maximum upload size to something like 1MB, or even less, would also be advised, we really shouldn't be a file hosting provider, but an Arch text paste service if at all. It's certainly helpful and nice to have a paste service to quickly share a config, snippet, error log or similar, but I'm very concerned about the potential of misuse beyond this. I understand this all may go way beyond a simple "let's host rustypaste" idea, I don't really want to exhaust our devops team even more with such tasks, and we also had reasons to lock down our hedgedoc to a staff only service. Sincerely Levente
On 01-09-2024 12:26, Levente Polyak wrote:
Sep 1, 2024 11:49:46 Morten Linderud <foxboron@archlinux.org>:
I think it's important to specify how the moderation capabilities are. Public paste services are *bounds* to be abused and can distribute malicious files and illegal files. This needs to be dealt with and should probably not cause too much friction.
This is also my main concern, both in terms of users as well as putting more burden on devops to comply with official complaint requests that come in through the hosting provider.
So if we want to see something like this, I strongly recommend we put in some measures. I'd like to see a proper moderation tool, as well as a native way to report a violation, so it's easy to moderate. On top, I'd like to see this being connected to our keycloak via OIDC as an identity provider, much like Ubuntu One does, not allowing arbitrary unauthenticated use. Massively limit the maximum upload size to something like 1MB, or even less, would also be advised, we really shouldn't be a file hosting provider, but an Arch text paste service if at all.
It's certainly helpful and nice to have a paste service to quickly share a config, snippet, error log or similar, but I'm very concerned about the potential of misuse beyond this. I understand this all may go way beyond a simple "let's host rustypaste" idea, I don't really want to exhaust our devops team even more with such tasks, and we also had reasons to lock down our hedgedoc to a staff only service.
+1 For users there are plenty of pastebin alternatives, for staff we can use md.archlinux.org to share notes, pastes. Regarding abuse, see for example what 0x0.st does against detecting awful nsfw content https://git.0x0.st/mia/0x0
Thanks, everyone, for the feedback on the matter! The concerns are clearer to me now, and I've given this a bit more thought.
it's important to specify how the moderation capabilities are.
My initial idea was to use a configuration like this (with some parts stripped): ``` [server] max_content_length = "20MB" [paste] random_url = { type = "alphanumeric", length = 6 } mime_blacklist = [ "application/x-dosexec", "etc"] default_expiry = "1h" delete_expired_files = { enabled = true, interval = "1h" } ``` This would mean that, for example, we would allow certain file types with a maximum size of 20MB, and files would be deleted after 1 hour. Additionally, there is a `delete_tokens` option, which makes it possible to delete files remotely.
I'd like to see a proper moderation tool, as well as a native way to report a violation, so it's easy to moderate.
I think a config like above would be plausable for **quickly** sharing files, which is what `rustypaste` is designed for. If we go beyond that, we might need a more complex service, which would require more effort to set up and maintain.
On top, I'd like to see this being connected to our keycloak via OIDC as an identity provider, much like Ubuntu One does, not allowing arbitrary unauthenticated use.
`rustypaste` does not yet support OIDC: <https://github.com/orhun/rustypaste/issues/286> It was intended to be self-hosted very quickly and easily—and frankly, I've never needed such a feature myself.
I don't really want to exhaust our devops team even more with such tasks, and we also had reasons to lock down our hedgedoc to a staff only service.
I hear you; the legal side of things should be well thought out before proceeding with this. The only thing I'd like to add here is to maybe reconsider the idea with the configuration above. Perhaps using aggressive retention and soft-launching this in the first stage would be a good experiment. I'm all for implementing OIDC logins if we decide to go further with this.
For users there are plenty of pastebin alternatives, for staff we can use md.archlinux.org to share notes, pastes.
Most of the options out there are not quite reliable. The reason I don't use md.archlinux.org is that it feels like too much effort just to share a file. Also, I'm not sure if I can use it in my shell scripts. (For example, I have a function that captures the screen and uploads it to a pastebin service.)
Something public is a pain because unfortunately there are many people who do not respect.
People are people; we can't change that. There are many public services out there, and I would be up for the challenge of maintaining one for Arch Linux.
I agree, too. If it should be only for Arch Linux users, we can add a validation question like the one in the forum's log in page.
That's a good/fun idea :) Let me know what you think!
participants (5)
-
David Runge
-
Jelle van der Waa
-
Levente Polyak
-
Morten Linderud
-
Orhun Parmaksiz