[pacman-dev] Fw: Pacman support for IPFS

[@RubenKelevra]

This message was originally send to Allan, who recommended to move the
discussion to the pacman-dev mailing list:



I tested a bit around and setup a mirror (via a cluster-configuration)
in IPFS for the packages - but it's kind of ugly.

I was wondering if there's interest from you or the team to implement
native IPFS support in pacman?

The idea is, to extend the current database definition, by an
additional field, which holds the Content-ID necessary to fetch a
package from IPFS.

If there's an API-endpoint configured in the pacman config, pacman will
query the IPFS API to resolve a IPNS-public key[0].

The public key will lead to a folder listing, which holds the latest
databases, like this path[1] does.

The Content-ID of each database could be saved as symbolic link in the
db-patch. If one of the Content-IDs changed, the new version is fetched.

With the Content-IDs the updated packages can be fetched from the
network.

So if there's no API-endpoint configured, the feature wont be active.

If it's been activated, pacman can fetch packages much quicker, since
IPFS can fetch from multiple sources, like BitTorrent and will also
connect to other peers on the local network via mDNS. That's especially
interesting if you run a large amount of computers and currently need a
rsync mirror server or a cache in the network or you're a home user with
extremely limited bandwidth.

---

To maintain a reliably redundancy of packages there's a need for a
cluster installation. A IPFS cluster[2] is basically just a small daemon
which runs and holds a database which Content-IDs should be maintained
on which cluster peer.

So if a package gets added to the database, the Content-ID would be
added to the dataset of the cluster and the Content-ID of the old
package get's removed.

And the new version of the folder with the databases would be replaced
as well.

The cluster definition isn't static, everybody can join the cluster and
can hold the cluster data. There are some examples of clusters which
are open for copies here[3].

You can set on each element how many copies the cluster should hold,
at maximum and minimum. The cluster will start to replicate the maximum
amount and if the redundancy drops below minimum additional copies will
be made.

You can also specify -1 which means every cluster member will hold a
copy - this makes sense for example for the databases.

---

How to add new package versions:

A package maintainer would do the normal sha256/md5/signing and
additionally add the file to a IPFS client (for example on a server) or
his computer. The Content-ID generated while adding the file to IPFS.
The content ID will be added to the database definition.

The new database file is then added to to IPFS as well, and the folder
of the current database listing (in IPFS) is altered to reference the
new Content-ID of the database file.

This will alter the Content-ID of the folder, since the references has
changed.

The new Content-ID of the folder and the package then get added to the
cluster, which will fetch both new files from the computer of the
maintainer or the server where they was added.

Now the IPNS-Record can be switched to the new Content-ID of the
folder, any everyone can fetch the new database.




[0] https://docs-beta.ipfs.io/concepts/ipns/
[1] https://ipfs.io/ipns/pkg.pacman.store/arch/x86_64/default/db
[2] https://cluster.ipfs.io/
[3] https://collab.ipfscluster.io/

Best regards,


Ruben


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-encrypted
Size: 11 bytes
Desc: not available
URL: <https://lists.archlinux.org/pipermail/pacman-dev/attachments/20200405/f933b4b0/attachment.pgp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/octet-stream
Size: 3332 bytes
Desc: not available
URL: <https://lists.archlinux.org/pipermail/pacman-dev/attachments/20200405/f933b4b0/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 228 bytes
Desc: Digitale Signatur von OpenPGP
URL: <https://lists.archlinux.org/pipermail/pacman-dev/attachments/20200405/f933b4b0/attachment.sig>