[arch-general] How to cache pacman mirror for LAN
Hello, I'm building a computer cluster with about 20 nodes, all of them running ArchLinux. One of them is the 'login' node connected to the Internet, other nodes share Internet connection via the login node being a router. https://wiki.archlinux.org/index.php/Internet_Share Now, when I update the system (pacman -Syu) or install package on all node, I think it's a bad idea because it will consume the mirror bandwidth proportional to the number of the nodes. Maybe it's not a big issue for 20 nodes, but I'd like to learn nevertheless for future use, that: Is there a way to cache the pacman transaction at the login node, so that the communication between the login node and the mirror is constant, and the rest of communications only take place within the LAN? Is seting up a pacman mirror at the login node is the correct solution? Best, Takayuki Muranushi
On Sat, Jun 4, 2011 at 5:32 PM, Takayuki Muranushi <muranushi@gmail.com> wrote:
Hello,
I'm building a computer cluster with about 20 nodes, all of them running ArchLinux. One of them is the 'login' node connected to the Internet, other nodes share Internet connection via the login node being a router. https://wiki.archlinux.org/index.php/Internet_Share
Now, when I update the system (pacman -Syu) or install package on all node, I think it's a bad idea because it will consume the mirror bandwidth proportional to the number of the nodes. Maybe it's not a big issue for 20 nodes, but I'd like to learn nevertheless for future use, that:
Is there a way to cache the pacman transaction at the login node, so that the communication between the login node and the mirror is constant, and the rest of communications only take place within the LAN? Is seting up a pacman mirror at the login node is the correct solution?
Best,
Takayuki Muranushi
Haven't used it myself, but this [1] might do the trick. [1] http://xyne.archlinux.ca/projects/pacserve/
You can also share the pacman directory /var/cache/pacman/pkg via NFS or some other protocol. Whenever a new package is requested, it gets downloaded to your login node which stores it for later use in case another client needs it. The drawback is, that you can't upgrade two machines at the same time, because they would interfere each other. Have a look at: https://wiki.archlinux.org/index.php/Network_Shared_Pacman_Cache#Network_sha... 2011/6/5 Gary Wright <wriggary@gmail.com>
On Sat, Jun 4, 2011 at 5:32 PM, Takayuki Muranushi <muranushi@gmail.com> wrote:
Hello,
I'm building a computer cluster with about 20 nodes, all of them running ArchLinux. One of them is the 'login' node connected to the Internet, other nodes share Internet connection via the login node being a router. https://wiki.archlinux.org/index.php/Internet_Share
Now, when I update the system (pacman -Syu) or install package on all node, I think it's a bad idea because it will consume the mirror bandwidth proportional to the number of the nodes. Maybe it's not a big issue for 20 nodes, but I'd like to learn nevertheless for future use, that:
Is there a way to cache the pacman transaction at the login node, so that the communication between the login node and the mirror is constant, and the rest of communications only take place within the LAN? Is seting up a pacman mirror at the login node is the correct solution?
Best,
Takayuki Muranushi
Haven't used it myself, but this [1] might do the trick.
Thank you Gary, Simon. Pacserve will be best solution if it works. I'll give it a try. Also, NFS solution is OK because most of the time I'll run the update automatically and I'll write a script to handle the timing. 2011/6/5 Simon Schneider <schneida.simon@gmail.com>:
You can also share the pacman directory /var/cache/pacman/pkg via NFS or some other protocol. Whenever a new package is requested, it gets downloaded to your login node which stores it for later use in case another client needs it. The drawback is, that you can't upgrade two machines at the same time, because they would interfere each other. Have a look at: https://wiki.archlinux.org/index.php/Network_Shared_Pacman_Cache#Network_sha...
2011/6/5 Gary Wright <wriggary@gmail.com>
On Sat, Jun 4, 2011 at 5:32 PM, Takayuki Muranushi <muranushi@gmail.com> wrote:
Hello,
I'm building a computer cluster with about 20 nodes, all of them running ArchLinux. One of them is the 'login' node connected to the Internet, other nodes share Internet connection via the login node being a router. https://wiki.archlinux.org/index.php/Internet_Share
Now, when I update the system (pacman -Syu) or install package on all node, I think it's a bad idea because it will consume the mirror bandwidth proportional to the number of the nodes. Maybe it's not a big issue for 20 nodes, but I'd like to learn nevertheless for future use, that:
Is there a way to cache the pacman transaction at the login node, so that the communication between the login node and the mirror is constant, and the rest of communications only take place within the LAN? Is seting up a pacman mirror at the login node is the correct solution?
Best,
Takayuki Muranushi
Haven't used it myself, but this [1] might do the trick.
I use apt-cacher-ng configured for archlinux repositories :) http://aur.archlinux.org/packages.php?ID=40662 https://bbs.archlinux.org/viewtopic.php?id=120068
I use a central server for simply caching with NFS (as in the wiki basicly) for five machines at home. And it works great and not having to add other packages just feels more simple in my imho. Was unaware about not being able to use the cache simultaneously though and has never been an issue for me. Will have to check that out, thanks for the info! If you were to write a script for that maybe it could be handy to put in the wiki? Cheers, Henrik Andersson 2011/6/5 Takayuki Muranushi <muranushi@gmail.com>:
Thank you Gary, Simon. Pacserve will be best solution if it works. I'll give it a try. Also, NFS solution is OK because most of the time I'll run the update automatically and I'll write a script to handle the timing.
2011/6/5 Simon Schneider <schneida.simon@gmail.com>:
You can also share the pacman directory /var/cache/pacman/pkg via NFS or some other protocol. Whenever a new package is requested, it gets downloaded to your login node which stores it for later use in case another client needs it. The drawback is, that you can't upgrade two machines at the same time, because they would interfere each other. Have a look at: https://wiki.archlinux.org/index.php/Network_Shared_Pacman_Cache#Network_sha...
2011/6/5 Gary Wright <wriggary@gmail.com>
On Sat, Jun 4, 2011 at 5:32 PM, Takayuki Muranushi <muranushi@gmail.com> wrote:
Hello,
I'm building a computer cluster with about 20 nodes, all of them running ArchLinux. One of them is the 'login' node connected to the Internet, other nodes share Internet connection via the login node being a router.
https://wiki.archlinux.org/index.php/Internet_Share
Now, when I update the system (pacman -Syu) or install package on all node, I think it's a bad idea because it will consume the mirror bandwidth proportional to the number of the nodes. Maybe it's not a big issue for 20 nodes, but I'd like to learn nevertheless for future use, that:
Is there a way to cache the pacman transaction at the login node, so that the communication between the login node and the mirror is constant, and the rest of communications only take place within the LAN? Is seting up a pacman mirror at the login node is the correct solution?
Best,
Takayuki Muranushi
Haven't used it myself, but this [1] might do the trick.
participants (5)
-
Gary Wright
-
Henrik Andersson
-
Hyacinthe Cartiaux
-
Simon Schneider
-
Takayuki Muranushi