[arch-general] Virtualization accross hardware border
Dear list, if I understand the offering of Amazons cloud service correctly, there, you can install an OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and down just as you need it. While I can to this using e.g. KVM+qemu on a single machine, I want to be able to bind together a bunch of machines such that they appear as a single big machine. Is there a way to do this? I my question clear enought to be answered? Thanks, Wolfgang
You might want to look into docker: https://www.docker.com/ On Mon, Feb 8, 2016, 12:59 AM Wolfgang Mader <Wolfgang_Mader@brain-frog.de> wrote:
Dear list,
if I understand the offering of Amazons cloud service correctly, there, you can install an OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and down just as you need it. While I can to this using e.g. KVM+qemu on a single machine, I want to be able to bind together a bunch of machines such that they appear as a single big machine. Is there a way to do this? I my question clear enought to be answered?
Thanks, Wolfgang
if I understand the offering of Amazons cloud service correctly, there, you can install an OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and down just as you need it.
Well, yes and no. You can scale up resources (e.g. increase RAM) but this requires a full restart of your virtual machine. See: https://serverfault.com/questions/591533/can-ec2-instances-dynamically-add-r... Moreover, adding more and more resources will stop working at some point. That's why you'd be looking to add several independent virtual machine running your application (Amazon EC2), which connect to one big database (another Amazon EC2), and put all of those computing machines in from of a load balancer (Amazon ELB).
While I can to this using e.g. KVM+qemu on a single machine, I want to be able to bind together a bunch of machines such that they appear as a single big machine. Is there a way to do this?
You can achieve it with both solutions, is it Amazon (EC2+ELB), or your own dedicated server(s) where you'll be likely to use KVM for virtualization and HAProxy, nginx or other solutions for load balancing.
From a technological point of view, both ways are the same.
-- Damian Nowak CEO, Virtkick www.virtkick.com
On 02/09/2016 12:30 AM, Damian Nowak wrote:
if I understand the offering of Amazons cloud service correctly, there, you can install an OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and down just as you need it. Well, yes and no. You can scale up resources (e.g. increase RAM) but this requires a full restart of your virtual machine. See: https://serverfault.com/questions/591533/can-ec2-instances-dynamically-add-r...
Moreover, adding more and more resources will stop working at some point. That's why you'd be looking to add several independent virtual machine running your application (Amazon EC2), which connect to one big database (another Amazon EC2), and put all of those computing machines in from of a load balancer (Amazon ELB).
While I can to this using e.g. KVM+qemu on a single machine, I want to be able to bind together a bunch of machines such that they appear as a single big machine. Is there a way to do this?
You can achieve it with both solutions, is it Amazon (EC2+ELB), or your own dedicated server(s) where you'll be likely to use KVM for virtualization and HAProxy, nginx or other solutions for load balancing.
From a technological point of view, both ways are the same.
Thanks for your answer. I still try to wrap my head around the topic, so some of my assumptions my not hold. To me it sounds, that the proposed approach only works for web-apps running in a http-server, since the load balancing is done via HAProxy and nginx. For my needs, I want to run "usual" software, specifically R, the statistics language. Utlimately, I want to bind several physical hosts together to appear as one host on OS level, such that e.g. htop would show the total number of cores accross all bound boxes.
On Tue, 9 Feb 2016 09:13:28 +0100 Wolfgang Mader <Wolfgang_Mader@brain-frog.de> wrote:
[snip] For my needs, I want to run "usual" software, specifically R, the statistics language. Utlimately, I want to bind several physical hosts together to appear as one host on OS level, such that e.g. htop would show the total number of cores accross all bound boxes.
What you're describing is a classic clustering environment, like a Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll never be as smooth as "a single OS on a single logical host across multiple machines", but it's as close as you'll get. ~Celti
On 02/09/2016 09:23 AM, Patrick Burroughs (Celti) wrote:
On Tue, 9 Feb 2016 09:13:28 +0100 Wolfgang Mader <Wolfgang_Mader@brain-frog.de> wrote:
[snip] For my needs, I want to run "usual" software, specifically R, the statistics language. Utlimately, I want to bind several physical hosts together to appear as one host on OS level, such that e.g. htop would show the total number of cores accross all bound boxes. What you're describing is a classic clustering environment, like a Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll never be as smooth as "a single OS on a single logical host across multiple machines", but it's as close as you'll get.
~Celti This sounds like the thing I am after.
Much apprechiated!
For my needs, I want to run "usual" software, specifically R, the statistics language. Utlimately, I want to bind several physical hosts together to appear as one host on OS level, such that e.g. htop would show the total number of cores accross all bound boxes.
There are often optimized distributed solutions for specific tasks. So if your goal is to run R across multiple machines, check out something like http://www.distributedr.org Chris -- echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3
participants (5)
-
Alexander Terry
-
Christoph Gysin
-
Damian Nowak
-
Patrick Burroughs (Celti)
-
Wolfgang Mader