On 02/09/2016 12:30 AM, Damian Nowak wrote:

...

if I understand the offering of Amazons cloud service correctly, there, you can install an OS, say arch, on a virtualized machine and scale CPU, RAM, etc. freely up and down just as you need it. Well, yes and no. You can scale up resources (e.g. increase RAM) but this requires a full restart of your virtual machine. See: https://serverfault.com/questions/591533/can-ec2-instances-dynamically-add-r...

Moreover, adding more and more resources will stop working at some point. That's why you'd be looking to add several independent virtual machine running your application (Amazon EC2), which connect to one big database (another Amazon EC2), and put all of those computing machines in from of a load balancer (Amazon ELB).

...

While I can to this using e.g. KVM+qemu on a single machine, I want to be able to bind together a bunch of machines such that they appear as a single big machine. Is there a way to do this?

You can achieve it with both solutions, is it Amazon (EC2+ELB), or your own dedicated server(s) where you'll be likely to use KVM for virtualization and HAProxy, nginx or other solutions for load balancing.

From a technological point of view, both ways are the same.

Thanks for your answer. I still try to wrap my head around the topic, so some of my assumptions my not hold. To me it sounds, that the proposed approach only works for web-apps running in a http-server, since the load balancing is done via HAProxy and nginx. For my needs, I want to run "usual" software, specifically R, the statistics language. Utlimately, I want to bind several physical hosts together to appear as one host on OS level, such that e.g. htop would show the total number of cores accross all bound boxes.

On 02/09/2016 09:23 AM, Patrick Burroughs (Celti) wrote:

On Tue, 9 Feb 2016 09:13:28 +0100 Wolfgang Mader wrote:

...

[snip] For my needs, I want to run "usual" software, specifically R, the statistics language. Utlimately, I want to bind several physical hosts together to appear as one host on OS level, such that e.g. htop would show the total number of cores accross all bound boxes. What you're describing is a classic clustering environment, like a Beowulf cluster. Look into something like LinuxPMI/OpenMosix. It'll never be as smooth as "a single OS on a single logical host across multiple machines", but it's as close as you'll get.

~Celti This sounds like the thing I am after.

Much apprechiated!