Edit: e2 not n2 - https://www.google.com/amp/s/cloudblog.withgoogle.com/produc...

But when you say "the price would vary according to how much RAM or CPU resources you use" you get into the real complexity: resource sharing. If your VM temporarily gives up some RAM, can another VM use that RAM? This is very hard to do, because the provider doesn't know when/if you'll want that RAM back. They don't want a physical server to get into a situation where the RAM demand is higher than the installed RAM because there is no good solution to that scenario. If you're running hundreds of "micro" VMs/containers on one server you can rely on statistical multiplexing and luck, but it doesn't really work for large workloads.

A provider called NearlyFreeSpeech has been charging based on "the use of one gigabyte of RAM for one minute, or the equivalent amount of CPU power" since even before EC2 existed AFAIK, but I suspect this complexity is more scary than attractive for most people. https://www.nearlyfreespeech.net/services/hosting

Consider memory usage -- but operating systems (and some applications) are designed to grab all the memory they can, and use it for caching etc. So it's hard for the VM host to known when it can grab memory and stop billing the user for it.

But there is this idea called memory ballooning -- you have a little process running on the VM guest OS that grabs lots of memory, but is actually in cahoots with the host, and just tells the host -- "hey I got all this memory, you can take it back and use it somewhere else".

Okay, so doesn't ballooning solve the problem? There are a few problems with it -- you can't balloon when you need the memory, because it's not fast enough. So you have to balloon pro-actively. And you don't know how much to balloon, so you have to guess: do it wrong and the guest OS will start swapping, or it might activate its OOM killer and start killing processes.

So making memory usage follow the application is kinda sorta possible but comes with hairy problems. What about CPU? CPUs usage already follows the application so you could just measure and bill accordingly -- except that nothing is gained if memory doesn't also follow usage.

All in all it's way simpler to get towards this goal with clearly-defined higher level services like Lambda.

There are videos of it happening, but no blog post yet.

So running a t3 would allow you to pay for a baseline and then only pay for the CPU you end up needing beyond that baseline.

Doing the same in a VM might be possible (the technologies exist) but it's often the task or workload that needs to be modified to support it and when you are already doing that a step to containers, functions or horizontal scaling is just as easy (or hard). Horizontal scaling based on load is pretty common (classic ASGs but also overcapacity cost bidding based scaling).

It raises some isolation concerns, however. To make this work, the host needs to know how much memory is allocated to a given tenant, and that's difficult without having access to the OS running inside the machine, which is easy with containers, but not so much with VMs. The tenant can turn off virtual CPU cores in the VM and the hypervisor can pick that signal up (at least in Linux) but I'm not sure it's possible right now to do the same to virtual memory modules. I'd love if AWS allowed me to do that because the boot process of some of my workloads is more CPU-bound than the rest of the machine's lifetime and having a bunch of extra cores at boot would do wonders. If, under memory pressure, I could "plug in" more virtual memory modules, it'd also be quite nice.

There could be a market opportunity, but whoever does it would need to beat the current incumbents in price or this would not fly. Also there would be some difficulty scaling up - all these extra CPUs and memory would need to come from somewhere and would necessarily fail (or need to trigger a live VM migration, which, IIRC, nobody does in cloud environments right now) if the host box's resources are fully allocated.

Complications: a lot of oses and some programs will tune themselves on startup to the size of the ram, if the baloon is large, you would have a lot of ram used in bigger kernel structures that might be never used. There's probably a practical limit on how much your memory could grow. Provisioning would be complex, as you mentioned.

Google cloud is documented to do live migration away from hosts that need maintenance, so it seems possible to at least potentially move guests if they want more ram and can't get it on the current system. You could also make expandable guests colocate with preemptable guests, and preempt guests when expansion was needed. Presumably, you could also make a tier with premptable ram, but that seems pretty specialized, I'm not sure if you would get enough users to meet provisioning needs.

Some physical systems have hotplug ram, but it's not very common. The expense doesn't seem worth the gains in most cases.

[1] https://en.m.wikipedia.org/wiki/Memory_ballooning

I guess the missing piece is that it doesn't leave a good way for the guest to request more RAM (maxing out the balloon?).

One thing that would make this easier would be a shift towards systems like LegoOS[0]. Instead of a host-centric design, you directly attach hardware components to a "fast enough" network, each with a little bit of local controlling logic. So there are no hosts per se, just a pool of CPUs, a pool of RAM, a pool of disks etc.

It wouldn't be suitable for workloads where locality dominates flexibility. But in the long run, it could be very useful for scale economies. Especially since different workloads have different patterns of demand for CPU, RAM and disk.

[0] https://www.usenix.org/system/files/osdi18-shan.pdf

you could bill based on how much has/hasn't been ballooned out

CPU: AWS already have burstable instance types, which bill you more for usage if exceed your quota: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstabl...

As an experiment I deployed several apps to Azure S1 tier and they appear to be good bit slower than running on a VM on our Dell R740 on-premises server.

When ARM released on AWS, it was a double punch because it has no hyperthreading on the chips so you get a full core at a cheaper price.

You can't unless the provider is exposing those kind of metrics to the system. Some systems, such as VMware can expose CPU allocation time, CPU Wait, and other metrics that show much time you're waiting for resources

Azure S1s are shared CPU. My guess is that Azure is running 4 VMs per core or something, so you can get more then 25% of CPU if there isn't any contention, but you might only be 25% of that core

However we kind of moved on to disposable servers which can be scaled on demand. In which case you just add additional servers or adjust server type depending on requirements. Same with containers.

The need to add RAM or CPU to a running instance was from the time you used to have a singe long living instance serving an application.

Simply allocate each vm one core for each hyperthread and record the CPU time used in total.

Nobody actually does this because it makes billing complicated. Both practically and for sales.

My guess is this would result in overall less revenue as well, AWS is for sure making lots of money selling the same cpu time to a dozen people not actually using it.

They "for sure" aren't; their prices are so high precisely because the resources are guaranteed (in most cases).

Good resource utilization will only happen when you have non-interactive tasks that can be scheduled to smooth the demand. When I worked at Google, this is something we had; plenty of people were willing to run their mapreduces overnight, and take advantage of the purchased interactive capacity that other services needed during the day. In the real world, I've never had anything like that. Users use our website during the day. Employees use the internal apps during the day. Developers send their code off to CI during the day. That means from 9-5 there isn't enough capacity (builds have to wait) and from 5-9 the computers are just sitting there bored. I don't see a way around it, even at cloud provider scale, just because so many users are like me. I might be willing to send CI builds off to some part of the world where it's nighttime -- if you'll give me the CPUs for free. Nobody appears to have written software to do this (maybe the big CI providers do this in the background; but I haven't noticed a lot of latency while ssh-ing to CircleCI jobs for example), so we sit here using computers very inefficiently.

Ultimately, computers are cheap enough that it doesn't matter. I have a 10 core workstation that sit idle most of the time. Having a fast build when I need one is nice; and the cost to have those cores sitting idle is minimal. I would be happy spending even more money on my workstation for that reason. I imagine people treat their servers the same way, and aren't tempted to tweak things because their workloads are interactive and autoscaling introduces latency.

If the resources were actually guaranteed that simply wouldn't happen.

Noisey neighbors wouldn't be a phrase people even knew if they actually guaranteed resources.

It's easier with CPUs where you can just yield back if there's no work.