To my mind, the success of VPS has hinged on the fact it's been the only solution to fill the shared and dedicated server gap, not the ideal solution.
I understand perfectly why developers and site owners choose to use VPS. We've become disillusioned with the underperformance of most shared hosting and eventually traffic outgrows even quality shared hosting (congratulations on the success).
So with the looming cost and complication of dedicated servers, we’re forced to accept the rather drab compromise of a VPS. Buying more resources but actually getting less grunt per pound than a dedicated server.
The trouble is Virtual Machines create a complete clone of the OS which then needs to run on each VPS on the host server. That's a lot of overhead to be paying for.
If your operating system uses say 300MB then your 512MB VPS starts looking as useful as a wet piece of toast with the 212MB memory left available to run your entire webserver (Apache, PHP, MySQL, etc).
This inefficiency is then exacerbated by the fact many VPS solutions are hopelessly oversold. In an effort to claw back some of that lost value and capability from the poor utilization of base hardware. Many vendors load up more VPS machines than can be optimally supported by a system.
The whole thing always struck me as a non-solution.
Providers have buried their heads in sand ignoring the damage they were doing to their own reputation and the hosting industry generally. Meanwhile, customers have been presented with only one executable strategy, buy more resources.
"If you're 512MB VPS isn't big enough, buy a 1024MB VPS"
- hardly indicative of a progressive and innovative industry.
It's actually easy for website owners to reconcile oneself with the above because you're still paying less than you would be for a dedicated server. Moreover, your site is now running and who cares about how efficiently your providers hardware is being used anyway.
However, unfortunately at some point comes the realisation that with VPS we have all the configuration, security and management problems of a dedicated server.
This level of control can be desirable, the ability to have complete customisation of your server can be a real boon. However the time and expertise required to do so successfully comes at a cost itself.
A bag full of the latest wizard tools make squeezing performance and managing VPS... well, more manageable. Technology like Chief, Puppet, Vagrant and LXC give us enterprise levels of automation, free of charge (expect time) and with a much lower level of expertise than previously required.
These awesome tools do much more than allow us to claw a bit of value back from VPS however. They are actually creating a new breed of solutions and it's these tools and next generation solutions that we at HudHost are passionate about offering you.
It's our belief at HudHost that every hour of a developers time spent figuring out their hosting is at a minimum an hour lost in creating awesome apps and websites. Even worse, it’s often an hour lost that could have been spent with our kids, being outdoors.