Monday, 13 June 2011

Are infrastructure people doing cloud the wrong way?

So here we are – banging on about the cloud story, flexible infrastructure, infrastructure as a service, platform as a service etc…
Most things I see / here about / vendor presentations are really focused towards how do we make the infrastructure layer more flexible… but are we not just continuing to dig ourselves a hole and perpetuating a legacy way of working?
We are busily trying to re-produce the legacy environments we have today, utilising traditional written application layers that have a reliance on infrastructure layer. Hell even the vendors are doing it – you only need to look at the approach that Oracle are taking with Exadata / Exalogic, VCE with vBlock, FlexPod etc etc……
Surely there is a better way – sat here doing some thinking over the weekend and came to the conclusion that we more and more need to turn this cloud stuff  on its head! (from an infrastructure perspective)
Why are we not provisioning a core infrastructure that is essentially very dumb, and rather than trying to offer  infrastructure services that provide resilience, high availability, scalability, disaster recovery, encryption, capacity planning (the list goes on), let’s start putting requirements in to the App Dev community  around how they have to code to / principles that they should work with? They are a smart bunch – and given the challenge and the promise of more agile delivery would probably jump at the chance to embrace this challenge.
They type of development principles / rules that I had in my mind were:
·         The application will scale out and spin up engines as it needs to become more performance or requires more bandwidth
·         The application will be coded in such a way that resilience is provided at the app layer, and has the capability. If stateful data movement is required – a publisher / subscriber (Pub/Sub)  bus must be used
·         App owners and development folk know / understand their data better than any infrastructure dude can – if they need encryption – put it in at the application layer
·         The application will be coded in such a way that DR is provided at the app layer (Using approaches such a dual commit / write many places etc.)
·         From this point forwards let’s use a converged framework for development (tools / libraries etc).
·         As the application sees BUSINESS TRANSACTIONS go out of tolerance provision some more app instances / put application logic in to deal with this scenario (no doubt some of you grid types are saying we have been doing this for ages ;-)
Admittedly some smart stuff needs to happen at the middle layer to take care of spread and movement of data, ensuring transaction data is published to / subscribed in the appropriate way, but the interfaces between app and infrastructure should be unhooked and we should be taking more advantage of pub / sub message busses to interconnect
Once you start doing this, and all the stuff that infrastructure people worry about starts falling away and the App Dev community don’t have the dependency on infrastructure – the opportunity to be more agile starts becoming reality. Let’s also face it – application development are far closer to the business than infrastructure are
Let’s start ripping out the complexity of tin / infrastructure layer and start moving up the stack and do the clever stuff (just as an aside – one of the things I am struggling with (for the enterprise) is how you ensure that items such as data are catered for in major DR events (i.e. holding state somewhere else) – I still think the concept of Prod / DR may exist for a while – so this needs to be catered for in any model you propose.
At the moment you have the likes of the infrastructure people trying to solve a problem that simply can’t fix it (we can for the legacy way of working – but not the brave new world!)
OK – so maybe this does not solve the legacy problems – but if we start tackling this way of working for NEW application development, and continue with some of the current practices for legacy applications – its got to start improving things

If we carry on doing the things in the same old way, the various infrastructure vendors will continue to empty out pockets of cash, and more importantly we continue to deploy things in the same old legacy way.  We should really re-focus that cash on a better way of doing things.
I am probably preaching to the converted – but it makes sense to me!
And now, finally – for a diagram!


  1. Have no idea what you are going on about.
    I will stick to design, my mind started thinking about Tequila after paragraph 2. I should have stuck it out past grade 8 at school because it all sounded very impressive

  2. I think you'll find what you are describing is PaaS. Create a platform that fits the needs of applications and write them to match. Then the underlying infrastructure can be simplified to deliver it. We'll still need operating systems of some sort, but with simplified delivery layers you can throw a lot of it away.

  3. thanks for comment Chris - agree, i am describing PaaS - however - we seldom look at changing the way that applications are coded to take advantage of this paradigm - hence the blog article... Developers love infrastructure people solving low level problems - ultimately - most of the problem areas (scale-out, DR, Resilience, Security etc...) of this needs to be solved at the application layer!

  4. I don't disagree but the other dynamic I expect to see is people shift to SaaS services from the vendors who have taken care of all this thinking for you and deployed in this way already in many cases. As demand increases, and competition increases, SaaS will come dowm in price. We will look at some things we do internally (in hind sight) and it seem crazy that we ever did it internally and picked up all the DR, application architecture, integration, cost modelling charges etc etc...

    I know I know InfoSec, compliance etc...but the ecosystem of security tools and mitigation steps will also continue to develop at a pace to address the concerns. and ServiceNow are already 2 examples that have corporate IT managers thinking - why did I ever take on those headaches internally! And mature application architecture functions should already be building the model you describe Mr S if the build (not buy) route is the preferred option.