Saturday, 1 May 2010

What sort of things would I like to see included in storage array federation??

Hi there all, so there have been a good few discussions going on between
a bunch of us, namely: @storageanarchy @3parfarley @ianhf @storagebod
@stuiesav (myself) and numerous others (apologies if i have missed
anyone) around the subject of storage federation...

Firstly - a big "Get fixed soon" to Marc Farley - Take a nose at his
Blog - really quite humbling to see what technology is capable of...
forget about all this storage and infrastructure malarkey - Keeping
people healthy - that is where it is at... Take a nose:

Now - back to business - Just a little rant... honestly just a little
baby one.....I truly truly think it would be of great help to both the
industry (vendors and customers) if some firm definition of terms were
made, and in this example - why federation does not equal virtualisation
and virtualisation does not equal federation (I'm still struggling with
it whilst writing this article - please don't flame me ;-) I really
believe that the governance and standards bodies could really help here
by writing a dictionary of terms etc.

Right - back to the topic in hand - As well as twitter traffic there
have been a couple of good blog postings, one from @3parfarley which can
be found here:
another from @ianhf which can be found here:
and finally some of my own thoughts on some of the hype etc that is
surrounding this topic which can be found here:

Views and thoughts have been shared and its been a good discussion. I
thought i might just add a few more views in terms of the type of
functions / behaviors that i would expect from a federated storage

This really is not a hard spec of items, but really some rough
scribblings that i thought peeps may be interested... so here goes for a
bit of "rough":

Day 1 functions that i would like to see (again rough...):

* Allows non-intrusive life-cycle in and out of arrays
* Allows a re-balance of data across arrays
* Load balances (guess this couples with the above point)
* Scales (isn't limited by lun counts, array counts etc) as an
example, i dont want a federated layer becoming a bottleneck
* Storage array agnostic and can federate data spread across arrays
of disparate type
* Understands performance characteristics of arrays that are being
federated (look and listen approach)
* Ability not just to tier on disk within array but accross arrays
based on holistic performance of arrays and can place data into
different locations
* Is not tied to geometries / lun boundaries / like for like
configs, can move and translate
* Ability to off replication (not sure this is a federation function
- but rep between dislike arrays is attractive)
* Can spoof previous geometries of configurations to allow easier
migrations between host types / arrays etc
* Has a policy engine that can define what should be where, when
data needs to be evacuated / drained (in the event of system
lifecycle) and define where data should be right-placed.
* Any host able to map to any component within the federated array
configuration without the pain of additional lun masking, zoning
etc etc etc (i.e. help me with the provision nightmare that is
involved with moving data around)
* Federate over short distances where possible (i.e. arrays within
a DC complex or upto 80km spread)
* Move the smarts normally associated with the array (such as
snapshot, replication etc) upto the federation layer

I would like to put in day-1, the ability to federate to differernt
types of storage, i.e. file system, block, object stores, links into
various API's etc - but i guess this could be asking just a little too

Day-2 functions (even more rough....)

* Ability to link into an Infrastructure federation layer that works
/ behaves at a global enterprise level
* Offer the concept of connectors between federation layer,
application, OS and array so that data can truely be moved to an
appropriate tier based on API, Application definition and policy -
a big ask, but as we move to these cloud based services - its
gonna need to be tackled - maybe there is a federation that needs
to happen between cloud services??
* Start offering concept of de-dupe (or maybe i mean
single-instance) across federated infrastructure - learn whats out
there and stop writing the same thing everywhere - I am guessing
the amount of metadata required to do this is huge - but it may be
less than the data set itself ;-) (hey - i never said this was
gonna be easy!)
* Global federation - based on centralised federation layer, tie in
with application definitions and location information - if an
application gets deployed in one region or many - the ability to
handle that, and right-place provision is key. This could play
such a huge part in time-to-market discussions - and the fact you
can just tell an infrastructure cloud to provision ready to accept
app would just rock!
* Ability to federate from a storage service that may be in house,
and movement / lifecycle out to a cloud service and vice versa...
This coupled with a policy engine would rock... (hmmm never used
that term before - whats that all about)

Right - that is where I am up to now... Will add to this over time -
However, I am on a train whizzing to Nottingham for a friends bulls
party (maybe might even have a beer or two)...

I am aware that i may be dipping out of storage federation and bumping
into infrastructure federation but hey.... there ya go!

Would be really great to hear other peoples views / add to list / tell
me i am mad as a box of frogs etc..

Have a great one!!


1 comment:

  1. Nice Post Stuart! (and thanks for the references). You've listed a number of excellent ideas here that I think will help people understand how Storage Federation is distinct from virtualization.

    It's definitely good to get more meat on the Federation bone.