Research Duquesne Advisory delivers in-depth analyses of Information and Communications Technologies, their implementations and their markets. Research is based on critical observation of the market by the analysts and their on-going contacts with the vendor community, together with hands-on, practical experience in consulting engagements.

The future of data centre LANs and SANs

Unified physical fibre infrastructure and virtualisation ... and a new organisational model

The data centre LAN is at a crossroads. Scale economics will increasingly dictate the dominance of Ethernet. The question is not what, but when.

For data centre teams, cross-disciplinary collaboration – and the establishment of a common “culture” - between server, storage and network specialists becomes mandatory, as virtualisation creates a new paradigm with stronger interactions and dependencies between these domains.

Significant time will be needed, evolution paths will vary but management issues will be critical: software solutions will rapidly mature, but data centre teams will need a clear roadmap including an all-important skill set adjustment and reorganisation.

The future of data centre LANs and SANs

Data centre interconnects: the physical and protocol infrastructure perspective

Traditionally, server, storage and networking have all had their own “technology sets” inside the data centre: from the interconnect standpoint, this meant specialised server cluster interconnects such as Infiniband, Fibre Channel SANs, Ethernet/IP-based iSCSI SANs, and Ethernet LANs, together with a lot of proprietary technologies in the mainframe arena.

Most importantly, these technologies are usually mastered by specialised staff, and organisational boundaries have reflected technology domain borders (i.e. servers, storage, network). In each of these technology “silos”, skill sets and best practices differ.

For the purpose of this analysis, we leave aside the specific case of mainframe infrastructures. Nevertheless, while the “mainframe subset” of any data centre remains a “world in itself”, we also expect it to increasingly open up and integrate with the rest of the infrastructure along the same standards.

As long as scarcity and high cost of effective bandwidth remained the rule, these “specialised” networking technologies and solutions were the only possible answers to the ever growing interconnect requirements inside the data centre. Standard, cost-effective networking solutions such as Ethernet could not provide a relevant answer to data centre requirements, just as standard, Intel x86-based server technology needed years to become relevant to IT production environments.

However, these days are now gone, also for networking technologies. The emergence and industrialisation of high-speed, lossless enhanced Ethernet changes the economics of data centre interconnects the same way it is driving ATM switches out of WAN infrastructures for one single reason: economics. Volume production of 10, 40 and soon 100 GB Enhanced Ethernet chipsets and adapters paves the way for unified, Ethernet-based physical infrastructures in the data centre. The performance gap between Fibre Channel and Ethernet for SANs is already gone – and SAN adapter technology providers (e.g. Qlogic, Emulex) now compete head-on with conventional network adapter providers (e.g. Broadcomm, Intel), to provide Converged Network Adapters (CNAs) which support Fibre Channel over Ethernet (FCoE).

While FCoE pre-requires the new, 10 GB “enhanced lossless Ethernet” and its data centre implementation, Data Centre Bridging (DCB), smaller and/or less critical block-oriented SANs based on the iSCSI protocol have already developed over traditional Ethernet infrastructures, notably among smaller or medium sized businesses that did not have Fiber Channel equipment. As iSCSI reliability and performance improve, this technology will coexist (and to a large extent compete) with FCoE on converged Ethernet data centre infrastructures, together with file-oriented network attached storage (NAS), mainly using NFS and CIFS.

For a very large majority of applications, the same evolution spells the close of Infiniband's potential window of opportunity as a data centre connectivity solution beyond high performance clustering. Despite some technically interesting possibilities (such as those recently exposed by a team of Google researchers), production volumes will drive diverging costs, restricting Infiniband to niche, mostly machine-level applications, as performance advantages (e.g. very low latency) will remain but become marginal and/or irrelevant for business application types.

The performance gaps between all these technologies have narrowed to a point where costs and return on investment will impose – indeed, already impose, at least for new installations - Ethernet as the LAN and SAN infrastructure of the future for data centres. Indeed, the amount of R&D investments going into Ethernet technologies will enable them to become increasingly effective from both the cost and energy-consumption standpoints.

Data centre “topology” must be revisited in parallel with this evolution towards a “unified fabric”. Despite the potential reduction of the number of ports, thanks to fabric unification and to virtualisation, optimising connectivity at the rack (with “top-of-rack” switches) and/or row level (with “end-of-row” switches) remains essential. “Clean” architectural designs will be essential to cope with the needed evolution and the generalisation of virtualisation. One of the immediate effects of virtualisation is the growing pre-eminence of network storage (SAN and NAS). Direct attached storage declines steadily as a result, reinforcing the need for network performance, reliability and versatility.

On another note, energy consumption optimisation will drive fibre adoption at all levels across the data centre, as LED-and-fibre energy requirements outperform TwinAx/SFP+ cabling solutions while eliminating cable length and insulation issues.

Obviously, these evolutionary changes will take time; they involve equipment replacement and, in some cases, standard maturation. For example, while Enhanced Ethernet and DCB are now well on track, FCoE still needs to address several technical and standardisation issues before delivering its full benefits. But the trend towards unified, Enhanced Ethernet-based data centre networking is clear, and the portfolio of available technologies already enables organisations to plan, and start implementing, their own roadmaps towards a “unified fabric”.

Data centre interconnects: the logical perspective

Virtualisation is clearly the key discipline that all data centre teams must develop in the 2010 decade. Server and application virtualisation cannot yield its full benefits without the matching storage and networking virtualisation. Virtual machine “mobility” inside the infrastructure requires that connectivity should “follow”.

In addition, virtualisation imposes much closer cooperation across previously relatively distinct expertise domains – servers, storage, and network experts need to cooperate closely and develop a coherent, cross-disciplinary approach of data centre architecture and operation.

Obviously, the unified data centre infrastructure requires proper automation and management tools. While most vendors now recognise this need, some have had a head start: a vendor like Blade Network Technologies has long emphasised this aspect in its offering. HP and Brocade are now advancing rapidly. Users will need to thoroughly assess this point when selecting a technology solution, since automation and management will greatly impact their ability to move smoothly to the new paradigm of data centre networking.

Automation and management solutions will rapidly mature while organisations need time to evolve their infrastructures – e.g., transition to FCoE, pre-requiring Enhanced Ethernet - and gradually generalize virtualisation. Assuming “business-as-usual” long-term continuation of existing designs would be a severe mistake.

Even though many organisations will initially maintain specialised, distinct Ethernet-based LAN and SAN infrastructures in their data centre, the target infrastructure should remain a unified fabric as tomorrow's data centres will need to be increasingly flexible and versatile.

Such infrastructures will not eliminate complexity from the information system. Indeed, information systems are complex, and will be increasingly complex. They will make complexity more manageable; this will increasingly require standardised, straightforward infrastructures that move complexity to “higher level layers” where complexity indeed has business value.

The virtualised data centre needs a new organisational model

We believe the rhythm of evolution should be dictated by the data centre staff skill set evolution roadmap rather than by pure technology considerations. However, this will only be feasible if data centre managers initiate the evolution as early as possible, rather than being later “pushed” by vendor offerings. The evolution will be profound, requiring server, storage and network specialists to work together more closely. It will also dictate that architects, developers and production staff cooperate and communicate more effectively, breaking technology “silos” the same way (or faster than) application silos need to be broken.

In such an evolution, the specific expertise and best practices developed in each domain must be preserved and leveraged. The portfolio of technologies enables this preservation: as an example, FC experts will continue to apply their FC knowledge, tools and practices in an FCoE environment.

Indeed, this organisational question is by far the most important for data centre teams. Technology evolution enables changes that are required to match business requirements. In this evolutionary process, virtualisation is the “mother of all changes” and will ultimately enable the truly agile organisation. Virtualisation skills are therefore paramount for all data centre professionals – irrespective of the infrastructure sourcing and delivery solutions they select, be it some kind of cloud computing or more traditional production environments. Rethinking communication and cooperation between highly specialised and skilled professionals will enable organisations to retain technology control, independently of their sourcing strategies. Optimising the data centre requires a holistic approach, rather than mere component-level optimisations.

Organisations should assess data centre network vendor offerings with this perspective in mind:

  • Which vendors can best accompany the necessary evolution of our skill sets, and transfer the appropriate expertise and know-how to the data centre and IT staff rather than create a vendor lock-in?

  • Where can we find the proper experience and expertise (rather than a marketing message) on “cloud computing” or any other packaging for the true core technology of the future: virtualised, cost-effective, standard-based production infrastructures that can match business requirements?

The advent of a market sea change?

These changes open a window of opportunity for vendors, both established players and – to a degree – new entrants.

Reinforced by the acquisition of 3Com and therefore equipped with an almost full offering, HP has engaged in an aggressive campaign to shake up the networking market and contest Cisco's dominance. The data centre will be one of the key battlefields, more because of criticality than sheer size. HP, however, will need more than cost-oriented messages to significantly displace Cisco, which enjoys a technical customer base which is extremely “loyal” (some might say “indoctrinated”). The recent acquisition of 3PAR, although it brings an FC-based solution to HP's technology portfolio, will bring muscle to its storage offering while 3PAR technology will be able to evolve towards the new standards over time. Indeed, 3PAR’s strong point is not in the protocol choice but the mastery of software and ASIC components design – a largely network-technology-agnostic competence.

A specialist vendor such as Brocade has both a challenge and an opportunity, as Ethernet dominance opens an opportunity for new entrants to compete for the previously well-isolated SAN market. The Ethernet expertise acquired with its Foundry acquisition has enabled Brocade to develop an aggressive approach for the converged data centre network market. Like all other current offerings, Brocade's “Virtual Cluster Switching” (VCS) convergence solution presents several proprietary aspects, enabling the creation of a homogeneous (i.e., Brocade-only) “virtual fabric” that interoperates neatly with the “rest of the network”. There is no alternative to such proprietary convergence architectures today, as standards are neither completely defined nor stabilised. Brocade’s challenge will be to convince users that they can leverage this type of offering to optimise their data centre infrastructures and engage a cost-effective SAN/LAN convergence without waiting for a still far-fetched, and maybe hypothetic, full standardisation in this space.

With the acquisition of Blade Technology Networks (which had long standing alliances - probably to level of dependency - with IBM, notably in terms of patent access), IBM gains reinforced momentum in the data centre networking market. Blade's early investment in network virtualisation as a key technology brings a key asset to actually deliver next generation data centre networking infrastructure, and may help IBM to develop the proper ecosystem around its offerings.

The data centre networking market excites vendors' appetite, not only because of its size and profitability, but also because it represents a strongly structuring component of tomorrow's data centres. Vendor franchise boundaries are blurring: Cisco's UCS spans the server and storage arena, while HP and IBM “re-enter” the networking field. In such a context, marketing hype can only flourish; savvy users will want to define their own evolution roadmaps, keeping in mind the need to gradually evolve infrastructures and skillsets while optimising costs and performance.

Bottom line for users

User organisations should start planning for the new data centre network infrastructure paradigm – “everything on Ethernet” - while bearing in mind that this evolution will occur over a significant period of time, and be part of a more “holistic” vision of data centre architectures. Solutions such as FCoE and DCB, even though still incomplete and/or partly proprietary, offer an already credible transition path for FC/SAN users while enabling continued leverage of existing solutions and infrastructures.

Note: In a series of forthcoming notes in this series, we will look at vendor offerings, evaluate the state-of-the-art in virtualised environments management, and propose an assessment framework for user organisations planning the evolution of their infrastructures.

Tuesday, December 7th 2010
Duquesne Advisory
Newsletter To subscribe to the Duquesne Advisory Newsletter, please enter your e-mail address.

Duquesne Advisory

Duquesne Advisory is a European firm, dedicated to researching, understanding and advising clients worldwide on opportunities and trends in Information and Communications technology.


Duquesne Advisory delivers in-depth analyses of Information and Communications Technologies, their implementations and their markets. Research is based on critical observation of the market by the analysts and their on-going contacts with the vendor community, together with hands-on, practical experience in consulting engagements.


The analysts of Duquesne Advisory leverage the Firm’s ongoing market and technology research to undertake high added value consulting engagements for both ICT users and ICT providers. Focused on client service, their approach is rigorous and methodical, and at the same time pragmatic and operational.