Coraid Blog

Automating Storage Management for Virtualization and Cloud Architectures

While IT budgets are largely flat, end users reliant on computing resources and services to get their jobs done are making more demands. To best meet these rising demands, IT needs more labor-reducing automation to keep up with the increasing needs of their companies.

Our Ethernet SAN technology simplifies the network and storage arrays, and our recent announcement introducing EtherDrive SAN Manager (ESM) 1.5 our newest management platform ties them together with automation and visibility across all resources making managing petabytes of shared storage as simple as managing local disk.

Coraid EtherDrive SAN Manager 1.5 leverages the scale out architecture of EtherDrive and the single namespace of CorOS to monitor and configure storage as a pooled resource.  With a graphical user interface and rich REST API, administrators gain visibility and control over their entire storage network.

EtherDrive SAN Manager includes the Coraid ESM for vCenter Plug-in, which provides VMware administrators visibility and control over virtual machine storage, including virtual machine-to-storage mappings and the coordination and scheduling of array snapshots.  It also includes a Microsoft VSS hardware provider that communicates with Microsoft Windows Server and Microsoft Hyper-V to quiesce applications before storage snapshots, ensuring data consistency.

What’s New in EtherDrive SAN Manager 1.5 

• Graphical User Interface: Provides the visibility and operational controls to provision, configure, monitor, and troubleshoot storage resources including the EtherDrive SRX and EtherDrive VSX.
• Physical Inventory: By leveraging the single namespace of CorOS, provides a detailed list of every physical component in the infrastructure including disks, logical volumes, appliances, and more, available in multiple views and with the ability to filter and sort to find specific resources.
• Alerts and Alarms: Provides proactive alerts and enables quick drill downs from system to appliance to disk to isolate issues.
• Web-style, REST Interface: Enables programmatic control of the storage infrastructure.  With exactly the same rich API used by the graphical user interface, storage administrators and cloud operators can use a broad array of management tools to orchestrate their storage environment. 
• The Coraid ESM for vCenter Plug-in: Enhances Coraid EtherDrive to provide VMware administrators visibility and control over virtual machine storage, including virtual machine-to-storage mappings and the coordination and scheduling of array snapshots for data protection.
• Microsoft VSS Hardware Provider: Communicates with Microsoft Windows Server and Microsoft Hyper-V to quiesce applications before storage snapshots, ensuring data consistency.

Virtual server and cloud environments present unique storage management challenges and require interfaces that provide visibility, control, and automation of the virtual infrastructure system.  Coraid's groundbreaking Ethernet SAN platform eliminates multiple layers of traditional SAN complexity, and EtherDrive SAN Manager enables powerful automation in virtualized and cloud computing environments.

Ethernet SAN Gets Top Marks from Palmer’s College

Did you know Coraid has over 130 education customers, colleges, universities, and school districts all over the world?  Some common themes these institutions share are their need to manage storage growth, sometimes doubling year over year, tight budgets, and a need for easy to manage storage that is not so complex it takes trained experts to keep it up and running.

Palmer College in the UK is a good example of a college that because of data retention regulations (emails and student records must be kept for five years and old students’ work for three years) is trying to keep up with significant growth in storage requirements and management challenges.

To address their storage requirements many education institutions are turning to server virtualization to improve their IT efficiency only to find that Fibre Channel shared storage will end up costing more than what they save in server consolidation.   And iSCSI does not provide the performance they need.

This is why so many education institutions turn to Coraid Ethernet SAN storage, because they have found that by leveraging the power of Ethernet SAN our customers get the performance of Fibre Channel without the cost or the complexity.  Our scale-out architecture based on commodity hardware and Ethernet is a proven, reliable and affordable alternative to Fibre Channel and iSCSI SANs.

Here is what Dan Byne, Network and system developer at Palmer's College said, “Our Coraid systems have curbed our needs for additional IT staff; the storage infrastructure is much easier and quicker to manage now. What took a working week before only takes a day.  The Coraid Ethernet appliances are very fast and the performance/cost and capacity/cost ratios are fantastic. Like most colleges, our IT budgets are very tight so we need to make every penny work as hard as possible; this is why we purchased Coraid systems.”

Fibre Channel does not = Cloud

Check out this new article on VMBlog.com by our CEO Kevin Brown titled “Why you can't Build a Cloud with Fibre Channel”.  In the article Kevin writes about how cloud economics require a new storage paradigm.  “Take a look at cloud computing giants like Google and Amazon in the era of Big Data and you won’t find cloud stacks built on legacy storage technology”, writes Kevin.  These cloud service providers realized Fibre Channel was too expensive and instead built massive data centers using commodity hardware and virtualization software. 

Cloud storage architectures, whether public or private, represent a huge design shift from traditional enterprise storage. To minimize cost, cloud environments need to scale out leveraging lower cost building blocks instead of scaling up behind expensive storage hardware.  We see many enterprises turn to Coraid EtherDrive to help them replicate the benefits and economics of this consumer “cloud storage” model in their own data centers.  

Why? Because of its disruptive economics at 1/5^th the cost of Fibre Channel, its high performance at 1,800 MB/s and its extreme flexibility scaling up to 65,536 shelves per network. Also, Ethernet SAN eliminates multiple layers of complex legacy SAN topology, including multi-pathing software, port bonding, and switch configuration.

If you are looking to build a private or a public cloud try Ethernet SAN; it makes cloud storage affordable, scalable and easy to use.

Ethernet SAN Optimizes Microsoft Hyper-V

A recent Coraid Survey suggests that the high cost of storage purchases is diluting the benefits of server virtualization.  More than half of IT respondents reported that for their virtualization project they spent 3 times more on new storage than on their virtualization software.  The high cost of storage is one reason our customers turn to Coraid because Ethernet SAN offers a simpler storage topology with better performance at a lower cost than traditional storage arrays.

A new customer deployment at Duplo International really illustrated this well.  Duplo decided to use Microsoft Hyper-V to virtualize their servers in order to increase utilization.  Here is what Glyn Nutting, the Financial Director at Duplo said, “we knew that server virtualization would help us reduce our IT spend. However it would require a shared storage environment and our concern was that this would be so expensive that it would negate our server consolidation savings.”

Duplo was pretty methodical in reviewing all the storage options available to them.  They liked Fibre Channel for the performance, but found it too expensive and complex, and they were not prepared to sacrifice performance by deploying an iSCSI solution.  Instead Duplo turned to Ethernet SAN technology which leverages Layer-2 Ethernet to provide a high-performance, scalable architecture built with off-the-shelf, affordable hardware.  Duplo initially looked at Coraid because of the low TCO but were then blown away by the performance and the simplicity.  

Why do customers find Ethernet SAN so simple?  Because Ethernet is a technology they know well.  With Ethernet SAN storage customers can set up an array in 60 seconds; three commands; it's done. After that it's self-service in the hypervisor.  Shared storage is configured as if its local storage, special multi-pathing software goes away, port zoning and bonding go away. The whole idea of a fixed connection network goes away.  Ethernet SAN strips away the complexity of legacy storage networks.

We hear these stories all the time.  Traditional storage arrays are too complex and too expensive.  They were designed for the mainframe era and do not scale well under the unique demands of virtual infrastructures.  By partnering with Coraid and leveraging Ethernet Duplo got an easy to use, high performance and affordable shared storage solution.

Partnership Accelerates Ethernet SAN Acceptance

Our announcement with Arista Networks to team up to accelerate the adoption of Ethernet SAN received plenty of media interest with a number of articles published in the press already.  Interest is high within IT organizations and with the media and analysts for an alternative to the legacy storage protocols (Fibre Channel, FCoE and iSCSI) that can meet the demands of modern, virtualized data centers.

Server virtualization features like live migration strain legacy storage networks that were designed for static, mainframe deployments.  This makes management difficult and drags down the ROI of virtualization and cloud projects.  Our shared goal with Arista is to enable extensive virtualization based on end-to-end Ethernet, flexible topologies, and low cost scale out hardware without compromising on performance.

In fact, by eliminating complex protocol layers Coraid EtherDrive storage enables near-bare-metal performance using Arista's ultra-low-latency 10GbE switches.  To demonstrate this, we tested performance using the IOMETER workload generator via simulated application workloads based on streaming media services. The tests confirm Arista and Coraid’s ability to deliver predictably scalable performance of 1,800 MB/sec from 36 SATA drives. Put into perspective, a single shelf was able to drive enough bandwidth to saturate a 10Gbps interface.  And with the deep buffers of the Arista 7500 family of switches featuring more than 40ms of per port packet buffering, conditions such as fan-in, which arise when multiple servers need to access the same storage data, are well handled.

If you are considering storage for a server virtualization or cloud project, Coraid and Arista can provide a high performance solution with the simplicity and cost of scale out hardware.

New SAN Virtualization Appliance

Virtual is reality in today’s data centers. With this new reality comes greater complexity and new challenges. Customers I talk with realize their storage is strategic to the success of their business. These customers want virtualized storage that is fast, flexible, and reliable. To address this growing demand Coraid has released a new storage virtualization product that changes the virtualization game by simplifying and reducing the complexity to deploy, manage and grow storage in virtual environments.

The EtherDrive® VSX virtualization appliance brings storage virtualization features to Ethernet SANs, including logical volume management, synchronous mirroring, snapshots and cloning. The VSX virtualization appliance is designed to provide virtualization across the SAN not just within a single storage array like most other storage systems. The VSX appliances can support multiple Coraid disk arrays over Ethernet with no direct cabling, enabling customers to simply add capacity without the need to configure or pay for more software. This ability to start small and scale up without having to buy more software or upgrade from a small storage box to a big storage box reduces cost, reduces complexity and represents a real change in the storage virtualization market.

IT organizations are expected to do more with the same or fewer resources. To keep up, IT administrators require the ability to allocate storage for any application based on the latest business needs, and to do so non-disruptively, quickly and easily. The VSX represents a new class of storage appliance that provides “enterprise-class” storage virtualization for an entire Ethernet SAN, allowing IT administrators to respond to complex storage needs quickly and reliably.

Virtualization's Dirty Little Secret

Virtualization has brought undeniable benefits to enterprise IT—reduced costs, server consolidation, increased flexibility, automated management and more. Coupled with powerful, scale-out x86 servers, virtualization enables enterprises of all sizes to move toward cloud architectures that achieve massive scale using commodity hardware, multi-tenant segmentation, elastic expansion, and automated self-service.

Unfortunately, all of the cost and agility benefits of enterprise virtualization come to a screeching halt when data reaches today’s connection based legacy storage network. In fact, legacy storage networking technologies such as Fibre Channel, Fibre Channel over Ethernet (FCoE), and iSCSI all evolved from mainframe-era one-to-one connection based designs. Mainframe protocols like FICON and ESCON ran on Fibre Channel, and the array designs grew out of this tradition. They require expensive custom networks, and manual configuration of every connection between servers and storage. Providing new storage to a VM takes hours of work, and days or weeks of man hours, instead of a few clicks. That’s not cloud!

Beyond the complexity and agility issues, legacy storage is fundamentally out of sync with cloud economics. Services like Amazon are offering storage for as little as $0.05 per gigabyte per month. Want to build your own cloud? Call up your favorite storage vendor, and they will charge you between $2.00 and $10.00 per gigabyte, before you even turn on the power. Not exactly a formula for profitability. The dirty little secret is that virtualization players have started taking storage costs off their ROI calculator -- when you upgrade to faster arrays to handle increased server density, storage costs blow away most of the savings from server consolidation.

Fortunately, there’s a new class of Ethernet SAN storage that mirrors the commodity hardware economics of the cloud. The protocols grew out of the Linux market, where they have been open and native in the Linux kernel since 2005. By 2009, Ethernet SAN technology was integrated with VMware, and had been adopted by more than 1000 customers worldwide. The new generation of SAN storage, built on bare-metal 10 Gigabit Ethernet and off-the-shelf storage servers, delivers faster performance than Fibre Channel at less than 20% of the cost, while eliminating multiple layers of complex storage management. Storage arrays can be deployed in as little as 60 seconds. Today more than 1300 user organizations, including Fortune 100 enterprises, movie studios, cloud providers, and the U.S. Department of Defense, have deployed Ethernet SAN.

To sum up, the bad news is that legacy storage technologies from top vendors kill the ROI of virtualization and complicate the evolution towards cloud architectures. The good news for 2011 is that next-generation Ethernet SAN technologies are arriving to improve storage performance, cut costs, and ease the way for scale-out cloud deployments in the enterprise.

10 Facts about Ethernet SAN

Real Ethernet SAN, ATA over Ethernet (AoE), hasn’t received the media hype of Fibre Channel over Ethernet (FCoE), but those who use it understand how leveraging the power of Ethernet unencumbered by Fibre Channel can deliver a flexible high performance scale-out storage solution that is designed from the ground up for simplicity, and virtualization, with a 5-8x price performance advantage over Fibre Channel.

Here are 10 key facts about Ethernet SAN from Coraid:

1) ATA over Ethernet has been around for over a decade. It is an open standard protocol and is included in the official Linux kernel 2.6.11.

2) Coraid is the market share leader of Ethernet SAN storage solutions providing a family of Enterprise class scale-out storage arrays called EtherDrive®.

3) The AoE initiator breaks down I/O requests to match the MTU size of the Ethernet network with no encapsulation and sends I/O requests in parallel across all available links.

4) ATA over Ethernet is connectionless and doesn't use sequence numbers, relying instead on per-request tags to keep I/O straight.

5) Storage arrays are identified by target and LUN with "major.minor" numbers, and the network can scale with additional clients and targets to over 65,000 nodes.

6) Retransmissions are handled by the protocol, and tagging eliminates worries about out-of-order requests.

7) ATA over Ethernet traffic uses multiple initiators, network links, and targets without any special protocols or configuration.

8) There's no security in the AoE protocol itself: There are no session controls, so any client can read or write data to any storage system, though access control lists based on MAC address and "config string" are implemented.

9) ATA over Ethernet isn't routable, so it's limited in practice to local-area networks.

10) Practically any Ethernet switch can be used for AoE, just so long as it supports 9k jumbo frames.

Redefining Storage Economics – Q&A with CORAID CEO Kevin Brown and CTO/Founder Brantley Coile

This week, CORAID announced a $10 million venture investment, a 400 Terabyte deployment at the U.S. National Institutes of Health, and an expanded executive and advisory team.  http://www.coraid.com/

Join us for a Q&A with new CEO Kevin Brown and CTO/founder Brantley Coile:

Q: Congratulations on the funding and new additions to the team. Why are investors and industry players interested in the storage sector, which has been dominated by large vendors for some time?
Kevin: Virtualization and cloud computing, along with recent economic trends, have created an inflection point in the storage industry. Virtualization adoption has created a whole new generation of SAN buyers that have zero affinity with Fibre Channel, iSCSI, or the dominant vendors. If you want to use VMware VMotion™ and other similar features, you need a high-performance SAN. Storage costs have increased from 15% to 50% of many IT projects, and prices have stayed high. A lot of customers are frustrated and looking for a new approach.

Q: Why have the large storage vendors failed to adapt?
Brantley: Legacy SANs are built on mainframe-era protocols and architectures, which arose when you were connecting a couple of large servers or mainframes to a couple of large storage frames. FICON became Fibre Channel, which inspired iSCSI. The large vendors have a huge technology base and a business model that’s built around preserving that status quo. Unfortunately, the genetic profile of these solutions is poorly adapted to the modern virtual or cloud datacenter.

Q: What are some of the technical challenges for storage in these environments?
Brantley: There are lots of issues. For example, if you’ve replaced your large server with a rack full of x86 servers, you now need a huge number of expensive HBAs, cables, and Fibre Channel switch ports. Instead of predictable static application workloads, virtual environments are constantly moving workloads around, creating bottlenecks at the network layer and tremendous head contention on the drive level. SANs weren’t designed for constant change and flexibility, but that’s exactly what virtualization and cloud architectures are about. If you have 50 servers with 30 VMs each, and want to use iSCSI to get away from Fibre Channel, you need 1500 TCP/IP connections to talk to a storage array 10 feet away! That’s a crushing amount of overhead.

Q: Where does CORAID fit into this?
Kevin: Brantley invented the Cisco PIX firewall and LocalDirector, pioneered the use of stateful packet inspection, and holds one of the fundamental patents on network address translation (NAT). He specializes in disruptive inventions. Starting in the Linux community around 2005, Brantley and his team quietly introduced a radical new idea for storage networking: an Ethernet SAN called EtherDrive® that eliminates legacy-era SAN complexity, delivers massive throughput between servers and drives, and enables customers to scale out using commodity building block arrays. It’s designed from the ground up for virtualization and simplicity, and we now have over 1100 customers. Just as Dell and Compaq transformed the server market, EtherDrive is poised to do the same in the storage market. We’re here to reinvent storage economics.

Brantley Coile, Coraid CTO, to Speak at OpenSource World

Brantley Coile, CTO of Coraid and inventor of the ATA-over-Ethernet (AoE) protocol will be speaking at OpenSource World™ (formerly LinuxWorld®), scheduled to be held August 12-13, at Moscone Center West in San Francisco. The inventor of AoE lightweight SAN protocol will discuss how the technology has commoditized Open Storage.  OpenSource World focuses on innovative solutions in real-world environments using open source, open standards and open architecture as part of an integrated IT infrastructure. Coile’s session, “AoE: A Standard Protocol Has Commoditized Storage” will be held Wednesday, August 12, 2009 at 2 PM.

The talk provides an insider's view of the popular open source AoE (ATA over Ethernet) protocol, highlighting its applicability in virtualization. Offering significant cost and performance benefits over traditional NAS, iSCSI and Fibre Channel for server virtualization environments, AoE delivers on speed. This talk will explore the openness of the technology and how that development paradigm helped the technology evolve to its pervasiveness, extending to virtualization today. Case study examples will be shown  that explore how AoE storage achieves extremely high performance at a significantly lower cost that iSCSI or Fibre Channel solutions in Xen, Citrix, and VMware virtualization environments.