Thoughtput

Haven't We Seen This Movie?

Stacey Higginbotham launched a post today on GigaOm titled Next Up: I/O Virtualization? discussing options to consolidate Ethernet, Fibre Channel, and InfiniBand connections on a single link. It is great to see companies tackling the path management mess created by trying to jam 6 PCI cards in a 1U server.

We spent some time discussing I/O virtualization and virtual I/Os with Greg Schulz at The StorageIO Group and have our own post and short podcast at Defining Virtual I/O Acceleration.

But more importantly, this continued activity in the I/O virtualization arena triggers the following...

How the heck did we go from:
1. We have Ethernet to
2. We have Ethernet and Fibre Channel to
3. We have Ethernet, Fibre Channel and InifiniBand to
4. We can put everything back on 10Gig Ethernet.

Is it just me, or does this seem like we could have saved a few billion dollars over the last few years and skipped a couple of steps?

I know there were things Fibre Channel could do before 10Gig Ethernet came around. And I'm sure someone will be happy to point out the interconnect efficiencies of InfiniBand. But let's be honest with each other...haven't we all seen this movie again and again?

Top 6 Ways to Cut Storage Costs

We talk a lot about performance at Gear6. But equally important to the performance angle of scalable caching appliances is the opportunity to reduce storage costs. We put together this short video to outline the cost-saving advantages of scalable caching appliances.

Scalable NAS and the (un)Scalable Disk Latency Problem

The primary benefit of scalable NAS solutions is being able to increase capacity while maintaining a global namespace and providing access to one coherent file storage pool. Capacity is increased by adding more nodes to the storage pool where typically new nodes are similar to existing ones in terms of memory and storage capacity. It is important to note that while adding nodes can increase bandwidth capacity, the bulk of data is still retrieved from disk and therefore adding nodes to a clustered file server does not decrease latency. 

But does the latency remain the same? Herein lies the problem: as more nodes are added, the overhead of coordinating file operations between the nodes increases. Therefore, latency increases as more nodes are added to a clustered NAS solution. Regardless of the implementation, clustered NAS solutions must coordinate file coherency between the nodes since data modifications by one of the nodes must somehow be synchronized with the other nodes to make certain that no data corruption takes place. Even when reading data, overhead exists to either propagate or coordinate the read requests to the correct node. And as node count increases, the chance that a file request would initially be sent to the right node decreases, making request propagations more frequent.

So how do you enjoy the capacity scale of clustered NAS solutions without being impacted by increased latency?

This is where a scalable, centralized caching appliance can comes to the rescue. As opposed to other solutions, a caching appliance serves data direct from memory instead of disk, thereby reducing latency to sub-milliseconds response times. Furthermore a central appliance does not need to coordinate file operations between other appliances/nodes and can guarantee high performance levels regardless of the capacity and scale of the underlying file system. With a centralized caching appliance enhancing a clustered NAS solution, capacity can be scaled and managed with a global namespace while performance also scales and high bandwidth and low latency levels can be maintained. A centralized caching appliance also provides the benefit of being able to scale performance by growing the cache without the need to over-provision capacity in the storage system as well.

So do clustered NAS solutions really scale? Definitely, they scale in capacity and data management. But if you want to also enjoy the benefit of performance management and scale, nothing can replace the need for centralized caching.

Learning the Components: Memory and Storage

I came across this video on YouTube when reviewing some of our Gear6 material. The "memory" and "storage" terms have linked us in the videosphere.

Take this with a grain of salt...I think this is two tech guys explaining the differences between memory and storage to a Computer 101 class. But the analogies they use are entertaining, and the similarities between the data center problems of slower disks and faster memory are striking.

An Easier Way Out Of The Attic

Stacey Higginbotham at GigaOm wrote a short post about startups working to speed storage access. She has a great analogy about how storage access relates to keeping things stored in your attic. My favorite selection is here...

VMware benchmarks protocols

VMware recently published a paper benchmarking storage protocol performance, comparing:

• Fibre Channel
• Hardware iSCSI (with iSCSI and TCP/IP offload engine)
• Software iSCSI
• NFS

According to the description, the focus was on cached runs to showcase protocol as opposed to system performance. They also chose 100% sequential workloads, stating that randomness has almost no effect on throughput or response time in these cases.

The one tricky part of the test is that Fibre Channel was measured using a 2Gb/s connection, and all of the IP based protocols using a 1Gb/s. That said, results reached about wire speed in most scenarios, which validates one longstanding networking maxim...when you have networks that are point-to-point, full-duplex, and switched, it generally doesn't matter which protocol you run. What is more important is do you like the overall architecture, it the solution easy to maintain and operate, and does it deliver favorable cost.

The takeaway from VMware (noting that FC tests were at 2Gb/s and others at 1Gbs/s):

...although Fibre Channel has the best performance in throughput, latency, and CPU efficiency among the four options, iSCSI and NFS are also quite capable and may offer a better cost‐to‐performance ratio in certain deployment scenarios.

I also came across an interesting blog entry also related to this paper's release.

On some in-house application-specific benchmarking that I’ve done, I actually saw overall better performance with an NFS datastore than with a software iSCSI datastore on the same filer.    from Thinking Sysadmin blog

I hope virtualization fans will take these results for what they are worth...all storage protocols are valid in certain cases, but people should choose architectures, not protocols. For network attached storage fans, there should no longer be any hesitation about virtualization deployments. We covered this earlier in a post, NFS on the Rise for Virtualization. In particular, for NAS fans looking to have guaranteed performance, the complement of a scalable caching appliance in virtualization environments can make a lot of sense. We covered this in Overcoming Virtualization I/O Pitfalls.

 

 

Is Caching A Tier?

We've seen the emergence of the term "Tier Zero." It typically refers to the highest performing tier of storage within a storage array. But having this top tier also comes the responsibility to manage data up and down the tiers. That means defining policies for each tier, determining which data gets placed on each tier, protection for each tier, and capacity management for each tier, including tier zero.

Managing capacity within a memory-centric storage tier zero can be tricky. Because tiers in a block-based storage array represent LUNs, the systems are not particularly dynamic in the expansion or contraction of LUNs. (This is not to say that those capabilities do not exist, but it still isn’t that easy.) That rigidity, along with the required assignments of migration policies, coupled with the need to snapshot, backup, and protect each tier, can lead to a long list of to-do items for already overburdened administrators.

Centralized caching represents a top-performing I/O delivery engine which can be thought of looking like a top tier of storage, but is actually quite different.

Instead of taking memory and trying to make it look like a disk or persistent tier, centralized caching takes memory and makes it look like an intelligent cache. This means that the cache is dynamically populated from as much storage as you would like to support. Caching enhances existing storage configurations compared to replacing them.

When tier zero appears as a LUN, you are effectively saying to the application “you can have as much performance as you like, as long as it fits in this LUN.” With centralized caching you are saying, “you can have as much performance as you like across any amount of storage, and if you need more caching resources you can add them on-the-fly.” Quite a different approach, and one Gear6 sees as a much more effective way to deploy memory in the data center.

A reference deployment might include a scalable caching appliance strategically located in the network to provide rapid client responses and access to a clustered file system. The clustered file system does its function well…which is to provide infinite, easy-to-manage capacity. The caching appliance does its function well…which is to dynamically apply performance resources to shifting disk and system hotspots based on application requests. This is a winning combination.

So is caching a tier? Perhaps. But more importantly, caching is a more sophisticated, more dynamic approach than creating a fixed tier zero of memory based storage. Caching frees administrators from the hassles of figuring out what should or should not go in memory, letting applications drive own their needs and allowing for cache scalability to deliver maximum application performance.