« March 2008 | Main | May 2008 »
There has been plenty of recent market activity on the reach of memory in to the data center. We covered this in a series of posts, including:
1) Media
2) Architecture and Implementation
3) Installation and Operation
4) Overall Data Center Impact
This post is simply a collection of the entire series. Enjoy!
Overall Data Center Impact
This is a continuation of our series on Memory in the Data Center, part I, part II, and part III.
The final area to explore is the overall impact these new memory based approaches will have on the data center. In particular we want to keep the major themes in mind of spindle reduction, consolidation, and a need to reduce power, space, and cooling.
Memory in all of its various shapes and forms will be responsible for keeping our data centers from spiraling out of control with disk drive proliferation. Coupling high-speed memory with low-cost, high-capacity drives delivers both the performance and capacity requirements for modern data centers in an efficient footprint.
But as with all data center transformations, this will not happen overnight. And there will be stages of deployment as data center managers find the best way to enhance their current infrastructure with new memory-based solutions.
The most visible aspect of the data transformation will be the spread of silicon (in terms of processing cores and memory) to complement the storage layer. Of the three primary layers in the data center – servers, networks, and storage – the storage component is the last to rely on physical moving parts. Some belie this as “rotating rust” but I prefer to think of it as the most cost effective means of retaining high-capacity persistent data. But today’s robust application servers demand more in terms of IOPS and bandwidth than what a typical disk-based storage system can provide.
Disks are not going away, and neither is tape. But we will see memory advance to complement, and eventually displace, some disk-based systems that have relied on spindles for performance.
The big wins in adding memory to the data center will not come by trying to replace disk, but rather by effectively enhancing it. We often mistake the decline in memory prices as justification that disks will go away sooner, but I see it differently. The decline of memory prices makes the use of both technologies more applicable….performance when needed through memory, and the benefits of a high-capacity, low-cost persistent storage layer for a never-ending amount of content.
Over time (and I use these words carefully because it may be five years, or it might be ten) memory as a persistent storage media will become a more effective means in and of itself. But this will require more in terms of reconfiguring data centers than might be fully realized. New persistent storage systems that rely on memory for persistent storage not only need the basic components in place, but also years of maturity to where all of the exceptions and error codes can be easily handled in a standards-based manner understood by multiple vendors. That day is a bit further off than the current headlines might indicate.
In the meantime, we have new ways to make use of memory that did not exist a few years ago giving us the ability to:
Individually, these capabilities might seem straightforward. Taken together, they expand and open up the architectural options for data center managers. But most importantly, they represent an immediate opportunity for IT professionals to dramatically improve current data center performance without causing unnecessary and premature overhauls of existing data center equipment.
By freeing data center managers from having to buy disks for performance, they will now be able to change the way they purchase and configure storage. The impact will reshape our map of modern data centers.
Final Conclusions
The world is moving away from configuring for performance with disk. As the following chart shows, the spending on performance-optimized storage is declining because customers are tired of paying twice the amount per Gigabyte. And there is an overwhelming trend to move more things to capacity-optimized storage.
Why isn't this happening faster?
This is a continuation from Memory in the Data Center - Part II, and Memory in the Data Center - Part I.
So far we've covered a bit about media, and also architecture. Now let’s talk about
installation, use, and administration across these different approaches.
Perhaps it’s best to step back for a second and look at the differences between the memory-as-disk approach compared to the memory-as-cache approach.
In the memory-as-disk approach administrators must decide what size memory footprint they need. This is far easier said than done because few tools exist to figure out the "active data set" within an existing LUN. So inevitably manual intervention in this process leads to gross over provisioning of memory resources for the entire data set as compared to the active, frequently used data.
The next task is to create and provision LUNs, and then migrate data to the new LUN(s). Once in place, the LUN must be protected with backup and recovery procedures, overall storage management, and the ongoing monitoring to determine if that data set is growing beyond the LUN size. If the data set exceeds the size of the LUN, administrators must be able to rapidly provision additional space (of the same speed and performance) while not disrupting the application.
In the memory-as-caching approach, the goal is to enhance the existing disk infrastructure as a seamless complement that delivers performance without creating additional maintenance and management items.
With caching, particularly when deployed as a network resource, applications request data from any amount of storage capacity, but will “view” that infinite capacity through a caching appliance. The network-based cache only retains the actively used data, dynamically populating the cache based on application requests and letting all unused (but important) data remain on disk-based, protected persistent storage.
By enhancing a traditional or clustered file system with a network-based cache, customers get the capacity depth of a their chosen file system, and the optimized performance of a cache sized perfectly to their active data set.
As workloads shift and change, the intelligence of a network-based caching appliance automatically adjusts to the data I/O patterns. This continuously alleviates hot-spots on and across systems.
Using Technology Efficiently
An important trigger for market growth and adoption is how to use more memory, more effectively in the data center. This memory can be in any shape or form but we need to turn our attention to the system level implementation and specifically how we do use memory to remove action items from the administrative to-do list, not add to it.
Using memory as a persistent storage device will tend to add way more to-do items than it might alleviate. Using memory as an intelligent cache will remove action items and migrate data centers to more automated, dynamic operation.
Stay tuned for the final installment Memory in the Data Center - part IV - Overall Data Center Impact
EMC has finally closed the Iomega acquisition. Erick Schonfeld at TechCrunch has an insightful statement...
"Perhaps EMC could combine Iomega with Mozy, an online storage service it bought last year for $76 million. The trend in consumer storage, as with consumer software, is to offer both online and offline capabilities."
When one company owns everything from the external USB disk drive through online backup, and a suite of services for sharing in between, then we should be backing up more often, transferring data more often, and someone will be profiting more often.
One more example of EMC's current have-no-fear acquisition strategy.
As spotted on GigaOM, Google is hosting a conference for developers of web applications.
Google I/O is a developer gathering focused on pushing the boundaries of web applications using Google and open web technologies. Google engineers and web development leaders will lead you through two days full of in-depth breakout sessions on the latest technologies, hands-on Code Labs, and informal Q&A at Fireside Chats.
Now that is a great conference name! Click here for the conference website.
The amount of activity around web services and cloud computing recently is unbelievable. We had our first cloud computing roundtable recently and are planning more soon.
Here are a few interesting tidbits that popped up in my email box this week.
Massive Memory Systems. Memory is the new disk! ....Disk will become the new tape, and will be used in the
same way, as a sequential storage medium (streaming from disk is
reasonably fast) rather than as a random-access medium (very slow).
Tons of opportunities there to develop new products that can offer
10x-100x performance improvements over the existing ones.
click here for the full blog post from Computing at Scale
High-performance computing, mondo memory and new style applications
Just as storage and networking have been disaggregated from the
computer, some amount of memory and processing, at least for
specialized purposes, may also migrate on to the network
click here for the full blog post from Server Specs: A SearchDataCenter.com blog
AutoTrader plans VMware move from iSCSI to NFS
(more evidence of the NFS train leaving the VMware station...)
AutoTrader.com said it's getting ready to shift VMware data stores from
the iSCSI side to the NFS side of its NetApp 3040 disk arrays, saying
that the move will make VMware storage management easier and more
efficient.
click here for the full story from Beth Pariseau at SearchStorage.com
