Linear Throughput Scaling
With Himalaya, a single system can be scaled across multiple racks to provide up to 10’s of Gigabytes per second of I/O, with near linear scaling of throughput as the system grows. Each individual AmpliStor Controller delivers up to 1 Gigabyte per second of aggregate throughput, enabling a single system rack with multiple controllers to provide multi-gigabytes per second of throughput. Multi-rack systems can be constructed that scale throughput across racks and controllers, to dial-in any level of performance required. Moreover, the system utilizes the power of all disk drives and processors in the “pool” in aggregate, to harness the power of hundreds or thousands of disk spindles working in parallel.
The Bottom Line – eXtreme Throughput
Hands on testing by experts at DeepStorage.net demonstrate Himalaya’s predecessor, AmpliStor, can ingest data at near line speed. The same is true for Himalaya.
They tested the performance and resiliency of the system with 846TB of raw capacity across 24 storage nodes. Below is a summary of their findings.
The Amplistor system:
- Routinely delivered just under 3 Gigabytes per second of sustained aggregate throughput from a single system, requiring less than one data center rack
- Delivered single controller get and put throughput of over 950MBps essentially saturating the 10Gbps
- Delivered aggregate throughput of over 2900MBps across three controllers which is more than 90% linear scalability
- Single object read (GET) performance of over 300MBps, and write (PUT) performance over 350MBps more than twice what a replication based system can deliver
- Maintained 90% GET and 86% of PUT performance while reconstructing data during rebuilding 55 simultaneous disk failures out of a pool of 288 disk drives (19% of the drives)
Read more in the eXtreme performance validation report
Patented BitSpread technology takes full advantage of latest Intel® Xeon® processors
Example: (1 GB/s throughput per controller) x (3 controllers per rack) x (4 racks) = 12 GB/s throughput
Industry’s First Rateless Erasure Coding
This throughput is essential for many of today’s mission-critical applications in media and entertainment, finance and medical industries. Enhancements to Amplidata’s patented BitSpread Erasure Coding software, the industry’s first rateless erasure coding, now allows AmpliStor to drive 10 Gigabit Ethernet network interfaces to full capacity per storage controller. Himalaya-based controllers provide 10 Gigabit Ethernet network interfaces for ingest and read by applications, as well as dedicated 10 Gigabit Ethernet interfaces for back-end storage fabric connectivity. BitSpread fully utilizes the available processing power, including multiple CPU cores and all network bandwidth, to enable these levels of throughput. BitSpread leverages Intel® Xeon® processors to further enhance low-level instruction sets that increase performance.
Himalaya incorporates a new object caching capability that allows Controllers to be configured with variable sized object caches in either solid state (SSD) or hard-drive (HDD) based caches. The capability further optimizes the performance of Himalaya for applications where objects are accessed repeatedly, or for geographically-spread deployments where objects are retrieved over higher-latency (WAN) based networks. This avoids repeated transfers of objects and allows immediate access once objects have been cached in the controllers.