INFINIDAT White Paper - Storage Architecture - 161007 - A4

INFINIDAT
Storage
Architecture
White Paper
Abstract
The INFINIDAT enterprise storage solution is based upon the unique and patented
INFINIDAT storage architecture. The INFINIDAT storage architecture is a fully abstracted
set of software-driven storage functions layered on top of off-the-shelf hardware. By
shipping the software with a highly tested hardware reference platform, INFINIDAT can
deliver the first true enterprise-class software-defined storage.
This paper discusses how INFINIDAT achieves multi-petabyte capacity in a single 42U rack,
mainframe-class reliability with an unprecedented 99.99999% (7 nines) availability, and
over 1M IOPS of performance with sub-microsecond latency.
2
Introduction
The INFINIDAT enterprise storage solution is based upon the unique and patented
INFINIDAT storage architecture. The INFINIDAT storage architecture is a fully abstracted
set of software-driven storage functions layered on top of off-the-shelf hardware. The
result is multi-petabyte capacity in a single 42U rack, mainframe-class reliability with
an unprecedented 99.99999% (7 nines) availability, and over 1M IOPS of performance.
Automated provisioning, management, and application integration provide a system
that is incredibly efficient and easy to use. By separating the storage innovation from the
hardware, INFINIDAT allows for the rapid adoption of the latest and most cost-effective
hardware. In addition, by shipping the software with a highly tested hardware reference
platform, INFINIDAT can deliver the first true enterprise-class software-defined storage.
INFINIDAT Hardware
INFINIIDAT is built as a mesh architecture. Our current-generation products use three
nodes (N=3) per rack. These nodes represent the intelligence in the system. Each
independent node contains a server, DRAM, and flash cache:
• Flash Cache – up to 200TB
• RAM – up to 3TB
HOST CON NEC TIV IT Y
INFINIDAT includes three connectivity layers:
• Front end — FC and Ethernet connectivity for client access,
Server Server
replication, and management
Server
RAM RAM RAM
• High-speed InfiniBand — Low latency node-to-node communication
Flash Flash Flash
• Back end — SAS connectivity from each node to all the drive
Cache Cache Cache
enclosures in the system
ALL NOD ES ARE CON NEC TED TO AL L DRIVES
3
Availability
The architecture was designed to sustain multiple failures without disrupting client
operations. At the hardware connectivity level, this is done with the following redundancy:
• All front-end connections are redundant, with floating IP addresses and
software-defined FC targets, which can failover between physical ports/nodes
• InfiniBand disconnection is solved by routing the InfiniBand messages through
another node
• SAS connectivity failures redirect IO to the InfiniBand link as a temporary path of
accessing drives via another node
Below the node layer are eight drive enclosures. The drive enclosures can be filled to
include 480 drives of up to 8TB each. All nodes utilize all drives and disk drive failures
are handled by our de-clustered RAID data layout called InfiniRaid™. This architecture
guarantees that all the nodes receive an even load. In the event of a drive failure, all the
remaining drives work in parallel to support the restore process.
INFINIDAT Storage Architecture
H O S T CO NNEC TI V I T Y
Server Server Server

RAM RAM RAM
Flash Flash Flash
Cache Cache Cache
AL L N O D E S ARE CO NNEC TED TO ALL D RI V ES
4
Caching
Each of the nodes in the system is equipped with multiple SSD drives, which act as a
global cache for all the nodes (no data block is cached twice). A read IO sent to node 1
can get a cache hit from node 2’s SSDs to minimize latency. With up to 200TB of SSD
cache per system, InfiniBox can cache almost all the active dataset and avoid getting
reads from slow, spinning drives.
Logical Architecture
INFINIDAT’s logical architecture leverages small sections of disks as RAID members
(instead of the entire disk). This minimizes disk contention and allows each of the nodes
in the system to use all the drives all the time for maximum performance.
Incoming Data
Incoming Data
14 data sections 2 parity sections
As data comes into the system it is aggregated into 14 optimized sections, each with its
own DIF (data integrity field) and lost-write protection field. These protect the data from
logical corruptions as well as disk-level errors. InfiniBox then adds two parity sections to
complete a RAID 14 stripe.
data sections 2 parity sections
The data is then sent to a group of 16 disks, with each RAID stripe always landing in
different disks. This minimizes the common data between any two drives, accelerating
double-failure RAID rebuilds. This also guarantees no hot spots can be created, as all
data is evenly spread across all the disks in the system. We also do this without any
hot spares.
5
As a result, if the system sustains a double drive failure, it can immediately calculate
which RAID stripes were shared by these two drives and rebuild the RAID groups with the
higher risk first. Since the recovery is performed from all the drives at the same time, the
system can rapidly recover (typically less than 10 minutes) from a double disk failure to
protected mode.
By designing both the software and hardware stacks to be highly resilient, the INFINIDAT
storage architecture is built to provide an unprecedented 7 nines availability (99.99999%
uptime).
Capacity
The InfiniBox is able to store an unprecedented amount of capacity in a single 42U
rack. This is a function of two things. First, as a part of our innovative data layout we
are able to achieve as much as 72% capacity utilization within the system. Secondly,
we have architected an intelligent compression architecture. INFINIDAT’s compression
architecture allows you store more data on the InfiniBox with absolutely no impact to
I/O latency. Our flagship system is capable of storing multiple petabytes of effective
capacity in a single 42U rack (assuming a conservative 2:1 compression ratio).
Performance
With almost all the reads coming from SSD, the spinning drives run at a low utilization,
offering the little IO that doesn’t hit the cache a low latency response.
Writes are buffered in a large protected memory cache that sustains all the changes to
the blocks before they “cool down” enough to be sent to disk. When writes are finally sent
to the disks, they are written to all the disks simultaneously, giving the system the ability
to write at full drive speed to all the drives. Writes that are being accessed (hot data) are
also written to the SSD cache layer to accelerate reads.
Conclusion
The INFINIDAT storage architecture was designed for the most critical of business
applications. The architecture supports an extremely high level of availability, has a
very small footprint, and is extremely high performance. The architecture also allows for
virtually unlimited scalability and carefully balances functionality and cost. The INFINIDAT
storage architecture is at the heart of the InfiniBox family of storage arrays. Unlike
traditional storage architectures, this new architecture is designed to eliminate storage
trade-offs.
www.infinidat.com | [email protected]
W P - S T O R A R C - 1 6 1 0 0 7 | © I N F I N I D A T 2 0 16 6

INFINIDAT White Paper - Storage Architecture - 161007 - A4

Uploaded by

Copyright:

Available Formats

INFINIDAT White Paper - Storage Architecture - 161007 - A4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

INFINIDAT White Paper - Storage Architecture - 161007 - A4

Uploaded by

Copyright:

Available Formats

INFINIDAT

ALL NOD ES ARE CON NEC TED TO AL L DRIVES

INFINIDAT Storage Architecture

Server Server Server

AL L N O D E S ARE CO NNEC TED TO ALL D RI V ES

14 data sections 2 parity sections

You might also like