Optimizing Video Data Storage in a Virtualized Environment – Intro to SCRIBE

BCDVideo’s Hyperconverged Infrastructure for Video Surveillance (HCI-VS), powered by Scale Computing, utilizes the Scale Computing Reliable Independent Block Engine (SCRIBE) embedded in each node to manage all storage as one logical pool.  The real benefits of SCRIBE come from the intelligent distribution of blocks redundantly across the cluster to maximize availability and performance for each virtual machine.

Unlike many other hyperconverged infrastructures that require a tier of flash/SSD storage in each node, HCI-VS is designed to operate entirely with spinning hard drives for greater storage scalability. The HCI-VS leverages SCRIBE’s wide striping to distribute I/O and capacity across the entire cluster, achieving levels of performance that are industry leading for video surveillance.

How it works
HCI-Data-Redundancy
The SCRIBE storage engine allocates storage for block I/O requests in chunks ranging from as large as 1MB to as small as a 512 byte disk sector. SCRIBE also ensures that two or more copies of every chunk are written to the storage pool in a manner that not only creates the required level of redundancy (equivalent to a RAID 10 approach) but also aggregates the I/O and throughput capabilities of all the individual disks in the cluster, commonly known as wide striping.

As data is received from the LAN to the cluster, data is written at the block level as the VM issues and receives I/O requests from the network. SCRIBE utilizes the network backplane to wide stripe data across the cluster, isolating that traffic from the LAN network resources.

For example, 10GB of data is being written to a cluster of 3 BCDVideo BCD218X-HCI nodes which contain a total of (18) or (9) hard drives. In this example, each node has (18) 12TB spinning disks, thus contributing approximately 200TB of RAW storage capacity per node, or 100TB of effective usable storage space to the cluster after accounting for two copies of all data blocks being stored.

That 10GB of data (or 10240MB) would require 10240 chunks (1MB each) to store the first copy, plus 10240 chunks to store a second copy of each chunk. Not factoring in other I/O that may be occurring at the same time, you could picture each of the 18 disks doing approximately 1150 1MB writes in order to store that 10 GB redundantly across the HC3 cluster.

The benefits
Of course, other I/O is likely going on in the cluster at the same time, further leveraging the I/O capabilities and throughput of all the disks in the cluster. This means that the more nodes you add to the cluster, the more aggregate performance you are able to achieve. The benefits of wide striping across a large number of disks apply to both read and write I/O.

In addition, video data is uniquely large, complex, and cumbersome. In a traditional set-up, performance is limited by the performance of disks in a RAID set, as well as the RAID controller itself. For high performance, the RAID controller would need a large cache to reach maximum levels of bandwidth per server. The HCI-VS eliminates both bottlenecks, as RAID controllers and sets are not used, and SCRIBE writes data directly to disk at the block level. This combination of features enables virtualization to be optimized for video data, resulting in lower latency and more efficient use of hardware.

Source: bcdvideo.com
0 Comments