Real-Time Data Storage

In many industrial fields, big data has become increasingly important. Numerous applications already benefit from ever-growing data volumes, while others have only become possible because of them. Especially in measurement and testing, these data can be used effectively to generate highly precise results and evaluations.

In addition to data acquisition, storage is a crucial factor for efficient processing. Large data sets must not only be written to storage media reliably and at high throughput rates, but often also need to be read back at the same speed. Based on the Storage Module, Kithara therefore focuses specifically on real-time data storage to ensure reliable integration into demanding applications.

The Storage Module enables real-time data storage on SSDs via the NVMe interface. Data can be written and read continuously at throughput rates of several gigabytes per second. The UDF file system is also supported, allowing efficient packet writing.

With RAID (Redundant Array of Independent Disks), the already high throughput can be increased even further by combining multiple SSDs. With RAID 0, the achievable data rate scales with the number of SSDs used and can reach the double-digit Gbyte/s range. Read and write operations remain within the real-time context of Kithara RealTime Suite.

For efficient storage of measurement data, MDF (Measurement Data Format) is available directly within the real-time environment. Support for MDF 4.2 enables large volumes of complex, hierarchically structured measurement data to be stored and accessed immediately. Losslessly captured raw measurement data can therefore be made available directly as MDF files for further processing. MDF is a de facto standard for measurement applications, particularly in the automotive industry, and is designed for fast and space-efficient storage of recorded data. The format also provides high-performance functions for efficient sorting and compression of data sets.

The Wireshark-supported PCAPng file format can also be used. In combination with real-time capabilities, network packets can be captured directly and stored for later evaluation. The recorded data can then be analyzed conveniently with Wireshark.

Modules

Real-time data storage with the modular system of Kithara RealTime Suite consists of the following components:

Storage Module

Real-time data storage on SSD drives

  • Common
  • Features
  • Supported Hardware
  • Extensions

The Storage Module introduces real-time data storage via SSD with the most recent capacities. This way, via the NVMe interface, large amounts of data can be written as well as read with speeds of several Gbyte/s. UDF is provided as a file system, which also allows for packet writing.

Maximum of one storage device; additional devices require the Storage Device Extension.

  • Real-time data storage via NVMe SSD with the most recent capacities
  • High-speed reading and writing with several Gbyte/s
  • UDF file system
  • Requires RealTime Tasking Module

In connection with the Storage Module, all storage devices that are compatible with the NVMe specifications 1.0 to 1.4 can be used on request.

  • Intel NVMe SSD P3700/P3600/P3520/750 Series
  • Intel NVMe SSD DC P3520 Series
  • Intel NVMe SSD DC P4511 Series
  • Intel NVMe SSD 600p Series
  • Samsung NVMe SSD 950 Series
  • Samsung NVMe SSD 960 Series
  • Samsung NVMe SSD 970 Series
  • Samsung NVMe SSD 980 Series
  • Samsung NVMe SSD 980 PRO Series
  • Corsair NVMe SSD MP510 Force Series
  • HGST NVMe SSD Ultrastar SN200/SN260 Series
  • Micron NVMe SSD 9300 MAX
  • Seagate NVMe FireCuda 530
  • Toshiba NVMe SSD XG6
  • VirtualBox NVMe disk

Storage Device Extension

Extension for using additional Storage Devices

  • Extended Level: up to 4
  • Unlimited Level: unlimited

Storage RAID Extension

Real-time data storage with multiple NVMe SSDs in RAID 0 groups, multiplied storage capacity and throughput rates.

PCAPng Module

Real-time data storage in PCAPng format

  • Common
  • Features

The PCAPng Module (PCAP Next Generation) allows for real-time data storage of complex hierarchically structured measurement data in PCAPng format. PCAPng is a Wireshark-supported data format for capturing data packets within networks. This way, Wireshark is used for the later analysis of stored measurement data.

  • Real-time data storage in PCAPng format
  • Wireshark used for reading stored data sets such as measurement data or even image data
  • Requires Storage Module

MDF Module

Real-time data storage in MDF

  • Common
  • Features

The MDF Module allows for real-time data storage of complex hierarchically structured measurement data in Measurement Data Format. MDF 4.2 is a binary file format provided by ASAM for the evaluation or long-term storage of recorded and calculated data sets. The file size depends solely on the hardware used (allowing file sizes in the terabyte range).

  • Real-time data storage in MDF
  • Support for MDF 4.2, downward compatible to earlier versions
  • Writing of files with virtually any size (264 Bytes)
  • Storing of raw messages from the busses CAN, LIN, FlexRay and automotive Ethernet
  • Requires Storage Module