Image Capturing and Processing

Innovations in industrial machine vision are constantly trying to further increase performance levels while opening up new technologies to the mainstream. At the same time, there is also an increasing demand for industrial real-time capabilities of vision processes in order to be executed within a fixed minimal timeframe.

Kithara RealTime Suite adresses these key issues and enables the development of cost-efficient as well as performance-efficient machine vision solutions for industrial fields such as automated quality assurance, robotics, autonomous cars or medical technology. This includes image capture, image processing as well as subsequent control reactions all in the same real-time context.

For industrial machine vision two camera standards have proven to be particularly trendsetting and future-proof—GigE Vision® and USB3 Vision®. Both standards feature a similar set of benefits that make them highly suitable for a vast range of industrial fields and applications. Besides high data rates, both interfaces can also be optimally used for almost any task due to inexpensive and interchangeable standard hardware. Both interfaces can be configured with GenICam 2.0 which allows for the utilization of default functions as well as access to manufacturer-specific features.

Machine vision in real time

Task and solution

In machine vision, Windows is often essential, due to most vision applications running primarily on Windows OS. However, without real-time capabilities, developers are usually confronted with certain issues:

  • Individual data packets of GigE Vision or USB3 Vision cameras are not transferred, therefore, images are corrupted or only partial transferred.
  • The reaction time is delayed by the Windows application and is therefore, too long and not deterministic, thus causing issues for subsequent processes.

An appication example for the Camera Module of Kithara RealTime Suite is quality assurance in the packaging industry. In this field the assessment whether a product meets the specification or not has to take place in a very short amount of time in order to remove an unsuitable workpiece from the production line. The Camera Module of Kithara RealTime Suite solves this task impressively. To avoid the loss of data packets the Camera Module provides large-sized memory buffer and real-time network or XHCI drivers. The reaction to received GigE Vision or USB3 Vision images immediately takes place in a real-time context within a few microseconds. This also allows for immediate control reactions by triggering subsequent real-time tasks.

The need for high-performance real-time systems in machine vision is especially apparent with high resolution images at high frame rates, where regular Windows functions bottleneck the processing of incoming data packages.

Machine vision in real time

Modules

Real-time machine vision with the modular system of Kithara RealTime Suite consists of the following components:

Camera Module

Image acquisition with GigE Vision and USB3 Vision cameras in real-time

  • Common
  • Features
  • Extensions

The Camera Module serves the purpose of capturing image data of industrial cameras according to GigE Vision or USB3 Vision standards in real time. It provides all necessary functions for detection and management and for creating real-time streams.

Large buffer memory along with real-time network drivers reliably prevent any loss of data packets. The reaction to incoming GigE Vision or USB3 Vision images takes place instantly in the context of the real-time environment within a few microseconds. An immediate real-time task can be triggered, in order to execute an instant control reaction.

Thanks to the real-time drivers, the Camera Module allows for maximal data throughputs according to interface and system configuration, even up to the theoretical maximum. For the real-time processing of image data, refer to Halcon and OpenCV extension.

The results of the image processing can be given to the process even in the real-time context without any delay, e. g. over EtherCAT or CANopen. Accordingly, automation solutions can be easily combined with several other functions within Kithara RealTime Suite.

The Camera Module requires Network Module for GigE Vision or USB RealTime Module for USB3 Vision respectively.

Maximum of two Cameras, above requires Camera Device Extension.

Requires Network Module and/or USB RealTime Module

General features

  • Image acquisition with GigE Vision or USB3 Vision cameras in real time
  • Event or request based
  • Multiple simultaneous cameras, hot-plugging, configuration in accordance with GenICam 2.0
  • Every GigE Vision compliant camera is supported

Control

  • Automatic connection handling
  • Full access to camera configuration space
  • Read and write camera memory
  • Set custom heartbeat timeout
  • Start and stop acquisition of images by software
  • Set acquisition mode by software
  • GenICam-XML configuration file automatically downloaded
  • Access to GenICam features to configure your camera
  • Error handling

Streams

  • One or multiple streams per camera possible
  • Acquisition of images in real time
  • Memory-efficient image capture, no unnecessary copying
  • High-performance acquisition, no unnecessary copying
  • Set your own buffer size and buffer count
  • Provides information about missed or incomplete images

GenICam features

  • Numbers all categories
  • Set and retrieve all accessible camera features like image format, pixel format, frame rate, exposure time and more

Image Processing

For the processing of image data see: Halcon Extension or OpenCV Extension

Camera Device Extension

Extension for using additional Cameras

  • Extended Level: up to 8
  • Unlimited Level (requires Extended Level): unlimited

Camera Acceleration Extension

The Camera Acceleration Extension provides efficient real-time drivers for the GigE Vision framegrabber acceleration card PGC-1000 by PLC2. Embedded in Kithara RealTime Suite, the PGC-1000 is able to almost completely offload the CPU for image acquisition processes within machine vision applications. This allows, for instance, to simultaneously capture up to four GigE Vision camera streams each with 10 Gbit/s in a real-time context. The real-time synchronization of multiple cameras via the PTP Module by Kithara is also possible.

  • Real-time support for PGC-1000
  • Almost completely offloads the CPU for GigE Vision image acquisition
  • Connection speeds of up to 4 × 10 GBit/s or 64 × 1 GBit/s
  • Synchronisation of multiple cameras via PTP Module

Halcon Extension

Real-time image processing with Halcon

  • Common
  • Features

Halcon by Munich company MVTec is a program library with algorithms for image processing and machine vision. It includes its own integrated development environment HDevelop as well as over 2000 operators. The Halcon Extension allows for the implementation of this program library into the Kithara real-time system. This enables the triggering of immediate control reactions to processed images as well as their integration into automation systems with EtherCAT or CANopen.

The Halcon license needs to be acquired separately from MVTec.

  • Real-time image processing with Halcon
  • Over 2000 operators, including matching, segmentation, 3D matching, OCR, deep learning and more
  • Integrated development environment HDevelop
  • Automatic parallelization
  • Execute the HDevEngine directly from the real-time context
  • Immediate control reaction to processed image data, for example with EtherCAT or CANopen
  • Supports Halcon Versions 10 to 13, 18.11 Steady as well as 20.11 Steady
  • The Halcon Extension is an extension of the Library Module

OpenCV Extension

Real-time image processing with OpenCV

  • Common
  • Features

OpenCV is a Open Source program library with algorithms for image processing and machine vision. Among others it includes algorithms for face recognition, 3D functionality, hair segmentation, various fast filters as well as functions for camera calibration. The OpenCV Extension allows for the implementation of this program library into the Kithara real-time system. This enables the triggering of immediate control reactions to processed images as well as their integration into automation systems with EtherCAT or CANopen.

  • Real-time image processing with OpenCV
  • Algorithms for face recognition, 3D functionality, hair segmentation, various fast filters as well as functions for camera calibration as well as deep learning
  • Automatic parallelization
  • Immediate control reaction to processed image data, for example with EtherCAT or CANopen
  • Supports OpenCV Versions OpenCV 3.0, 3.4 and 4.1
  • The OpenCV Extension is an extension of the Library Module

EtherCAT® and Safety over EtherCAT® are registered trademarks and patented technologies, licensed by Beckhoff Automation GmbH, Germany.
CANopen® is a registered trademark of the CAN in Automation e. V.
GigE Vision® is a registered trademark of the Automated Imaging Association.
USB3 Vision® is a registered trademark of the Automated Imaging Association.