Primary Studies on the real.iZ VS-1000 3D Vision System

This document is the primary studies on the real.iZ VS-1000 3D Vision System for fruit detection. Please go to manuals and odos imaging for more information and always regard the manuals as your prioritized reference.

The contents are listed as follows:

  1. structure and operating principles
  2. features
  3. indoor experiment
  4. outdoor experiments
  5. range photos processing
  6. portable solutions
  7. future challenges
  8. help seeking

1.structure and operating principles

Picture1

The yellow outlined part is the illumination module which can send out laser. The blue one is the sensor module, the main part of the camera, which can receive the laser. The basic principle of measuring the range is time-of-flight.

Basic operating steps are like this:

(1)fix three parts and lens

if you do not need to move the 3 parts often, you can just assemble them together like the upper picture.

Remember to use a hex key (you can find some in a small package in the box) to disassemble them. See Figure 5.3 in realiz-1kVS_technical_manual .

Then mount the lens on the camera. Kindly note that the ring coming along with the lens may not be mounted. Otherwise you will get blur photos.

(2)connect the cables together

Altogether there are two cables for power supply and two cables for data transfer for illumination parts, and the example connecting picture is shown as blow.

camera2

As for the sensor module, connect it with the Ethernet cable to your laptop and the power cable line to 110v outlet.

  • Please note that all the cable lines have a connector pin like the picture shown below. So please make sure the direction of the pin is correct before plugging in! Otherwise, twist the connector and make sure it can fit.

camera3

(3)open odim_streamer.exe

Basic operations and parameter adjusting can be achieved by odim_streamer.exe. A basic operation guide can be found in part.3 Quickstart Guide in realiz-1kVS_user_interface_manual.

The other two tools in the same file may not be used often unless you are required to. Their functions are illustrated as blow:

  • interface_firmware_updater.exe – the real.iZ VS-1000 system utilises a 3rd party GigE interface board (from www.pleora.com). This utility is used to upgrade the firmware on this interface board.
  • odim_file_uploader.exe – this utility is used to upgrade the operating firmware contained within the real.iZ VS-1000.

Open odim_streamer.exe, select and connect the device.

After press ‘Play‘ button, you may click create display to get a display window and have a view of the photos taken continuously by default.

If you want to save photos and .txt files, please press ‘Streamer’ button then ‘save’ then click ‘enable’, and here you can choose and click what you want to save in files. Usually Intensity and Range are choosed for components, BMP and TXT are clicked for format.

Parameters can be adjusted mostly in Device Control. And you can choose levels from beginner to expert or guru to control more parameters.

  • One important parameter is the ComponentTupleRateActual, which reflects the frame rate. The speed is not only related to ComponentTupleRate, but also related to Scan3dAccumulationCount and the active Region of Interest (ROI) utilised in the camera (described in the User Interface Manual, Section 5.2, Configuring the Image Format). By default the system is set to 1280×1024 resolution – maximum resolution but slowest rate of image capture. Reducing this (to, eg 1024×768) or (800×600) will have a very positive effect on frame rate.

Here are some suggested reduced ROI parameters setting:

    • 1024 x 768 (OffsetX and OffsetY are both 128 to maintain the ROI in the centre of the image)
    • 800 x 600 (OffsetX = 240 and OffsetY = 212 to maintain ROI in the centre of the image)

As for Scan3dAccumulationCount, you may do some experiments like adjusting ROI and Count at the same time observing the ComponentTupleRateActual and decide what parameter combination you would like to use for a balance between the speed and accuracy.

  • Another parameter you may use often is single, multiple or continuous tuple. This can decide how many photos you want to save to files.
  • If you would like to take both intensity photos and range photos, please also set range parameters referring to Table 5.3 in realiz-1kVS_user_interface_manual, especially AcquisitionComponentSelector and AcquisitionComponentEnable parameters.
  • Scan3dAxisMax and Scan3dAxisMin are simply numerical thresholds that are applied to the range data. Use of these does not have any impact on the range capability of the system. Please refer to Sections 8.5.2 and 8.5.4 of the User Interface Manual. They should simply be used to eliminate range data from objects that are outwith those thresholds.

Maximum range capability of the system is a function of a number of factors, although most simply can be described as the ability of the system to detect the reflection of a pulse from the target object. In practice, this will be in the region of 4-6 meters, however very reflective objects may be detected a longer ranges, and very non-reflective objects may not be detected at closer ranges.

2.features

You may refer to the company’s website for detailed features like precision, etc.

3.indoor experiment

camera4

Arrange the objects in different distance. And set the parameters so that you can get both intensity and range photos at an acceptable frame rate and press ‘play’.

You may first use display window to have a quick view at the view and adjust the lens’ focus so that the intensity photo is clear. Please make sure to keep the aperture open to full.

You may set how many photos you want and also its ROI. After that, you can save the photos and .txt file in your laptop.

This is what I got for this specific experiment:

camera5intensity photo

camera6range photo

And also a .txt file is saved at the same time including intensity and range information, which may be used in the image processing part.

4.outdoor experiment

Outdoor fruit photos are what the project require for.

One important thing about outdoor experiments is the strong light. You are strongly recommended to keep the aperture open to full so that the sensor module can accept as much laser as possible, therefore can improve its range performance. Instead of changing aperture, you may adjust exposure time to a smaller amount to get acceptable intensity photos.

Also filter MidOpt bp880 is suggested by the odos imaging Support for outdoor use.

So here are some outdoor photos I took (original photos are very large, these are processed ones):

camera7

camera8

5.range photos processing

To make the range photos easy to read for human, that is to say, you can have an idea of how far away one object is just by looking at the image with color bar, so we need to use Matlab to process the images.

Here is my major programming idea:

camera9

And an easy example is like this:

A=load(‘119791934839.txt’);
A=(A(:,3)*0.000426365-9.89156)/2;
A1=reshape(A,1280,1024);
A2=A1′;
A3=(A2+abs(A2))/2;
imagesc(A3)

And after adding the color bar and setting the color map, you may get this:

camera10

camera11

And you can read the distance of an object by reading the color bar on its side. You may also add more color to the tree so that you can get more details on the range photos by setting suitable color map. You may also consider color coordination for a sense of aethetics. 🙂

And kindly note that, when calculating the physical distance by using .txt file, please note that whilst the output is consistent across both 10 and 16 bit pixels, in 10 bit mode, the numerical values for both range and intensity are indeed shifted by 6 bits (decimal 64), greatly quantising the values for each pixel. We would therefore not recommend use of .txt output with anything other than 16 bit pixels.

The use of the binary output is more recommended by odos imaging Support for futher study. Better still, is the use of your own acquisition software.

6.portable solutions

To take outdoor photos, batteries and open ended power cables are required.

Open ended power cables can be ordered through the odos company. These are optionally available in 3m, 6m and 9m lengths – they have a Binder 99 0410 00 04 connector on one end, and are open ended on the other for connection to a 12V supply.

And suitable rechargeable batteries that can provide at least 6A at 12V, and have substantial charge (so they can power up the system for a long time) are required for portable use.
The reason the battery should support at least 6A is that you need to power 15+2*25=65W at 12V -> ~5.5A. To be on the safe side you want your battery to support at least 6A (and ideally higher, say ~10A).

And a camera pedestal may be needed for convenience.

7.future challenges

As far as it goes, incapablity of auto focusing may be one of the major future challenges. The real.iZ VS-1000 camera is a precision industrial camera, and as such has no ability to provide an “auto-focus” control to a motorised lens (this feature is commonly found on many consumer cameras).

And challenge may be that there are some invalid pixels on the range photo whose R value cannot reflect its real distance. Therefore either take an average or fix by their neighbor pixels may be one of the solutions.

Also, a lot of experiments need to be done such as finding the balance between frame rate and precision, and measuring how accurate range photos are.

Kindly note that an easy way to do the measurement is to find a large flat piece of object, put it about one meter away from the camera and let the object occupy most of the range photos so that you can easily pick a pixel to calculate. There may be some invalid pixels you may not choose for calculation. Kindly note that before calculating the distance, make sure the precision is what you want, otherwise set Scan3dAccumulationCount and ROI.

odim::streamer is a simple application that is designed only to show the most basic functionality of our system, and to quickly confirm interface to the computer. The source code for odim::streamer is included in the software examples also on the USB stick. The odim_samples in the U disk are C++ / .NET source code. As per Section 4.1 of the User Interface Manual, the following development environments are supported:

  • Visual Studio 8, Visual Studio 9, Visual Studio 10
  • .NET using Visual Studio 10, Version 4.0 .NET framework
  • GCC Red Hat Enterprise Linux 6 (other linux distros may also be compatible)

Also, the VS-1000 has an industry standard GigE/Genicam interface, and is therefore compatible with a wide range of image processing libraries (eg Labview/Matlab, OpenCV, Halcon, Matrox MIL, etc), both Matlab and Labview are commonly used in University environments.

8.help seeking

You may send an email to odos imaging Support([email protected]asking for help and usually within 2 days they will reply professionally to you with patience. If you have questions about this document, you may send an email to me. My email is [email protected].

Thank you very much for your time reading this document.

Hu Xinyu (Cher)