In an industrial environment, parts of different shapes, size, material, and optical properties (reflectance, absorption, etc.) need to be handled every day. These parts have to be picked and placed in a specific orientation for processing, and the automation of these pick and place activities from an environment (containers or other) where the parts are randomly held is generally referred to as bin picking. This task poses the challenge for a robot end effectors (devices that are attached to the end of a robotic arm) to know the exact 3D location, dimensions and orientation of the objects it wants to grip. In order to navigate around the walls of the box and other objects inside the box, the robot’s machine vision system needs to acquire depth information in addition to the 2D camera information.
The challenge of the 3D-image capture of objects for bin-picking can be addressed by a structured light technique. The structured light technology-based 3D scanner/camera works by projecting a series of patterns onto the object being scanned and captures the pattern distortion with a camera or sensor. A triangulation algorithm then calculates the data and outputs a 3D-point cloud. Image processing software like Halcon by MVTech calculates the objects position and the robot arm’s optimal approach path (Figure 1).
Figure 1: Example of matching pipe couplings with their respective 3D model using Halcon (Source: Halcon by MVTech)
DLP technology provides a high speed pattern projection capability through a micro-mirrors matrix assembled on top of a semiconductor chip know as a digital micro-mirror device (DMD) as depicted in Figure 2. Each pixel on the DMD represents one pixel in the projected image, allowing pixel accurate image projection. The micro-mirrors can be transitioned at ~ 3us to reflect the light incident on the object through a projection lens or onto a light dump. The former achieves a bright pixel on the projected scene, while the later create a dark pixel. DLP technology also offers unique advantage with its ability to project patterns over a wide wavelength range (420 nm – 2500 nm) using various light sources like lamps, LEDs and lasers.
DLP technology- powered structured light for bin picking offers several advantages:
Figure 2: DLP chips contain millions of micromirrors that are individually controlled at high speeds and deliberately reflect light to create projected patterns
TI’s evaluation modules for DLP technology enable fast implantation of structured light into the machine vision workflow. To demonstrate this capability, Factory Automation and Control systems engineers mounted a DLP LightCrafter 4500 evaluation board at a set distance and angle to a monochromatic camera. The DLP evaluation board is triggered by the camera through a trigger cable connecting the two to each other; see Figure 3.
Figure 3: Structured light setup including the DLP Products LightCrafter 4500 (left), a Point Grey forward-looking infrared Flea3 camera (right) and calibration (back)
Both the board and camera are connected to a PC via USB, and the whole setup is directed at a calibration board. Software from the Accurate Point Cloud Generation for 3D Machine Vision Applications Using DLP® Technology reference design is then used to calibrate the camera and projector for parameters like focal length, focal point, lens distortion, translation and rotation of the camera relative to the calibration board. The reference design user guide walks through the process step by step.
Recalibration is only necessary if the camera is moved relative to the DLP products board.
After the setup is complete, it’s possible to create point clouds of real-world targets. Those clouds are output by the software in an arbitrary file format, which is then read and displayed by some brief code developed in Halcon’s HDevelop platform. Figure 4 shows a point cloud with color coding for depth information taken of a box filled with coffee mugs.
Figure 4: Captured mug (left) and acquired point cloud acquired with DLP powered structured light of several mugs in a box displayed in Halcon HDevelop (right)
Halcon’s surface matching can determine the mug’s 3D pose by comparing the point cloud with a 3D CAD model of a mug. Doing this the robot arm now “sees” the object and the optimal approach path of the robot arm can be calculated allowing it to pick objects from boxes avoiding obstacles in an unstructured and changing environment.
All content and materials on this site are provided "as is". TI and its respective suppliers and providers of content make no representations about the suitability of these materials for any purpose and disclaim all warranties and conditions with regard to these materials, including but not limited to all implied warranties and conditions of merchantability, fitness for a particular purpose, title and non-infringement of any third party intellectual property right. No license, either express or implied, by estoppel or otherwise, is granted by TI. Use of the information on this site may require a license from a third party, or a license from TI.
TI is a global semiconductor design and manufacturing company. Innovate with 100,000+ analog ICs andembedded processors, along with software, tools and the industry’s largest sales/support staff.