March 2001
An Overview of Machine Vision
By John F. Reid
Contributing Editor
Machine vision is the use of digital imaging for measurement, inspection and control. Machine-vision applications use a material’s optical properties to provide information that can define material characteristics. For food products and biological materials, the optical properties may be related to the product’s physical characteristics, such as size, shape, projected area, etc.
The primary elements of a machine-vision sensing system are the scene, the illumination system, the image of the scene, a description of the scene formed from the image and an application response to the description formed from the image. Image formation is a process that converts the scene into a digital image for computer processing. Image analysis uses computer-implemented operations to transform the raw image into a scene description based on higher-order measures representing characteristics of the digital image.
These elements form a model by which a machine-vision application can be subdivided into smaller tasks.
Looking at light
Machine vision is a unique sensing method because excitation energy can be regulated and even structured in a pattern to get a response from the biological material in the scene. The illumination system is designed to consider the spectral quality of the illumination and the spatial distribution of illumination across the scene.
The spectral properties of a biological material can be used to identify regions where the material exhibits a unique electromagnetic response. The illumination system provides electromagnetic radiation to excite the image sensor to form an image. Configuring the elements of a machine-vision application requires an understanding of the biological material’s electromagnetic response, spectral characteristics of illumination sources and spectral sensitivities of machine-vision sensors. The engineer needs to match these elements so that the image measures the desired biological material characteristics.
The most common illumination sources are incandescent and fluorescent lights. Some applications, such as microscopy, select special illumination sources to generate a specific type of response from the biological material. These illumination sources may even be outside the visible spectrum. Some examples include ultraviolet light, infrared light and X-rays. In precision agriculture applications, the illumination may come from ambient conditions (sun light).
The spatial distribution requirements for illumination are an important characteristic of an application. The position of light sources relative to the viewed object and the image sensor forms two general categories of illumination: frontlighting and backlighting. In backlighting, the scene is located between the light source and the image sensor. The images formed by backlighting are silhouettes of the scene or indicate the transmitted light through objects. In frontlighting, the light source is positioned on the same side of the scene as the image sensor. Images formed in frontlighting are a result of the bi-directional reflectance properties of the scene, which depends on surface properties, light source and light receiver. Grading fruit or vegetables for surface defects is an example where frontlighting might be used.
Structured lighting results from generating light in the form of dots, lines or patterns that can be interpreted to represent changes in the features of the scene. Light striping is a type of structured lighting that is used to measure object thickness. A low-power laser beam is dispersed into a line of light that is projected across the surface of an object. Triangulation between the camera, the light source and the object is used to compute the object’s profile based on the displacement of the laser line pattern.
Proper construction of the illumination system can simplify the image-processing techniques needed in an application. Providing a method of imaging the essential food product information can save computer time in determining product quality.
Making the scene
The scene is defined by the image sensor’s field-of-view. Scene complexity can be classified as either a controlled or an uncontrolled environment. In a controlled environment, there is control over many of the environmental variables like illumination, image contrast, position and orientation of objects or knowledge of the objects imaged that can simplify the image processing. As an example, a vision system to inspect carrots for surface defects could be set up with precise light-level control and knowledge that the only objects passing under the camera are singulated carrots on a conveyor belt. The application is controlled in the sense that information about the types of objects in the scene simplifies the types of information that can be extracted from the images.
An uncontrolled environment is a scene where there is less precise knowledge of the scene’s characteristics, such as unregulated illumination or uncertainty about the number or location of objects in the scene, unknown scene boundaries. Most applications in this latter category are the challenging applications for machine vision.
A good example of a machine-vision application is a sensor developed to sense information for selective harvest for green asparagus in the field. The machine-vision sensor is part of a system to determine if spears are harvestable and to determine the precise location of spears in the image. This application represents an uncontrolled environment since the types of objects that could show up in the image are not controlled very well. It is common for applications to have a mixture of controlled and uncontrolled variables. A practical objective for engineering design of machine vision applications is to structure the environment to be controlled, if possible.
Optical characteristics of the biological material are important in the selection of both the illumination source and the image sensor. An understanding of these scene properties can be used to simplify the image processing requirements of an application.
Image, formation and sensors
An image sensor measures a response from the scene in the form of a two-dimensional (2-D) digital image. A signal transmission technique sends the analog image to a digitizer to convert the signal in a form that can be processed by a computer. The array of image pixels form a signal that can be used to measure photometric and morphometric characteristics of objects within the image. Photometric characteristics are properties of image pixels that are related to the response to the illumination. Morphometric characteristics are size and shape characteristics of objects represented by collections of pixels in the image.
Two common sensors used in machine-vision applications are monochrome cameras and color cameras. Monochrome cameras are used to measure an image intensity response most often related to the reflectance or transmitted light from a scene. Camera technology for forming the image may be based on vacuum-tube sensors or solid-state sensing elements. The former are best for applications requiring high-signal resolution, while the latter are known for their ruggedness and durability. Solid-state cameras typically are sensitive to light beyond the visible spectrum into the near infrared. Optical filters may be used with cameras to isolate portions of the spectrum where a scene has a response that simplifies the machine-vision application.
Color camera systems also are available in tube and solid-state forms. A typical goal of color sensing is to mimic the perceived color response that human vision observes. For example, color could be used to detect debris in a process line of fruits or vegetables. Numerous industrial color sorters are based on this principle. Color also can be detected by using a black and white sensor with bandpass filters for specific portions of the visible spectrum. A vector color signal generally is formed from the response of red, green and blue sensors in the camera.
Lenses and filters are an important element for the image sensor. Lenses control the portion of the scene, or field-of-view, that is projected onto the camera’s sensing elements. Filters control the spectral characteristics of the illumination sensed by the camera. For many applications, optical filters are important for processing the image before it strikes the photosensitive digital array.
Most applications require some knowledge of the geometrical relationship between the image sensor and the scene. Spatial calibration permits the measurement of the scene’s characteristics from the image. Intensity, or color calibration, characterizes the response of individual pixels.
After the signal is sensed at the camera, it is transmitted to an image digitizer to form an array of numbers representing the picture. Many systems utilize the transmission standards developed for television. However, other systems use custom- transmission schemes to meet speed or resolution requirements.
An A/D converter processes the transmitted signal from the camera and converts it into an array of picture elements (pixels) representing the 2-D scene projection. The varying signal between the synchronizing pulses describes the analog intensities for one line of the image. A digitizer breaks this line up into pixels having a digital representation of intensity.
The primary considerations of the selection of the image digitization system are the spatial resolution and the signal resolution. The spatial resolution is the number of pixels used to represent the 2-D image. The image is arranged into a number of rows and columns of cells, typically representing a rectangular portion of the projected scene. Each cell of the image holds a value proportional to the response of the camera at that particular location in the image sensor.
In monochrome imaging systems, each pixel holds a single value called a gray level. In color systems, pixel values are vector values representing the response of sensing elements representing the red, green and blue portions of the visible spectrum. Another way to think of a color image is a gray level image of the red, green and blue sensor responses. Colors other than red, green and blue are represented by mixing the proper proportions of these primary color signals.
Commercial vision systems have spatial resolutions up to 1024 pixels by 1024 pixels in an image. Many vision systems have 256 gray levels for each pixel. A color vision system would have 3 values per pixel. A single 1024 pixels by 1024 pixels color image could occupy over 3 MB of memory. Increased spatial (256 pixels by 240 pixels vs. 1024 pixels by 768 pixels) and intensity resolution (monochrome vs. color) generally increase image storage requirements and image processing time.
The final analysis
The goal of image processing in a machine-vision application is to extract information from the image that can be used in the application response or understanding of the scene. Image analysis operations can be performed in hardware or software. Hardware operations have the advantage of speed, but many applications are developed with general hardware operations and software to define the processing operations. Computer programming skills are needed to perform image processing. Many commercial vision systems come with some basic operations programmed for application. Newer products are using innovative technology to help minimize the programming requirements by using a windowing environment and object-oriented operations.
The raw image captured by the vision sensor is enhanced by image processing, which simplifies the image so that features related to the desired signal are enhanced. Typically, the data goes through a reduction process where the enhanced image is represented in a simpler form. One method of representation is the calculation of connected pixel properties. Represented features are used to make decisions from the image. Features that represent useful scene properties have to be classified based on the arrangement of pixels and pixel values in the image. A representation process in image processing transforms the image from a connected set of pixels into higher-level descriptions of size and shape (area, length, location, elongation, shape factors, etc.).
Image-processing operations for image enhancement can be categorized as point, frame, region and geometric operations based on the computations that take place in the image. Point operations change the output pixel of an image based on the pixel’s intensity value. Frame operations take the values of pixels from the same location in multiple images and perform an operation to create a value for the same pixel in an output image. Region-processing operations create a new value for a pixel based on the values of pixels in some defined neighborhood around a pixel. Geometric operations alter images based on a geometric transformation.
Image classification techniques can be very simple rules based on observed image properties or can be more sophisticated relationships derived from pattern recognition or neural network analysis of image features. An engineer has to be able to select the key features that will make the sensor operate successfully.
Applying information
The application response uses the information from images for sorting products, evaluating biological material properties and detecting the occurrence of events or process control. In measurement, information can be collected on the photometric or morphometric characteristics of the image. This information is often an essential element to learn the characteristics that are useful for an automated system. In inspection or control, the image information is used to actuate a device or another appropriate action to the information detected in the image. Engineering this area requires an understanding of controls, actuators, computer interfacing, and the interaction of biological materials with mechanical systems.
Applied machine vision for biological systems can provide new and innovative sensors for inspection, measurement and control. This technology integrates many types of engineering expertise together. Some useful areas of study for machine vision practitioners include computer programming, artificial intelligence, image processing, optical sensing, controls, computer interfacing and mechanical systems design. Additional skills for biological applications include physical properties of biological materials.
John F. Reid, Ph.D. served on the University of Illinois faculty for 14 years, where he developed the field of applied machine vision. Reid’s applications of machine vision have included quality evaluation of food processes, automated assessment of the state of bioprocesses, grain quality evaluation and plant health sensing. Additionally, he has used machine vision in a number of robotic applications, including selective harvesting of crops and vehicle navigation. Reid is the manager of Technology Development Support at the John Deere Technical Center, Moline, IL.
Back to top
3400 Dundee Rd. Suite #100
Northbrook, IL 60062
Phone: 847-559-0385
Fax: 847-559-0389
E-Mail: info@foodproductdesign.com
Website: www.foodproductdesign.com
|
SupplySide
