Using a newly developed polarized image sensor, solid-state cameras are now available that allow product analysis and defect detection to be performed more effectively.
Keith Wetzel, Director Of New Product Development Imperx, Inc., 6421 Congress Ave, Suite 105, Boca Raton, FL 33487, USA
Designers of machine vision and image processing systems have multiple choices in choosing which ameras, lighting, computers and I/O peripherals with which to deploy. Indeed, developers can choose from multiple monochrome, color, infrared (IR), UV) and multispectral cameras or combinations of these with which to perform industrial inspection tasks. Perhaps one of the most overlooked of these methodologies is the use of polarization cameras – cameras that do not require polarization filters to analyze products such as plastics and film and can remove any reflected glare on products imaged on the production line.
Today industrial automation systems often use machine vision systems to inspect, measure, sort and analyze products for defects. Such systems can be broadly defined in two classes: those that are either active or passive. While, active machine vision systems may employ structured laser light or pattern projection systems with which to illuminate an object, passive based systems use simpler forms of illumination such as LED front lights, dome lights or backlights with which to illuminate the object to be inspected. In both types of systems, a CCD or CMOS-based camera is used to capture the reflected image data.
Today industrial automation systems often use machine vision systems to inspect, measure, sort and analyze products for defects. Such systems can be broadly defined in two classes: those that are either active or passive.
Monochrome, Color Or Polarized
In such systems, the data captured can be monochrome or color. In many cases, such as part sorting, for example, it may not be necessary to use a color camera if the part is illuminated at the correct wavelength since different parts may exhibit a different grey-scale value when captured by the camera. In other applications, however, for example, if the system needs to determine whether an exact color is present, a color camera must be used.
To obtain such color images, sensor vendors have implemented the Bayer filter (or versions of it) onto their monochrome CCD and CMOS image sensors (Figure 1). Named after Bryce Bayer, the Kodak engineer who invented it, the Bayer filter uses twice as many green (G) elements as red (R) or blue to mimic the physiology of the human eye. In this manner, the color of individual pixels on the image sensor can be interpolated to generate a color image. What is important about Bayer’s invention is that it marked one of the first times that filters were used in image sensors to render color images from monochrome images.
In a development that somewhat mimics this design, engineers at Sony Semiconductor Solutions Corp. (Tokyo, Japan; www.sony-semicon.co.jp) have used a similar concept in the design of the IMX250MZR CMOS-based polarizer image sensor. Instead of filtering the incoming light by wavelength – as is used in the Bayer filter – the sensor filters the incoming light based on the orientation of its vibrations perpendicular to its direction of travel. (Figure 2). Using this imager, a number of companies, including Imperx, have developed polarization cameras that can be used in applications such as machine vision, object recognition and automatic target detection.
Most light sources such as the sun and artificial lighting are un-polarized and consist of electromagnetic waves that oscillate perpendicular to the direction of propagation. A light wave in which vibrations perpendicular to the direction of travel are confined to one plane is said to be polarized.
Many different ways exist to polarize light including the use of dichroic, crystalline and wire grid polarizers. Dichroic polarizers use a laminated dyed polymer film between two polished and anti-reflection coated glass windows. While dichroic polarizers use laminated polymer films, crystalline polarizers take advantage of the birefringence in crystalline materials to modify the incident light. Wire grid polarizers, on the other hand, use an array of microscopic wires such that only a specific polarization is transmitted through the wire grid polarizer and the outgoing wave will have a single linear polarization.
One of the most important characteristics of all polarizers is their extinction ratio. This is the ratio of the power of a plane-polarized beam transmitted through a polarizer compared with the transmitted power when the polarizer filter’s axis is perpendicular to the beam’s polarization plane. For the Sony IMX250MZR, this varies from 40-150 depending on the wavelength of light (Figure 3).
In the design of the Sony IMX250MZR CMOS-based polarizer image sensor, wire grid polarizers are used to capture polarized light.
To do so, four separate 3.45μm pixels are used to filter this light at orientations of 0o, 90o, 45o and 135o angles. In essence, the imager uses lithographically fabricated onchip wire grid polarizers placed above each pixel to filter incoming light.
Placing the polarizer close to the on-chip photodiode improves the extinction ratio (the ratio of minimum to maximum transmission) and incident angle dependence. Data from these four quadrants can then be used to interpolate the direction of linear polarization (DoLP) and the angle of linear polarization (AoLP) using Stokes parameters, originally conceived by Sir George Stokes at Cambridge University in England in 1852.
To calculate both the DoLP and AoLP, it is necessary to first measure the total intensity of light (I). This is accomplished by adding the intensities of the vertically and horizontally polarized pixels.
By also calculating the intensity difference between polarized components of the electromagnetic wave parallel and perpendicular to the reference plane (Q) and the intensity difference between polarized components in planes 45° and −45° to the reference plane (U), both the DoLP and AoLP can be computed using the following equations:
The DoLP provides information about the incident light that is linearly polarized, for example, specular surfaces reflect light with high DoLP values. While most natural objects will be characterized by low DoLP values, man-made objects such as plastic will have high DoLP values. DoLP is a useful measure for other reasons – it can be used to analyze stress-induced birefringence in transparent materials since the birefringence of such material is directly proportional to the DoLP.
Interestingly, AoLP represents the polarization angle of the light as it reaches the camera sensor and provides the angle of the surface from which the light is reflected. If the distance between the camera and the object is known, the 3D shape of the object can then be obtained.
Using the Sony sensor, a number of companies including Imperx have developed solid-state cameras. Imperx’s first camera to leverage this technology is the C2420Z, a Camera Link-based camera that features a native resolution of 2464 x 2056 in a 2/3” optical format that can run at speeds as fast as 97 fps. Data from this camera this then represented with four different polarization angles. Images from each polarization angle can then be viewed or saved in raw data format.
Since specialized software is required to calculate the DoLP and the AoLP, Imperx has enlisted the expertise of Polaris Sensor Technologies (Huntsville, AL; USA; www.polarissensor.com) to develop algorithms to provides a real-time display of polarized images. Using this software, developers can visualize polarized images and calculate parameters such as DoLP and AoLP.
Polarizing cameras can be used in several different applications including material stress analysis. In such applications, light is first passed through a polarizer to produce linearly polarized light. This light is then passed through a birefringent material such as plastic, polystyrene and polycarbonate stress in the material causes the light to be broken up into two polarized components
Components which are parallel and perpendicular to the direction of the stress in the material will lag each other in phase. After these polarized components are captured by the camera, only the part of each of the components in the transmission plane will emerge resulting in two coplanar components with a phase difference. The phase difference is proportional to the stress and results in constructive and destructive interference resulting in an interference pattern of varying colors that can be used to determine the stress at various points in the material (Figure 4).
Such polarization cameras can also be used to reduce the specular glare associated with light being reflected from non-metallic surfaces such as smooth surfaces or surfaces covered with grease, oil or liquid. To reduce this glare, a polarizing filter can be placed over the light source. In the past, it was necessary to place a separate filter over a CCD or CMOS camera to reduce this glare. Now, however, since the polarizing filter is incorporated into the camera, such separate filters is not required.
Figure 5 (a): Reflections from of a sealed cap illuminated by ambient light and a white LED light source. This results in specular glare that renders the cap unreadable. After installing polarizing film over the light source and either using a polarizing filter on a standard monochrome camera or, by using a polarizing camera, will greatly reduce any specular reflections (Image courtesy MidOpt).
Figure 5a shows the reflections from a sealed cap illuminated by ambient light and a white LED light source. This result in specular glare that renders the cap unreadable. After installing polarizing film over the light source and either using a polarizing filter on a standard monochrome camera or, by using a polarizing camera, greatly reduces any specular reflections (Figure 5b).
Similarly, Figure 6 shows the effect of a polarizing filter in reducing glare in light reflected from water. While Figure 6a is the original image, Figure 6b was imaged with a polarizer, eliminating the reflection of clouds and sky observed in
Other applications that can leverage the power of polarization cameras include remote sensing, haze removal and surveillance applications. With the introduction of solidstate cameras that incorporate on-chip polarizing sensors, systems developers no longer need to employ external polarizing filters to solid-state cameras. Interestingly, while sensors such as the Sony IMX250MZR polarize visible light, further developments may lead to imagers that perform the same function in other parts of the electromagnetic spectrum such as infrared (IR). Needless to say, these more specialized imagers will find more niche applications in such areas as security and surveillance.
A: Imperx, Inc, 6421 Congress Ave, Suite 204, Boca Raton, FL 33487, USA
T: +1 561 989 0006