Optical sorting

Last updated

Optical sorting (sometimes called digital sorting) is the automated process of sorting solid products using cameras and/or lasers.

Contents

Depending on the types of sensors used and the software-driven intelligence of the image processing system, optical sorters can recognize an object's color, size, shape, structural properties and chemical composition. [1] The sorter compares objects to user-defined accept/reject criteria to identify and remove defective products and foreign material (FM) from the production line, or to separate product of different grades or types of materials.

Optical sorters are in widespread use in the food industry worldwide, with the highest adoption in processing harvested foods such as potatoes, fruits, vegetables and nuts where it achieves non-destructive, 100 percent inspection in-line at full production volumes.[ citation needed ] The technology is also used in pharmaceutical manufacturing and nutraceutical manufacturing, tobacco processing, waste recycling and other industries. Compared to manual sorting, which is subjective and inconsistent, optical sorting helps improve product quality, maximize throughput and increase yields while reducing labor costs. [2]

History

Optical sorting is an idea that first came out of the desire to automate industrial sorting of agricultural goods like fruits and vegetables. [3] Before automated optical sorting technology was conceived in the 1930s, companies like Unitec were producing wooden machinery to assist in the mechanical sorting of fruit processing. [3] In 1931, a company known as “the Electric Sorting Company” was incorporated and began the creation of the world’s first color sorters, which were being installed and used in Michigan’s bean industry by 1932. [4] In 1937, optical sorting technology had advanced to allow for systems based on a two-color principle of selection. [4] The next few decades saw the installation of new and improved sorting mechanisms, like gravity feed systems and the implementation of optical sorting in more agricultural industries. [5]

In the late 1960s, optical sorting began to be implemented to new industries beyond agriculture, like the sorting of ferrous and non-ferrous metals. [6] By the 1990s, optical sorting was being used heavily in the sorting of solid wastes. [6]

With the large technological revolution happening in the late 1990s and early 2000s, optical sorters were being made more efficient via the implementation of new optical sensors, like CCD, UV, and IR cameras. [5] Today, optical sorting is used in a wide variety of industries and, as such, is implemented with a varying selection of mechanisms to assist in that specific sorter’s task.

The sorting system

Optical sorting achieves non-destructive, 100 percent inspection in-line at full production volumes. SortingPotatoStrips.jpg
Optical sorting achieves non-destructive, 100 percent inspection in-line at full production volumes.

In general, optical sorters feature four major components: the feed system, the optical system, image processing software, and the separation system. [7] The objective of the feed system is to spread products into a uniform monolayer so products are presented to the optical system evenly, without clumps, at a constant velocity. The optical system includes lights and sensors housed above and/or below the flow of the objects being inspected. The image processing system compares objects to user-defined accept/reject thresholds to classify objects and actuate the separation system. The separation system — usually compressed air for small products and mechanical devices for larger products, like whole potatoes — pinpoints objects while in-air and deflects the objects to remove into a reject chute while the good product continues along its normal trajectory.

The ideal sorter to use depends on the application. Therefore, the product's characteristics and the user's objectives determine the ideal sensors, software-driven capabilities and mechanical platform.

Sensors

Optical sorters require a combination of lights and sensors to illuminate and capture images of the objects so the images can be processed. The processed images will determine if the material should be accepted or rejected.

There are camera sorters, laser sorters and sorters that feature a combination of the two on one platform. Lights, cameras, lasers and laser sensors can be designed to function within visible light wavelengths as well as the infrared (IR) and ultraviolet (UV) spectrums. The optimal wavelengths for each application maximize the contrast between the objects to be separated. Cameras and laser sensors can differ in spatial resolution, with higher resolutions enabling the sorter to detect and remove smaller defects.

Cameras

Shape sorting enables the detection of same-color defects and foreign material. SortingGreenBeans.jpg
Shape sorting enables the detection of same-color defects and foreign material.

Monochromatic cameras detect shades of gray from black to white and can be effective when sorting products with high-contrast defects.

Sophisticated color cameras with high color resolution are capable of detecting millions of colors to better distinguish more subtle color defects. Trichromatic color cameras (also called three-channel cameras) divide light into three bands, which can include red, green and/or blue within the visible spectrum as well as IR and UV. The interaction of different materials with parts of the electromagnetic spectrum make these contrasts more evident than how they appear to the naked human eye.

Coupled with intelligent software, sorters that feature cameras are capable of recognizing each object's color, size and shape; as well as the color, size, shape and location of a defect on a product. Some intelligent sorters even allow the user to define a defective product based on the total defective surface area of any given object.

Lasers

While cameras capture product information based primarily on material reflectance, lasers and their sensors are able to distinguish a material's structural properties along with their color. This structural property inspection allows lasers to detect a wide range of organic and inorganic foreign material such as insects, glass, metal, sticks, rocks and plastic; even if they are the same color as the good product.

Lasers can be designed to operate within specific wavelengths of light; whether on the visible spectrum or beyond. [8] For example, lasers can detect chlorophyll by stimulating fluorescence using specific wavelengths; which is a process that is very effective for removing foreign material from green vegetables. [9]

Camera/laser combinations

Sorters equipped with cameras and lasers on one platform are generally capable of identifying the widest variety of attributes. Cameras are often better at recognizing color, size and shape while laser sensors identify differences in structural properties to maximize foreign material detection and removal.

Hyperspectral Imaging

Hyperspectral image of "sugar end" potato strips shows invisible defects. Hyperspectral image of "sugar end" potato strips.jpg
Hyperspectral image of "sugar end" potato strips shows invisible defects.

Driven by the need to solve previously impossible sorting challenges, a new generation of sorters that feature multispectral and hyperspectral imaging Optical Sorters. [10]

Like trichromatic cameras, multispectral and hyperspectral cameras collect data from the electromagnetic spectrum. Unlike trichromatic cameras, which divide light into three bands, hyperspectral systems can divide light into hundreds of narrow bands over a continuous range that covers a vast portion of the electromagnetic spectrum. This opens the door for more detailed analysis that leads to a more consistent product. Using IR alone might detect some defects, but combining it with a broader range of the spectrum makes it more effective. Compared to the three data points per pixel collected by trichromatic cameras, hyperspectral cameras can collect hundreds of data points per pixel, which are combined to create a unique spectral signature (also called a fingerprint) for each object. When complemented by capable software intelligence, a hyperspectral sorter processes those fingerprints to enable sorting on the chemical composition of the product. This is an emerging area of chemometrics.

Software-driven intelligence

Once the sensors capture the object's response to the energy source, image processing is used to manipulate the raw data. The image processing extracts and categorizes information about specific features. The user then defines accept/reject thresholds that are used to determine what is good and bad in the raw data flow. The art and science of image processing lies in developing algorithms that maximize the effectiveness of the sorter while presenting a simple user-interface to the operator.

Object-based recognition is a classic example of software-driven intelligence. It allows the user to define a defective product based on where a defect lies on the product and/or the total defective surface area of an object. It offers more control in defining a wider range of defective products. When used to control the sorter's ejection system, it can improve the accuracy of ejecting defective products. This improves product quality and increases yields.

New software-driven capabilities are constantly being developed to address the specific needs of various applications. As computing hardware becomes more powerful, new software-driven advancements become possible. Some of these advancements enhance the effectiveness of sorters to achieve better results while others enable completely new sorting decisions to be made.

Platforms

The considerations that determine the ideal platform for a specific application include the nature of the product – large or small, wet or dry, fragile or unbreakable, round or easy to stabilize – and the user's objectives. In general, products smaller than a grain of rice and as large as whole potatoes can be sorted. Throughputs range from less than 2 metric tons of product per hour on low-capacity sorters to more than 35 metric tons of product per hour on high-capacity sorters.

Channel sorters

The simplest optical sorters are channel sorters, a type of color sorter that can be effective for products that are small, hard, and dry with a consistent size and shape; such as rice and seeds. For these products, channel sorters offer an affordable solution and ease of use with a small footprint. Channel sorters feature monochromatic or color cameras and remove defects and foreign material based only on differences in color.

For products that cannot be handled by a channel sorter – such as soft, wet, or nonhomogeneous products – and for processors that want more control over the quality of their product, freefall sorters (also called waterfall or gravity-fed sorters), chute-fed, sorters or belt sorters are more ideal. These more sophisticated sorters often feature advanced cameras and/or lasers that, when complemented by capable software intelligence, detect objects' size, shape, color, structural properties, and chemical composition.

Freefall and chute-fed sorters

Freefall sorters inspect product in-air during the freefall and chute-fed sorters stabilize product on a chute prior to in-air inspection. The major advantages of freefall and chute-fed sorters, compared to belt sorters, are a lower price point and lower maintenance. These sorters are often most suitable for nuts and berries as well as frozen and dried fruits, vegetables, potato strips and seafood, in addition to waste recycling applications that require mid-volume throughputs.

Belt sorters

Optical sorters can function within visible light wavelengths as well as the IR and UV spectrums. SortingPeas.jpg
Optical sorters can function within visible light wavelengths as well as the IR and UV spectrums.

Belt sorting platforms are often preferred for higher capacity applications such as vegetable and potato products prior to canning, freezing or drying. The products are often stabilized on a conveyor belt prior to inspection. Some belt sorters inspect products from above the belt, while other sorters also send products off of the belt for an in-air inspection. These sorters can either be designed to achieve traditional two-way sorting or three-way sorting if two ejector systems with three outfeed streams are equipped.

ADR systems

A fifth type of sorting platform, called an automated defect removal (ADR) system, is specifically for potato strips (French fries). Unlike other sorters that eject products with defects from the production line, ADR systems identify defects and actually cut the defects from the strips. The combination of an ADR system followed by a mechanical nubbin grader is another type of optical sorting system because it uses optical sensors to identify and remove defects.

Single-file inspection systems

The platforms described above all operate with materials in bulk; meaning they do not need the materials to be in a single-file to be inspected. In contrast, a sixth type of platform, used in the pharmaceutical industry, is a single-file optical inspection system. These sorters are effective in removing foreign objects based on differences in size, shape and color. They are not as popular as the other platforms due to decreased efficiency.

Mechanical graders

For products that require sorting only by size, mechanical grading systems are used because sensors and image processing software is not necessary. These mechanical grading systems are sometimes referred to as sorting systems, but should not be confused with optical sorters that feature sensors and image processing systems.

Practical usage

Waste and recycling

Optical sorting machines can be used to identify and discard manufacturing waste, such as metals, drywall, cardboard, and various plastics. [11] In the metal industry, optical sorting machines are used to discard plastics, glass, wood, and other non-needed metals. [12] The plastic industry uses optical sorting machines to not only discard various materials like those listed, but also different types of plastics. Optical sorting machines discard different types of plastics by distinguishing resin types. Resin types that optical sorting machines can identify are: HDPE, PVC, PLA, PE, and others. [12]

Optical sorting also aids in recycling since the discarded materials are stored in bins. Once a bin is full of a given material, it can be sent to the appropriate recycling facility. [13] Optical sorting machines’ ability to distinguish between resin types also aids in the process of plastic recycling because there are different methods used for each plastic type. [14]

Food and drink

In the coffee industry, optical sorting machines are used to identify and remove underdeveloped coffee beans called quakers; quakers are beans that contain mostly carbohydrates and sugars. [15] A more accurate calibration offers a lower total number of defective products. [15] Some coffee companies like Counter Culture use these machines in addition to pre-existing sorting methods in order to create a better tasting cup of coffee. [15] One limitation is that someone has to program these machines by hand to identify defective products. [15]

However, this science is not limited to coffee beans; food items such as mustard seeds, fruits, wheat, and hemp can all be processed through optical sorting machines. [16]

In the wine manufacturing process, grapes and berries are sorted like coffee beans. [17] Grape sorting is used to ensure no unripe/green parts to the plant are involved in the wine making process. [17] In the past, manual sorting via sorting tables was used to separate the defective grapes from the more effective grapes. [17] Now, mechanical harvesting provides a higher effectiveness rate compared to manual sorting. [17] At different points in the line, materials are sorted out via several optical sorting machines. [17] Each machine is looking for various materials of differing shapes and sizes. [17]

The berries or grapes can then be sorted accordingly using a camera, a laser, or a form of LED technology with regard to the shape and form of the given fruit. The sorting machine then discards any unnecessary elements. [18]

[19]

See also

Related Research Articles

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Machine vision</span> Technology and methods used to provide imaging-based automatic inspection and analysis

Machine vision is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to many technologies, software and hardware products, integrated systems, actions, methods and expertise. Machine vision as a systems engineering discipline can be considered distinct from computer vision, a form of computer science. It attempts to integrate existing technologies in new ways and apply them to solve real world problems. The term is the prevalent one for these functions in industrial automation environments but is also used for these functions in other environment vehicle guidance.

<span class="mw-page-title-main">Thermographic camera</span> Imaging device using infrared radiation

A thermographic camera is a device that creates an image using infrared (IR) radiation, similar to a normal camera that forms an image using visible light. Instead of the 400–700 nanometre (nm) range of the visible light camera, infrared cameras are sensitive to wavelengths from about 1,000 nm to about 14,000 nm (14 μm). The practice of capturing and analyzing the data they provide is called thermography.

<span class="mw-page-title-main">Thermography</span> Infrared imaging used to reveal temperature

Infrared thermography (IRT), thermal video and/or thermal imaging, is a process where a thermal camera captures and creates an image of an object by using infrared radiation emitted from the object in a process, which are examples of infrared imaging science. Thermographic cameras usually detect radiation in the long-infrared range of the electromagnetic spectrum and produce images of that radiation, called thermograms. Since infrared radiation is emitted by all objects with a temperature above absolute zero according to the black body radiation law, thermography makes it possible to see one's environment with or without visible illumination. The amount of radiation emitted by an object increases with temperature; therefore, thermography allows one to see variations in temperature. When viewed through a thermal imaging camera, warm objects stand out well against cooler backgrounds; humans and other warm-blooded animals become easily visible against the environment, day or night. As a result, thermography is particularly useful to the military and other users of surveillance cameras.

<span class="mw-page-title-main">Image scanner</span> Device that optically scans images, printed text

An image scanner—often abbreviated to just scanner—is a device that optically scans images, printed text, handwriting or an object and converts it to a digital image. Commonly used in offices are variations of the desktop flatbed scanner where the document is placed on a glass window for scanning. Hand-held scanners, where the device is moved by hand, have evolved from text scanning "wands" to 3D scanners used for industrial design, reverse engineering, test and measurement, orthotics, gaming and other applications. Mechanically driven scanners that move the document are typically used for large-format documents, where a flatbed design would be impractical.

<span class="mw-page-title-main">Konica Minolta</span> Japanese technology company

Konica Minolta, Inc. is a Japanese multinational technology company headquartered in Marunouchi, Chiyoda, Tokyo, with offices in 49 countries worldwide. The company manufactures business and industrial imaging products, including copiers, laser printers, multi-functional peripherals (MFPs) and digital print systems for the production printing market. Konica Minolta's Managed Print Service (MPS) is called Optimised Print Services. The company also makes optical devices, including lenses and LCD film; medical and graphic imaging products, such as X-ray image processing systems, colour proofing systems, and X-ray film; photometers, 3-D digitizers, and other sensing products; and textile printers. It once had camera and photo operations inherited from Konica and Minolta but they were sold in 2006 to Sony, with Sony's Alpha series being the successor SLR division brand.

<span class="mw-page-title-main">Coordinate-measuring machine</span> Device for measuring the geometry of objects

A coordinate-measuring machine (CMM) is a device that measures the geometry of physical objects by sensing discrete points on the surface of the object with a probe. Various types of probes are used in CMMs, the most common being mechanical and laser sensors, though optical and white light sensors do exist. Depending on the machine, the probe position may be manually controlled by an operator, or it may be computer controlled. CMMs typically specify a probe's position in terms of its displacement from a reference position in a three-dimensional Cartesian coordinate system. In addition to moving the probe along the X, Y, and Z axes, many machines also allow the probe angle to be controlled to allow measurement of surfaces that would otherwise be unreachable.

A staring array, also known as staring-plane array or focal-plane array (FPA), is an image sensor consisting of an array of light-sensing pixels at the focal plane of a lens. FPAs are used most commonly for imaging purposes, but can also be used for non-imaging purposes such as spectrometry, LIDAR, and wave-front sensing.

<span class="mw-page-title-main">Automated guided vehicle</span> Type of portable robot

An automated guided vehicle (AGV), different from an autonomous mobile robot (AMR), is a portable robot that follows along marked long lines or wires on the floor, or uses radio waves, vision cameras, magnets, or lasers for navigation. They are most often used in industrial applications to transport heavy materials around a large industrial building, such as a factory or warehouse. Application of the automatic guided vehicle broadened during the late 20th century.

<span class="mw-page-title-main">Hyperspectral imaging</span> Multi-wavelength imaging method

Hyperspectral imaging collects and processes information from across the electromagnetic spectrum. The goal of hyperspectral imaging is to obtain the spectrum for each pixel in the image of a scene, with the purpose of finding objects, identifying materials, or detecting processes. There are three general types of spectral imagers. There are push broom scanners and the related whisk broom scanners, which read images over time, band sequential scanners, which acquire images of an area at different wavelengths, and snapshot hyperspectral imagers, which uses a staring array to generate an image in an instant.

The following are common definitions related to the machine vision field.

<span class="mw-page-title-main">Waste sorting</span> Environmental practice of separating waste categories to make it easy to recycle

Waste sorting is the process by which waste is separated into different elements. Waste sorting can occur manually at the household and collected through curbside collection schemes, or automatically separated in materials recovery facilities or mechanical biological treatment systems. Hand sorting was the first method used in the history of waste sorting. Waste can also be sorted in a civic amenity site.

Automated X-ray inspection (AXI) is a technology based on the same principles as automated optical inspection (AOI). It uses X-rays as its source, instead of visible light, to automatically inspect features, which are typically hidden from view.

Electro-optical MASINT is a subdiscipline of Measurement and Signature Intelligence, (MASINT) and refers to intelligence gathering activities which bring together disparate elements that do not fit within the definitions of Signals Intelligence (SIGINT), Imagery Intelligence (IMINT), or Human Intelligence (HUMINT).

Thermographic inspection refers to the nondestructive testing (NDT) of parts, materials or systems through the imaging of the temperature fields, gradients and/or patterns ("thermograms") at the object's surface. It is distinguished from medical thermography by the subjects being examined: thermographic inspection generally examines inanimate objects, while medical thermography generally examines living organisms. Generally, thermographic inspection is performed using an infrared sensor.

<span class="mw-page-title-main">Finger tracking</span> High-resolution technique in gesture recognition and image processing

In the field of gesture recognition and image processing, finger tracking is a high-resolution technique developed in 1969 that is employed to know the consecutive position of the fingers of the user and hence represent objects in 3D. In addition to that, the finger tracking technique is used as a tool of the computer, acting as an external device in our computer, similar to a keyboard and a mouse.

<span class="mw-page-title-main">Sensor-based sorting</span>

Sensor-based sorting, is an umbrella term for all applications in which particles are detected using a sensor technique and rejected by an amplified mechanical, hydraulic or pneumatic process.

<span class="mw-page-title-main">Colour sorter</span> Machines that separate items by colours

Colour sorters or color sorters are machines used on production lines in bulk food processing and other industries. They sort items by color, detecting passing items' colors and using mechanical or pneumatic devices to divert items with colors outside the acceptable range or to create distinct groups.

Active thermography is an advanced nondestructive testing procedure, which uses a thermography measurement of a tested material thermal response after its external excitation. This principle can be used also for non-contact infrared non-destructive testing (IRNDT) of materials.

<span class="mw-page-title-main">Aphelion (software)</span> Image processing and analysis software suite

The Aphelion Imaging Software Suite is a software suite that includes three base products - Aphelion Lab, Aphelion Dev, and Aphelion SDK for addressing image processing and image analysis applications. The suite also includes a set of extension programs to implement specific vertical applications that benefit from imaging techniques.

References

  1. Computer vision technology for food quality evaluation. Sun, Da-Wen. (1st ed.). Amsterdam: Elsevier/Academic Press. 2008. ISBN   978-0-12-373642-0. OCLC   228148344.{{cite book}}: CS1 maint: others (link)
  2. Nondestructive testing of food quality. Irudayaraj, Joseph, 1961-, Reh, Christoph. (1st ed.). Ames, Iowa: Blackwell Pub./IFT Press. 2008. ISBN   978-0-470-38828-0. OCLC   236187975.{{cite book}}: CS1 maint: others (link)
  3. 1 2 "About us". Unitec Group. Archived from the original on 2017-03-01. Retrieved 2020-04-02.
  4. 1 2 "Optical Sorting Technology Explained - Satake USA". www.satake-usa.com. Archived from the original on 2019-10-24. Retrieved 2020-04-02.
  5. 1 2 "Optical Sorting Systems - Satake USA". www.satake-usa.com. Archived from the original on 2014-11-27. Retrieved 2020-04-02.
  6. 1 2 "The History Of MSS | Optical Sorting & Recycling Technology". Archived from the original on 2019-04-26. Retrieved 2020-04-02.
  7. Detecting foreign bodies in food. Edwards, M. C. (Michael Charles). Boca Raton: CRC Press. 2004. ISBN   1-85573-839-2. OCLC   56123328.{{cite book}}: CS1 maint: others (link)
  8. US 5675419,van den Bergh, Herman; Lane, Marvin& Mallon, John,"Scattered/transmitted light information system",published 1997-10-07
  9. Zheng, Hong; Lu, Hongfei; Zheng, Yueping; Lou, Heqiang; Chen, Cuiqin (2010-12-01). "Automatic sorting of Chinese jujube (Zizyphus jujuba Mill. cv. 'hongxing') using chlorophyll fluorescence and support vector machine" . Journal of Food Engineering. 101 (4): 402–408. doi:10.1016/j.jfoodeng.2010.07.028. ISSN   0260-8774. Archived from the original on 2015-09-24. Retrieved 2013-12-03.
  10. "StackPath". www.vision-systems.com. Archived from the original on 2013-12-08. Retrieved 2020-03-24.
  11. US EPA, OLEM (2015-05-26). "Typical Wastes Generated by Industry Sectors". US EPA. Archived from the original on 2015-06-22. Retrieved 2020-04-02.
  12. 1 2 "Optical Sorting Applications And Sorting Machines | MSS". Archived from the original on 2020-03-24. Retrieved 2020-04-02.
  13. "A Look at Optical Sorting". Waste360. 2015-09-10. Archived from the original on 2015-10-13. Retrieved 2020-04-02.
  14. "Plastic recycling", Wikipedia, 2020-04-01, archived from the original on 2023-08-13, retrieved 2020-04-02
  15. 1 2 3 4 "Optical Sorting Q&A". Counter Culture Coffee. 2018-09-21. Archived from the original on 2020-03-24. Retrieved 2020-04-02.
  16. "Intel Seed Ltd. | Grain and Forage Seed Manitoba | Seed Cleaning and Conditioning Manitoba | HOME". www.intelseed.ca. Archived from the original on 2014-10-26. Retrieved 2020-04-02.
  17. 1 2 3 4 5 6 Beyerer, Jürgen; León, Fernando Puente (2013). OCM 2013 - Optical Characterization of Materials - conference proceedings. KIT Scientific Publishing. ISBN   978-3-86644-965-7. Archived from the original on 2023-08-13. Retrieved 2020-12-07.
  18. Lafontaine, Magali; Freund, Maximilian; Vieth, Kai-Uwe; Negara, Christian (2013-12-01). "Automatic Fruit Sorting by Non-Destructive Determination of Quality Parameters Using Visible/Near Infrared to Improve Wine Quality: I. Red Wine Production". NIR News. 24 (8): 6–8. doi:10.1255/nirn.1403. S2CID   95941922.
  19. Optical Inspection, Measurement and Sorting Machine for Fasteners, Washer, Screw, Bol, Nut, Precious Parts, Insert, O Ring etc., archived from the original on 2023-03-16, retrieved 2023-03-16