The Application and Prospects of Machine Vision in Fruit Quality Inspection and Grading
Traditional manual fruit sorting is being replaced by machine vision technology, ushering in a revolution in efficiency in modern agriculture.
At an agricultural production, education, and research base in Zhongzhuang Town, Yiyuan County, Shandong Province, boxes of apples of varying shades of green, red, and size are automatically sorted by an intelligent device. This device accurately sorts and removes diseased and inferior fruit based on size, sugar content, and color.
This machine vision-based fruit inspection system is gradually changing the traditional practice of fruit farmers relying on experience and touch to determine fruit grade, becoming a crucial component of agricultural modernization.
1 Technological Transformation in Fruit Quality Inspection
The global population is growing exponentially at an annual rate of approximately 1.09%, leading to an increase in demand for food and other basic necessities. Against this backdrop, reducing postharvest losses has become a key challenge in the agricultural sector.
Fruits and vegetables are particularly beneficial to humans, providing a variety of vitamins, minerals, and antioxidants. However, their perishable nature makes efficient and proper handling crucial to prevent spoilage. Sorting and grading are the most critical, difficult, and time-consuming steps in the post-harvest chain.
Traditional manual screening methods are highly susceptible to fruit damage and are only suitable for small-scale operations. With population growth and resource dwindling, agricultural production urgently needs more efficient and accurate quality inspection technologies.
Machine vision technology has emerged as a response to this need. It uses computers to replicate human vision, replacing the human eye's perception of the objective three-dimensional world. This interdisciplinary field, encompassing artificial intelligence, neurobiology, psychophysics, computer science, image processing, and pattern recognition, offers new solutions for fruit quality inspection.
Fruit quality inspection primarily encompasses both external and internal quality. Traditional external quality inspection primarily utilizes grading machines, which grade fruit based on metrics such as size and weight. However, this method cannot accurately assess color, texture, and surface defects.
With the advancement of machine vision technology, computer vision-based systems have gained significant attention for fruit quality assessment and grading. These technologies are efficient, fast, consistent, time-saving, reliable, and cost-effective, enabling product processing tailored to market needs. Once developed, they require little to no specialized knowledge and can be applied to large-scale production.
2 Core Technical Methods of Machine Vision
Machine vision fruit inspection systems typically consist of two core subsystems: a defect detection system and a mechanical sorting system. In terms of hardware architecture, the transport and sorting module uses a motor-driven conveyor belt to transport the fruit, while a robotic arm connected to a servo motor sorts the fruit into corresponding bins based on the inspection results.
The image acquisition module uses a color camera, coupled with an LED light source to eliminate shadows, to capture RGB images of the fruit in a closed environment. A microcontroller coordinates image processing results with mechanical movements to achieve closed-loop control.
Technical approaches are primarily categorized into traditional image processing algorithms and deep learning methods.
Image processing solutions pre-process the RGB image into a color space such as grayscale or HSV to optimize lighting consistency. Then, threshold segmentation is performed to extract the target area. Morphological operations such as dilation and erosion are then used to remove noise and enhance contours. Finally, the defect area percentage is calculated.
For example, in apple defect detection, the system determines the image processing window, uses the Sobel operator and Hilditch to refine edges, and identifies the centroid point to represent the fruit diameter, thereby detecting the overall size and appearance. Deep learning solutions combine public datasets with self-collected images to build a training library and improve model generalization through data augmentation techniques such as rotation, flipping, and blurring. Customized convolutional network structures can be designed to address the characteristics of different fruits.
In a study conducted at the Faisalabad University of Agriculture, customized convolutional networks were designed for mangoes and tomatoes, respectively: the mango model used a 7-layer convolutional structure, and the tomato model used a 5-layer convolutional structure, both using a softmax classifier for output.
In recent years, the YOLO series of algorithms has become a new choice for fruit detection. By optimizing the backbone network structure and introducing a dynamic attention mechanism, YOLOv8 can more accurately capture changes in fruit skin texture, color differences, and morphological characteristics, significantly improving the accuracy of identifying rotten areas.
The latest YOLOv10 even eliminates the need for non-maximum suppression (NMS), reducing computational overhead and further improving detection efficiency.
3 Technical Advantages and Breakthroughs
Compared to traditional manual inspection, machine vision systems offer multiple technical advantages. In terms of inspection efficiency, a four-channel citrus machine can process 12 to 15 tons of fruit in an hour, equivalent to nearly a week's work for one worker in the past.
In terms of inspection accuracy, a CNN-based deep learning model has a verified accuracy rate of 95% for mango defect detection, and a 93.5% for tomato. In actual applications, the intelligent sorting equipment has an overall accuracy rate of 97% for surface defect detection, and 95% for internal quality inspection.
The machine vision system possesses multi-parameter inspection capabilities, capable of simultaneously evaluating multiple parameters of fruit characteristics, including size, shape, color, and surface defects.
For size detection, researchers translate and rotate the fruit to obtain images at different angles, calculate the fruit's equatorial radius and area, and estimate its size by treating the fruit as an ellipsoid.
For color detection, some fruits have a single color evenly distributed across the skin (primary color), while others (such as peaches, apples, and tomatoes) have secondary colors that can serve as a good indicator of ripeness.
Surface defect detection is another advantage of machine vision systems, which can detect surface blemishes, damage, and scratches on fruit. For example, the reddish-brown color of Golden Delicious apples can be detected and classified using a specific algorithm.
The economic benefits are equally significant. After implementing intelligent sorting equipment, processing costs for enterprises have been significantly reduced, from 600-800 yuan per ton to 100 yuan per ton, a cost reduction of over 80%. This not only improves agricultural production efficiency but also brings tangible economic benefits to fruit farmers.
4 Practical Application Cases
The application of machine vision in fruit inspection has been effective in many core fruit-producing areas across China. In Wuming District, Nanning, Guangxi, a core production area for Wogan oranges, intelligent equipment has successfully addressed the need to classify Wogan oranges based on external defects.
Local manufacturers previously relied on workers to visually sort Wogan oranges, which was inefficient and unable to identify internal lesions. Intelligent equipment can distinguish between "rough-skinned fruit," "ulcerated fruit," and "sun-shaped fruit," significantly improving the standardization of Wogan oranges and making Wuming Wogan a nationally renowned fruit brand.
Regarding apple inspection, researchers have developed a system specifically for inspecting and grading apples based on their external quality. The system first captures three images covering the entire surface of the apple and then extracts surface features.
The apple's shape is described using the Fourier operator, and a neural network based on the L-M algorithm is used to grade the apples by shape. Color detection converts the image's RGB values into a histogram (HIS) pattern, generating a chromaticity histogram. A particle swarm optimization algorithm is then used to optimize the neural network for color grading.
A research team at the University of Agriculture, Faisalabad, developed specialized inspection systems tailored to the characteristics of mangoes and tomatoes. Experimental evaluations showed that the image processing algorithm achieved defect detection accuracies of 89% and 92% for mangoes and 95% for tomatoes, respectively. Using a CNN architecture, the verification accuracy for the two fruits reached 95% and 94%.
In commercial applications, smart devices have been used to inspect and package over 20 varieties of fruit, including apples, citrus, oranges, prunes, winter dates, and plums. These devices have been used in nearly 10 provinces, municipalities, and autonomous regions, including Yunnan, Guangxi, Hubei, and Xinjiang, and have cumulatively inspected and packaged millions of tons of various fruits.
5 Challenges and Future Development Trends
Although machine vision has made significant progress in fruit inspection, it still faces several challenges. The complex characteristics of fruit surfaces, such as color variations, diverse textures, and irregular shapes, pose challenges for accurate inspection.
Threshold segmentation can easily fail when the contrast between the fruit peel and the background is low or when the lighting is uneven. For example, the yellow-green gradient on the mango peel can be misidentified as a defect, resulting in a high false positive rate.
The differences between different fruit varieties also increase the difficulty of inspection, necessitating the development of customized solutions for each fruit.
Future development trends will show the following characteristics: Technological convergence will drive fruit inspection towards multimodality, combining spectral detection, X-ray detection, electronic nose detection, and nuclear magnetic resonance detection to achieve more comprehensive quality assessment.
Dynamic detection capabilities will be a key direction. Future fruit quality inspection technology will evolve from static detection to dynamic detection, placing higher demands on hardware stability and image processing software accuracy.
Embedded integration will make systems more lightweight, and lightweight models based on edge computing will become a research focus to meet real-time processing requirements.
Intelligent decision-making will continue to improve, evolving from single-quality inspection to comprehensive quality management, integrating big data analysis to predict fruit shelf life and market demand. The fruit sorting industry has gone through four stages of development: Phase 1.0 focused on size, Phase 2.0 added grading and classification by weight, Phase 3.0 placed greater emphasis on color, and Phase 4.0 began exploring the detection of external defects and internal quality.
Currently, Chinese companies have achieved a level of mechanization comparable to global standards in fruit sorting, and are even at the forefront of artificial intelligence algorithms.
With the continuous advancement of technology, fruit visual inspection systems will become increasingly intelligent and sophisticated. New algorithms such as YOLOv10 have eliminated the need for NMS, reducing computational overhead. In the future, we may see even more lightweight models embedded directly into smartphones or portable devices, allowing consumers to conduct fruit quality inspections at any time.
The application of smart devices in the Wogan (Wugan) production area in Wuhan has shown that intelligently sorted Wogan not only improves quality but also expands market channels and leads to sustained growth in farmers' income. This "technological trend" is fundamentally changing the image of traditional agriculture as "dull and unrefined," infusing agricultural production with innovation and vitality.
