In the context of new engineering construction, cultivating students’ ability to solve complex engineering problems has become a focal point of engineering education reform. To help students master robotics and computer vision, a multidisciplinary and innovative experimental teaching program has been designed and developed for a tomato-like intelligent harvesting robot, specifically targeting the field of agricultural robots.

The methodology adopted in this experimental teaching program follows the paradigm of "problem discovery, scheme design, problem-solving." Initially, detailed challenges faced by intelligent harvesting of tomato-like fruits were scrutinized. Ensuring freshness and flavor requires staged harvesting and quick marketing. The long production cycle and short harvesting window of cherry tomatoes necessitate manual intervention. In addition, diverse growth postures of cherry tomato clusters and variations in fruit ripeness pose challenges for harvesting robots performing "fruit picking" tasks. These challenges lead to low harvesting efficiency and difficulties in difficulties in effective graded harvesting. To address these issues, a robot vision detection scheme based on cascade vision is proposed. This scheme achieves precise identification, ripeness detection, and positioning of tomato-like fruits. During the prediction stage of the YOLOv5 network, an additional detection branch is introduced to handle object detection and maturity grading of tomatoes simultaneously. By excluding non-target objects during data annotation and filtering for non-target objects, the computational burden on the network is reduced, thereby improving harvesting efficiency. Additionally, MobileNetv3 is introduced to classify the positional relationship between the fruit and its stem, guiding the end effector to approach the target fruit at the correct angle. To accurately and reliably locate the three-dimensional position of cherries, a cascaded vision-based localization detection method is proposed, combining the advantages of individual single-modal methods and three-dimensional point clouds. Ultimately, a meticulously crafted cherry tomato dataset is constructed, and hand-eye calibration of the robot platform is performed. Three robot harvesting experiments were conducted using cherry tomatoes to validate the effectiveness of the proposed scheme.

The tomato detection and maturity grading tests demonstrate high average recognition accuracy in detecting the ripeness of single tomatoes and clustered targets at various ripeness stages. Incorporating MobileNetv3 led to the clear classification of the positional relationship between fruit and stem, effectively guiding the end effector toward precision. In the final robotic grasping experiment, compared to fixed-angle harvesting, a notable enhancement in harvesting efficiency is observed, along with a reduction in the average time taken per fruit harvested. Consequently, the designed harvesting scheme for tomato-like fruits, based on cascaded visual detection, provides crucial technical support for achieving graded and staged harvesting, significantly improving overall harvesting efficiency.

The design and experimental teaching process of the visual solution enhances students’ understanding of theoretical knowledge in machine vision and deep learning while improving their practical skills in operating robot platforms can be enhanced. This helps cultivate students’ ability to independently solve engineering problems and innovate, enhances their interest in artificial intelligence, and lays a solid foundation for nurturing automation-related talents with interdisciplinary competence.