Tomato Harvesting Robot Based On Solid Work
Tomato Harvesting Robot Based On Solid Work
Tomato Harvesting Robot Based On Solid Work
Abstract
Designing and development of agricultural robot is always a challenging issue, because of robot intends to work
an unstructured environment and at the same time, it should be safe for the surrounded plants. Therefore,
traditional robots cannot meet the high demands of modern challenges, such as working in confined and
unstructured work spaces. Based on current issues, we developed a new tomato harvesting robot arm with a
flexible backbone structure for working in confined and extremely constrained spaces. Moreover, we optimized a
tomato detaching process by using newly designed gripper with passive stem cutting function. Moreover, by
designing the robot we also developed ripe tomato recognition by using machine learning. This paper explains
the proposed continuum robot structure, gripper design, and development of tomato recognition system.
Keywords: Tomato harvesting, gripper, tomato detection, design, agricultural robot.
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 370
robot can not be compared with manual operation[2]. are designed to harvest tomatoes at a certain height
The bottleneck of commercialization of harvesting along the guide rail. Two wheel drive control and
robot is low efficiency and high cost. The differential steering control can not meet the demand
combination of human and robot is a feasible way to of automatic harvesting of fruits and vegetables.
improve the success rate of robot harvesting line Moreover, the picking arm is usually replaced by the
method. If we can promote agricultural robots, we industrial robot arm, which has the disadvantages of
can predict labor saving, automation, production high cost and control complexity. The harvesting time
growth and so on. In addition, it is possible to of citrus is 3-7 s, melon is 15 s, cucumber is 10 s,
transform facts such as management of crops, eggplant is 1 min, and the harvest rate is less than
environment, cultivation, state and quality. From the 90%[4]. Fruits and vegetables are mainly identified
perspective of safety, appropriate guidance can be by color features, gray threshold and geometric shape.
provided to consumers. There are many serious These algorithms may be affected by illumination and
challenges to overcome, but it is possible to establish environmental factors, and it is difficult to identify
a new robot market in agriculture. Among all kinds of overlapping fruits.
fruits and vegetables, tomato is one of the main fruits
that consume a lot of fruits and vegetables. Many 1.4 Research Content
tomatoes are planted in houses and other facilities,
but due to the high temperature and humid working The task is to build a tomato harvesting robot,
environment and long harvest period, labor is large which can detect and harvest mature tomatoes, and
and labor needs to be saved. learn the automatic system in the process to pick
tomatoes from tomato plants safely and accurately. It
1.2 Research Status is expected that modern farms will produce higher
yield per unit area, provide high-quality products in a
In Japan, the Netherlands, Britain, France, Italy, sustainable way at lower cost, and have less
the United States, Israel and other countries, various dependence on labor force. Implementing digital
robots have been developed to harvest tomatoes, agriculture and the response of precision management
cucumbers, grapes and oranges. Kyoto University has in a specific location to this expectation, which not
developed a tomato harvesting robot manipulator only depends on the sensor technology, but
with five degrees of freedom. Okayama University continuous collection of field data only through the
has developed a 7-DOF robot, which is composed of proper use of agricultural robots is feasible.
motion system, vision system, end effector, Agricultural scientists, farmers and growers also
manipulator and control system. Later, a new tomato face the challenge of sustainably producing more
harvesting robot was developed, which was food on less land in a way that meets the projected
composed of vision system, manipulator, control needs of 9.8 billion people by 2050. This is
system and rotating arm. It takes about 15 seconds for equivalent to feeding a new city with 200000 people
each tomato from recognition to input, and the a day. Integrated digital tools, sensors, and control
success rate is 50% - 70%. In addition, the Korean technology accelerates the design and development of
Agricultural Mechanization Research Institute has agricultural robot technology, showing great potential
also developed a series of tomato harvesting and potential the benefits of modern agriculture.
robots[3]. The vision system can accurately These developments start from the collection of
determine the surface color of tomatoes, so that the accurate and detailed information to digitize the
robot can selectively pick mature tomatoes. However, timely spatio-temporal information of plants and
the potential disadvantages of these robots are slow fields to complete the complex nonlinear control
response and clumsiness. tasks of robots navigation. Robots and manipulators
in the field of agriculture have become an important
1.3 Domestic Research Status part of digital agriculture and precision
agriculture[6].
In China, the research of tomato harvesting robot
has made great progress, such as manipulator, image 2 Overall Design of Tomato Picking Robot
recognition and motion control. Panasonic
demonstrated the initial version of the harvesting 2.1 Gripper Design
robot and exhibited it at the latest international
robotics exhibition in December 2015. The robot can We use solid works software to design the gripper
pick tomatoes and move around the place. The robot of tomato harvesting robot. Solid works is a highly
puts tomatoes in the basket and replaces them once efficient 3D CAD software tool with integrated
they are full. The robot uses image sensing to detect analysis tools and design automation functions,
the color and shape of tomatoes. It also uses sensors which can help stimulate various physical behaviors
to identify the location of tomatoes. Usually, the such as motion, dynamics, stress, deflection,
current robot is not intelligent enough, and the vibration, temperature or fluid flow to adapt to all
success rate and pitching rate are far lower than types of design. Designers and engineers from all
expected. In addition, most tomato harvesting robots over the world and industries are using solid works to
2
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 371
3
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 372
4
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 373
is represented by the rotation of the connecting rod provide maximum torque of 0.941n. M and 2.422n.
(2). Each joint uses a specific servo motor to provide M respectively.
the torque required for movement. Each motor
receives shaft position control signal from matlab
code through Arduino microcontroller. The Cartesian
position of the object to be picked is detected by
using the camera and the available matlab plug-in,
which is identified by its color.
The s value of the parallel rectangular beam is the elbow muscle. Represents the total weight of the
highest, followed by the hollow cylindrical beam. payload and gripper.
This means that two parallel assembled rectangular
beams have the highest structural strength, 3.3 Recognition of Mature Tomato
considering that they are all made of the same
material with the weight ratio of the three links[11]. The goal of the recognition system is to find all
Therefore, the rectangular beam is used for the link of the tomatoes in the plant and know which are ripe
the manipulator and which are not. Therefore, it is very important to
understand the characteristics such as the color, shape,
type, weight and water ratio of tomatoes. The color of
the tomato is a good way to know if the tomato is
harvested. However, when tomatoes are fully ripe,
they must be eaten within a few days. As a result,
tomatoes are collected when they continue to ripen
during transportation, and the flavor of tomatoes is
less than that of tomatoes that have been preserved
for a longer time. One of the main difficulties in
using color as a visual discrimination parameter is
that it may suffer from uncontrolled or changing
lighting and shadows. There are many different
varieties of tomatoes and many different shapes[12].
The relationship between cross-sectional area and So let's focus on the round tomatoes. Weight is
cross-sectional area for three beams considered another factor in determining whether a tomato is ripe.
As tomatoes grow, their size increases with their
In order to determine the most appropriate weight. If the volume and density of a tomato are
cross-sectional shape of the link, three possible known, its weight can be easily determined.
geometric shapes are considered. Solid cylindrical
beam, hollow cylindrical beam and a group of two
rectangular beams. The section modulus s is a
geometric property that represents the strength based
on the shape or area of the cross section of the beam.
In order to ensure that the torque of the elbow and Fig 4: tomato detection
shoulder motors is sufficient to rotate the joint, the
maximum payload is 200g, the following calculation
is carried out. The figure shows the number of 3.3.1 YOLO Tomato Classifier
maximum required torque corresponding to the worst
case of full arm extension. From the moment of Mature tomato recognition is also a key part of
equilibrium equation, equation. The results show that the robot. In this study, we used machine learning to
the elbow needs at least 0.351n. M and the shoulder train neural networks to distinguish mature tomatoes
needs at least 1.259n. M. Therefore, according to the from immature tomatoes and other similar fruits. As a
servo motors available in the market, the servo tomato classifier, we use YOLO We developed three
motors selected for elbow joint and shoulder joint main filters to distinguish tomatoes. The first filter
5
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 374
detects the shape of the object, the second filter inflate the air bag, while the DC motor drives the
detects the color or (RGB) filter, and the third filter is clamp to clamp the stem properly.
machine learning. This is necessary because there are The pressure sensor can detect the pressure
many fruits or objects that may be very similar to between the fruit and the air bag under the premise of
tomatoes, such as apples or oranges. In addition, it reliably clamping the fruit, so as to ensure that there
should be possible to calculate the distance between is no damage on the fruit. Until it is stable, then drive
the camera and the tomato using the classification the double acting cylinder to shrink the sleeve to cut
camera. For this technique, we use a measurement and pick the tomato. Our harvesting tools are able to
method for a single camera by using the focal length grab tomatoes with suction cups and then use
of the camera. swinging blades. Natural changes in crop size, shape
and direction make it challenging and reliable to
select a single end effector pose to grab and cut each
tomato at the same time. In order to overcome this
difficulty, a key feature of our harvesting tool design
is the passive decoupling mechanism, which allows
sequential grasping and cutting operations. The
decoupling mechanism is a kind of flexible strip,
which is fixed on the main body of the end effector
together with the suction cup. The suction cup is also
magnetically connected to the lower side of the
cutting blade, allowing the manipulator to guide the
suction cup during the installation phase. After fixing,
lift the cutting blade to separate the suction cup from
the cutting blade. The suction cup is then attached to
the end effector only by a flexible piece, allowing the
cutting blade to move independently of the suction
Fig 5: fixture calibration (a) geometric formula (b) cup during the cutting operation.
6
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 375
T10 A1
link
d i i a i i
T20 A1 A2
T30 A1 A2 A3
L1 0
i a 1
0
T
0
L2 0
2 a 2
0
3 is a transformation matrix representing the
L3 0
3 a 3
0 visibility of the end effector relative to the basic
framework
A. Inverse kinematics of manipulator
The figure above shows the method that can be used
for inverse kinematics analysis
constant, while
i is the angle of the (i)th
connection, which is variable.
Is the homogeneous transformation moment that
relative to
o x y z i 1 i 1 i 1 i 1
i c d i i
In order to determine the required joint angle of the
0 0 0 1 end effector and the desired position (x, y) of the
manipulator in the work area[15], it is necessary to
(1) know the coordinates of the wrist and the direction of
the end effector
c i cos i c a i cos a i Give Way,
and
x y w w Wrist coordinates
c 1 s 1 0 a1c 1
1
s c 1 0 a1s 1 z is the angle made by the end effector relative to
A I
0 0 1 0
the x-axis, and then
x w
x a 3 cos
0 0 0 1
y w
y a 3 sin
c 2 s 2 0 a c 2
2 y
a tan 1 w
s 2 c 2 0 a s 2
x
A2 0
0 1
2
0 w
r a a 2a a
2
cos
0 0 0 1 1 2 1 2 (1)
r x y
2 2 2
TJI AI 1 AI 2 .....AJ w w
From I<j
β is obtained from equation 1
Give help to
oxyz j j j j
The position and 2
direction of is converted to since
r a 2r a 1 cos a 2
2 2 2
o x y z Transformation matrix
i i i i TJI 1 (2)
7
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 376
1
a 3c 3s 12 a 3s 3c 12
2 3 1 2 a 3c 3c12 a 3s 3s 12
0
4.2 manipulator Jacob j3 0
The upper joint angle is for the elbow down
configuration 0
Jacobian matrix is one of the most important tools in
1
robot motion control because it has been developed
joint and end effector speed. Jacobi's law transforms The whole matrix of the whole system is compiled by
the ratio of forces into joint coordinates. Given an Jacobi
N-linked manipulator
Jacobian
j
j j j .
1 2 3
j
j j .... j
1 2 n
8
GSJ© 2022
www.globalscientificjournal.com
GSJ: Volume 10, Issue 1, January 2022
ISSN 2320-9186 377
[1] R. Hussin*, M. RizonJuhari, Ng Wei Kang, R.C.Ismail, [14] Edan, Y., (1999). Food and agriculture robotic.
A.Kamarudin ―Digital ImageProcessing Techniques for Handbook of Industrial Robotic, Second edition. Chapter
Object Detection From Complex Background 60, pp. 1143-1155
Image‖International Symposium on Robotics and
Intelligent Sensors 2012 (IRIS 2012). [15] C. Ji, J. Zhang, T. Yuan, W. Li Research on key
technology of truss tomato harvesting robot in greenhouse
[2] Dameshwari Sahu, Ravindra Manohar Potdar. "Defect
Identification and MaturityDetection of Mango Fruits [16] Schertz, C.E. and Brown, G.K. (1968). Basic
Using Image Analysis."American Journal of Artificial considerations in mechanizing citrus harvest. Transaction
Intelligence. Vol. 1, No. 1, 2017, pp. 5-14. DOI: of ASAE. 343-346.
10.11648/j.ajai.20170101.12.
[17] Zion, M. Mann, D. Levin, A. Shilo, D. Rubinstein, I.
[3] Rahul Gautam, Ankush Gedam, Ashish Zade, and Ajay Shmulevich Harvested-order planning for a multiarm
Mahawadiwar ―Review onDevelopment of Industrial robotic harvester Comput. Electr. Agric., 103 (2014), pp.
Robotic Arm‖ International Research Journal of 75-81.
Engineeringand Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017www.irjet.net p-ISSN:
2395-0072.
[4] Srinath, A., Rao, C.S. and Jagadeesh, A., 2011. Genetic
algorithm based Hammingnumber technique for detection
of isomorphism in planetary gear trains. Int. J. of Applied
Engineering Research,6 (1), pp.33-42.
GSJ©9 2022
www.globalscientificjournal.com