sensors

Article
Monitoring Plant Status and Fertilization Strategy
through Multispectral Images
Matheus Cardim Ferreira Lima 1,2, * , Anne Krus 3 , Constantino Valero 3 ,
Antonio Barrientos 4 , Jaime del Cerro 4 and Juan Jesús Roldán-Gómez 4,5
1 Department of Agroforest Ecosystems, ETSI Agrónomos, Universidad Politécnica de Valencia,
46022 Valencia, Spain
2 Research and Extension Unit (AGDR), Food and Agriculture Organization of the United Nations (FAO),
00153 Rome, Italy
3 Department of Agroforest Engineering, ETSI Agronómica, Alimentaria y de Biosistemas, Universidad
Politécnica de Madrid, 28040 Madrid, Spain; [email protected] (A.K.); [email protected] (C.V.)
4 Centre for Automation and Robotics (CSIC-UPM), 28006 Madrid, Spain; [email protected] (A.B.);
[email protected] (J.d.C.); [email protected] or [email protected] (J.J.R.-G.)
5 Department of Computer Engineering, Higher Polytechnic School, Autonomous University of
Madrid (UAM), 28049 Madrid, Spain
* Correspondence: [email protected]




Received: 17 December 2019; Accepted: 9 January 2020; Published: 13 January 2020


Abstract: A crop monitoring system was developed for the supervision of organic fertilization status
on tomato plants at early stages. An automatic and nondestructive approach was used to analyze
tomato plants with different levels of water-soluble organic fertilizer (3 + 5 NK) and vermicompost.
The evaluation system was composed by a multispectral camera with five lenses: green (550 nm),
red (660 nm), red edge (735 nm), near infrared (790 nm), RGB, and a computational image processing
system. The water-soluble fertilizer was applied weekly in four different treatments: (T0: 0 mL, T1:
6.25 mL, T2: 12.5 mL and T3: 25 mL) and the vermicomposting was added in Weeks 1 and 5. The trial
was conducted in a greenhouse and 192 images were taken with each lens. A plant segmentation
algorithm was developed and several vegetation indices were calculated. On top of calculating
indices, multiple morphological features were obtained through image processing techniques. The
morphological features were revealed to be more feasible to distinguish between the control and
the organic fertilized plants than the vegetation indices. The system was developed in order to be
assembled in a precision organic fertilization robotic platform.

Keywords: multispectral image; computer vision; precision agriculture; vegetation indices;

morphological features

1. Introduction
Environmental protection allied with health concerns represents an increasingly important
trend in the consumer behaviour and has led to the development of green products and organic
markets [1]. These markets have experienced exponential growth in recent years, especially in
the organic markets that address multiple consumer concerns relating to health, food safety and
environmental conservation [2]. Tomatoes are one of the vegetable crops with a higher demand for
fertilization, with recommended doses close to 400 kg·ha−1 depending on soil type [3].
Deficient uses of organic fertilization can cause “abiotic diseases” on plants, decrease the
suppressive effect of the soil, and the readiness of plants to defend themselves against attacks
of plant pathogens [4]. Nutrient deficiency affects the plant growth, with consequences in the
production and the profitability of the organic field. On the other hand, the indiscriminate use of

Sensors 2020, 20, 435; doi:10.3390/s20020435 www.mdpi.com/journal/sensors

Sensors 2020, 20, 435 2 of 21

manure and soluble fertilizers prolong the vegetative state of the plant with the abundance of young
tissue, making the plants more susceptible to plant pathogen attacks for a longer time [4,5]. Further, a
considerable amount of the fertilizers cannot be totally absorbed by the plants. These compounds can
run off into waterways, leached into groundwater, or become lost in gaseous form. The leachate liquid
derived from the organic fertilizer can cause pollution of groundwater producing toxic algae blooms,
accelerating eutrophication, and reducing biodiversity [6].
Meta-analysis studies indicate that the optimization of the use of nitrogen fertilization in tomatoes
can decrease costs and environmental impact maintaining the same yield levels [7]. Focusing on
that, the European commission fomented projects to study cost-efficiency technologies and bring
innovations to reduce the dependency of contentious inputs in the organic production systems.
The SUREVEG (SUREVEG stands for “Strip-cropping and recycling of waste for biodiverse and
resource-efficient intensive vegetable production” and is funded via ERANET Core Organic Cofund.
More info: https://projects.au.dk/coreorganiccofund/core-organic-cofund-projects/sureveg/) Project
seeks to develop and implement innovations for intensive cropping systems using strip-cropping
and fertility strategies. These farming systems consists of inter-row cropping with different families
and species in the same plot. The increase in biodiversity enhances the resilience of the field against
soil-borne disease, and pest attacks, which at the same time increase the soil protection against erosion
and nutrient depletion.
One of the research lines aims to develop and assess smart technologies for the management
of strip-cropping systems. These smart technologies are focused on the use of precise fertilization
methods to reduce the dependency on biopesticides and non-organic fertilizers, improve soil fertility
in intensive vegetable cropping systems and therefore bring a positive impact on water quality and
landscape biodiversity.
The field of precision fertilization aims to optimize the use of resources in time and space. Smart
technologies were developed for monitoring the nutrient status of plants and control variable rate
applications in broad, monocrop, and conventional fields [8].
Initially these technologies were based on soil samples, yield mapping, and automatic guidance [9].
After that, the advances in sensor technology-enabled nondestructive optical approaches and the
use of satellite and unmanned aerial vehicles images were added to the system [10]. Nowadays
new embedded devices have been implemented in order to analyze the nutrient status of the crop in
real-time using high-resolution image sensors at plant level [11].
These sensors are based on multispectral images and use vegetation indices to obtain the best
correlation with the nutrient status of the arable crops [12–14].
In order to optimize the use of the spectral information acquired by the multispectral camera
for monitoring the nutrient content of plants, a principal component analysis approach can be used.
This methodology uses orthogonal transformations to convert the spectral measurements at different
wavelengths into an orthogonal system of eigenvectors. This approach combined with multiple linear
regression allows to create new vegetation indices, reconstruct the leaf reflectance spectra, and predict
the leaf biochemical contents with high accuracy [15].
These sensors are also being used for many applications regarding plant analysis. The multispectral
images can be used for counting plants in orchard fields [16], estimate biomass, productivity, canopy
traits [17], phenological state [18] and mortality of forest trees [19], and can also estimate the damage
level of insect pests in forest and crops [20], assess hydric deficiency [21], and estimate the quality
parameters (brix, texture, internal damages) of fruits in a non-destructive way [22,23].
In real-time sprayer systems, these devices are embedded in conventional tractors and are
connected to a global navigation satellite system (GNSS) and computer system that control selectively
foliar spray applications [8].
Other systems based on real-time analysis were developed in order to deliver site-specific herbicide
applications in arable crops. Besides the multispectral images and the vegetation indices, these systems
Sensors 2020, 20, 435 3 of 21

use a bicameral system, binarization techniques, and morphological algorithms to differentiate between
Sensors 2020, 20, x FOR PEER REVIEW 3 of 21
the crop and spontaneous plant species [24–27].
Both types of real-time precision spraying systems are commercially available or are in process of
Both types of real-time precision spraying systems are commercially available or are in process
being commercialized, and have shown a positive reduction in the use of chemical compounds with a
of being commercialized, and have shown a positive reduction in the use of chemical compounds
reduction of costs and an increase in yield levels (in the case of precision fertilization) compared to
with a reduction of costs and an increase in yield levels (in the case of precision fertilization)
traditional sprayers [8].
compared to traditional sprayers [8].
Image processing techniques with multispectral cameras from visible to near-infrared spectrum
Image processing techniques with multispectral cameras from visible to near-infrared spectrum
are also being used to provide non-destructive plant phenotype image datasets. These approaches
are also being used to provide non-destructive plant phenotype image datasets. These approaches
have allowed more precise and real-time, high throughput, and high-resolution data for the modelling
have allowed more precise and real-time, high throughput, and high-resolution data for the
and prediction of plant growth and morphological development in different conditions, with recent
modelling and prediction of plant growth and morphological development in different conditions,
applications in plant health analysis [28–30].
with recent applications in plant health analysis [28–30].
More advanced prototypes were made in order to insert these types of sensors in autonomous
More advanced prototypes were made in order to insert these types of sensors in autonomous
vehicles. Several terrestrial robotic platforms were developed using multispectral cameras seeking
vehicles. Several terrestrial robotic platforms were developed using multispectral cameras seeking
for a more automatic, low-energy cost and accurate analysis of the crop parameters compared to
for a more automatic, low-energy cost and accurate analysis of the crop parameters compared to
conventional equipped tractors and unnamed aerial vehicles [11,31,32].
conventional equipped tractors and unnamed aerial vehicles [11,31,32].
Some of these autonomous agricultural devices are capable of identifying weeds and control
Some of these autonomous agricultural devices are capable of identifying weeds and control
them using low-dose spraying, mechanical control or thermal control [33]. Other platforms can spray
them using low-dose spraying, mechanical control or thermal control [33]. Other platforms can spray
fertilizers in arable crops without the need for heavy tractors, reducing the compaction of the soil and
fertilizers in arable crops without the need for heavy tractors, reducing the compaction of the soil and
the
the physical
physical damage
damageto tothe
thecrops
crops[29].
[29].More
Morerecently, modular
recently, modularagricultural robots
agricultural were
robots developed
were developedfor
mapping different aspects of the cereal plants in parcels of breeding trials [31].
for mapping different aspects of the cereal plants in parcels of breeding trials [31].
Besides
Besides the the advances
advances in in the
the robotic
robotic agricultural
agricultural platforms, most of
platforms, most of them
them were
were developed
developed forfor
conventional
conventional orchards and arable crops, presenting a lack of technologies in the context of
orchards and arable crops, presenting a lack of technologies in the context of organic
organic
horticulture.
horticulture. For For this
this reason,
reason, the
the SUREVEG
SUREVEG team team has
has developed
developed the
the robotic
robotic prototype
prototype shown
shown in in
Figure
Figure 1.1.

Figure
Figure 1. Robotic prototype
1. Robotic prototype developed
developed by
by the
the SUREVEG
SUREVEG team.
team.

This
This prototype
prototypeisisa cart withwith
a cart a manipulator robot robot
a manipulator and multiple sensors and
and multiple actuators.
sensors The proposed
and actuators. The
sensors
proposed aresensors
laser scanners
are lasertoscanners
build three
to dimensional models of themodels
build three dimensional plants,of
which will be which
the plants, used towill
study
be
their
used growth,
to studyand a multispectral
their growth, and camera, which will
a multispectral be used
camera, to study
which will the state and
be used healththe
to study of state
the crops.
and
Moreover,
health of thea system
crops.toMoreover,
apply treatments
a systemto to
theapply
soil will be installed
treatments in the
to the soilprototype, consisting
will be installed in of a
the
prototype, consisting of a tank to store the liquid treatment and a sprayer to apply it. Finally, the
manipulator robot will be available to accurately place sensors and actuators at the target points.
Having in mind this long-term goal of automatically monitoring crops and applying treatments,
this work is focused on the collection of information about the plants by using multispectral imagery.
Sensors 2020, 20, 435 4 of 21

tank to store the liquid treatment and a sprayer to apply it. Finally, the manipulator robot will be
Sensors 2020, 20, x FOR PEER REVIEW 4 of 21
available to accurately place sensors and actuators at the target points.
Having in mind this long-term goal of automatically monitoring crops and applying treatments,
Specifically, the paper describes the acquisition of multispectral images of tomato plants with
this work is focused on the collection of information about the plants by using multispectral imagery.
different levels of organic fertilization, as well as the estimation of their nutritional states in the early
Specifically, the paper describes the acquisition of multispectral images of tomato plants with different
stages through multiple vegetation indices and morphological features by using computer vision
levels of organic fertilization, as well as the estimation of their nutritional states in the early stages
techniques.
through multiple vegetation indices and morphological features by using computer vision techniques.
2.2.Materials
Materialsand
andMethods
Methods

2.1.
2.1.Location
Locationand
andGrowing
GrowingConditions
Conditions
The experiment was
The experiment wascarried
carried
outout
on on a greenhouse
a greenhouse in order
in order to the
to obtain obtain the different
different spectral
spectral responses
responses of plants. This greenhouse is in the facilities of the School of Agronomic,
of plants. This greenhouse is in the facilities of the School of Agronomic, Food and Biosystems Food and
Biosystems
EngineeringEngineering of the
of the Technical TechnicalofUniversity
University of Madrid,
Madrid, which which
are located in are located
Madrid, in Madrid,
Spain Spain
(40◦ 260 19.9” N
(40°26′19.9″
◦ 0 N 3°44′15.7″ W). The tomato (Mina F1 cv.) seedlings were initially cultivated
3 44 15.7” W). The tomato (Mina F1 cv.) seedlings were initially cultivated in 3-l pots with height and in 3-l pots
with height
diameter ofand diameter
15 cm, of 15acm,
filled with mixfilled with
of peat a mix of(50%)
substrate peat and
substrate (50%)
coconut and
fiber coconut
(50%). Afterfiber (50%).
5 weeks of
After 5 weeks of experiment, the plants were transplanted to 8-l
experiment, the plants were transplanted to 8-l pots with the same mixture. pots with the same mixture.

2.2.
2.2.Trial
TrialDesign
Designand
andFertilization
Fertilization
The
Theexperiment
experimentwas wasdesigned
designedas asrandomized
randomizedblocks,
blocks,withwithfour
fourtreatments
treatmentsto tobe
beapplied
appliedto tofour
four
groups
groups of of plants
plants (T0,
(T0, T1,
T1, T2
T2 and
and T3).
T3). The
Thetrial
trialdesign
designisisdescribed
describedby byFigure
Figure 2,2, which
which shows
shows thethe
specific
specific moments
moments when when thethetreatments
treatmentswerewereapplied,
applied,and andthethe multispectral
multispectral images
images werewere acquired.
acquired. The
The fertilizations
fertilizations werewere carried
carried out weekly
out weekly usingusing water-soluble
water-soluble organic
organic fertilizer
fertilizer (3% of (3% of Nitrogen
Nitrogen and 5%andof
5% of potassium)
potassium) obtained obtained from
from beet beet vinasse
vinasse and phosphorite,
and phosphorite, and registered
and registered for use in for use production
organic in organic
production (COMPO-Fertilizante
(COMPO-Fertilizante HuertoThe
Huerto y Frutales). y Frutales). The initial
initial volume volumeand
of irrigation of irrigation and
fertilization fertilization
was 300 mL of
was 300 mL of solution. One week after transplant, plants were assigned to the
solution. One week after transplant, plants were assigned to the different treatments according to the different treatments
according to the(T0:
fertilizer label fertilizer
0 mL, label (T0:mL,
T1: 6.25 0 mL,T2:T1: 6.25
12.5 mL mL,
andT2:T3:12.5
25 mL
mL,andwhichT3: corresponds
25 mL, whichtocorresponds
T0: 0 g of N
to T0:0 0g gofofK,NT1:
and and 0 ggof
0.15 ofK,N T1:
and0.15
0.30ggof
of N
K,and
T2: 0.30 g of of N
K,and
T2: 0.60
0.30 gg of
of K,
N T3:
and0.60
0.60ggofofN,K,1.20
T3: g0.60 g
of K).
of N,plants
The 1.20 galsoof K). The plants
received also receivedoftwo
two supplements supplements ofone
vermicomposting: vermicomposting:
in the first transplantone in(T0:
the 0first
mL;
transplant
T1: 75 mL;(T0: T2: 0150
mL;mL;
T1:T3:
75 mL; T2: 150
300 mL) andmL; T3: 300
another mL)
after theand another
second after the(T0:
transplant second
0 mL;transplant
T1: 237.5(T0:mL;
0T2:
mL;475T1:mL;
237.5 T3:mL;
950T2: 475Images
mL). mL; T3:of950
themL). Images
plants wereof the plants
acquired were
with acquired
7-days withand
intervals 7-days intervals
processed to
and processed
compute to compute
vegetation indices vegetation indices
and analyze them. and analyze them.

Figure 2. Chronology of plant treatment and image acquisition during the experiment.
Figure 2. Chronology of plant treatment and image acquisition during the experiment.
2.3. System Overview
2.3. System Overview
The general system was composed of images acquired from the greenhouse experiment with
The general
different levels ofsystem was composed
fertilization. of images
The evaluation acquired
system was from the greenhouse
composed of a bracketexperiment with
support stands
different levels of fertilization.
clamp supporting the sensor andThethe
evaluation system(battery).
power supply was composed of a bracket
The platform support
has the stands
possibility of
clamp
height adjustment for the sensor. The distance between the sensor and the bottom of the pot was setof
supporting the sensor and the power supply (battery). The platform has the possibility at
height adjustment
0.7 m during for the
the first sensor.
5 weeks andThe distance
moved between
to 1.4 m afterthe sensor
Week and the
5 (Figure 3),bottom of the
since the potwere
plants was set
too
at 0.7 m during the first 5 weeks and moved to 1.4 m after Week 5 (Figure 3), since the plants were
too high to be observed from the initial distance. The higher distance between the camera and the
plant makes the plant image smaller, so the leaf area represented by the pixel area became smaller
after that modification. In order to model the data, regression analysis using the mean values was
performed, so the results could be extrapolated for the next weeks.
commercial system.
The Sequoia Camera has also a sunshine sensor that is used to calibrate the images depending
on the sunlight. This makes it possible to compare photos over time, despite variations in light during
photo shoots. The sunshine sensor is attached on the upper part to the system, facing the sky and
Sensors
correcting the2020, 20, 435and it also contains a GPS/GNSS module, a magnetometer and inertial 5 of 21
signal
measurement system (Figure 3).
The images
high to bewere takenfrom
observed withthe
andinitial
without a blackThe
distance. background placedbetween
higher distance to help the
the segmentation.
camera and the plant
The developed algorithms showed different efficiency in segmentation with
makes the plant image smaller, so the leaf area represented by the pixel area became or without
smallerthe
after that
background. The pots were tagged with a coloured stamp in order to take the images in the same
modification. In order to model the data, regression analysis using the mean values was performed,
positionsoduring
the resultsthe could
experiment (Figure 3b).
be extrapolated The
for the sensor
next weeks.communicates with a smartphone or
computer via Wifi protocol in order to store the acquired images.

b)

Figure 3. Overview of the image acquisition system. (a) Conceptual image acquisition scheme. (b)
Figure 3. Overview of the image acquisition system. (a) Conceptual image acquisition scheme.
Bracket support with camera and signal correction device measuring tomato plants in early stages.
(b) Bracket support with camera and signal correction device measuring tomato plants in early stages.
2.4. Computational System for Image Analysis
The camera used for image acquisition was a multispectral camera (model Sequoia, Parrot Drones,
TheParis, France, 2017)
computational whichused
system was for
originally designed
the image for is
analysis use in agricultural
composed of fourUnmanned
major steps: Aerial
pre-Vehicles
(UAV’s). Its internal sensor is composed of four spectral bands which
processing, calculation of vegetation indices, image segmentation, and morphological analysis. register the reflected light
coming from the vegetation and can be used to distinguish plant vigor based on reflectance levels [34].
2.4.1. Image
ThesePre-Processing
bands are: green (550 nm wavelength, 40 nm bandwidth), red (660 nm wavelength, 40 nm
Thebandwidth),
Sequoia sensor red edge (735 nm
produces wavelength,
tagged 10 nm
image file bandwidth)
format and near
(TIFF) images withinfrared
a size of (790 nm×wavelength,
1280 960
pixels. The camera was designed to take images from a minimum distance of 30 m to the target [26]. system.
40 nm bandwidth). Additionally, a 16-megapixel RGB camera is also fitted into the commercial
When the images The Sequoia Camera
were taken has
with also a sunshine
a shorter distance sensor that is used
a displacement to calibrate
occurs betweenthe theimages depending
4-channel
images on duethe sunlight.
to an This geometry
unexpected makes it possible
between to thecompare photos
sensors. In orderover time, the
to correct despite variationsofin light
displacement
the images, an algorithm that shifts an image by a specified number of pixels in either the x- orfacing
during photo shoots. The sunshine sensor is attached on the upper part to the system, y- the
directionsky(orand correcting
both) was used. the signal and it also contains a GPS/GNSS module, a magnetometer and inertial
measurement system
The parameters used to define (Figurethe3).number of pixels and the direction that each image should be
The images were taken
shifted were obtained using a sample image with andwith
without a black background
a referential point and then placed to help
applied the segmentation.
subsequently
The developed
to all collected data. The algorithms
red image showed different selected
was arbitrarily efficiencyasinthe
segmentation
reference, andwiththen
or without
the green, thenear-
background.
The pots were tagged with a coloured stamp in order to take the images in the same position during the
experiment (Figure 3b). The sensor communicates with a smartphone or computer via Wifi protocol in
order to store the acquired images.

2.4. Computational System for Image Analysis

The computational system used for the image analysis is composed of four major steps:
pre-processing, calculation of vegetation indices, image segmentation, and morphological analysis.

2.4.1. Image Pre-Processing

The Sequoia sensor produces tagged image file format (TIFF) images with a size of 1280 × 960
pixels. The camera was designed to take images from a minimum distance of 30 m to the target [26].
Sensors 2020, 20, 435 6 of 21

When the images were taken with a shorter distance a displacement occurs between the 4-channel
images due to an unexpected geometry between the sensors. In order to correct the displacement
of the images, an algorithm that shifts an image by a specified number of pixels in either the x- or
y-direction (or both) was used.
The parameters used to define the number of pixels and the direction that each image should be
shifted were
Sensors 2020, 20,obtained
x FOR PEERusing a sample image with a referential point and then applied subsequently
REVIEW 6 of 21
to all collected data. The red image was arbitrarily selected as the reference, and then the green,
near-infrared
infrared (NIR)(NIR) and red-edge
and red-edge imagesimages were shifted
were shifted in to
in order order
be intothe
be same
in theposition
same position as the
as the red red
image.
image. The parameters
The parameters used forused for the distances
the distances (0.7–1.4(0.7–1.4 m) are shown
m) are shown in Tablein1.Table 1.

Table
Table1.1. Shift factors (x- and y-direction) used for overlaying the different images using parrot sequoia
in shorter distance
distance ofof plant
plant samples.
samples. (Red image used as reference).
reference).

Images
Images ShiftFactor
Shift Factor
Green
Green [50,23]
[50, 23]
NearInfrared
Near Infrared [41, −34]
[41, −34]
Red Edge
Red Edge [68,−11]
[68, −11]

central area
The central area of the images was defined as area of interest (AOI) and the consequent processes
were used
were usedinin this
this areaarea in order
in order to reduce
to reduce the computational
the computational process
process and and the
optimize optimize the plant
plant recognition
and binarization. The images were clipped with AOI-x: 200:900 and AOI-y 200:1000, reducing200:1000,
recognition and binarization. The images were clipped with AOI-x: 200:900 and AOI-y the size
reducing
of thefrom
the image size of
960the× image
1280 tofrom
700 ×960 × 1280
800 to 700
squared × 800
pixels squared
(Figure 4). pixels (Figure 4).

Figure 4. Types of images produced by the sensor after shifting and clipping the area of interest (700
×
× 800 resolution). (Bluish
(Bluish colors
colors indicate
indicate regions
regions with lower levels of reflectance and reddish
reddish colors
colors
indicates regions
regions with
with higher
higher levels
levels of
of reflectance).
reflectance).

2.4.2. Vegetation Indices

2.4.2. Vegetation Indices
Vegetation indices (VIs) computed from multispectral images are quite simple and effective
Vegetation indices (VIs) computed from multispectral images are quite simple and effective
parameters for the qualitative and quantitative evaluation of vegetation vigour, cover and growth
parameters for the qualitative and quantitative evaluation of vegetation vigour, cover and growth
dynamics among other applications. There is a vast implementation of these indices using different
dynamics among other applications. There is a vast implementation of these indices using different
airborne and satellite platforms with recent advances using unmanned aerial vehicles (UAV) and
airborne and satellite platforms with recent advances using unmanned aerial vehicles (UAV) and
tractors. Due to the complexity of the instrumentation platforms, light spectra combinations and
resolutions used, there is no unified mathematical expression that attends all applications of VIs [26].
According to the same authors [35], customized algorithms have been studied for several
applications combining visible light radiation, mainly the spectral region correspondent with the
green region from vegetation, and nonvisible spectra in order to obtain proxy quantifications of the
vegetation surface. Therefore, for precise measurement applications, the VIs are optimized and
Sensors 2020, 20, 435 7 of 21

tractors. Due to the complexity of the instrumentation platforms, light spectra combinations and
resolutions used, there is no unified mathematical expression that attends all applications of VIs [26].
According to the same authors [35], customized algorithms have been studied for several
applications combining visible light radiation, mainly the spectral region correspondent with the
green region from vegetation, and nonvisible spectra in order to obtain proxy quantifications of the
vegetation surface. Therefore, for precise measurement applications, the VIs are optimized and usually
constructed according to the specific application requirements, and a validation procedure is needed,
along with customized methodologies.
Eleven VIs retrieved from the literature were adopted to find correlations between the fertilization
status of the tomato plants and the spectral response. One of the vegetation indices were also used as a
filter parameter for binarization of the plant leaves despite the background (image segmentation).
The VIs were chosen based on the literature and in the available bands present at the sensor. The
list of indices, abbreviations, formula and traditional application of the VIs can be observed in Table 2.
For use in this study, some closest Sequoia reflectance bands were substituted for the traditional
narrowband wavelengths.

Table 2. Vegetation Indices (VIs) used to create spectral profiles of the tomato plants from the
multispectral data with formulae and traditional applications (obtained from [36]).

Index Abbreviation Formula Application

Normalized Difference ρNIR – ρRED Measuring green vegetation through
NDVI ρNIR+ ρRED
Vegetation Index normalized ration ranging from −1 to 1.
ρNIR − ρGREEN Modification of NDVI, more sensitive to
Index GNDVI ρNIR+ ρGREEN chlorophyll content.
Normalized Difference ρNIR − ρRedEdge Modification of NDVI, using Red-Edge
RENDVI ρNIR+ ρRedEdge
Vegetation Index information related to plant health.
Modification of NDVI used to
Green Normalized Difference ρNIR2 − ρRED
NLI emphasize linear relations with
Vegetation Index ρNIR2 + ρRedEdge
vegetation parameters.
Red-Edge Normalized ρNIR − ρRED Variation of NDVI in order to reduce
OSAVI 1.5 ∗ ρNIR+ ρRED+0.16
Difference Vegetation Index the soil effect
ρNIR
Nonlinear Vegetation Index GRVI ρGREEN
Related with leaf production and stress
 ρNIR 
−1 A combination of renormalized NDVI
Optimized Soil Adjusted q ρRED 
MSR and SR to improve sensitivity to
Vegetation Index ρNIR
ρRED +1 vegetable characteristics
Ratio of NIR scattering to chlorophyll
ρNIR
Green Ratio Vegetation Index SR ρRED
and light absorption used for simple
vegetation distinction
ρRedEdge − ρRED Modification of NDVI, using Red-Edge
Modified Simple Ratio NDRER ρRedEdge+ ρRed instead of NIR.
ρNIR − ρGREEN Index used in areas with high
Simple ratio SPI2 ρNIR− ρRED variability in canopy structure
Normalized Difference ρNIR − ρRedEdge Index to assess chlorophyll content in
LCI ρNIR+ ρRED
Red-Edge/Red areas of complete leaf coverage.
Note: For use in this study, some closest Sequoia reflectance bands were substituted by the traditional narrowband
wavelengths as described by [36] and showed in the Table 2.

2.4.3. Plant Extraction (Segmentation)

Digital image processing and computer vision approaches are powerful tools for plant analysis
because they allow plant physiological and physical features to be measured non-destructively with
high temporal and quantitative resolution [37]. After the image acquisition, the image analysis
process starts with the extraction of the numerical data that describes the object in the image. First
the background pixels must be separated from the object of interest through a process called object
segmentation. The accuracy of this process decreases as the image quality decreases [38]. Multiple
Sensors 2020, 20, 435 8 of 21

Sensors 2020, 20, x FOR PEER REVIEW 8 of 21

plant segmentation methods have been proposed to handle the variation among different sensors
including
Otherthe use of were
systems adaptative thresholding
developed and machine
using a fixed learning
threshold [39].standardization method [38].
and image
Other systems
Normalized differencewerevegetation
developedindexusing(NDVI)
a fixed has
threshold and image
been widely usedstandardization method
for distinguishing [38].
between
Normalized difference vegetation index (NDVI) has been widely used for distinguishing
plant and soil and plant species through satellite imagery. This occurs because the near infrared (NIR) between
plant and soil
combined withand the plant species
red image througha satellite
provides imagery.
significantly higherThis occurs because
reflectance than soiltheinnear infrared
natural light
(NIR) combined
conditions with the red
[40]. However, NDVI image provides
is known for apresenting
significantly higher related
problems reflectance than soil at
to saturation in extreme
natural
light conditions
values. Aiming [40]. However,
to design NDVI
a robust is known for
segmentation presenting
algorithm thatproblems
could berelated
used into different
saturationsoilat
extreme values.
conditions, Aiming
the fixed to design
threshold wasa robust segmentation
avoided. Instead, wealgorithm that couldhistogram
used an automatic be used insegmentation
different soil
conditions, the fixed threshold was avoided. Instead, we used an automatic histogram
that extracted the 92% higher reflectance values from the NDVI images when the NDVI values were segmentation
that extracted
superior to 0.1.the 92% higher
Another filter reflectance
based in thevalues from the
NIR image wasNDVI
testedimages when reliable
but without the NDVI values
results were
(Figure
superior
5). to 0.1. Another filter based in the NIR image was tested but without reliable results (Figure 5).

Figure 5.5.Computer
Computervision
vision process
process based
based in NDVI
in NDVI imageto used
image used extracttotheextract the plant
plant from from the
the background.
background.
Top Top images: (left),
images: Near-infrared Near-infrared (left),and
NDVI (center), NDVI
RGB(center), and RGB
(right); bottom (right);
images: bottomimages.
binarized images:
binarized images.
The images were converted from unit8 to double type in order to allow the pixels to admit decimal
valuesThe
and images were converted
calculations. from processing
All the image unit8 to double type
analysis wasin performed
order to allow the pixels
in MATLAB to admit
2013b (The
decimal values
MathWorks and
Inc., calculations.
Natick, All the
MA, USA). image
After step, the bwarefilt
thisprocessing analysisfunction
was performed
of MATLABin MATLAB 2013b
was applied.
(Thefunction
This MathWorks allowsInc., Natick, MA,
extracting USA). After
all connected this step,
components thethe
from bwarefilt function
binary image, of MATLAB
where the area ofwas
the
applied.
objects is This
in thefunction
specified allows
rangeextracting
producingall connected
a new binary components fromonly
image containing the binary image,
the objects thatwhere
meet the
area of theInobjects
criterion. is inthe
this case, thecriterion
specifiedwas
range producing
to extract a newobject
the larger binaryofimage containing
the image only the
(with more objects
connected
that meet the criterion.
components). This allowedIn this
thecase, the criterion
system to eliminatewasnoises
to extract
and the larger object
unconnected of theThe
pixels. image (with
resulting
more connected
binary image (mask) components). Thismultiplied
(Figure 5) was allowed the by system to eliminate
the vegetation indicesnoises andinunconnected
images, pixels.
order to extract just
Thevalues
the resulting binary
of VIs presentimage (mask)
in the plant(Figure 5) was6).
tissue (Figure multiplied by the vegetation indices images, in
order to extract just the values of VIs present in the plant tissue (Figure 6).

2.4.4. Morphological Analysis

Nutrient deficiency in tomato plants can cause several symptoms besides the change in
reflectance of leaves. It is common to observe changes in the plant growing behavior, specially related
with morphological aspects of the leaves and shoots.
Nitrogen deficiency can cause restricted shoot growth and small erect leaflets. Phosphorous
deficiency can cause restricted shoot growth and small stiffed curved leaves. Deficiency of potassium
can cause scorched and curled symptoms in old leaves. Zinc and Iron deficiencies can cause stunting.
Boron and Calcium deficiencies can cause changes in the leaflets making them curved and deformed.
Copper deficiency symptoms can be observed in the margins of leaflets and younger leaves that curl
into a tube shape, the terminal leaves can become very small, stiff and contorted and the stem growth
become somewhat stunted [41].
from the background. The vegetation indices results were stored by date and treatment in image
format seeking a visual confirmation of the segmentation process and a visual representation of the
spectral responses (Figures 6–8).
The images were taken in the same position during the weeks in order to observe the growth
behavior of the plants in the different treatments. Both aspects could be observed during this time:
Sensors 2020, 20, 435 9 of 21
morphological traits and vegetation indices (Figure 8).

Figure 6. Example of vegetation indices obtained through the Parrot Sequoia camera and segmentation
Figure 6. Example of vegetation indices obtained through the Parrot Sequoia camera and
algorithm. NDVI image and phenological state (BBCH scale) of the T0 treatment (left) and the T3
segmentation algorithm. NDVI image and phenological state (BBCH scale) of the T0 treatment (left)
treatment (right).
and the T3 treatment (right).
2.4.4. Morphological Analysis
Nutrient deficiency in tomato plants can cause several symptoms besides the change in reflectance
of leaves. It is common to observe changes in the plant growing behavior, specially related with
morphological aspects of the leaves and shoots.
Nitrogen deficiency can cause restricted shoot growth and small erect leaflets. Phosphorous
deficiency can cause restricted shoot growth and small stiffed curved leaves. Deficiency of potassium
can cause scorched and curled symptoms in old leaves. Zinc and Iron deficiencies can cause stunting.
Boron and Calcium deficiencies can cause changes in the leaflets making them curved and deformed.
Copper deficiency symptoms can be observed in the margins of leaflets and younger leaves that curl
into a tube shape, the terminal leaves can become very small, stiff and contorted and the stem growth
become somewhat stunted [41].
From the binary plant datasets, it is possible to measure plant size, shape, area and other features
in an automated way and correlate plant phenotypes with experimental treatments [42,43].
Several other morphological properties can be extracted from binary images using the regionprops
function of MATLAB. This function uses the distribution of the pixels to calculate shape parameters and
has been recently used for image analysis in medical studies [44], industry [45], plant pathology [46]
and plant growth analysis.
These parameters can provide reliable information about the changes in the plants’ area, perimeter,
shape and growth behaviour.
Not all parameters resulting from the regionprops function were used, and the description of the
morphological properties used for the tomatoes plants analysis can be observed in Table 3.
Sensors 2020, 20, 435 10 of 21

Table 3. Morphological properties calculated for the tomato plants at early stages images with different
levels of organic fertilization.

Morphological Property Description

Area Actual number of pixels in the region, returned as a scalar.
Number of pixels in the image that specifies the convex hull, with all pixels
Convex Area within the hull filled in (set to on), returned as a binary image (logical). The
image is the size of the bounding box * of the region.
Eccentricity of the ellipse that has the same second-moments as the region,
returned as a scalar. The eccentricity is the ratio of the distance between the
Eccentricity foci of the ellipse and its major axis length. The value is between 0 and 1. (0
and 1 are degenerate cases. An ellipse whose eccentricity is 0 is a circle, while
an ellipse whose eccentricity is 1 is a line segment.)
Diameter of a circle with the same area as the region, returned as a scalar.
Diameter Equivalent q
Computed as 4 ∗ Area π .

Number of objects in the region minus the number of holes in those objects,
Euler Number
returned as a scalar.
Ratio of pixels in the region to pixels in the total bounding box *, returned as a
Extent
scalar. Computed as the area divided by the area of the bounding box *.
Filled Area Number of on pixels in filled image, returned as a scalar.
Angle between the x-axis and the major axis of the ellipse that has the same
Orientation second-moments as the region, returned as a scalar. The value is in degrees,
ranging from −90 degrees to 90 degrees.
Length (in pixels) of the major axis of the ellipse that has the same normalized
Major Axis Length
second central moments as the region, returned as a scalar.
Length (in pixels) of the minor axis of the ellipse that has the same normalized
Minor Axis Length
second central moments as the region, returned as a scalar
Distance around the boundary of the region returned as a scalar. This
Perimeter function computes the perimeter by calculating the distance between each
adjoining pair of pixels around the border of the region
Proportion of the pixels in the convex hull that are also in the region, returned
Solidity
as a scalar. Computed as area/convex area
* Bounding box: Smallest rectangle containing the region.

2.4.5. Statistical Analysis

The experiment used randomized blocks, with four treatments and six repetitions. Each plot was
composed of six pots. The statistical analysis carried out through the ANOVA and Tukey’s HSD test
with 99% of confidence for average comparisons using the statistical and machine learning toolbox
from MATLAB software. ANOVA and Tukey’s HSD test were also performed with 95% confidence,
but since there it was no difference in terms of significant factors, the 99% was chosen for this study.
For the significant predictor factors, a functional boxplot and a regression analysis (using the week
means) was performed in the same software.

3. Results and Discussion

Images were collected weekly (one image per plant) during the course of eight weeks.
The segmentation of the plant tissue from the background was needed to quantify the spectral
response from each plant and to calculate the mean values of the evaluated parameters.
This process also enabled the morphological analysis of the plant growth behavior. Multiple
segmentation codes were tested to remove the background, and the code based on the normalized
difference vegetation index (NDVI) was the most efficient.
All the other bands (red, red edge and green) alone were not able to differentiate the plant tissue
from the background. The vegetation indices results were stored by date and treatment in image
format seeking a visual confirmation of the segmentation process and a visual representation of the
spectral responses (Figures 6–8).
Sensors 2020, 20, x FOR PEER REVIEW 11 of 21

Sensors 2020, 20, 435 11 of 21

Sensors 2020, 20, x FOR PEER REVIEW 11 of 21

Figure 7. Example of vegetation indices obtained through the Parrot Sequoia camera and
segmentation algorithm. Modified Simple Ratio (MSR) image and phenological state (BBCH scale) of
theFigure
T0Figure7. Example
treatment (left)of
7. Example vegetation
and indicesindices
thevegetation
of T3 treatment obtained
(right). throughthrough
obtained the Parrot
theSequoia
Parrotcamera
Sequoiaandcamera
segmentation
and
algorithm. Modified Simple Ratio (MSR) image and phenological state (BBCH
segmentation algorithm. Modified Simple Ratio (MSR) image and phenological state (BBCHscale) of the T0scale)
treatment
of
(left)
theand the T3 treatment
T0 treatment (left) and(right).
the T3 treatment (right).

Figure 8. Evolution of the morphology and Normalized Difference Vegetation Index of the tomato
plants according to the fertilization treatments, the DAT (days after transplant) and the phenological
state (BBCH scale).
Sensors 2020, 20, 435 12 of 21

The images were taken in the same position during the weeks in order to observe the growth
behavior of the plants in the different treatments. Both aspects could be observed during this time:
morphological traits and vegetation indices (Figure 8).
The average values of the different vegetation indices per plant were calculated and compared
during each week of the experiment. Parameters that showed significant difference and correspondent
p-values are displayed below (Table 4).

Table 4. p-values for the parameters (that presented significative difference) analyzed with multispectral
images in tomato plants with different organic fertilization levels. (ANOVA Significance test 99%).

Time After Transplant (Days)

08-mar 15-mar 22-mar 29-mar 05-abr 12-abr 19-abr 26-abr
0 7 14 21 28 36 43 50
NDVI n.s n.s n.s n.s n.s n.s 0.02 ** 0.05 **
MSR n.s n.s n.s n.s n.s n.s 0.074 ** 0.03 **
GRVI n.s n.s n.s n.s n.s n.s 0.013 ** 0.103 **
Area n.s n.s n.s 0.002 ** <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 **
MajorAxisLength n.s n.s n.s <0.0001 <0.0001 ** 0.001 ** 0.0008 ** <0.0001 **
MinorAxisLength n.s n.s n.s 0.0007 ** <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 **
ConvexArea n.s n.s n.s <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 **
FilledArea n.s n.s n.s 0.02 ** <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 **
Perimeter n.s n.s n.s 0.0016 ** <0.0001 ** <0.0001 ** <0.0001 ** 0.0002 **
EquivDiameter n.s n.s n.s <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 ** <0.0001 **
**: Significative values with 99% of confidence.

Aiming to identify the most reliable parameters to predict the fertilization level of organic
tomato, boxplots were created with the whole amount of data, without week distinction. After that,
a comparison test (Tukey’s HSD) was performed to find which parameters are more robust to use in an
automatic and nondestructive fertilization level analysis model.
The graphical representation of the boxplots and Tukey’s comparison test can be seen in Figure 9.
In the case of the morphological aspects, some of the parameters presented significant results with
distinction between the control (T0) and the treatments (T1, T2, and T3) (Figure 9). On the other
hand, none of the vegetation indices presented a significative difference (with a 99% or 95% level of
confidence) to predict the nutritional level of organic tomato plants independent of the week after
transplant (Figure 10).
The results presented in Table 4, as well as Figures 9 and 10 indicate that the morphological
parameters are more related to the fertilization level of tomato plants at early stages than the spectral
responses. These parameters were selected to create regression models in order to predict the nutritional
status of the tomato crop.
The regression models were created using the MATLAB function curvefit, the mean values of
the control (T0), and the recommended dose (T3), for the first five weeks and excluding outliers. The
selected parameters were: area, filled area, convex area, perimeter, equivalent diameter, major axis
length and minor axis length (Table 5).
Sensors 2020, 20, 435 13 of 21
Sensors 2020, 20, x FOR PEER REVIEW 13 of 21

Figure 9. Boxplots and Tukey

Figure 9. Tukey Test
Testwith
with99%
99%of
of probability
probability using
using all
all data
data (50
(50 days)
days) representing
representing morphological
morphological parameters
parameters of
of tomato
tomato plants
plants with
with different
different levels
levels of
of
organic fertilization. T0, T1, T2, and T3 were the four groups of plants treated with different amounts of fertilizer and vermicompost (see Section
organic fertilization. T0, T1, T2, and T3 were the four groups of plants treated with different amounts of fertilizer and vermicompost (see Section 2). 2).
Sensors 2020, 20, 435 14 of 21
Sensors 2020, 20, x FOR PEER REVIEW 14 of 21

Figure
Figure 10.10.Boxplots
Boxplotsand
andTukey
TukeyTest
Testwith
with99%
99%ofofprobability
probabilityusing
usingall
alldata
data (50
(50 days)
days) representing vegetation indices
representing vegetation indices of
oftomato
tomatoplants
plantswith
withdifferent
differentlevels
levelsofoforganic
organic
fertilization.
fertilization.T0,
T0,T1,
T1,T2,
T2,and
andT3
T3were
werethe
thefour
fourgroups
groupsofofplants
plantstreated
treatedwith
withdifferent
differentamounts
amountsofof fertilizer
fertilizer and
and vermicompost (see Section
vermicompost (see Section 2).
2).
 Sensors 2020, 20, 435 15 of 21

Sensors 2020, 20, x FOR PEER REVIEW 15 of 21

Table 5. Linear and polynomial regressions correlating morphological parameters were acquired using
multispectral
Table 5. Linearimages to weeks after
and polynomial transplant
regressions in different
correlating organic fertilization
morphological parameters levels
wereinacquired
tomato plants
at early
using stages. images to weeks after transplant in different organic fertilization levels in tomato
multispectral
plants at early stages.
Parameter Tr. Regression R2
Parameter Tr. Regression 2
T0 f (x) = 10070x + 26550 𝑅 0.937
Area 2
T0 T3 𝑓 𝑥 f (x) = +2825x
10070𝑥 26550 + 3078x + 8909
0.937 0.983
Area
T3 T0 f (x) = 6766x + 8284 0.983 0.953
Filled Area 𝑓 𝑥 2825𝑥 + 3078𝑥 +28909
T3 f (x) = 3118x + 2072x + 13130 0.977
T0 𝑓 𝑥 6766𝑥 + 8284 0.953
Filled Area T0 f (x) = 164.5x + 2211 0.559
Perimeter 2 − 1065x + 0.977
T3 T3 𝑓 𝑥 3118𝑥
f (x+
)= 381.7x
2072𝑥 + 13130 3265 0.941
T0 T0 𝑓 𝑥 f (x+
164.5𝑥 )= 21.93x + 114.40.559
2211 0.938
Eq. Diameter
Perimeter T3 f (x) = 52.65x + 80.43 0.971
T3 𝑓 𝑥 381.7𝑥 − 1065𝑥 +23265 0.941
T0 f (x) = 2984x − 8845x + 56880 0.957
Convex Area T0 T3 𝑓 𝑥 f (21.93𝑥 + 114.42 − 60980x + 0.938
x) = 19530x 95950 0.992
Eq. Diameter
T3 T0 𝑓 𝑥 f (52.65𝑥
x) = 16.53x 2 0.971
+ 80.43 − 112.2x + 522.2 0.933
Major Axes 2
T3 f (x) = 27.41x − 106.9x + 452.5 0.988
T0 𝑓 𝑥 2984𝑥 − 8845𝑥 + 56880 0.957
Convex Area T0 f (x) = −4.833x2 + 61.4x + 77.95 0.976
Minor Axes T3 + 95950+ 69.480.992
T3 𝑓 𝑥 19530𝑥 −f60980𝑥
(x) = 71.54x 0.964
T0 𝑓 𝑥 16.53𝑥 − 112.2𝑥 + 522.2 0.933
Major Axes
Seeking for a more robust, simple
T3 and
𝑓 𝑥
reliable prediction model, functional
27.41𝑥 − 106.9𝑥 + 452.5 0.988 boxplots were created
using the mean values of each plant T0 and T3 for the first four weeks and plotted together. The
T0 𝑓 𝑥 −4.833𝑥 + 61.4𝑥 + 77.95 0.976
functional boxplots represent
Minor Axes the mean data of each week without the outliers with an interval of
confidence of 95% (Figure 11). T3 𝑓 𝑥 71.54𝑥 + 69.48 0.964

Figure11.
Figure Functionalboxplots
11.Functional boxplots showing
showing morphological
morphological responses
responses extracted
extracted from
frommultispectral
multispectralimages
correlating
images with number
correlating of weeksofafter
with number transplant
weeks and fertilization
after transplant treatments with
and fertilization a confidence
treatments with interval
a
of 95%. T0, T1, T2, and T3 were the four groups of plants treated with different amounts
confidence interval of 95%. T0, T1, T2, and T3 were the four groups of plants treated with different of fertilizer
and vermicompost
amounts (seevermicompost
of fertilizer and Section 2). (see Section 2).

Discussion
Sensors 2020, 20, 435 16 of 21

Discussion
The system used for image acquisition showed to be effective for tomato plants at early stages.
The same system could be adjusted for tomato plants at late stages, as well as for other plant species
and seedlings in different stages (depending on the height adjustments). The pre-processing technique
showed to be adequate for the overlay of the Parrot Sequoia images in the different heights tested
0.70 m and 1.40 m.
Plant segmentation is the cornerstone of the image analysis at plant level. Diverse authors used
multispectral images to access the nutritional level of the plants. These works are usually focused in
small zones or areas, or in multiple plants in a parcel as the minimum management unit. In order to
perform precise fertilization at the plant level, the segmentation and extraction of the plant of interest
must be also precise. The algorithm developed using the normalized difference index was able to
distinguish plant tissue from soil, pot, floor and metal platform extracting the whole plant apart of
the background (Figures 4–6). The use of NDVI for vegetation extraction and monitoring is widely
used for satellite imagery [47–49] and this study proves that can also be used with high-resolution
plant images. The code was applied in different conditions with good efficacy in extract the plant. The
visual inspection was needed to guarantee that the plants were well-segmented and also provides
the distribution of the vegetation indices intensity through the leaf tissue. The segmentation also
allows morphological analysis during the time, and this system can be useful for digital phenotyping
projects. However, the code can present limitations for morphological analysis of plants with presence
of senescent leaves since the reflectance of yellow and/or brown leaves are different from the active
vegetative tissue.
The vegetation indices used in this study did not show significant differences between the
treatments during the initial stages of the plant development, but there were significant differences in
the 6th week after the transplant between T0 and T2 and T0 and T1. Among all the studied vegetation
indices, the NDVI, GRVI, OSAVI, SR, and MSR were sensible to these differences. Different results
were found by [50], that using the crop circle ACS 470 and the SPAD 502 sensors concluded that the
vegetation indices based on the red band (NDVI and red vegetation Index) were the best predictors for
nitrogen levels in tomatoes maintaining the relationships during the crop.
According to the authors [50], the SPAD readings, the GNDVI, and the GVI were good predictors
of the nitrogen status of the crop in the beginning of the cycle, with low accuracy in the later part of
the crop cycle. Using the same sensor (Circle ACS 470) and guided sampling method, [51] developed
a yield prediction map of tomato crop. Their study used the NDVI index as a predictor parameter
and the correlation between the vegetation index and the yield were of 0.67 and 0.71 in two different
regions of Spain. Another study conducted by [52] showed the possibility to use vegetation indices to
improve the use of nitrogen fertilization in tomatoes cultivated in protected environment. Applying
the spectral sensors ASD Field Spec HandHeld 2, and the Minolta SPAD 502, the authors compared the
spectral response of tomato plants in the reference plot and with lower levels of nitrogen fertilization.
The use of NDVI and the SPAD measurements as a nitrogen level parameter allowed to reduce
significantly the use of nitrogen fertilization maintaining the same yield levels and fruit quality
compared to the reference plot [52]. On the other hand, during three years of field trials with different
levels of nitrogen fertilization [53] evaluated vegetation indices of tomato plots through the use
of a multispectral radiometer (MSR-87, Cropscan Inc., Rochesters, MA, USA). In their study [53],
sixteen vegetation indices were analyzed and the NDVI did not show a strong correlation with nitrogen
concentration on leaves. It was concluded that the best vegetation index for nitrogen evaluation in
tomato was the relation between NIR and the reflectance present in the range of 560 nm (NIR/R560).
Some factors can be responsible for the divergence observed between literature results.
The first factor is the type of fertilizer, in all the previous studies, mineral nitrogen fertilization
was used. This type of fertilizer is more soluble and less stable and can be absorbed and translocated
faster than the organic amendments. The crop stage during the measurements can also be a factor. In
the previous studies, the measurements started after the 29th day after transplant where the plants
Sensors 2020, 20, 435 17 of 21

were developed more, and the deficiencies symptoms were more present. The type of sensor is
also an important factor because this was the first study to use the Parrot Sequoia sensor at plant
level to monitor nutrient deficiency in tomatoes. [54], showed the differences in measurements in
different commercial spectral sensors for nitrogen management, and among the studied factors was
concluded that the measuring distance, the device temperature and the light intensity can influence
the performance of the sensors.
The most interesting results were obtained through the analysis of the morphological traits of
the tomato plants. Seven of the twelve aspects analyzed showed significant difference between the
control and the fertilized treatments. Only three weeks after the transplant, the plants started to exhibit
a significant difference in the plant area, convex area, filled area, minor axis length and equivalent
diameter discriminating T3 and T2 of T1 and the control. In the same period, the perimeter was
significantly different between the recommended dose T3 and the control, without difference between
T2 and T1. The length of the major axis could discriminate between the control against the treatments
but not between the treatments already in the third week. In the following weeks, from the fourth week
to the end of the study, all the seven morphological traits (area, convex area, filled area, equivalent
diameter, perimeter, minor axis length and minor axis length) showed significant differences between
the control and the treatments, except for the major axis length in the seventh week after transplant.
These results are in agreement with the results described in the literature. Mooy et al. [55] showed
the positive influence of soluble organic fertilizer in tomato plant height and the number of leaves. Other
studies also demonstrated that the nitrogen and phosphorous fertilization influences the fresh shoot
weight, plant height, stem diameter, leaf number, and leaf area of tomato seedlings [56–59]. In field
trials, the nitrogen fertilization produced an increase in the leaf area index and in the above-ground
weight of tomato plants [60] and the use of NPK induce an increase in plant height and in the number
of leaves [61]. Besides the slight difference in the morphological parameters between the fertilized
treatments T1, T2, and T3, there was no significant difference with 1% of probability, because, for the
functional boxplot analysis, the treatment with the recommended dose (T3) was chosen for comparison
with the control (T0).
The use of functional boxplots as a predicting model is interesting because they can exclude
outliers and can be used as a threshold parameter [62]. As can be observed, the functional boxplots
displays the range of the observed values until the 4th week after transplant and not until the Week 7.
At Week 5 the plants were too high to be observed with 0.70 m height as was planned for the robotic
prototype, so the camera bracket was set for a 1.4 m height. This adjustment in the camera distance
cause differences in the relationship between morphological parameters and the number of pixels.

4. Conclusions
The computer vision methodology developed in this article was suitable for monitoring the
development of tomato var. Mina F1 at early stages with different levels of organic fertilization in a
protected environment. The methodology involves consecutive steps, and using an automatic and
non-destructive approach provides several morphological and spectral aspects of the plants that can
be used in a robotic platform.
After the pre-processing, the NDVI threshold showed to be crucial and effective for plant extraction.
Among the several spectral (vegetation indices) responses observed, the NDVI, OSAVI, simple ratio,
GRVI, and modified simple ratio were the only ones that showed a small difference between treatments
in the sixth and seventh weeks after transplant, but without practical applications.
Studying the evolution of a crop by using multiples images allowed to extract several morphological
features. The automatic morphological analysis was able to distinguish between the control and the
fertilized treatments from the third week after transplant until the end of the experiment with 99% of
confidence according to the Tukey’s honest significant differences test.
Among the studied morphological parameters, area, convex area, filled area, perimeter, major
axis length, minor axis length and equivalent diameter showed to be useful to distinguish between the
Sensors 2020, 20, 435 18 of 21

control and fertilized treatments. Convex area was the parameter that presented the highest difference
between control and the fertilized plants according to the functional boxplot analysis.
This was the first study to relate the use of morphological features extracted automatically with
the organic fertilization level in vegetable plants. The image analysis system was able to distinguish
plant tissue from soil, floor, and metal structures. Therefore, most of the image processing steps
can be extrapolated to other vegetable cultures, although future analysis is necessary to evaluate its
robustness. Moreover, the Parrot Sequoia camera fulfilled the requirements to be used in the robotic
platform and besides be originally developed for aerial images this study proved the capacity of its use
at short distances.
Additionally, the same methodology can be used for analyzing other aspects related with
morphology features in tomato plants, such as disease severity, hydric and salinity stress or any
environmental modification that can cause aboveground changes in the morphology or spectral
responses of vegetable plants. This technique has potential to be used in many branches of plant health
studies and provides an innovative and automatic way to measure the severity of abiotic disorders.
In a subsequent investigation, we are developing an extended version of the methodology to be
applied using the robotic platform at the field level. Trials are being conducted in an experimental line
with different vegetable crops to study the best image acquisition interval in a scenario with movement.
Other trials using the robotic platform are being conducted in an experimental field area with different
crops such as onion, pumpkins, tomatoes, and zucchini with different levels of fertilization. In these
trials, the multispectral camera is being integrated with laser sensors (Lidar) for a multiscale analysis
of the crops.

Author Contributions: Data Curation, A.K.; Funding Acquisition, A.B. and J.d.C.; Investigation, M.C.F.L., A.K.
and C.V.; Methodology, M.C.F.L.; Project Administration, A.B. and J.d.C.; Resources, C.V., A.B., J.d.C. and J.J.R.-G.;
Software, M.C.F.L. and A.K.; Supervision, C.V.; Visualization, M.C.F.L. and A.K.; Writing-Original Draft, M.C.F.L.;
Writing-Review & Editing, M.C.F.L., C.V. and J.J.R.-G. All authors have read and agreed to the published version
of the manuscript.
Funding: Transnational funding bodies, being partners of the H2020 ERA-net project, CORE Organic Cofund,
and the cofund from the European Commission.
Acknowledgments: This article has been developed as a result of a mobility stay funded by the Erasmus+—KA1
Erasmus Mundus Joint Master Degrees Programme of the European Commission under the PLANT HEALTH
Project. The authors acknowledge the financial support for this project provided by transnational funding
bodies, being partners of the H2020 ERA-net project, CORE Organic Cofund, and the cofund from the
European Commission.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Hidalgo-Baz, M.; Martos-Partal, M.; González-Benito, O. Attitudes vs. purchase behaviors as experienced
dissonance: The roles of knowledge and consumer orientations in organic market. Front. Psychol. 2017, 8, 248.
[CrossRef]
2. Pino, G.; Peluso, A.M.; Guido, G. Determinants of regular and occasional consumers’ intentions to buy
organic food. J. Consum. Aff. 2012, 46, 157–169. [CrossRef]
3. Ribeiro, A.C.; Guimarães, P.T.G.; Alvarez, V.V.H. Recomendações Para o Uso de Corretivos e Fertilizantes em
Minas Gerais—5◦ Aproximação, 1st ed.; Universidade Federal de Viçosa: Viçosa, Brazil, 1999; p. 359.
4. Agrios, G.N. Introduction to Plant Pathology; Elsevier Academic Press Publication: Amsterdam, The Netherlands,
2005; pp. 257–262.
5. Brown, J.F.; Ogle, H.J. Plant Pathogens and Plant Diseases; Rockvale Publications for the Division of Botany:
Armidale, Australia, 1997; pp. 156–157.
6. Wang, Y.; Ying, H.; Yin, Y.; Zheng, H.; Cui, Z. Estimating soil nitrate leaching of nitrogen fertilizer from
global meta-analysis. Sci. Total Environ. 2019, 657, 96–102. [CrossRef]
7. Du, Y.D.; Niu, W.Q.; Gu, X.B.; Zhang, Q.; Cui, B.J. Water-and nitrogen-saving potentials in tomato production:
A meta-analysis. Agric. Water Manag. 2018, 210, 296–303. [CrossRef]
Sensors 2020, 20, 435 19 of 21

8. Pedersen, S.M.; Lind, K.M. Precision Agriculture–From Mapping to Site-Specific Application. In Precision
Agriculture: Technology and Economic Perspectives; Springer: Cham, Switzerland, 2017; p. 25.
9. Chen, C.; Pan, J.; Lam, S.K. A review of precision fertilization research. Environ. Earth Sci. 2014, 71, 4073–4080.
[CrossRef]
10. Candiago, S.; Remondino, F.; De Giglio, M.; Dubbini, M.; Gatteli, M. Evaluating multispectral images and
vegetation indices for precision farming applications from UAV images. Remote Sens. 2015, 7, 4026–4047.
[CrossRef]
11. Pallottino, F.; Antonucci, F.; Costa, C.; Bisaglia, C.; Figorilli, S.; Menesatti, P. Optoelectronic proximal sensing
vehicle-mounted technologies in precision agriculture: A review. Comput. Electron. Agric. 2019, 162, 859–873.
[CrossRef]
12. Prey, L.; Von Bloh, M.; Scgumidhalter, U. Evaluating RGB imaging and multispectral active and hyperspectral
passive sensing for assessing early plant vigor in winter wheat. Sensors 2018, 18, 2931. [CrossRef]
13. Kitic, G.; Tagarakis, A.; Cselyuszka, N.; Panic, M.; Birgermajer, S.; Sakulski, D.; Matovik, J. A new low-cost
portable multispectral optical device for precise plant status assessment. Comput. Electron. Agric. 2019, 162,
300–308. [CrossRef]
14. Freidenreich, A.; Barraza, G.; Jayachandran, K.; Khoddamzadeg, A.A. Precision Agriculture Application for
Sustainable Nitrogen Management of Justicia brandegeana Using Optical Sensor Technology. Agriculture
2019, 9, 98. [CrossRef]
15. Liu, L.; Song, B.; Zhang, S.; Liu, X. A Novel Principal Component Analysis Method for the Reconstruction of
Leaf Reflectance Spectra and Retrieval of Leaf Biochemical Contents. Remote Sens. 2017, 9, 1–23. [CrossRef]
16. Osco, L.P.; de Arruda, M.D.S.; Junior, J.M.; da Silva, N.B.; Ramos, A.P.M.; Moryia, É.A.S.; NobuhiroImai, N.;
RobertoPereira, D.; EduardoCreste, J.; TakashiMatsubara, E.; et al. A convolutional neural network approach
for counting and geolocating citrus-trees in UAV multispectral imagery. ISPRS J. Photogramm. Remote Sens.
2020, 160, 97–106. [CrossRef]
17. Wallis, C.I.; Homeier, J.; Peña, J.; Brandl, R.; Farwig, N.; Bendix, J. Modeling tropical montane forest biomass;
productivity and canopy traits with multispectral remote sensing data. Remote Sens. Environ. 2019, 225,
77–92. [CrossRef]
18. Zeng, L.; Wardlow, B.D.; Xiang, D.; Hu, S.; Li, D.A. Review of vegetation phenological metrics extraction
using time-series, multispectral satellite data. Remote Sens. Environ. 2020, 237, 1–20. [CrossRef]
19. Meddens, A.J.; Hicke, J.A.; Vierling, L.A. Evaluating the potential of multispectral imagery to map multiple
stages of tree mortality. Remote Sens. Environ. 2011, 115, 1632–1642. [CrossRef]
20. Huang, Y.; Lan, Y.; Hoffmann, W.C. Use of airborne multi-spectral imagery in pest management systems.
Agricultural Engineering International: CIGR J. 2008, 10, 1–14.
21. West, H.; Quinn, N.; Horswell, M. Remote sensing for drought monitoring & impact assessment: Progress,
past challenges and future opportunities. Remote Sens. Environ. 2019, 232, 1–14.
22. Lu, R. Multispectral imaging for predicting firmness and soluble solids content of apple fruit. Postharvest
Biol. Technol. 2004, 31, 47–157. [CrossRef]
23. Lleó, L.; Barreiro, P.; Ruiz-Altisent, M.; Herrero, A. Multispectral images of peach related to firmness and
maturity at harvest. J. Food Eng. 2009, 93, 229–235. [CrossRef]
24. Gerhards, R.; Oebel, H. Practical experiences with a system for site-specific weed control in arable crops
using real-time image analysis and GPS-controlled patch spraying. Weed Res. 2006, 46, 185–193. [CrossRef]
25. Christensen, S.; Søgaard, H.T.; Kudsk, P.; Nørrenmark, M.; Lund, I.; Nadimi, E.S.; Jørgensen, R. Site-specific
weed control technologies. Weed Res. 2009, 49, 233–241. [CrossRef]
26. Ávila-Navarro, J.; Franco, C.A.; Rasmussen, J.; Nielsen, J. Color classification methods for perennial weed
detection in cereal crops. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications,
Proceedings of the 23rd Iberoamerican Congress, CIARP 2018, Madrid, Spain, 19–22 November 2018; Springer:
Cham, Switzerland, 2019; pp. 117–123.
27. Partel, V.; Kakarla, S.C.; Ampatzidis, Y. Development and evaluation of a low-cost and smart technology
for precision weed management utilizing artificial intelligence. Comput. Electron. Agric. 2019, 157, 339–350.
[CrossRef]
28. Ampatzidis, Y.; De Bellis, L.; Luvisi, A. iPathology: Robotic applications and management of plants and
plant diseases. Sustainability 2017, 9, 1010–1024. [CrossRef]
Sensors 2020, 20, 435 20 of 21

29. Cavender-Bares, K.; Bares, C. Robotic Platform and Method for Performing Multiple Functions in Agricultural
Systems. U.S. Patent Application n. 16/188,422, 14 March 2019.
30. Veys, C.; Chatziavgerinos, F.; AlSuwaidi, A.; Hibbert, J.; Hansen, M.; Bernotas, G.; Smith, M.; Yin, H.; Rolfe, S.;
Grieve, B. Multispectral imaging for presymptomatic analysis of light leaf spot in oilseed rape. Plant Methods
2019, 15, 4. [CrossRef]
31. Grimstad, L.; From, P. The Thorvald II agricultural robotic system. Robotics 2017, 6, 24–38. [CrossRef]
32. Shamshiri, R.R.; Weltzien, C.; Hameed, I.A.; Yule, I.J.; Grift, T.E.; Balasundram, S.K.; Pitonakova, L.;
Ahmad, D.; Chowdhary, G. Research and development in agricultural robotics: A perspective of digital
farming. Int. J. Agric. Biol. Eng. 2018, 11, 1–14. [CrossRef]
33. King, A. The future of agriculture. Nature 2017, 544, S21–S23. [CrossRef]
34. Xue, J.; Fan, Y.; Su, B.; Fuentes, S. Assessment of canopy vigor ınformation from kiwifruit plants based on
a digital surface model from unmanned aerial vehicle ımagery. Int. J. Agric. Biol. Eng. 2019, 12, 165–171.
[CrossRef]
35. Xue, J.; Su, B. Significant Remote Sensing Vegetation Indices: A Review of Developments and Applications.
J. Sens. 2017, 2017, 1–17. [CrossRef]
36. Moriarty, C.; Cowley, D.C.; Wade, T.; Nichol, C.J. Deploying multispectral remote sensing for multi-temporal
analysis of archaeological crop stress at Ravenshall, Fife, Scotland. Archaeol. Prospect. 2019, 26, 33–46.
[CrossRef]
37. Fahlgren, N.; Gehan, M.A.; Baxter, I. Lights, camera, action: High-throughput plant phenotyping is ready for
a close-up. Curr. Opin. Plant Biol. 2015, 24, 93–99. [CrossRef]
38. Berry, J.C.; Fahlgren, N.; Pokorny, A.A.; Bart, R.S.; Veey, K.M. An automated, high-throughput method
for standardizing image color profiles to improve image-based plant phenotyping. PeerJ 2018, 6, e5727.
[CrossRef]
39. Yogamalagan, R.; Karthikeyan, B. Segmentation techniques comparison in image processing. Int. J. Eng.
Technol. 2013, 5, 307–313.
40. Soille, P. Morphological image analysis applied to crop field mapping. Image Vis. Comput. 2010, 18, 1025–1032.
[CrossRef]
41. Van Eysinga, J.R.; Smilde, K.W. Nutritional Disorders in Glasshouse Tomatoes, Cucumbers and Lettuce; Centre for
Agricultural Publishing and Documentation: Wageningen, The Netherlands, 1981.
42. Veley, K.; Berry, J.; Fentress, S.; Schachtman, D.; Baxter, I.; Bart, R. High-throughput profiling and analysis of
plant responses over time to abiotic stress. Plant Direct 2017, 1, e00023. [CrossRef]
43. Liang, Z.; Pandey, P.; Stoerger, V.; Xu, Y.; Qiu, Y.; Ge, Y.; Schnable, C. Conventional and hyperspectral
time-series imaging of maize lines widely used in field trials. Gigascience 2018, 7, 1–11. [CrossRef]
44. Tariq, M.; Siddiqi, A.; Narejo, G.B.; Andleeb, S. A Cross Sectional Study of Tumors Using Bio-medical
Imaging Modalities. Curr. Med. Imaging Rev. 2019, 15, 66–73. [CrossRef]
45. Jiang, J. Design of Meshing Assembly Algorithms for Industrial Gears Based on Image Recognition.
In International Conference on Advanced Information Systems Engineering; Springer: Cham, Switzerland, 2019;
pp. 64–72.
46. Singh, K.; Kumar, S.; Kaur, P. Automatic detection of rust disease of Lentil by machine learning system using
microscopic images. Int. J. Electr. Comput. Eng. (IJECE) 2019, 9, 660–666. [CrossRef]
47. Hsieh, H.; Chung, C.; Huang, C. Using the NDVI and mean shift segmentation to extract landslide areas in
the Lioukuei Experimental Forest region with multi-temporal FORMOSAT-2 images. Taiwan J. For. Sci. 2017,
32, 203–222.
48. Bhandari, A.K.; Kumar, A.; Singh, K. Feature Extraction using Normalized Difference Vegetation Index
(NDVI): A Case Study of Jabalpur City. Procedia Technol. 2012, 6, 612–621. [CrossRef]
49. Manakos, I.; Braun, M. Land Use and Land Cover Mapping in Europe: Practices & Trends, Remote Sensing
and Digital Image Processing 18; Springer Science + Business Media: Berlin/Heidelberg, Germany, 2014;
pp. 363–381.
50. Padilla, F.M.; Peña-Fleitas, M.T.; Gallardo, M.; Thompson, R.B. Threshold values of canopy reflectance indices
and chlorophyll meter readings for optimal nitrogen nutrition of tomato. Ann. Appl. Biol. 2015, 166, 271–285.
[CrossRef]
Sensors 2020, 20, 435 21 of 21

51. Fortes, R.; Prieto, M.H.; García-Martin, A.; Córdoba, A.; Martínez, L.; Campillo, C. Using NDVI and guided
sampling to develop yield prediction maps of processing tomato crop. Span. J. Agric. Res. 2015, 16, 1–9.
[CrossRef]
52. Oliveira, T.F.; Pinto, F.A.C.; Silva, D.J.H. Spectral Vegetation Indices applied to Nitrogen Sufficiency Index:
A Strategy with Potential to Increase Nitrogen Use Efficiency in Tomato Crop. Eng. Agrícola 2019, 39, 118–126.
[CrossRef]
53. Gianquinto, G.; Orsini, F.; Fecondini, M.; Mezzetti, M.; Sambo, P.; Bona, S. A methodological approach for
defining spectral indices for assessing tomato nitrogen status and yield. Eur. J. Agron. 2011, 35, 135–143.
[CrossRef]
54. Kipp, S.; Mistele, B.; Schimidhalter, U. The performance of active spectral reflectance sensor as influenced
by measuring distance, device temperature and light intensity. Comput. Electron. Agric. 2014, 100, 24–33.
[CrossRef]
55. Mooy, L.M.; Hasan, A.; Onsili, R. Growth and yield of Tomato (Lycopersicum esculentum Mill.) as influenced
by the combination of liquid organic fertilizer concentration and branch pruning. IOP Conf. Ser. Earth
Environ. Sci. 2019, 260, 1–8. [CrossRef]
56. Nicola, S.; Basoccu, L. Nitrogen and N, P, K relation affect tomato seedling growth, yield and earliness. In III
International Symposium on Protected Cultivation in Mild Winter Climates; ISHS Acta Horticulturae 357; ISHS:
Buenos Aires, Argentina, 1994; Volume 357, pp. 95–102.
57. Basoccu, L.; Nicola, S. Supplementary light and pretransplant nitrogen effects on tomato seedling growth
and yield. Hydroponics Transpl. Prod. 1994, 396, 313–320. [CrossRef]
58. Melton, R.R.; Dufault, R.J. Nitrogen, phosphorus, and potassium fertility regimes affect tomato transplant
growth. HortScience 1991, 26, 141–142. [CrossRef]
59. Scholberg, J.; Mcneal, B.L.; Boote, K.J.; Jones, J.W.; Locascio, S.J.; Olson, S.M. Nitrogen stress effects on growth
and nitrogen accumulation by field-grown tomato. Agron. J. 2000, 92, 159–167. [CrossRef]
60. Elia, A.; Giulia, C. Agronomic and physiological responses of a tomato crop to nitrogen input. Eur. J. Agron.
2012, 40, 64–74. [CrossRef]
61. Hariyadi, B.W.; Nisak, F.; Nurmalasari, I.R.; Kogoya, Y. Effect of Dose and Time of Npk Fertilizer Application
on the Growth and Yield of Tomato Plants (Lycopersicum esculentum Mill). Agric. Sci. 2019, 2, 101–111.
62. Whitaker, R.T.; Mirzargar, M.; Kirby, R.M. Contour boxplots: A method for characterizing uncertainty in
feature sets from simulation ensembles. IEEE Trans. Vis. Comput. Graph. 2013, 19, 2713–2722. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).