Achiri 2018

Download as pdf or txt
Download as pdf or txt
You are on page 1of 4

SAR and AIS Fusion for Maritime Surveillance

Lotfi Achiri, Raffaella Guida, Pasquale Iervolino


Surrey Space Centre
University of Surrey
Guildford, UK
{L.Achiri, R.Guida, P.Iervolino}@surrey.ac.uk

Abstract— This paper presents a novel approach to fuse AIS payload [5]. The potential data availability of SAR images
Synthetic Aperture Radar (SAR) images and Automatic in S-Band and X-Band in addition to the AIS data provided
Identification System (AIS) data for maritime surveillance. The from the same platform encourage the research community to
procedure consists of four steps. First, ship detection is performed investigate methods and develop algorithms to make best use of
in the SAR image using a Constant False Alarm Rate (CFAR) these data sets.
algorithm; then feature extraction (ship position, heading and size)
is performed on ships detected in the SAR image, the third step The AIS system was initially developed for ship collision
consists in identifying the detected ships and extracting the same avoidance by broadcasting the ship information to other ships
features from the AIS data. The final step is to feed the fusion and AIS base stations located along coast lines. This
block with both features vectors extracted separately from the information can be dynamic including ship position, heading
SAR and AIS. Here the arithmetic mean function is established. and speed or static including ship name, IMO number and size
The algorithm is tested using simulated SAR images and AIS data. [6].
Preliminary results of the fusion of SAR and AIS data are
presented and discussed. In maritime domain, several algorithms and methods have
been developed for ship detection using SAR data. The most
Keywords—SAR, AIS, ship detection, ship monitoring, data common ship detector algorithm is the CFAR (Constant False
fusion, maritime surveillance. Alarm Rate) detector where the outcomes of the detection stage
can be validated with the AIS data as a ground truth [7].
I. INTRODUCTION However, the AIS data are often not a reliable source of
information because of the presence of some errors and
It is well known that the most efficient and cost-effective uncertainties while reporting ships information, for example,
way of international trade can be done via sea which represents error related to the ship position, heading or simply some data
more than 80% of the global trade [1]. Therefore, ship can be missing in the AIS report [8].
monitoring (including detection and identification) plays a
critical role to improve the security and safety at sea in maritime Because of the aforementioned reasons, we propose a novel
surveillance domain allowing, for example, the reporting of any approach where the AIS data are used as additional source of
illegal activities such as illegal fishing, illegal immigration and information by fusing it with SAR data to improve the security
hijacking. SAR (Synthetic Aperture Radar), is an active in maritime monitoring. First, the SAR image is pre-processed,
microwave remote sensing radar system, that can acquire data then the CFAR detector is applied to generate a map of detected
independently from daylight and weather condition [2]. ships. This will be followed by extracting the ship features (the
ship position, heading, length and width) from the map of
In the last decade, maritime surveillance domain has detected ships. These features are then concatenated so that
increased its importance and several new SAR platforms have every ship is represented by one feature vector. The same
been launched recently to enhance the security and safety at sea. features are considered for the AIS data. Vectors of features
For example, in June 2016, the successful launch of the extracted from both sources of information are fed to the fusion
Maritime Monitoring and Messaging Microsatellite (M3MSat) block where a fusion strategy based on the arithmetic mean
developed by the Canadian Space Agency (CSA) with the function is implemented [9].
collaboration of the Defense Research and Development
Canada (DRDC) is providing a potential SAR and AIS This paper is organised as follows. Section II outlines the
(Automatic Identification System) data for ship monitoring [3]. methodology and flowchart of the fusion of the SAR and AIS
Similarly, in February 2018, the successful launch of Paz data. This is followed by the preliminary results of the
satellite, the SAR observation Spanish satellite developed by implementation of the algorithm in section III. The conclusions
HisdeSAT started providing a high-resolution X-band SAR and and future work are discussed in section IV.
AIS data [4]. Also, NovaSAR-S mission, a low-cost SAR
satellite developed by Surrey Satellite Technology Ltd (SSTL) II. METHODOLOGY
with the collaboration of Airbus Defense and Space (Airbus
DC) and the support of the UK government will deliver S-Band This section outlines in detail the methodology of the
SAR data for maritime applications in addition to its secondary proposed approach. First, the features extraction from SAR
image and AIS data are presented. This is followed by fusing

978-1-5386-6282-3/18/$31.00 ©2018 IEEE


SAR and AIS feature vectors for ship monitoring in the
flowchart presented in Fig. 1. SAR image AIS Data

A. SAR processing
1) Pre-processing Pre-processing Pre-processing
Pre-processing consists of preparing the SAR image before
applying the detection algorithms. If the input is a Single Look
Complex product, pre-processing includes the calibration step CFAR
where the pixel digital number is converted to a calibrated radar
backscatter coefficient, the generation of the intensity or the
amplitude images, speckle filtering and multi-looking to reduce Feature Feature
the false alarm. Finally, the land masking is applied to consider Extraction Extraction
only the sea pixels. The land masking step is very important
because the ship detector algorithms usually produce high false
rate detection over land area [10].
Ship association
2) CFAR detector
The CFAR detector is the most common SAR-based ship
detector. It is based on two main steps. First the sea clutter is Fusion
modelled by a suitable statistical distribution. Once the sea
clutter is modelled, a threshold value is set in order to have a
constant false alarm rate according to [10]: Outcomes

Fig. 1. The flowchart for SAR and AIS data fusion.


=1− ( ) = ( ) (1)
passing by the origin and perpendicular to , . The dominant
where PFA is the false alarm probability, f(x) is the sea clutter orientation in the image which is the ship heading, corresponds
distribution and T is the threshold value. Thus, a pixel or group to the highest coefficient of Radon transform. These headings
of pixels are classified as ship if their intensity values are bigger are saved in the heading vector.
than the threshold T.
c) Ship size
3) Feature extraction
In this paper, the ship is simplified as a rectangular shape
The feature extraction is the crucial part of the proposed and the length and the width of the ship can be estimated in two
method where, for each ship detected in the previous step, steps. First, the ship signature is rotated, using the heading angle
corresponding position, heading and size are extracted from the extracted by Radon transform, and made parallel to y-axis.
SAR image. Then, the projection of the nonnull pixels over the x-axis
corresponds to the width of the ship, likewise, the projection of
a) Ship position nonnull pixels over the y-axis corresponds the length of the
The ship position (x, y) is the centre of the detected ship ship. Both projections should be multiplied by the pixel spacing
represented by its azimuth and slant range in the SAR image. to get a value of the ship dimension in meter. Finally, the length
This can be extracted by isolating every detected ship in and the width of the ship are saved in the size vector.
separated sub-image, identifying the smallest window of pixels
covering the entire ship (and only one at a time) and extracting B. AIS processing
its centre of mass. The window centre is considered as the ship The AIS data are the record of the dynamic and the static
position. The coordinates of this centre are saved in the position information of the AIS signal broadcast by ships over a period
vector. of time. The final file of the AIS data may contain duplicated
b) Ship heading information of the same ship because the same ship could report
its information more than once at the same time. Also, some
Then, the 2-D Radon transform [11] is applied to information (i.e. ship name, ship length, heading) may not be
determine the dominant heading of the ship defined by (2). provided for the monitored area. Another anomaly that may
occur in AIS data are the ship position reported at zero latitude
( , )= ( , ) ( − and zero longitude. Knowing the anomalies that the AIS data
(2) suffer from, a first step is removing them for the ship of interest
− ) (i.e. those detected also in the SAR image) [12].
where R(.) is the Radon transform, I(x,y) is the image intensity,
C. AIS and SAR fusion
is the Dirac function, is the smallest distance from the origin
of the coordinate system to the line , defined by , : = At the first stage of the development of the proposed
− , is the angle between the x-axis and the line approach, before the fusion stage, it is necessary to match the
ships with their corresponding AIS reports. This is done by
evaluating the absolute distance between the coordinates of the x
y
position of the ships detected in the SAR image and the ship
positions reported in the AIS data. Here, the minimum distance
method is used to find the SAR/AIS matching.
The fusion block for the SAR and AIS data is based on the
arithmetic mean function [9]. The features extracted from SAR
image including ship positions, heading and size (length, width)
are concatenated and assembled into one vector for each ship.
Same features are considered from the AIS data for every ship
from the SAR image. We assume that the SAR and AIS data
are acquired at the same time over a scene with low shipping
density and that all ships in the SAR image are broadcasting
their AIS signals. Finally, we assume that the SAR image Fig. 3. The outcomes of the CFAR block represent a map of detected ships.
azimuth direction is pointing to the geographic North. This last
assumption is made only for sake of computation simplicity but projection of nonnull pixels on the x-axis corresponds to the
will be relaxed in future. width of the ship. As anticipated, these features extracted from
the SAR simulated image together with ship position and
heading are concatenated in one vector which represents an
III. EXPERIMENTAL RESULTS
input for the fusion block.
The algorithm has been tested and evaluated with a
simulated data sets. The SAR image is simulated in the stripmap Table 1 summarises the outcomes of the fusion block of 10
mode with a spatial resolution of 6 meters. The size of the image ships along with their absolute and the relative error of
is 2200×3300 pixels in azimuth and range direction respectively, measurement. In the mathematical expressions of the errors, the
consequently covering an area of 13.2 km by 19.8 km. The SAR fusion results and the AIS data are considered the measured and
image contains 10 ships as depicted in fig.2. The exponential the reference data, respectively. The average absolute error
distribution is assumed for the sea clutter intensity (the simulated shows a very good result in terms of ship heading with an error
SAR data are not multi-looked here). Its mean value is set equal lower than 2° and ship length (3.27m at average); while worse
to that corresponding single scattering from rough sea within results are obtained in terms of ship width (7.13 m at average)
Geometric Optics (GO) approximation which takes the radar and ship positions (3.9 and 5.9 pixels in x and y direction
parameters (frequency, look angle and spatial resolution), respectively). The average relative error presents a relative error
roughness parameters and the dielectric of the saline water as an of 1.77 % for the ship length and a larger relative error of
input [13]. Then, the ship backscatter is simulated using the 24.18% for the ship width. This can be due to the width
canonical model presented in [13]. The AIS data of the 10 ships extraction technique that should be further investigated. The
is created based on real information provided in [14]. The ship drawback of this method is the necessity to extract all features
positions and headings are generated randomly. The speeds from both SAR and AIS sources; but unfortunately, this is not
instead are the average speed recorded in [14]. always possible (i.e. missing information in some AIS reports
or impossibility to extract the ship sizes from SAR data where
After the pre-processing stage, the detection stage is only one or two pixels are detected). However, using other
performed using CFAR algorithm where the sea-clutter is fusing methods such us the weighted mean or the vote strategy
modelled by exponential distribution. The PFA is set equal to [9] can potentially ameliorate the present results even though
10 . The outcome of the detection stage is shown in Fig. 3. some data are missing from one source or another. In addition,
while Fig. 4. shows the effect on the image after applying the the absence of ground truth has driven the authors to use the
rotation to extract the ship dimensions. The projection of nonnull same AIS data for now to evaluate the error generated for each
pixels on y-axis corresponds to the length of the ship, whilst the feature in the fusion block. This can be avoided if a second
source of AIS data is available. In this case, the additional AIS
x source can be used as reference data.
y
Finally, the overall performance of the proposed architecture
can be improved by reducing the error generated during the
feature extraction stage. For instance, evaluating the azimuth

Fig. 2. Simulated SAR image intensity of the sea clutter in the stripmap Fig. 4. Ship signature after detection with CFAR algorithm and rotation.
mode with 10 ships.
things, it is necessary to evaluate the Doppler shift due to the
TABLE I. THE FUSION RESULTS OF SAR AND AIS DATA FOR 10 radial velocity of the ship in order to improve the current
SHIPS. findings and the technique in general. The authors are now
Position
investigating complex scenarios on both simulated and real data
Heading Length Width sets and comparing different fusion methods.
x y [°] [m] [m]

Ship 1 653 1284.5 178.5 346.5 54.3 REFERENCES


Ship 2 1698.5 1946 338.5 399.85 61.2 [1] Internatinal Marine Organization, “Introduction to IMO,” Internet:
http://www.imo.org/en/About/Pages/Default.aspx, [28 April 2018].
Ship 3 1711 673 57.5 366 69.2 [2] F. T. Ulaby et al., Microwave Radar and Radiometric Remote Sensing,
Ship 4 1313 1652 139 329.5 55.5 Ann Arbor, MI, USA:Univ. Michigan Press, 2014.
[3] Canadian Space Agency, “Maritime Monitoring and Messaging
Ship 5 679.5 18.56 240.5 165.2 29.2 Microsatellite (M3MSat),” Internet: http:// www.asc-
csa.gc.ca/eng/satellites/m3msat/default.asp, [24 April 2018]
Ship 6 655 651 109.5 129.1 28.9
[4] B. Gomez Miguel et al., "PAZ SAR System Validation Plan," EUSAR
Ship 7 3132.5 1070 146.5 194.1 35.9 2014; 10th European Conference on Synthetic Aperture Radar, Berlin,
Germany, 2014, pp. 1-4.
Ship 8 1105 1902 333 145.5 28.5
[5] R. Bird, P. Whittaker, K. Hall, C. Slim and M. Cohen, “NovaSAR: Next
Ship 9 911 656.5 301 183.3 28.8 generation developments,” 2015 IEEE 5th Asia-Pacific Conference on
Synthetic Aperture Radar (APSAR), Singapore, 2015, pp. 94-98.
Ship 10 1851 1452.5 224.5 139.6 29 [6] International Telecommunication Union, “Technical characteristics for an
automatic identification system using time division multiple access in the
Average of the
3.9 5.9 VHF maritime mobile frequency band”, internet: http:// www.
absolute error 1.45 ° 3.27 m 7.13 m
pixels pixels itu.int/dms_pubrec/itu-r/rec/m/R-REC-M.1371-5-201402-I!!PDF-E.pdf,
over 10 ships
[27 October 2017].
Average of the [7] R. Pelich, N. Longépé, G. Mercier, G. Hajduch and R. Garello, "AIS-
relative error Based valuation of Target Detectors and SAR Sensors Characteristics for
- - - 1.77 24.18
in [%] over 10 Maritime Surveillance," in IEEE Journal of Selected Topics in Applied
ships Earth Observations and Remote Sensing, vol. 8, no. 8, pp. 3892-3901,
Aug. 2015.
[8] A. Harati-Mokhtari, A. Wall, P. Brooks, J. Wang “Automatic
displacement due to the radial velocity of the ship due to the identification system (AIS): Data reliability and human error
Doppler effect can be used for a better estimation of the ship implications” Journal of Navigation. vol. 60, no. 3, pp. 373–389, 2007.
position. Preliminary results encourage the authors to test and [9] C. A. Shipp, L. I. Kuncheva, Relationships between combination methods
evaluate the algorithm using real SAR and AIS data. The results and measures of diversity in combining classifiers, Information Fusion,
of real SAR and AIS data acquired respectively from Sentinel-1 vol. 3, no. 2, 2002, pp. 135 – 148.
and ExactEarth provider will be presented at the conference. [10] D. J. Crisp, “The state-of-art in ship detection in synthetic aperture radar
imagery,” Aust. Gov., Dept. Def., Def. Sci. Technol. Org. (DSTO).,
Australia, Tech. Rep. DSTO-RR-0272, 2004.
IV. CONCLUSION [11] S. Tabbone, L. Wendling, J.P. Salmon “A new shape descriptor defined
The paper presents a novel approach to fuse SAR and AIS on the Radon transform” Computer Vision and Image Understanding, vol.
102, no. 1, pp. 42-51, 2006.
data for maritime surveillance. The fusion is performed on the
[12] J. English, R. Hewitt, D. Power and J. Tunaley, "ICE-SAIS — Space-
decision level using the arithmetic mean function where based AIS and SAR for improved Ship and Iceberg Monitoring," 2013
features including ship position, heading, length and width are IEEE Radar Conference (RadarCon13), Ottawa, ON, 2013, pp. 1-6.
extracted from both data sets in order to enhance the ship [13] P. Iervolino and R. Guida, "A Novel Ship Detector Based on the
monitoring operation. The approach has been tested on Generalized-Likelihood Ratio Test for SAR Imagery," in IEEE Journal
simulated data sets and with the assumption that all ships in the of Selected Topics in Applied Earth Observations and Remote Sensing,
SAR image broadcast their AIS data with no temporal gap vol. 10, no. 8, pp. 3616-3630, Aug. 2017.
between them. Preliminary results are very promising for the [14] “Maritime traffic” , Internet: www. marinetraffic.com, [24 April 2018].
fusion of SAR and AIS data for ship monitoring. Among other

You might also like