Validation of Spaceborne and Modelled Surface Soil Moisture Products with Cosmic-Ray Neutron Probes

Abstract

The scale difference between point in situ soil moisture measurements and low resolution satellite products limits the quality of any validation efforts in heterogeneous regions. Cosmic Ray Neutron Probes (CRNP) could be an option to fill the scale gap between both systems, as they provide area-average soil moisture within a 150–250 m radius footprint. In this study, we evaluate differences and similarities between CRNP observations, and surface soil moisture products from the Advanced Microwave Scanning Radiometer 2 (AMSR2), the METOP-A/B Advanced Scatterometer (ASCAT), the Soil Moisture Active and Passive (SMAP), the Soil Moisture and Ocean Salinity (SMOS), as well as simulations from the Global Land Data Assimilation System Version 2 (GLDAS2). Six CRNPs located on five continents have been selected as test sites: the Rur catchment in Germany, the COSMOS sites in Arizona and California (USA), and Kenya, one CosmOz site in New South Wales (Australia), and a site in Karnataka (India). Standard validation scores as well as the Triple Collocation (TC) method identified SMAP to provide a high accuracy soil moisture product with low noise or uncertainties as compared to CRNPs. The potential of CRNPs for satellite soil moisture validation has been proven; however, biomass correction methods should be implemented to improve its application in regions with large vegetation dynamics.

1. Introduction

Soil moisture partitions the incoming rainfall into runoff and infiltration, and controls water and energy fluxes, as well as exchange of trace gases at the Earths’ surface [1,2,3,4]. Accurate information about soil moisture temporal and spatial variation is therefore important for further application [2], e.g., for flood and drought forecasts [5,6], as well as for climate impact studies [7]. Coarse scale, but moderate temporal resolution global surface soil moisture can be obtained by satellite remote sensing, mostly by microwave sensors [8,9]. Currently several satellite missions provide global surface soil moisture products, such as: the Soil Moisture Active Passive (SMAP) [10], the Soil Moisture and Ocean Salinity (SMOS) [11], the METOP-A/B Advanced Scatterometer (ASCAT) [12,13], and the Advanced Microwave Scanning Radiometer 2 (AMSR2) [14]. Before their application to solving various scientific or societal problems, satellite data have to be evaluated, and their validity and accuracy have to be assessed.

Specific airborne validation campaigns have been performed over limited periods of time to fill the scale gap between in situ measurements and coarse satellite observations. For instance, the SMEX02 [15] and SMEX03 [16] campaigns were conducted to validate AMSR-E X- and C-band soil moisture retrievals. The CanEx-SM10 focused on large Canadian agricultural and boreal forest regions to validate SMOS and AMSR-E [17]. Other airborne SMOS validation campaigns were performed in Europe [18,19,20,21], and Australia [22], complementary to long-term comparisons against tower-based radiometry [23]. In the case of SMAP, extensive field experiments were conducted during the Soil Moisture Active Passive Experiments (SMAPEx) in southeastern Australia [24], and the SMAP validation experiment (SMAPVEX) in the USA [25,26]. These airborne or ground-based validation activities have the advantage that, in addition to soil moisture, a validation of brightness temperatures can also be performed to identify potential artefacts in early processing steps [18,27]. However, such aircraft measurements are very costly, and therefore limited in duration.

For the validation of longer periods of time soil moisture sensor networks are typically used, e.g., REMEDHUS [28,29], U.S. watershed networks [30,31,32], U.S. Soil Climate Analysis Network (SCAN)/SNOwpack TELemetry (SNOTEL) network [33], European networks [34,35], AMMA (African Monsoon Multidisciplinary Analysis) [36], or OzNet [37,38]. Most of these network data have been merged in the International Soil Moisture Network (ISMN) [39,40] and further used for validation purposes [41].

According to Crow et al. [42], Gruber et al. [43] and Miralles et al. [44], the validation of satellite based soil moisture products typically faces the problem of scale mismatch, and recommendations for the measurement density requirements for ground-based soil moisture networks are given. As the recommended measurements density cannot always be met due to financial constraints, satellite soil moisture products are also validated against hydrological model simulations [18,45]. Reviews about coarse scale soil moisture remote sensing accuracies can be found in Wagner et al. [13] and Kerr et al. [46].

For the continuous validation of satellite based soil moisture products typically in situ electromagnetic (EM) soil water content sensors are used, e.g., time or frequency domain reflectometry sensors [47,48,49]. However, the application of EM sensors has some limitations. First, EM sensors only probe a very small soil volume, which makes it difficult to relate data from single soil moisture stations to satellite data due to soil heterogeneity. Second, EM sensors cannot be continuously operated in agricultural fields with regular tillage. In these cases, soil moisture stations are typically installed outside the agricultural fields, e.g., neighboring grasslands, and thus the soil moisture measurements are unlikely to represent the main land use observed by the satellite. Thirdly, in most cases EM sensors are horizontally installed at certain depths, e.g., 5 cm, and thus do not represent well the near-surface soil moisture detected by the satellite sensor (the sensitivity of microwave soil moisture sensors decreases sharply with depth [50]).

In recent years, several new non-invasive measurement technologies have emerged in an attempt to address the abovementioned issues of EM sensors [51]. One promising sensor is the cosmic-ray neutron probe (CRNP) [52,53], which measures the intensity of ambient low-energy tertiary neutrons above the land surface. A detailed description of the cosmic-ray neutron sensing technique can be found in Zreda et al. [52]. The footprint of the CRNP typically has a radius between 150 m and 250 m, weakly dependent on soil moisture content [54] and depths between 0.1 m and 0.7 m, strongly dependent on soil moisture content [55]. Given this large footprint area, the CRNP method is better suited for the validation of satellite based soil moisture products [56] because it integrates out small-scale spatial variations in soil moisture such as those in Figure 1 of Zreda et al. [52].

However, the application of the CRNP method is hampered by its sensitivity to additional sources of hydrogen (e.g., above- and below-ground biomass, humidity of the lower atmosphere, lattice water of the soil minerals, organic matter and water in the litter layer, intercepted water in the canopy, and soil organic matter), e.g., Bogena et al. [57], Franz et al. [58], Heidbüchel et al. [59]. In case that these hydrogen sources are temporally stable, their influence can be accounted for by estimating the contribution of each source and subtracting its contribution during the transformation of the neutron counts into volumetric soil moisture contents [52]. Recently, Baatz et al. [60] developed a simple empirical approach to correct for biomass effects, which can, in principle, also be applied to temporally dynamic biomass effects.

Three recent publications have shown first comparisons between CRNP and satellite soil moisture time series. Kędzior and Zawadzki [61] compared SMOS Level 3 data, the Global Land Data Assimilation System (GLDAS), and CRNP data for the year 2013 from the CRNP station Derlo in Poland. They found a correlation between SMOS and CRNP of R = 0.59. Most analyses were performed on normalized soil moisture time series ranging from 0 to 1. Moreover, they employed the triple collocation (TC) method [62], but focusing on the TC error only. Evans et al. [63] introduced the COSMOS-UK network and compared to the Joint UK Land Environment Simulator (JULES) hydrological model. A comparison to ASCAT has been presented to show the applicability of such a network. Correlations between ASCAT and CRNPs at two locations were given with R² = 0.46 and R² = 0.59 with RMSE in units of saturation index (-) of 0.2 and 0.17, respectively. Kim et al. [64] compared different AMSR2 soil moisture retrieval methods with COSMOS data, however, here the focus lied on the evaluation of AMSR2. It was found that AMSR2 shows rapid decreases in correlation coefficients during wet conditions, in cold regions (mean annual temperature <290 K), as well as under dense vegetation cover.

The importance of footprint-scale soil moisture reference data will increase in near future due to downscaled global soil moisture products such as those by Merlin et al. [65], Das et al. [66], Piles et al. [67], Rüdiger et al. [68], high resolution radar soil moisture retrievals [69,70,71], or future satellite missions such as Tandem-L [72,73]. Therefore, in this study a more in-depth analysis and comparison of different currently available soil moisture products from SMOS, SMAP, AMSR-2, ASCAT and the Global Land Data Assimilation System Version 2 (GLDAS2) simulation with CRNP time series is performed to find out how useful cosmic-ray soil moisture data can be. In addition to standard validation scores the Triple Collocation method is performed to give more in-depths views on retrieval accuracies of the different sensors and their causes.

2. Test Regions

For the validation of coarse scale satellite soil moisture products, examples from six CRNP sites from four networks in different regions of the world were selected, see Figure 1 and

Table 1. Coordinates and land cover information for the CRNP sites within the six focus regions.

Region	Station	Lat	Lon	Land Cover
Rur catchment	Merzenhausen	50.930	6.297	crops
	Rollesbroich1	50.622	6.304	grassland
	Rollesbroich2	50.624	6.305	grassland
	Gevenich	50.989	6.324	crops
	Ruraue	50.862	6.427	grassland
	Wildenrath	51.133	6.169	clearing
	Wuestebach	50.505	6.331	spruce
	Aachen	50.799	6.025	crops
	Heinsberg-Schleiden	51.041	6.104	grassland
	Kall-Sistig	50.501	6.526	grassland
Arizona	Lucky Hills	31.742	−110.052	shrubland
	Kendall	31.737	−109.942	grassland
California	Tonzi Ranch	38.432	−120.966	oak savanna, grassland
NSW	Yanco	−35.005	146.299	grassland
Kenya	Mpala North	0.486	36.870	shrubland
	Klee	0.283	36.867	savanna
Karnataka	Singanallur	12.142	77.229	crops

.

2.1. TERENO Site Rur Catchment

The catchment of the Rur River is located in the Belgian–Dutch–German border region near the city of Aachen and can be divided in two main regions: The southern part covers the bedrock of the Eifel Mountains, whereas the northern part is characterized by soils evolved from loess [74,75]. In accordance with these local hydrogeological and climatic differences, land use and vegetation is clearly distinguishable: Forest and grassland characterize the south, whereas in the north fertile agricultural land predominates [76,77,78]. Soils can be locally very heterogeneous due to their origin in floodplain deposits and loess [79]. The Rur catchment is part of the Terrestrial Environmental Observatories (TERENO) network [80,81]. Several point-scale soil moisture sensors have been already used to evaluate airborne active and passive L-band retrievals [18,78,82] and to validate SMOS [18,83], ASCAT [83], and AMSR2 products [14]. Data are available from the International Soil Moisture Network (ISMN) [39] and the TEODOOR data base [84], and is directly delivered to NASA for SMAP validation [85]. The Rur catchment CRNP network comprises ten probes (CRS1000, HydroInnova LLC, 2009) which were installed in 2011 and 2012 [86].

2.2. Walnut Gulch, Arizona, USA

Two sites within the Walnut Gulch Experimental Watershed, AZ, USA, are used in this study. The Lucky Hills is a shrubland site located east of the town of Tombstone. The vegetation is mainly Chihuahuan desert shrub species [87], and the soils are sandy loams, with some caliche, 3.6 wt % (weight percent) organic carbon and 1.5 wt % lattice water [52]. The Kendall site is a semi-desert grassland site located to the east of the Lucky Hills. The vegetation is mainly C4 grasses with sparse shrubs, and the soils are mostly gravely loams, with 0.8% organic carbon and 2.4% lattice water [52]. Average annual rainfall within the watershed is 350 mm [88]. About half of it falls in the summer time (July–September) from convective storms associated with the North American Monsoon. The Walnut Gulch watershed is heavily instrumented for hydrological and atmospheric research. In addition to two CRNPs, it has two flux towers, a network of 88 rain gauges [88] and a network of TDR probes for soil moisture determination, as well as flumes and weirs for surface runoff assessment [87]. Because of low vegetation mass throughout the year, this site can be seen as a benchmark for the accuracy to be expected after vegetation correction for CRNPs.

2.3. Tonzi Ranch, California, USA

The Californian site Tonzi Ranch is located in the lower foothills of the Sierra Nevada. This site is an oak-grass savanna consisting of scattered blue oak trees with occasional gray pine trees, and grazed grassland. Growing season of the understory is confined to the wet season only, typically from October to early May, after which grass senesces rapidly. In contrast, the deciduous blue oak trees are dormant during the rainy winter months and reach maximum LAI in April. The Mediterranean climate comes with a mean annual temperature of 16.3 °C and total annual precipitation of 560 mm [89]. Soils are composed of red sandy clay loam with a clay hardpan located around 30–40 cm depth. The soil moisture observation network design comprises multiple sites, each with a 10–30 node cluster taking measurements from up to 4 sensors installed at different depths. Tonzi Ranch is FLUXNET/AmeriFlux, SoilSCAPE site and SMAP validation site as well [85].

2.4. CosmOz Site New South Wales, Australia

The CRNP in New South Wales (NSW), Australia, is part of the CosmOz network, which consists of 15 probes distributed across the country. Hawdon et al. [90] provides a comprehensive summary of the correction and calibration procedures applied to all cosmic-ray probes in CosmOz (cosmoz.csiro.au). The CRNP chosen for this study was installed in 2011 in an open, semi-arid grassland field in the central area of the Murrumbidgee river catchment, close to the town of Yanco. The predominant soil type in the area is silty loam, with 0.284% organic carbon, and 4.4% lattice water. The average annual rainfall in the region is 400 mm. This CRNP is within the Yanco validation site, which has been extensively used for the validation of satellite missions in the past, such as SMOS, SMAP, and AMSR-2 [22,24]. Other instruments in the area include soil moisture monitoring stations from the OzNet network [38], a weather station, and a flux tower from the OzFlux network.

2.5. COSMOS Sites Kenya

Two sites within the Mpala Research Centre and Ranch, approximately 250 km north of Nairobi, the capital city of Kenya, are used here. Two COSMOS probes were installed in 2011 at two sites: KLEE and Mpala North. The KLEE site (KLEE standing for Kenya Long-term Exclusion Experiment) is a semi-arid savanna composed of mainly acacia trees and grass understory [91,92]. The soils are black clay vertisols, known locally as “black cotton”, which have low hydraulic conductivity (poor drainage) and a tendency to swell on wetting and shrink and crack on drying. They contain 1.11 wt % of organic carbon and 6.00 wt % of lattice water [52]. The average precipitation rate is 650 mm/y [91]. The Mpala North site, located approximately 20 km north of KLEE, is a semi-arid shrubland composed of mixed acacia savanna with understory of grasses and succulents [93]. The soils are sandy loams overlying bedrock at a depth of approximately 2 m. The top 30 cm of soil contain 0.81 wt % of organic carbon and 2.88 wt % of lattice water [52]. The area receives an average of 500 mm/y of rain [91,93] The average biomass is estimated at approximately 0.5 kg/m² [93]. The cosmic-ray probe is collocated with a 20 m high eddy covariance tower installed by Princeton University in 2009.

2.6. Karnataka, India

The Singanallur site in Peninsular India is located on the southern part of Deccan Plateau in the south of Karnataka State, approximately 100 km southwest of Bangalore. The site is part of the new COSMOS-India network that will consist of seven CRNP sites distributed across India by February 2017. The climate of the region is semi-arid and can be classified under the tropical Indian summer monsoon. Mean annual rainfall is 700 mm, with the majority of rainfall falling during the southwest monsoon between June and November. The Singanallur CRNP was installed at an open area of agricultural land in the catchment of the Cauvery river basin during September 2015. The main soil type is predominantly sandy loam. Agricultural management in this region consists of small scale farming practices, with two main crop production periods during the monsoon (June to November) and winter (December to February) seasons. Diverse varieties of crops (Maize, Sorghum, vegetables etc.) are grown in the region. Crops are mostly rain fed during the monsoon season and about 15%–20% are irrigated using groundwater, especially during winter. The CNRP data handling procedures for COSMOS-India are based on Evans et al. [63]. Other hydrological monitoring in the area includes soil moisture monitoring (Stevens HydraProbes), weather variables (OTT climate station) and groundwater levels in bore wells.

3. Soil Moisture Data Sets

Satellite surface soil moisture products from AMSR2, ASCAT, SMAP, and SMOS, as well as the land data assimilation product GLDAS2 are analyzed in comparison to CRNPs over the six test regions. In order to better inter-compare the different data sets, we selected the period from 1 April 2015 to 31 March 2016 for all sensors, except the Karnataka region in India which has been evaluated for the period 7 September 2015 to 6 September 2016 to include the most recent full year CRNP records. Even though the different sensors measure different physical quantities, such as brightness temperature, backscatter or cosmic-ray neutron intensity, the final computed variable is soil moisture. Please note that in the following descriptive sections the orbit direction—ascending or descending—varies between satellite missions and therefore also the overpass time, i.e., the morning overpass used here can be from descending or ascending node product. Using the early morning overpass for passive sensors is important, because they minimize the difference between canopy and soil temperatures and thermal differences between land cover types within a pixel helping to minimize soil moisture retrieval errors originating from the use of a single effective temperature to represent the surface soil and canopy temperatures [94,95].

3.1. AMSR-2

The Japan Aerospace Exploration Agency (JAXA) Global Change Observation Mission-1st Water (GCOM-W1) satellite hosts the AMSR2 sensor [14]. Together with its precursor AMSR-E, the time series of available data expands from mid-2002 until present, with a few months gap in between. Having a C- and an X-band radiometer on board, soil moisture in this study is retrieved from the C-band sensor by the Land Parameter Retrieval Model (LPRM) [96], where the land surface temperature derived separately from AMSR2’s Ka-band (36.5 GHz) is one input. The data are delivered at a spatial resolution of 0.25°, resampled from the nominal 46 km. In this study, we used the descending orbit data recorded at 1:30 am local time to be closer to the other sensors morning overpass times. Data have been filtered by the soil moisture retrieval uncertainty parameter <0.2.

3.2. ASCAT

ASCAT comprises two active C-band systems onboard the METOP-A/B platforms [13], with a third system planned for launch in 2018. Data are acquired on descending node at 9:30 am local time morning. As the only active system in this study, the slightly later overpass time does not imply impacts due to vertical temperature gradients. The Vienna University of Technology soil moisture change detection algorithm Water Retrieval Package (WARP) after Naeimi et al. [97] utilizes backscatter measurements at six different azimuth angles to calculate soil moisture. The angular information is used to characterize the vegetation contribution and their temporal variability to be eliminated before soil moisture inversion. A basic assumption is that at the spatial scale of the scatterometer measurements roughness and land cover are stable in time [13]. Several products are derived from ASCAT, and distributed by the EUMETSAT Satellite Application Facility on Support to Operational Hydrology and Water Management (H-SAF). In this study, the EUMETSAT H109 has been used for 2015, and the EUMETSAT H110 for 2016, both with a 12.5 km spatial sampling. The soil moisture is given as a saturation index, which is a relative measure of surface soil moisture ranging between wilting point and saturation. The soil moisture index was converted to soil moisture in cm³·cm⁻³ by multiplication with the porosity provided by the Harmonized World Soil Database (HWSD) [98]. All data with a soil moisture noise designation larger than 20% have been excluded from analysis.

3.3. SMAP

The National Aeronautics and Space Administration (NASA) mission SMAP [10] was designed to record co-located L-band active radar and passive radiometer measurements at a constant incidence angle of 40°. Because the radar system stopped function after few weeks of operation, here only the radiometer product is validated. The SMAP radiometer yields ~40 km resolution (−3 dB footprint) measurements at low-Earth orbit, so that brightness temperatures and derived products are posted on a 36 km fixed Earth grid. This employs the Equal-Area Scalable Earth–2 (EASE2) grid [99] for perfect nesting of the different spatial resolutions. For Level 3 data used in this study (Version 3, validated release), individual half-orbit granules from Level 2 acquired over 24 h are composed to produce daily multi-orbit global maps. The data point observed closest to 6 am local time (descending orbit) is selected for the final product. The current baseline Single Channel Algorithm after Jackson [100] and a dielectric mixing model converts horizontally polarized brightness temperatures to soil moisture. The vegetation opacity parameterization is derived from Moderate Resolution Imaging Spectroradiometer (MODIS) Normalised difference Vegetation Index (NDVI) data.

3.4. SMOS

The European Space Agency (ESA) Earth Explorer mission SMOS [11] operates at the L-band frequency and overpasses the area under investigation in the morning at 6 am local time from ascending orbit. The advantage of SMOS is the multiple incidence angle measurements for combined vegetation opacity and soil moisture retrieval by the SMOS processor. It makes use of the L-Band Microwave Emission of the Biosphere (L-MEB) formulation [101] and the dielectric mixing model by Mironov et al. [102] in order to retrieve surface soil moisture. In this study, we used the daily Level 3 product (Version 3.0), i.e., the Level 2 Soil Moisture Processor equivalent of Version 6.20 [103]. It optimizes the soil moisture inversion over the dominant land cover fraction, whereas all other fraction contributions are calculated using fixed parameters. Therefore, the retrieved soil moisture represents only the surface fraction on which inversion has been attempted, i.e., the dominant only. As the ellipsoidal footprint size varies with the incidence angle, the spatial resolution varies. As a rule of thumb, 90% of the signal originates from a circular area with a radius of 25 km [33]. All data with a soil moisture data quality index larger than 0.1 and a radio frequency interference (RFI) probability larger than 0.2 [104] have been excluded.

3.5. GLDAS2-NOAH

The Global Land Data Assimilation System Version 2 (GLDAS2) [105] provided by a joint effort of NASA’s Goddard Space Flight Center and NOAA’s National Centers for Environmental Prediction (NCEP) is driven by the long-term high resolution meteorological forcing data set from Sheffield et al. [106]. From the available four land surface models, we selected the NOAH model [107] for use in this study to calculate water and energy fluxes at the soil-vegetation-atmosphere continuum. It is worth noting that NOAH is not coupled to the atmosphere, and it is a gridded 1D column model with vertical fluxes only. GLDAS2 integrates MODIS snow cover and leaf area index by data assimilation techniques. The output soil moisture is given for four soil layers. Here, we selected the top level of 0–10 cm depth for comparison with the other data sets, and the 1° spatial resolution GLDAS2 data. The temporal resolution is 3 h, but the daily mean values are used in this study.

3.6. Cosmic-Ray Neutron Probes

A detailed description of the cosmic-ray neutron sensing technique can be found in Zreda et al. [52]. Here, only the basics of the method are presented. High-energy, secondary cosmic-ray neutrons are created in the Earth’s atmosphere when incoming cosmic-ray particles interact with atmospheric nuclei [52,54]. The secondary neutrons form a cascade that propagates through the atmosphere and eventually reaches the land surface. Some interactions of high-energy neutrons with nuclei in the atmosphere and the solid Earth form fast neutrons. Those neutrons are particularly sensitive to collisions with hydrogen nuclei and therefore they can be used to measure the concentration of hydrogen in materials at the land surface. Since soil moisture is one of the largest sources of hydrogen present in terrestrial systems, it largely controls the intensity of fast neutrons in the lower atmosphere [52]. Thus, changes in the intensity of fast neutrons are strongly and inversely correlated with soil moisture changes.

The CRNPs were calibrated using the simple calibration function proposed by Desilets et al. [108] and Bogena et al. [57] using gravimetric soil samples:

$${\theta }_v={\rho }_{bd}\left(a_0{\left(\frac{N_{corr}}{N_0}-a_1\right)}^{-1}-a_2\right).$$

(1)

where ${\theta }_v$ is the volumetric soil water content (cm³·cm⁻³), ${\mathrm{N}}_0$ is the count rate over dry soil under the same reference conditions and $a_i$ are fitting parameters and ${\rho }_{bd}$ refers to the dry soil bulk density (g·cm⁻³). $N_0$ refers to intensity over dry soil and is used as a calibration parameter determined by independent methods using measured neutron intensity with simultaneously within the CRNP footprint in situ measured soil moisture.

At each site in the Rur catchment, 18 soil samples were taken along three circles with distances of 25, 75 and 175 m from each CRNP, six samples evenly distributed along each circle. Each sample was extracted with a 50.8 × 300 mm round HUMAX soil corer (Martin Burch AG, Switzerland). The samples were split into 6 sub-samples with 5 cm length each and oven dried at 105 °C for 48 h to measure dry soil bulk density and soil moisture. Lattice water was determined for each site using a heat conductivity detector. The fast neutron counts were corrected for influences of atmospheric pressure, incoming neutrons and atmospheric humidity, see Baatz et al. [86] for more details. A vegetation correction such as that of Baatz et al. [60] is not considered so far because of the limited availability of biomass dynamics data at the respective CRNP stations.

Being a relatively new soil moisture monitoring method, the accuracy of CRNP soil moisture data is still a topic of discussions [51,59,109]. As mentioned earlier, CRNP observes a footprint of several hundreds of metres; therefore a direct validation using data from single point in situ sensors (e.g., TDR) is unfeasible given the difference in spatial scale. For this reason, data from soil moisture networks covering the CRNP footprint with many in situ sensors have been used in the literature for the validation of CRNP. For instance, a validation study in the Rollesbroich site (Germany) demonstrated that CRNP measurements compare very well to a TDR soil moisture sensor network with a RMSD of 0.032 cm³·cm⁻³ [86]. Even under very humid conditions in a forest environment with high amount of additional hydrogen pools (e.g., biomass, organic matter, litter layer, etc.) reducing the CRNP sensitivity to soil moisture, the CRNP accuracy as compared to a soil moisture sensor network ranges in RMSD between 0.0274 and 0.0325 cm³·cm⁻³ depending on the calibration method [57]. Other CRNP validation studies in different climatic, pedological, and land use settings have demonstrated that CRNP provide reliable measurements of soil moisture when calibrated appropriately, and thus can serve as reference for larger scale soil moisture products. Nevertheless, in future the effects of time-variable crop biomass on CRNP measurements should be accounted for using appropriate correction methods, e.g., Baatz et al. [60].

COSMOS data processing is done online following the procedures described in Zreda et al. [52]. The calculated soil moisture values, as well as all ancillary and intermediate data, are disseminated hourly at the COSMOS server (cosmos.hwr.arizona.edu); the same data, averaged over 12 h, are available from the International Soil Moisture Network ([39]; https://ismn.geo.tuwien.ac.at). We note here that for many sites displayed on the COSMOS server the correction for atmospheric humidity has to be made in post processing using relative humidity and temperature data from any collocated instruments (such as weather stations or eddy covariance towers).

The cosmic-ray neutron measurement reflects soil moisture in the top few decimeters of soil [110], whereas L-band microwave satellite sensors provide a value for the top 0–5 cm soil layer. According to Zreda et al. [53] the penetration depth of the sensor ranges from 12 to 76 cm depending on soil water content, which was also confirmed by Köhli et al. [54]. They found that an exponential decrease with depth is a good estimate of the sensor’s vertical sensitivity. In microwave remote sensing studies, the soil moisture content is typically assumed to be uniform to the penetration depth of the sensor and the effects of non-uniformity of moisture with depth are not considered. It has been demonstrated that EM waves also show a strong non-linear propagation characteristic depending on soil water content [111]. This means that microwave soil moisture sensors as well as CRNPs are most sensitive to the first centimeters of the soil. Therefore, CRNP are well suited for the validation of satellite soil moisture products. An indirect estimation of soil moisture profile information from CRNPs is not done in this study, although possible [112,113,114], nevertheless, the higher penetration depth needs to be taken into consideration [115]. The available high temporal resolution Level 3 online CRNP data (1 hourly) has been converted to daily mean (0 h–24 h) values.

4. Description of the Metrics Used for Validation

As a first approach, three standard validation metrics on direct comparison and absolute values given in Entekhabi et al. [116] have been used. A brief summary of these metrics is presented hereafter.

The Root-Mean-Square Deviation (RMSD) is given by:

$$RMSD=\sqrt{E[{\left({\theta }_{Product}-{\theta }_{CRNP}\right)}^2]},$$

(2)

where $E[.]$ is the expectation operator, ${\theta }_{Product}$ is the soil moisture of the product to be evaluated (spaceborne or modelled), and ${\theta }_{CRNP}$ is the soil moisture of the CRNP measurements, with all soil moisture values given in units of volumetric soil moisture. Please note that, when several CRNP fall within a grid pixel, the value of ${\theta }_{CRNP}$ in this equation is the linear average of all those CRNP data. As this metric can be compromised by bias in the time series, the unbiased RMSD (ubRMSD) is calculated by removing the bias from each data sets:

$$ubRMSD=\sqrt{E\left[{\left(\left({\theta }_{Product}-E\left[{\theta }_{Product}\right]\right)-\left({\theta }_{CRNP}-E\left[{\theta }_{CRNP}\right]\right)\right)}^2\right]}.$$

(3)

Both metrics are related through:

$$RMS{D}^2=ubRMS{D}^2+bia{s}^2,$$

(4)

with

$$bias=E\left[{\theta }_{Product}\right]-E\left[{\theta }_{CRNP}\right].$$

(5)

On the other hand, the Pearson’s pairwise correlation coefficient (R) is given by:

$$R=\frac{cov\left({\theta }_{Product},{\theta }_{CRNP}\right)}{{\sigma }_{{\theta }_{Product}}{\sigma }_{{\theta }_{CRNP}}}.$$

(6)

where the numerator is the covariance of both soil moisture time series, and the denominator is the product of the standard deviations of both time series.

The study was then extended to consider the Triple Collocation (TC) method [62], which helps understanding the random error variances of the remote sensing products. These can be used as additional quality scores of soil moisture time series. Several applications of TC to soil moisture time series have been published [43,117,118,119,120,121,122,123,124,125,126,127]. Gruber et al. [128] list the assumptions made to perform the TC analysis: (i) Linearity between the true soil moisture signal and the observations; (ii) signal and error stationarity; (iii) independency between the errors and the soil moisture signal (error orthogonality); and (iv) independency between the errors of three soil moisture time series (zero error cross-correlation). These assumptions lead to the error model $\mathit{i}={\alpha }_i+{\beta }_i\theta +{\epsilon }_i$ with $i\in \left[X,Y,Z\right]$. $X$, $Y$, and $Z$ denote the time series to be compared in TC. By this linear scaling, the penetration depth difference can partly be removed [125]. In this study, we used $X$ as the cosmic-ray neutron probe reference, $Y$ as the satellite remote sensing product from AMSR2, ASCAT, SMOS or SMAP, and $Z$ as the GLDAS2 model simulation. Here, we assume that these are spatially and temporally collocated datasets. The temporal accordance is given, but the spatial extent differs between products. Moreover, the CRNP reference value is calculated by averaging the station data located within each product pixel. This can be seen in Figure 1. In the following we use the covariance notation to explain the TC scores, i.e., the scores are combinations of the covariances ($cov(.)$) of the three individual data sets. The scaling factor $\beta$ for each time series is given by:

$$\begin{gathered} {\beta }_X=1 \\ {\beta }_Y=\frac{cov\left(X,Z\right)}{cov\left(Y,Z\right)} \\ {\beta }_Z=\frac{cov\left(X,Y\right)}{cov\left(Z,Y\right)} \end{gathered}$$

(7)

with the fact that ${\beta }_X=1$ indicates that $X$ serves as the reference value. Here, we present the absolute error standard deviation ${\sigma }_{{\epsilon }_i}=\sqrt{{\sigma }^2{}_{{\epsilon }_i}}$ by:

$$\begin{gathered} {\sigma }_{{\epsilon }_X}=\sqrt{\left|var\left(X\right)-\frac{cov\left(X,Y\right)cov\left(X,Z\right)}{cov\left(Y,Z\right)}\right|} \\ {\sigma }_{{\epsilon }_Y}=\sqrt{\left|var\left(Y\right)-\frac{cov\left(Y,X\right)cov\left(Y,Z\right)}{cov\left(X,Z\right)}\right|} \\ {\sigma }_{{\epsilon }_Z}=\sqrt{\left|var\left(Z\right)-\frac{cov\left(Z,Y\right)cov\left(Z,X\right)}{cov\left(Y,X\right)}\right|}. \end{gathered}$$

(8)

Another measure is the signal-to-noise ratio (SNR), given by:

$$\begin{gathered} SN{R}_X=-10log\left(\frac{var\left(X\right)cov\left(Y,Z\right)}{cov\left(X,Y\right)cov\left(X,Z\right)}-1\right) \\ SN{R}_Y=-10log\left(\frac{var\left(Y\right)cov\left(X,Z\right)}{cov\left(Y,X\right)cov\left(Y,Z\right)}-1\right) \\ SN{R}_Z=-10log\left(\frac{var\left(Z\right)cov\left(X,Y\right)}{cov\left(Z,X\right)cov\left(Z,Y\right)}-1\right). \end{gathered}$$

(9)

The SNR is a logarithmic measure and typically given in decibel units (dB) to make it symmetric around zero. A value of zero identifies a signal variance that is equal to the noise variance, and every ±3 dB corresponds to an additional doubling/halving of the ratio between them. The SNR in dB is not affected by previous scaling.

5. Results and Discussion

The time series of ASCAT, AMSR2, SMAP, SMOS and GLDAS2 were compared to the CRNP references in Figure 2 for the Rur catchment, in Figure 3 for Arizona, in Figure 4 for California, in Figure 5 for New South Wales, in Figure 6 for Kenya, and in Figure 7 for Karnataka. These figures identify already advantages and drawbacks of the different soil moisture observation strategies by focusing on specific microwave frequencies, or active and passive technologies. In the following, the results are discussed in detail.

5.1. Standard Metrics

The standard statistics scores for each satellite and modelled soil moisture data set as compared to CRNP at each site are listed in

Table 2. Standard validation metrics for the satellite soil moisture products from AMSR2, ASCAT, SMAP and SMOS as well as from the GLDAS2 simulation compared to the CRNP reference. n is the number of valid data points to calculate the metrics.

Product	Region	n	R	RMSD	Bias	ubRMSD
AMSR2	Rur	356	0.0984	0.1154	0.0388	0.1087
	Arizona	259	0.5049	0.0737	−0.0568	0.0469
	California	278	0.6061	0.0871	−0.0074	0.0868
	NSW	278	0.5454	0.1115	0.0104	0.1111
	Karnataka	212	0.4069	0.3345	0.2732	0.1931
	Kenya	221	0.7541	0.1556	0.1464	0.0529
ASCAT	Rur	366	0.7882	0.0733	−0.0412	0.0606
	Arizona	361	0.1688	0.0504	−0.0356	0.0356
	California	328	0.8383	0.0830	−0.0579	0.0595
	NSW	277	0.7876	0.0802	−0.0314	0.0974
	Karnataka	214	0.8586	0.0522	0.0019	0.0521
	Kenya	221	0.7720	0.0969	−0.0531	0.0810
SMAP	Rur	206	0.8536	0.0682	−0.0457	0.0505
	Arizona	133	0.7138	0.0305	−0.0133	0.0273
	California	177	0.9146	0.0375	−0.0026	0.0373
	NSW	171	0.8818	0.0745	−0.0439	0.0599
	Karnataka	141	0.7605	0.0529	−0.0271	0.0452
	Kenya	129	0.8136	0.0464	−0.0103	0.0453
SMOS	Rur	226	0.6927	0.1096	−0.0813	0.0732
	Arizona	184	0.6544	0.0452	0.0048	0.0448
	California	193	0.8435	0.0689	−0.0213	0.0653
	NSW	188	0.8674	0.0862	−0.0578	0.0637
	Karnataka	108	0.4769	0.0819	0.0158	0.0800
	Kenya	146	0.5896	0.1419	−0.1226	0.0706
GLDAS2	Rur	366	0.6837	0.1145	−0.1052	0.0453
	Arizona	366	0.7717	0.0947	0.0899	0.0298
	California	366	0.8977	0.0489	0.0214	0.0440
	NSW	366	0.7514	0.1042	−0.0390	0.0966
	Karnataka	348	0.8753	0.0740	0.0644	0.0365
	Kenya	366	0.5672	0.0546	0.0111	0.0535

. SMAP provides the highest correlation R > 0.71 for all regions. On the other hand, the goodness of fit for AMSR2 in the Rur catchment and for ASCAT in Arizona with CRNP measurements is too low and therefore inadequate for further utilization. AMSR2 responds to precipitation events quite well, but it is not able to cover the drying phases and the moisture dynamics. This could be an indication of the inability of the C-band system to penetrate the dense vegetation in the West of Germany [129] or problems with the LPRM parameterization. The accuracy of ASCAT in Arizona will be discussed in Section 5.2 in more detail. SMOS observations over India are heavily affected by RFI, since the filter considering the RFI probability reduced significantly the number of observations. The remainder data still show RFI impact (Figure 7). Although also at L-band, SMAP Level 3 data are not affected by RFI. This is because SMOS measures brightness temperatures over a single 24 MHz passband centred at 1.413 GHz [130], whereas SMAP measures in the same band, but separated into 16 sub-bands with 1.5 MHz widths. This provides time and frequency diversity and enables detection and mitigation of RFI.

The bias of ASCAT, SMAP and also SMOS shows a trend to underestimate CRNP records at most sites. For ASCAT this might be related to uncertainties in the soil porosity information used for scaling the soil moisture time series. For the other sensors, the problem is twofold: When attributing the bias to the spaceborne systems, the dielectric mixing model, the soil texture and further auxiliary data used to retrieve soil moisture could contribute to the systematic difference. Otherwise, when attributing the bias to the CRNP reference, the horizontal and vertical scale mismatch may introduce systematic deviations. e.g., for Kenya in

Figure 6 it is remarkable that the two CRNP stations observe very similar soil moisture dynamics, but with large absolute difference indicating significant differences in soil properties, e.g., soil porosity, between both sites. Local rainfall events and disparity in average annual rainfall increase the difference. This indicates that this region in Kenya is very heterogeneous with large absolute soil moisture variability, which might not be well captured by only two CRNP. Thus, more CRNP sites would be needed for a reliable validation of satellite soil moisture products in the area.

Soil moisture products of SMAP, SMOS and ASCAT show in the Rur catchment (Figure 2) a strong underestimation of soil moisture during the period from April to end of July 2015. During this period, the vegetation in central Europe typically shows the highest biomass growth rates. This change in vegetation status is accounted for in the satellite retrieval models, but up to now is not included in the standard processing of CRNP data. Thus, the discrepancies between satellite and CRNP data products seem to be grounded in the different data processing procedures. To increase the reliability of CRNP soil moisture products, a biomass correction of CRNP data would be required in regions with temporally varying vegetation activity.

A similar trend is shown in NSW, where at the end of the wet season of the Australian winter higher CRNP soil moisture was observed than any remote measurement. However, given that the winter biomass in Yanco is very low, the reason for these discrepancies is believed to be due to standing water. This CRNP is located in an area prone to ponding, and the frequent rain registered in the period from June to August 2015 caused ponding within the CRNP footprint for weeks. The CRNP clearly captures this situation, but it does only partly emerge at the scale of the satellite products as it was not that areally extensive. SMAP, SMOS and partly ASCAT observe these very wet conditions, but AMSR2 and the GLDAS2 simulations do not react commensurably.

The RMSD in Table 2 is given for completeness; we are now focusing only on the ubRMSD. The target accuracy of SMOS and SMAP of 0.04 cm³·cm⁻³ is not fulfilled by any product for all sites. For NSW drier conditions were retrieved well, but wetter conditions were generally underestimated. In the Rur catchment SMAP, SMOS and ASCAT are better able to identify the right soil moisture magnitude during wet conditions, but dry phases were underestimated. Thus, removing the bias for these data sets does not significantly improve the accuracy. In India, AMSR2 shows adequate record of soil moisture variation between the two monsoon seasons and the dry phase in between. However, soil moisture conditions during the winter monsoon starting October 2015 have been heavily overestimated and lasted too long according to AMSR2 records. Here interception as well as vegetation effects may still play a role in soil moisture time series. Similarly, the summer monsoon starting in June 2016 was moderately overestimated by AMSR2. In Arizona, the smallest soil moisture dynamic range has been recorded, which is the reason for the lowest ubRMSD for each data set. A comparison to other sites is not directly possible. These findings indicate that the use of standard validation scores such as R, RMSD and ubRMSD is not always suitable for in-depths estimation of soil moisture product accuracies. The TC method is able to overcome these limitations.

It was already shown that the simulated GLDAS2 data shows lower dynamic variability than measured soil moisture data products [131]. This indicated that the model driving forces as well as auxiliary data such as soil texture or land cover information are not able to represent the full heterogeneity of Earth’s characteristics. However, GLDAS2 time series are able to cover well the relative dynamics for the analyzed sites. This should be kept in mind when interpreting the TC results in the following section.

5.2. Triple Collocation Results

In

Table 3. Triple collocation validation scores for the remotely-sensed soil moisture products from AMSR2, ASCAT, SMAP and SMOS compared to the cosmic ray reference ($X$) and the GLDAS2 time series as third ($Z$) data set.

Product	Region	${\mathit{\alpha }}_{\mathit{Y}}$	${\mathit{\beta }}_{\mathit{Y}}$	${\mathit{\sigma }}_{{\mathit{\epsilon }}_{\mathit{Y}}}$ (${\mathit{\sigma }}_{{\mathit{\epsilon }}_{\mathit{X}}}$,${\mathit{\sigma }}_{{\mathit{\epsilon }}_{\mathit{Z}}}$)	$\mathit{S}\mathit{N}{\mathit{R}}_{\mathit{Y}}$ (dB)
AMSR2	Rur	0.0388	5.5595	0.0894 (0.0387, 0.0171)	−18.5235
	Arizona	−0.0568	0.7281	0.0442 (0.0136, 0.0199)	−2.8311
	California	−0.0074	1.2337	0.0790 (0.0326, 0.0251)	−1.3439
	NSW	0.0104	1.9909	0.0766 (0.0562, 0.0279)	−2.3658
	Karnataka	0.2732	0.5911	0.1875 (0.0278, 0.0191)	−5.8664
	Kenya	0.1464	0.7246	0.0472 (0.0167, 0.0502)	2.5727
ASCAT	Rur	−0.0412	0.7165	0.0546 (0.0149, 0.0197)	3.0057
	Arizona	−0.0356	4.8117	0.0286 (0.0081, 0.0263)	−14.8701
	California	−0.0579	0.9389	0.0577 (0.0135, 0.0419)	4.0987
	NSW	−0.0314	1.5420	0.0438 (0.0534, 0.0304)	4.7676
	Karnataka	0.0019	0.7176	0.0406 (0.0216, 0.0280)	6.7020
	Kenya	−0.0531	0.3511	0.0460 (0.0432, 0.0504)	8.8528
SMAP	Rur	−0.0457	0.5393	0.0034 (0.0293, 0.0214)	28.3029
	Arizona	−0.0133	0.6073	0.0190 (0.0146, 0.0216)	5.0957
	California	−0.0026	1.0080	0.0276 (0.0251, 0.0358)	9.8749
	NSW	−0.0439	0.9796	0.0036 (0.0600, 0.0295)	30.0646
	Karnataka	−0.0271	1.2505	0.0424 (0.0098, 0.0338)	1.6155
	Kenya	−0.0103	0.7023	0.0401 (0.0112, 0.0495)	3.7934
SMOS	Rur	−0.0813	0.7563	0.0632 (0.0317, 0.0182)	1.9996
	Arizona	0.0048	0.5054	0.0371 (0.0133, 0.0212)	1.4378
	California	−0.0213	0.7881	0.0565 (0.0245, 0.0374)	5.3544
	NSW	−0.0578	0.9994	0.0265 (0.0581, 0.0272)	12.6628
	Karnataka	0.0158	1.2542	0.0765 (0.0218, 0.0257)	−4.5250
	Kenya	−0.1226	0.5929	0.0467 (0.0443, 0.0500)	3.9003

the Triple Collocation results are listed. As ${\alpha }_Y$ is the same as the bias in the section before, it has already been discussed. A ${\beta }_Y$ of 1 indicates that no scaling is needed, ${\beta }_Y$ smaller (higher) than 1 indicate that the dynamic range of the satellite product is higher (lower) than that of the CRNP and GLDAS2 time series. For the L-band systems SMAP and SMOS ${\beta }_Y$ is consistently lower than 1 except for Karnataka, indicating that in those sites the dynamic scaling needs to reduce the dynamic range to match the range of the CRNP observations. In the SMOS and SMAP over Karnataka, the summer monsoon season was slightly underestimated. Here it is obvious that the penetration depth difference plays a role between high soil moisture dynamics at shallow depths (roughly 0–5 cm [11]) of SMOS and SMAP and damped temporal variability at larger soil depths for CRNPs. For the C-band systems ASCAT and AMSR2 the picture is indifferent. A ${\beta }_Y$ larger and also smaller than 1 have been calculated for the specific regions and sensors. Though ASCAT fits the magnitude of soil moisture temporal variability in the Rur catchment, it overestimates it as compared against the Kenya sensors and underestimates it against Arizona and New South Wales. AMSR2 overestimates the dynamics for Arizona, India and Kenya, and underestimates these for Australia, California and the Rur catchment. For the latter region an additional problem seems to hamper the soil moisture retrieval results, because a scaling factor of 5.5 is the highest we have found and designates low information content and high (random) noise. In addition to these findings, we expect less dynamic variability in a variable that represents a larger area, i.e., the areal difference between the hundreds of meters of a CRNP footprint and the tens of kilometers of remote sensing pixel/footprint is still evident.

The high noise of AMSR2 for the Rur catchment is also identified by the low $SN{R}_Y$ of −18 dB. In addition, in Arizona, California, Karnataka and NSW the negative $SN{R}_Y$ indicate that the noise is larger than the soil moisture signal in the respective time series. For Kenya, AMSR2 provides valuable data with almost the double soil moisture signal contribution as compared to (random) noise. SMOS provides soil moisture time series at adequate accuracy for all sites, with best results for the Australian site. Here, the soil moisture signal contribution to the time series is four times higher than the noise. Highly dynamic soil moisture with change in covering vegetation water content is well-captured by SMOS. Moderate RFI impact is still included in SMOS records over India.

For ASCAT the very low moisture conditions in Arizona lead to problems in soil moisture retrieval. For the active C-band sensor a $SN{R}_Y$ of −14 dB indicates that no moisture information with reference to CRNP and GLDAS2 is given. The reason can be identified as follows: The penetration depth of the scatterometer increases with soil dryness. During wet periods a shallow penetration depth is given with a soil roughness similar to the soil surface small scale topography. With decreasing soil moisture a larger soil volume is sampled with increasing roughness due to subsoil gravel and stone contributions to an increasing volume scattering. Several semi-empirical and theoretical models for describing backscatter from a rough soil surface have been published so far, but the correct parameterization of global roughness still poses significant challenges [132], and was the reason to circumvent this problem by using the change detection algorithm [97]. The changing roughness conditions should be accounted for in future retrieval models before ASCAT application in very dry regions. The ASCAT $SN{R}_Y$ values for the Rur catchment, New South Wales, and Kenya indicate a very good performance for wetter regions.

In addition to Table 3 the absolute TC error standard deviation ${\sigma }_{{\epsilon }_Y}$ is also seen in Figure 8. In principle a direct comparison of ${\sigma }_{{\epsilon }_Y}$ from different triplets is not valid, because these errors are strictly related to the choice of the triplet and thus cannot be compared to errors derived from another triplet [127]. The extension of this method to a multiple collocation analysis as developed by Pan et al. [133] would solve this issue. However, this would drastically reduce the number of common data points as the considered remote sensing time series are not continuous (e.g., SMAP provides a three daily resolution). Scipal et al. [134] found that at least 100 observation triplets are required for a reliable estimation of the error variance. Thus, much longer time series would be needed for the multiple collocation analysis, which is not available for all data sets. To overcome this problem, we considered in our validation study triple combinations of the individual satellite data products, the continuous GLDAS2 time series as well as the related continuous CRNP time series to maximize likelihood for adequate comparability. The appropriateness of this approach is supported by the estimated TC error standard deviations for the GLDAS2 time series, i.e., ${\mathsf{\sigma }}_{{\mathsf{\epsilon }}_{\mathrm{Z}}}$, listed in Table 3. At the Kenya site ${\mathsf{\sigma }}_{{\mathsf{\epsilon }}_{\mathrm{Z}}}$ for different satellite product-related triplets varies between 0.0495 and 0.0504. This narrow range indicates close relationships of the inner TC statistics between the triplets so that comparability is given. The ranges of the other sites are 0.0171–0.0214 (Rur), 0.0272–0.0304 (NSW), 0.0199–0.0263 (Arizona), 0.0191–0.0338 (Karnataka), and 0.0251–0.0419 (California), respectively. This indicates that a direct comparability of the California site is valid for SMOS and SMAP as well as to a certain extent for ASCAT, but the direct comparability to the AMSR2 triplet is limited. Thus, despite the issues related to the triple collocation method, we are confident that our approach still provides useful information for the users of remote sensing soil moisture products.

AMSR2 shows in all sites the largest ${\mathsf{\sigma }}_{{\mathsf{\epsilon }}_{\mathrm{Y}}}$ which can only partly be explained by the smaller penetration depth of the passive C-band system. The clear differentiation in a wet and a dry season helps all sensors to provide valuable soil moisture retrievals for Kenya. Similarly, the low vegetation cover at Arizona is a clear asset for soil moisture monitoring by microwave radiometry. In NSW the L-band systems show a clear advantage over the C-band systems. The passive AMSR2 record may be influenced by external signal contributions, because the wet period during Australian winter was not observed. For ASCAT the general soil moisture level and precipitation response is adequate. However, during dry periods low precipitation amounts are directly observed by strong peaks, in contrast to CRNP where the precipitation response does not show large dynamics in soil moisture. This could be related to the penetration depth difference and a potential decoupling of surface soil layers from subsoil [135]. Therefore, at this special site CRNPs may not be used for C-band satellite soil moisture validation. For Tonzi Ranch in California SMAP provides adequate accuracy for further utilization. Here, AMSR2 shows strong sensitivity to rainfall and dry conditions with decoupled soil surface layer, which can be attributed to penetration depth differences between CRNP and AMSR2. SMOS and ASCAT show similar accuracy with ${\sigma }_{{\epsilon }_Y}<0.06$. In India SMAP and ASCAT provide data with a low TC error standard deviation ${\sigma }_{{\epsilon }_Y}$ of ~0.04, whereas AMSR2 overestimates the monsoon conditions and SMOS is strongly affected by RFI. For the Rur catchment SMAP, SMOS and ASCAT provide adequate TC error standard deviations and good soil moisture accuracy. However, SMAP shows a strong linear relationship to CRNP spatial soil moisture average. Overall, SMAP provides lowest TC errors and provides stable accuracy levels over all the target regions.

As the GLDAS2 data set shows much lower dynamic range as the other data sets, the absolute $SN{R}_Y$ and ${\sigma }_{{\epsilon }_Y}$ results for all sensors could be to a certain extent overestimated, but they still identify the relative quality level between the different remote sensing products.

6. Conclusions and Outlook

This study was conducted in order to identify a soil moisture validation reference in Cosmic Ray Neutron Probes (CRNP) for coarse scale global soil moisture products. Instead of point-scale in situ measurements, CRNPs provide footprint-scale soil moisture average over a few hundreds of meters, and decimeters of soil depth, thus better representing the soil moisture value at the scale of satellite products. Over showcase sites on five continents, i.e., in Germany, Arizona and California (USA), Kenya, Karnataka (India) and Australia, satellite products from SMOS, SMAP, AMSR-2, ASCAT and the GLDAS2-NOAH simulation have been validated against CRNPs. Both standard validation methods and the Triple Collocation (TC) approach have been used for this purpose.

Standard soil moisture validation scores indicated good correlation for most data sets and most sites. The ability of soil moisture records to better capture the moisture levels at dry or wet periods differs from site to site. The interpretation of these results is challenging, because the scale mismatch of satellite footprints, simulation pixels and CRNP footprints still exists and the general soil moisture level may differ within the coarse grid by retaining the general relative dynamics. Moreover, due to different dynamic ranges of the analyzed data sets and the different sites, direct comparisons of the scores are debatable.

To overcome these issues, the TC method identified further structural quality differences between the satellite soil moisture products, by relating their dynamics to the CRNP observations and the GLDAS2 simulations. An asset for further interpretation is the linear scaling towards the reference. Results indicated that the utilization of scatterometers over very dry regions gains large noise fractions, because the changing soil surface roughness conditions are hard to be covered by available soil moisture retrieval models. An external estimation of the roughness temporal variability could solve the known problem.

Future resolution-enhanced global soil moisture records will close the scale gap to CRNPs, increasing the importance for footprint-scale soil moisture references for direct comparison. Therefore, the results of this study encourage the extension of the CRNP network globally. However, an operational biomass correction for the CRNP measurements needs to be developed before using it as a reference in regions characterized by high vegetation dynamics. The TC method also allows a direct comparison of CRNP data and remotely sensed soil moisture by a certain normalization of penetration depth differences due to pre-TC scaling. This is true for all regions except for NSW in Australia, where CRNPs may not be used for C-band satellite soil moisture product validation by the TC method, as penetration depth differences and top soil—sub soil decoupling introduces larger validation errors. Further investigation is needed here.

In general, this study identifies region characteristics where specific sensors and retrieval methods show good performance, whereas in other regions the complementary sensors provide better results. Therefore, a concept like the ESA Climate Change Initiative (CCI) soil moisture data product [136], where active and passive microwave sensors data were combined to a global soil moisture time series dating back to 1978, would be extremely useful. Here, a hierarchical system identifies the best performance for a location and selects a single or a combination of sensors to provide a high accuracy soil moisture product. However, this result is scaled towards the GLDAS-NOAH climatology, evaluated also in this study. As shown by this study the successor GLDAS2 has a much lower dynamic range than the satellite products and also the CRNP observations, indicating a reduced dynamic range of the CCI product. SMAP shows over the selected regions relatively low bias, an adequate dynamic range as identified by the TC error covariances, and the highest signal-to-noise ratio in this study. Therefore, we recommend generating a global long-term soil moisture data set based on the SMAP, as soon as a long enough time series becomes available.