RWAPI on L2 product - CESBIO

SMOS
The training course
SMOS L2 SOIL MOISTURE product
Use of the Matlab toolbox: <strong>RWAPI</strong>
SMOS Workshop
27-29 September
Arles, France

Aim of this talk
� Give an overview of the soil moisture product: UDP, User Data Product
� How to interpret some flags
� reference to the main documents
� Use of matlab toolbox « rwapi », developed by ARRAY Inc.
But
NOT present every field of the L2 Soil Moisture product
No word about the DAP (if you have heard of it...courses at CESBIO !)

3
Outline
I) Documents
II) How to derive on the Soil Moisture retrieval
III) Read the L2 products with the <strong>RWAPI</strong>
IV) Examples

4
Outline
I) Documents
II) How to derive on the Soil Moisture retrieval
III) Read the L2 products with the <strong>RWAPI</strong>
IV) Examples

Documents
2 doc.
� Spec Level 2, describes all Inputs/Outputs files of the L2
� ATBD: algorithm to derive the soil moisture
Where ?
� CESBIO website: http://www.cesbio.ups-tlse.fr/fr/smos/smos_atbd.html
� SMOSBlog

Documents
Spec L2 = all the details of Inputs / Output files

7
Outline
I) Documents
II) How to derive on the Soil Moisture retrieval
III) Read the L2 products with the <strong>RWAPI</strong>
IV) Examples

● L2 products: per ½ orbit
UDP: User Data Product
L2 Soil Moisture course refresher
● Ascending orbits: ~6 a.m. solar local time
Descending orbits: ~6 p.m. solar local time
● 2 files: *.HDR : Header
*.DBL : Datablock,
BINARY file, contains the data
● L2 soil moisture Grid : isea4h9, spatial resolution ~15 km
BUT
~15km
Soil Moisture NOT retrieved over an hexagonal area
●
●
●
●
●
●
●
●

L2 Soil Moisture course refresher
9
Antenna Pattern
Weighted function : WEF
To account for the radiometric sensitivity of
SMOS instrument
Emission of the Surface
dielectric Model
● Dobson model (Dobson et al. 1985)
Mironov's model (Mironov et al, 2007)
Nominal wet soil
Water (Pure and Saline)
● Simple empirical relationship
Frozen soil
Snow and Ice
Barren rocks
Urban
Vegetation Layer
Radiative transfer model : L-Meb
L band Microwave Emission of the Biosphere
Wigneron et al. 2007
�Atmosphere
Water
Low Vegetation
Forest

L2 Soil Moisture course refresher
Surface classification
� For each DGG
� Definition of a Working Area
- 123 km x 123 km
- Surface classification: Ecoclimap, at ~4km
� Ecoclimap reclassified to 12 main classes
10
Low vegetation
Forest
Water
Barren / rock soil
Urban area
Wetlands
Strong Topography
Moderate Topography
Wet Snow
Mixed Snow
frozen soil
Ice cover
Nominal =
low vegetation
Forest
water
~123 km
⇒ Soil moisture derived ONLY
LOW VEGETATION or FOREST fraction
~123 km

L2 Soil Moisture course refresher
Surface classification
��Selection of a branch of the Decision tree
ATBD, Table 18, « branches of the Decision Tree »
��Selection of models for each of the classes of a Working Area
ATBD, Table 20 « Selected models »
4 dielectric models (ATBD, Table 19 « Structure of forward model »)
MN = nominal: L-MEB (Dobson's)
MW & MS = Water (Dobson's)
MD = Cardioid model
Snow
Source: ATBD

12
Outline
I) Documents
II) How to derive on the Soil Moisture retrieval
III) Read the L2 products with the <strong>RWAPI</strong>
IV) Examples

L2 soil moisture product: UDP -User Data Product-
� Parameters of the model
- SM
- TAU
- Hr : roughness
- Surface Temperature
- Dielectric constant
- TB @ 42.5° (from models!)
- their DQX = index, quality of the retrieval NOT an error compared to in-situ...
- ...
⇒ Retrieved = value
Not retrieved = filled value, e.g. -999
� Information about the conditions: Science Flags
forest, topography, rain, snow...
� Evaluation of the retrieval : Confidence flags
retrieval failed (FL_NO_PROD)
is retrieved soil moisture within an expected range? its DQX ?
quality of the models
RFI contaminations
� Conditions of the retrieval: Processing Flags
model used, initial tau conditions, ..
Source: Spec L2, Table 4.9 SM_SWATH Data Set Record

Tools to read SMOS products
*) BEAM + SMOS beam toolbox (See Ahmad's presentation)
● Where to find it : http://www.brockmann­consult.de/cms/web/beam/software
● What for ?
→ L1C, L2 UDP
→ export data to ascii file
*) <strong>RWAPI</strong> (Array Inc)
● Matlab tool to read and extract data L1C, L2 (UDP, DAP), input data …
● Where to find it:
Contact Ali Mahmoodi : ali@array.ca …. offer him a beer
Access to ftp site to download the toolbox

<strong>RWAPI</strong> with Matlab
� Load a L2 product
Matlab function : <strong>RWAPI</strong>.product.m
Input = name of the directory containing the *HDR and *DBL
Important !

Load Data for ALL nodes of the L2 soil moisture product
�load all DGG

17
Outline
I) Documents
II) How to derive on the Soil Moisture retrieval
III) Read the L2 products with the <strong>RWAPI</strong>
IV) Examples
Read retrieved data
Flags

Extract information for 1 DGG and understand the results
Example : 87406 th DGG of the L2 product

Example 1:
Retrieved parameters
SM = 0.1672
TAU = -999 (= not retrieved)
..... WHY ?!

Example 1:
SM retrieved but not TAU ....
Is it expected ? a bug ? ... How to find why ?
1) Call Philippe Richaume....
I do !!
Or
2) look at the FLAGS of the UDP L2 product
tables of the documents : ATBD and the L2 specification product

Example 1
Fraction of →
working area
S_Tree1 : branch of the Decision Tree (see ATBD, table 18)
� Threshold
ATBD, Table 17, « Decision
Tree stage one thresholds »
WO : open water
TS : strong topography
TM : moderate topo.
SW: wet snow
SM : Mixed snow
FRZ : frozen soil
FO : Forest
NO : Nominal
WL : Forest
EB : Barren
TI : Ice
EU : Urban
S_Tree1 = 12 ⇒ FNO > TH_NO: low vegetation > 40% of the Working area

Example 1
S_Tree2 : conditions of the retrieval
Use Table under table 4-9, doc. Spec L2 to analyse the binary
understand the flag = convert to a binary

Example 1
00000001
� Results : Retrieval conditions
- R2
- Very low initial vegetation
optical depth : TAU
- Model used MN : L-Meb!
S_Tree2 : conditions of the retrieval
Spec'L2, Table under table 4-9

Example 1
S_Tree2 = 00000001
��R2
�� Very low initial TAU
�� Model used MN : L-Meb
From Table: �
Initial std on Parameters
to be RETRIEVED
� Nil � not derived
� Only SM retrieved
S_Tree2: conditions of the retrieval
Source : ATBD, Page 89, Table 23

R2 R3 R4 ... what 's the difference ?
� The more TBL1C the more parameters can be retrieved
See in the L2 product Value of M_AVA0 and M_AVA
� If retrieval failed in R3 configuration
� retrieval tried with R2, i.e. less parameters to be derived
� R4: no more

Note:
Initial TAU compared to 2
thresholds: TH_Tau_R_ 23
TH_Tau_R_34
Defined in the AUX_CNFSMF:
Configuration input file of
L2 processor
Source : ATBD, Page 89, Table 23

Example 2
DGG Id = 223130
SM = - 999
TAU = 0.4587
SM not retrieved ....BUT TAU !!!!!!!

Example 2

Example 2
Fraction of the →
working area
S_Tree1 = 17 heterogeneous surface
Working Area composed with
Nominal < 40 %
Forest < 60 %
etc..
Threshold defined
in the ATBD, Table 17,
« Decision Tree stage
one thresholds »
WO : open water
TS : strong topography
TM : moderate topo.
SW: wet snow
SM : Mixed snow
FRZ : frozen soil
FO : Forest
NO : Nominal
WL : Forest
EB : Barren
TI : Ice
EU : Urban

Example 2
S_Tree2 =00100110
00100110
��Retrieval conditions
● R3
● Medium initial TAU
● Model for the dielectric constant MD : cardioid
Spec'L2, Table under table 4-9

Example 2
S_Tree2 =00100110
��Retrieval conditions
● R3
● Medium initial TAU
● Model for the dielectric constant MD : cardioid
4 dielectric models (ATBD, Table 19 « Structure of forward model »)
MN = nominal: L-MEB
Water = MW & MS
Cardioid = MD

Example 2
S_Tree2 gives
● R3
● Medium initial TAU
● Model used MD : cardioid
��Parameters retrieved
● Tau and the
● Dielectric constant
(CADIOID model)
Source : ATBD, Page 89, Table 23

33
Outline
I) Documents
II) How to derive on the Soil Moisture retrieval
III) Read the L2 products with the <strong>RWAPI</strong>
IV) Examples
Read retrieved data
Flags

Flags

Science Flags
Source, Spec' L2 document, Table 4-11 « Structure of the Science Flags in the DSR »

Science Flags
→ from Table 4.11 « Structure of the Science Flags in the DSR », Spec. L2 document
FL_TAU_FO : optical thickness > a defined threshold
FL_External : FL_Rain or FL_TEC or N_Sky>0
FL_Nominal
FL_Forest : fraction of forest > defined threshold
FL_Scene_T
FL_Non_Nom

Confidence Flags
Source: spec. L2 document

Confidence Flags
5 th field: FL_NO_PROD = 1
���������������� retrieval failed
See Table 4-10 « Structure of the Confidence Flag in the DSR »

No value reported in the UDP � No retrieval

Processing Flags
Source: Spec'L2, Table 4-12

Processing Flags
R3 retrieval was tried, failed � R2 attempted

Time series
2 solutions
� <strong>RWAPI</strong> matlab
loop to read all your products
Find your DGG
extract the data you want
� Beam (see Ahmad's presentation)
either
- From the Graphic User Interface : Task bar� Tools ��Export SMOS Grid Points
- Linux, shell command (see Beam Help: export grid point)
« sh /path_to_beam/beam-4.9.0.1/bin/export-grid-points.sh -box $lonmin $lonmax $latmin $latmax
-o myfile /path2myproducts/SM_????_MIR_*UDP*»

Time series
Matlab example : function with a loop to read several L2 soil moisture products
%% Extract Temporal serie
serie = ones(length(List_Product),5) ;
for ifile = 1:length(List_Product) ;
% cd to <strong>RWAPI</strong> directory
cd /home/mialon/Documents/SMOS/ToolBox/<strong>RWAPI</strong>/V14/test ;
produitSMOS = <strong>RWAPI</strong>.product(List_Product{ifile}) ;
% load dataset
d=produitSMOS.dataset(1) ;
% extract all DGGid of the product
dggid=d(:).Grid_Point_ID ;
% find index of your DGG
idxdgg = find(your_DGGid,dggid) ;
% time of the overpass
time = cat(2,double(d(idxdgg).Mean_Acq_Time.Days'),...
double(d(idxdgg).Mean_Acq_Time.Seconds'),...
double(d(idxdgg).Mean_Acq_Time.Microseconds')) ;
serie(ifile,1) = d(idxdgg).Grid_Point_ID ;
% extract Soil moisture
serie(ifile,2) = d(idxdgg).Retrieval_Results_Data.Soil_Moisture ;
% extract Soil Moisture DQX
serie(ifile,3) = d(idxdgg).Retrieval_Results_Data.Soil_Moisture_DQX ;
% extract TAU
serie(ifile,4) = d(idxdgg).Retrieval_Results_Data.Optical_Thickness_Nad ;
% extract TAU DQX
serie(ifile,5) = d(idxdgg).Retrieval_Results_Data.Optical_Thickness_Nad_DQX ;
% release memory !
delete(d) ;
delete(produitSMOS) ;
end

44
� CESBIO website
Thanks for your attention...
http://www.cesbio.ups­tlse.fr/fr/indexsmos.html
ATBD document available = soil moisture retrieval algorithm
http://www.cesbio.ups­tlse.fr/fr/smos/smos_atbd.html
� CESBIO blog : many info, results, tools …
http://www.cesbio.ups­tlse.fr/SMOS_blog/
� ESA website
http://earth.esa.int/SMOS/

Find DGG coordinates from any latitude / longitude
Matlab function on SMOSBlog, tool page
● Download the matlab fct :find_nearestDGG.m
● .txt : file containing all the DGG and their coordinates
Node of the grid DGG
Each DGG has an ID (fixed : does not change with products)

Example how to use find_nearestDGG.m
Arles, DGG Id : 2018331 Latitude : 43.744 ° Longitude : 4.619°