See also the AVHRR processing overview document.

AAPP (ATOVS and AVHRR Pre-processing Package from UK Met Office)

Purpose as documented in the AAPP Scientific Description (PDF) and AAPP Software Description (PDF):

For the NOAA satellites, the decommutation task performs the extraction of data particular to each
instrument from the HRPT format. The data from each sounder (HIRS, MSU, AMSU, MHS) and from
the imager (AVHRR) are put into separate files. The following tasks perform the computation of the
calibration coefficients, the satellite navigation (satellite position calculation on its orbit) and image
navigation (longitude and latitude position of the pixels, taking into account the sensor geometry, solar
angles, etc.). The pre-processing tasks achieve the conversion of the raw HRPT data (Earth and
atmosphere VIS and IR radiances) into physical parameters (brightness temperatures, albedo) and
perform data mapping between the measurement grids of the different instruments (AVHRR/HIRS,
AMSU-MHS/HIRS mapping...).

and again,

The core AAPP can be broken down to five major tasks:

1. Decommutation (only useful for direct acquisition of NOAA data)
2. Calibration/satellite navigation/localisation
3. Preprocessing step 1 (atovin): Apply calibration coefficients, convert radiances to brightness temperatures.
4. Preprocessing step 2 (atovpp): Instrument mapping on another instrument grid
5. AVHRR mapping on HIRS and cloudmak (avh2hirs).

The following figures (Figure 2, Figure 3, Figure 4, Figure 5) show the different processing chains according to the origin of the data. The figures don’t make mention of pre-NOAA-K satellites, for which the MSU instrument was used instead of AMSU-A and AMSU-B/MHS. The different levels of AAPP data are defined as follows:

• level0: raw HRPT data.
• level1a: separated data for each instrument.
• level1b: geo-referenced and calibrated data (reversible: calibration coefficients are separated from raw data).
• level1c: geo-referenced and calibrated temperatures and albedo (non-reversible: calibration coefficients are applied to numerical data). In the case of IASI, level 1c implies Gaussian apodisation.
• level1d: mapped and filtered data (cloud mask).

So our processing stream is;

1. Generate 1A from HRPT: hrptidf, decommutation, chk1btime
2. Generate 1B from 1A: avhrcl
3. Possibly Generate cloud mask

It looks like alot of this can be achieved with the script AAPP_RUN_NOAA_script. This script does the following;

1. Receives as input the absolute pathname of the HRPT data file and the year of the data (this parameter is not present in the HRPT format)
2. With the tool hrpidf.exe, it extracts the satellite name, the day of the year and the time of the data
3. Using orbital bulletins and either tbusing or tleing, the satellite is navigated. Velocity/position files are also created for several days with satpos.
4. With the date, the time of the data and the satellite position file, it gets the orbit number (sdh2orbnum).
5. Then, it calls different modules to make the decommutation, navigation/localisation, calibration, mapping, cloud mask tasks (decommutation, hirscl/hirscl_algoV4, msucl, amsuacl, amsubcl, mhscl, avhrcl, atovin, atovpp, avh2hirs).
6. For AVHRR, HIRS and MSU, before and after navigation/calibration task, AAPP_RUN calls tools (prhavh, prhirs, prhmsu) to write level 1B headers and first records into ASCII files (phavh_before_calib.log, phavh_before_calib.log, …).
7. At the end, it renames all output files to include information in the file names: Satellite name, date and time, orbit number.
   

One thing to note about AAPP is the following; "The AAPP standard distribution uses the data exactly in the format described in the NESS/NOAA TM 107 - Rev. 1 (Planet, W. G. (Editor), 1988) and also in the NOAA-KLM User's Guide, but expects the original downlinked 10 bit words unpacked by being stored right justified in 16 bit words." NOAA files from Murdoch and Curtin are currently reaching iVEC as 10bit packed (NB - subject to change).

There are currently unpackers written to read WASTAC HRPT variants and output 16 bit formats compatible with AAPP. See WASTAC svn:

https://wastac.ivec.org/wastacsvn/scripts/trunk/noaa_unpack/

Also the output format is not HDF - it is fairly close to the NOAA KLM style HRPT.

Cognac AAPP Installation

1. Download AAPP 6.1 and unpack
2. Unpack 6.2 over the top of source tree
3. Unpack 6.3 over the top of the source tree
   
4. Configure, compile and make for cognac. Intel's Fortran compiler ifort was tried but undescribed internal errors halt it from completing the build.
   

$ module load intel-fc/8.1.033
$ ./configure --fortran-compiler=ifort --c-compile=icc --site-id=WAU --prefix=/projects/wastac/tools/aapp-6 --station=perth
$ make
$ make install

Normally then you'd have to set AAPP_PREFIX to /projects/wastac/tools/aapp-6, but this is not required if using the aapp-6 modulefile created for wastac on cognac (ie "module load aapp-6"). Although because the wastac-all module loads aapp-6, you can just use the wastac-all module.

AAPP Quickstart

Here is the procedure to process NOAA L0 HRPT data using AAPP on cognac to produce L1B AAPP format HRPT (close to NOAA's HRPT L1 format but not identical - see AAPP's Formats documentation). We start with an example NOAA 15 file called n15.0703102251_45870 and assume the necessary "module load aapp" has been completed (either manually or by a "module load wastac-all" to set required environment variables).

$ module load aapp-6                                               # Loads aapp variables $AAPP_PREFIX etc
$ noaa_unpack.py -t n15.0703102251_45870                           # extracts tle, header and data (to 16 bit)
3494 frames read
$ ls
n15.0703102251_45870.header
n15.0703102251_45870.tle
n15.0703102251_45870.dat
$ cp n15.0703102251_45870.tle ${AAPP_PREFIX}/AAPP/orbelems/tle_db  # Will find a way around this soon
$ tleing -s noaa15 -f n15.0703102251_45870.tle                     # Ingest TLE
$ AAPP_RUN_NOAA -Y 2007 -i AVHRR -o . n15.0703102251_45870.l1b     # Generate L1B using $AAPP_PREFIX/ATOVS_ENV6
$ ls
avhrcl.log
decommutation.log
decommutation.par
hrpt_noaa15_20070310_2251_45870.l1b
n15.0703102251_45870.header
n15.0703102251_45870.tle
n15.0703102251_45870.l1b

Notes on above:

• This procedure is not setup nicely for multiple users each having their own customisations just yet. So right now either you use the standard AAPP_RUN_NOAA script below, or you make a copy of this (from $AAPP_PREFIX/AAPP/bin) and ATOVS_CONF (from $AAPP_PREFIX) and change the two files locally. Please don't change the system ones!
• noaa_unpack is a custom utility written for wastac. It can extract the (ASDA) header, tle and 10 bit data, then unpack the latter to 16 bit, and writes all of these to separate files. It's written in Python and is quite slow currently, so this could be improved, but it was easier to prototype as a script initially as the implementation was quite tricky. It and unit test is available at https://wastac.ivec.org/wastacsvn/scripts/trunk/noaa_unpack/ but is installed to /projects/wastac/tools/bin currently. Files are written to current working dir.
  
• It's possible we could run the scripts called by AAPP_RUN_NOAA separately, but AAPP_RUN_NOAA is easiest to get going with.
• Note the -o switch for AAPP_RUN_NOAA - it should be the destination L1B directory as several files are written. We're using CWD here.
  
• By omitting the AVHRR flag to AAPP_RUN_NOAA, all instrument files are generated
• All AAPP processing config settings (instruments, platform names, locations of navigational assets) are stored in ${AAPP_PREFIX}/ATOVS6_ENV but it's possible to provide our own by creating $HOME/ATOVS6_ENV. For not the former has been customised so you may not need to do this.