SURFRAD Solar Retrospective verification

This document is a guide for verifying retro cases against SURFRAD/SOLRAD data in the MATS MySQL database. This network(s) of 14 stations are pulled from the GML ftp server on a daily basis and are stored indefinitely within the database.

Updated: April 11, 2025
POC: MATS Service account (mats.gsl@noaa.gov)

NOTE: Station data is stored indefinitely inside the surfrad3 database, but if you happen to come across missing data, please contact the MATS team

Process Retro grids¶

1. Log onto the HPC (Jet or Hera) where your output grids are stored as the “role.amb-verif” role account. If you cannot become role.amb-verif, please let us know and we will have you added.

2. Go to the directory ~role.amb-verif/surfrad3/retro. If your retro is not already linked, create the soft links to the cycle-level directory of your output with a case name of your choosing to highlight the experiment, with the available conventions:

   ln -s {case_cycle_directory} {experiment_name}

3. Go to the ~role.amb-verif/surfrad3 directory and run the following command. Note that on Hera you’ll use gen_surfrad.tcsh instead of gen_surfrad.sh.

   gen_surfrad.sh {start_epoch} {end_epoch} {reprocess? 0 - no, 1 - Yes} {max forecast hours} {experiment_name}

4. If the command runs correctly, it will create text files in the ~role.amb-verif/surfrad3/tmp/ directory. These will be loaded into the surfrad3 database, and the needed table will be created, when the job ~role.amb-verif/surfrad3/load_surfrad.tcsh runs. This job is currently in cron, and runs every 10 minutes.

5. Results should appear on MATS the following morning, but a metadata scan can be run to expedite the process. As role.amb-verif on Hera/Jet, run:

   ~role.amb-verif/mats_metadata/update_metadata_surfrad.py {case_name}

   for instantaneous station verification, and

   ~role.amb-verif/mats_metadata/update_metadata_surfrad_avg.py {case_name}

   for spatially averaged station data as a function of grid resolution.

   If you see a >0 number of records (‘numrecs’) found in the output, your retro is stored in the database with metadata enabled.

6. The final step is to refresh the metadata inside the MATS client by refreshing the Surface Radiation app page a few times in your browser.

If you encounter any issues during any of the steps outlined here, either on Jet or Hera, please contact the MATS team (mats.gsl@noaa.gov)