Restoring retro data for visibility, ceiling, surface

Restoring visibility¶

1. First check whether you need to restore obs. You can run this command to see if the needed obs are there:

   use visibility;
   
   select from_unixtime(floor(time/86400)*86400) as day
   ,count(*) as N
   from obs
   where 1=1
   and time >= <start_secs>
   and time <= <end_secs>
   group by day
   order by day;

2. If the data you need aren’t in the visibility.obs file, check whether they exist in the madis3.obs table:

   use madis3;
   
   select from_unixtime(floor(time/86400)*86400) as day
   ,count(*) as N
   from obs o, stations s where 1=1
   and o.sta_id = s.id
   and net = 'METAR'
   and time >= <start_secs>
   and time <=  <end_secs>
   group by day
   order by day;

3. If not, check the obs_retro table:

   use madis3;
   
   select from_unixtime(floor(time/86400)*86400) as day
   ,count(*) as N
   **from obs_retro** where 1=1
   and time >= <start_secs>
   and time <=  <end_secs>
   group by day
   order by day;

4. If the data exist in either madis3.obs or madis3.obs_retro, on hera or jet, run:

   cd ~role.amb-verif/visibility
   ./get_vis_obs_retro.pl <start_secs> <end_secs>

   This will check the madis3.obs table for data for each hour in the requested time window. If there is no data in madis3.obs, the script will automatically check madis3.obs_retro.
   In either case, it will put any data it finds into the visibility.obs table.

5. If there is no data in the madis3.obs_retro (or madis3.obs) table, log on to emb-prod-db and run this command:

   ~amb-verif@emb-prod-db:~/madis51/retro> load_retro.pl <start_secs> <end_secs>

   where <start_secs> and <end_secs> are seconds since 1/1/70 for the start end end of the time window you want to load. This job may take several hours for 10 days worth of data. This loads only METAR stations into the madis3 database.

Then rerun step 1.

Restoring surface¶

See Surface Verification

Restoring ceiling¶

See Retro Ceiling verification