Median filtering Script

The script generates various median images. These images are made by using the imcombine function of the IRAF image reduction package. Every night images are taken using the blue and red filters, these images are taken at an exact sideral second, they have an interval of a certain amount of seconds between each exposure. For example, an image would be taken in the blue filter at sideral second 34000, then a red filter image at 34015, a blue at 34030 and so on. Since the blue filter and red filters have differant pixel values these cannot be combined. The median script does the following.

Day1 Blue34000, Day2 Blue34000 Day3 Blue34000 , imcombine into a Bluemedian34000

Day1 Red34015, Day2 Red34015, Day3 Red34015 , imcombine into a Redmedian34015

After all the images over the last 9 days are processed, a whole lot of median images are produced. These medians will be used by another script to generate a divided image that will hopefully only show objects moving in the image, such as meterorites, planes and most importantly clouds.

Image Generation(acquisition machine)

The images will be generated in fits format by the acquisition program cloud.c that will be running on a separate machine due to the excessive CPU usage of the manufacturers driver which polls the parallel port constantly and doesn't use DMA. This might be improved upon at a later date.

The end result of this operation is:


Image Processing(working on the processing machine)

At this point an IRAF script called changeformat.cl will now process this fits image and generate several new images for the web page, the backup directory and the the last n nights directory.

The Images that are used in the creation of the median images are:

Median Filter IRAF script(working on the processing machine)

An IRAF script will have to be scheduled to run every mourning to check that new files have entered the /home/data/lastnnights/ directory and if they have, it will then eliminate the oldest night in the directory.

This script will then take the last 10 nights that have in their headers the keywords SUN=NIGHT and MOON=MOONDOWN and will do a median filter on them using the IRAF task IMCOMBINE. This was found to be a lot faster than IMSUM which is another IRAF task that can do median filters.
 
 
Number of Images
Time taken for IMCOMBINE(sec)
Time taken for IMSUM(sec)
3
0.50
0.66
7
1.92
1.54
11
3.50
2.98
15
5.24
29.60
19
7.07
127.98

It should take about 4 hours or less to median filter an entire set of images and have them ready for the nighttime subtraction.

For every sidereal second a median filter file for that second will be generated and left in the /home/data/medians/ directory. These files will be used to subtract from the incoming files of the next night, and inside the IMAGE PROCESSING SCRIPT, a last subtracted image and animation's of the subtracted images will be done. 

Backup to DVD or tape C script

As said earlier in the IMAGE PROCESSING SCRIPT, a gzipped FITS and a thumbnail PNG were sent to the /home/data/backupfiles/ directory. A C script will be scheduled to copy these files to a DVD disc recorder, upon successful copy the files will be transferred to the /home/data/archive/ directory were they will be stored for future web viewing. The C script shall check to see that there is an empty DVD in the disc drive and will then start to copy the files if for some reason, the disc is full or there is no disc then an email message will be sent to a person on the mountain to replace the disc. A check will also be run to compare the files on the DVD to the one on the hard drive to insure that they have copied correctly.

The objective therefore of this scheduled script is to backup the gzipped FITS and PNGs to DVD and then transfer them to the /home/data/archive/ directory.

Clean up  script for the archive directory

The gzipped FITS and PNGs in this directory will be viewed by the web, and will be downloadable, however due to disk space constraints we will not keep all the data online. Since all the data up to this point has been backed up on DVD, this data will slowly be erased using a cleanup algorithm.

This algorithm will erase files older than a certain date, but using different ratios to cut them off, an example could be that we keep only every 4th image from files older than 2 months, every 8th file after 4 months, every 16th file after 6 months and maybe after a year only 1 or 2 images of the night in question.

This cleanup script will probably be run once a day.