Table of Contents -- Index -- PS reprint -- PDF reprint

Astronomical Data Analysis Software and Systems VII
ASP Conference Series, Vol. 145, 1998
Editors: R. Albrecht, R. N. Hook and H. A. Bushouse

The IRAF Mosaic Data Reduction Package

Francisco G. Valdes
IRAF Group, NOAO, PO Box 26732, Tucson, AZ 85726

 

Abstract:

The IRAF Mosaic Data Reduction Package, mscred, processes data from mosaics of CCDs such as the NOAO CCD Mosaic Camera . A brief design of the package was presented earlier by Valdes (1997a). Since then a first implementation of the software has been completed. This paper summarizes the current status of the software and our plans for future developments.

           

1. Current Status

Most of the components of the original design (Valdes 1997a) for an IRAF Mosaic Data Reduction Package have been implemented in the first version of the mscred package. This has allowed complete, end-to-end reductions of data from the NOAO CCD Mosaic Camera. The package operates on multiextension FITS format (MEF) files consisting of a global header and individual image extensions for each amplifier (Valdes 1997b). The raw data files are processed so that each image extension is flux and coordinate system calibrated. Mosaic cameras have gaps and misalignments between the CCD elements so, when a complete image of the sky is desired, multiple ``dithered" exposures are taken. The mscred package provides tasks to resample the data into a final image with the geometric effects (gaps, alignments, and optical distortions) removed. This process can also improve the signal-to-noise and eliminate cosmic rays and cosmetic defects. Observations have been obtained with the NOAO CCD Mosaic Camera and processed with the mscred package which produce images of high scientific and aesthetic quality despite the engineering grade CCDs used in the current instrument.

The functionality of the mscred package can be broken down into the following categories: (1) display, (2) basic CCD calibrations, (3) coordinate registration, (4) mosaicing, (5) taping, and (6) miscellaneous. The package is used both at the telescope and after the observing run for quick-look or full reductions and for data analysis. Some of the categories apply to both uses as, for example, the display of mosaic data. In this paper the package tasks are identified with their names in bold font.

Being able to display an exposure as an approximation of a complete (mosaiced) image of the sky is a prime requirement both at the telescope and during data reduction. We are developing a new display capability for this purpose as part of the NOAO Mosaic Data Handling System (Tody 1997). The mscred package provides an interim task (mscdisplay) to display multiextension data as an image in a standard display server such as Ximtool. Mscdisplay includes real-time capabilities to display the data while a readout is in progress. Related tools allow users to interact with the displayed mosaic exposure (even during readout) to evaluate focus ( mscfocus) and to do quick-look analysis (mscexamine) including PSF fitting, statistics, graphics, and celestial coordinate measurements.

Basic CCD calibration provides for combining sequences of calibration exposures (zerocombine, darkcombine, and flatcombine) and the standard CCD calibration operations of overscan subtraction, trimming, bad pixel replacement, zero level subtraction, dark count subtraction, and flat-fielding (ccdproc). The input and output of these operations are MEF files.

The mscred package places emphasis on having an accurate celestial coordinate system (called the world coordinate system or WCS). Using prototype astrometry tools in the mscred package (msctpeak) an accurate WCS consisting of independent solutions for each CCD relative to a common reference pointing including all optical distortion and alignment terms has been derived for the NOAO Mosaic at the Kitt Peak National Observatory Mayall 4-meter and 0.9-meter telescopes (Davis, 1998). This was done using exposures of astrometric fields. This WCS is part of the raw data produced by the Mosaic Data Capture Agent (DCA) (Tody & Valdes 1998) with the coordinate system reference point set to the telescope pointing coordinates.

The zero point of the WCS, which is initially set by the telescope pointing, can be adjusted to a precise absolute coordinate or to common coordinates in a dithered set of exposures by displaying the exposures and identifying one or more reference stars (msczero). Given that the raw data already have relatively good coordinates there is a task (still evolving) that takes a set of overlapping exposures and either a set of coordinates or random regions and registers their WCS using cross-correlation (mscregister).

Using the WCS, the multiple images from each amplifier in a mosaic can be resampled to make a single image on a uniform pixel grid having a standard WCS, such as a tangent plane projection (mscimage). By using one exposure as a reference, multiple dithered exposures can all be resampled to the same pixel grid system (the same tangent point and pixel scale) so that the images may be stacked (mscstack) to make a final image without further resampling. The stacking process excludes the gaps and may include use of bad pixel masks and various scaling and pixel rejection algorithms found in the standard IRAF imcombine routine. Since combining dithered exposures is a common operation a higher level task ( mscdither) combines mscregister, mscimage, and mscstack to directly produce a final image.

Tasks for taping of data are only included as an interim measure until generic IRAF tasks include direct support for disk FITS files in MEF format. For MEF data the duties of the taping tasks (mscwfits and mscrfits) are simply to transfer the FITS file to and from tape with the appropriate FITS blocking, efficient listing of the contents of tapes with multiextension files, and recording the disk filenames and restoring the files to disk with their original filenames. The tape is a valid FITS tape.

The mscred software is packaged as a standard IRAF external package for IRAF version 2.11 and later. Although this is an early version of the software it has been made available to users of the NOAO Mosaic Camera and other interested parties developing mosaic cameras. Releases of the software will be made periodically as new features are added.

2. Future Work

There are many things which still need to be added. These range from minor improvements to a few major research and development items. The major items are discussed in the following sections. The minor items consist of an improved syntax to interface MEF data to existing IRAF tasks that operate on lists of images, expanding the CCD processing task ccdproc to provide for incremental reductions, using a better WCS representation, and a task to restore flux conservation in flat-fielded data.

A wildcard syntax is needed to easily select a set of image extensions from an MEF file rather than the current requirement that each extension be listed explicitly. It takes special care to produce a good flat-field for a wide-field mosaic so for quick-look and initial reductions it is desired to apply archival calibration data, such as a high quality master sky flat, and then continue with incremental calibration using data acquired during the course of observing. Ccdproc needs to be modified to easily support incremental calibration.

The WCS representation for a wide-field optical image is better given as a radial projection (as proposed for a FITS world coordinate system standard) although a general polynomial distortion residual will still be required. Currently a tangent plane projection is used in combination with a separate text file defining a polynomial distortion function. Another property of wide-field images, such as with the NOAO Mosaic at the 4-meter telescope, which is not obvious at first is that the pixel area (square arc seconds per pixel) may vary significantly. This means the sky and object counts vary with position. Flat-fielding attempts to make the sky counts constant which leads to flux errors. A task based on the WCS is required to restore the correct flux per pixel to flat-fielded data prior to doing any photometry. This only affects the MEF files because the resampling operation ( mscimage) naturally accounts for the varying pixel areas.

2.1. Pixel Masks

Pixel masks assign integer codes to each pixel. IRAF provides a pixel mask format which is very compact for masks containing regions of constant value. In mscred pixel masks are used to identify bad pixels with codes values for cosmetic defects, saturated pixels, and cosmic rays. The masks are assigned to data exposures and the software uses these assignments to determine bad pixel information for the data pixels. The current software supports the first category of predetermined cosmetic defects for replacement by interpolation, avoiding bad data in automatic display scaling, excluding bad data from statistical sampling of scaling factors for combining, and exclusion during the stacking of dithered exposures.

The issues that still have to be addressed are updating other pieces of the the software to add to the mask, such as the flagging of saturated pixels, additional uses of the bad pixel information, such as during resampling, and storage of the bad pixel information in multiextension FITS files. The last topic requires mapping the compact IRAF pixel mask format to a FITS format; most likely as a binary table extension.

2.2. Pixel Uncertainty Information

The propagation of pixel value uncertainty information naturally starts with the raw data. The mscred tasks need to be expanded to propagate the pixel uncertainties from the raw data during each step that modifies or transforms the pixel values. There are two development stages that need to be completed. The first is to define the data format representing the pixel uncertainties and the second is to understand how the uncertainties propagate in operations such as flat-fielding, resampling, and combining with pixel rejection.

Research in representing the pixel uncertainties is needed to, hopefully, find a compact description requiring much less than one uncertainty value for each data pixel. Preliminary research suggests a combination of a scaling relative to the pixel data, header keywords, and mapping to a finite set of discrete values that give uncertainties to a useful precision. A key feature of this is the use of pixel masks which can be stored in a compact format as described previously.

2.3. Astrometry

The mscred package supports a coordinate system that is quite accurate. The software maintains and propagates this coordinate system. Much of the coordinate system description is fairly static and only terms relating to zero points and rotations need to be calibrated on an individual exposure or run basis. Currently the instrument support personnel provide the static part of the coordinate system description and the mscred package provides tools to modify the zero point to yield absolute coordinates and to register overlapping exposures. The problem is that if users want to modify anything but the zero point they have to do a complete astrometric solution which requires a good astrometric field with many stars.

The desired enhancements are to let users to have more control of the coordinate system calibration and to integrate catalog servers to ease the determination of a zero point for absolute coordinates. The first part relates to allowing adjustments of the coordinate system representation short of requiring a compete new astrometric solution. For instance with just a few good astrometric objects users should be able to adjust the scale and rotation in addition to the zero point.

2.4. Data Reduction Agent

The Data Reduction Agent (DRA) is an ambitious part of the NOAO Data Handling System which was described in the original design. It is not directly a part of the mscred package. However, this pipeline tool is intended to be portable with the mscred package and be closely tied to the mscred functionality. As work progresses on the DRA there may be enhancements of the mscred package to support the automatic reduction of mosaic data in a data handling system environment.

References:

Davis, L. 1998, this volume

Tody, D. 1997, in Astronomical Data Analysis Software and Systems VI, ASP Conf. Ser., Vol. 125, eds. Gareth Hunt and H. E. Payne (San Francisco, ASP), 451

Tody, D. and Valdes, F. 1998, this volume

Valdes, F. 1997a, in Astronomical Data Analysis Software and Systems VI, ASP Conf. Ser., Vol. 125, eds. Gareth Hunt and H. E. Payne (San Francisco, ASP), 455

Valdes, F. 1997b, in Astronomical Data Analysis Software and Systems VI, ASP Conf. Ser., Vol. 125, eds. Gareth Hunt and H. E. Payne (San Francisco, ASP), 459

Footnotes

...

National Optical Astronomy Observatories, operated by the Association of Universities for Research in Astronomy, Inc. (AURA) under cooperative agreement with the National Science Foundation.

...

© Copyright 1998 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA