Table of Contents -- Index -- PS reprint -- PDF reprint

Astronomical Data Analysis Software and Systems VII
ASP Conference Series, Vol. 145, 1998
Editors: R. Albrecht, R. N. Hook and H. A. Bushouse

Pipeline Processing and Quality Control for Echelle Data

G. Morgante and F. Pasian
Osservatorio Astronomico di Trieste, 34131 Trieste, Italy
Email: pasian@ts.astro.it

P. Ballester
European Southern Observatory, D-85748 Garching, Germany

 

Abstract:

In the framework of a collaboration between ESO and OAT concerning the development of a data reduction pipeline and the related quality control subsystems software for UVES/VLT, a detailed feasibility study is being performed on the basis of data extracted from the EMMI/NTT archive.

The pipeline reduction is based on an accurate set of ``pre-calibrated solutions" and will also result in a powerful tool for the release and the distribution of the scientific data, from a simple quick look at the data just collected to a complete echelle reduction with the highest possible accuracy.

           

1. Overview

It is ESO policy to perform data reduction for VLT instruments in ``pipeline" mode, either at the observing premises or at headquarters. The result of the pipeline will be a ``standard quality" data product, certified by the Observatory.

In order to verify if a pipeline was feasible for echelle data, in particular for UVES (Ultraviolet-Visible Echelle Spectrograph), a test on EMMI (ESO Multi-Mode Instrument) data acquired in echelle mode and extracted from the NTT archive was performed:

• a preliminary processing pipeline was defined using the MIDAS echelle package;
• a set of pre-calibrated solutions was available, and other have been defined, to optimize the processing speed while achieving results within an acceptable level of accuracy;
• archive data were processed in pipeline mode;
• quality control on the data was performed, with the purpose of determining the instrumental stability.

It is important to note that the approach followed in the development of the work has been to minimize assumptions, or a priori knowledge of the EMMI instrument, of its data and parameters; the decision was made to rely uniquely on the contents of the archive and of the keywords of the files stored therein. As a consequence, an evaluation of the completeness of the archive of EMMI echelle data was also performed.

2. Key Items

The key points of the work were identified to be the following:

• test of the performance of the preliminary version of the automatic reduction chain compared with the interactive calibration;
• adaptability of the pipeline procedures on data collected with the R4 grating;
• verification of instrument stability over time with particular reference to geometrical and photometric stability;
• analysis of the accuracies expected from wavelength and flux calibrations for the eventual definition of new calibration strategies;
• evaluation of different effects, such as atmospheric variations, differential refraction, the impact of mechanical deformations on optical performance and instrument maintenance;
• definition of procedures for quality control tests for EMMI data;
• estimation of CPU and hardware requirements to run the final pipeline;
• the completeness of the information stored in the archive, and the corresponding requirements to be imposed to operations to ensure such completeness;
• requirements to be imposed to data processing to ensure that the inevitable instrument instabilities are correctly recovered by the data reduction software, both in pipeline and in interactive modes.

3. Pipeline Processing and Pre-calibrated Solutions

The MIDAS environment, and in particular its Echelle Package, include proper tools for building pipeline DRS for generic echelle instruments, and in the future for UVES, with some minor improvements. In the pipeline built for EMMI, pre-calibrated solutions (i.e., tables containing the results of interactive processing on selected ``reference" spectra) were used for order detection and wavelength calibration as starting points. The UPDATE/ORDER MIDAS command was used to re-define the positions of the orders starting from the pre-calibrated order table.

The wavelength calibration has been performed using IDENT/ECHELLE with the GUESS method (i.e., searching for the optimal solution starting from a pre-defined position of calibration lines). The result was accepted only after checking the plot of residuals and the ``percentage of identification among the half brighter lines", that should be the highest, and in any case above 50%.

The analysis of applicability for the pre-calibrated solutions has verified that, for a stable instrumental configuration and performance, the current procedures are quite adequate. Minor software upgrades could be made to take care of extreme situations; e.g., for EMMI a non-rigid shift in order positions due to a changed field lens was corrected with a custom modification of the UPDATE/ORDER command. A new set of pre-calibrated solutions shall however be computed by staff whenever a manual intervention on the instrument is made; such interventions shall be appropriately reported in the archive. New pre-calibrated solutions should be computed during every test period, to minimize a-posteriori software recovery.

4. Quality Control

For an efficient quality control to take place, the setup of proper operational procedures is necessary:

• at the telescope:
  • test frames must be regularly acquired (once during every test period):
    • bias and long dark frames are useful to monitor the detector parameters and features;
    • flat fields for different instrumental configurations provide additional information on the instrument stability;
  • scientific data sets must contain all calibration frames necessary for re-using the data and assessing their quality:
    • the optical performance of the instrument can be monitored by the analysis of the flat field stability and standard stars fluxes;
    • a list of standard stars of a very early spectral type or with a featureless spectrum should be selected in order to evaluate and subtract the atmospheric contribution;
    • the ThAr spectra allow the geometrical stability and the dispersion relation accuracy over time to be checked;
  • information on manual interventions must be regularly logged and archived;
  • instruments should always be in a stable and documented state.
• at headquarters:
  • whenever a modification to an instrument occurs, the related processing script (or parameters) should be updated;
  • previous versions of processing scripts should be archived;
  • the test data should be promptly processed to produce a proper trend analysis on instrument behaviour.

5. Archive Completeness

In this work, it was decided to rely uniquely on the contents of the archive and of the keywords of the files, because this is the standard approach an ESO user would take in accessing data he/she would have not acquired directly. In accordance with the results of Péron et al. (1994), some incompleteness in the archive of EMMI echelle data has been identified. In the future, for EMMI but especially for UVES, the following are key items for archive completeness:

• routine acquisition (e.g., in service mode) of all frames necessary for proper reduction of the scientific data;
• indication in the archive or in the header of the scientific frame, or in a dedicated Observation Summary Table (OST) of the calibration files considered by ESO as the ones to be used for an ``observatory standard" reduction of data;
• creation of data sets (one per scientific frame), including science data, calibration frames (acquired with the science data, or extracted from the archive), and the OST;
• correctness of the keywords in the frame headers.

Acknowledgments:

The work has been carried out under ESO Contract No. 47980/DMD/96/7513 to OAT. Useful information and valuable suggestions were provided during meetings at ESO by H.Dekker, S.d'Odorico, H.Hernsberge, R.Gilmozzi, P.Grosbøl, L.Kaper, B.Leibundgut, P.Molaro, P.Nielsen, and L.Pasquini. Useful discussions with P.Bonifacio and S.Monai of OAT are gratefully acknowledged.

References:

Ballester, P. 1992, in Proc. 4^th ESO/ ST-ECF Data Analysis Workshop, ESO Conf. and Workshop Proc. No. 41, 177

Ballester, P., Rosa, M. R. 1997, Astr. Astroph., in press, ESO Scientific Preprint No. 1220

Morgante, G., Pasian, F. 1997, Feasibility Study of Pipeline and Quality Control Software for Echelle Data - Final Report, OAT Technical Report

Péron, M., Albrecht, M. A., Grosbol, P. 1994, in Proc. ESO/OAT Workshop Handling and Archiving Data from Ground-based Telescopes, M.A.Albrecht & F.Pasian eds., ESO Conf. and Workshop Proc. No. 50, 57

Verschueren, W., Hernsberge, H. 1990, in Proc. 2^nd ESO/ST-ECF Data Analysis Workshop, ESO Conf. and Workshop Proc. No. 34, 143

© Copyright 1998 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA