Cookbook: Using SPICE to extract a spectrum from ultradeep IRS observations of faint 24 micron sources

In this cookbook, we demonstrate the data reduction strategies used for ultradeep IRS spectroscopy. This page follows the IRS Best Practices memo for the same observation, but we now include optimal extraction in SPICE which was not previously available. The ultradeep proof of concept data (PID=252) included observations in SL1, LL2, and LL1 We use the LL1 data here. The target was a z=2.69 ULIRG. A total of 12 hours on source integration was performed, using mapping-mode to place the target at 6 positions along the slit.

This example includes some preprocessing before using SPICE. We assume that you are comfortable with some flavor of standard image processing software such as IRAF or IDL. We will describe the basic procedures and trust that they are straightforward to implement in your favorite package.

Requirements:

• For a better understanding of the wavelength range and names of the modules see the IRS pocketguide.

• Download the data associated with the LL2 observations in PID=252. Using Leopard, search by program and then find AORKEYs 15733248, 15732992, and 15732736. This involves going to Query->By Program->Typing in 252->OK. Then select the BCD data box on the lower right and File->Download Selections which will transfer the zip files from the archive to your hard drive. You can have Leopard UnZip the files for you.
  The downloaded data will be in three subdirectories beneath the location you selected in Leopard. In each, the BCD data will be located in a "ch2/bcd/" subdirectory. The files of interest are the bcd.fits, bmask.fits files.

Pre-Processing:

The basic calibrated data (BCD) files have been processed by the SSC pipeline, which includes ramp fitting, dark subtraction, droop correction, linearity correction, flat fielding, and wavelength calibration. Further processing of the two dimensional dispersed frames is required before spectral extraction:

1. Latent charge removal: A small fraction (1-2%) of charge on the detector persists between frames despite the resetting of the detector that occurs prior to every integration. This latent charge decays slowly over time and is removed by the annealing process. In the case of very faint sources, the source of latent charge is the zodiacal background. Over the course of a six hour AOR, this charge can build up to a significant level. You can remove the latent signal by fitting the slope of the background with time.

   Measure the background independently for each row of each frame as the median of that row. Next, fit the slope of those median values and subtract that amount row-by-row from each frame. The background in BCD images is the residual after ``skydark'' subtraction in 3D. In the deep observations, the residual sky level is about 50 electrons per second, and the latent charge built up to about 5 electons per second by the end of the AOR (see Figure 1).

Figure 1: The latent charge accumulated in a pixel as a function of time (DCE number). The latent charge was measured in the median of 400 pixels.

2. Rogue Pixel Interpolation: Unstable, or ``rogue'', pixels are those which are useable in some AORs and not in others, depending on the recent history of the detector. A mask of known rogue pixels is provided by the SSC, but you can identify further suspect pixels from the data. In many LL observations, rogue pixels are not too much of a problem, because they tend to be subtracted away. The very long time baseline of the current dataset means that they can still show up even after subtracting a delta sky (see below).

   We recommend using the IRSCLEAN program to find rogue pixels in the 2D data. Using the Agressive option, you can have the program flag more pixels than are in the SSC mask. You can further identify suspect pixels by searching for pixels with abnormally high variance with time (inspecting them in IRAF or IDL, not within IRSCLEAN). Finally, though it may be unsatisfying, some visual inspection is often used to find any rogues that still sneak through. Once you have a list of rogue pixels, they can be interpolated over using IRSCLEAN.

3. Residual Sky Subtraction: Once the BCD data are ``cleaned'', you will need to create residual sky images from the data. The observations were taken in mapping mode, so a good sky frame can be created using the other five map positions. A median combination is reasonable, although a resistant mean (removing outliers before calculating the mean) with time is also a good way to calculate the sky value in each pixel. The sky should be calculated for each AOR separately, to keep the frames close in time. If the sky appears too noisy, it is also possible to combine skies from multiple AORs taken close in time. After sky frames are created, they should be subtracted from each BCD frame. In some cases, you may wish to scale the sky frame to the mode of the sky in the LL1 order for best results.

4. Combine the stack of each Map Position: Once the frames have been corrected for latents, cleaned of rogue pixels, and sky subtracted, they can be combined into final frames. Each map position should be combined separately, because the spectra are not rectilinear on the array thus preventing shift-and-add. Again, a resistent mean is a reasonable approach to combining the frames. The final frame will need a valid FITS header, so attach it to, for example, the first header in the stack (unless your software propagates the header for you).

5. Measure the Uncertainty Frame: Given the high redunancy for each pixel in this ultradeep obseration, you can create a new uncertainty frame, rather than progating the BCD uncertainties. You can estimate the uncertainty in the final, combined frames as the standard deviation of each pixel with time. Save this uncertainty frame along with the final frames to be input into SPICE. Again, make sure you have a valid FITS header.

6. Combine the Bmasks: In principle, the bmask.fits files should be combined along with the BCD frames. This should be done with a bit-wise AND. The high redundancy and rogue pixel cleaning means that few transient pixel problems will survive and so only permanently bad pixels will need to be excluded from the final extraction. For a quick look, you could even use just one of the mask files and see what happens.

Optimal Extraction with SPICE

You will need to extract the spectrum from each of the map positions independently. We will walk through the extraction of the third map position as an example here. Repeat the same analysis for the other positions. Given the faint continuum of the object, it is usually not possible to do this in batch mode (the RIDGE module won't find the trace). Launch the SPICE GUI, and choose Open Spice Generic Template-->Point Source with Optimal Extract from the file menu. This will bring up the optimal extraction flow. Then:

7. Load the combined data, uncertainty and mask files that you created in the preprocessing step. You can specify and output directory at this step. Run just the Initialize Parameters and Files module using the stack button in the corner of the module. The dispersed image will be displayed in the FITS window.

8. Run the PROFILE module. There is no need to change the order, so leave the setting as "default". After you click the stack button, the spatial profile will be calculated and displayed in the plot window (see Figure 2). The profile includes other objects in the slit, as well as significant scatter due to noise.

   

   Figure 2: The SPICE GUI after running PROFILE for the third map position. Several objects can be seen in the slit. The object of interest is the middle one.

9. Because the target (in the middle of the slit in this case) is not the brightest object in the slit, you cannot run the RIDGE in the default mode. Instead, you will need to select "Manual". This will bring up the input field to specify the PERCENT across the slit at which you would like to place the RIDGE line. You can use the slider at the bottom of the plot window to move the percentage across, as indicated by a dotted blue line in the plot window. (Note -- Clicking the plot window will move the dotted line but not the percentage in the RIDGE module). When you have the dotted line in a reasonable spot, run the RIDGE module. The trace and the extraction window will be overlayed on the FITS image. It may take a few tries to find exactly the right percentage. To rewind the RIDGE module, just click on the rewind button that has replaced the stack button in the module window.

10. Now you can run the optimal extraction. You should not need to change the default parameters, as the optimal weighting will ensure that only the most signifcant pixels contribute to the extraction. The spectrum will be displayed in the plot window.



Figure 3: The SPICE GUI after running Optimal extract. The source is quite faint, but a very weak PAH feature can be seen at about 29 microns (redshifted 7.7 and 8.6).

Finally, run the POINT SOURCE TUNE module to convert from electrons per second to Jy.

Post-Processing

After extracting the 1D spectrum from each of the six map positions, you will need to coadd them to obtain the final spectrum. The output of SPICE is a spectral table (in your specified output directory) *.spect.tbl, where the * is the name of your input FITS file with the .fits suffix stripped off. The output file is an IPAC table, with the following columns:

|order  |wavelength |flux                |error               |bit-flag            |
|int    |real       |real                |real                |int                 |
|       |micron     |                    |                    |                    |
|       |           |Jy                  |Jy                  |                    |

The 1D spectra themselves can simply be averaged. The uncertainties should be added in quadrature.

Figure 4: The final spectrum, after averaging the six SPICE extractions. The 1-sigma error array is plotted as a dashed line. The target lies at z=2.69, so the wavelength scale is shown in both the observed and rest frames.

Return to the Data Analysis Cookbooks page.