MOPEX and APEX FAQ

General Questions about MOPEX (including APEX)

Q. Should I be using the new MOPEX GUI or the command-line version?
A. The latest release of the command-line and GUI versions of MOPEX (including APEX) are identical in terms of the results they produce. If you are a new user then we strongly recommend using the GUI as it is more user-friendly and has enhanced, built-in help pages. If you are running an operating platform other than Solaris 8+, Mac OSX 10.3+ or Linux RedHat then we also recommend using the GUI, as many users have experienced Perl library linking problems while running the command-line version on other platforms. If you are a seasoned MOPEX user, and/or want to run large batch jobs then you may want to consider using the command-line version, as it is marginally faster and can be scripted. Having said that, even experienced users may find the new GUI help pages useful.

Q. How do I get hold of the command-line version of MOPEX (including APEX) now you've launched the new GUI?
A. The command-line version of MOPEX is distributed with the GUI - you should download the GUI and follow the installation instructions on the MOPEX Download page. Note that the environment variable file for MOPEX has changed from mopex.csh in previous versions to mopex-script-env.csh.

Q. I'm using a flavor of Linux other than RedHat and I'm getting strange error messages when I try to run the command-line version of MOPEX (including APEX). What should I do?
A. The SSC only supports the command-line version of MOPEX on the following platforms: Solaris 8+, Mac OSX 10.3+ and Linux RedHat. If you are running a different platform or flavor of Linux then you are probably experiencing Perl library linking problems (although it may not be obvious from the error messages you are seeing). You should try upgrading to the new release of MOPEX, but if that doesn't work then you should try using the MOPEX GUI.

Q. I've read all of the documentation and the FAQ pages, and I still can't find the answer to my question. How do I get help?
A.If you are having problems with SSC-supported software (MOPEX/APEX, Bandmerge, SPICE, CUBISM, IRSCLEAN, IRSFRINGE, or The GeRT) then please drop us an email at the Spitzer Helpdesk and we'll be happy to help. If you are having problems with Contributed Software then please contact the appropriate distributor.

Mosaicing - mosaic.pl

Q. What do I need to start using mosaic.pl?
A. A set of images with their full path name listed in a file

Q. Are there any restrictions on the size of the mosaic due to the projection approximations?
A. The projection used here is exact, it involves no approximations. The only requirement here is that the final mosaic can not span more than one hemisphere.

Q. What kinds of images can be processed?
A. Any images in the FITS format, but the following keywords are required:
BITPIX, NAXIS, NAXIS1, NAXIS2, CRVAL1, CRVAL2, CRPIX1, CRPIX2, CTYPE1, CTYPE2.
Either the CD matrix or CDELT1, CDELT2, CROTA2 keywords are also required.

Q. How fast is the processing?
A. The slowest step is image interpolation. On a 500 MHz Sunblade with 512 MB of memory, it takes ~20s to process a 256x256 image.

Q. Are there any restrictions on the mosaic pixel size?
A. There are no restrictions on the mosaic pixel size. If the mosaic pixel size is so small that you don't have enough computer memory to coadd all the interpolated images at once, you should split the mosaic into several tiles. In order to do so, a suggested tile size is specified in the namelist:

&MOSAICCOADDIN
TILEMAX_X = 1000,
TILEMAX_Y = 1000,
&END

Q. Can I do drizzle interpolation?
A. Yes. The drizzle option is set in the namelist:

&MOSAICINTIN
# INTERP_METHOD = 1(default), 2(Drizzle), 3(Grid),
INTERP_METHOD = 2,
DRIZ_FAC = 0.8;
&END

Q. When do I use the GRID option for image interpolation?
A. GRID should be used when a quick-look mosaic is needed. It can also be used for background matching (overlap.pl). The gain in speed is up to 10 times as fast as scheme one, without much loss in the quality of the interpolated image. The price you pay is the fidelity of the interpolated images. It should not be used for photometry purposes. It should also not be used while doing outlier rejection. Don't forget to set GRID_RATIO in the namelist.

Q. Which modules should be turned on in the namelist for various outlier rejection options?
A. For single frame outlier rejection, the following should be set to 1:
run_medfilter = 1
run_detect_radhit = 1

For multiframe outlier rejection the following should be set to 1:
run_mosaic_outlier = 1
USE_OUTLIER_FOR_RMASK = 1

For dual outlier rejection the following should be set to 1:
run_medfilter = 1
run_detect_outlier = 1
run_mosaic_proj = 1
run_mosaic_dual_outlier = 1
run_level = 1
USE_DUAL_OUTLIER_FOR_RMASK = 1

For multiframe box outlier rejection, the following should be set to 1: run_mosaic_box_outlier = 1
USE_BOX_OUTLIER_FOR_RMASK = 1

Interpolated images are required for multiframe and dual outlier rejection. The following module should be run once before running mosaic_rmask:
run_mosaic_covg = 1

Q. How do I set the pixel size of the mosaic?
A. The can set the pixel size directly using the following keywords in the namelist:
MOSAIC_PIXEL_SIZE_X = -0.0003
MOSAIC_PIXEL_SIZE_Y = 0.0003

MOSAIC_PIXEL_SIZE_X is negative since CDELT1 is negative. Alternatively, you can set the ratio of the input image pixel size to the mosaic pixel size using the following keywords in the namelist:
MOSAIC_PIXEL_RATIO_X = 2
MOSAIC_PIXEL_RATIO_Y = 2

Q. I'm confused. In some places it says that I should use THRESH_OPTION = 2 for the mosaic_outlier sigma computation option, because it is more aggressive and that THRESH_OPTION = 1 doesn't work. Yet, in some other places it says that I should use THRESH_OPTION = 1 because it is the robust option. Which one is it?
A. Both statements are true, but they apply to different versions of MOPEX. For older versions, up to and including mopex_093005 you should use TRESH_OPTION = 2, as the only option that actually detects outliers. However, this option is too aggressive, and so for all subsequent versions of MOPEX, THRESH_OPTION = 1 was modified to use the median and median absolute deviation. This results in more reliable outlier detection with fewer false alarms. This option should be used for all versions from mopex_030106 onwards.

Q. I'm getting the following error message when I run mosaic.pl:

Pipeline Module MOSAIC_COADD Version 2.5
Processing time Tue Jan 20 18:11:26 2004
nmings 10
Error in allocating memory for Coverage Image for Tile 0
mosaic_coadd Error Message 0x0071: MALLOC_FAILED.

A. This is an indication that you are out of RAM. The chances are that you are trying to make a very big mosaic. First, make sure that this is what you actually want to do. Check the keywords NAXIS1, NAXIS2 in the fiducial image frame fif.tbl. It is possible that a huge fiducial image frame was created because one or more images in the set have bad pointing. Verify that all the images have reasonable pointing. (headerlist.tbl). If you believe that the problem is real, i.e. you are actually trying to make a mosaic as big as it is in fif.tbl, then use the tiling capabilities by specifying the tile size for the appropriate module in the namelist. For coaddition (mosaic_coadd) and mosaic_covg the keywords are:
TILEMAX_X = 1000,
TILEMAX_Y = 1000,

For multiframe (mosaic_outlier) and dual outlier rejection (mosaic_dual_outlier) the keywords are:
TILE_XSIZ = 500,
TILE_YSIZ = 500,

Q. I have a list of images covering a big area on the sky, but I only want to male a mosaic covering a small portion of that area. How can I do it?
A. First, create the fiducial image frame based on the full list. In order to do so, run mosaic.pl with only one step set to 1 in the namelist:
run_fiducial_image_frame = 1

This will create FIF.tbl. Now, set run_fiducial_image_frame = 0. Modify FIF.tbl to specify the reduced field, then run mosaic.pl with the following setting:
run_mosaic_geom = 1

Specify all the pertinent input lists in the namelists. For each input list there will be a corresponding reduced list created in the output directory (OUTPUT_DIR). The lists will contain only the images that have some overlap with the image frame specified in the modified FIF.tbl. E.g. if you have the following setting in your namelist:
OUTPUT_DIR = mosaic
IMAGE_STACK_FILE_NAME = data/MIPS70_ImageList.txt
DCE_STATUS_MASK_LIST = DmaskList.txt

After you run mosaic.pl there will be two lists created:
mosaic/geom_MIPS70_ImageList.txt
mosaic/geom_DmaskList.txt

Set run_mosaic_geom = 0. Now use the reduced lists in the namelist:
IMAGE_STACK_FILE_NAME = mosaic/geom_MIPS70_ImageList.txt
DCE_STATUS_MASK_LIST = mosaic/geom_DmaskList.txt

and proceed with mosaicing.
The program uses a namelist parameter MARGIN. The parameter gives a margin in terms of the number of pixels around the FIF. Normally, one would want to set to a negative number to avoid including input images that only have a marginal overlap with the FIF.

Q. After I have run overlap.pl the mosaic doesn't have the patchiness, but it now displays an overall gradient. Is it real or was it introduced by overlap.pl?
A. The chances are that it's neither of the above. Most likely the overall gradient in the mosaic image comes from the gradients in the input images, which were not properly flat-fielded. Perl script flatfield.pl, included in the MOPEX distribution, should take care of this problem.

Q. I have run medfilter to create background-subtracted images for overlap.pl. It took me forever because I used a big window size. I don't want to do it again for mosaic.pl and apex.pl. Is there an easy way to re-use those background-subtracted images?
A. Yes. If a subdirectory name is specified as a full path, it is not treated as a subdirectory of OUTPUT_DIR and the name is used as specified. All you have to do is specify MEDFILTER_DIR to point to the location where the background-subtracted images created by overlap.pl were saved. For example, let's say that in the overlap namelist you have:
OUTPUT_DIR = output_overlap
MEDFILTER_DIR = Medfilter

and you are running overlap.pl and mosaic.pl in the same directory /home/NGC0001/.

In order to re-use the background-subtracted images, you would write in the mosaic namelist:
MEDFILTER_DIR = /home/NGC0001/output_overlap/Medfilter

Q. The MOPEX module prf_estimate is failing with a Perl error, or with an error about incorrect "flux" columns in a PRF file. What's wrong?
A. The Perl error is most likely a memory limit because you are trying to derive a PRF from too many stars x too many frames x too big a fitting area x too fine sampling. For example, if you have a big fitting area, you could reduce this by lowering PostStamp_Xsize and PostStamp_Ysize.

The second error is due to MOPEX having trouble with the flux column in the input table. This is a bug that remains to be fixed. You can get around it by having a table with just RA and Dec, or RA and Dec and X and Y. It will then determine an approximate flux for each PRF star for its fitting.

Q. I have made a mosaic using mosaic.pl. Do I have to redo it when I run apex.pl?
A. No, you don't have to redo mosaicing. The script apex.pl can use the tiles created by mosaic.pl and saved in the coadd directory (COADDER_DIR). If the products of mosaic.pl are to be used by apex.pl, then the following option should be used in the namelist used by mosaic.pl:
keep_coadded_tiles = 1

Then set OUTPUT_DIR to be the same in both the apex.pl and mosaic.pl namelists. The run_mosaic_int and run_mosaic_coadd steps can and should be skipped (set to 0). This is especially relevant if outlier rejection is performed as part of mosaicking, since apex.pl does not do outlier rejection.

Q. The detection list - mosaic_detect.tbl - has no detections.
A. Parameter Detection_Threshold is set too high. Try lowering it.

Q. I don't want to do passive deblending. Is there a way to turn it off?
A. The way to avoid doing passive deblending is to remove the deblend_size and deblend_id columns. To do so, set the following in the namelist:
run_select_detect = 1

and add the list of columns to copy:
select_detect_columns = "srcid,x,y"

The perl script, select, will create a new detection table without the deblend_size and deblend_id colummns, and passive deblending will not be performed.

Q. The values of chi2/dof's in the extraction table - extract.tbl - are way too high.
A. Check the uncertainty images - they could be underestimated.

Q. All of the chi2/dof's in the extraction table - extract.tbl - are ~0.1 or lower.
A. Check the uncertainty images - they could be overestimated.

Q. Can I do point source extraction on the mosaiced image?
A. Yes. First you should produce a mosaic with the corresponding uncertainty image. Then you run apex_1frame.pl setting INPUT_IMAGE_FILE = <mosaic_image_file_name> and SIGMA_IMAGE_FILE = <mosaic_uncertainty_image_file_name>. Use the appropriate PRF file.

Q. I only want to do aperture photometry for a given set of sources in the sky. Is there an easy way to do that without using apex.pl or apex_1frame.pl?
A. Yes. You can run the module aperture on the command line with the appropriate setting of the parameters on the command line or in the namelist. See the aperture manual (pdf format) for more details. There you will also find an explanation of how to measure aperture photometry for a single point in the sky in a set of images.

Q. My apex.pl run crashed with the following obscure message:

Perl system() return code = 6
sourcestimate ran with signal 6
System Exit Code ( sourcestimate): 6

What's going on?

A. Chances are you ran out of RAM because of big blends of detections in the detection table. The program sourcestimate performs simultaneous fitting of all the detections in the detection table with the same value of the blend_id column (passive deblending). In order to bypass this problem, you need to clean the detection table by removing huge "blends" of detections. You can do it by inserting the follwing two lines in the namelist:
run_select_detect = 1
select_detect_conditions = "blend_size < 100"

The actual value for the blend_size limit may vary with the data set. In order to find the value applicable for your data you can run select directly on the detect table, by typing the following on the command line:
select from mosaic_detect.tbl for "blend_size > 100"

select will print out the number of rows with blend_size > 100. If there are none then you can decrease the limit.