Spec2Pipeline for LRS-SLIT (MIRI LRS-SLIT CALSPEC2) for point source¶
Spec2Pipeline applies additional instrumental corrections and calibrations to count rate products that result in a fully calibrated individual exposure.
Official documentation for Spec2Pipeline can be found here:
https://jwst-pipeline.readthedocs.io/en/latest/jwst/pipeline/calwebb_spec2.html
The Spec2Pipeline for LRS-SLIT comprises a linear series of steps. The steps in order are:
Step |
Description |
|---|---|
|
attach WCS object to each science exposure |
|
subtract dedicated background exposures |
|
if APT source type set to ‘UNKNOWN’, sets the ‘SRCTYPE’ keyword to ‘EXTENDED’ for MRS |
|
correct the exposure using the flat-field reference file |
|
apply flux (photometric) calibration |
|
extract 1D spectrum from 3D cube |
Input data¶
An example of running the file through the Spec2Pipeline is now shown using a simple simulated fullband, 2-point dither observation of a point source (power law plus emission lines spectrum) with the MIRI LRS-SLIT produced with MIRISim v2.3 and brought to level 2A using the Detector1Pipeline.
Imports¶
# imports
import os, glob, shutil
import numpy as np
from matplotlib.colors import LogNorm
import matplotlib.pyplot as plt
from subprocess import call
from jwst import datamodels
from jwst.associations.lib.member import Member
from jwst.associations.asn_from_list import asn_from_list
from jwst.associations.lib.rules_level2_base import DMSLevel2bBase
# set the CRDS_CONTEXT
os.environ["CRDS_CONTEXT"] = "jwst_0641.pmap"
Spec2Pipeline can be run by passing indivual exposures or datamodels but in this sample case we produce an association file so all can be run through Spec2Pipeline in a single call. It is assumed that the science file are in ‘LRS-SLIT_point_source/science’ and the background file is in ‘LRS-SLIT_point_source/background’.
# load the science and background files to lists
my_science_files = glob.glob('LRS-SLIT_point_source/science/*rate.fits')
my_background_files = glob.glob('LRS-SLIT_point_source/background/*rate.fits')
my_science_files = sorted(my_science_files)
# setup an empty level 2 association structure
asn = asn_from_list(my_science_files, rule=DMSLevel2bBase)
asn.data['products'] = None
# set the association name
asn_name = 'my_point_source'
# set some metadata
asn['asn_pool'] = asn_name + '_pool'
asn['asn_type'] = 'spec2'
for n, sci in enumerate(my_science_files):
asn.new_product('{}_dither{}'.format(asn_name, str(n+1)))
sci_member = Member({'expname': sci, 'exptype': 'science'})
new_members = asn.current_product['members']
new_members.append(sci_member)
for bkg in my_background_files:
bkg_member = Member({'expname': bkg, 'exptype': 'background'})
new_members.append(bkg_member)
# print the association and save to file
name, ser = asn.dump()
print(ser)
asn_file = asn_name + '_lvl2_asn.json'
with open(asn_file, 'w') as f:
f.write(ser)
{
"asn_type": "spec2",
"asn_rule": "DMSLevel2bBase",
"version_id": null,
"code_version": "0.17.1",
"degraded_status": "No known degraded exposures in association.",
"program": "noprogram",
"constraints": "No constraints",
"asn_id": "a3001",
"asn_pool": "my_point_source_pool",
"products": [
{
"name": "my_point_source_dither1",
"members": [
{
"expname": "LRS-SLIT_point_source/science/det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits",
"exptype": "science"
},
{
"expname": "LRS-SLIT_point_source/background/det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits",
"exptype": "background"
}
]
}
]
}
Import Spec2Pipeline and print the docstring to show some information
from jwst.pipeline import Spec2Pipeline
print(Spec2Pipeline.__doc__)
Spec2Pipeline: Processes JWST spectroscopic exposures from Level 2a to 2b.
Accepts a single exposure or an association as input.
Included steps are:
assign_wcs, background subtraction, NIRSpec MSA imprint subtraction,
NIRSpec MSA bad shutter flagging, 2-D subwindow extraction, flat field,
source type decision, straylight, fringe, pathloss, barshadow, photom,
resample_spec, cube_build, and extract_1d.
The association file is passed as input. The output level 2B files will be saved in my_output_dir as _cal.fits. The cubes and 1D spectra will also be saved.
Parameters used:
save_results: boolean, optional, default=False
save the results to file
output_dir : boolean, optional, default is the working directory
the location to save the output
steps : dict, optional, default None
pass parameters to steps
The output filenames will be determined by the asn_name in the association file which was set in the cell above.
# user specified
my_output_dir = 'LRS-SLIT_science'
# the output directory should be created if it doesn't exist
if not os.path.exists(my_output_dir):
os.mkdir(my_output_dir)
# run the pipeline
Spec2Pipeline.call(asn_file, save_results=True, output_dir=my_output_dir)
2020-10-16 11:44:51,049 - CRDS - ERROR - Error determining best reference for 'pars-backgroundstep' = Unknown reference type 'pars-backgroundstep'
2020-10-16 11:44:51,095 - CRDS - ERROR - Error determining best reference for 'pars-assignwcsstep' = Unknown reference type 'pars-assignwcsstep'
2020-10-16 11:44:51,143 - CRDS - ERROR - Error determining best reference for 'pars-imprintstep' = Unknown reference type 'pars-imprintstep'
2020-10-16 11:44:51,190 - CRDS - ERROR - Error determining best reference for 'pars-msaflagopenstep' = Unknown reference type 'pars-msaflagopenstep'
2020-10-16 11:44:51,236 - CRDS - ERROR - Error determining best reference for 'pars-extract2dstep' = Unknown reference type 'pars-extract2dstep'
2020-10-16 11:44:51,287 - CRDS - ERROR - Error determining best reference for 'pars-flatfieldstep' = Unknown reference type 'pars-flatfieldstep'
2020-10-16 11:44:51,333 - CRDS - ERROR - Error determining best reference for 'pars-pathlossstep' = Unknown reference type 'pars-pathlossstep'
2020-10-16 11:44:51,381 - CRDS - ERROR - Error determining best reference for 'pars-barshadowstep' = Unknown reference type 'pars-barshadowstep'
2020-10-16 11:44:51,429 - CRDS - ERROR - Error determining best reference for 'pars-photomstep' = Unknown reference type 'pars-photomstep'
2020-10-16 11:44:51,474 - CRDS - ERROR - Error determining best reference for 'pars-masterbackgroundnrsslitsstep' = Unknown reference type 'pars-masterbackgroundnrsslitsstep'
2020-10-16 11:44:51,521 - CRDS - ERROR - Error determining best reference for 'pars-wavecorrstep' = Unknown reference type 'pars-wavecorrstep'
2020-10-16 11:44:51,568 - CRDS - ERROR - Error determining best reference for 'pars-flatfieldstep' = Unknown reference type 'pars-flatfieldstep'
2020-10-16 11:44:51,614 - CRDS - ERROR - Error determining best reference for 'pars-sourcetypestep' = Unknown reference type 'pars-sourcetypestep'
2020-10-16 11:44:51,661 - CRDS - ERROR - Error determining best reference for 'pars-straylightstep' = Unknown reference type 'pars-straylightstep'
2020-10-16 11:44:51,710 - CRDS - ERROR - Error determining best reference for 'pars-fringestep' = Unknown reference type 'pars-fringestep'
2020-10-16 11:44:51,755 - CRDS - ERROR - Error determining best reference for 'pars-pathlossstep' = Unknown reference type 'pars-pathlossstep'
2020-10-16 11:44:51,803 - CRDS - ERROR - Error determining best reference for 'pars-barshadowstep' = Unknown reference type 'pars-barshadowstep'
2020-10-16 11:44:51,850 - CRDS - ERROR - Error determining best reference for 'pars-photomstep' = Unknown reference type 'pars-photomstep'
2020-10-16 11:44:51,898 - CRDS - ERROR - Error determining best reference for 'pars-resamplespecstep' = Unknown reference type 'pars-resamplespecstep'
2020-10-16 11:44:51,946 - CRDS - ERROR - Error determining best reference for 'pars-cubebuildstep' = Unknown reference type 'pars-cubebuildstep'
2020-10-16 11:44:51,993 - CRDS - ERROR - Error determining best reference for 'pars-extract1dstep' = Unknown reference type 'pars-extract1dstep'
2020-10-16 11:44:52,037 - CRDS - ERROR - Error determining best reference for 'pars-spec2pipeline' = Unknown reference type 'pars-spec2pipeline'
2020-10-16 11:44:52,050 - stpipe.Spec2Pipeline - INFO - Spec2Pipeline instance created.
2020-10-16 11:44:52,052 - stpipe.Spec2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2020-10-16 11:44:52,053 - stpipe.Spec2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2020-10-16 11:44:52,055 - stpipe.Spec2Pipeline.imprint_subtract - INFO - ImprintStep instance created.
2020-10-16 11:44:52,056 - stpipe.Spec2Pipeline.msa_flagging - INFO - MSAFlagOpenStep instance created.
2020-10-16 11:44:52,058 - stpipe.Spec2Pipeline.extract_2d - INFO - Extract2dStep instance created.
2020-10-16 11:44:52,061 - stpipe.Spec2Pipeline.master_background - INFO - MasterBackgroundNrsSlitsStep instance created.
2020-10-16 11:44:52,063 - stpipe.Spec2Pipeline.master_background.flat_field - INFO - FlatFieldStep instance created.
2020-10-16 11:44:52,064 - stpipe.Spec2Pipeline.master_background.pathloss - INFO - PathLossStep instance created.
2020-10-16 11:44:52,065 - stpipe.Spec2Pipeline.master_background.barshadow - INFO - BarShadowStep instance created.
2020-10-16 11:44:52,068 - stpipe.Spec2Pipeline.master_background.photom - INFO - PhotomStep instance created.
2020-10-16 11:44:52,070 - stpipe.Spec2Pipeline.wavecorr - INFO - WavecorrStep instance created.
2020-10-16 11:44:52,073 - stpipe.Spec2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2020-10-16 11:44:52,074 - stpipe.Spec2Pipeline.srctype - INFO - SourceTypeStep instance created.
2020-10-16 11:44:52,076 - stpipe.Spec2Pipeline.straylight - INFO - StraylightStep instance created.
2020-10-16 11:44:52,078 - stpipe.Spec2Pipeline.fringe - INFO - FringeStep instance created.
2020-10-16 11:44:52,079 - stpipe.Spec2Pipeline.pathloss - INFO - PathLossStep instance created.
2020-10-16 11:44:52,081 - stpipe.Spec2Pipeline.barshadow - INFO - BarShadowStep instance created.
2020-10-16 11:44:52,083 - stpipe.Spec2Pipeline.photom - INFO - PhotomStep instance created.
2020-10-16 11:44:52,085 - stpipe.Spec2Pipeline.resample_spec - INFO - ResampleSpecStep instance created.
2020-10-16 11:44:52,088 - stpipe.Spec2Pipeline.cube_build - INFO - CubeBuildStep instance created.
2020-10-16 11:44:52,093 - stpipe.Spec2Pipeline.extract_1d - INFO - Extract1dStep instance created.
2020-10-16 11:44:52,171 - stpipe.Spec2Pipeline - INFO - Step Spec2Pipeline running with args ('my_point_source_lvl2_asn.json',).
2020-10-16 11:44:52,191 - stpipe.Spec2Pipeline - INFO - Step Spec2Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': 'LRS-SLIT_science', 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': True, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_bsub': False, 'fail_on_exception': True, 'steps': {'bkg_subtract': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sigma': 3.0, 'maxiters': None}, 'assign_wcs': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'slit_y_low': -0.55, 'slit_y_high': 0.55}, 'imprint_subtract': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'msa_flagging': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'extract_2d': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'slit_name': None, 'extract_orders': None, 'tsgrism_extract_height': None, 'wfss_extract_half_height': 5, 'grism_objects': None, 'mmag_extract': 99.0}, 'master_background': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': True, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'force_subtract': False, 'save_background': False, 'user_background': None, 'inverse': False, 'steps': {'flat_field': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_interpolated_flat': False, 'user_supplied_flat': None, 'inverse': False}, 'pathloss': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}, 'barshadow': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}, 'photom': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}}}, 'wavecorr': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'flat_field': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_interpolated_flat': False, 'user_supplied_flat': None, 'inverse': False}, 'srctype': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'straylight': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'method': 'ModShepard', 'roi': 50, 'power': 1.0}, 'fringe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'pathloss': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}, 'barshadow': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}, 'photom': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}, 'resample_spec': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'pixfrac': 1.0, 'kernel': 'square', 'fillval': 'INDEF', 'weight_type': 'exptime', 'single': False, 'blendheaders': True}, 'cube_build': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': True, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': False, 'input_dir': '', 'channel': 'all', 'band': 'all', 'grating': 'all', 'filter': 'all', 'output_type': 'band', 'scale1': 0.0, 'scale2': 0.0, 'scalew': 0.0, 'weighting': 'emsm', 'coord_system': 'skyalign', 'rois': 0.0, 'roiw': 0.0, 'weight_power': 0.0, 'wavemin': None, 'wavemax': None, 'single': False, 'xdebug': None, 'ydebug': None, 'zdebug': None, 'skip_dqflagging': False}, 'extract_1d': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'smoothing_length': None, 'bkg_order': None, 'log_increment': 50, 'subtract_background': None, 'use_source_posn': None, 'apply_apcorr': True}}}
2020-10-16 11:44:52,269 - stpipe.Spec2Pipeline - INFO - Prefetching reference files for dataset: 'det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits' reftypes = ['apcorr', 'area', 'barshadow', 'camera', 'collimator', 'cubepar', 'dflat', 'disperser', 'distortion', 'drizpars', 'extract1d', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'fringe', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'msaoper', 'ote', 'pathloss', 'photom', 'regions', 'resol', 'sflat', 'specwcs', 'wavecorr', 'wavelengthrange', 'wfssbkg']
2020-10-16 11:44:52,294 - CRDS - INFO - Fetching /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_apcorr_0007.fits 262.1 K bytes (1 / 5 files) (0 / 13.0 M bytes)
2020-10-16 11:44:53,860 - CRDS - INFO - Fetching /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_extract1d_0004.json 745 bytes (2 / 5 files) (262.1 K / 13.0 M bytes)
2020-10-16 11:44:54,319 - CRDS - INFO - Fetching /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_flat_0472.fits 12.7 M bytes (3 / 5 files) (262.8 K / 13.0 M bytes)
2020-10-16 11:44:57,468 - CRDS - INFO - Fetching /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_photom_0072.fits 14.4 K bytes (4 / 5 files) (13.0 M / 13.0 M bytes)
2020-10-16 11:44:58,351 - CRDS - INFO - Fetching /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_specwcs_0003.fits 46.1 K bytes (5 / 5 files) (13.0 M / 13.0 M bytes)
2020-10-16 11:44:59,018 - stpipe.Spec2Pipeline - INFO - Prefetch for APCORR reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_apcorr_0007.fits'.
2020-10-16 11:44:59,020 - stpipe.Spec2Pipeline - INFO - Prefetch for AREA reference file is 'N/A'.
2020-10-16 11:44:59,021 - stpipe.Spec2Pipeline - INFO - Prefetch for BARSHADOW reference file is 'N/A'.
2020-10-16 11:44:59,022 - stpipe.Spec2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2020-10-16 11:44:59,024 - stpipe.Spec2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2020-10-16 11:44:59,025 - stpipe.Spec2Pipeline - INFO - Prefetch for CUBEPAR reference file is 'N/A'.
2020-10-16 11:44:59,027 - stpipe.Spec2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2020-10-16 11:44:59,029 - stpipe.Spec2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2020-10-16 11:44:59,031 - stpipe.Spec2Pipeline - INFO - Prefetch for DISTORTION reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_distortion_0028.asdf'.
2020-10-16 11:44:59,033 - stpipe.Spec2Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_drizpars_0001.fits'.
2020-10-16 11:44:59,033 - stpipe.Spec2Pipeline - INFO - Prefetch for EXTRACT1D reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_extract1d_0004.json'.
2020-10-16 11:44:59,036 - stpipe.Spec2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2020-10-16 11:44:59,037 - stpipe.Spec2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is 'N/A'.
2020-10-16 11:44:59,038 - stpipe.Spec2Pipeline - INFO - Prefetch for FLAT reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_flat_0472.fits'.
2020-10-16 11:44:59,040 - stpipe.Spec2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2020-10-16 11:44:59,041 - stpipe.Spec2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2020-10-16 11:44:59,042 - stpipe.Spec2Pipeline - INFO - Prefetch for FRINGE reference file is 'N/A'.
2020-10-16 11:44:59,043 - stpipe.Spec2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2020-10-16 11:44:59,044 - stpipe.Spec2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2020-10-16 11:44:59,045 - stpipe.Spec2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2020-10-16 11:44:59,047 - stpipe.Spec2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2020-10-16 11:44:59,049 - stpipe.Spec2Pipeline - INFO - Prefetch for MSAOPER reference file is 'N/A'.
2020-10-16 11:44:59,050 - stpipe.Spec2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2020-10-16 11:44:59,051 - stpipe.Spec2Pipeline - INFO - Prefetch for PATHLOSS reference file is 'N/A'.
2020-10-16 11:44:59,052 - stpipe.Spec2Pipeline - INFO - Prefetch for PHOTOM reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_photom_0072.fits'.
2020-10-16 11:44:59,053 - stpipe.Spec2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2020-10-16 11:44:59,055 - stpipe.Spec2Pipeline - INFO - Prefetch for RESOL reference file is 'N/A'.
2020-10-16 11:44:59,057 - stpipe.Spec2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2020-10-16 11:44:59,058 - stpipe.Spec2Pipeline - INFO - Prefetch for SPECWCS reference file is '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_specwcs_0003.fits'.
2020-10-16 11:44:59,059 - stpipe.Spec2Pipeline - INFO - Prefetch for WAVECORR reference file is 'N/A'.
2020-10-16 11:44:59,060 - stpipe.Spec2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2020-10-16 11:44:59,061 - stpipe.Spec2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2020-10-16 11:44:59,062 - stpipe.Spec2Pipeline - INFO - Starting calwebb_spec2 ...
2020-10-16 11:44:59,081 - stpipe.Spec2Pipeline - INFO - Processing product my_point_source_dither1
2020-10-16 11:44:59,082 - stpipe.Spec2Pipeline - INFO - Working on input LRS-SLIT_point_source/science/det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits ...
2020-10-16 11:44:59,276 - stpipe.Spec2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:44:59,277 - stpipe.Spec2Pipeline.assign_wcs - INFO - Step assign_wcs parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2020-10-16 11:45:00,045 - stpipe.Spec2Pipeline.assign_wcs - INFO - Created a MIRI mir_lrs-fixedslit pipeline with references {'distortion': '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_distortion_0028.asdf', 'filteroffset': None, 'specwcs': '/Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_specwcs_0003.fits', 'regions': None, 'wavelengthrange': None, 'camera': None, 'collimator': None, 'disperser': None, 'fore': None, 'fpa': None, 'msa': None, 'ote': None, 'ifupost': None, 'ifufore': None, 'ifuslicer': None}
2020-10-16 11:45:00,136 - stpipe.Spec2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS 0.000379612 -0.000035778 0.000379612 0.000080412 359.999082432 0.000080412 359.999082432 -0.000035778
2020-10-16 11:45:00,137 - stpipe.Spec2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS 0.000379612 -0.000035778 0.000379612 0.000080412 359.999082432 0.000080412 359.999082432 -0.000035778
2020-10-16 11:45:00,138 - stpipe.Spec2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2020-10-16 11:45:00,144 - stpipe.Spec2Pipeline.assign_wcs - INFO - Step assign_wcs done
2020-10-16 11:45:00,254 - stpipe.Spec2Pipeline.bkg_subtract - INFO - Step bkg_subtract running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>, ['LRS-SLIT_point_source/background/det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits']).
2020-10-16 11:45:00,258 - stpipe.Spec2Pipeline.bkg_subtract - INFO - Step bkg_subtract parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'bsub', 'search_output_file': True, 'input_dir': '', 'sigma': 3.0, 'maxiters': None}
2020-10-16 11:45:00,877 - stpipe.Spec2Pipeline.bkg_subtract - INFO - Step bkg_subtract done
2020-10-16 11:45:00,949 - stpipe.Spec2Pipeline.imprint_subtract - INFO - Step imprint_subtract running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>, []).
2020-10-16 11:45:00,950 - stpipe.Spec2Pipeline.imprint_subtract - INFO - Step imprint_subtract parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2020-10-16 11:45:00,951 - stpipe.Spec2Pipeline.imprint_subtract - INFO - Step skipped.
2020-10-16 11:45:00,952 - stpipe.Spec2Pipeline.imprint_subtract - INFO - Step imprint_subtract done
2020-10-16 11:45:01,008 - stpipe.Spec2Pipeline.msa_flagging - INFO - Step msa_flagging running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,010 - stpipe.Spec2Pipeline.msa_flagging - INFO - Step msa_flagging parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2020-10-16 11:45:01,010 - stpipe.Spec2Pipeline.msa_flagging - INFO - Step skipped.
2020-10-16 11:45:01,012 - stpipe.Spec2Pipeline.msa_flagging - INFO - Step msa_flagging done
2020-10-16 11:45:01,070 - stpipe.Spec2Pipeline.srctype - INFO - Step srctype running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,072 - stpipe.Spec2Pipeline.srctype - INFO - Step srctype parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2020-10-16 11:45:01,080 - stpipe.Spec2Pipeline.srctype - INFO - Input EXP_TYPE is MIR_LRS-FIXEDSLIT
2020-10-16 11:45:01,081 - stpipe.Spec2Pipeline.srctype - INFO - Input SRCTYAPT = None
2020-10-16 11:45:01,081 - stpipe.Spec2Pipeline.srctype - WARNING - SRCTYAPT keyword not found in input; using SRCTYPE instead
2020-10-16 11:45:01,082 - stpipe.Spec2Pipeline.srctype - INFO - Input source type is unknown; setting default SRCTYPE = POINT
2020-10-16 11:45:01,084 - stpipe.Spec2Pipeline.srctype - INFO - Step srctype done
2020-10-16 11:45:01,140 - stpipe.Spec2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,141 - stpipe.Spec2Pipeline.flat_field - INFO - Step flat_field parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_interpolated_flat': False, 'user_supplied_flat': None, 'inverse': False}
2020-10-16 11:45:01,231 - stpipe.Spec2Pipeline.flat_field - WARNING - Keyword CDP_PARTIAL_DATA does not correspond to an existing DQ mnemonic, so will be ignored
2020-10-16 11:45:01,232 - stpipe.Spec2Pipeline.flat_field - WARNING - Keyword CDP_LOW_QUAL does not correspond to an existing DQ mnemonic, so will be ignored
2020-10-16 11:45:01,232 - stpipe.Spec2Pipeline.flat_field - WARNING - Keyword CDP_UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2020-10-16 11:45:01,254 - stpipe.Spec2Pipeline.flat_field - WARNING - Keyword DIFF_PATTERN does not correspond to an existing DQ mnemonic, so will be ignored
2020-10-16 11:45:01,400 - stpipe.Spec2Pipeline.flat_field - INFO - Step flat_field done
2020-10-16 11:45:01,454 - stpipe.Spec2Pipeline.straylight - INFO - Step straylight running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,456 - stpipe.Spec2Pipeline.straylight - INFO - Step straylight parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'method': 'ModShepard', 'roi': 50, 'power': 1.0}
2020-10-16 11:45:01,457 - stpipe.Spec2Pipeline.straylight - INFO - Step skipped.
2020-10-16 11:45:01,458 - stpipe.Spec2Pipeline.straylight - INFO - Step straylight done
2020-10-16 11:45:01,520 - stpipe.Spec2Pipeline.fringe - INFO - Step fringe running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,521 - stpipe.Spec2Pipeline.fringe - INFO - Step fringe parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2020-10-16 11:45:01,522 - stpipe.Spec2Pipeline.fringe - INFO - Step skipped.
2020-10-16 11:45:01,523 - stpipe.Spec2Pipeline.fringe - INFO - Step fringe done
2020-10-16 11:45:01,577 - stpipe.Spec2Pipeline.pathloss - INFO - Step pathloss running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,578 - stpipe.Spec2Pipeline.pathloss - INFO - Step pathloss parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}
2020-10-16 11:45:01,579 - stpipe.Spec2Pipeline.pathloss - INFO - Step skipped.
2020-10-16 11:45:01,580 - stpipe.Spec2Pipeline.pathloss - INFO - Step pathloss done
2020-10-16 11:45:01,632 - stpipe.Spec2Pipeline.barshadow - INFO - Step barshadow running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,633 - stpipe.Spec2Pipeline.barshadow - INFO - Step barshadow parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}
2020-10-16 11:45:01,634 - stpipe.Spec2Pipeline.barshadow - INFO - Step skipped.
2020-10-16 11:45:01,635 - stpipe.Spec2Pipeline.barshadow - INFO - Step barshadow done
2020-10-16 11:45:01,685 - stpipe.Spec2Pipeline.photom - INFO - Step photom running with args (<ImageModel(1024, 1032) from det_image_seq1_MIRIMAGE_P750Lexp1_rate.fits>,).
2020-10-16 11:45:01,686 - stpipe.Spec2Pipeline.photom - INFO - Step photom parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}
2020-10-16 11:45:01,697 - stpipe.Spec2Pipeline.photom - INFO - Using photom reference file: /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_photom_0072.fits
2020-10-16 11:45:01,698 - stpipe.Spec2Pipeline.photom - INFO - Using area reference file: N/A
2020-10-16 11:45:01,793 - stpipe.Spec2Pipeline.photom - INFO - Using instrument: MIRI
2020-10-16 11:45:01,794 - stpipe.Spec2Pipeline.photom - INFO - detector: MIRIMAGE
2020-10-16 11:45:01,794 - stpipe.Spec2Pipeline.photom - INFO - exp_type: MIR_LRS-FIXEDSLIT
2020-10-16 11:45:01,795 - stpipe.Spec2Pipeline.photom - INFO - filter: P750L
2020-10-16 11:45:01,825 - stpipe.Spec2Pipeline.photom - INFO - subarray: FULL
2020-10-16 11:45:01,826 - stpipe.Spec2Pipeline.photom - INFO - PHOTMJSR value: 60.567
2020-10-16 11:45:01,929 - stpipe.Spec2Pipeline.photom - INFO - Step photom done
2020-10-16 11:45:01,994 - stpipe.Spec2Pipeline.resample_spec - INFO - Step resample_spec running with args (<ImageModel(1024, 1032) from LRS-SLIT_science/my_point_source_dither1_cal.fits>,).
2020-10-16 11:45:01,996 - stpipe.Spec2Pipeline.resample_spec - INFO - Step resample_spec parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': True, 'skip': False, 'suffix': 's2d', 'search_output_file': True, 'input_dir': '', 'pixfrac': 1.0, 'kernel': 'square', 'fillval': 'INDEF', 'weight_type': 'exptime', 'single': False, 'blendheaders': True}
2020-10-16 11:45:02,104 - stpipe.Spec2Pipeline.resample_spec - INFO - Drizpars reference file: /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_drizpars_0001.fits
2020-10-16 11:45:02,141 - stpipe.Spec2Pipeline.resample_spec - WARNING - /Users/patrickkavanagh/anaconda3/anaconda3/envs/jwst7.6/lib/python3.8/site-packages/numpy/lib/nanfunctions.py:1113: RuntimeWarning: All-NaN slice encountered
r, k = function_base._ureduce(a, func=_nanmedian, axis=axis, out=out,
2020-10-16 11:45:02,249 - stpipe.Spec2Pipeline.resample_spec - INFO - Resampling slit None (387, 44)
2020-10-16 11:45:02,918 - stpipe.Spec2Pipeline.resample_spec - INFO - Drizzling (1024, 1032) --> (387, 44)
2020-10-16 11:45:02,958 - stpipe.Spec2Pipeline.resample_spec - INFO - Update S_REGION to POLYGON ICRS 179.999087909 -0.000075806 180.000404817 -0.000075806 180.000404817 0.000033658 179.999087909 0.000033658
2020-10-16 11:45:03,112 - stpipe.Spec2Pipeline.resample_spec - INFO - Saved model in LRS-SLIT_science/my_point_source_dither1_s2d.fits
2020-10-16 11:45:03,113 - stpipe.Spec2Pipeline.resample_spec - INFO - Step resample_spec done
2020-10-16 11:45:03,175 - stpipe.Spec2Pipeline.extract_1d - INFO - Step extract_1d running with args (<SlitModel(387, 44) from my_point_source_dither1_s2d.fits>,).
2020-10-16 11:45:03,177 - stpipe.Spec2Pipeline.extract_1d - INFO - Step extract_1d parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': True, 'skip': False, 'suffix': 'x1d', 'search_output_file': True, 'input_dir': '', 'smoothing_length': None, 'bkg_order': None, 'log_increment': 50, 'subtract_background': None, 'use_source_posn': None, 'apply_apcorr': True}
2020-10-16 11:45:03,205 - stpipe.Spec2Pipeline.extract_1d - INFO - Using EXTRACT1D reference file /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_extract1d_0004.json
2020-10-16 11:45:03,207 - stpipe.Spec2Pipeline.extract_1d - INFO - Using APCORR file /Users/patrickkavanagh/crds_mirror/references/jwst/miri/jwst_miri_apcorr_0007.fits
2020-10-16 11:45:03,288 - stpipe.Spec2Pipeline.extract_1d - INFO - Turning on source position correction for exp_type = MIR_LRS-FIXEDSLIT
2020-10-16 11:45:03,289 - stpipe.Spec2Pipeline.extract_1d - WARNING - spectral_order is None; using 1
2020-10-16 11:45:03,302 - stpipe.Spec2Pipeline.extract_1d - INFO - Applying position offset of -8.28 to xstart and xstop
2020-10-16 11:45:03,303 - stpipe.Spec2Pipeline.extract_1d - INFO - Using extraction limits: xstart=8.217715601436794, xstop=18.217715601436794, ystart=0, ystop=386
2020-10-16 11:45:03,367 - stpipe.Spec2Pipeline.extract_1d - INFO - Applying Aperture correction.
2020-10-16 11:45:03,646 - stpipe.Spec2Pipeline.extract_1d - INFO - Saved model in LRS-SLIT_science/my_point_source_dither1_x1d.fits
2020-10-16 11:45:03,647 - stpipe.Spec2Pipeline.extract_1d - INFO - Step extract_1d done
2020-10-16 11:45:03,648 - stpipe.Spec2Pipeline - INFO - Finished processing product my_point_source_dither1
2020-10-16 11:45:03,648 - stpipe.Spec2Pipeline - INFO - Ending calwebb_spec2
2020-10-16 11:45:03,931 - stpipe.Spec2Pipeline - INFO - Saved model in LRS-SLIT_science/my_point_source_dither1_cal.fits
2020-10-16 11:45:03,931 - stpipe.Spec2Pipeline - INFO - Step Spec2Pipeline done
[<ImageModel(1024, 1032) from my_point_source_dither1_cal.fits>]
We can plot the extracted spectrum using the x1d.fits file in the output directory
# read the file to datamodel
spec_file = glob.glob(os.path.join(my_output_dir,"*x1d.fits"))[0]
dm = datamodels.open(spec_file)
fig, axs = plt.subplots(1, 1, figsize=(12, 7))
# plot the spectrum
axs.plot(dm.spec[0].spec_table['WAVELENGTH'][1:-1], dm.spec[0].spec_table['FLUX'][1:-1], c='b', marker='.',
markersize=0, linestyle='-', linewidth=2)
axs.set_ylabel(r'Flux (Jy)')
axs.set_xlabel(r'Wavelength ($\mu$m)')
axs.set_xlim(5.0,12.0)
axs.set_ylim(0.005,0.035)
plt.grid()
plt.show()
2020-10-16 11:53:54,303 - stpipe - WARNING - /Users/patrickkavanagh/anaconda3/anaconda3/envs/jwst7.6/lib/python3.8/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.
and should_run_async(code)
