Full Command Documention¶
Documented below are the primary commands used to run the BOSS Data Reduction Pipeline. However, there are numerous other routines included in this package, which are called by these commands and have their own internal documentation.
Full Bash and Python Command Usage¶
BOSS_log¶
usage: BOSS_log [-h] [-m MJD] [-y] [-o {apo,lco}] [-l] [--new_ref] [-c] [-r]
[-e] [-s]
Build BOSS Exposure Log
options:
-h, --help show this help message and exit
-m MJD, --mjd MJD MJD
-y, --yesterday current mjd-1
-o {apo,lco}, --observatory {apo,lco}, --obs {apo,lco}
Manually set observatory
-l, --long Long/detailed version of log
--new_ref Calculate new reference values in fratio and w_shift
and show in place of fratio and w_shift (edit to code
to save new value is required)
-c, --hide_hart, --hide_hartmann
Hide cleaned version of Hartmann Logs as a table
-r, --hart_raw Print raw form (instead of table form) of Hartmann
Logs
-e, --hide_error Hide SOS Error and Workings
-s, --hide_summary Hide data summary table
SOS¶
usage: SOS [-h] (-r | -b | -j) (-c | -t | -d) [--unlock] [-e EXP]
[-m [MJD ...]] [--nodb] [--no_gz] [--no_reject]
[--clobber_fibermap] [--sdssv_sn2] [--no_sn2_15] [--bright] [-n]
[-o] [-v]
SOS process for reducing BOSS data on the Moutain
options:
-h, --help show this help message and exit
-r, --red Red Camera Process
-b, --blue Blue Camera Process
-j, --joint Both Camera Processes
-c, --catchup Run Catchup on the night or (MJD)
-t, --redoMode Save outputs of MJD or exposure to sosredo
-d, --test Save outputs and logs to sosredo/dev
--unlock Unlock Locked Files
-e EXP, --exp EXP exposure id (or range of exp id 500-510) (with or
without leading zeros)
-m [MJD ...], --mjd [MJD ...]
MJD
--nodb skip opsdb load
--no_gz Overrides the requirement for '.gz' compressed files
(experimental)
--no_reject Overrides the Calibration rejection (use with caution)
--clobber_fibermap, -f
Clobbers the existing spfibermap files
--sdssv_sn2 Report a second set of SN2 values with updated fit
parameters
--no_sn2_15 Skip reporting a set of SN2 values with a fiducial mag
of 15 for engineering fields
--bright Display BOSS_only Bright Time Operation SN2_15 for all
fields
-n, --no_arc2trace Skip Utilizing arc2trace refinements
-o, --forcea2t Force arc2trace for all fields (even if flat exists
for field)
-v, --verbose prints the only (or red if joint) active SOS process
to terminal
boss_arcs_to_traces¶
Some IP addresses of users launching heavy query showers have temporarily been disabled. Please contact the Gaia helpdesk (https://www.cosmos.esa.int/web/gaia/gaia-helpdesk) for advice. Workaround solutions for the Gaia Archive issues following the infrastructure upgrade: https://www.cosmos.esa.int/web/gaia/news#WorkaroundArchive
usage: boss_arcs_to_traces MJD
Routine to transfer trace locations from an initial arc/flat pair given
subsequent arc frames only
options:
-h, --help show this help message and exit
--mjd MJD MJD to process
--outdir OUTDIR output directory
--obs OBS observatory (lco|apo)
--vers VERS BOSS_SPECTRO_REDUX version
--threads THREADS number of threads
--cams CAMS Supply the camera for operation with SOS files
--fitsname FITSNAME Supply the FitsName for SOS error reporting
--sosdir SOSDIR Base SOS output directory
--clobber clobber?
--no_hash Skip updating the file hash
build_combined_html¶
usage: build_combined_html [-h] [--force] sosdir
build SOS combine index page
positional arguments:
sosdir Base SOS output directory
options:
-h, --help show this help message and exit
--force Force update
clean_run¶
usage: clean_run [-h] --clean_type
{all,spec2d,comb,spec1d,post,merge,reformat,spcalib}
[--topdir TOPDIR] [--run1d RUN1D] [--run2d RUN2D] [--epoch]
[--reset] [--remove_redux] [--dry] [--verbose]
[--field FIELD] [--mjd MJD] [--fmjd [FMJD ...]]
Clean BOSS DRP Run
options:
-h, --help show this help message and exit
--clean_type {all,spec2d,comb,spec1d,post,merge,reformat,spcalib}, --clean {all,spec2d,comb,spec1d,post,merge,reformat,spcalib}
Pipeline Step to start the cleaning
--topdir TOPDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--run1d RUN1D Optional override value for the enviro variable $RUN1D
--run2d RUN2D Optional override value for the enviro variable $RUN2D
--epoch Clean up epoch run
--reset if clean_type == all, then remove plans and redux
--remove_redux if clean_type == all, then remove redux
--dry Print Files to be removed rather then remove
--verbose Print Files paths (with wildcards) to be removed
--field FIELD, -f FIELD
Run for a single Field
--mjd MJD, -m MJD Run for a single MJD
--fmjd [FMJD ...] List of Field-MJDs to clean
cleanup_backups¶
usage: cleanup_backups [-h] [--topdir TOPDIR] [--run2d RUN2D] [--epoch]
[--custom CUSTOM] [--backups BACKUPS]
Clean spAll backup files
options:
-h, --help show this help message and exit
--topdir TOPDIR Boss Spectro Redux base directory
--run2d RUN2D Run2d
--epoch run for the epoch coadds
--custom CUSTOM Name of custom Coadd
--backups BACKUPS Number of backups to keep
cronplot_QA.bash¶
Usage: cronplot_QA.bash module [options]
Description:
Load the correct module and execute the QA plotting script.
Options:
-l Use LCO observations (default is APO).
-c Include the --clobber_lists option.
-n Disable linking (default is False).
-e Include the --epoch option.
-w Generate HTML output (default is False).
-h Display this help message and exit.
Example:
cronplot_QA.bash myModule -l -c -n -e -w "test.html"
cronrun.bash¶
usage: cronrun.bash module 'script'
daily_log¶
usage: daily_log [-h] [--obs OBS [OBS ...]] [--mjd [MJD ...]]
[--mjdstart MJDSTART] [--mjdend MJDEND] [--epoch]
[--custom CUSTOM] [--topdir TOPDIR] [--run1d RUN1D]
[--run2d RUN2D] [--email] [--fast] [--refresh]
[--refresh_error] [--refresh_critical] [--force]
Build/load BOSS Pipeline Status Pages
options:
-h, --help show this help message and exit
--obs OBS [OBS ...] Observatory for status update
--mjd [MJD ...] Update these MJDs
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
--epoch Run for epoch Coadds
--custom CUSTOM Name of custom Coadd
--topdir TOPDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--run1d RUN1D Optional override value for the enviro variable $RUN1D
--run2d RUN2D Optional override value for the enviro variable $RUN2D
--email Send each mjd status as email
--fast Skip updating index until end
--refresh Refresh all the existing Status logs for obs
--refresh_error Refresh existing Status logs for obs with errors
--refresh_critical Refresh existing Status logs for obs with critical
errors
--force Refresh Summaries pages
field_directory¶
usage: field_directory [-h] [--run2d RUN2D] [--topdir TOPDIR] field
Prints the path to a Field Directory
positional arguments:
field FieldID
options:
-h, --help show this help message and exit
--run2d RUN2D idlspec2d Run2d version
--topdir TOPDIR idlspec2d Run2d topdir
fieldlist¶
usage: fieldlist [-h] [--create] [--topdir TOPDIR] [--run1d [RUN1D ...]]
[--run2d [RUN2D ...]] [--outdir OUTDIR]
[--skipcart [SKIPCART ...]] [--epoch] [--basehtml BASEHTML]
[--logfile LOGFILE] [--debug] [--noplot]
Build/load BOSS Fieldlist
options:
-h, --help show this help message and exit
--create, -c Create Fieldlist
--topdir TOPDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--run1d [RUN1D ...] Optional override value for the enviro variable $RUN1D
--run2d [RUN2D ...] Optional override value for the enviro variable $RUN2D
--outdir OUTDIR Optional output directory (defaults to topdir/$RUN2D)
--skipcart [SKIPCART ...]
Option list of cartridges to skip
--epoch Produce FieldList for epoch coadds
--basehtml BASEHTML html path for figure (defaults to relative from
topdir)
--logfile LOGFILE Manually Set logfile (including path)
--debug Overrides the logger of the simplified error messages
and prints standard python errors
--noplot Skips updating the sky plots
fieldmerge¶
usage: fieldmerge [-h] [--run2d RUN2D] [--indir INDIR] [--skip_line]
[--include_bad] [--legacy] [--skip_specprimary]
[--update_specprimary] [--lite] [--XCSAO] [--field FIELD]
[--mjd MJD] [--clobber] [--bkup] [--verbose]
[--logfile LOGFILE] [--epoch] [--programs [PROGRAMS ...]]
[--datamodel DATAMODEL] [--line_datamodel LINE_DATAMODEL]
[--outroot OUTROOT] [--remerge_fmjd [REMERGE_FMJD ...]]
[--remerge_mjd REMERGE_MJD] [--merge_only] [--allsky]
[--custom CUSTOM] [--run1d RUN1D] [--limit LIMIT]
[--ndays MJDSTART]
Build BOSS spAll Summary File
options:
-h, --help show this help message and exit
--run2d RUN2D Optional override value for the enviro variable $RUN2D
--indir INDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--skip_line skip the generation of spAllLine.fits
--include_bad include bad fields
--legacy Include columns used by SDSS-IV and depreciated in
SDSS-V
--skip_specprimary Skip creation of specprimary and associated columns
--update_specprimary Keep existing specprimary and associated columns and
only update new row (and their secondaries)
--lite Produce lite version of spAll file
--XCSAO Include XCSAO columns
--field FIELD, -f FIELD
Run for a single Field
--mjd MJD, -m MJD Run for a single MJD
--clobber Clobber all spAll-field-mjd files
--bkup Backup existing spAll files
--verbose Log columns not saved
--logfile LOGFILE Manually set logfile
--epoch Produce spAll for epoch coadds
--programs [PROGRAMS ...]
List of programs to include
--datamodel DATAMODEL
Supply a spAll datamodel file (defaults to
$IDLSPEC2D/datamodel/spall_dm.par
--line_datamodel LINE_DATAMODEL
Supply a spline datamodel file (defaults to
$IDLSPEC2D/datamodel/spzline_dm.par
--outroot OUTROOT Path and root of filename for output (defaults to
spectra/full or summary)
--remerge_fmjd [REMERGE_FMJD ...], -r [REMERGE_FMJD ...]
Field-MJD to replace in spAll
--remerge_mjd REMERGE_MJD
MJD to replace in spAll
--merge_only, -o Skip Building new spAll-Field-MJD files and just merge
existing
--allsky Build spAll for Allsky Custom Coadd
--custom CUSTOM Name of Custom Coadd
--run1d RUN1D Optional override value for the enviro variable $RUN1D
(only for custom allsky coadds)
--limit LIMIT Limit number of Field-MJD spAll files to read before
save
--ndays MJDSTART Limit update to last ndays
filecheck¶
usage: filecheck [-h] cmd file
Check File (uncompressed or gz) favor/instrument/quality
science:
return "true" if the fits file is a science frame. This
is determined by flavor=science in the header. If flavor
is not in the header, "false" is returned.
test:
return "true" if the fits file is a test frame. This is
determined by quality=test in the header. If quality
is not in the header, "false" is returned
excellent:
return "true" if the fits file is a excellent frame. This is
determined by quality=excellent in the header. If quality
is not in the header, "true" is returned
boss:
return "true" if the plPlugMapM file is a boss frame.
this is determined by instrument=boss in the header.
If instrument is not in the header, "false" is returned.
positional arguments:
cmd file check command
file fits file
options:
-h, --help show this help message and exit
flag_manual_cal¶
usage: flag_manual_cal [-h] -o {apo,lco} -m MJD -f FIELD [-e EXPID] -t
{arc,flat} [--nogit]
Build spManCal.par file to flag manual alternative calibration frames for
spPlan
options:
-h, --help show this help message and exit
-o {apo,lco}, --observatory {apo,lco}, --obs {apo,lco}
observatory
-m MJD, --mjd MJD MJD
-f FIELD, --field FIELD
FieldID
-e EXPID, --expid EXPID
Exposure ID to manually set the calibration frame
exposure ID
-t {arc,flat}, --type {arc,flat}
Calibration Type
--nogit Skip automatic git add
flatlib¶
usage: flatlib [--help]
{reduce,build,plot,analyze,lowfiber,csv,timeSeries,end2end} ...
Build and analyze a library of flats to check for Fiber throughput Issues
positional arguments:
{reduce,build,plot,analyze,lowfiber,csv,timeSeries,end2end}
Sub-command help
reduce Reduce/link the spFlats
build Build the flat library fits file
plot Plot Raw and Reduced Flat
analyze Run Full analysis on Flat library
lowfiber Check for Low fibers
csv Export CSV only
timeSeries Plot Throughout Time Series only
end2end Plot Throughout Time Series only by running through
all the steps (FPS only)
options:
--help, -h show this help message and exit
Subparser 'reduce' help:
usage: flatlib reduce [-h] [--dir DIR] [--run2d RUN2D] [--lco] [--link]
[--deep] [--nodes NODES] [--nosubmit] [--no_run]
[--link_all] [--link_traceflat] [--mjd [MJD ...]]
[--fps] [--plates] [--legacy]
Reduce/link the spFlats
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
--link Link Pre-existing spFlat Files
--deep Check Pre-existing plans for completion
--nodes NODES Number of nodes to use
--nosubmit Dont submit the job
--no_run, -n Just link (if set), but do not run new spFlat files
--link_all, -a Link all spFlat files regardless of spPlanTrace file
--link_traceflat, -c Link all spTraceFlat files
--mjd [MJD ...], -m [MJD ...]
MJDs to Run
--fps Catch up FPS
--plates Catch up Plates
--legacy Catch up Legacy
Subparser 'build' help:
usage: flatlib build [-h] [--dir DIR] [--run2d RUN2D] [--lco]
Build the flat library fits file
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
Subparser 'plot' help:
usage: flatlib plot [-h] [--dir DIR] [--run2d RUN2D] [--lco] [--save SAVE]
[--mjd MJD [MJD ...]] [--flats FLATS [FLATS ...]]
Plot Raw and Reduced Flat
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
--save SAVE, -s SAVE Save Directory
--mjd MJD [MJD ...], -m MJD [MJD ...]
List of mjds to plot
--flats FLATS [FLATS ...], -f FLATS [FLATS ...]
List of reduced flats to plot (overrides mjd and obs)
Subparser 'analyze' help:
usage: flatlib analyze [-h] [--dir DIR] [--run2d RUN2D] [--lco]
[--mjd MJD [MJD ...]] [--noplot]
Run Full analysis on Flat library
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
--mjd MJD [MJD ...], -m MJD [MJD ...]
List of mjds to plot alone
--noplot Plot Flat
Subparser 'lowfiber' help:
usage: flatlib lowfiber [-h] [--dir DIR] [--run2d RUN2D] [--lco]
[--mjd MJD [MJD ...]] [--threshold THRESHOLD]
Check for Low fibers
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
--mjd MJD [MJD ...], -m MJD [MJD ...]
List of mjds to plot alone
--threshold THRESHOLD, -t THRESHOLD
Threshold to flag lowfibers
Subparser 'csv' help:
usage: flatlib csv [-h] [--dir DIR] [--run2d RUN2D] [--lco]
Export CSV only
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
Subparser 'timeSeries' help:
usage: flatlib timeSeries [-h] [--dir DIR] [--run2d RUN2D] [--lco]
[--mjd MJD [MJD ...]] [--mjdstart MJDSTART]
[--TraceIDs]
Plot Throughout Time Series only
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
--mjd MJD [MJD ...], -m MJD [MJD ...]
List of mjds to plot alone
--mjdstart MJDSTART MJD to start reduction (later steps use all mjd
included in --mjd or all if not set) to plot alone
--TraceIDs, -t Label with Trace FiberIDs rather then slit FiberIDs
Subparser 'end2end' help:
usage: flatlib end2end [-h] [--dir DIR] [--run2d RUN2D] [--lco] [--link]
[--deep] [--nodes NODES] [--nosubmit] [--no_run]
[--link_all] [--link_traceflat] [--TraceIDs]
Plot Throughout Time Series only by running through all the steps (FPS only)
options:
-h, --help show this help message and exit
--dir DIR, -d DIR Flat Library Directory
--run2d RUN2D Override $RUN2D
--lco, -l Run for LCO data
--link Link Pre-existing spFlat Files
--deep Check Pre-existing plans for completion
--nodes NODES Number of nodes to use
--nosubmit Dont submit the job
--no_run, -n Just link (if set), but do not run new spFlat files
--link_all, -a Link all spFlat files regardless of spPlanTrace file
--link_traceflat, -c Link all spTraceFlat files
--TraceIDs, -t Label with Trace FiberIDs rather then slit FiberIDs
fluxcorr_prior¶
usage: fluxcorr_prior [-h] [--xythrucorr] planfile
Try solving with a prior that fluxcorr = 1
positional arguments:
planfile name of the plan file
options:
-h, --help show this help message and exit
--xythrucorr Apply XY throughput Correction
idlspec2d_version¶
usage: idlspec2d_version [-h]
Prints the IDLspec2D BOSS_DRP version
options:
-h, --help show this help message and exit
loadSN2Value¶
usage: loadSN2Value [-h] [-v] [-u] [--sdssv_sn2] fits confSum
Load SOS SN2 values into OpsDB
positional arguments:
fits The fits file is the science frame output from sos-reduce
confSum confSummary-file
options:
-h, --help show this help message and exit
-v, --verbose verbose
-u, --update update (An error will occur if the exposure has already been
processed, unless set)
--sdssv_sn2 Load sdssv_sn2
manage_coadd_Schema¶
usage: manage_coadd_Schema [-h] [--coaddfile COADDFILE] [--topdir TOPDIR]
[--run2d RUN2D] [--name NAME] [--DR] [--rerun1d]
[--active] [--carton [CARTON ...]]
[--SDSSIDS [SDSSIDS ...]] [--program [PROGRAM ...]]
[--legacy [LEGACY ...]] [--use_catid]
[--use_firstcarton] [--cadence CADENCE] [--show]
[--mjd [MJD ...]]
Manage Custom Coadds
options:
-h, --help show this help message and exit
--coaddfile COADDFILE, -f COADDFILE
File to store Coadding Schema (Default:
{topdir}/{run2d}/fields/SDSSV_BHM_COADDS.par)
--topdir TOPDIR Override value for the environment variable
$BOSS_SPECTRO_REDUX.
--run2d RUN2D Override value for the environment variable $RUN2D
--name NAME Name of Custom Coadd
--DR DR/IPL Coadding
--rerun1d, -r Provides flag for coadd to be rerun though 1D analysis
--active, -a Activate (or deactivate) a Coadding Schema
--carton [CARTON ...], -c [CARTON ...]
list of cartons
--SDSSIDS [SDSSIDS ...], -i [SDSSIDS ...]
list of SDSS_IDS (or CatalogIDs if use_catid is set)
--program [PROGRAM ...], -p [PROGRAM ...]
list of programs
--legacy [LEGACY ...], -l [LEGACY ...]
list of Legacy Tags to include
--use_catid, -u Use CatalogIDs rather then SDSS_IDs
--use_firstcarton Use Firstcarton only for carton match (dont look at
db)
--cadence CADENCE, -t CADENCE
Number of days between coadd epochs
--show, -s Show Configurations
--mjd [MJD ...] Use data from these MJDs.
parse_runtime¶
usage: parse_runtime [-h] [-a] [-s] file_path [file_path ...]
Process log file to calculate elapsed times for SOS.
positional arguments:
file_path Path to the log file
options:
-h, --help show this help message and exit
-a, --all Combine all daily logs of this format
-s, --stamp Add Date stamp to output file
plot_qa¶
usage: plot_qa [-h] [-r [RUN2D ...]] [-t [TEST ...]] [--test_path TEST_PATH]
[--mjds_low [MJDS_LOW ...]] [--mjds_high [MJDS_HIGH ...]]
[--clobber_lists] [--lco] [--publish] [--html]
[--html_name HTML_NAME] [--fast_opsdb] [--epoch] [--cron]
Plot QA
options:
-h, --help show this help message and exit
-r [RUN2D ...], --run2d [RUN2D ...]
List of run2ds
-t [TEST ...], --test [TEST ...]
List of True/False test run2d (corresponding to run2d)
--test_path TEST_PATH
test Run2d path modification
--mjds_low [MJDS_LOW ...]
List of mjd lower limits - use None for no limit
(corresponding to run2d)
--mjds_high [MJDS_HIGH ...]
List of mjd upper limits - use None for no limit
(corresponding to run2d)
--clobber_lists Clobber list of fieldIDs
--lco Flag for LCO vs APO
--publish create publication version of plot
--html Produces Plotly interactive HTML versions of the plots
--html_name HTML_NAME
Name of HTML file (default = BOSS_QA-{obs}.html)
--fast_opsdb, -f Skips OpsDB queries for SOS SN2 (and uses cached if
available)
--epoch, -e produce plots for epoch coadds
--cron, -c Produce cronlogs
read_sos¶
usage: read_sos [-h] [--exp EXP] [--nocopy] [--no_hash] directory mjd
Create Fiber info Summary for SOS
positional arguments:
directory SOS Directory
mjd mjd
options:
-h, --help show this help message and exit
--exp EXP, -e EXP Exposure Name
--nocopy, -n Prevent copy to combined Directory
--no_hash Skip updating the file hash
readfibermaps¶
usage: readfibermaps [-h] [-p SPPLAN2D] [--topdir TOPDIR] [-c] [--fast]
[--datamodel DATAMODEL] [-s] [--release RELEASE]
[--remote] [--v_targ V_TARG] [--confSummary CONFSUMMARY]
[--ccd {b2,r2,b1,r1}] [--mjd MJD] [--log]
Produces spfibermap file corresponding to a spplan2d (or single confSummary
file for SOS)
options:
-h, --help show this help message and exit
-p SPPLAN2D, --spplan2d SPPLAN2D
spplan2d file for idlspec2d run
--topdir TOPDIR Alternative output directory (defaults to location of
spplan2d file or /data/boss/sos/{mjd} for SOS)
-c, --clobber overwrites previous spfibermap file
--fast When using --no_db, streamlines process and only gets
parallax from MOS target files
--datamodel DATAMODEL
Supply a datamodel file (defaults to
$IDLSPEC2D/datamodel/spfibermap_dm.par or
$IDLSPEC2D/datamodel/spfibermap_sos_dm.par for SOS)
-s, --SOS produces spfibermap for SOS
--release RELEASE sdss_access data release (defaults to sdsswork),
required if you do not have proprietary access,
otherwise see https://sdss-
access.readthedocs.io/en/latest/auth.html#auth
--remote allow for remote access to data using sdss-access
--v_targ V_TARG SDSS-V MOS Targeting Product Version (for no Database
access use)
SOS:
Options of use with SOS only
--confSummary CONFSUMMARY
confSummary file for SOS (required for with --SOS)
--ccd {b2,r2,b1,r1} CCD for SOS
--mjd MJD MJD of observation
--log creates log file in topdir
run_PyXCSAO¶
usage: run_PyXCSAO [-h] [--run1d RUN1D] [--epoch] [--custom CUSTOM] fitsfile
Runs pyXCSAO to determine RVs
positional arguments:
fitsfile fits file
options:
-h, --help show this help message and exit
--run1d RUN1D, -r RUN1D
run1d name
--epoch run for epoch Coadds
--custom CUSTOM Name of Custom Coadd
sdR_hdrfix¶
usage: sdR_hdrfix [-h] [--mjd MJD] --obs {APO,LCO} [--clobber]
[--cameras {b1,b2,r1,r2,??}] [--no_update] [--nogit] [--bad]
[--test] [--FF {0,1} {0,1} {0,1} {0,1}]
[--FFS {0,1} {0,1} {0,1} {0,1} {0,1} {0,1} {0,1} {0,1}]
[--NE {0,1} {0,1} {0,1} {0,1}]
[--HGCD {0,1} {0,1} {0,1} {0,1}]
[--HEAR {0,1} {0,1} {0,1} {0,1}] [--arc] [--flat]
[--hartmann {Out,Right,Left,Closed}]
[--quality {excellent,test,bad}]
[--flavor {bias,dark,flat,arc,science,smear}]
[--exptime EXPTIME] [--tai-beg TAI_BEG]
[--cartid {FPS-S,FPS-N}] [--fieldid FIELDID]
[--confid CONFIGID] [--designid DESIGNID] [--key KEY]
[--value VALUE]
expid
Create the files used by the pipeline to fix the header meta data of the BOSS
exposures
positional arguments:
expid Exposure ID
options:
-h, --help show this help
message and exit
--mjd MJD, -m MJD mjd of file (default:
latest MJD)
--clobber clobber sdHdrFix file
--cameras {b1,b2,r1,r2,??} Cameras for hdr
update (?? for all
cameras) [default:??]
--no_update, -u Skip updating SOS
Logs for quality
flags
--nogit Skip automatic git
add
Required arguments:
--obs {APO,LCO} Set Observatory
Optional Quality Update (exclusive)
At current only use if still exposing or don't run SOS after for Science Frames
(skip and note in Night Log (and/or email) if uncertain):
--bad, -b flag as quality=bad
--test, -t flag as quality=test
Optional lamp/screen keys to Update (1:on, 0:off):
--FF {0,1} {0,1} {0,1} {0,1} Flat Field Lamp
--FFS {0,1} {0,1} {0,1} {0,1} {0,1} {0,1} {0,1} {0,1} Flat Field Screen
--NE {0,1} {0,1} {0,1} {0,1} Ne arc lamp
--HGCD {0,1} {0,1} {0,1} {0,1} HeCd arc Lamp
--HEAR {0,1} {0,1} {0,1} {0,1} HeAr arc Lamp
--arc short cut to set all
relevant arc lamps to
1 1 1 1
--flat short cut to set FF =
1 1 1 1 & FFS = 1 1 1
1 1 1 1 1
--hartmann {Out,Right,Left,Closed} Hartmann Door Status
Optional Common keys to Update
At current only use if still exposing or don't run SOS after for Science Frames
(skip and note in Night Log (and/or email) if uncertain):
--quality {excellent,test,bad} Set Quality flat of
exposures
Optional Specialized Keys to Update
At current only use if still exposing or don't run SOS after
(skip and note in Night Log (and/or email) if uncertain):
--flavor {bias,dark,flat,arc,science,smear} Type/Flavor of
exposure
--exptime EXPTIME Exposure length (s)
--tai-beg TAI_BEG Starting time (tai)
of exposure
--cartid {FPS-S,FPS-N} Cartridge Mounted
--fieldid FIELDID FieldID
--confid CONFIGID ConfigureID
--designid DESIGNID DesignID
Manually update a key
At current only use if still exposing or don't run SOS after
(skip and note in Night Log (and/or email) if uncertain):
--key KEY, -k KEY header keyword to
update (required if
value is set)
--value VALUE, -v VALUE updated header
keyword value
(required if key is
set)
one or more update options are required
slurm_Summary¶
usage: slurm_Summary [-h] [--module MODULE] [--topdir TOPDIR] [--run2d RUN2D]
[--run1d RUN1D] [--walltime WALLTIME] [--fast]
[--mem MEM] [--daily] [--epoch] [--custom CUSTOM]
[--full] [--monitor] [--no_submit] [--merge_only]
[--no_fieldlist] [--backup BACKUP] [--limit LIMIT]
[--n_iter N_ITER] [--ndays NDAYS] [--email_start]
[--skip_specprimary] [--update_specprimary]
[--utah_daily] [--verbose] [--defaults]
Create daily field merge slurm job
options:
-h, --help show this help message and exit
--module MODULE, -m MODULE
module file to use (ex bhm/master[default] or
bhm/v6_0_9)
--topdir TOPDIR Boss Spectro Redux base directory
--run2d RUN2D Run2d
--run1d RUN1D Run1d
--walltime WALLTIME, -w WALLTIME
Job wall time (format hh:mm:ss) default = "40:00:00"
--fast use fast allocation
--mem MEM memory in bytes
--daily only run if daily run has been run today
--epoch run for the epoch coadds
--custom CUSTOM Name of custom Coadd
--full Use a full cluster node
--monitor Monitor job and send email at completion with the logs
--no_submit Create slurm job but do not submit it
--merge_only Run fieldmerge in merge_only mode
--no_fieldlist Skip Running Fieldlist
--backup BACKUP Number of backups to keep, or None to not create
backup
--limit LIMIT Limit number of new field-mjds to update
--n_iter N_ITER number of iterations of field merge to run
--ndays NDAYS Limit spAll update to last ndays
--email_start Send email at start of run
--skip_specprimary Skip building specprimary in fieldmerge
--update_specprimary Keep existing specprimary and associated columns and
only update new row (and their secondaries)
--utah_daily Load tagged daily run into Pipelines.boss_drp database
table
--verbose Run Fieldmerge with verbose
--defaults Sets --full --merge_only --backup 3 --monitor
--update_specprimary --ndays 10
slurm_readfibermap¶
usage: slurm_readfibermap [-h] [--topdir TOPDIR] [--run2d RUN2D] [--clobber]
[--apo] [--lco] [--v_targ V_TARG] [--mjd [MJD ...]]
[--mjdstart MJDSTART] [--mjdend MJDEND]
[--mem_per_cpu MEM_PER_CPU] [--walltime WALLTIME]
[--ppn PPN] [--nbundle NBUNDLE]
Create daily field merge slurm job. Without access to the SDSS Slurm package,
it prints the commands for manual execution
options:
-h, --help show this help message and exit
--topdir TOPDIR Boss Spectro Redux base directory
--run2d RUN2D Run2d
--clobber Clobber spfibermaps
--apo run apo
--lco run lco
--v_targ V_TARG SDSS-V MOS Targeting Product Version (for no Database
access use)
Select MJDs:
--mjd [MJD ...] MJD dates to reduce; default="*"
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
Slurm Options:
--mem_per_cpu MEM_PER_CPU
Memory allocated per CPU
--walltime WALLTIME Wall time in hours
--ppn PPN Number of processors per node
--nbundle NBUNDLE Number of jobs to bundle
slurm_runfix¶
usage: slurm_runfix [-h] [--obs OBS [OBS ...]] [--mjd [MJD ...]]
[--mjdstart MJDSTART] [--mjdend MJDEND] [--epoch]
[--topdir TOPDIR] [--run1d RUN1D] [--run2d RUN2D] [--full]
[--running] [--nosubmit] [--nbundle NBUNDLE]
[--nodes NODES] [--shared] [--no_write]
[--walltime WALLTIME]
Check for failed runs and setup the runs to clean and rerun the crashed field-
mjds
options:
-h, --help show this help message and exit
--obs OBS [OBS ...] Observatory for status update
--mjd [MJD ...] Update these MJDs
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
--epoch Run for epoch Coadds
--topdir TOPDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--run1d RUN1D Optional override value for the enviro variable $RUN1D
--run2d RUN2D Optional override value for the enviro variable $RUN2D
--full Rerun full pipeline regardless of crashed step
--running Select Field-MJDs with
--nosubmit Build, but not submit redux files
--nbundle NBUNDLE Number of jobs to bundle
--nodes NODES Number of Nodes
--shared Node sharing
--no_write skip writing and submitting job
--walltime WALLTIME Wall time in hours
slurm_sos¶
usage: slurm_sos [-h] [--apo] [--lco] [--mjd [MJD ...]] [--mjdstart MJDSTART]
[--mjdend MJDEND] [--no_reject] [--clobber_fibermap] [-n]
[-o] [--no_sdssv_sn2] [--no_sn2_15] [--bright]
[--mem_per_cpu MEM_PER_CPU] [--walltime WALLTIME]
[--nodes NODES] [--ppn PPN] [--no_submit] [--nbundle NBUNDLE]
Create SOS slurm job. Without access to the SDSS Slurm package, it prints the
commands for manual execution
options:
-h, --help show this help message and exit
--apo run apo
--lco run lco
Select MJDs:
--mjd [MJD ...] MJD dates to reduce; default=Today
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
SOS Options:
--no_reject Overrides the Calibration rejection (use with caution)
--clobber_fibermap, -f
Clobbers the existing spfibermap files
-n, --no_arc2trace Skip Utilizing arc2trace refinements
-o, --forcea2t Force arc2trace for all fields (even if flat exists
for field)
--no_sdssv_sn2 Skip reporting a second set of SN2 values with updated
fit parameters
--no_sn2_15 Skip reporting a set of SN2 values with a fiducial mag
of 15
--bright Display BOSS_only Bright Time Operation SN2_15
Slurm Options:
--mem_per_cpu MEM_PER_CPU
Memory allocated per CPU
--walltime WALLTIME Wall time in hours
--nodes NODES Number of nodes to use; default=1
--ppn PPN Number of processors per node
--no_submit Skip submitting process to queue
--nbundle NBUNDLE Number of jobs to bundle
slurm_spTrace¶
usage: slurm_spTrace [-h] [--topdir TOPDIR] [--run2d RUN2D] [--mjd [MJD ...]]
[--daily] [--mjdstart MJDSTART] [--mjdend MJDEND] [--lco]
[--clobber] [--debug] [--skip_plan] [--nodes NODES]
[--walltime WALLTIME] [--maxjobs MAXJOBS]
Create spTrace slurm jobs. Without access to the SDSS Slurm package, it prints
the commands for manual execution.
options:
-h, --help show this help message and exit
--topdir TOPDIR Boss Spectro Redux base directory
--run2d RUN2D Run2d
--mjd [MJD ...] Use data from these MJDs.
--daily Run in daily mode (only use MJDs specified)
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
--lco Build Run files for LCO
--clobber Clobber the existing Plan files
--debug Run in debug mode
--skip_plan Skip creating plans and use currently existing plans
--nodes NODES Number of nodes to use to run arc2trace
--walltime WALLTIME Wall time in hours (default=72:00:00)
--maxjobs MAXJOBS Max Number of Parallel jobs per node
sos_hash¶
usage: sos_hash MJD
Create or check the SOS file hash
options:
-h, --help show this help message and exit
--mjd MJD MJD to process
--redo, -t use sosredo directory (same as SOS -t or --redoMode option)
--test, -d use sosredo/dev directory (same as SOS -d or --test option)
--utah, -u use utah test directory (same as SOS --utah option)
--create Create the Hash file
--check Check the SOS Hash
--transfer Check the SOS Hash of a Utah transfer
--lco Build/check for lco at Utah
--dummy Create a dummy file to prevent an empty hash file
sos_log2html¶
usage: sos_log2html [-h] [-l LOGFILE] [-f HTMLFILE] [--obs OBS]
[--copydir COPYDIR] [--fps] [--sdssv_sn2] [--sn2_15]
[--bright]
mjd sosdir
Plot the Science frame for SOS
positional arguments:
mjd MJD of reduction
sosdir Path to SOS save directory (ie. folder that contains
logfile-?????.fits)
options:
-h, --help show this help message and exit
-l LOGFILE, --logfile LOGFILE
Name of logfile (default: logfile-{mjd}.fits)
-f HTMLFILE, --htmlfile HTMLFILE
Name of output htmlfile (default: logfile-{mjd}.html)
--obs OBS, -o OBS Observatory of observations (default: apo)
--copydir COPYDIR, -c COPYDIR
Where to save the htmls
--fps build for FPS reductions
--sdssv_sn2 Include SDSSV SN2 V2
--sn2_15 Include Mag 15 SN2
--bright Include Mag 15 SN2 for all exposures
sos_plot¶
usage: sos_plot [-h] [--ccd CCD [CCD ...]] [--redo] [--mask_end]
[--outdir OUTDIR]
mjd expid
Plot the Science frame for SOS
positional arguments:
mjd MJD of reduction
expid Exposure ID to plot
options:
-h, --help show this help message and exit
--ccd CCD [CCD ...] CCDs to plot; default=['b1', 'r1']
--redo If set use sosredo rather then sos reductions
--mask_end Mask end of the spectra during plotting
--outdir OUTDIR Where to save the figure (default:
/data/boss/sos/tests/)
spSpec_reformat¶
usage: spSpec_reformat [-h] --field FIELD --mjd MJD [--topdir TOPDIR]
[--run2d RUN2D] [--run1d RUN1D] [--custom CUSTOM]
[--plot] [--epoch] [--lsdr10] [--allsky]
Build Spec Files
options:
-h, --help show this help message and exit
--field FIELD, -f FIELD
Run for a single Field
--mjd MJD, -m MJD Run for a single MJD
--topdir TOPDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--run2d RUN2D Optional override value for the enviro variable $RUN2D
--run1d RUN1D Optional override value for the enviro variable $RUN2D
--custom CUSTOM Name of Custom Coadd schema
--plot, -p Create spec plots
--epoch, -e Run for epoch Coadds
--lsdr10 Include Legacy Survey DR10 links on HTML
--allsky Reformat for Allsky Custom Coadd
spplan¶
usage: spplan [-h] [--skip2d] [--skip1d] [--topdir TOPDIR] [--run2d RUN2D]
[--lco] [--logfile LOGFILE] [--verbose VERBOSE] [-c]
[--release RELEASE] [--remote] [--override_manual]
[--mjd [MJD ...]] [--mjdstart MJDSTART] [--mjdend MJDEND]
[--field [FIELD ...]] [--fieldstart FIELDSTART]
[--fieldend FIELDEND] [--legacy] [--plates] [--fps] [--sdssv]
[--no_commissioning] [--no_dither] [--matched_flats]
[--nomatched_arcs] [--minexp MINEXP] [--multiple_flat]
[--multiple_arc] [--manual_noarc] [--plate_epoch] [--quick]
Produce the spPlan2d and spPlancomb files for the pipeline run
options:
-h, --help show this help message and exit
General:
General Setup Options
--skip2d Skip spplan2d
--skip1d Skip spplan1d
--topdir TOPDIR Base run2d directory to BOSS_SPECTRO_REDUX
environmental variable
--run2d RUN2D Run2d to environmental variable
--lco Build Run files for LCO
--logfile LOGFILE Optional logfile (Including path)
--verbose VERBOSE Provide information about nonutlized frames
-c, --clobber overwrites previous plan file
--release RELEASE sdss_access data release (defaults to sdsswork),
required if you do not have proprietary access,
otherwise see https://sdss-
access.readthedocs.io/en/latest/auth.html#auth
--remote allow for remote access to data using sdss-access
--override_manual Override/clobber manually edited plan
MJD/Field Filtering:
MJD/Field Filtering Options
--mjd [MJD ...] Use data from these MJDs.
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
--field [FIELD ...] Use data from these fields.
--fieldstart FIELDSTART
Starting Field
--fieldend FIELDEND Ending Field
--legacy Include legacy (BOSS/eBOSS) plates
--plates Include SDSS-V plates
--fps Include FPS Fields
--sdssv Include both SDSS-V Fields & Plates
--no_commissioning Exclude SDSS-V FPS Commission Fields
--no_dither Exclude Dither fields
RUN2D:
spPlan2d Setup Options
--matched_flats Require Flat from a field/plate
--nomatched_arcs Allow Arc from another field/plate
--minexp MINEXP Min Science Exposures in Plan (default=1)
--multiple_flat Find all possible flat calibration frames
--multiple_arc Find all possible arc calibration frames
--manual_noarc if nomatched_arcs is False, builds spplan with
unmatched arcs and mark as manual
RUN1D:
spPlancomb Setup Options
--plate_epoch Use a variable max epoch length for plate coadd
--quick Use the list of new spPlan2d as a filter for fields
spplan_epoch¶
usage: spplan_epoch [-h] [--topdir TOPDIR] [--run2d RUN2D] [--run1d RUN1D]
[--mjd MJD] [--mjdstart MJDSTART] [--mjdend MJDEND]
[--field FIELD] [--fieldst FIELDSTART]
[--fieldend FIELDEND] [--fps] [--sdssv] [--clobber]
[--minexp MINEXP] [--lco] [--logfile LOGFILE]
[--abandoned] [--started] [--min_epoch_len MIN_EPOCH_LEN]
[--release RELEASE] [--remote] [--v_targ V_TARG]
Builds the spPlancombepoch files
options:
-h, --help show this help message and exit
--topdir TOPDIR Override value for the environment variable
$BOSS_SPECTRO_REDUX.
--run2d RUN2D Override value for the environment variable $RUN2D
--run1d RUN1D Override value for the environment variable $RUN1D
--mjd MJD Use data from these MJDs.
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
--field FIELD Look for the input data files in topdir/fieldid;
default to search all subdirectories. Note that this
need not be integer-valued, but could be for example
'0306_test'.
--fieldst FIELDSTART Starting fieldid
--fieldend FIELDEND Ending fieldid
--fps Only produce epoch coadds for FPS Fields
(Fields>16000)
--sdssv Only produce epoch coadds for SDSS-V Fields
(Fields>15000)
--clobber If set, then over-write conflicting plan files
--minexp MINEXP Set minimum number of Science Frames for plan creation
--lco Create Plans for LCO
--logfile LOGFILE, -l LOGFILE
File for logging
--abandoned Create plans for abandoned epochs
--started Create plans for started epochs (including unfinished)
--min_epoch_len MIN_EPOCH_LEN
minimum length of epoch required to produce plan
--release RELEASE sdss_access data release (defaults to sdsswork),
required if you do not have proprietary access,
otherwise see https://sdss-
access.readthedocs.io/en/latest/auth.html#auth
--remote allow for remote access to data using sdss-access
--v_targ V_TARG SDSS-V MOS Targeting Product Version (for no Database
access use)
spplan_target¶
usage: spplan_target [-h] (--manual | --batch) [--name NAME]
[--coaddfile COADDFILE] [--topdir TOPDIR] [--run2d RUN2D]
[--run1d RUN1D] [--clobber] [--logfile LOGFILE] [--DR]
[--cartons [CARTONS ...]] [--catalogids [CATALOGIDS ...]]
[--program [PROGRAM ...]] [--mjd [MJD ...]]
[--mjdstart MJDSTART] [--mjdend MJDEND]
[--coadd_mjdstart COADD_MJDSTART] [--rerun1d]
[--use_catid] [--use_firstcarton] [--useDB] [--lco]
[--apo]
Build SDSSID/CatalogID Combine Plan
options:
-h, --help show this help message and exit
--manual Manaully run a Coadd Schema (from coaddfile if only
name is set)
--batch Batch run all active Coadd Schema in batch file
located {topdir}/{run2d}/{name}
--name NAME Name of Custom Coadd
--coaddfile COADDFILE
File of store Coadding Schema
--topdir TOPDIR Override value for the environment variable
$BOSS_SPECTRO_REDUX.
--run2d RUN2D Override value for the environment variable $RUN2D
--run1d RUN1D Override value for the environment variable $RUN1D
--clobber If set, then over-write conflicting plan files
--logfile LOGFILE File for logging
--DR DR/IPL Batch Coadding
--cartons [CARTONS ...]
list of cartons
--catalogids [CATALOGIDS ...]
list of sdss_ids (or catalogids)
--program [PROGRAM ...]
list of programs
--mjd [MJD ...] Use data from these MJDs.
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
--coadd_mjdstart COADD_MJDSTART
First Coadd MJD to include
--rerun1d Provides flag for coadd to be rerun though 1D analysis
--use_catid, -u Uses CatalogID rather then sdss_id
--use_firstcarton Use Firstcarton only for carton match (dont look at
db)
--useDB Use sdss targetdb instead of the Semaphore targeting
flag (if not use_firstcarton)
--lco Create Plans for LCO
--apo Create Plans for APO
spplan_trace¶
usage: spplan_trace [-h] [--topdir TOPDIR] [--run2d RUN2D] [--mjd_plans]
[--lco] [--logfile LOGFILE] [--verbose] [-c]
[--release RELEASE] [--remote] [--override_manual]
[--mjd [MJD ...]] [--mjdstart MJDSTART] [--mjdend MJDEND]
Produces spPlanTrace
options:
-h, --help show this help message and exit
General:
General Setup Options
--topdir TOPDIR Topdir to override the BOSS_SPECTRO_REDUX environmental
variable
--run2d RUN2D Run2d to environmental variable
--mjd_plans Only build plans for MJDs with spPlan2d
--lco Build Run files for LCO
--logfile LOGFILE Optional logfile (Including path)
--verbose Provide information about nonutlized frames
-c, --clobber overwrites previous plan file
--release RELEASE sdss_access data release (defaults to sdsswork),
required if you do not have proprietary access,
otherwise see https://sdss-
access.readthedocs.io/en/latest/auth.html#auth
--remote allow for remote access to data using sdss-access
--override_manual Override/clobber manually edited plan
MJD/Field Filtering:
MJD/Field Filtering Options
--mjd [MJD ...] Use data from these MJDs.
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
sxpar.py¶
usage: sxpar.py [-h] [-v] fitsfile keyword
Simply parse a fits header
positional arguments:
fitsfile The fits file to read
keyword Header keyword to parse
options:
-h, --help show this help message and exit
-v, --verbose verbose
sxpar_retry.py¶
usage: sxpar_retry.py [-h] [-v] fitsfile keyword
Simply parse a fits header, retrying if failed
positional arguments:
fitsfile The fits file to read
keyword Header keyword to parse
options:
-h, --help show this help message and exit
-v, --verbose verbose
update_flags¶
usage: update_flags [-h] [--run2d RUN2D] [--topdir TOPDIR] [--clobber]
[--custom [CUSTOM ...]] [--nobackup]
Update SDSSV Targeting flats inn the summary files
options:
-h, --help show this help message and exit
--run2d RUN2D idlspec2d Run2d version
--topdir TOPDIR idlspec2d Run2d topdir
--clobber Clobber spTargeting file
--custom [CUSTOM ...]
List of name of custom coadd schema
--nobackup Skip backup of existing
update_opfiber¶
usage: update_opfiber [--help] {refine,guess} ...
Prints updated/refined values for opfibers using spFlat traces
positional arguments:
{refine,guess} Sub-command help
refine Refine the opFiberFPS parameters using a spFlat
guess Takes a processed image frame produced using the /sawraw
flag in sdssproc (or indirecly via spreduce2d) as an input.
It then uses either the bundlefiber list of number of fibers
per bundle (supplied as input or via opFiberFPS) combined
with the scipy peak finding algarithm to create a first
guess of the peak fiber and bundle gaps. It uses the median
flux of the 11 central pixel (along the dispersion axis) to
build the flux array
options:
--help, -h show this help message and exit
Subparser 'refine' help:
usage: update_opfiber refine [-h] [-p PRECISION] fitsfile
Refine the opFiberFPS parameters using a spFlat
positional arguments:
fitsfile File name and path of the spFlat (refine) or sdProc
(guess) file to use
options:
-h, --help show this help message and exit
-p PRECISION, --precision PRECISION
Precision of the reported fiberspacing and bundle gaps
Subparser 'guess' help:
usage: update_opfiber guess [-h] [-p PRECISION] [-b [BUNDLEFIBERS ...]]
[-m MJD] [-f] [--min_peak_sep MIN_PEAK_SEP]
[--min_peak_height MIN_PEAK_HEIGHT]
fitsfile
Refine the opFiberFPS parameters using a spFlat
positional arguments:
fitsfile File name and path of the spFlat (refine) or sdProc
(guess) file to use
options:
-h, --help show this help message and exit
-p PRECISION, --precision PRECISION
Precision of the reported fiberspacing and bundle gaps
-b [BUNDLEFIBERS ...], --bundlefibers [BUNDLEFIBERS ...]
list of number of fibers per bundle (defaults to
reading from latest opFiberFPS entry)
-m MJD, --mjd MJD MJD of new updated opFiber Fiberparameter entry
(defaults to latest entry of opFiberFPS)
-f, --plot Whether to plot the flux slice and detected peaks
--min_peak_sep MIN_PEAK_SEP
the minimum seperation between detected peak (ie
fibers are no closer then this; default=6)
--min_peak_height MIN_PEAK_HEIGHT
the minimum flux level to be detected as a peak
(default=5000)
uubatchpbs¶
usage: uubatchpbs [-h] [--sdssv] [--sdssv_fast] [--sdssv_noshare] [--apo]
[--lco] [--bay15] [--merge3d] [--obs [OBS ...]]
[--topdir TOPDIR] [--run1d RUN1D] [--run2d RUN2D]
[--idlutils_1d IDLUTILS_1D] [--no_reject] [--MWM_fluxer]
[--map3d {bayestar15,bay15,merge3d}] [--no_healpix]
[--noxcsao] [--dist] [--skip_specprimary] [--no_merge_spall]
[--skip2d] [--only1d] [--onestep_coadd] [--fibermap_clobber]
[--saveraw] [--debug] [--no_db] [--fast_no_db FAST_NO_DB]
[--release RELEASE] [--v_targ V_TARG] [--a2t]
[--field [FIELD ...]] [--fieldstart FIELDSTART]
[--fieldend FIELDEND] [--mjd [MJD ...]]
[--mjdstart MJDSTART] [--mjdend MJDEND] [--no_write]
[--shared] [--fast] [--mem_per_cpu MEM_PER_CPU]
[--walltime WALLTIME] [--nodes NODES] [--ppn PPN]
[--nosubmit] [--clobber] [--nbundle NBUNDLE] [--epoch]
[--custom CUSTOM] [--allsky] [--single_mjd] [--coadd_only]
[--1dpost] [--email]
Build idlspec2d redux and submit to slurm. Without access to the SDSS Slurm
package, it prints the commands for manual execution
options:
-h, --help show this help message and exit
Short cuts:
--sdssv --mwm --no_merge_spall --no_reject --shared --merge3d
--sdssv_fast --sdssv --fast --shared --merge3d
--sdssv_noshare --sdssv (without --shared)
--apo Run apo only
--lco Run lco only
--bay15 Set map3d to bayestar15 model
--merge3d Set map3d to best 3d model
idlspec2d Run options:
--obs [OBS ...] Observatory {apo,lco}
--topdir TOPDIR Optional override value for the environment variable
$BOSS_SPECTRO_REDUX
--run1d RUN1D Optional override value for the enviro variable $RUN1D
--run2d RUN2D Optional override value for the enviro variable $RUN2D
--idlutils_1d IDLUTILS_1D
idlutils override version of spec1d
--no_reject Deactivate Rejection in Coadd
--MWM_fluxer, --mwm
--map3d {bayestar15,bay15,merge3d}
Name of 3d dustmap to use with MWM_fluxer
(default=None)
--no_healpix, --nohp Turn off copy to healpix
--noxcsao Skip pyXCSAO
--dist unsets --nodist and reactivates the flux distortion
corrections
--skip_specprimary Skip Calculation of Specprimary
--no_merge_spall Skip building full SpAll File
--skip2d Skip fullinputs
--only1d run spec1d step only (eg. spreduce1d_empca, XCSAO)
--onestep_coadd Use legacy one step version of coadd
--fibermap_clobber Clobber spfibermap fits file
--saveraw Save sdssproc outputs
--debug Save extraction debug files
--no_db skip Database operations
--fast_no_db FAST_NO_DB
When using --no_db, streamlines process and only gets
parallax from MOS target files
--release RELEASE sdss_access data release (defaults to sdsswork),
required if you do not have proprietary access,
otherwise see https://sdss-
access.readthedocs.io/en/latest/auth.html#auth
--v_targ V_TARG SDSS-V MOS Targeting Product Version (for no Database
access use)
--a2t Force Use of Arc2Trace
Select Fields:
--field [FIELD ...], -f [FIELD ...]
Plate/Field numbers to reduce default="*"
--fieldstart FIELDSTART
Starting Field/Plate number
--fieldend FIELDEND End Field/Plate number
Select MJDs:
--mjd [MJD ...], -m [MJD ...]
MJD dates to reduce; default="*"
--mjdstart MJDSTART Starting MJD
--mjdend MJDEND Ending MJD
Slurm Options:
--no_write skip writing and submitting job
--shared Node sharing
--fast Use SDSS fast queue
--mem_per_cpu MEM_PER_CPU
Memory allocated per CPU
--walltime WALLTIME Wall time in hours
--nodes NODES Number of Nodes
--ppn PPN Number of processors per node
--nosubmit Build, but not submit redux files
--clobber Clobber redux
--nbundle NBUNDLE Number of jobs to bundle
Custom Coadd Options:
--epoch Epoch Coadds
--custom CUSTOM Name of custom Coadd Schema
--allsky All Sky Coadds
--single_mjd Run Each Custom MJD coadd+1dpost as seperate job
--coadd_only Run spspec_target_merge only
--1dpost Run 1d analysis and post processing only
Email outputs:
--email Email log using $DAILY_DIR/etc/emails
uurundaily¶
usage: uurundaily [-h] [--module MODULE] [--apo] [--lco] [--mjd [MJD ...]]
[--range_mjd RANGE_MJD] [--no_dither] [--epoch]
[--no_traceflat] [--no_fibermap] [--no_prep]
[--skip_plan [{pipe,trace,True,all}]]
[--clobber [{spPlans,fibermap,trace,True,all} ...]]
[--no_healpix] [--summary] [--saveraw] [--debug] [--tagged]
[--daily] [--dev] [--dist] [--no_merge3d] [--fast]
[--nosubmit] [--noslurm] [--batch] [--nodb] [--monitor]
[--allemail] [--pause PAUSE] [--walltime WALLTIME]
[--mem_per_cpu MEM_PER_CPU] [--nbundle NBUNDLE]
Process the BOSS data for a single MJD end-to-end (including plan files)
options:
-h, --help show this help message and exit
--module MODULE Module for daily run
Field-MJD Selection:
Arguments to control the Field-MJD Selection to run
--apo Run for APO Only
--lco Run for LCO Only
--mjd [MJD ...] Manually run for a single/list of mjd (does not update
nextmjd.par)
--range_mjd RANGE_MJD
Manually run for a range of mjds (does not update
nextmjd.par)
--no_dither Skip Dither Engineering Fields
--epoch Run Epoch Coadds
Pipeline Steps:
Arguments to control which steps of the full pipeline are run
--no_traceflat Skip Building and using TraceFlats
--no_fibermap Skip Pre-Run of readfibermap
--no_prep Skip building TraceFlats and spfibermaps before
pipeline run
--skip_plan [{pipe,trace,True,all}]
Skip the given plan {pipe,trace,all (flagging
--skip_plan with no name will default to all)}
--clobber [{spPlans,fibermap,trace,True,all} ...]
Clobber uubatchpbs + a combo of spPlan, fibermap, and
TraceFlat run {fibermap,trace, all (flagging --clobber
with name will default to all)}
--no_healpix Skip linking of spec files to the MWM_HEALPIX
Directory
--summary Build Summary Files
Debug:
Arguments to saving of optional debugging files
--saveraw save sdssproc outputs
--debug save extraction debug files
Shortcut Options:
Arguments to set multiple options at once for various defaults
--tagged sets --merge3d --no_dither --monitor --allemail
--daily sets --merge3d --fast --monitor --allemail
--no_healpix
--dev sets --merge3d --no_dither --monitor --no_healpix
Pipeline Options:
Arguments to set the misc pipeline options
--dist unsets --nodist and reactivates the flux distortion
corrections
--no_merge3d Skip using prototype 3D Dustmap (in merge mode)
--fast turn on fast user for slurm
--nosubmit Skip submitting uubatch job (ideal for allowing
editting of plans)
--noslurm Skip creating uubatch job
--batch run for multiple mjds in a single batch
--nodb skip Database operations
--monitor Monitors pipeline status
--allemail Email intermediate log using all emails in
$DAILY_DIR/etc/emails (defaults to first email only)
--pause PAUSE Pause time (s) in status updates
--walltime WALLTIME Wall time in hours
--mem_per_cpu MEM_PER_CPU
Memory allocated per CPU
--nbundle NBUNDLE Number of jobs to bundle
IDL Command Usage¶
spreduce2d.pro¶
;+
; NAME:
; spreduce2d
;
; PURPOSE:
; Calling script for SPREDUCE that reduces a night of data according
; to a plan file.
;
; CALLING SEQUENCE:
; spreduce2d, [ planfile, docams=, /do_telluric, /xdisplay, $
; /writeflatmodel, /writearcmodel, /bbspec ]
;
; INPUTS:
;
; OPTIONAL INPUTS:
; planfile - Name(s) of output plan file; default to reducing all
; plan files matching 'spPlan2d*.par'
; docams - Cameras to reduce; default to ['b1', 'b2', 'r1', 'r2']
; do_telluric- Passed to EXTRACT_OBJECT
; xdisplay - Send plots to X display rather than to plot file
; writeflatmodel - passed to SPCALIB via SPREDUCE to trigger writing
; out of flat model info to file.
; writearcmodel - passed to SPCALIB via SPREDUCE to trigger writing
; out of arc model info to file.
; bbspec - use bbspec extraction code
; noreject - Override Bad calibration rejection (use with caution)
;
; Optional Keywords:
; MWM_fluxer - Utilize MWM optional settings (ie gaia reddening and different S/N cuts)
;
;
; OUTPUT:
;
; COMMENTS:
; The following environment variables must be set:
; BOSS_SPECTRO_DATA
; SDSSCORE
; SPECFLAT_DIR
; Look for raw FITS data files in BOSS_SPECTRO_DATA/MJD.
; Look for obsSummary files in SDSSCORE/MJD.
; Look for spectroscopic flat files in SPECFLAT_DIR.
;
; EXAMPLES:
;
; BUGS:
; This routine spawns the Unix command 'mkdir'.
;
; PROCEDURES CALLED:
; cpbackup
; idlspec2d_version()
; idlutils_version()
; splog
; spreduce
; yanny_free
; yanny_par()
; yanny_read
;
; INTERNAL SUPPORT ROUTINES:
;
; REVISION HISTORY:
; 02-Nov-1999 Written by David Schlegel, Princeton.
; Apr-2010 Added "write[flat,arc]model" pass-through (A. Bolton, Utah)
; 15-Aug-2011 Added pass-through for spatial split of sky model (A. Bolton, Utah)
; 15-Nov-2018: Modified for use only one spectrograph for the BHM (HJIM)
;-
rm_combine_script.pro¶
;+
; NAME:
; rm_combine_script
;
; PURPOSE:
; Script to process epochs with the xyfit custom flux calibration
;
; CALLING SEQUENCE:
;
; INPUTS:
; planfile - Name(s) of output plan file
;
; OPTIONAL INPUTS:
; run2d - Name of the run2d
; finaldir - Additional subdirectory for output
; xyfit - Compute 2d flux corrections in the xy focal plane
; bscore - Fraction of best exposure score to use as a threshold for discarding exposures
; minsn2 - Minimum S/N^2 to include science frame in coadd; default
; to 0 to only include those with S/N > 0.
; Note that all exposures with a score less than 0.2 times
; the score of the best exposure are discarded; for those
; purposes, the score used is the worst of all 4 cameras.
;
;
; Optional Keywords:
; MWM_fluxer - Utilize MWM optional settings (ie gaia reddening and different S/N cuts)
; nofcorr - Skip the step to generate and use the spFluxcorr* files
; nodist - Skip the step to generate and use the spFluxdistort* files
; radec_coadd - Coadd using ra-dec matching rather then catalogID matching
; no_reject - Turns off rejection in the coadding
; onestep_coadd - Legacy algorithm for coadd. Coadding blue+red and all exposures
; at the the same time.
; epoch - Epoch Coadd flag for input and outputs
; legacy - Flag for Pre-SDSSV 2 Spectrograph data at APO
; plates - Flat for SDSSV 1 Spectrograph plate data at APO
; loaddesi - Load the DESI (JG) models for fluxing
; skipfluxing - Skip the step to generate spFluxcalib* files
; skipfcorr - Skip creation of flux-correction vectors and use prexisting spFluxcorr* files
;
; OUTPUT:
;
; COMMENTS:
; EXAMPLES:
;
; BUGS:
; This routine spawns the Unix command 'mkdir'.
;
; PROCEDURES CALLED:
; get_field_dir
; djs_filepath
; rm_spcombine_v5
;
;
spreduce1d_empca.pro¶
;+
; NOTE: same as spreduce1d, but uses different QSO PCA templates
; NAME:
; spreduce1d
;
; PURPOSE:
; 1-D reduction of spectra from 1 plate
;
; CALLING SEQUENCE:
; spreduce1d, [ platefile, fiberid=, run1d=, /doplot, /debug, chop_data= ]
;
; INPUTS:
;
; OPTIONAL INPUTS:
; platefile - Plate file(s) from spectro-2D; default to all files
; matching 'spPlate*.fits'
; fiberid - If specified, then only reduce these fiber numbers;
; this must be a vector with unique values between 1 and
; the number of fibers in the plate file
; run1d - Optional override value for the environment variable $RUN1D
; doplot - If set, then generate plots. Send plots to a PostScript
; file spDiagDebug1d-$PLATE-$MJD.ps unless /DEBUG is set.
; debug - If set, then send plots to the X display and wait for
; a keystroke after each plot; setting /DEBUG forces /DOPLOT.
; chop_data - If set, then trim wavelength range to the specified range
; in vacuum Ang (if a 2-element array), or to a default
; trim range of [3600,10400] Ang.
;
; OUTPUTS:
;
; OPTIONAL OUTPUTS:
;
; COMMENTS:
; Input files are read from the current directory.
; Output files are written to the subdirectory $RUN1D.
;
; Names of output files are derived from PLATEFILE.
; For example, if PLATEFILE='spPlate-0306-51690.fits', then
; ZALLFILE = 'spZall-0306-51690.fits'
; ZBESTFILE = 'spZbest-0306-51690.fits'
; ZLINEFILE = 'spZline-0306-51690.fits'
;
; EXAMPLES:
;
; BUGS:
;
; DATA FILES:
; $IDLSPEC2D_DIR/templates/TEMPLATEFILES
;
; PROCEDURES CALLED:
; cpbackup
; dfpsclose
; dfpsplot
; djs_filepath()
; elodie_best()
; fileandpath()
; filter_thru()
; mrdfits()
; mwrfits
; qaplot_fcalibvec
; splog
; skymask()
; speclinefit
; star_dvelocity()
; struct_addtags()
; sxaddpar
; sxdelpar
; sxpar()
; synthspec()
; vdispfit
; zfind()
; zrefind()
;
; REVISION HISTORY:
; 28-Jun-2000 Written by D. Schlegel, Princeton
; 2010-2011: various template-related tweaks and Z_NOQSO, A. Bolton, Utah
; 01-Oct-2012: Adding ZNUM_NOQSO to the Z_NOQSO section, Joel Brownstein, Utah
spcalib_qa.pro¶
; NAME:
; spcalib_qa
;
; PURPOSE:
; Compare photometric accuracy of standards
;
; CALLING SEQUENCE:
; SpCalib_QA, [run2d=, fieldid=, mjd=, /rerun, /nobkup, /epoch, /catchup]
;
; INPUTS:
;
; OPTIONAL INPUTS:
; field - field to include
; mjd - MJD to include
; run2d - RUN2D version of reduction
; rerun - Rerun for all field-mjds in spAll
; nobkup - Don't backup output and log file
; catchup - Run for missing field-mjds spAll
; epoch - run for epoch coadds
;
; OUTPUTS:
;
; OPTIONAL OUTPUTS:
;
; COMMENTS:
; Depends on the spAll files (either full run2d version or field-mjd version)
;
; EXAMPLES:
;
; BUGS:
;
; DATA FILES:
;
; Function Called:
; mpfitfun
; field_to_string
; djs_filepath
; mrdfits
; sdss_flagval
;
; External PROCEDURES CALLED:
; plot
; XYOUTS
; cpbackup
;
; Internal PROCEDURES CALLED:
; std_hist
;
; REVISION HISTORY:
; 21-June-2022 Written by S. Morrison (UIUC)
spspec_target_merge.pro¶
;+
; NAME:
; spspec_target_merge
;
; PURPOSE:
; To create spSpec and spFullsky target level coadds (independent of field-mjd)
;
; CALLING SEQUENCE:
;
; INPUTS:
; customplan - The spPlanCustom file for the coadd
;
; OPTIONAL KEYWORDS:
; topdir - the daily coadd base directory
;
; OUTPUTS:
;
; OPTIONAL OUTPUTS:
;
; COMMENTS:
;
; EXAMPLES:
;
; BUGS:
;
; PROCEDURES CALLED:
;
; REVISION HISTORY:
;
;-