.. title:: IDLspec2D: Dependencies Pipeline Dependencies ===================== Setup Requirements ------------------- A number of system environmental variables and paths are required to run the BOSS DRP. Running at the University of Utah CHPC ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ At Utah, modules (`github.com/sdss/sdss_modules `_) are used to maintain the the required packages and paths. For the BOSS pipeline, the bhm modules are used to manage these for the BOSS pipeline. Environmental Variables ^^^^^^^^^^^^^^^^^^^^^^^ :: export DATABASE_PROFILE="pipelines" export RUN2D="|idlspec2d_version|" export RUN1D="|idlspec2d_version|" export BOSS_SPECTRO_REDUX="/uufs/chpc.utah.edu/common/home/sdss50/sdsswork/bhm/boss/spectro/redux" export BOSS_SPECTRO_DATA_N="/uufs/chpc.utah.edu/common/home/sdss50/sdsswork/bhm/boss/spectro/apo" export BOSS_SPECTRO_DATA_S="/uufs/chpc.utah.edu/common/home/sdss50/sdsswork/bhm/boss/spectro/lco" export SDHDRFIX_DIR="/uufs/chpc.utah.edu/common/home/sdss50/software/git/sdss/sdsscore_test/main" export IDLSPEC2D_DIR="/uufs/chpc.utah.edu/common/home/sdss50/software/git/sdss/idlspec2d/$RUN2D" export SDSSCORE_DIR="/uufs/chpc.utah.edu/common/home/sdss50/software/git/sdss/sdsscore_test/main" export DAILY_DIR='$HOME/daily/ export BOSS_QA_DIR='$BOSS_SPECTRO_REDUX' .. _Paths: Paths ^^^^^ :: export IDL_PATH="$IDL_PATH:$IDLSPEC2D_DIR" export PATH="$PATH:$IDLSPEC2D_DIR/bin" export PYTHONPATH="$PYTHONPATH:$IDLSPEC2D_DIR/python" Compiling Internal Functions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The idlspec2d package contains a shared object library and C scripts. If you have issues with these, you can run ``evilmake all`` within the main idlspec2d directory ($IDLSPEC2D_DIR). ``evilmake`` is included within idlutils, so follow the directions on (`sdss.org/dr18/software/packages/idlutils/ `_) to install idlutils first. Compiling Internal Documentation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This documentation can be compiled locally by navigating to ``$IDLSPEC2D_DIR/docs/sphinx`` and run ``make all`` (This can also be run with ``evilmake all`` in the same directory or ``evilmake doc`` within ``$IDLSPEC2D_DIR``). This will compile the html, singlehtml, and pdf forms of the documentation within the associated subdirectoires of ``$IDLSPEC2D_DIR/docs/sphinx/_build``. Dependencies ------------------- * idl * python(3.7-3.11) * SDSS Collaboration Package Dependencies * `idlutils `_: idlutils is a collection of IDL functions and routines used by a variety of SDSS software. * `sdssdb `_: sdssdb contains the source catalogs, targeting catalogs, and operational databases. * `semaphore `_: provides codes and reference files to decode and understand the sdss_target_flags * `sdss-tree `_: provides environment variables to manage paths for SDSS data as they organized on the Science Archive Server (SAS) * `sdss_access `_: provides a convenient way of navigating local and remote file system paths from the Science Archive Server (SAS) * `pyVista `_: * `SDSS Slurm `_: Required for use of uurundaily at Utah, all other command can be manually (at Utah or elsewhere) run without access to the slurm manager provided by this package * `sdsstools `_: * SDSS Product Dependencies * `sdsscore `_: sdsscore is an SDSS-V product that contains the FPS fiber configuration files and header correction files to change the header exposure values (sdHdrFix-{mjd}.par) * `elodie `_: A database of high and medium-resolution stellar spectra (Prugniel+, 2001) used by spec1d to classify spectra and determine stellar parameters. * `dust `_: A catalog of dust extinction models, including the SFD model. * `speclog `_: speclog is an SDSS product that contains information about SDSS BOSS plate operations including seeing measured by the guides (guiderMon-{MJD}.par, plate plug maps (plPlugMapM-{plateid}-{mjd}-{plugid}.par, and plate header correction files to change the header exposure values (sdHdrFix-{mjd}.par) * `platelist `_: platelist is an SDSS product that contains information on the plate designs and plugging. The plateHoles files include additional metadata associated with the targets on a plate * `specflat `_: specflat is an SDSS product that contains master calibration frames and bad pixel masks for use in the idlspec2d pipeline. * `gaia/dr2 `_: idlspec2d utilizes gaia_source/csv to calculate the distance to standard stars from GAIA DR2 proper motion. * External Dependencies * `pyDL `_: a package that consists of python replacements for IDL function, both built-in and from external astronomical libraries * `dustmaps `_: provides a unified interface for several 2D and 3D maps of interstellar dust reddening and extinction. idlspec2d makes use of the Bayestar 2015 dustmaps (`Green, Schlafly, Finkbeiner et al. 2015 `_) * `PyXCSAO `_: a python package designed to replicate the functionality of `IRAF XCSAO `_. * `numpy `_: a standard Python package for arrays and high-level mathematical functions * `astropy `_: a collection of astronomy packages written in Python * `matplotlib `_: a python plotting library * `healpy `_: a Python package based on the Hierarchical Equal Area isoLatitude Pixelization (HEALPix) scheme * `tqdm `_: a progress bar for Python * `pandas `_: a python package designed for data manipulation and analysis * `h5py `_: a python interface between numpy and HDF5 data * `scipy `_: a python package for scientific and technical computing * `pillow `_: a python image file processing library * `jinja2 `_: a python templating engine used to build the HTMLs produced by the pipeline * `termcolor `_: a python package for color formatting of terminal outputs (not required but recommended) * `plotly `_: a python package for interactive plots (not required for the core pipeline but used by some of the supplementary tools)