Welcome to the pyhrs documentation! pyhrs is a package for the reduction and analysis of data from the High Resolution Spectrograph on the Southern African Large Telescope
Documentation¶
pyhrs is an ffiliated package for the AstroPy package. The documentation for this package is here:
HRS Data Reduction (pyhrs)¶
Introduction¶
Note
pyhrs works only with astropy version 1.0.0 or later. It also requires ccdproc version 0.3.0 or later.
The pyhrs package provides steps for reducing and extracting data from the High Resolution Spectrograph on the Southern African Large Telescope. The package includes the following sub-packages to help with the processing and reduction of the data:
- A class describing a single HRS order, HRSOrder, that includes position, wavelength, and flux properties
- A class describing the HRS spectrograph, HRSModel, to allow for accurate modeling of the HRS Spectrograph
- Raw data can be processed using hrsprocess for both the blue and red arms.
- Orders can be identified in the images using create_orderframe
- wavelength_calibrate_arc can be used to calculate the wavelength calibration.
Once the data are reduced and calibrated, extraction of the object spectra can proceed based on the prefered method of the user.
Processing your data¶
For more information about how to process your data, please check out
Processing HRS Data¶
hrsprocess includes steps for the basic CCD processing necessary for HRS data. The code provides a wrapper for tasks from ccdproc to provide specific reductions for HRS data. In addition, it provides several functions for creating calibration frames for the reduction of HRS data.
Note
hrsprcess expects files to follow the SALT naming conventions
Processing Data frames¶
Data frames can be process using the tasks blue_process and red_process. The user can select from several options included in these programs, but certain aspects are hard wired to provide convenient functions for data reductions. For example, to process blue data:
>>> from pyhrs.hrsprocess import blue_process
>>> ccd = blue_process('H201411170015.fits', masterbias=masterbias)
This will return an ccdproc.CCDData object that has had the overscan corrected, trimmed, gain corrected, had the master bias subtracted, and positioned such that the orders increase from the bottom to the top and the dispersion goes from the left to the right. Flatfielding and calibration from a spectrophotometric standard will only be applied in later steps.
Convenience functions for Processing Science Data:
- blue_process: process data from the HRS blue camera
- red_process: process data from the HRS red camera
- hrs_process: convenience function for processing HRS data
The functions all pass appropriate parameters to ccd_process. This tasks wraps functions from ccdproc for processing CCD images. ccd_process has a number of steps, which are all optional, that include overscan subtraction, trimming, creating error frames, masking, gain correction, and subtracting a master bias.
Processing Calibration Frames¶
Calibration frames can also be created using several convenience functions. For example, passing a list of filenames to create_masterbias will process the data and combine them to create the master bias frame.
>>> from pyhrs.hrsprocess import create_masterbias
>>> masterbias = create_masterbias(['H201411170015.fits', 'H201411170016.fits']
This will process each frame and return a masterbias ccdproc.CCDData object.
In addition, masterflats can be produced using:
>>> from pyhrs.hrsprocess import create_masterflat >>> masterflat = create_masterflat(['H201411170020.fits', 'H201411170021.fits', 'H201411100022.fits']
Finding Orders¶
A critical step to the reduction of HRS data is finding orders in the images. Typically, images of flat fields or of a bright object can be used to identify orders in the frame. The data must first be processed. Unfortunately, a truly perfect data set for the identification of the orders is rarely available as the response function across the CCD can widely vary.
Below we outline some tasks that can be used for identifying orders in HRS images and the steps that are used to process the data to make the identification possible.
Normalizing a flat field frame¶
Due to the changing response function across the CCD, a fiber flat image can show significant vignetting in both the vertical and horizontal direction. To remove the vignetting in the vertical direction, the normalize_image task can be used.
The normalize_image task fits a function to a fiber flat image after masking the image. Either a 1D or 2D function can be fit to the image, and the fitting function should be specified by the user. For the best performance, it is best to either apply an already existing order frame or to smooth the image and mask areas of low response.
Here is an example of the steps needed to normalize an image:
>>> from astropy.io import fits
>>> from astropy import modeling as mod
>>> from scipy import ndimage as nd
>>> from pyhrs import normalize_image
>>> hdu = fits.open('HFLAT.fits')
>>> image = nd.filters.maximum_filter(image, 10)
>>> mask = (image > 500)
>>> norm = normalize_image(image, mod.models.Legendre1D(10), mask=mask)
>>> norm[norm<10000] = 0
This will produce a ndarry where good areas all have the same value and bad areas will have values set to zero. This significantly simplifies the process of identifying orders in the image. However, orders at the very top of the image with little or no signal will still be difficult to detect.
Creating an order frame¶
The next step is to create an order frame. An order frame is defined as an image where each pixel associated with an order is identified and labeled with that order. To produce the order frame, an initial detection kernal (based on a user input) is convolved with a single column in the image. The first maximum identified is assocatied with the initial input order given by the user. To identify the full 2D shape of the order, all pixels above a certain threshold and connected are identified. These pixels are then given the value of the initial order. Once the order is identified, all pixels associated with this order are set to zero. The detection kernal is then updated based on the 1D shape of this order, the order is incremented, and the process is repeated until all orders are identified in the frame.
All of these steps are accomplished running the create_orderframe task. An example of running this task is the follow:
>>> norm[norm>500] = 500
>>> xc = int(norm.shape[0]/2)
>>> detect_kern = norm[30:110, xc]
>>> frame = create_orderframe(norm, 84, xc, detect_kern, y_start=30, y_limit=4050)
This will produce the order frame for the blue arm in medium resolution mode. It will identify all orders up to the limit of y-position of 4050. For the red arm, the first order is 53 for the medium resolution mode.
Wavelength Calibration¶
Wavelength calibration requires the identification of known lines in a spectrum of an arc to determine the transformation between pixel space and wavelength.
HRSModel¶
HRSModel is a class for producing synthetic HRS spectra. HRSModel is based on the PySpectrograph.Spectrograph class. It includes a simple model for both arms that is based on the instrument confirguration and the spectrograph equation. After adjusting for offsets in the fiber position relative to the CCD, the model can return an approximation for the transformation that is accurate to within a few pixels.
The residuals between the model and the actual solution though are well described by a quadratic equation. This quadratic does slowly vary accross the orders and is different between the two arms. Due to the change in the fiber position between the different resolutions, this quadratic can change between the different configurations as well.
For these reasons, the initial guess for the wavelength solution is based on HRSModel plus a quadratic correction. The correction can either be calculated manual or by automatically fitting a single row of an order.
Calibrating a single order¶
To calibrate a single order, the following steps are carried out:
- Curvature due to the optical distortion is removed from the spectra and a square representation of the 2D spectra is created. Only integer shifts are applied to the data
- A model of the spectrograph is created based on the order, camera, and xpos offset that are supplied. A small correction described by a quadratic equation is added to the transformation calculated from the model.
- In each row of the data, peaks are extracted and matched with a line in the atlas of wavelengths that is provided. Due to the accuracy of the initial guess, lines are matched to within an angstrom and any line that might be blended is rejected.
- Once the first set of peaks and lines are matched up, a new solution is calculated for the given row. Then the processes of matching lines and determining a wavelength solution is repeated using this new solution. The best result from each line is saved.
- Using all of the matched lines from all lines, a ‘best’ solution is determined. Everything but the zeroth order parameter of the fit is fixed to a slowly varying value based on the overall solution to all lines. See fit_solution for more details.
- Based on the best solution found, the process is repeated for each row but only determing the zeropoint.
- Based on the solution found, a wavelength is assigned to each pixel
All of these steps are carried out by wavelength_calibrate_order. In the end, this task returns an HRSOrder object with wavelengths correspond to every pixel where a good solution was found. In addition, it also returns the x-position, wavelength, and solution for the initial row.
Calibrating an arc¶
For full automated calibration of an arc, wavelength_calibrate_arc can be used. In this task, it applies wavelength_calibrate_order to each of the orders in the frame. It uses pyhrs.HRSModel for the first guess but takes the quadratic correction from the solution of the nearest order. It starts with the initial order and the first row is also set by the user.
Processing HRS Data¶
hrsprocess includes steps for the basic CCD processing necessary for HRS data. The code provides a wrapper for tasks from ccdproc to provide specific reductions for HRS data. In addition, it provides several functions for creating calibration frames for the reduction of HRS data.
Note
hrsprcess expects files to follow the SALT naming conventions
Processing Data frames¶
Data frames can be process using the tasks blue_process and red_process. The user can select from several options included in these programs, but certain aspects are hard wired to provide convenient functions for data reductions. For example, to process blue data:
>>> from pyhrs.hrsprocess import blue_process
>>> ccd = blue_process('H201411170015.fits', masterbias=masterbias)
This will return an ccdproc.CCDData object that has had the overscan corrected, trimmed, gain corrected, had the master bias subtracted, and positioned such that the orders increase from the bottom to the top and the dispersion goes from the left to the right. Flatfielding and calibration from a spectrophotometric standard will only be applied in later steps.
Convenience functions for Processing Science Data:
- blue_process: process data from the HRS blue camera
- red_process: process data from the HRS red camera
- hrs_process: convenience function for processing HRS data
The functions all pass appropriate parameters to ccd_process. This tasks wraps functions from ccdproc for processing CCD images. ccd_process has a number of steps, which are all optional, that include overscan subtraction, trimming, creating error frames, masking, gain correction, and subtracting a master bias.
Processing Calibration Frames¶
Calibration frames can also be created using several convenience functions. For example, passing a list of filenames to create_masterbias will process the data and combine them to create the master bias frame.
>>> from pyhrs.hrsprocess import create_masterbias
>>> masterbias = create_masterbias(['H201411170015.fits', 'H201411170016.fits']
This will process each frame and return a masterbias ccdproc.CCDData object.
In addition, masterflats can be produced using:
>>> from pyhrs.hrsprocess import create_masterflat >>> masterflat = create_masterflat(['H201411170020.fits', 'H201411170021.fits', 'H201411100022.fits']
Finding Orders¶
A critical step to the reduction of HRS data is finding orders in the images. Typically, images of flat fields or of a bright object can be used to identify orders in the frame. The data must first be processed. Unfortunately, a truly perfect data set for the identification of the orders is rarely available as the response function across the CCD can widely vary.
Below we outline some tasks that can be used for identifying orders in HRS images and the steps that are used to process the data to make the identification possible.
Normalizing a flat field frame¶
Due to the changing response function across the CCD, a fiber flat image can show significant vignetting in both the vertical and horizontal direction. To remove the vignetting in the vertical direction, the normalize_image task can be used.
The normalize_image task fits a function to a fiber flat image after masking the image. Either a 1D or 2D function can be fit to the image, and the fitting function should be specified by the user. For the best performance, it is best to either apply an already existing order frame or to smooth the image and mask areas of low response.
Here is an example of the steps needed to normalize an image:
>>> from astropy.io import fits
>>> from astropy import modeling as mod
>>> from scipy import ndimage as nd
>>> from pyhrs import normalize_image
>>> hdu = fits.open('HFLAT.fits')
>>> image = nd.filters.maximum_filter(image, 10)
>>> mask = (image > 500)
>>> norm = normalize_image(image, mod.models.Legendre1D(10), mask=mask)
>>> norm[norm<10000] = 0
This will produce a ndarry where good areas all have the same value and bad areas will have values set to zero. This significantly simplifies the process of identifying orders in the image. However, orders at the very top of the image with little or no signal will still be difficult to detect.
Creating an order frame¶
The next step is to create an order frame. An order frame is defined as an image where each pixel associated with an order is identified and labeled with that order. To produce the order frame, an initial detection kernal (based on a user input) is convolved with a single column in the image. The first maximum identified is assocatied with the initial input order given by the user. To identify the full 2D shape of the order, all pixels above a certain threshold and connected are identified. These pixels are then given the value of the initial order. Once the order is identified, all pixels associated with this order are set to zero. The detection kernal is then updated based on the 1D shape of this order, the order is incremented, and the process is repeated until all orders are identified in the frame.
All of these steps are accomplished running the create_orderframe task. An example of running this task is the follow:
>>> norm[norm>500] = 500
>>> xc = int(norm.shape[0]/2)
>>> detect_kern = norm[30:110, xc]
>>> frame = create_orderframe(norm, 84, xc, detect_kern, y_start=30, y_limit=4050)
This will produce the order frame for the blue arm in medium resolution mode. It will identify all orders up to the limit of y-position of 4050. For the red arm, the first order is 53 for the medium resolution mode.
Wavelength Calibration¶
Wavelength calibration requires the identification of known lines in a spectrum of an arc to determine the transformation between pixel space and wavelength.
HRSModel¶
HRSModel is a class for producing synthetic HRS spectra. HRSModel is based on the PySpectrograph.Spectrograph class. It includes a simple model for both arms that is based on the instrument confirguration and the spectrograph equation. After adjusting for offsets in the fiber position relative to the CCD, the model can return an approximation for the transformation that is accurate to within a few pixels.
The residuals between the model and the actual solution though are well described by a quadratic equation. This quadratic does slowly vary accross the orders and is different between the two arms. Due to the change in the fiber position between the different resolutions, this quadratic can change between the different configurations as well.
For these reasons, the initial guess for the wavelength solution is based on HRSModel plus a quadratic correction. The correction can either be calculated manual or by automatically fitting a single row of an order.
Calibrating a single order¶
To calibrate a single order, the following steps are carried out:
- Curvature due to the optical distortion is removed from the spectra and a square representation of the 2D spectra is created. Only integer shifts are applied to the data
- A model of the spectrograph is created based on the order, camera, and xpos offset that are supplied. A small correction described by a quadratic equation is added to the transformation calculated from the model.
- In each row of the data, peaks are extracted and matched with a line in the atlas of wavelengths that is provided. Due to the accuracy of the initial guess, lines are matched to within an angstrom and any line that might be blended is rejected.
- Once the first set of peaks and lines are matched up, a new solution is calculated for the given row. Then the processes of matching lines and determining a wavelength solution is repeated using this new solution. The best result from each line is saved.
- Using all of the matched lines from all lines, a ‘best’ solution is determined. Everything but the zeroth order parameter of the fit is fixed to a slowly varying value based on the overall solution to all lines. See fit_solution for more details.
- Based on the best solution found, the process is repeated for each row but only determing the zeropoint.
- Based on the solution found, a wavelength is assigned to each pixel
All of these steps are carried out by wavelength_calibrate_order. In the end, this task returns an HRSOrder object with wavelengths correspond to every pixel where a good solution was found. In addition, it also returns the x-position, wavelength, and solution for the initial row.
Calibrating an arc¶
For full automated calibration of an arc, wavelength_calibrate_arc can be used. In this task, it applies wavelength_calibrate_order to each of the orders in the frame. It uses pyhrs.HRSModel for the first guess but takes the quadratic correction from the solution of the nearest order. It starts with the initial order and the first row is also set by the user.
Extracting Spectra¶
PyHRS¶
The PyHRS package is for the reduction of data from the High Resolution Spectrograph on the Southern African Large Telescope. The goals of the package are to provide tools to be able to produce scientific quality reductions for the the low, medium, and high resolution modes for HRS and to prepare data for more specialized code for the reduction of high stability observations.
The package includes the following classes and functions: - HRSModel - hrsprocess - HRSOrder - hrstools
HRSModel¶
HRSModel is a class for producing synthetic HRS spectra. HRSModel is based on the PySpectrograph.Spectrograph class. It only includes a simple model based on the instrument confirguration and the spectrograph equation.
hrs_process¶
hrsprocess includes steps for the basic CCD processing necessary for HRS data. It also includes steps necessary for creating calibration frames.
HRSOrder¶
HRSOrder is a class descirbe a single order from an HRS image. The order then has different tools for identifying regions, extracting orders, and defining properties of different orders such as wavelengths and calibrations.
hrsextract¶
hrsextract includes all steps necessary to extract a single, one-dimensional HRS spectrum.
HRStools¶
HRStools includes generally utilies used across different functions and classes.
Reference/API¶
pyhrs Module¶
pyhrs is a package for reducing data from the High Resolution Spectrograph on the Southern African Large Telescope
| background(b_arr[, niter]) | Determine the background for an array |
| blue_process(infile[, masterbias, error, ...]) | Process a blue frame |
| calc_weights(x, y, m[, yerr]) | Calculate weights for each value based on deviation from best fit model |
| ccd_process(ccd[, oscan, trim, error, ...]) | Perform basic processing on ccd data. |
| create_masterbias(image_list) | Create a master bias frame from a list of images |
| create_masterflat(image_list[, masterbias]) | Create a master flat frame from a list of images |
| create_orderframe(data, first_order, xc, ...) | Create an order frame from from an observation. |
| fit_order(data, detect_kernal, xc[, order, ...]) | Given an array and an overlapping detect_kernal, |
| fit_wavelength_solution(sol_dict) | Determine the best fit solution and re-fit each line with that solution |
| hrs_process(image_name[, ampsec, oscansec, ...]) | Processing required for HRS observations. |
| iterfit1D(x, y, fitter, model[, yerr, ...]) | Iteratively fit a function. |
| match_lines(xarr, farr, sw, sf, ws[, rw, ...]) | Match lines in the spectra with specific wavleengths |
| ncor(x, y) | Calculate the normalized correlation of two arrays |
| normalize_image(data, func_init, mask[, ...]) | Normalize an HRS image. |
| red_process(infile[, masterbias, error, rdnoise]) | Process a blue frame |
| test([package, test_path, args, plugins, ...]) | Run the tests using py.test. |
| wavelength_calibrate_arc(arc, order_frame, ...) | Wavelength calibrate an arc spectrum from HRS |
| wavelength_calibrate_order(hrs, slines, ...) | Wavelength calibration of a single order from the HRS arc spectra |
| xcross_fit(warr, farr, sw_arr, sf_arr[, dw, nw]) | Calculate a zeropoint shift between the observed arc |
| CCD([name, height, width, xpos, ypos, ...]) | Defines a CCD by x and y position, size, and pixel size. |
| Detector([name, ccd, zpos, xpos, ypos, ...]) | A class that describing the Detector. |
| Grating([name, spacing, order, height, ...]) | A class that describing gratings. |
| HRSModel([grating_name, camera_name, slit, ...]) | HRSModel is a class that describes the High Resolution Specotrgraph on SALT |
| HRSOrder(order[, region, flux, wavelength, ...]) | A class describing a single order for a High Resolutoin Spectrograph observation. |
| Optics([name, diameter, focallength, width, ...]) | A class that describing optics. |
| Slit([name, height, width, zpos, xpos, ...]) | A class that describing the slit. |
| Spectrograph([camang, gratang, grating, ...]) | A class describing a spectrograph and functions related to a spectrograph. |
| SpectrographError | Exception Raised for Spectrograph errors |
