Options
This page details all the available options. This information is also
available on the command line by invoking goquartical
or
goquartical help
. The following is broken up by configuration section.
Note
These options can be specified either via command line or .yaml file e.g.:
input_ms.path=path/to.ms
or
input_ms:
path: path/to.ms
input_ms
Options pertaining to the input measurement set.
input_ms.path
Path to input measurement set.
Default: N/A
input_ms.data_column
Name of column to use as data.
Default: DATA
input_ms.sigma_column
When given, the weights will be reinitialized as 1/sigma_column**2. This should be preferred over weight_column to ensure that the correct data weights are used and not those from a previous calibration run. Mutually exclusive with input_ms.weight_column.
Default: N/A
input_ms.weight_column
Column to read weights from. An empty string will result in all weights being treated as unity if sigma_column is not set. Mutually exclusive with input_ms.sigma_column (which is preferred).
Default: N/A
input_ms.time_chunk
Chunk data up by this number of timeslots. This limits the amount of data processed at once. Smaller chunks allow for a smaller RAM footprint and greater parallelism, but this sets an upper limit on the solution intervals that may be employed. Specify as an integer number of timeslots, or a value with a unit (e.g. ‘300s’). 0 means use full time axis.
Default: 0
input_ms.freq_chunk
Chunk data by this number of channels. See time_chunk for info. Specify as an integer number of channels, or a value with a unit (e.g. ‘128MHz’). 0 means use full frequency axis.
Default: 0
input_ms.is_bda
If set True, the input measurement set is assumed to have been averaged in a baseline dependent fashion.
Default: False
input_ms.group_by
Input data will be partitioned into separate xarray datasets based on the values of the specified columns. Multiple column names may be given as a list, e.g. [SCAN_NUMBER, FIELD_ID, DATA_DESC_ID].
Choices:
SCAN_NUMBER
FIELD_ID
DATA_DESC_ID
Default: [‘SCAN_NUMBER’, ‘FIELD_ID’, ‘DATA_DESC_ID’]
input_ms.select_corr
Select correlations from the input data. These are specified as integer values and must be given as a list e.g. to select the first and last correlations in a measurement set with four correlations, use [0, 3].
Choices:
0
1
2
3
Default: N/A
input_ms.select_fields
Select fields from the input data. These are specified as integer values and must be given as a list e.g. to select fields 2 and 6 use [2, 6].
Default: []
input_ms.select_ddids
Select data descriptor IDs (spectral windows) from the input data. These are specified as integer values and must be given as a list e.g. to select ddids/SPWs 0 and 2, use [0, 2].
Default: []
input_ms.select_uv_range
Select a range of uv values to include when calibrating. Practically, this treats points outside the range as having zero weight. Use zero to indicate an open interval, e.g. [100, 0] would select all values greater than 100, [0, 10000] would select all values less than 10000 and [100, 10000] would select values greater than 100 but less than 10000.
Default: [0, 0]
input_model
Options pertaining to the input model.
input_model.recipe
Input model recipe. This is a string which describes how the model data should be constructed. Currently, measurement set columns and Tigger lsms are supported. Multiple columns and lsms can be specified and combined using colons (:), addition (+) and negation (~). As an example consider COL1, COL2 and LSM1. “COL1” will simply use a single MS column as input. “COL1:COL2” will create a direction dependent model with COL1 as the first direction and COL2 as the second. “COL1~COL2” will create a model with a single direction by subtracting COL2 from COL1. “COL1+COL2” will create a model with a single direction by adding COL1 and COL2. Tigger lsms can be used interchangeably with columns (using the same syntax) but also support the use of tagged directions. “LSM1” will create a model with a single direction from the sky model. “LSM1@dE” will create a model with multiple directions, one for each cluster tagged with dE in the lsm. Combining all of the above it is possible to do things like “COL1~LSM1:LSM1@dE” which will create a model with multiple directions, the first given by COL1 minus LSM1 and the remainder given given by the dE tagged clusters in LSM1. Leaving this value unset (the default) will use an identity model.
Default: N/A
input_model.beam
Path to beams. Apply beams during predict if specified eg. ‘beam_$(corr)_$(reim).fits’ or ‘beam_$(CORR)_$(REIM).fits’.
Default: N/A
input_model.beam_l_axis
Determines the orientation of the beam l-axis. Note that ~ indicates flipping the orientation of the axis.
Choices:
X
~X
Y
~Y
L
~L
M
~M
Default: X
input_model.beam_m_axis
Determines the orientation of the beam m-axis. See beam_l_axis.
Choices:
X
~X
Y
~Y
L
~L
M
~M
Default: Y
input_model.invert_uvw
The UVW coordinates will be negated if this option is specified. Enabled by default to match the Casa convention.
Default: True
input_model.source_chunks
The number of sources to predict simultaneously. Has a large impact on memory footprint.
Default: 500
input_model.apply_p_jones
Determines whether P-Jones (parallactic angle rotation) is applied to the model. This affects both measurement set columns and predicted components. Care must taken when using this option and output.apply_p_jones_inv.
Default: False
output
Options pertaining to output.
output.gain_directory
Name of directory in which QuartiCal gain outputs will be stored. Accepts both local and s3 paths. QuartiCal will always produce gain outputs.
Default: gains.qc
output.log_directory
Name of directory in which QuartiCal logging outputs will be stored. s3 is not currently supported for these outputs.
Default: logs.qc
output.log_to_terminal
Enable or disable logging to terminal.
Default: True
output.overwrite
Whether or not the contents of the output directory may be overwritten. Will trigger an error when False and output.directory already exists.
Default: False
output.products
The desired output data products. Multiple data products can be specified as a list e.g. [residual, corrected_residual]. Any required measurement set outputs should be specified here. Note that the output names of the desired products should be specified via output.columns.
Choices:
corrected_data
corrected_residual
residual
weight
corrected_weight
model_data
Default: N/A
output.columns
Output MS column names for visibility outputs (if applicable). Column names will be used in order, matching the order of output.products. Multiple columns can be specified as a list e.g. [COL1, COL2, COL3].
Default: N/A
output.flags
If True, write out flags to FLAG and FLAG_ROW. This can be disabled if you want QuartiCal to leave the measurement set flags unaltered.
Default: True
output.apply_p_jones_inv
Determines whether the inverse of P-Jones (parallactic angle rotation) is applied to the output visibilitites. This has the effect of derotating the output visibilities into the sky frame. Care must taken when using this option and input_model.apply_p_jones.
Default: False
output.subtract_directions
Which model directions to subtract when generating residuals. Must be specified as a list of integers e.g. [0, 5, 7]. The default will subtract all directions.
Default: N/A
output.net_gains
Merge subsets of gains into an effective/net gain per antenna per time per channel. This is formed by multiplying all the specified gains together. This can be used to reduce the computational load of solution transfer by transferring net/effective gains rather than each individual term. Accepts a list or a list of lists e.g. [“K”, “G”, “X”, “B”] or [[“K”, “G”], [“X”, “B”]]. Results will be written to outputs.gain_directory as e.g. KGXB-net.
Default: N/A
output.compute_baseline_corrections
Enable or disable computation of baseline-based corrections. Functionality is currently limited to a solution per-channel, per-chunk. These solutions are useful for analysis and are stored in output.gain_directory.
Default: False
output.apply_baseline_corrections
Enable or disable application of baseline-based corrections. Extreme caution advised - this can and will lead to overfitting.
Default: False
mad_flags
Options pertaining to MAD (Median Absolute Deviation) flagging.
mad_flags.enable
Enables the MAD flagging routines.
Default: False
mad_flags.whitening
Determines whether and how the residuals are whitened (multiplied by the square root of the weights) prior to performing MAD flagging. “native” whitening will use the original weights (specified in the input_ms section). “robust” will use the weights produced when solver.robust=True. “disabled” will result in the MAD estimate being performed on the unwhitened residuals.
Default: disabled
mad_flags.threshold_bl
Multiplicative factor which determines whether or not a chi-squared value is considered to deviate significantly from the median of a given baseline. Values greater than MAD_bl*threshold_bl will be flagged. Set to zero to disable flagging on this statistic.
Default: 5
mad_flags.threshold_global
Multiplicative factor which determines whether or not a chi-squared value is considered to deviate significantly from the median of a given data chunk. Values greater than MAD*threshold_global will be flagged. Set to zero to disable flagging on this statistic.
Default: 10
mad_flags.max_deviation
Multiplicative factor which determines whether or not the MAD estimate on a given baseline is considered to deviate too much from the global MAD estimate. If the MAD estimate over all baselines is X, and the MAD estimate on a specific baseline is X_bl, baselines for which X_bl > max_deviation*X will be flagged. Set to zero to disable flagging on this statistic.
Default: 0
mad_flags.use_off_diagonals
Controls whether or not the mad flagger will be run on the off-diagonal elements of the residual. This is disabled by default as the residual will tend to contain structure in the absence of a polarised model and adequate leakage calibration.
Default: False
dask
Options pertaining to Dask (and therefore parallelism).
dask.threads
Number of threads to use in the dask scheduler. Setting to zero (the default) will use all available resources.
Default: N/A
dask.workers
Number of workers to use in the dask distributed scheduler. Advanced users only.
Default: 1
dask.address
Distributed scheduler address.
Default: N/A
dask.scheduler
Which dask scheduler to use. The default, threads, is the most appropriate for non-cluster use.
Choices:
threads
single-threaded
distributed
Default: threads
dask.scheduler_plugin
Enable or disable the dask scheduler plugin.
Default: True
solver
Options pertaining to all solvers (as opposed to specific terms).
solver.terms
Gain terms for which we are solving. Multiple terms can be specified as a list e.g. [G,B,dE]. Each term specified here has its own set of arguments which can be specified as (gain).(option). e.g. G.time_interval.
Default: [‘G’]
solver.iter_recipe
Specifies the iterations to be performed per gain term. This argument expects a list as long or longer than solver.terms. If solver.terms was given as [K,G,B], an iteration recipe of [20,10,5] would do 20 iterations for K, 10 for G and 5 for B. To loop over the gains multiple times, use a longer list e.g. [20,10,5,15,5,0] would do the same as the first example but then do an additional 15 iterations for K, 5 for G and skip B. Setting to zero effectively disables solving for that term and can be used in conjunction with iterpolation.
Default: [25]
solver.propagate_flags
Controls whether gain flags/flags raised inside the solver are propagated and ultimately written to the FLAG column. This should almost always be enabled so that data associated with diverging gains is properly flagged.
Default: True
solver.robust
Enable robust reweighting in solvers. Note that this only works when the solver.iter_recipe loops through the chain multiple times. The reweighting step only happens once per traversal of the chain.
Default: False
solver.threads
Number of Numba threads per Dask thread (enables nested parallelism) to be used when running the solvers. The total number of threads used will be dask.threads*solver.threads; if this product exceeds the number of available threads, performance will suffer.
Default: 1
solver.convergence_fraction
The fraction of gain values which must converge before a solver will exit prematurely.
Default: 0.99
solver.convergence_criteria
The change in the value of the gain below which it considered to have converged. Set to zero to iterate for the number of interations specified in solver.iter_recipe.
Default: 1e-06
solver.reference_antenna
A reference antenna to use for terms which require one. QuartiCal will also guarantee zero phase on the specified antenna for diagonal terms. Specify as the integer index of the antenna - antenna names are not currently supported.
Default: 0
gain
Options pertaining to a specific gain/Jones term.
Warning
This help is generic - users will not typically write gain.option
but
will instead use the labels specified by solver.gain_terms
. Thus, for
solver.gain_terms="[G,B]"
, options would be specified using
G.option
or B.option
.
gain.type
Type of gain to solve for.
Choices:
complex
diag_complex
amplitude
delay
delay_and_offset
phase
tec_and_offset
rotation_measure
rotation
crosshand_phase
leakage
Default: complex
gain.solve_per
Determines whether this term should be solved per antenna (conventional) or over the entire array (doesn’t vary with antenna).
Choices:
antenna
array
Default: antenna
gain.direction_dependent
Determines whether this term is treated as direction dependent.
Default: False
gain.pinned_directions
If this term is direction dependent, this can be used to exclude integer indexed directions during gain updates i.e. effectively disable updates in the given directions. Accepts a list of integer values e.g. ‘[0,3]’.
Default: [0]
gain.time_interval
Number of timeslots/amount of time to include in a single solution. Specify as an integer number of timeslots, or a value with a unit (e.g. ‘300s’). 0 means use full time axis.
Default: 1
gain.freq_interval
Number of channels/bandwidth to include in a single solution. Specify as an integer number of channels, or a value with a unit (e.g. ‘128MHz’). 0 means use full frequency axis.
Default: 1
gain.load_from
Load solutions from given database.
Default: N/A
gain.interp_mode
Set interpolation mode. THIS OPTION IS CURRENTLY UNAVAILABLE.
Choices:
reim
ampphase
amp
phase
Default: reim
gain.interp_method
Set interpolation method.
Choices:
2dlinear
2dspline
Default: 2dlinear
gain.respect_scan_boundaries
Determines whether solution intervals may span multiple scans. This only works when input_ms.group_by does not include SCAN_NUMBER. Can be used in conjunction with time_interval to solve a term per scan even when data is not partitioned by scan (by setting this to True and time_interval to 0).
Default: True
gain.initial_estimate
Controls whether or not a term will be populated with an initial estimiate where applicable. Currently only supported for delay terms.
Default: False