Overview of the filtering process

Coverage filtering

All positions in every sample are filtered simultaneously, whether the genotype if REF or ALT in VCF. Any position will be masked (genotype is set to unknown, GT=.) if the coverage is lower than minimum_coverage_for_calling.

Frequency filtering

Filtering over the frequency of observation at each position is processed separately for each sample. Multiallelic entries from the VCF are splitted and normalized with vt. Then, for alternative genotype entries, the genotype is set to unknown if the ratio of observation of alternate base(s) (FORMAT/AD[0:1] in the VCF) over the total coverage of the position (FORMAT/DP) is lower than minimum_alternate_fraction_for_calling. Similarly, for reference genotype entries, the genotype is set to unknown if the ratio of the number of observations of reference base(s) (FORMAT/AD[0:0]) over the total coverage of the position (FORMAT/DP) is lower than minimum_alternate_fraction_for_calling. After the filtering is done, all samples are merged back together in the same VCF.

Type filtering

Entries in the VCF are filtered according to their type. We preferentially keep only entries defined as single nucleotide polymorphisms, but indel or bnd can also be extracted from the genotyping of freebayes, or indel or mixed from the genotyping of GATK.

Core genome filtering

Core genome filtering is performed at the end. Using any bed file defined using the mapping reference, only the positions in the filtered final VCF overlapping the bed regions are kept. See Core genome determination for the bed file definitions.