SAM (file format)

Last updated
SAM file format
Filename extension
.sam
Developed by
Type of format Bioinformatics
Extended from Tab-separated values
Website samtools.github.io/hts-specs/

Sequence Alignment Map (SAM) is a text-based format originally for storing biological sequences aligned to a reference sequence developed by Heng Li and Bob Handsaker et al. [1] It was developed when the 1000 Genomes Project wanted to move away from the MAQ mapper format and decided to design a new format. The overall TAB-delimited flavour of the format came from an earlier format inspired by BLAT’s PSL. The name of SAM came from Gabor Marth from University of Utah, who originally had a format under the same name but with a different syntax more similar to a BLAST output. [2] It is widely used for storing data, such as nucleotide sequences, generated by next generation sequencing technologies, and the standard has been broadened to include unmapped sequences. The format supports short and long reads (up to 128 Mbp [3] ) produced by different sequencing platforms and is used to hold mapped data within the Genome Analysis Toolkit (GATK) and across the Broad Institute, the Wellcome Sanger Institute, and throughout the 1000 Genomes Project.

Contents

Format

The SAM format consists of a header and an alignment section. [1] The binary equivalent of a SAM file is a Binary Alignment Map (BAM) file, which stores the same data in a compressed binary representation. [4] SAM files can be analysed and edited with the software SAMtools. [1] The header section must be prior to the alignment section if it is present. Headings begin with the '@' symbol, which distinguishes them from the alignment section. Alignment sections have 11 mandatory fields, as well as a variable number of optional fields. [1]

ColFieldTypeBrief description
1QNAMEStringQuery template NAME
2FLAGIntbitwise FLAG
3RNAMEStringReferences sequence NAME
4POSInt1- based leftmost mapping POSition
5MAPQIntMAPping Quality
6CIGARString CIGAR string
7RNEXTStringRef. name of the mate/next read
8PNEXTIntPosition of the mate/next read
9TLENIntobserved Template LENgth
10SEQStringsegment SEQuence
11QUALStringASCII of Phred-scaled base QUALity+33

Description

From the specification: [4]

  1. QNAME: Query template NAME. Reads/segments having identical QNAME are regarded to come from the same template. A QNAME ‘*’ indicates the information is unavailable. In a SAM file, a read may occupy multiple alignment lines, when its alignment is chimeric or when multiple mappings are given.
  2. FLAG: Combination of bitwise FLAGs [5]
  3. RNAME: Reference sequence NAME of the alignment. If @SQ header lines are present, RNAME (if not ‘*’) must be present in one of the SQ-SN tag. An unmapped segment without coordinate has a ‘*’ at this field. However, an unmapped segment may also have an ordinary coordinate such that it can be placed at a desired position after sorting. If RNAME is ‘*’, no assumptions can be made about POS and CIGAR.
  4. POS: 1-based leftmost mapping POSition of the first matching base. The first base in a reference sequence has coordinate 1. POS is set as 0 for an unmapped read without coordinate. If POS is 0, no assumptions can be made about RNAME and CIGAR.
  5. MAPQ: MAPping Quality. It equals −10 log10 Pr{mapping position is wrong}, rounded to the nearest integer. A value 255 indicates that the mapping quality is not available.
  6. CIGAR: Concise Idiosyncratic Gapped Alignment Report (CIGAR) string.
  7. RNEXT: Reference sequence name of the primary alignment of the NEXT read in the template. For the last read, the next read is the first read in the template. If @SQ header lines are present, RNEXT (if not ‘*’ or ‘=’) must be present in one of the SQ-SN tag. This field is set as ‘*’ when the information is unavailable, and set as ‘=’ if RNEXT is identical RNAME. If not ‘=’ and the next read in the template has one primary mapping (see also bit 0x100 in FLAG), this field is identical to RNAME at the primary line of the next read. If RNEXT is ‘*’, no assumptions can be made on PNEXT and bit 0x20.
  8. PNEXT: Position of the primary alignment of the NEXT read in the template. Set as 0 when the information is unavailable. This field equals POS at the primary line of the next read. If PNEXT is 0, no assumptions can be made on RNEXT and bit 0x20.
  9. TLEN: signed observed Template LENgth. If all segments are mapped to the same reference, the unsigned observed template length equals the number of bases from the leftmost mapped base to the rightmost mapped base. The leftmost segment has a plus sign and the rightmost has a minus sign. The sign of segments in the middle is undefined. It is set as 0 for single-segment template or when the information is unavailable.
  10. SEQ: segment SEQuence. This field can be a ‘*’ when the sequence is not stored. If not a ‘*’, the length of the sequence must equal the sum of lengths of M/I/S/=/X operations in CIGAR. An ‘=’ denotes the base is identical to the reference base. No assumptions can be made on the letter cases.
  11. QUAL: ASCII of base QUALity plus 33 (same as the quality string in the Sanger FASTQ format). A base quality is the phred-scaled base error probability which equals −10 log10 Pr{base is wrong}. This field can be a ‘*’ when quality is not stored. If not a ‘*’, SEQ must not be a ‘*’ and the length of the quality string ought to equal the length of SEQ.

Bitwise flags

The FLAG field is displayed as a single integer, but is the sum of bitwise flags to denote multiple attributes of a read alignment. [4] Each attribute denotes one bit in the binary representation of the integer.

Bitwise Flags
IntegerBinaryDescription (Paired Read Interpretation)
1000000000001template having multiple templates in sequencing (read is paired)
2000000000010each segment properly aligned according to the aligner (read mapped in proper pair)
4000000000100segment unmapped (read1 unmapped)
8000000001000next segment in the template unmapped (read2 unmapped)
16000000010000SEQ being reverse complemented (read1 reverse complemented)
32000000100000SEQ of the next segment in the template being reverse complemented (read2 reverse complemented)
64000001000000the first segment in the template (is read1)
128000010000000the last segment in the template (is read2)
256000100000000not primary alignment
512001000000000alignment fails quality checks
1024010000000000PCR or optical duplicate
2048100000000000supplementary alignment (e.g. aligner specific, could be a portion of a split read or a tied region)

The FLAG attributes are summed to get the final value, e.g. a SAM row resulting from an Illumina paired-end FASTQ record having the FLAG value 2145 would indicate:

Flag ValueMeaningFlag Sum
1read is paired1
32read2 was reverse complemented33
64read197
2048Supplementary alignment2145

Optional fields

From the specification for Sequence Alignment/Map Optional Fields Specification (SAMtags): [6]

The type may be one of A (character), C (integer 0-255), f (real number), H (hexadecimal array), i (integer), or Z (string). It may be a single value or B (general array).

TagTypeDescription
AMiThe smallest template-independent mapping quality in the template
ASiAlignment score generated by aligner
BCZBarcode sequence identifying the sample
BQZOffset to base alignment quality (BAQ)
BZZPhred quality of the unique molecular barcode bases in the OX tag
CBZCell identifier
CCZReference name of the next hit
CGB,IBAM only: CIGAR in BAM's binary encoding if (and only if) it consists of >65535 operators
CMiEdit distance between the color sequence and the color reference (see also NM)
COZFree-text comments
CPiLeftmost coordinate of the next hit
CQZColor read base qualities
CRZCellular barcode sequence bases (uncorrected)
CSZColor read sequence
CTZComplete read annotation tag, used for consensus annotation dummy features
CYZPhred quality of the cellular barcode sequence in the CR tag
E2ZThe 2nd most likely base calls
FIiThe index of segment in the template
FSZSegment suffix
FZB,SFlow signal intensities
GC?Reserved for backwards compatibility reasons
GQ?Reserved for backwards compatibility reasons
GS?Reserved for backwards compatibility reasons
H0iNumber of perfect hits
H1iNumber of 1-difference hits (see also NM)
H2iNumber of 2-difference hits
HIiQuery hit index
IHiQuery hit total count
LBZLibrary
MCZCIGAR string for mate/next segment
MDZString for mismatching positions
MF?Reserved for backwards compatibility reasons
MIZMolecular identifier; a string that uniquely identifies the molecule from which the record was derived
MLB, CBase modification probabilities
MMZBase modifications / methylation
MQiMapping quality of the mate/next segment
NHiNumber of reported alignments that contain the query in the current record
NMiEdit distance to the reference
OAZOriginal alignment
OCZOriginal CIGAR (deprecated; use OA instead)
OPiOriginal mapping position (deprecated; use OA instead)
OQZOriginal base quality
OXZOriginal unique molecular barcode bases
PGZProgram
PQiPhred likelihood of the template
PTZRead annotations for parts of the padded read sequence
PUZPlatform unit
Q2ZPhred quality of the mate/next segment sequence in the R2 tag
QTZPhred quality of the sample barcode sequence in the BC tag
QXZQuality score of the unique molecular identifier in the RX tag
R2ZSequence of the mate/next segment in the template
RGZRead group
RT?Reserved for backwards compatibility reasons
RXZSequence bases of the (possibly corrected) unique molecular identifier
S2?Reserved for backwards compatibility reasons
SAZOther canonical alignments in a chimeric alignment
SMiTemplate-independent mapping quality
SQ?Reserved for backwards compatibility reasons
TCiThe number of segments in the template
U2ZPhred probability of the 2nd call being wrong conditional on the best being wrong
UQiPhred likelihood of the segment, conditional on the mapping being correct
X??Reserved for end users
Y??Reserved for end users
Z??Reserved for end users

See also

Related Research Articles

<span class="mw-page-title-main">Sequence alignment</span> Process in bioinformatics that identifies equivalent sites within molecular sequences

In bioinformatics, a sequence alignment is a way of arranging the sequences of DNA, RNA, or protein to identify regions of similarity that may be a consequence of functional, structural, or evolutionary relationships between the sequences. Aligned sequences of nucleotide or amino acid residues are typically represented as rows within a matrix. Gaps are inserted between the residues so that identical or similar characters are aligned in successive columns. Sequence alignments are also used for non-biological sequences such as calculating the distance cost between strings in a natural language, or to display financial data.

In bioinformatics and biochemistry, the FASTA format is a text-based format for representing either nucleotide sequences or amino acid (protein) sequences, in which nucleotides or amino acids are represented using single-letter codes.

The printf family of functions in the C programming language are a set of functions that take a format string as input among a variable sized list of other values and produce as output a string that corresponds to the format specifier and given input values. The string is written in a simple template language: characters are usually copied literally into the function's output, but format specifiers, which start with a % character, indicate the location and method to translate a piece of data to characters. The design has been copied to expose similar functionality in other programming languages.

In bioinformatics, sequence assembly refers to aligning and merging fragments from a longer DNA sequence in order to reconstruct the original sequence. This is needed as DNA sequencing technology might not be able to 'read' whole genomes in one go, but rather reads small pieces of between 20 and 30,000 bases, depending on the technology used. Typically, the short fragments (reads) result from shotgun sequencing genomic DNA, or gene transcript (ESTs).

Mach-O, short for Mach object file format, is a file format for executables, object code, shared libraries, dynamically loaded code, and core dumps. It was developed to replace the a.out format.

<span class="mw-page-title-main">UCSF Chimera</span>

UCSF Chimera is an extensible program for interactive visualization and analysis of molecular structures and related data, including density maps, supramolecular assemblies, sequence alignments, docking results, trajectories, and conformational ensembles. High-quality images and movies can be created. Chimera includes complete documentation and can be downloaded free of charge for noncommercial use.

<span class="mw-page-title-main">Phred quality score</span> Measurement in DNA sequencing

A Phred quality score is a measure of the quality of the identification of the nucleobases generated by automated DNA sequencing. It was originally developed for the computer program Phred to help in the automation of DNA sequencing in the Human Genome Project. Phred quality scores are assigned to each nucleotide base call in automated sequencer traces. The FASTQ format encodes phred scores as ASCII characters alongside the read sequences. Phred quality scores have become widely accepted to characterize the quality of DNA sequences, and can be used to compare the efficacy of different sequencing methods. Perhaps the most important use of Phred quality scores is the automatic determination of accurate, quality-based consensus sequences.

The Binary Universal Form for the Representation of meteorological data (BUFR) is a binary data format maintained by the World Meteorological Organization (WMO). The latest version is BUFR Edition 4. BUFR Edition 3 is also considered current for operational use. BUFR was created in 1988 with the goal of replacing the WMO's dozens of character-based, position-driven meteorological codes, such as SYNOP, TEMP and CLIMAT. BUFR was designed to be portable, compact, and universal. Any kind of data can be represented, along with its specific spatial/temporal context and any other associated metadata. In the WMO terminology, BUFR belongs to the category of table-driven code forms, where the meaning of data elements is determined by referring to a set of tables that are kept and maintained separately from the message itself.

<span class="mw-page-title-main">UGENE</span>

UGENE is computer software for bioinformatics. It works on personal computer operating systems such as Windows, macOS, or Linux. It is released as free and open-source software, under a GNU General Public License (GPL) version 2.

FASTQ format is a text-based format for storing both a biological sequence and its corresponding quality scores. Both the sequence letter and quality score are each encoded with a single ASCII character for brevity.

The Variant Call Format (VCF) is a standard text file format used in bioinformatics for storing gene sequence variations. The format was developed in 2010 for the 1000 Genomes Project and has since been used by other large-scale genotyping and DNA sequencing projects. VCF is a common output format for variant calling programs due to its relative simplicity and scalability. Many tools have been developed for editing and manipulating VCF files, including VCFtools, which was released in conjunction with the VCF format in 2011, and BCFtools, which was included as part of SAMtools until being split into an independent package in 2014.

<span class="mw-page-title-main">DNA nanoball sequencing</span>

DNA nanoball sequencing is a high throughput sequencing technology that is used to determine the entire genomic sequence of an organism. The method uses rolling circle replication to amplify small fragments of genomic DNA into DNA nanoballs. Fluorescent nucleotides bind to complementary nucleotides and are then polymerized to anchor sequences bound to known sequences on the DNA template. The base order is determined via the fluorescence of the bound nucleotides This DNA sequencing method allows large numbers of DNA nanoballs to be sequenced per run at lower reagent costs compared to other next generation sequencing platforms. However, a limitation of this method is that it generates only short sequences of DNA, which presents challenges to mapping its reads to a reference genome. After purchasing Complete Genomics, the Beijing Genomics Institute (BGI) refined DNA nanoball sequencing to sequence nucleotide samples on their own platform.

<span class="mw-page-title-main">XQuery API for Java</span> Application programming interface

XQuery API for Java (XQJ) refers to the common Java API for the W3C XQuery 1.0 specification.

Pileup format is a text-based format for summarizing the base calls of aligned reads to a reference sequence. This format facilitates visual display of SNP/indel calling and alignment. It was first used by Tony Cox and Zemin Ning at the Wellcome Trust Sanger Institute, and became widely known through its implementation within the SAMtools software suite.

SAMtools is a set of utilities for interacting with and post-processing short DNA sequence read alignments in the SAM, BAM and CRAM formats, written by Heng Li. These files are generated as output by short read aligners like BWA. Both simple and advanced tools are provided, supporting complex tasks like variant calling and alignment viewing as well as sorting, indexing, data extraction and format conversion. SAM files can be very large, so compression is used to save space. SAM files are human-readable text files, and BAM files are simply their binary equivalent, whilst CRAM files are a restructured column-oriented binary container format. BAM files are typically compressed and more efficient for software to work with than SAM. SAMtools makes it possible to work directly with a compressed BAM file, without having to uncompress the whole file. Additionally, since the format for a SAM/BAM file is somewhat complex - containing reads, references, alignments, quality information, and user-specified annotations - SAMtools reduces the effort needed to use SAM/BAM files by hiding low-level details.

Heng Li is a Chinese bioinformatics scientist. He is an associate professor at the department of Biomedical Informatics of Harvard Medical School and the department of Data Science of Dana-Farber Cancer Institute. He was previously a research scientist working at the Broad Institute in Cambridge, Massachusetts with David Reich and David Altshuler. Li's work has made several important contributions in the field of next generation sequencing.

<span class="mw-page-title-main">Binary Alignment Map</span>

Binary Alignment Map (BAM) is the comprehensive raw data of genome sequencing; it consists of the lossless, compressed binary representation of the Sequence Alignment Map-files.

Compressed Reference-oriented Alignment Map (CRAM) is a compressed columnar file format for storing biological sequences aligned to a reference sequence, initially devised by Markus Hsi-Yang Fritz et al.

The BED format is a text file format used to store genomic regions as coordinates and associated annotations. The data are presented in the form of columns separated by spaces or tabs. This format was developed during the Human Genome Project and then adopted by other sequencing projects. As a result of this increasingly wide use, this format had already become a de facto standard in bioinformatics before a formal specification was written.

References

  1. 1 2 3 4 5 Li, H.; Handsaker, B.; Wysoker, A.; Fennell, T.; Ruan, J.; Homer, N.; Marth, G.; Abecasis, G.; Durbin, R. (2009). "The Sequence Alignment/Map format and SAMtools" (PDF). Bioinformatics. 25 (16): 2078–2079. doi:10.1093/bioinformatics/btp352. ISSN   1367-4803. PMC   2723002 . PMID   19505943.
  2. Edmunds, Scott (2021-02-17). "Play it again, SAMtools. Q&A with the SAMtools team on 12 years of providing bioinformatics "glue"". GigaScience. Retrieved 2021-03-20.
  3. Dörpinghaus, J.; Weil, V.; Schaaf, S.; Apke, A. (2023). Computational Life Sciences: Data Engineering and Data Mining for Life Sciences. Studies in Big Data. Springer International Publishing. p. 447. ISBN   978-3-031-08411-9 . Retrieved 2023-07-19.
  4. 1 2 3 "SAM/BAM Format Specification" (PDF). samtools.github.io.
  5. "Explain SAM Flags". broadinstitute.github.io. Retrieved 2023-11-04.
  6. "Sequence Alignment/Map Optional Fields Specification" (PDF). samtools.github.io.