Annual Scientific Report 2015

Genomics Technology Infrastructure
Our team is responsible for developing technology to store, process, access and
disseminate genomic annotation. We provide Ensembl and Ensembl Genomes
with its production infrastructure, and maintain database application
programming interfaces (APIs), web services, schemas and data-mining tools.
Genomic annotation, a foundational service provided
by EMBL-EBI, is fundamental to many downstream
programs for interpreting genomic data sets. We provide
a single software stack, empowering our users to explore
multiple datasets on many species, in reference to
several genome assemblies, and ensure a consistent
interface-to-annotation experience with minimal
recoding required.
For the past 15 years, Ensembl has developed APIs for
data access through its MySQL databases. For many
bioinformatics communities, these APIs, including
the highly used Variant Effect Predictor (VEP), are a
vital component of many analysis methods. They are
essential to Ensembl’s genome-annotation pipelines,
allowing our solutions to be rolled out quickly to new
species, and enable the Ensembl website to serve over 4
million users a year.
Our team is also responsible for developing new
methods for data access. Our current focus is on
providing efficient web service APIs to access genomic
annotation. Bioinformatics is no longer a language
dominated by a small set of programing languages; we
can increase the value of our APIs by ensuring they
appeal to as large a consumer base as possible. The
Ensembl REST API, first released in 2012, was our
first foray into providing these types of API and one we
continue to improve upon.
We also provide access to genome annotation via our
public FTP sites and data-mining tool, BioMart. Both
solutions provide ‘de-normalised’ representations of
Ensembl data customised for genome-wide analysis and
batch processing. We build these resources for Ensembl
and Ensembl Genomes using the same codebases,
ensuring consistent, high quality and correct data across
both projects.
BioMart, developed at EMBl-EBI, is a well-used
service, especially by R programmers through the
Bioconductor biomaRt project. EMBL-EBI projects
(e.g. Gene Expression Atlas, UniProt, Reactome) build
their own resources from annotations provided by these
alternative, efficient access methods.
Major achievements
In 2015 we issued a beta release of a new Track Hub
Registry service, which makes community-submitted
datasets more discoverable. The Registry was developed
as part of our BBSRC funded project ProteoGenomics.
Track Hubs enable users visualise genomic annotation
data within a genome browser, using a UCSC Genome
Browser text format to group and configure these
datasets into ‘tracks’. Ensembl has supported Track
Hubs since their use in the ENCODE project, when
they required more user involvement. Initially, it was
necessary to know the location of a dataset, then and
manually attach it to the Ensembl Browser. Now, the
Track Hub Registry enables users to publish Track Hub
locations and provides a comprehensive search interface
(see Figure). We see Track Hubs and the Track Hub
Registry as the natural successor to the Distributed
Annotation System (DAS).
In 2015 use of our REST API continued to grow, serving
over 70 million requests for data. We made REST
available for all Ensembl and Ensembl Genomes species,
and provided access to annotation based on GRCh37.
We released our first Global Alliance for Genomics and
Health (GA4GH) endpoints, which provide access to
Ensembl-hosted variation data. We also expanded the
amount of data returned from a number of endpoints,
providing more comprehensive information about genes
and variations. We improved the performance of our
endpoints, notably of retrieving genomic variations in
protein coordinates and the VEP.
We developed the first set of programmatic interfaces to
access CRAM data from Perl. Our work, funded as part
of the Crop Bioinformatics Initiative from the BBSRC,
enabled CRAM data attachment into Ensembl; making
Ensembl genome browsers among the first to support
the next-generation format.
In 2015 we built new layers in our indexing schemes,
FAIDX and Tabix, providing fast access to FASTA
formatted sequence and tab-delimited data files,
respectively. The library is available from CPAN under
an Apache 2.0 license.
89
2015 EMBL-EBI Annual Scientific Report