- Course overview
- Search within this course
- Introduction to the Solr API: accessing IMPC data programmatically
- What is Apache Solr?
- Important definitions: query, field, core, document, parameter
- Quiz 2: get yourself familiar with Solr terminology
- What is the difference between an IMPC parameter and a Solr parameter?
- Using simple Solr syntax in your browser
- Output of the simplest request in your browser
- A Python module to access IMPC data: installation and available functions
- Quiz 3: explain Solr request
- Solr query syntax: simplified explanation
- How to use the solr_request function from the impc-api python package
- How to perform a query: q parameter
- Exercise 1: getting familiar with the core
- How to request a limited number of documents: rows parameter
- Exercise 2: requesting three documents
- How to get specific fields: fl parameter
- Exercise 3: selecting specific fields
- Quiz 4: basic Solr parameters
- Filtering data in Solr: narrowing down your results
- How to query a specific field: filter by value
- Exercise 4: filtering by a single field
- How to filter numbers: range search
- Exercise 5: changing the p-value threshold
- How to combine multiple filters: Boolean operators
- Exercise 6: applying multiple filters
- How to exclude data: NOT operator
- Why parentheses are important: combine multiple Boolean operators
- Quiz 5: Boolean operators
- How to handle with null values: exclude empty fields
- Exercise 7: explore null values
- Downloading data: getting large results efficiently
- How to download large dataset effectively: pagination
- How to download the data: batch_solr_request function
- What formats are available for downloading: wt parameter
- Exercise 8: download the data
- What is the difference: JSON vs CSV
- What you need to keep in mind: query responsibly
- Quiz 6: request only necessary data
- Advanced Solr query techniques: faceting and iterating over entities
- Understanding IMPC data: resources and assistance
- Your feedback
All materials are free cultural works licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license, except where further licensing details are provided.
How IMPC data is generated: data processing workflow
The IMPC generates phenotypic data by analysing mice using a standardised set of protocols designed to learn as much as possible about different aspects of biology in a high throughput fashion (figure 2). After validation and quality control checks, ETL (Extract, Transform, Load), statistical, and annotation pipelines are applied to identify mutant phenotypes and make associations of phenotypes to specific genes. Based on the results, human-disease associations are generated. Finally, the data is prepared for release and made available on the IMPC website.
Figure 2. Processing workflow.
For more details on data generation, explore this section of the tutorial The International Mouse Phenotyping Consortium: Finding phenotypes for your gene of interest.
In this tutorial you will learn how to access the following programmatically:
- Raw experimental data
- Statistical results
- Hits with significant p-values and assigned mp_terms
- Images
- Human disease association data