- Course overview
- Search within this course
- Overview of key IMPC concepts and tools
- Introduction to the Solr API: accessing IMPC data programmatically
- What is Apache Solr?
- Important definitions: query, field, core, document, parameter
- Quiz 2: get yourself familiar with Solr terminology
- What is the difference between an IMPC parameter and a Solr parameter?
- Using simple Solr syntax in your browser
- Output of the simplest request in your browser
- A Python module to access IMPC data: installation and available functions
- Quiz 3: explain Solr request
- Solr query syntax: simplified explanation
- How to use the solr_request function from the impc-api python package
- How to perform a query: q parameter
- Exercise 1: getting familiar with the core
- How to request a limited number of documents: rows parameter
- Exercise 2: requesting three documents
- How to get specific fields: fl parameter
- Exercise 3: selecting specific fields
- Quiz 4: basic Solr parameters
- Downloading data: getting large results efficiently
- How to download large dataset effectively: pagination
- How to download the data: batch_solr_request function
- What formats are available for downloading: wt parameter
- Exercise 8: download the data
- What is the difference: JSON vs CSV
- What you need to keep in mind: query responsibly
- Quiz 6: request only necessary data
- Advanced Solr query techniques: faceting and iterating over entities
- Understanding IMPC data: resources and assistance
- Your feedback
How to filter numbers: range search
Data can be filtered by a range of numeric values using the syntax field:[lower_bound to upper_bound] in the q parameter. An asterisk (*) can be used for either or both endpoints to specify an open-ended range query. See examples in the table below.
field:[* TO 100] | Finds all field values less than or equal to 100. |
field:[100 TO *] | Finds all field values greater than or equal to 100. |
field:[* TO *] | Finds any document with a value between the effective values of -Infinity and +Infinity for that field type. |
Different types of brackets can be used:
[&]denote an inclusive range query, meaning it matches values including both the upper and lower bounds.{&}denote an exclusive range query, meaning it matches values between the upper and lower bounds, but excludes them.
You can mix these types, so one end of the range is inclusive and the other is exclusive. For example: field:{1 TO 10] means values greater than 1 but less than or equal to 10.
Let’s use our knowledge gained so far to break down the query below:
num_found, df = solr_request(
core='statistical-result',
params={
'q': 'p_value:[1e-8 TO 0.001}',
'fl': 'marker_symbol,top_level_mp_term_name,effect_size,p_value',
'rows': 3
}
)
In this example, we request data with a p-value less than or equal to 10−8 (inclusive) and a p-value greater than 0.001 (exclusive).