- Course overview
- Search within this course
- Overview of key IMPC concepts and tools
- Introduction to the Solr API: accessing IMPC data programmatically
- What is Apache Solr?
- Important definitions: query, field, core, document, parameter
- Quiz 2: get yourself familiar with Solr terminology
- What is the difference between an IMPC parameter and a Solr parameter?
- Using simple Solr syntax in your browser
- Output of the simplest request in your browser
- A Python module to access IMPC data: installation and available functions
- Quiz 3: explain Solr request
- Filtering data in Solr: narrowing down your results
- How to query a specific field: filter by value
- Exercise 4: filtering by a single field
- How to filter numbers: range search
- Exercise 5: changing the p-value threshold
- How to combine multiple filters: Boolean operators
- Exercise 6: applying multiple filters
- How to exclude data: NOT operator
- Why parentheses are important: combine multiple Boolean operators
- Quiz 5: Boolean operators
- How to handle with null values: exclude empty fields
- Exercise 7: explore null values
- Downloading data: getting large results efficiently
- How to download large dataset effectively: pagination
- How to download the data: batch_solr_request function
- What formats are available for downloading: wt parameter
- Exercise 8: download the data
- What is the difference: JSON vs CSV
- What you need to keep in mind: query responsibly
- Quiz 6: request only necessary data
- Advanced Solr query techniques: faceting and iterating over entities
- Understanding IMPC data: resources and assistance
- Your feedback
How to perform a query: q parameter
q is a mandatory Solr parameter that defines the query. This includes boolean operators, quotation marks, parentheses, wildcard characters such as asterisks and question marks, range operators, proximity searches, and more. We will learn more in the following sections.
Let’s try to request everything from one core using the command below:
num_found, df = solr_request(
core='statistical-result',
params={
'q': '*:*'
}
)
Take into account that this request returns only the first 10 documents by default. To request everything effectively, we need to use the batch_solr_request function and pagination. In the following sections, you will learn how to download large amounts of data, but for now, we will focus on using the solr_request function for prototyping.
Let’s do our first exercise!