First, we use two ontologies to create the original phenotype dataset P, i.e. PTO and Arabidopsis Hormone Database 2.0 (http://ahd.cbi.pku.edu.cn/cgi-bin/phenotypeBrowse.pl .) (Jiang et al., 2011). PTO is an important controlled vocabulary that describes phenotypic traits in plants. Each trait is a distinguishable, characteristic, quality or phenotypic feature of a developing or mature plant or a plant part. Arabidopsis Hormone Database 2.0 provides a systematic and comprehensive view of genes participating in plant hormonal regulation of the model organism A. thaliana. Its phenotypic ontology was developed to describe precisely myriad hormone-regulated morphological processes with standardized vocabularies in Arabidopsis.

When processing PTO, we extract ‘name’ and ‘synonym’ from every term in the ontology. Approximately 84% of these names are associated with synonyms; on average, each name has 1.07 synonyms. For example, the phenotype ‘alkali soil sensitivity’ has two synonyms: ‘AlkS’ and ‘alkali sensitivity’. Not all of terms in these ontologies appear in the literature. We found 805 terms in abstracts after removing duplicate entries. We combined these into a complete phenotype dataset P.

We followed the word embedding method published in (Xing et al., 2017). First, we used the collected PubMed texts to train the word-embedding model, which gave each word or phrase a distributed representation in low and dense dimensional vector space. By finding phrases with high similarity to phenotypic entities in P, the original ontology of the phenotype is expanded as P_update. Therefore, we can obtain more sentences containing phenotypic information.

Because some phenotypic synonyms contained in P are abbreviated forms, they may not represent as phenotype in the text and are incorrectly identified. For example, the abbreviation ‘AC’ in the ontology corresponds to the full name of ‘leaf sheath auricle color’. However, in the sentence ‘Many of these proteins have complex domain architectures with AC or GC centers …’ (PMID: 26721677), ‘AC’ is not a phenotype. The previous method did not consider abbreviation recognition such as this, so we required post-processing of word-embedding results. After obtaining a high similarity phenotype phrase, we recognized and revised the abbreviation.

We used (Xu et al., 2009) algorithms for identifying abbreviations in the biological literature, matching pairs of all abbreviations and full names in the processed texts. When we used an updated phenotype dataset P_update to reidentify the phenotype in the literature, if there was an abbreviated form, it was first matched with a full name. Only the full name of the abbreviation also in P_update, remained as a phenotype, otherwise it was deleted. The abbreviation recognition and revision can increase pipeline precision value and identify phenotypes more accurately.

Using the word-embedding results, we classified and tagged PubMed texts as input for the sentence-embedding (Le and Mikolov, 2014) method. The trained model can find sentences containing phenotypic information, acquiring new phenotypic sentences. To improve diversity of phenotype recognition, we transformed the unsupervised sentence-embedding model into a weakly supervised model. We used the results of word-embedding as positive samples, S_pos, and combined the NCLE algorithm for negative samples, S_neg, for the training of the Sen2Vec model.

Sentence embedding can aggregate similar phenotypic expressions. We found that large-scale gathered sentences have a similar sentence context structure. For example, the more similar sentences with ‘Solute import across the pollen plasma membrane, which occurs via proteinaceous transporters, is required to support pollen development and also for subsequent germination and pollen tube growth’ always have the same structure ‘be required {prep_*} + [phenotype]’, such as:

‘During pollination, constant communication between male pollen and the female stigma is required for pollen adhesion, germination, and tube growth’.

Due to many similar sentences, it is time-consuming to identify all phenotypic sentences and analyze their phenotype with expert evaluation. Therefore, we used sentence structure to automate extraction of complicated long/short phenotypic sentences of non-phrase types. These structures may contain complex phenotypic descriptions, likely with punctuation, prepositions, and conjunctions. We designed an automated algorithm to find frequently occurring sentence templates and with this, we extracted relatively complex descriptions of phenotypic long/short sentences from many sentence-embedding results.

At present, there are few studies about automatic generation of sentence templates in NLP. We borrowed the idea of modular algorithms from Sentence Pattern Extraction arChitecturte (SPEC) systems in (Michal et al., 2011) and proposed our own solution for combinatorial explosion problem.

With the SPEC algorithm, a ‘sentence template’ is considered as n-element ordered combination of sentence elements. It generates all possible combinations of patterns from a sentence and selects the frequency occurrence combination as a sentence pattern. However, we focused on the phenotype-containing structure and created the algorithm Phenotypic Sentence Template Extraction arChitecture (PSTEC) which consists of three components:

Preprocessing

Preprocessing: We tokenized all positive sentences S_pos of sentence embedding. Because we must extract phenotype-containing sentence structures, we treated phenotypic phrases as a whole and replaced phenotypic descriptions appearing in the sentence with ‘PHE’.

Generation ordered combinations: In every n-element sentence, there is k-number of ordered combination groups (1 ≤ k ≤ max). After processing all sentences in corpora, we choose a combination of frequencies greater than a threshold fre as a k-length template. Because the phenotype-containing template is not too long, so we set max as the length of the element threshold. We set two restrictions to prevent the combination explosions:

Combination of the k-element must include the specific word ‘PHE’

After iteration processing, we obtained all ordered, not duplicated, high frequency combinations for all values of k from the range of {1, …, max} as k-element sentence templates.

Insertion of a wildcard: During combination, we combined the original word order. To improve the applicability of templates, we specified whether the elements appeared next to each other or were separated. Therefore, we placed a wildcard between all non-subsequent elements using one heuristic rule. If an absolute difference of word order assigned to the two subsequent elements of a combination >1, we added a wildcard between them. An example of PSTEC algorithm appears in Figure 2.

When we obtained the high-frequency max-element sentence templates, we applied these templates to the results of a large number of sentence embeddings. Extracting the description of the more complex phenotypes in sentences that are highly similar to the positive samples improved phenotype recognition.

First, we searched all related genes in the UniProt database (http://www.uniprot.org ) using ‘A.thaliana’ as a key word and obtained 129 648 records. Each record contained the fields ‘Organism’, ‘Gene locus’, ‘Gene name’. Although we use Arabidopsis as a keyword, the results included other species, such as ‘Oryza sativa subsp. japonica (Rice)’. After processing, we obtained 89 287Arabidopsis gene ID and gene name pairs and these were used as a dictionary to identify genes.

Due to the large number of gene names and not a gene locus in the literature, part of the gene name is not in the dictionary. Therefore, we use gene lexical rules and semantic description rules in the text to improve gene recognition. Gene name spelling had some character-level rules as follows:

All capital letters.

Therefore, we used two rule types, mixed character-levels and contextual-levels, to identify the gene. When an input sentence contained these expressions: Expression of, Accumulation of, Expression levels/patterns of, Targets of, mRNA abundance of, Transcript profiles/levels of, and the ‘NNP’ (Proper noun, singular) tagged parts in the part-of-speech (POS) tagged sentence complies with our character-level rules, we extracted this special expression as a gene. For example, with the POS tagged sentence: “…HTR4K27Q (‘NNP’) overexpression (‘NN’) lines (‘NNS’) exhibited (‘VBD’) deregulated (‘JJ’) expression (‘NN’) of (‘IN’) H3K27me3-enriched (‘NNP’) genes (‘NNS’).” (PMID: 27926813) contains the specific contextual-level description ‘Expression of ’, and the ‘NNP’ tagged words satisfy the third and fourth character-level rules. Thus, we can identify gene ‘H3K27me3-enriched’.

Then, we used all sentences that contained the phenotype as input, and the output is two entities that cooccur in sentences. These sentences were used as input to subsequent relationship extraction.

To the best of our knowledge, there is a limited document annotation corpus of gene–phenotype relationships in Arabidopsis species. Currently we are only concerned with gene–phenotype relationships in single sentences. Most relationship recognition systems are not generic and portable so we used the open information extraction (OpenIE) system for this specific relationship identification. OpenIE can extract assertions from massive corpora without a specified vocabulary (Fader et al., 2011) from open-domain corpora, such as the Internet and Wikipedia, but in recent years, OpenIE has used biological literature for systematic testing.

We used an existing OpenIE system, OLLIE (Schmitz et al., 2012) as a relationship phrase recognition tool. OLLIE improved several shortcomings of the state-of-the-art system, extracting only relationships mediated by verbs and ignoring context, extracting tuples not asserted as factual. OLLIE is popular for information extraction and used in many fields, such as Question-Answer (Berant et al., 2013), knowledge graphs (Nickel et al., 2016), and named entities’ network (Tariq et al., 2017). OLLIE uses high-precision results of the previous generation OpenIE system i.e. REVERB (Fader et al., 2011). With many syntactic analyses of sentences that contain relationships, learning relationship patterns can be extended to find relationships of new input sentences.

We input the co-occurring sentences into the OLLIE system and extracted relationship sentences and their corresponding relationships. OLLIE automatically gives NP pairs of sentences as arguments in the relationship. However, these NP pairs contain too much noise, and the partially extracted arguments are not genes or phenotypes. Therefore, we limited our screening to eligible relationship groups. For the first (agr1) and third (agr2) parts of one triple, we need map them to the previous phenotype and gene entity list. When one or some genes and phenotypes are in each of the two arguments, we consider the relationship as a gene–phenotype relationship and stored such a relationship.

We used Word2Vec (https://code.google.com/p/word2vec ) to train a skip-gram model with a 4 D size, i.e. 300, 500, 700 and 900. Due to a lack of standards for this topic, we needed expert evaluation and annotation. Therefore, the results of word embedding first were semi-automatically classified and then manually evaluated by one expert and confirmed by another. Ultimately, the word-embedding method can extend original phenotype datasets P, increasing 1303 new phenotype data by up to 161.86%. We used the extended dataset P_update to match the phenotypic descriptions in the abstracts. Mapping sentences numbered 88 243. After abbreviations were identified and revised, 87 613 sentences containing phenotypes were obtained.

Some examples of phenotypes recognized by the word-embedding method appear in Table 1. ‘Ontology term’ as the original input, using the similarity mechanism to get ‘Phenotype’ and the corresponding ‘Similarity Score’. ‘Class’ represents the corresponding categories in the PTO 10 basic categories (10 basic categories are: TO: 0000277 biochemical trait; TO: 0000283 biological process trait; TO: 0000183 other miscellaneous trait; TO: 0000357 plant growth and development trait; TO: 0000017 plant morphology trait; TO: 0000597 quality trait; TO: 0000133 stature or vigor trait; TO: 0000392 sterility or fertility trait; TO: 0000164 stress trait; TO: 0000371 yield trait). Table 1.

Examples of word-embedding results

As shown in Table 1, the word-embedding method can find a phenotypic description according to the syntax and context of the text. For example, for the same ontology term ‘leaf curling’ (TO: 0002681), the method can extract similar words by considering syntax (‘leaf curling’—‘curly leaves’) and context semantics (‘leaf curling’—‘altered leaf shape’). Some new phenotypes are not synonyms of their corresponding original ontology terms. For example, the new phenotype ‘serrated leaves’ is not synonymous with ‘leaf curling’. This may because the contextual environment that describes the new phenotype and the original term is similar, but the semantics of expression are not the same.

We used Doc2Vec (http://radimrehurek.com/gensim/models/doc2vec.html ) to train the PV-DBOW model, and the trained corpora are positive/negative labeled abstracts. Then, we used the results of word embedding S_pos as inputs and acquired candidate sentences with similarities greater than Sim after calculating for cosine distance with S_pub. A reasonable Sim value greatly influenced the results. After testing, if Sim was too high (>0.4), high similarity sentences were too few and an average of 1.2 high-similarity sentences was obtained for each original sentence. If Sim was too low (¡0.2), we get a lot of dissimilar sentences. Therefore, we set Sim as 0.3, and an average of 4.5 high-similarity sentences was obtained for each original sentence.

The sentence-embedding method can find many candidate phenotypic sentences, which contain many non-phrase, complex long/short phenotypic sentences. For example, the phenotypic structure ‘response to …stress’ in the sentence ‘GmaPHO1 genes had altered expression in response to salt, osmotic, and inorganic phosphate stresses’. Such phenotypic descriptions are special and numerous and can improve relationship identification. Therefore, we designed a PSTEC algorithm to automatically generate phenotypic sentence templates for extracting them.

We tested and selected template length max and template frequency fre of the PSTEC algorithm. When max is too long (>6), the template will contain a lot of noise, such as too many prepositions and stop words. When max is too short (<4), the template cannot contain complete template structure information. Thus, we set max as 5. The size of fre directly affects the efficiency and uptime of the algorithm. After testing, we set fre as 100 and only kept templates that appeared more often than 100 in the corpus. Ultimately, we obtained 250 sentence templates. There are many types of duplicate templates and high frequency but not intention-containing templates, such as ‘Show/Suggest + prep_*’. Therefore, we merged and selected these results. Table 2 shows 5 high frequency sentence templates that can recognize combination type phenotypes (‘Tolerance to salt/drought/methyl viologen stress in Arabidopsis’), with environmental or time factors (‘Hypocotyl growth in response to unilateral blue-light illumination’) and are rich in diversity of phenotypes. Meanwhile, we noticed that phenotypes recognized by different templates may differ. For example, the ‘Respond’ template can identify more ‘stress trait’ types. Table 2.

Examples of sentence templates

We can extend 1314 phenotypic descriptions using the sentence template. After merged results of word-embedding, we expanded 2409 phenotypic expressions and increased them 2.99-fold compared to the original phenotype dataset P.

Using the phenotype recognition cascaded approach can improve the identification of phenotypes in the literature and improve relationship identification. To illustrate the importance of phenotypic recognition in relationship extraction and to verify the accuracy of our approach, we establish two baselines for performance comparison.

B1: Using the traditional ontology-based method (Müller et al., 2004) to recognize phenotype and extracting the gene and relationship using method described in this article.

We randomly selected 100 abstracts to identify the relationships by expert verification and to calculate Precision, Recall, F1-Measure. Results are shown in Table 3. Table 3.

Performance of baselines compared with our pipeline

Among the different methods on phenotype recognition, the effect of recognizing the gene–phenotype relationship using only ontology-based efforts is the poorest. Because of loss of many phenotypes, recall value in relationship recognition is low. For example, the phenotype ‘NaCl stress-sensitive phenotype’ is not in ontology, so the relationship (MCK1; complemented; NaCl stress-sensitive phenotype) cannot be found. However, we can identify this phenotype using the proposed approach and obtain relationships with the best recall. This is because we recognized the phenotypic phrase and the more complex phenotypic long/short sentences based on the sentence template. As the integrity of the phenotype increased, the precision is improved. For this sentence, ‘…a structurally related Arabidopsis MADS-box gene involved in the negative control of Arabidopsis flowering time, …’ (PMID: 15539492), due to the template: ‘(gene) involve + {prep.} + PHE’, we can identify the whole phenotypic description ‘negative control of Arabidopsis flowering time’, and get the relationship (MADS-box gene; involved in; negative control of Arabidopsis flowering time). However, the first two baselines only extracted part of the whole expression ‘flowering time’ and missed the complete relationships. Thus, our approach can extend relationship extraction by improving phenotype recognition.

Compared with the B3 baselines, which only change the relation extraction method, our pipleline also has the best performance. Because the syntactic rule method misses many results and only getting 26.58% recall value, its F1-Measure is about 36.00%.

We have considered to use generic tools such as GNormPlus (Wei et al., 2015), GenNorm (Wei and Kao, 2011) and so on for gene identification but found that these tools identify the gene of all species that appear in the text. Therefore, noise information is mixed in the targeted identification of Arabidopsis gene information, which requires expert screening. So, we finally chose a more targeted rule- and dictionary-based approach and obtained 88.76% precision value in the above test dataset. This is slightly higher than the results given in the article (Wei et al., 2015) by GNormPlus (precision 87.1%) and GenNorm (precision 78.9%).

Although the proposed pipeline can improve the effectiveness of final relationship identification compared with baselines, there are misidentifications and omissions due to the following reasons:

Error of relationship recognition. The OLLIE system is limited as it can only identify the relationship in a single sentence, and the length of the sentence cannot be too long. Sentences >20 words have increased errors for relationship analysis (Schmitz et al., 2012). For example, the sentence ‘Hence, the narrow organ shape, reduced plant height, and reduced whorl 4 organ primordia are consistent with a general reduction of cell number, and, perhaps, reflect a role of SEU in promoting cell proliferation’ can be assessed by OLLIE to get this relationship (whorl 4 organ primordia; perhaps reflect; a role SEU in promoting cell proliferation). The wrong relationship association results in inaccurate identification of it.

These errors reduce precision and recall because each case results in an incorrect or incomplete relationship extraction.

The TAIR database (Lamesch et al., 2012) is one of the most informative databases for storing Arabidopsis information, which contains a gene–phenotype relationships dataset. This information was manually extracted from 555 full texts. To verify pipeline effectiveness, we calculated coverage of relationship for these papers. Because some documents cannot be downloaded, we retrieved only 481 full papers. Preprocessing the TAIR dataset by deleting irrelevant fields, i.e. ‘Phenotype not described’ and ‘No visible phenotype’ was done and we retrieved 1397 sets of gene–phenotype relationships. We noticed that there are duplicate types of relationships in the dataset. For example, the gene ‘MSSP1’ is related with:

Under normal growth temperature conditions, the double mutant leaves’ content in glucose and fructose is slightly reduced (30%) in a similar fashion to that observed with the tmt1 single mutants.

As they are the same type, we treat them as the identical relationships. We applied our pipeline to this dataset, extracted data were compared with processed TAIR datasets by four experts and offered coverage of 70.94%. Moreover, our pipeline can identify 373 new relationships, which the TAIR dataset does not include. The results are shown in Supplementary Material. We had limited coverage for a few reasons:

Many relationships in TAIR come from cross-sentence or even cross-paragraph relationships. Such relationships are unrecognizable to our pipeline that only extracts from a single sentence, so there is the main reason of limited coverage. However, due to redundancy of much information, our pipeline use repetitive relationships extracted from many studies to compensate extraction of the relationship representations in small samples. Such work cannot be done manually.

After analysis, we found that articles in the TAIR dataset are relatively old (most prior to 2000). Due to limitations to manual reading, this dataset failed to update gene–phenotype relationships as the literature grew, so scalability was poor. However, with our pipeline we can quickly find relationships for updated literature, greatly improving efficiency for summarizing data.