User Guide: Top Associated Concepts with a Target Concept

Last update: July 2021.

This document explains how this Shiny visualization tool works.

Summary

This tool shows a table of the concepts associated with a particular target concept selected by the user. The concepts in the table are ordered by decreasing strength of association.

• The user can select variants of the data used as a basis for calculating the associations: abstracts and/or full articles, co-occurrences counted by sentence or by paper.
• The "strength of association" is measured in the data by Pointwise Mutual Information (PMI), a statistical measure based on how often two concepts occur together with respect to how often each of them occurs on its own.
• The user can refine the search of associated concepts:
  • by setting a minimum frequency threshold, which removes the least "important" concepts
  • by filtering the semantic types of the related concepts, for instance in order to visualize only the top genetic concepts.

Data

The data used in this application has been precomputed using the biomedical text content from Medline and PubMed Central (PMC) as well as the UMLS metathesaurus and PubTatorCentral.

• Two sources are proposed to the user:
  • The KD data is extracted with the KnowledgeDiscovery system by Jake Lever. Concepts are identified by string-matching based on the UMLS metathesaurus.
  • The PTC data is extracted directly from PubTatorCentral, which provides a pre-annotated version of the biomedical literature.
• Three views are proposed:
  • "abstracts only": the Medline abstracts
  • "articles only": the PMC full articles
  • "abstracts+articles": both the Medline abstracts and PMC articles (duplicate abstracts which appear in both are discarded)

The associations between concepts may change depending on which dataset is selected by the user. For example abstracts are shorter and usually focus only on the most important concepts, whereas full papers contain more details and therefore more specific concepts.

The PMI association value (see below) is calculated based on how often any two concepts appear together in the data and how often they appear individually. The fact that two concepts "appear together" can be interpreted at different levels, so for every dataset two levels of co-occurrence are proposed to the user:

• "by paper" means that every pair of concepts found in the same document (abstract or article) are counted as a co-occurrence.
• "by sentence" means that a pair of concept is counted as a co-occurrence only if the two concepts appear in the same sentence.

Of course with the former option much more co-occurrences are counted. Some of the concept pairs might be remotely related (if at all), especially in the case of full articles. On the other hand, the latter option is restrictive and tends to only capture the clearest cases of relations between concepts, so it might be more accurate but it might also not cover all of relations.

Target Selection

The user can select a target concept among a predefined list of ND concepts.

Technical note: a free choice of target concept is not possible because of the volume of data and the intensive precomputations required. Thus this list is currently limited to the set of diseases which appear as descendants of the concept "Neurodegenerative Disorders" in either MeSH or UMLS. It might be possible to extend the list with more concepts in the future. The interface might also be improved in the future with a search box in order to make the selection more user-friendly.

Association Measure

Pointwise Mutual Information (PMI) is the default measure used as indicator of the strength of the association between two concepts. This value is based on how often two concepts \(A\) and \(B\) occur together with respect to how often each of them occurs on its own. In other words, it does not only take into account how often two concepts are found together (the joint probability \(p(A,B)\), which would be biased towards frequent concepts), it makes it relative to each concept frequency (\(p(A)\) and \(p(B)\)). This way a rare concept \(A\) might be found to be strongly associated with a frequent concept \(B\) if \(B\) almost always appears when \(A\) does (high conditional probability \(p(B|A)\)), even though \(A\) usually does not appear when \(B\) does (low conditional probability \(p(A|B)\)).

The PMI value has no predefined bounds, its minimum and maximum depend on the probabilities of the concepts \(A\) and \(B\).

• A high positive value denotes a high association, i.e. \(A\) and \(B\) tend to "attract each other".
• A value of zero (or close to zero) denotes the absence of interdependency, i.e. \(A\) and \(B\) appear together only by chance.
• A negative value denotes a negative association, that is \(A\) and \(B\) tend to "repulse each other".

The tool also proposes several other ways to measure association between concepts, presented below. It is recommended to choose the measure by trial and error based on the visible top result: one measure might be suitable for the desired goal for a particular target while another meaure works better in a different content.

• Normalized Pointwise Mutual Information (NPMI). In text data PMI is often considered biased towards rare events. NPMI is a variant of PMI known to reduces this bias, and it has the important advantage to be normalized: the value is between 0 and 1, making it easier to interpret.
• PMI^2 and PMI^3 are two other variants of PMI meant to give more importance to more frequent events. As a result these measures focus on the most general concepts, PMI^3 even more strongly than PMI^2.
• Mutual Information (MI). MI is closely related to PMI but it is more complex. It does not only reflect how much two concepts tend to appear together, it also takes into account how much they don't. This means that MI can be high also if two concepts tend to "avoid each other". While this could potentially be useful in general, it seems inadequate in the present application due to the very high number of concepts in the data. The MI results are also harder to interpret since there can be different reasons for a value to be high or low. For these reasons we do not recommend using it, but the option is available in the tool.

A few other less standard options are also proposed.

Technical note: the MI value is calculated based on the 2x2 contingency table corresponding to the presence or absence of each of the two concept, i.e. four cases are considered: neither A nor B is present, only A is present, only B is present, or both are present.

Joint Frequency Thresholding

The user can select a minimum joint frequency, i.e. the minimum number of times two concepts must appear together to be selected. This makes it possible to filter out cases where two concepts occur rarely together (even if they have a high association value), and consequently push pairs which may have a slightly lower association value up to the the top of the table. This is useful because there are many rare concepts which appear always accompanied by the target concept, however they are often too specific to be considered as an important indicator of the target concept. Some rare concepts may also appear by chance with the target, as opposed to more frequent concepts. The more often a co-occurrence event happens, the more one can be confident that the association value is meaningful.

The frequency threshold can also be seen as a way to adjust the level of generality of the observed concepts: increasing the threshold shows relationships involving high-level concepts, while decreasing leads to more specific relationship. Importantly, finding the desired level of generality may depend on the target concept, i.e. different concepts may require different threshold values.

Filtering by semantic type

The concepts shown in the results table can be filtered by semantic category/group.

The concepts which do not belong to any of the selected semantic types are filtered out from the results, causing the other concepts to be pushed up to the top.

Viewing the Results Table

The table on the right side of the tool shows the concepts related to the target, ordered by decreasing association score (PMI by default), after applying the selected filters (see above).

• With the KD source selected, every concept is identified by a CUI (Concept Unique Id). The CUI links to the UMLS page describing this concept in detail (this requires a UTS account).
  • See Usage Tips below about creating a UTS account.
  • There is a technical glitch which prevents redirecting the user to the UMLS concept page if they were not previously connected with their UTS account: the UTS website login page appears, but after login the user is redirected to the general UMLS page instead of the specific concept page. In this case the user has to click a second time on the CUI link, and this time the correct page appears. This happens only the first time the user clicks a link, since they will already be signed afterward.
• With the PTC source selected, the concept id is provided by PTC, which itself includes different data sources and therefore types of concepts ids. The concepts which originate from MeSH are provided with a link to the corresponding MeSH browser page.

Usage Tips

• The user's browser can be used to open several tabs or windows in order to see this document and manipulate the app at the same time.
• It is also possible to open several tabs or windows of the app itself in order to compare what happens in different configurations.
• It is possible to select some rows in the results table and copy/paste them in an external document. Normally this is supposed to preserve the formatting, but this probably depends on the software used.
• Access to the UMLS Terminology Services (UTS) (when clicking on a CUI link in the results table) requires the user to have a UTS account. Creating a UTS account is free (and it can be done through Google or Facebook authentication), but this is not an automatic process so the user may have to wait for the account to be validated. Once validated, the user just has to sign in in order to browse the concepts in the UMLS metathesaurus.

Examples

In the following examples the abstracts only and by sentence options are selected for the dataset. The "Amyotrophic Lateral Sclerosis (ALS)" concept is used as target, and the default PMI is used as association measure.

The following table shows the top 5 concepts obtained with the default minimum frequency 10:

Top 5 concepts associated with target ALS at min. frequency 10

concept	term	group	jointFreq	pmi	rank
C4475575	Radicava	CHEM	13	12.37871	18.0
C0154683	Other motor neuron disease	DISO	50	11.96223	22.0
C0678179	Rilutek	CHEM	11	11.58517	32.5
C3686938	Progressive motor neuron disease	DISO	42	11.46969	41.0
C1456383	IGFALS gene	GENE	1086	11.29005	43.0

For example it can be seen that "Other motor neuron disease" belongs to the disorder ("DISO") category, it appears together with ALS 50 times in total which represents a large proportion of the occurences of the CUI "Other motor neuron disease" but only a small proportion of the occurrences of the target ALS, which is much more frequent.

Filtering with Minimum Joint Frequency

The minimum frequency can be used to adjust the results to the desired level of generality. For example decreasing the minimum to zero gives the following top 5 concepts:

Top 5 concepts associated with target ALS at min. frequency 0

concept	term	group	jointFreq	pmi	rank
C4519182	AZD-7295	CHEM	1	12.58517	8.5
C0154758	Inflammatory and toxic neuropathy	DISO	1	12.58517	8.5
C0154684	Other anterior horn cell diseases	DISO	1	12.58517	8.5
C0154754	Hereditary and idiopathic neuropathy, unspecified	DISO	2	12.58517	8.5
C2317805	multifactorial amyotrophic lateral sclerosis	DISO	1	12.58517	8.5

This results in many rare concepts (only 1 or 2 occurrences) which have a high PMI with the target ALS because the latter always appears when they do. However this is hardly meaningful, especially in the case of concepts which appear only once.

Note: the rank of these concepts is identical because they have exactly the same PMI with the target. In such cases where several concepts are tied, the rank is the average of the ranks that would be obtained by ordering the tied concepts arbitrarily.

It is also possible to do the oposite, that is to observe only the most general related concepts by increasing the minimum frequency, for example at 100 below:

Top 5 concepts associated with target ALS at min. frequency 100

concept	term	group	jointFreq	pmi	rank
C1456383	IGFALS gene	GENE	1086	11.29005	43
C4024896	Motor neuron atrophy	DISO	525	11.05528	50
C0154682	Lateral Sclerosis	DISO	292	10.93107	58
C1428691	C9orf72 gene	GENE	1382	10.90755	60
C0524459	Lower motor neuron	ANAT	893	10.86020	62

With the rare concepts discarded, the results now show frequent concepts as strongest associations. One may notice that the top PMI values are not very far from the ones observed before without any filtering. This is often the case when the target concept is very frequent, since many other concepts have a quite strong association with it, i.e. there are only small variations in association power (PMI) across a very large set of concepts. The filtering options help visualizing the most relevant relations for a particular goal.

Filtering with Semantic Types

The below top 5 concepts are obtained by selecting only the "Genes and Molecular Sequences" in the semantic filter, with a min. frequency of 100:

Top 5 concepts in the 'Genes and Molecular Sequences' category for target ALS (min. frequency 100)

concept	term	group	jointFreq	pmi	rank
C1456383	IGFALS gene	GENE	1086	11.29005	43
C1428691	C9orf72 gene	GENE	1382	10.90755	60
C1420588	TARDBP gene	GENE	2325	10.55432	89
C1421305	UBQLN2 gene	GENE	124	10.39725	99
C1420306	SOD1 gene	GENE	4541	10.34614	100

Note that this filter leaves only around 40 genetic concepts in the table (thanks to the minimum frequency filter). This way one can observe the difference in association strength between different concepts: while the top concepts have a high PMI close to, the concepts found at the bottom of the table have a PMI close to 0 or even slightly negative. This means that they are not especially associated with the target, and it can be seen that most of them are indeed generic terms such as "Genes", "Alleles", "DNA sequence".