Overview
Features
Outputs
Node file descriptions and examples
Edge file descriptions and examples
How to use data
Overview
A Co-occurrence graph helps visualize the relationship between entities that co-occur on a document. For example, a co-occurrence graph of authors (co-author graph) shows the collaboration between authors on a set of documents. This feature is available in all of the iSearch datasets.
iSearch allows the creation of multiple types of co-occurrence graphs, including but not limited to: Authors, Conditions, and MeSH Keywords. In the case of Authors, a disambiguation algorithm is used. The algorithm uses machine learning and hierarchical clustering to generate a disambiguated author. Some of the features used by the machine learning include co-authors, author affiliation, MeSH keywords, journal name, citation network, linked grants, linked clinical trials, and linked patents.
Because real time disambiguated names are utilized, no docs file will be generated for export (if you have used the co-occurrence feature in past versions of iSearch or other tools, a docs file will be generated to show how names have been grouped into a single entity). Lastly, you will need to upload data exported using the co-occurrence feature into a network analysis tool in order to create the visualization as a subsequent step. Learn more below on how to use this data.
Where to find co-occurrence
Co-occurrence features
Notification and exported files to download
Features
To export a Co-occurrence Graph, click the Export button then select Co-occurrence Graph (50k limit).
Select Co-occurrence field from the drop down menu to generate a graph and optionally give a file title or adjust the default settings given below. Fields you can choose from include:
-
Grant Records: All PIs, RCDC, MeSH Plus, and Condition
-
Literature: Authors, MeSH Plus and Condition
- Clinical Trials: MeSH Plus and Condition
Minimum Edge Weight
You can also edit the Minimum Edge Weight, which is the minimum number of times values have to co-occur to appear it in the network. For example, in a co-author network, it is the minimum number of times authors have to publish together to generate an edge. For large networks setting a higher minimum edge weight can help reduce the “hairball.”
Maximum Values per Document
Additionally, you can edit the Maximum Values per Document, which is the number of values for a field in a document before the values are truncated to the first N and the last value. For example, if authors is selected and maximum values is set to 50, then only the first 50 authors and the last author are used in the co-occurrence graph generation.
Outputs
The output contains two files, with fields varying depending on the dataset you are exporting from:
- Node file: This file has attributes of each node/entity. That could mean the person, condition, MeSH terms or other option you have chosen to view co-occurrence on. See below for more details of what fields are included in each node file.
- Edge file: This file has information about the relationship between entities. That could mean the number of times authors have published together
Node file descriptions and examples
Literature
Literature node fields (if exporting on author)
| Field | Description |
| Authors | This is the unique OPA ID for an individual/author |
| Document IDs | Articles (PMIDs or preprint IDs) authored by this individual |
| Name | Disambiguated name for the individual |
| Other Names | Other names for person of interest above |
| Grants | Number of grants for this individual |
| Earliest Awarded FY | Earliest grant award year for this individual |
| Latest Awarded FY | Latest grant award year for this individual |
| Literature Count | Number of articles this individual has been listed as author on |
| Earliest Pub Year | Earliest publication year from the publications for this node |
| Latest Pub Year | Last publication year from the publications for this node |
Author example (literature node file)
Grant Records
Grant Records node fields (if exporting on PI)
| Field | Description |
| All PIs | This is the unique OPA ID for an individual/PI |
| Document IDs | Grant records attributed to this PI |
| Name | Disambiguated name for the individual |
| Other Names | Other names for person of interest above |
| Grants | Number of grants for this individual |
| Earliest Awarded FY | Earliest grant award year for this individual |
| Latest Awarded FY | Latest grant award year for this individual |
| Literature Count | Number of articles this individual has been listed as author on |
| Earliest Pub Year | Earliest publication year from the publications for this node |
| Latest Pub Year | Last publication year from the publications for this node |
Clinical Trials
We are currently working on disambiguation for clinical trial investigators, therefore, there is no person co-occurrence option at this time.
Nodes fields (if exporting on any other field besides person data)
| Field | Description |
| Co-occurrence field (Condition, MeSH or Topics) | This is the field you have chosen to export co-occurrence on, i.e. all intermediate level topics viewed in your search results |
| Document IDs | Documents within each co-occurrence field, i.e. document IDs in each intermediate level topic |
Condition example (node file)
Edge file description and examples
This file shows the relationship between the entities (nodes). The number of times they co-occur is the weight column, and the PMIDs, for example, that they co-occur in are listed. The data within these files are essentially the same regardless of the co-occurrence option/dataset selected (and despite the node files being unique).
All datasets
Example of co-occurrence export (Edges file for any co-occurrence option)
| Field | Description |
| Source | This is the source node for your co-occurrence option (this could be the co-author or condition, etc. shared between records) |
| Target |
This is the other node for your co-occurrence option (this could be the co-author or condition, etc. shared between records)
Source and target can often be used interchangeably, because the order which came first often is irrelevant, except in certain network analyses such as a co-citation network (which paper came first) |
| Weight | These are the number of documents shared between authors, conditions or other co-occurrence option you have chosen |
| Document IDs | These are the IDs of the documents shared between authors, conditions or other co-occurrence option you have chosen |
Author example (edges file)
Condition example (edges file)
How to use data
These files are meant to be used in a Network Analysis software, such as Cytoscape, which can be used to customize the visualization of connections and node attributes. You should review the files for general QA/QC and potentially ensuring names have been disambiguated correctly, depending on the uniqueness of your data. To review these files, you will also need to transform them from tsv format into csv or another readable format (statistical analyses software such as R can also be used).
There are detailed instructions showing how to check the data and more in our Network Analysis class slides/resources, which are on the class materials page of the OPA class materials/resources webpage.