bioRxiv · 10.1101/859611
Mining Archive.org's Twitter Stream Grab for Pharmacovigilance Research Gold
Abstract
In the last few years Twitter has become an important resource for the identification of Adverse Drug Reactions (ADRs), monitoring flu trends, and other pharmacovigilance and general research applications. Most researchers spend their time crawling Twitter, buying expensive pre-mined datasets, or tediously and slowly building datasets using the limited Twitter API. However, there are a large number of datasets that are publicly available to researchers which are underutilized or unused. In this work, we demonstrate how we mined over 9.4 billion Tweets from archive.orgs Twitter stream grab using a drug-term dictionary and plenty of computing power. Knowing that not everything that shines is gold, we used pre-existing drug-related datasets to build machine learning models to filter our findings for relevance. In this work we present our methodology and the 3,346,758 identified tweets for public use in future research.
Source connections
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Tekumalla, R., Asl, J. R., Banda, J. M.. 2019-12-03. Mining Archive.org's Twitter Stream Grab for Pharmacovigilance Research Gold. https://doi.org/10.1101/859611
Cite the original work for its findings. Save a collection to share your selection of sources.