Comparison of rates of nausea side effects for prescription medications from an online patient community versus medication labels: an exploratory analysis

Abstract

Background

While medication labels are considered the authoritative resource for medication information, emerging research suggests that patient-generated health data (PGHD) are a valuable tool to understand the ways in which patients experience medications in real world settings. However, the relationship between these two data sources has not been closely examined.

Methods

To understand how rates of medication side effects compare between a source of PGHD and medication labels, the current study compares adverse drug reaction rates from FDA medication labels with those self-reported by patients from an online patient community, PatientsLikeMe (PLM). The linear association between medication label and PLM nausea rates was evaluated using Spearman correlation, with an associated 95% confidence interval calculated based on 10,000 bootstrap iterations. The reporting ratio of PLM nausea rates to medication label nausea rates was defined for all treatments with non-zero medication label nausea rates. Lognormality of the distribution of this reporting ratio was assessed based on a Kolmogorov-Smirnov test (α = 0.05).

Results

Nausea rates for 163 medications were compared between the two data sources. Overall rates ranged from 0 to 60% for medication labels and 0 to 36% for PLM data with median rates of 6.4 and 3.7%, respectively. In general, nausea rates reported by patients in the online community were lower than those found in medication labels. This inconsistency was attributed to a variety of factors, including differences in data collection mechanisms and product use factors.

Conclusions

Quantifiable and consistent differences exist between side effect rates reported on medication labels and those self-reported by patients based on real-world use. In general, self-reported rates of nausea associated with medication use were lower than those reported in medication labels. Although considered a definitive resource for medication information, this discrepancy demonstrates that medication labels may not comprehensively describe the patient experience. Results suggest that a combination of information from different sources may provide a more rounded and holistic view on medication safety and tolerability.

Background

The term “side effect” typically refers to an unintended response to a medication that is related to the pharmacological properties of that particular medicine (World Health Organization, 2002). Prior research indicates that significant differences exist between reported rates of medication side effects during real-world use and those observed in clinical trials (Blenkinsopp et al., 2007; Hakobyan et al., 2011). Clinical trials follow rigorous protocols for extensive safety monitoring both during and after a trial; however, side effects, as observed in real-world settings, are not monitored as rigorously. Outside the controlled environment of clinical trials, several variables may affect the pharmacodynamics of a particular drug, including dosage, duration of use, indication, comorbidities, and use of concomitant medications. In both clinical trials and during real-world use, most side effects are reported through clinicians; this methodology may lead to underreporting, as clinicians may underestimate the severity of patient-reported symptoms or patients may withhold side effect information due to stigma (Pakhomov et al., 2008; Basch et al., 2009; Weingart et al., 2005).

Advancements in technology have led to the generation and availability of new types of medically-relevant data sources that have an array of potential health applications, including patient-generated health data (PGHD) (Wood et al., 2015). PGHD is defined as “health-related data—including health history, symptoms, biometric data, treatment history, lifestyle choices, and other information—created, recorded, gathered, or inferred by or from patients or their designees (i.e. care partners or those who assist them) to help address a health concern” (Shapiro et al., 2012). It is distinct from data generated in clinical settings because it allows for patients to directly report and record their own information. PGHD can complement and enrich other sources of medical evidence, and its use is advised when measuring concepts best known by the patient or best measured from the patient perspective, such as pain or nausea (Research Center for Devices and Radiological Health. Guidance for Industry Patient-Reported Outcome Measures, 2009).

PatientsLikeMe (PLM), an online patient community where patients track, share, and discuss health information, is one source of PGHD. Patients use the PLM platform to track their own health data, including medication side effects. This study seeks to explore the relationship between side effect rates reported on medication labels (i.e., package inserts) and those reported by patients in the PLM community. The value of gaining a better understanding of the ways in which side effects are experienced during real-world medication use is recognized by both the Food and Drug Administration (FDA) and the European Medicines Agency (Research Center for Devices and Radiological Health. Guidance for Industry Patient-Reported Outcome Measures, 2009; 21st Century Cures Act, HR 34, 114th Congress, 2015; European Medicines Agency, 2011).

To explore potential differences in side effect reporting, nausea, a relatively common side effect reported across several medication classes, was selected for further investigation. Nausea can be very debilitating and has the potential to seriously impact patient health outcomes. As nausea is a subjective experience, it is particularly at risk of being underestimated or underreported by providers. Additionally, patients may be able to more accurately attribute symptomatic reactions, such as nausea, to a specific drug as opposed to reactions where causality is less obvious, such as acute (e.g. heart attack, pulmonary embolism) or systemic (e.g. fatigue, generalized muscle pain) events (Research Center for Devices and Radiological Health. Guidance for Industry Patient-Reported Outcome Measures, 2009).

This exploratory analysis seeks to quantify potential differences in how nausea is self-reported by patients (PLM database) and by investigators in the context of clinical trials (medication labels). The side effect profiles of medications are critical components of regulator, payer, and physician decisions, as well as patients’ decisions about whether or not to take a particular drug. Quantifying the difference between side effect reporting in clinical trials versus PGHD offers an opportunity to better characterize the safety profile of medications and to improve the patient experience.

Methods

The aim of this exploratory analysis was to examine the relationship between side effect rates reported on medication labels and those self-reported by members of the PLM online patient community.

PLM data

The initial set of medications eligible for inclusion was selected based on available PLM data volume, and included 220 prescription medications with side effects reports from 50 or more patients on PLM. As FDA medication labels would be the source of comparison, only FDA-approved medications were included in the analysis. For each eligible medication, the PLM nausea rate was defined as the proportion of patients completing a treatment evaluation or side effect report who had reported nausea as a side effect. Data were aggregated by generic medication. All PLM data are as of January 1, 2016.

Medication label data

For all eligible treatments identified based on available PLM data, corresponding medication labels were retrieved using the text mining system i2E (Linguamatics, Cambridge, UK). Rule-based queries were used to identify the word “nausea” within the Adverse Reactions section of medication labels. Queries were designed in I2E OnDemand, which provides access to medication labels sourced from the DailyMed website sourced from the FDA. Data extraction was limited to the most recent version of the medication label as of May 2016. Unique Ingredient Identifier (UNII) codes were utilized to ensure that synonyms and/or lexical variants were identified for each medication. Nausea rates were recorded along with, when available, additional data from associated clinical trials (e.g. study sample size, nausea rate of placebo group).

Nausea rates reported in combination with vomiting (e.g. methotrexate reported a combined nausea/vomiting rate of 10% (Methotrexate sodium [package insert], 2003), but provided no rate associated uniquely with the symptom “nausea”), and instances where nausea was included as part of a list of side effects without a numerical approximation were categorized as having “no medication label nausea frequency information”. This occurred in a total of 49 nausea rates associated with 45 otherwise eligible generic treatments, which were excluded from subsequent analyses. If nausea was not mentioned on the medication label, the nausea rated was treated as 0% in subsequent calculations. When nausea rates were reported as an approximation or range, the nearest reported value (e.g. “>5%” as 5%) or midpoint of range (e.g. “2%–6%” as 4%) was used. Nausea rates from pediatric studies were excluded. The median of all reported nausea rates was calculated when multiple nausea rates were available for a medication.

Data processing and analysis

Data were analyzed using R version 3.2.3 (R Core Team, 2016). The linear association between medication label and PLM nausea rates was evaluated using Spearman correlation, with an associated 95% confidence interval calculated based on 10,000 bootstrap iterations. The reporting ratio of PLM nausea rates to medication label nausea rates was defined for all treatments with non-zero medication label nausea rates. In cases where the PLM nausea rate was 0, half the value of the lowest-detected PLM nausea rate (0.001) was added to enable calculation of log ratio. Lognormality of the distribution of the reporting ratio was assessed based on a Kolmogorov-Smirnov test (α = 0.05).. Confidence intervals for PLM nausea rates were calculated using the Clopper-Pearson exact method, and adjusted for multiple comparisons using Bonferroni adjustment.

Results

PLM data characteristics

As of January 1, 2016, the PLM community included 393,371 patients. After aggregating available data by generic ingredient, 220 prescription treatments met the inclusion criteria of 50 or more patients reporting side effect information (Fig. 1). Of these 220 medications, a total of 12 were excluded from subsequent analyses due to non-specific active ingredients (n = 5), lack of FDA approval (n = 4), ongoing research investigations which influence side effect reporting (n = 2), and not being a drug product (n = 1, oxygen therapy). All data generated or analyzed during this study are included in this published article [Additional file 1 and Additional file 2].

Fig. 1

Aggregation of available data by generic ingredient. A total of 163 medication met all of the inclusion criteria

Individual medication labels

Information related the medication’s nausea rates was identified and extracted from each medication label’s “Adverse Reactions” section for the 208 eligible medications. Of the 208 eligible medications, 72% (n = 150) of medications reported at least one numerical frequency for nausea. For medications in which the label did not list nausea (n = 13), the nausea rate was considered to be 0% for subsequent comparisons. Nausea was listed in the label 45 of these medications, but did not have a numerical frequency, and could not be included in the subsequent analysis. A total of 397 nausea rates were extracted for the remaining 150 medications that reported a numerical frequency. The number of nausea rates identified exceeded the number of medications as multiple rates may be reported for a single medication label and/or medications may have multiple brand products with product-specific labels. Of the 397 extracted nausea rates, approximately 5% were reported as a numerical approximation (n = 11) or range (n = 9). Examples of numerical approximations included: “At least 1%”, “fewer than 1 in 100” and “≥10%”.

Aggregate information for eligible medications

Medication label data were aggregated by active ingredient, and the median nausea rate was calculated for each of the 163 medications. Aggregated rates from medication labels ranged from 0% to 60%, with a median of 6% and an interquartile range of 3% to 14% (Fig. 2a). For the same 163 medications, nausea rates reported by the PLM community ranged from 0% to 36%, with a median of 4% and an interquartile range of 1% to 6% (Fig. 2b).

Fig. 2

Distribution of median reported nausea rates for 163 medications (aggregated by active ingredient). IQR = interquartile range

Comparison of PLM vs. medication label nausea rates

The Spearman correlation between PLM and medication label nausea rates was 0.59 [95% confidence interval 0.48, 0.68], demonstrating a moderate positive correlation between the two data sources. In both aggregate PLM and medication label data, cyclophosphamide had the highest observed nausea rate (Fig. 3). Medications with the largest absolute difference in reported nausea rates are shown in Table 1.

Fig. 3

PatientsLikeMe nausea rates versus medication label nausea rates. Selected medications are highlighted with text labels

Table 1 Medications with largest absolute differences in nausea rates between PLM data and medication label data

For 57% (93/163) of drugs, the label rate was within the Bonferroni-adjusted 95% confidence interval of the proportion of patients from PLM who reported nausea (Fig. 4). For the 43% (70/163) of drugs that did not fall within these confidence intervals, 86% (60/70) of label nausea rates fell above the upper bound of the PLM confidence interval. In total, 60 drugs were identified as having label nausea rates significantly higher than PLM nausea rates, and 10 drugs were identified as having label nausea rates significantly lower than PLM nausea rates.

Fig. 4

Bonferroni-adjusted confidence internals for PLM nausea rates versus medication label nausea rates. For each generic treatment, PLM nausea rates and their associated 95% Bonferroni-adjusted confidence intervals are positioned along the horizontal axis, and medication label nausea rates are shown as a black line. In cases where the medication label nausea rate falls outside of the associated 95% confidence interval based on PLM data, treatments are colored red, identifying instances in which drug label nausea rates and PLM nausea rates were identified as having notable differences, after accounting for multiple comparisons. Since not all drug labels reported the number of patients included in the associated study, it was not possible to assess the statistical significance of the differences between PLM and label nausea rates for each drug

To compare the degree of over or underreporting of nausea rates among patients from PLM relative to the rates reported in medication labels, the reporting ratio was calculated as the ratio of PLM nausea rates to medication label nausea rates. Across 150 treatments with non-zero medication label nausea rates, reporting ratios ranged from 0 to 13. The median reporting ratio was 0.38, indicating that, for half of all observed treatments, the PLM nausea rate was at or below 38% of the medication label nausea rate. The reporting ratio distribution was well approximated by a lognormal distribution (Fig. 5).

Fig. 5

Empirical and theoretical lognormal distributions of reporting ratio. Empirical distribution of the reporting ratio of PLM nausea rates to medication label nausea rates and a theoretical lognormal distribution with (natural log scale) mean − 1.01 and standard deviation 1.31. Distributions based on data from 150 treatments with non-zero medication label nausea rates

To explore factors that may contribute to different reporting rates, doxycycline, ritonavir, amoxicillin/clavulanate, mitoxatrone, and three organ transplant medications as a group (cyclosporine, mycophenolate mofetil, and tacrolimus) were selected for further investigation using both PLM data and detailed information from medication labels.

Ritonavir

In the current study, ritonavir had the highest observed magnitude of absolute difference between medication label (57.4%) and PLM (2.7%) nausea rates. This discrepancy cannot be attributed to a difference in indication for medication use, as the drug is listed as a treatment for HIV infection in both data sources. Instead, the observed difference is likely due to widely varying dosage information between the medication labels and PLM. In PLM, all but one patient reported a daily dosage of 100 mg/day (a single patient reported a daily dosage of 80 mg/day). In contrast, the medication label corresponds to patient experiences while taking a daily dose of 1200 mg/day.

Doxycycline

The doxycycline nausea rate in PLM (27.1%) exceeded the medication label nausea rate (8%) by the largest magnitude, which may be explained by differences in both dosing and indication for use. Indications for doxycycline use among patients from PLM included Lyme disease (37%) and acne (14%), with reported daily dosages ranging from 20 to 400 mg/day (median daily dose 100 mg/day). In comparison, information reported in the medication label was based on a clinical trial of patients with periodontal disease who received a daily dose of 40 mg/day (Periostat® [package insert], 2001).

Amoxicillin/Clavulanate

Like that of doxycycline, the nausea rate reported by patients from PLM for amoxicillin/clavulanate (11.5%) exceeded that reported in the medication label (2.6%). Amoxicillin/clavulanate is indicated for a wide range of conditions; indications included sinusitis, urinary tract infections, and other systemic infections in both PLM data and medication labels. Corresponding daily doses recommended by clinical guidelines also vary across indications, ranging from 750 to 4000 mg/day (Amoxicillin and clavulanate, 2016). Unlike doxycycline and ritonavir, there are no clear patient or product use characteristics that appeared responsible for differences in nausea rates between these data sources. Influenced by the relatively low number of patients who have evaluated amoxicillin/clavulanate on the PLM platform (n = 61), the aggregate medication label nausea rate for amoxicillin/clavulanate falls within the Bonferroni adjusted 95% confidence interval for the proportion of patients experiencing nausea in the PLM community.

Mitoxantrone

The aggregate nausea rate reported in the medication label for mitoxantrone (55%) was higher than that reported in PLM (20.3%). A more detailed examination of the medication label data suggests that the expected difference may be even more striking than the observed difference. The median medication label nausea rate of 55% is based on data collected over the course of three clinical trials, one with a nausea rate of 76% (n = 62, dose = 12 mg/m²), the second with a rate of 55% (n = 65, dose = 5 mg/m²), and a third trial where mitoxantrone was administered concurrently with methylprednisolone and had a nausea rate of 29% (n = 21) (Novantrone® [package insert], 2012). The current recommended daily dose of mitoxantrone is 12 mg/m², which is more than double the dose administered in the cohort of patients whose reported nausea rate was 55% (Marriott et al., 2010). Furthermore, administration of mitoxantrone in combination with an anti-emetic medication, such as methylprednisolone, would likely result in lower overall reported rates of nausea. It seems that the median medication label nausea rate (55%) may actually be an underestimate of the nausea rate observed during a clinical trial that more closely mirrors real-world prescribing practices (76%), leading to an even greater discrepancy between the nausea rates reported in PLM and those reported in the medication label.

Organ transplant medications

Our review also found that nausea rates reported on PLM were much lower than those reported in medication labels for the medications tacrolimus (PLM – 1.9%; medication label – 38.5%), mycophenolate mofetil (PLM rate – 9.7%; medication label – 54%), and cyclosporine (PLM rate – 6.9%; medication label – 14%) which are typically taken after organ transplantation to prevent rejection. Product use characteristics for these medications were examined in greater detail to determine what may be contributing to these wide differences. One potential explanation for this observed trend is that nausea rates reported in medication labels reflect the overall rate for nausea that patient experienced, but not the rate that is likely to be attributed to the medication alone. Traumatic, invasive transplant surgeries and/or pain medications are likely a significant contributor to nausea following a transplant. Patients evaluating treatments on PLM may differentiate between nausea associated with the overall experience of receiving an organ transplant and nausea caused by transplant-related medications, leading to lower reported rates for nausea as compared to those reported on medication labels.

Discussion

The goal of this study was to closely examine rates of patient-reported medication side effects from a real-world data source and from medication labels. This study showed that PLM nausea rates for 163 treatments ranged from 0% to 36%, with a median of 4%. Across a broad range of indications and dosages, PLM nausea rates were positively correlated to those reported on medication labels. Moreover, this study demonstrated a quantifiable relationship between the reporting ratio of PLM nausea rates to medical label nausea rates, which provides a useful basis for comparisons against alternative data sources (e.g. patient report to a physician).

Medication labels are often considered the definitive data source for information on side effects due the strenuous requirements of regulatory approval. However, any data source will have strengths and limitations. Many medications were approved decades ago, and updates to their labels are infrequent. The labels also do not contain information related to off-label indications. An estimated 20% of outpatient prescriptions in the United States are written for off-label use (Radley et al., 2006). For example, the label for amitriptyline contains information from clinical trials conducted over 50 years ago in patients with depression. However, prescribing practices have evolved significantly for amitriptyline so that the proportion of off-label use is very high (>80%) with the most frequent use in fibromyalgia, neuralgias and migraine prevention (Radley et al., 2006; Moore et al., 2015; Silberstein, 2015).

Differences in reporting rates of nausea between medication labels and in the PLM database may be influenced by various factors, including methodological differences in data collection and differences in indications for medication use. Data for all drugs from within the PLM database are collected through one consistent mechanism, whereas data collected through clinical trials for use in medication labels may be obtained using varying methods. Prior research has shown that the methods used to collect side effect data significantly influence the results obtained (Bent et al., 2006). For example, sexual side effects due to use of selective serotonin reuptake inhibitors ranged from 2% to 73% depending primarily on whether side effects were collected via a checklist, open-ended questioning, or by a detailed inquiry (Safer, 2002).

In the case of this study, we speculate that the general trend of lower nausea rates in the PLM database compared to those reported in corresponding medication labels is driven largely by differences in data collection mechanisms. On the PLM platform, patients report side effects using a standard method across all drugs. Once a patient indicates that they have experienced any particular side effect, they are presented with an entry field with an auto-completer that provides suggestions as the user types.

Although the FDA requires that studies investigating new drugs report adverse events, they do not specify a standard method for data collection. One common approach accepted by the FDA is to ask patients to complete a checklist of symptoms, which is likely to capture additional symptoms unrelated to the particular medication of interest (U.S. Department of Health and Human Services, Food and Drug Administration, 2005; Smith et al., 2013). Conversely, the process of collecting data in the PLM database is more open-ended than that used in clinical trials, leading patients to be more likely to report only their most bothersome side effects. Each approach has both advantages and disadvantages; however, the use of different mechanisms for data collection across clinical trials limits the ability of researchers to accurately compare side effect profiles across drugs. In contrast, the consistent method of collecting side effect data from the PLM platform is advantageous to making more valid comparisons among drugs’ safety profiles.

The examples highlighted in the Results section illustrate that differences in dosage and indication contribute to differences in reported nausea rates. There are likely other factors that additionally contribute to these differences, including concurrent treatments, comorbidities, disease severity, and other patient population characteristics (e.g. age, gender, location). However, information related to these subgroups was seldom reported in medication labels.

In addition to the methodological challenges associated with comparing side effects among many medications, differences in medication label data reporting presented additional challenges. Data can be presented quantitatively (e.g. 5%) or qualitatively (e.g. “commonly” or “frequently”). Nausea, in particular, is reported in various ways, often combined with rates of vomiting and reported as one value. The variety of methods used for 1) collecting information about side effects in clinical trials and 2) reporting information in medication labels limits the ability to make meaningful comparisons between medications.

Limitations

The side effect nausea can be reported in a variety of ways (i.e., qualitatively or quantitatively) on medication labels. In order to collect data that were consistent with information reported in medication labels, PLM nausea cases were identified only as instances in which patients specifically reported nausea. Patient reports of side effects known to have a high co-occurrence with nausea, such as vomiting, were not considered as nausea cases. In cases where nausea was not mentioned on a drug product label, the label nausea rate was approximated as zero for analysis purposes. FDA guidelines state that the Adverse Reactions section of a medication label does not include all adverse events that occurred during a clinical trial, but only those that meet an appropriate cutoff (e.g. all adverse reactions occurring at a rate of 10% or greater in the treatment group and at least twice the rate of placebo) (US Food and Drug Administration, 2006).

Due to variable reporting methods, aggregating nausea rates from medication labels proved particularly challenging. In many instances, more than one value was given for nausea as a side effect in a single medication label, and sample sizes were not always available for each individual rate, leading to challenges in calculating an overall nausea rate.

In addition, aggregating nausea rates across multiple labels for the same active ingredient was challenging due to inconsistencies or redundancies in the information. For example, the medication label for levetiracetam (Keppra®) does not list nausea as a side effect, but the label for extended-release levetiracetam (Keppra XR®) lists a nausea rate of 5% (Keppra® [package insert], 2013; Keppra XR® [package insert], 2016). Upon further investigation, it was found that nausea was previously listed in the levetiracetam (Keppra®) label, described as an “event reported by at least 1% of adult levetiracetam (Keppra)-treated patients but as or more frequent in the placebo groups.” This description, however, has been removed in updates to the medication label over time. Furthermore, it was often difficult to determine if nausea rates for different labels of the same active ingredient were from the same clinical trial, as was often the case for immediate-release and extended-release products.

Some medication labels included nausea rates for ingredients similar to the active ingredient, as was the case for the valproate sodium label, which contained information from trials of divalproex sodium. Similarly, some medication labels included nausea rates for a class of medications rather than for only the active ingredient. For example, the label for minocycline contains information on side effects of tetracyclines, a broad class of antibiotics that includes minocycline, but it did not provide information about side effects specific to minocycline (Minocycline hydrochloride [package insert], 2012).

The underreporting of side effect rates by patients in the PLM community may also be influenced by the existence of survival bias. Although patients are able to enter their entire medication history on PLM, they are prompted to enter more detailed information about experiences with current treatments. Patients, who stop taking a particular medication due to intolerable side effects, may not report a detailed history for that medication compared to medications they are currently taking. Prompting patients to report on their current treatments may lead to lower reported rates of nausea in the PLM database.

A common critique of Internet communities is that patients may not be able to accurately report their diagnosis and medication history, or that anonymity provides an opportunity for individuals to report misinformation. A previous study demonstrated that a large percentage of the patients on PLM users can be matched to real patients with confirmed identities, and there is a high degree of agreement between self-reported and insurance claims for diagnoses and medication history (Eichler et al., 2016). Other studies have also shown high degree of agreement between self-reported comorbidities and medical records (Ye et al., 2017).

Conclusions

In the current study, we observed a positive correlation between nausea rates reported in medication labels and those self-reported by patients in an online community. However, in general, nausea rates reported among patients in the PLM community were lower than those reported in medication labels. While there are likely many factors driving the discrepancy between these two sources, including differences in data collection mechanisms and product use factors, this study demonstrates a quantifiable relationship in side effect reporting frequencies between sources of real-world evidence and the controlled environment of clinical trials. Reporting frequencies of side effects have high impact on regulator, payer, physician, and patient decisions; therefore, collection of data that accurately reflect the patient experience is critical. Using self-reported patient data may be one way to enhance the capture of subjective elements of medication tolerability and safety.