Improving the safety of disposable auto-injection devices: a systematic review of use errors

Abstract

Auto-injection devices must be easy and intuitive to use, especially in emergency situations. However, there are many reports of safety-related issues affecting auto-injectors and pen injectors, due to usability deficits. To minimize this type of problem, the identification of potential use errors is an important and critical task in usability engineering. The aim of this systematic, qualitative literature review was to identify and catalog use errors related to disposable auto-injection devices. The key terms “auto-injector”, “usability”, and “safety/errors” were used to search in the PubMed and Scopus databases for articles from peer-reviewed journals and conference proceedings published in English or French between 2000 and 2017. Use errors, close calls, and operational difficulties were identified, extracted and hierarchically classified.

The analysis showed that of the 1415 initially identified publications, 38 met all the eligibility criteria. A qualitative analysis identified 232 instances of use errors and close calls, which were classified into 10 main categories and then 39 subcategories. The present results could be used to guide the design, evaluation and risk analyses of disposable auto-injection devices. Our approach is in line with the European Union’s latest regulations on improving the safety of medical devices - especially those concerning transparency and traceability.

Background

For patients suffering from certain allergies or certain chronic diseases (such as diabetes, rheumatoid arthritis, and multiple sclerosis) auto-injection devices represent a significant advance over manual injections with vials and syringes. In particular, these devices enable patients to administer their own medication (Thompson et al., 2013). Auto-injection devices improve dose accuracy (Kadiri et al., 1998; Clarke & Spollett, 2007; Keith et al., 2004) and foster adherence to the prescribed treatment regimen (Molife et al., 2009; Buysman et al., 2011). Furthermore, these devices can help to reduce injection pain (Kadiri et al., 1998; Molife et al., 2009; Graff & McClanahan, 1998) and increase the patients’ quality of life (Molife et al., 2009; Graff & McClanahan, 1998; Rubin & Peyrot, 2004).

Auto-injection devices can be classified according to the type of delivery system used (prefilled syringe-based systems vs. cartridge-based systems) (Frew, 2011). Auto-injectors based on prefilled syringes are designed for the single-use administration of fixed doses only. Cartridge-based systems (pen injectors) can be designed for both single-use and multiple-use administration if their reservoir can contain more than a single dose. In turn, cartridge-based systems can be divided into refillable systems and disposable systems. The present literature review is focused on disposable, syringe-based and cartridge-based auto-injection devices.

Auto-injection devices must be easy and intuitive to use (Stauder et al., 2014). A device’s design must also allow it to be used by people with no or only minimal medical training (Frew, 2011; Reimer et al., 2008). Ease of use is also essential because of the target user groups for auto- and pen injectors - elderly people, for example, who may have more difficulty learning how to use a device (Stauder et al., 2014). Auto-injection devices are also used by people suffering from impairments that make the devices more difficult to handle (Davidson et al., 2007; Pfützner et al., 2011). Furthermore, ease of use is especially important for devices that are intended to be used in medical emergencies (e.g. anaphylactic shock) (Frew, 2011). In this context, even a small difficulty experienced by users can delay or even prevent an injection - with possibly fatal consequences.

Developers of auto-injection devices must therefore focus on the intended context of use as well as the users and their characteristics. This includes physical characteristics (hand size, mobility, flexibility, strength, etc.), sensory performance and cognitive abilities (Fujioka et al., 2015; Food and Drug Administration, 2017). The application of information about human capabilities and limitations to device design in order to optimize a device’s performance and safety is known as human factors engineering or usability engineering (Lange et al., 2015; Russ et al., 2013). Usability is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” (International Standardization Organization, 1998). In this sense, usability goes beyond a device’s design (the legibility of text, shape, colors, etc.) and functionality. The concept of usability is more general, and deals with the match between a device and the users’ needs (Beuscart-Zéphir et al., 2013).

Good usability is no guarantee that a device will be useful or efficient. However, poor usability (characterized by the presence of usability flaws) makes devices prone to use errors and the corresponding negative safety outcomes. A use error is defined as “a user action or lack of user action while using the medical device that leads to a different result than that intended by the manufacturer or expected by the user” (International Electrotechnical Commission (IEC), 2015). However, one must distinguish between use errors, close calls, and operational difficulties. Close calls are events where users almost make a use error but realize in time and avoid it. The term can also describe cases in which a user commits a use error but quickly corrects his or her action before it has any consequences (Wiklund et al., 2016). Operational difficulties represent cases in which users have problems performing a task. Signs of this include needing several attempts to perform a task, as well as comments or non-verbal signs indicating confusion or frustration (Fujioka et al., 2015).

Figure 1 illustrates the propagation of a usability flaw, from violation of the corresponding design principle to the resulting use error and then the negative patient-related outcome.

Fig. 1

Schematic representation of the causal-and-effect chain between the violation of a usability design principle, the usability flaw, the use error, and the negative outcome

Various studies of auto-injection devices have reported safety issues that are due (at least in part) to usability flaws. For instance, one common negative outcome with auto- and pen injectors is an unintentional injection or a needle-stick injury caused by a use error related to incorrect orientation of the device (Frew, 2011; Simons et al., 2009; Simons et al., 2010; Sicherer et al., 2000; Fitzcharles-Bowe et al., 2007). This arises from a usability flaw: the mismatch between a pen-like design and non-pen-like handling (Simons et al., 2009; Gosbee, 2004). A device’s shape and size may also impair its portability. As a result, patients do not always carry the devices with them, even when this is required for certain applications (Frew, 2011; Gosbee, 2004).

In order to minimize this type of issue and to ensure that patients can take their treatment without running any risks, usability engineering is crucial for developing safe, effective devices (Fujioka et al., 2015). This requirement is also reflected in the guidelines issued by the regulatory authorities. In Europe (International Electrotechnical Commission (IEC), 2015) and the United States (Association for the Advancement of Medical Instrumentation (AAMI), 2001), medical device manufacturers must document the integration of safety-oriented usability engineering methods into the design and product development process (Lange et al., 2015; Schiro et al., 2017). A critical step in this process is the identification of use errors affecting existing devices, in order to avoid the same problems during the design and development of a new device (Schiro et al., 2017). In this context, it is necessary to systematically analyze the use errors associated with auto- and pen injectors. The present publication addresses the question which use errors related to disposable auto-injection devices have been identified in published studies. To the best of our knowledge, the present systematic review is the first to have provided an overview of the different types of use error affecting auto-injection devices.

Methods

As far as possible, the present review complied with international guidelines on methods (Higgins & Green, 2017; Center of Reviews and Dissemination, 2008) and reporting (Moher et al., 2009).

Eligibility criteria

The present review considered original studies reporting on the usability and/or usage of auto-injection devices and published in English or French between 2000 and 2017 in peer-reviewed journals or conference proceedings. The eligibility criteria were as follows:

• Publications about disposable auto-injection devices for use by patients, relatives and/or healthcare professionals were included. Publications about pumps, pre-filled syringes, and refillable pens were excluded.

• Publications giving objective descriptions of use errors or corresponding to sociotechnical, risk analysis or impact studies that addressed usability issues were included. Publications about the clinical or economic impact of auto- and pen injectors, and reports dealing solely with perceived usability (e.g. the assessment of feelings via questionnaires) and/or researchers’ hypotheses were excluded.

Information sources and search

We searched for eligible publications in two online databases (PubMed, the main medical literature database, and Scopus, the largest database on technologies) and in the reference lists of reviewed publications. For the literature search, three sets of key terms were defined:

• auto-injector: auto-injector, auto-injection, self-injection, injection pen, disposable injection pen, pen;

• usability: usability, human engineering, human factors, ergonomics, usage, user, ease of use, acceptability, user-computer interface, personal satisfaction, human factors engineering, validation testing, cognition, decision making, task performance and analysis, surveys and questionnaires, preference, handling study;

• safety/errors: error, risk, failure, hazard, risk assessment, risk factors, prevention and control, incident, failure mode and effects analysis, user error, adverse event, safety, safe.

For each set, key terms were combined with the operator “OR”. Lastly, the three sets were joined using the operator “AND” (see Fig. 2).

Fig. 2

Venn diagram for the literature research

Whenever possible, standardized key terms were used (e.g. MeSH terms for the PubMed search). With the help of a specialist librarian, we first developed the query for PubMed and then adapted it for Scopus (see the Additional file 1: Appendix S1 for the full set of queries). The searches were carried out on March 2nd, 2016, and updated on May 24th, 2017.

Study selection process

The selection was carried out by human factors researchers with expertise in healthcare technologies. Each phase of the review was over-inclusive; in the event of doubt, the item in question was included in the next step of the analysis. The reasons for rejecting publications were documented.

One reviewer (RM) was responsible for carrying out the search and combining the results. He (i) excluded duplicates, non-original research, non-peer-reviewed publications, and review articles; (ii) scanned the publications’ titles, in order to exclude those outside the scope of the present review (e.g. animal studies), and (iii) screened the abstracts of the remaining publications. For publications lacking an abstract (n = 3), the full publication was assessed directly. The results of the screening process were cross-checked by a second reviewer (TW), and an agreement score was calculated. For the most part, the two experts shared the same opinion on the abstracts; this yielded a high agreement score (Cohen’s kappa = 0.95). Disagreements were resolved by consensus.

In the next phase, three reviewers (MDZ, RM, and TW) screened the full texts of the selected publications. Firstly, the three reviewers trained together on three randomly chosen publications, in order to interpret the eligibility criteria as coherently as possible. Secondly, the remaining publications were shared equally between the three reviewers and screened independently. All results were cross-checked by a second expert. For all combinations of two reviewers, the agreement scores ranged from good to perfect (Cohen’s kappa = 0.82, 0.75 and 1.00). Disagreements were resolved by consensus. Lastly, the reference sections of the selected publications were checked for other suitable publications not identified during the database searches.

Data extraction and analysis

The selected publications were distributed equally among the three reviewers for data extraction. Online appendices (if available) were also analyzed. An Excel spreadsheet was used to record the items extracted from each publication, in three sections:

• 1. A description of the auto-injection device (if available):

• the brand name;

• the development phase (e.g. under development or market approval);

• the medical intention (e.g. anaphylactic shock, diabetes, etc.);

• the active substance.

• 2. Descriptions of the studies in which use errors were identified:

• design/methodology;

• participants

• 3. Descriptions of the reported events:

• use errors;

• close calls;

• operational difficulties.

The “device description” section aimed to get contextual information, in order to describe the results more efficiently. Problem descriptions that were not formulated clearly enough to enable the reviewer to understand the corresponding error or difficulty were not taken into account. Furthermore, close calls and operational difficulties were only taken into account and recorded if the corresponding problem was not described as a use error in the corresponding article. In this context, all issues that could not be unambiguously classified as use errors were considered to be operational difficulties. All extracted items were cross-checked by a second expert. Based on the extracted data, the three experts developed a hierarchical categorization scheme for the use errors and close calls. The scheme was applied in an interactive, inductive manner. Any difficulties and disagreements were discussed among the experts to produce clear, unambiguous, mutually exclusive categories and subcategories with a high internal consistency and a good degree of inter-reviewer agreement. At the end of the process, each use error and each close call was assigned to a unique category and subcategory. A similar categorization process was applied to operational difficulties. Since the three experts had developed the categorization schemes together, an agreement score was not calculated. Furthermore, the impact or severity of the use errors, close calls, and operational difficulties was not assessed.

Risk of bias

To assess the risk of bias in individual studies, the validity of those selected for qualitative analysis was assessed. Two eligibility criteria concerning each study’s methods and reporting quality were defined:

• Reporting: the study’s objective, the context of the evaluation, the device under evaluation, the device’s medical intention and/or intended users, and the results.

• Methods: the number of participants, the participants’ profile, the data collected, the analyses performed, the study design and/or the methods applied.

The reporting quality and the quality of the used methods were assessed in order to be able to judge the reproducibility of the included studies. During the analysis, two experts (RM and MDZ) rated each publication’s reporting and methods on two 5-point Likert scales ranging from 1 (poor reporting/methods) to 5 (very good reporting/methods). The publication was excluded if either of the two scores was equal to or lower than three.

Bias can also be induced by inter-publication differences in the level of detail of error descriptions. To decrease the potential for bias, only studies that gave precise, comprehensive reports on use errors were included in the review. Furthermore, not all errors in a given publication may have been described to the same extent (due to a journal’s word count limitation, for example); this is why only qualitative analyses of use errors, close calls and operational difficulties were taken into account in the present review.

Results

Study selection

The database searches identified a total of 1415 publications (Fig. 3); 340 publications were excluded because they were duplicates, non-original studies or non-peer-reviewed publications. Of the remaining 1075 publications, 809 were excluded on the basis of means of their title. A further 161 publications were excluded on the basis of their abstract, leaving 105 publications eligible for the full-text analysis. Next, following an examination of the full text, 73 publications were excluded - leaving 32 publications. Lastly, 6 additional publications were identified by searching within the reference sections of included publications. Hence, a total of 38 publications met the criteria for inclusion in our qualitative synthesis.

Fig. 3

PRISMA flow diagram for the literature analysis

Results of the bias assessment

Overall, the 38 included publications scored well with regard to reporting quality (mean score = 4.45; median = 5) and the quality of the methods used (mean = 4.63; median = 5). All publications scored more than 3 on both scales, and so all 38 were judged to be sufficiently valid for inclusion in the review. The full results of the assessment are available in Additional file 2: Appendix S2.

Characteristics of the included publications

The set of 38 included publications consisted exclusively of journal articles. The publications’ main characteristics are summarized in Additional file 3: Appendix S3. A total of 42 different disposable auto- and pen injectors had been evaluated, covering a broad range of medical intentions (most frequently in the treatment of allergies/anaphylaxis, diabetes or rheumatoid arthritis):

• rheumatoid arthritis (10 devices)

• allergies/anaphylaxis (9 devices)

• diabetes (8 devices)

• growth hormone therapy (5 devices)

• fertility treatment (4 devices)

• episodic migraine (2 devices)

• hepatitis (1 device)

• multiple sclerosis (1 device)

• overweight/obesity (1 device)

• opioid overdose (1 device)

• psoriasis (1device)

• systematic lupus erythematosus (1 device)

The above list contains 44 items because two of the devices were platform-based and used for the simulated treatment of both diabetes and rheumatoid arthritis.

Categories of identified use errors and close calls

Our qualitative analysis of the selected publications enabled us to identify and categorize 232 instances of use errors and 2 instances of close calls. These instances covered a broad range of errors that can occur when using a disposable auto- or pen injector.

The categorization process yielded 10 main categories, which were divided into 39 subcategories in order to provide more information about the use errors. The categories were related to either a certain process or a specific device component. A full list of the use errors and close calls is given in Additional file 4: Appendix S4. The categories of use errors are presented in Table 1.

Table 1 Categories of use errors and close calls

The most frequently reported use errors were related to an incorrect duration of injection (n = 28 publications), failure to remove the device’s protective/safety cap prior to an injection (n = 20), holding the device upside down (n = 17), incorrect choice of the injection site (n = 13) and errors related to the activation of an injection by pressing the device against the body (n = 13).

The “activation” category refers to the initiation of an injection. The three subcategories “failure to release the safety lock”, “failure to push the button (at all, or with enough force)” and “failure to press the device against the injection site (at all, or with enough force)” were related to the device’s design. Some devices require a safety lock to be released before an injection can be triggered (Phillips et al., 2011; Callis Duffin et al., 2016). Some users failed to release the safety lock. Depending on the device, an injection can be triggered either by pushing a button or by pressing the device against the body. For devices with a dedicated button, two types of use error were reported. Firstly, some users were not aware that they had to activate the device by pushing a button (Robinson et al., 2014; Salter et al., 2014; Brown et al., 2013). Secondly, some users were not able to push the button with enough force to trigger the injection (Fujioka et al., 2015; Callis Duffin et al., 2016; Matfin et al., 2015; Demary et al., 2014). The same distinction could be made for devices that have to be activated by pressing them against the body. Ten publications reported on users who failed to activate an injection because they did not press the device against the injection site (Sicherer et al., 2000; Salter et al., 2014; Brown et al., 2013; Umasunthar et al., 2015; Bakirtas et al., 2011; Mehr et al., 2007; Arga et al., 2011; Topal et al., 2013; Saleh-Langenberg et al., 2017; Hanna et al., 2016). Three publications mentioned that users did not press the device against the body hard enough to activate an injection (Guerlain et al., 2010a; Edwards et al., 2013; Raffa et al., 2017). Furthermore, one publication reported a “too short contact duration” between the device and the patient’s body; consequently, an injection could not be triggered (Schmid et al., 2013). Concerning the “unintentional activation” subcategory, one publication described a case in which the device was inadvertently activated during the dose setting (Rohrer et al., 2013).

The “compliance with procedures” category comprises errors related to failure to follow the formal procedure indicated in the IFU, and errors related to pre- or post-injection associated tasks. In this context, one publication mentioned that users tried to disassemble the device (Guerlain et al., 2010a). “Incorrect handling” refers to errors where users did not perform actions with the correct finger (Salter et al., 2014) or the correct hand (Hudry et al., 2017). The “incorrect storage” subcategory is related to cases in which users could not state the correct storage conditions for the auto-injection device (Sicherer et al., 2000; Domańska et al., 2017) or did not take the device out of the refrigerator long enough before the injection to reach room temperature (Schiff et al., 2016). With regard to post-injection tasks, three publications mentioned failure to massage the injection site after the injection (Salter et al., 2014; Brown et al., 2013; Saleh-Langenberg et al., 2017). Furthermore, two of the three publications stated that users failed to call the emergency medical services after the injection (Brown et al., 2013; Saleh-Langenberg et al., 2017). In the “not referring to the IFU” subcategory, one publication reported that patients did not follow this guidance when it would have been necessary (Jeannerot et al., 2016).

With regard to the “disposal” category, one must draw a distinction between the complete disposal of the device and (for devices that can be used to administer multiple doses) disposal of the needle only. One publication stated that the inner needle cap was used for disposal instead of the outer needle cap (Lange et al., 2014). In another case, the needle was not safely disposed of (Schertz et al., 2011). In relation to the device disposal, one publication reported that the auto-injector was not placed in a sharps container (Schiff et al., 2016). Another publication stated that the injection device was not put in a safe place after the injection (Brown et al., 2013).

The “dosage” category encompasses all errors related to checking the dose before or after the injection, the calculation or interpretation of remaining doses, and errors in the dose setup process itself. Errors related to an “incorrect dose setting” were most common. These refer to either setting up an incorrect dose (Stauder et al., 2014; Fujioka et al., 2015; Rohrer et al., 2013; Jeannerot et al., 2016; Lange et al., 2014; Schertz et al., 2011; Pfützner et al., 2012; Mahony et al., 2015; Saunders et al., 2012) or not setting a dose at all (Fujioka et al., 2015; Rohrer et al., 2013). A close call was also reported in this context: an incorrect dial-up of the remaining dose, which was corrected before the injection (Mahony et al., 2015). For the “dose misinterpretation/miscalculation” category, three publications were identified. One publication reported that users either misjudged the number of doses left in the disposable pen injector or could not determine this number (Stauder et al., 2014). Another publication reported that users miscalculated the doses when an injection had to be split across two devices because the medication remaining in a single pen injector would not be enough to deliver a complete dose (Fujioka et al., 2015). In this context, it was also reported that users did not realize that the device could not be set to give the required split dose because the device did not contain enough medication or was already empty (Jeannerot et al., 2016). Furthermore, one publication mentioned a close call in the context of split-dose calculation; the error was corrected by referring to the IFU before performing the injection (Mahony et al., 2015). Three publications reported that users failed to check the dose on the device prior to an injection (Stauder et al., 2014; Schertz et al., 2011; Saunders et al., 2012). Four publications mentioned that users failed to visually confirm after an injection that the correct dose had been administered (Phillips et al., 2011; Hudry et al., 2017; Schertz et al., 2011; Saunders et al., 2012).

With regard to the “hygiene” category, three types of use error were identified. Four publications reported that the injection site was not disinfected (Schiff et al., 2016; Domańska et al., 2017; Jeannerot et al., 2016; Pfützner et al., 2012). One of these also mentioned that users did not wash their hands before using the device (Schiff et al., 2016). Touching the needle/injection tip was described in three publications (Salter et al., 2014; Guerlain et al., 2010a; Topal et al., 2013).

The “injection” category encompasses all use errors directly related to the injection. The most frequently reported use error was an “incorrect duration of injection”. These errors are mostly related to an insufficient injection time, resulting in the administration of incomplete dosages. For instance, common adrenaline auto-injectors like the Anapen or EpiPen should be held in place for at least five seconds (Umasunthar et al., 2015; Bakirtas et al., 2011; Mehr et al., 2007; Arga et al., 2011; Hanna et al., 2016) to ensure complete delivery of the dose. In some studies of the Anapen/EpiPen, the recommended injection time was 10 s (Salter et al., 2014; Brown et al., 2013; Topal et al., 2013; Saleh-Langenberg et al., 2017). The users often failed to hold the device in place long enough while injecting, and thus failed to administer a full dose. Furthermore, we identified two instances of use errors related to an excessively long injection time (Schertz et al., 2011; Saunders et al., 2012). Creating a skin fold or pinching the skin is often recommended for the subcutaneous administration of medications. “Incorrect pinching/folding of skin for injection” refers to cases in which users failed to perform this action (Schiff et al., 2016) or did not do it as instructed (e.g. with the wrong fingers or not for the entire injection time) (Hudry et al., 2017). Another reported use error was “significant movement of the device during injection” (Guerlain et al., 2010a). For the “performance of several injections with same needle/device” subcategory, two types of use error were identified. One publication reported that users tried to perform more than one injection with the same device (Guerlain et al., 2010a). This study was a comparative evaluation of four different adrenaline auto-injector training devices. For one of the devices, the error was caused by incorrect feedback; the device continued to instruct the user to perform the injection even after they already had done so. For the other three devices, the reason was not stated. The other error included in this subcategory was related to a disposable pen injector that can deliver multiple dosages. In the corresponding study, participants failed to use a new needle for the second injection (Jeannerot et al., 2016). Another frequently reported injection-related error was the use of an “incorrect injection site”. For instance, adrenaline/epinephrine should be administered intramuscularly - preferable in the outer aspect of thigh. In the corresponding publications, however, some injections were not intramuscular or were performed in muscles other than those of the outer thigh (Sicherer et al., 2000; Salter et al., 2014; Brown et al., 2013; Guerlain et al., 2010a; Bakirtas et al., 2011; Mehr et al., 2007; Arga et al., 2011; Topal et al., 2013; Saleh-Langenberg et al., 2017; Schmid et al., 2013).

With regard to the “needle removal” category, one can distinguish between use errors related to removal of the needle from the device, and those related to removal of the needle from the skin. In the “incorrect removal from device” subcategory, two publications reported that users did not remove the needle from the device after an injection (Fujioka et al., 2015; Jeannerot et al., 2016). The other two references for this subcategory described that users did not unscrew the needle from the device with the needle cap, as was recommended (Schertz et al., 2011; Saunders et al., 2012). For the “incorrect removal from skin” subcategory, two references were identified (Schertz et al., 2011; Saunders et al., 2012). However, neither gave any further information about this use error.

The “device orientation” category refers to errors related to not holding a device in the correct way. The great majority of errors in this category concern holding a device upside down (Lange et al., 2015; Sicherer et al., 2000; Robinson et al., 2014; Matfin et al., 2015; Salter et al., 2014; Umasunthar et al., 2015; Bakirtas et al., 2011; Mehr et al., 2007; Arga et al., 2011; Topal et al., 2013; Saleh-Langenberg et al., 2017; Hanna et al., 2016; Schmid et al., 2013; Hudry et al., 2017; Domańska et al., 2017; Guerlain et al., 2010b; Andre et al., 2017). In conjunction with the “wrong angle” subcategory, one study reported that users did not place the device perpendicular to the thigh for the injection or did not withdraw the device straight out after the injection (Phillips et al., 2011).

The “preparation” category encompasses all use errors that can occur while setting up a device for an injection. This includes required checks on the device and liquid contained therein. For the “incorrect needle attachment” subcategory, two types of errors were defined. Six publications stated that users did not attach a needle to the auto-injection device (Stauder et al., 2014) or did not attach the needle correctly (Stauder et al., 2014; Fujioka et al., 2015; Lange et al., 2014; Schertz et al., 2011; Pfützner et al., 2012; Pfützner et al., 2010). With the exception of one publication (which reported that the needle was attached at the wrong angle (Lange et al., 2014)), no further information about these use errors was provided. The other error in this subcategory is related to the fact that users failed to hold the device in the required upright position during the needle attachment (Phillips et al., 2011). The “incorrect priming” subcategory refers to use errors related to checking for and removing air bubbles or performing a flow check. Four publications reported that users did not check for or remove an air bubble when present (Fujioka et al., 2015; Jeannerot et al., 2016; Lange et al., 2014; Schertz et al., 2011). One of those publications mentioned that participants did not set the device to the minimum dosage, as recommended for the priming process (Jeannerot et al., 2016). Another publication stated that the cartridge of the used device was not tapped before priming or that the needle was pointing downwards during priming (Lange et al., 2014). Furthermore, one publication reported that users primed the device twice (Stauder et al., 2014). In the latter study, participants had to perform several injections; the error was related to the fact that users unnecessarily primed the device again after the first injection. This publication also reported that users primed directly into the injection pad, rather than into a cup as recommended. With regard to the “incorrect device selection” subcategory, two publications mentioned that participants used the wrong device. In one study, participants had to pick up the correct auto-injector from a fridge containing several comparator product boxes (Fujioka et al., 2015). In the other study, participants used a training device instead of the real auto-injector (Raffa et al., 2017). The “failure to check device integrity” subcategory refers to cases in which users did not check whether the auto-injector was damaged (Schiff et al., 2016) or whether the tamper-evident seal was intact (Hudry et al., 2017). Two publications reported that users failed to check the integrity of the outer needle cap (Schertz et al., 2011; Saunders et al., 2012). With regard to the “failure to check product validity” subcategory, three publications mentioned that users failed to check the indicated expiration date (Schiff et al., 2016) or were carrying devices that had expired (Sicherer et al., 2000; Schmid et al., 2013). Furthermore, two publications reported that users failed to check the liquid (Hudry et al., 2017; Schiff et al., 2016). The last subcategory (“tampering with device”) refers to a publication that stated that users could not activate an injection because they had tampered with the device during its preparation (Andre et al., 2017).

The “protective/safety cap” category includes use errors divided into four subcategories. “Incorrect needle capping after the injection” refers to cases in which users failed to cover or cap the needle after the injection or did not cover it as recommended. Four publications reported that users did not cap the needle after an injection (Phillips et al., 2011; Hudry et al., 2017; Schertz et al., 2011; Saunders et al., 2012). Two publications state that the needle capping was not done as recommended (Jeannerot et al., 2016; Mahony et al., 2015). For instance, they capped the needle using one hand, rather than two (Mahony et al., 2015). Removal of the protective/safety cap prior to an injection concerned cases in which users did not remove the cap at all, and cases in which the users did not remove the cap as recommended. For the first case (“cap not removed”), users were not aware that they had to do this, or they simply forgot to do so (Fujioka et al., 2015; Sicherer et al., 2000; Robinson et al., 2014; Matfin et al., 2015; Salter et al., 2014; Brown et al., 2013; Guerlain et al., 2010a; Umasunthar et al., 2015; Bakirtas et al., 2011; Mehr et al., 2007; Arga et al., 2011; Topal et al., 2013; Hanna et al., 2016; Schmid et al., 2013; Rohrer et al., 2013; Hudry et al., 2017; Lange et al., 2014; Schertz et al., 2011; Guerlain et al., 2010b; Andre et al., 2017). In the second case (“incorrect cap removal”), users either did not hold the device correctly during cap removal (Phillips et al., 2011; Brown et al., 2013), or used the wrong hand (Saleh-Langenberg et al., 2017). One publication mentioned that users removed the cap manually rather than by extending the device’s injector shield while they were pointing it away from the body (Phillips et al., 2011). Use errors related to “cap removed in wrong order” were solely reported for the two adrenaline auto-injectors Anapen (Salter et al., 2014) and Twinject (Guerlain et al., 2010a), both of which have two safety caps.

Operational difficulties

All problems that could not clearly be identified as use errors were classified as operational difficulties. In some publications, operational difficulties were systematically documented. In most publications though, either only use errors were reported or there was no clear distinction between use errors and operational difficulties. As mentioned above, it must be noted that operational difficulties were only taken into account if the same issue had not been reported as a use error by the same publication.

We identified 15 instances of an operational difficulty. Table 2 gives the categories of operational difficulties in alphabetical order. A complete list of the identified operational difficulties is given in Additional file 5: Appendix S5.

Table 2 Categories of reported operational difficulties

“Activation” is related to a study in which users needed two attempts to activate the auto-injector on the injection site. The first attempt did not trigger the injection but the second attempt was successful (Jeannerot et al., 2016). “Buttons” refers to two descriptions of operational difficulties. One publication reported that subjects inadvertently pressed the dose button (Fujioka et al., 2015). Another study reported that users had problems distinguishing between the device’s dose setting and its activation button. In this context, they tried to dial the dose with the activation button (Rohrer et al., 2013). Four instances of difficulties with feedback mechanisms were identified. With regard to “feedback - audible”, two publications reported that participants failed to hear the clicks emitted by the device. In one case, they did not notice the click that indicated the start of the injection (Lange et al., 2015). In the other case, they did not notice the click indicating the end of the injection (Varunok et al., 2011). With regard to “feedback - visual”, two publications were identified. In these studies, participants had problems observing the viewing window during the injection, which deprived them of visual feedback during the injection progress (Lange et al., 2015; Lange et al., 2014). “Packaging” refers to a publication that reported difficulties with pulling the auto-injector out of its protective case (Guerlain et al., 2010b). A large proportion of the reported operational difficulties were related to the “safety cap” category, and concerned the cap’s removal. In contrast to the use errors described above, participants were aware in this case that the cap had to be removed prior to injection. However, they had difficulties doing so. With regard to difficulties with “site disinfection” and “washing hands”, participants either inconsistently completed these tasks or needed assistance to perform these tasks (Jeannerot et al., 2016).

Discussion

The goal of the present study was to present the current state of knowledge concerning use errors of disposable auto-injection devices. Our analysis identified a broad range of errors and categorized them with regard to a particular device component or a specific task.

Some of the identified errors are clearly attributable to shortcomings in usability. For example, the high frequency with which auto-injectors were held upside down is obviously due to insufficient identification of safety caps or the end with a needle (e.g. usage of inappropriate colors (Bakirtas et al., 2011)) or the symmetry of the two ends of a device (Guerlain et al., 2010a). One can assume that these design defects have contributed to the unintentional injections in fingers and hands mentioned in a number of publications (Stauder et al., 2014; Guerlain et al., 2010a; Bakirtas et al., 2011; Mehr et al., 2007; Arga et al., 2011). Unclear identification might also contribute to failure to remove the safety cap prior to injection – a relatively common use error. The risk of the occurrence of this type of error could be decreased by better design choices. One study demonstrated that simple design modifications (like changing the color of the safety cap and adding an arrow indicating the correct device orientation) were associated with a significantly lower frequency of needle-stick injuries and a significantly shorter application time (Bakirtas et al., 2011).

Difficulties related to the removal of safety caps can also be due (at least in part) to poor usability. One reason is that a device’s surface does not always provide enough grip to enable the cap to be removed easily - especially with wet hands (Guerlain et al., 2010b). Another reason is that the force required to remove the safety cap might not correspond to the physical capabilities of certain user groups (Lange et al., 2015). However, it must be noted that there is always a trade-off with regard to safety caps; they should neither come off too easily nor be too hard to pull off (Guerlain et al., 2010b).

One publication reported another design defect in which the needle cap lacked a solid backstop. When user covered the needle after the injection, the needle pierced the cap in two cases. In one case, this led to a needle-stick injury (Mahony et al., 2015). Usability evaluations and design reviews constitute an efficient means of identifying these issues and minimizing their occurrence.

Knowledge about use errors of existing, similar equipment is an important source of information on the design and development of new medical devices. A proper analysis of known problems is essential for ensuring that these issues will not occur with a newly developed device. One of the strengths of the present study is its systematic analysis of all use errors reported in the literature. To the best of our knowledge, this is the first review to have focused on use errors with disposable auto- and pen injectors. The present results should help to ensure the safety and intuitiveness of these devices during the development process. Furthermore, the present review should support manufacturers and researchers to perform risk analyses and device evaluations.

The number of publications in each error category gives an idea of the most common use errors associated with auto-injection devices. However, potential reporting and publication biases prevented us from performing a quantitative analysis of the various errors’ frequency of occurrence in the respective studies. Furthermore, we did not distinguish between devices as a function of their medical intention or active substance. Certain functions (such as dose adjustment) were not provided by all the devices reviewed here. This limitation aside, the use errors identified here are all relevant – irrespectively of the particular context of use, medical intention or active substance.

It should also be noted that the level of detail used to describe use errors differed substantially from one publication to another. This affects the interpretation of the results and the comparison of findings from different publications. To tackle this source of potential bias, we assessed each publication’s reporting quality and methodology. Thus, only high-quality studies with sufficiently precise descriptions of use errors were included in our analysis.

Another limitation relates to the fact that unfortunately, most of the publications did not report on the underlying usability flaws that led to the use errors. This might be because (i) the researchers did not look for the root causes of the use errors, (ii) the description of usability flaws was not a study objective or (iii) not all use errors could be clearly assigned to individual design features or user interface elements. In general, this did not affect our analysis of use errors but it might impede the interpretation and transferability of the reported results. Given that no information on the negative outcomes was provided for a large proportion of the identified use errors, we could not rate the severity of the various errors. In any case, this type of rating is not generalizable because it should always take account of the device’s intended use and specific characteristics.

Lastly, it has to be stated that regardless of the problems described above, the usability of disposable auto- and pen injectors has progressed significantly since the first devices came onto the market. This is not least due the increase in the number of companies working in this field. Redesigned needle covers, improved feedback mechanisms, and other optimizations facilitate the correct usage and enhance the safety of these devices.

The global auto-injector market is expected to grow further in coming years, and the technology is evolving rapidly. Hence, the present analysis should be updated regularly, in order to take the latest trends and developments into account. In this context, the present review is in line with the European Union’s latest adaptations of medical device regulations. One of the new regulations’ most important objectives is to improve transparency and traceability by establishing a European-wide database on medical devices and a system for unique device identification (UDI) (European Commission 2018). A centralized, complete documentation of all known safety-related issues, accessible to all stakeholders, will help to decrease the repeated occurrence of these problems. Accordingly, we are confident that the present review will help to further improve the safety of disposable auto- and pen injectors. In order to enhance the benefits of this work, the detailed data provided here should be extended by performing an in-depth analysis of incident reports to achieve a holistic view of use errors of disposable auto-injection devices (Marcilly et al., 2016). However, the results of this type of reviews will always be just a development aid and cannot replace the knowledge and experience of human factors engineering experts for the conception and development of medical devices.

Conclusion

The objective of the present review was to identify and classify previously reported use errors of disposable auto-injection devices. We identified 232 instances of a use error and two instances of a close call, which were classified into 10 categories and 39 subcategories. Furthermore, 15 instances of an operational difficulty were found and categorized. We hope that these results can be used as guidance in the design, evaluation and risk analysis of disposable auto- and pen injectors, in order to improve both the usability and safety of auto-injection devices for the benefit of their users.