Electronic patient-reported outcome

An electronic patient-reported outcome (ePRO) is a patient-reported outcome that is collected by electronic methods. ePRO methods are most commonly used in clinical trials, but they are also used elsewhere in health care. As a function of the regulatory process, a majority of ePRO questionnaires undergo the linguistic validation process. When the data is captured for a clinical trial, the data is considered a form of Electronic Source Data.

Methods

The two main methods currently used for ePRO are computers (including smartphones) and telephone systems.

Computers are most often touch-screen devices, ranging from wearables, hand-held devices and smartphones, through iPads and other Tablet computers. The smaller devices are often used as electronic diaries, designed to be used for symptom reporting on a daily basis. Larger devices are generally used in a clinical setting. Computers generally run dedicated ePRO applications - the use of the web for ePRO is not yet widespread. Typically a single question at a time is presented on the screen, with a set of possible response options. The user taps on the appropriate response with a finger or stylus, then moves on to the next questions.

Telephones normally use an interactive voice response system (IVR). The user calls into a dedicated phone line, and hears a spoken script that details the question, and the possible responses. Each response option is given a number, and the user presses the corresponding number key on the phone keypad to record the choice. IVR systems are more often used for diaries, with the patient phoning in e.g. from home, but they can be used in a clinic setting.

There are also a number of custom devices designed specifically for use as ePRO data collection devices.

Electronic diaries

Diaries are used when it is desirable to obtain frequent assessments over a period of time, for example when a condition fluctuates in severity. In such cases recall of severity over a period of time is unlikely to be accurate. Research has shown substantial bias in such summary recall, with ratings unduly influenced by how the patient is feeling at the time of making the rating, and by maximum severity rather than average severity during the assessment interval. ^[1] Diaries can overcome this problem by recording severity either on a momentary basis ("How bad is your pain right now?") or by recalling over short periods ("How bad has your pain been today?"). However, when diary data is collected on paper, it is not known when the ratings are actually made, and there is evidence that compliance may be quite poor. In one study, patients were given an instrumented paper diary that recorded covertly when it was opened. The study showed frequent cases of "back-filling", filling in a batch of entries sometimes, often days, after they were due, and even in some cases of "forward filling", completing entries before they were due. ^[2]

Electronic diaries automatically time-stamp all entries and can be set up to only allow entry within specified time windows. This improves compliance and ensures that true compliance can be documented. Documenting compliance is important if ePRO data are to be used to support regulatory applications. ^[3] ePRO applications typically achieve compliance rates of over 80%, often over 90%. ^{[4] [5]} Electronic diaries also have benefits in that they only allow valid, in-range entries to be made. Devices such as PDAs allow reminders to be given to patients when entries are due. Systems generally transfer data promptly to a central server, allowing tailored feedback to be given to patients. This can also improve compliance. Electronic diaries also eliminate the need for manual editing and entry of data, time-consuming and error-prone processes.

Site-based

The other main setting for ePRO is the clinic, with questionnaires completed when patients come in for their scheduled visits. In this supervised situation, compliance is less of an issue. The questionnaires used in site-based ePRO are often longer and more complex than those used in diaries, assessing quality of life and activities of daily living, for example, in some detail. They more often include branching logic ("if YES continue with the next question, if NO, go to question 34"). Such branching logic can be handled automatically by the ePRO application, and it is often not necessary for the patient even to know that branching is taking place. This makes it easier for the patient to use.

As with electronic diaries, the prevention of out-of-range or inconsistent entries, and the elimination of manual editing and data entry are important features of site-based ePRO. Missing data within a questionnaire can be reduced or eliminated, and this is important as missing data has been identified as a crucial quality issue in questionnaire data. ^[6]

Patient acceptability

From the early years of ePRO there has been concern about whether all patients can cope with computer technology. This is important, as if significant numbers of patients refuse to take part in clinical trials because of dislike of computers then there will be bias in the study population. One of the earliest ePRO studies used a LINC-2 minicomputer to collect patient data. The majority of patients preferred the computer to paper data collection. ^[7] Similar findings have been reported from many later studies. ^{[8] [9]}

Elderly patients, and those not familiar with computers, might be expected to have more problems. But these groups also show high acceptance of ePRO, and again often prefer it to paper. ^{[10] [11] [12]} Thus there seems to be no great barrier to recruiting representative patient samples in ePRO studies.

Validity and equivalence

Establishing the validity of an ePRO instrument is in principle no different from that for other methods, such as paper. However most of the instruments in current use have been validated in paper form, and we need to ask (1) whether the paper data can be used to establish validity of the electronic version, and (2) whether data from paper and electronic versions can be used interchangeably.

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has issued guidelines on establishing an equivalence between modes of administration. ^[13] Their approach is hierarchical and depends on the degree of change made during the process of migration from paper to electronic format. Three levels are suggested. At the lowest level, where the least change has been made, cognitive interviewing of patients as a check that they construe ePRO and paper, in the same way, is sufficient. This level includes both trivial changes (touch rather than circle a response choice) as well as changes that are supported by empirical findings in the literature. At the second level, equivalence studies comparing the scores obtained from the two modes should be carried out. At the third level, where most change has occurred, the ePRO instrument must be treated as a new instrument, and complete psychometric validation carried out.

There is a great deal of evidence supporting the general equivalence of paper and ePRO methods. Gwaltney and colleagues ^[14] have reported a meta-analysis in which they included 46 studies evaluating 278 scales. They concluded that there was good agreement between paper and ePRO, and no evidence of systematic bias. This general finding, of course, does not guarantee that any specific migration will lead to equivalence, and each case must be reviewed and documented.

It is not always necessary to validate an ePRO measure against a pre-existing paper version. In some cases, an instrument may be developed and validated from the beginning in electronic form. More commonly, perhaps, new instruments will be developed in parallel for paper and electronic use, as is the case with the PROMIS (Patient-Reported Outcomes Measurement Information System) initiative. ^[15]

Standards of validity must be maintained throughout every target language population. In order to ensure that developmental standards are consistent in translated versions of an ePRO instrument, the translated instrument undergoes a process known as Linguistic validation in which the preliminary translation is adapted to reflect cultural and linguistic differences between diverse target populations. This process also accounts for any formatting errors that may occur in languages using non-Roman fonts.

In practice

Several successful regulatory approvals have used ePRO data in recent years, including ketorolac for ocular pain, eszopiclone for insomnia, milnacipran for fibromyalgia, estradiol/levonorgestrel for post-menopausal symptoms, and ruxolitinib for myelofibrosis. ^[16] In the case of estradiol/levonorgestrel, detailed ePRO data on bleeding/spotting from a one-year clinical trial are presented in the patient information leaflet. ^[17]

As well as clinical trial use, ePRO methods may be used to support patients in regular care. An example of this is the collection of symptom data from patients undergoing chemotherapy, using handheld diaries. This allows clinic staff to monitor outpatients, and to identify the occurrence of adverse reactions that may require intervention. ^[18]

References

1. Stone AA, Broderick JE, Shiffman SS, Schwartz JE (2004). "Understanding recall of weekly pain from a momentary assessment perspective: absolute agreement, between- and within-person consistency, and judged change in weekly pain". Pain. 107 (1–2): 61–69. doi:10.1016/j.pain.2003.09.020. PMID   14715390. S2CID   34037052.
2. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR (2002-05-18). "Patient non-compliance with paper diaries". British Medical Journal. 324 (7347): 1193–1194. doi:10.1136/bmj.324.7347.1193. PMC   111114 . PMID   12016186.
3. "Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims" (PDF). Food and Drug Administration . 15 May 2020.
4. Meltzer EO, Kelley N, Hovell MF (2008). "Randomized, Cross-Over Evaluation of Mobile Phone vs Paper Diary in Subjects with Mild to Moderate Persistent Asthma". Open Respir. Med. J. 2: 72–79. doi:10.2174/1874306400802010072. PMC   2606655 . PMID   19412327.
5. Friedberg F, Sohl SJ (2008). "Memory for Fatigue in Chronic Fatigue Syndrome: The Relation Between Weekly Recall and Momentary Ratings". International Journal of Behavioral Medicine. 15 (1): 29–33. doi:10.1007/bf03003071. PMID   18444018. S2CID   144865647.
6. Bernhard J, Cella DF, Coates AS, Fallowfield L, Ganz PA, Moinpour CM, Mosconi P, Osoba D, Simes J, Hurny C (1998-03-15). "Missing quality of life data in cancer clinical trials: serious problems and challenges". Statistics in Medicine. 17 (5–7): 517–532. doi:10.1002/(sici)1097-0258(19980315/15)17:5/7<517::aid-sim799>3.3.co;2-j. PMID   9549801.
7. Slack WV, Hicks GP, Reed CE, Van Cura LJ (1966-01-27). "A computer-based medical-history system". New England Journal of Medicine. 274 (4): 194–198. doi:10.1056/NEJM196601272740406. PMID   5902618.
8. Crawley JA, Kleinman L, Dominitz J (2000). "User preferences for computer administration of quality of life instruments". Drug Information Journal. 34: 137–144. doi:10.1177/009286150003400119. S2CID   57179467.
9. Richter JG, Becker A, Koch T, Nixdorf M, Willers R, Monser R, Schacher B, Alten R, Specker C, Schneider M (2008). "Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires". Annals of the Rheumatic Diseases. 67 (12): 1739–1741. doi:10.1136/ard.2008.090209. PMID   18647853. S2CID   45468097.
10. Yarnold PR, Stewart MJ, Stille FC, Martin GJ (1996). "Assessing functional status of elderly adults via microcomputer". Perceptual and Motor Skills. 82 (2): 689–690. doi:10.2466/pms.1996.82.2.689. PMID   8724947. S2CID   36252525.
11. Allenby A, Matthews J, Beresford J, McLachlan SA (2002). "The application of computer touch-screen technology in screening for psychosocial distress in an ambulatory oncology setting". European Journal of Cancer Care. 11 (4): 245–253. doi:10.1046/j.1365-2354.2002.00310.x. PMID   12492461.
12. Tiplady B, Goodman K, Cummings G, Lyle D, Carrington R, Battersby C, Ralston S (2010). "Patient Reported Outcomes in Rheumatoid Arthritis: Assessing the equivalence of electronic and paper data collection". The Patient. 3 (3): 133–143. doi:10.2165/11535590-000000000-00000. S2CID   74444503.
13. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, Lenderking WR, Cella D, Basch E (2009). "Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report". Value in Health. 12 (4): 419–429. doi: 10.1111/j.1524-4733.2008.00470.x . PMID   19900250.
14. Gwaltney CJ, Shields AL, Shiffman S (2008). "Equivalence of Electronic and Paper-and-Pencil Administration of Patient-Reported Outcome Measures: A Meta-Analytic Review". Value in Health. 11 (2): 322–333. doi: 10.1111/j.1524-4733.2007.00231.x . PMID   18380645.
15. Fries, J.; Cella, D.; Rose, M.; Krishnan, E. & Bruce, B. (2009). "Progress in Assessing Physical Function in Arthritis: PROMIS Short Forms and Computerized Adaptive Testing". The Journal of Rheumatology. 36 (9): 2061–2066. doi: 10.3899/jrheum.090358 . PMID   19738214. S2CID   22351061.
16. McCallister E, Usdin S (5 December 2011). "A PROfessional Trial". BioCentury.
17. "DailyMed".
18. Kearney, N.; McCann, L.; Norrie, J.; Taylor, L.; Gray, P.; Gee-Lennon, M.; Sage, M.; Miller, M. & Maguire, R. (2009-01-04). "Evaluation of a mobile phone-based, advanced symptom management system (ASyMS-¬) in the management of chemotherapy-related toxicity" (PDF). Supportive Care in Cancer. 17 (4): 437–444. doi:10.1007/s00520-008-0515-0. hdl: 1893/845 . PMID   18953579. S2CID   9840362.

Further reading

• Byrom B, Tiplady B (eds.) ePRO: Electronic Solutions for Patient-Reported Data (2010) Farnham: Gower. ISBN   978-0-566-08771-4.
• Byrom B, Mühlhausen W Electronic Patient-reported Outcome Measures: An implementation handbook for clinical research(2018) ISBN   978-1-72028-110-8
• Stone AA, Shiffman S, Atienza AA, Nebeling L (eds.) The Science of Real-Time Data Capture (2007) Oxford: Oxford University Press. ISBN   978-0-19-517871-5.

External links

• International Society for Pharmacoeconomomics and Outcomes Research
• Patient-Reported Outcomes Measurement Information System