Routine health outcomes measurement

Last updated

Definition of health outcomes

Routine health outcomes measurement is the process of examining whether or not interventions are associated with change (for better or worse) in the patient's health status. This change can be directly measured (e.g. by rating scales used by the clinician or patient) or assumed by the use of proxy measurement (e.g. a blood test result). Interventions can be direct (e.g. medication) or indirect (e.g. change in the process of health care like integration care by different specialists). Some definitions of health outcomes measurement stipulate that the population or group has to be defined (different outcomes are expected for different people & conditions). A strong example is that of Australia's New South Wales Health Department: health outcome is

Contents

"change in the health of an individual, group of people or population which is attributable to an intervention or series of interventions" [1]

In its purest form, measurement of health outcomes implies identifying the context (diagnosis, demographics etc.), measuring health status before an intervention is carried out, measuring the intervention, measuring health status again and then plausibly relating the change to the intervention.

Health outcomes measurement and evidence-based practice

Evidence-based practice describes a healthcare system in which evidence from published studies, often mediated by systematic reviews or processed into medical guidelines is incorporated into clinical practice. The flow of information is one way; from research to practice. However many interventions by health systems and treatments by their staff have never been, or cannot easily be, subject to research study. Of the rest, quite a lot is from research that is graded as low quality. [2] All health staff intervene in their patients on the basis of both information from research evidence and from their own experience. The latter is personal, subjective and strongly influenced by stark instances which may not be representative. [3] However, when information on these interventions and their outcomes are collected systematically it becomes "practice-based evidence" [4] and can complement that from academic research. To date, such initiatives have been largely confined to primary care [5] and rheumatology. [6] An example of practice-based evidence is found in the evaluation of a simple intervention like a medication. Efficacy is the degree with which it can improve patients in randomised controlled trials– the epitome of evidence-based practice. Effectiveness is the degree with which the same drug improves patients in the uncontrolled hurly-burly of everyday practice; data which are much more difficult to come by. Routine health outcomes measurement has the potential to provide such evidence.

The information required for practice-based evidence is of three sorts: context (e.g. case mix), intervention (treatment) and outcomes (change). [7] Some mental health services are developing a practice-based evidence culture with the routine measurement of clinical outcomes [8] [9] and creating behavioral health outcomes management programs.

History of routine health outcomes measurement

Florence Nightingale

An early example of a routine clinical outcomes system was set up by Florence Nightingale in the Crimean War. The outcome under study was death. The context was the season and the cause of death– wounds, infections and any other cause. The interventions were nursing and administrative. She arrived just before the barracks in Scutari were accepting the first soldiers wounded at the battle of Inkerman in November 1854, and mortality was already high. She was appalled at the disorganisation and standards of hygiene and set about cleaning and reorganisation. However, mortality continued to rise. It was only after the sewers were cleared and ventilation improved in March 1856 that mortality fell. On return to the UK she reflected on these data and produced new sorts of chart (she had trained in mathematics rather than "worsted work and practising quadrilles") to show that it was most likely that these excess deaths were caused by living conditions rather than, as she initially believed, poor nutrition. She also showed that soldiers in peacetime also had an excess mortality over other young men, presumably from the same causes. Her reputation was damaged, however, when she and William Farr, Registrar General, collaborated in producing a table which appeared to show a mortality in London hospitals of over 90% compared with less than 13% in Margate. They had made an elementary error in the denominator; the true rate for London hospitals was actually 9% for admitted patients. [10] She was never too keen on hospital mortality figures as outcome measures anyway:

"If the function of a hospital were to kill the sick, statistical comparisons of this nature would be admissible. As, however, its proper function is to restore the sick to health as speedily as possible, the elements which really give information as to whether this is done or not, are those which show the proportion of sick restored to health, and the average time which has been required for this object…" [11]

Here she presaged the next key figure in the development of routine outcomes measurement

Ernest Amory Codman

Codman was a Boston orthopaedic surgeon who developed the "end result idea". At its core was

"The common sense notion that every hospital should follow every patient it treats, long enough to determine whether or not the treatment has been successful, and then to inquire 'if not, why not?' with a view of preventing similar failures in the future." [12]

He is said to have first articulated this idea to his gynaecologist colleague and Chicagoan Franklin H Martin, who later founded the American College of Surgeons, in a Hansom Cab journey from Frimley Park, Surrey, UK in the summer of 1910. He put this idea into practice in Massachusetts General Hospital.

"Each patient who entered the operating room was provided with a 5-inch by 8-inch card on which the operating surgeon filled out the details of the case before and after surgery. This card was brought up 1 year later, the patient was examined, and the previous years' treatment was then evaluated based on the patient's condition. This system enabled the hospital and the public to evaluate the results of treatments and to provide comparisons among individual surgeons and different hospitals" [13]

He was able to demonstrate his own patients’ outcomes and those of some of his colleagues but unaccountably this system was not embraced by his colleagues. Frustrated by their resistance, he provoked an uproar at a public meeting and thus fell dramatically from favour in the hospital and at Harvard, where he held a teaching post, and he was only able to fully realize the idea in his own, struggling small private hospital [14] although some colleagues continued with it at the larger hospitals. He died in 1940 disappointed that his dream of publicly available outcomes data was not even on the horizon, but hoped that posterity would vindicate him.

Avedis Donabedian

In a classic 1966 paper, Avedis Donabedian, the renowned public health pioneer, described three distinct aspects of quality in health care: outcome, process and structure (in that order in the original paper). [15] He had misgivings about solely using outcomes as a measure of quality, but concluded that:

"Outcomes, by and large, remain the ultimate validation of the effectiveness and quality of medical care." [15]

He may have muddied the waters a bit when discussing patient satisfaction with treatment (usually regarded as a measure of process) as an outcome, but more importantly it has become apparent that his three-aspect model has been subverted into what is called the "structure-process-outcomes" model, a directional, putatively causal chain that he never originally described. This subversion has been the justification for repeated attempts to improve process and thus outcomes by reorganizing the structure of health care, wittily described by Oxman et al. [16] Donabedian himself cautioned that outcomes measurement cannot distinguish efficacy from effectiveness: (outcomes may be poor because the right treatment is badly applied or the wrong treatment is carried out well), that outcomes measurement must always take into account context (factors other than the intervention may be very important in determining outcomes), and also that the most important outcomes may be the least easy to measure, so easily measured but irrelevant outcomes are chosen (e.g. mortality instead of disability).

Mortality as an outcome measure

Perhaps because of instances of scandalously poor care (for example at the Bristol Royal Infirmary 1984-1995 [17] ) mortality data have become more and more openly available as a proxy for other health outcomes in hospitals, [18] and even for individual surgeons. [19] For many people, quality of life is a greater consideration so factors such as physical symptoms, psychological, emotional and spiritual, and information and support needs may take greater precedence. Therefore, as an indicator of the quality and safety of health care institutions, mortality remains important, but for individuals, it may not be the key goal. [20]

Principles of routine health outcomes measurement

  1. All three dimensions (context, intervention as well as outcomes) must be measured. It is not possible to understand outcomes data without all three of these.
  2. Different perspectives on outcomes need to be acknowledged. For instance, patients, carers and clinical staff may have different views of what outcomes are important, how you would measure them, and even which were desirable [21]
  3. Prospective and repeated measurement of health status is superior to retrospective measurement of change such as Clinical Global Impressions. [22] The latter relies on memory and may not be possible if the rater changes.
  4. The reliability (statistics) and validity (statistics) of any measure of health status must be known so that their impact on the assessment of health outcomes can be taken into account. In mental health services these values may be quite low, especially when carried out routinely by staff rather than by trained researchers, and when using short measures that are feasible in everyday practice.
  5. Data collected must be fed back to them to maximize data quality, reliability and validity. [23] Feedback should be of content (e.g. relationship of outcomes to context and interventions) and of process (data quality of all three dimensions)

Current status of routine health outcomes measurement

Why is routine health outcomes measurement so rare? One can find reports of routine health outcomes measurement in many medical specialties and in many countries. However, the vast majority of these reports are by or about enthusiasts who have set up essentially local systems, with little connection with other similar systems elsewhere, even down the street. In order to realise the full benefits of an outcomes measurement system we need large-scale implementation using standardised methods with data from high proportions of suitable healthcare episodes being trapped. In order to analyse change in health status (health outcomes) we also need data on context, as recommended by Donabedian [15] and others, and data on the interventions being used, all in a standardised manner. Such large-scale systems are only at present evident in the field of mental health services, and only well developed in two locations: Ohio [8] and Australia, [9] even though in both of these data on context and interventions are much less prominent than data on outcomes. The major challenge for health outcomes measurement is now the development of usable and discriminatory categories of interventions and treatments, especially in the field of mental health.

Benefits of routine health outcomes measurement

Aspirations include the following benefits

Risks of routine health outcomes measurement

  1. If attempts are made to purchase or commission health services using outcomes data, bias may be introduced that will negate the benefits, especially in the service provider produces the outcomes measurement. See Goodhart's Law
  2. Inadequate attention may be paid to the analysis of context data, such as case mix, leading to dubious conclusions. [27]
  3. If data are not fed back to clinicians participating then data quality (and quantity) is likely to fall below the thresholds necessary for reasonable interpretation. [28]
  4. If only a small proportion of episodes of health care have completed outcomes data, then these data may not be representative of all episodes, although the threshold for this effect will vary from service to service, measure to measure.
  5. Some risks of bias, widely foretold, [29] are proving to be insubstantial but need guarding against

Practical issues in routine health outcomes measurement

Experience suggests that the following factors are necessary for routine health outcomes measurement

  1. an electronic patient record system with easy extraction from data warehouse. Entry of outcomes data can then become part of the everyday entry of clinical data. Without this, aggregate data analysis and feedback is very difficult indeed.
  2. resources and staff time set aside for training and receiving feedback
  3. resources and personnel to extract, analyse and proactively present outcomes, casemix and, where available, intervention data to clinical teams
  4. regular reports on data quality as part of a performance management process by senior managers can supplement, but not replace, feedback

Shared decision making [30]

Outcome measurement is therefore an important but neglected tool in improving quality of healthcare provision. It has been argued that it is vital that the patient has been meaningfully involved in decisions about whether or not to embark on an intervention (e.g. a test, an operation, a medicine). This is especially so if the decision is fateful (i.e. cannot be reversed). [31] Although a process rather than an outcome measure, the degree with which patients have been involved in shared decision making is clearly important. [32]

Related Research Articles

Evidence-based medicine (EBM) is "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients". The aim of EBM is to integrate the experience of the clinician, the values of the patient, and the best available scientific information to guide decision-making about clinical management. The term was originally used to describe an approach to teaching the practice of medicine and improving decisions by individual physicians about individual patients.

<span class="mw-page-title-main">Medical guideline</span> Document with the aim of guiding decisions and criteria in healthcare

A medical guideline is a document with the aim of guiding decisions and criteria regarding diagnosis, management, and treatment in specific areas of healthcare. Such documents have been in use for thousands of years during the entire history of medicine. However, in contrast to previous approaches, which were often based on tradition or authority, modern medical guidelines are based on an examination of current evidence within the paradigm of evidence-based medicine. They usually include summarized consensus statements on best practice in healthcare. A healthcare provider is obliged to know the medical guidelines of his or her profession, and has to decide whether to follow the recommendations of a guideline for an individual treatment.

Clinical governance is a systematic approach to maintaining and improving the quality of patient care within the National Health Service (NHS). Clinical governance became important in health care after the Bristol heart scandal in 1995, during which an anaesthetist, Dr Stephen Bolsin, exposed the high mortality rate for paediatric cardiac surgery at the Bristol Royal Infirmary. It was originally elaborated within the United Kingdom National Health Service (NHS), and its most widely cited formal definition describes it as:

A framework through which NHS organisations are accountable for continually improving the quality of their services and safeguarding high standards of care by creating an environment in which excellence in clinical care will flourish.

In the healthcare industry, pay for performance (P4P), also known as "value-based purchasing", is a payment model that offers financial incentives to physicians, hospitals, medical groups, and other healthcare providers for meeting certain performance measures. Clinical outcomes, such as longer survival, are difficult to measure, so pay for performance systems usually evaluate process quality and efficiency, such as measuring blood pressure, lowering blood pressure, or counseling patients to stop smoking. This model also penalizes health care providers for poor outcomes, medical errors, or increased costs. Integrated delivery systems where insurers and providers share in the cost are intended to help align incentives for value-based care.

A hierarchy of evidence is a heuristic used to rank the relative strength of results obtained from scientific research. There is broad agreement on the relative strength of large-scale, epidemiological studies. More than 80 different hierarchies have been proposed for assessing medical evidence. The design of the study and the endpoints measured affect the strength of the evidence. In clinical research, the best evidence for treatment efficacy is mainly from meta-analyses of randomized controlled trials (RCTs). Systematic reviews of completed, high-quality randomized controlled trials – such as those published by the Cochrane Collaboration – rank the same as systematic review of completed high-quality observational studies in regard to the study of side effects. Evidence hierarchies are often applied in evidence-based practices and are integral to evidence-based medicine (EBM).

Patient safety is a discipline that emphasizes safety in health care through the prevention, reduction, reporting and analysis of error and other types of unnecessary harm that often lead to adverse patient events. The frequency and magnitude of avoidable adverse events, often known as patient safety incidents, experienced by patients was not well known until the 1990s, when multiple countries reported significant numbers of patients harmed and killed by medical errors. Recognizing that healthcare errors impact 1 in every 10 patients around the world, the World Health Organization (WHO) calls patient safety an endemic concern. Indeed, patient safety has emerged as a distinct healthcare discipline supported by an immature yet developing scientific framework. There is a significant transdisciplinary body of theoretical and research literature that informs the science of patient safety with mobile health apps being a growing area of research.

Transitional care refers to the coordination and continuity of health care during a movement from one healthcare setting to either another or to home, called care transition, between health care practitioners and settings as their condition and care needs change during the course of a chronic or acute illness. Older adults who suffer from a variety of health conditions often need health care services in different settings to meet their many needs. For young people the focus is on moving successfully from child to adult health services.

Maintenance of Certification (MOC) is a recently implemented and controversial process of physician certification maintenance through one of the 24 approved medical specialty boards of the American Board of Medical Specialties (ABMS) and the 18 approved medical specialty boards of the American Osteopathic Association (AOA). The MOC process is controversial within the medical community, with proponents claiming that it is a voluntary program that improves physician knowledge and demonstrates a commitment to lifelong learning. Critics claim that MOC is an expensive, burdensome, involuntary and clinically irrelevant process that has been created primarily as a money-making scheme for the ABMS and the AOA.

A depression rating scale is a psychometric instrument (tool), usually a questionnaire whose wording has been validated with experimental evidence, having descriptive words and phrases that indicate the severity of depression for a time period. When used, an observer may make judgements and rate a person at a specified scale level with respect to identified characteristics. Rather than being used to diagnose depression, a depression rating scale may be used to assign a score to a person's behaviour where that score may be used to determine whether that person should be evaluated more thoroughly for a depressive disorder diagnosis. Several rating scales are used for this purpose.

Outcomes research is a branch of public health research which studies the end results of the structure and processes of the health care system on the health and well-being of patients and populations. According to one medical outcomes and guidelines source book - 1996, Outcomes research includes health services research that focuses on identifying variations in medical procedures and associated health outcomes. Though listed as a synonym for the National Library of Medicine MeSH term "Outcome Assessment ", outcomes research may refer to both health services research and healthcare outcomes assessment, which aims at Health technology assessment, decision making, and policy analysis through systematic evaluation of quality of care, access, and effectiveness.

<span class="mw-page-title-main">Quality of life (healthcare)</span> Notion in healthcare

In general, quality of life is the perceived quality of an individual's daily life, that is, an assessment of their well-being or lack thereof. This includes all emotional, social and physical aspects of the individual's life. In health care, health-related quality of life (HRQoL) is an assessment of how the individual's well-being may be affected over time by a disease, disability or disorder.

Comparative effectiveness research (CER) is the direct comparison of existing health care interventions to determine which work best for which patients and which pose the greatest benefits and harms. The core question of comparative effectiveness research is which treatment works best, for whom, and under what circumstances. Engaging various stakeholders in this process, while difficult, makes research more applicable through providing information that improves patient decision making.

Case management is the coordination of community-based services by a professional or team to provide quality mental health care customized accordingly to individual patients' setbacks or persistent challenges and aid them to their recovery. Case management seeks to reduce hospitalizations and support individuals' recovery through an approach that considers each person's overall biopsychosocial needs without making disadvantageous economic costs. As a result, care coordination includes traditional mental health services but may also encompass primary healthcare, housing, transportation, employment, social relationships, and community participation. It is the link between the client and care delivery system.

Behavioral health outcome management (BHOM) involves the use of behavioral health outcome measurement data to help guide and inform the treatment of each individual patient. Like blood pressure, cholesterol and other routine lab work that helps to guide and inform general medical practice, the use of routine measurement in behavioral health is proving to be invaluable in assisting therapists to deliver better quality care.

The Donabedian model is a conceptual model that provides a framework for examining health services and evaluating quality of health care. According to the model, information about quality of care can be drawn from three categories: “structure,” “process,” and “outcomes." Structure describes the context in which care is delivered, including hospital buildings, staff, financing, and equipment. Process denotes the transactions between patients and providers throughout the delivery of healthcare. Finally, outcomes refer to the effects of healthcare on the health status of patients and populations. Avedis Donabedian, a physician and health services researcher at the University of Michigan, developed the original model in 1966. While there are other quality of care frameworks, including the World Health Organization (WHO)-Recommended Quality of Care Framework and the Bamako Initiative, the Donabedian Model continues to be the dominant paradigm for assessing the quality of health care.

Symptom targeted intervention (STI) is a clinical program being used in medical settings to help patients who struggle with symptoms of depression or anxiety or adherence to treatment plans but who are not interested in receiving outpatient mental health treatment. STI is an individualized therapeutic model and clinical program that teaches patients brief, effective ways to cope with difficult thoughts, feelings, and behaviors using evidence-based interventions. Its individualized engagement process employs techniques from solution-focused therapy, using a Rogerian, patient-centered philosophy. This engagement process ensures that even challenging, at-risk, and non-adherent patients are able to participate.

Health care quality is a level of value provided by any health care resource, as determined by some measurement. As with quality in other fields, it is an assessment of whether something is good enough and whether it is suitable for its purpose. The goal of health care is to provide medical resources of high quality to all who need them; that is, to ensure good quality of life, cure illnesses when possible, to extend life expectancy, and so on. Researchers use a variety of quality measures to attempt to determine health care quality, including counts of a therapy's reduction or lessening of diseases identified by medical diagnosis, a decrease in the number of risk factors which people have following preventive care, or a survey of health indicators in a population who are accessing certain kinds of care.

Although several medications have been approved in different countries as of April 2022, not all countries have these medications. Patients with mild to moderate symptoms who are in the risk groups can take nirmatrelvir/ritonavir or remdesivir, either of which reduces the risk of serious illness or hospitalization. In the US, the Biden Administration COVID-19 action plan includes the Test to Treat initiative, where people can go to a pharmacy, take a COVID test, and immediately receive free Paxlovid if they test positive.

Harlan M. Krumholz, MD, SM is an American cardiologist, leading research scientist, and the Harold H. Hines, Jr. Professor of Medicine at Yale School of Medicine, where he has been on faculty since 1992. A pioneer in the development of the field of outcomes research, his groundbreaking contributions to science have directly led to improvements in healthcare outcomes for patients and populations. He is an international expert in the science to evaluate and improve the quality and efficiency of care, reduce disparities, improve integrity in medical research, promote better health policies and regulations, and promote patient-centeredness in research and clinical care. He is the founder and director of the Yale New Haven Hospital Center for Outcomes Research and Evaluation.

<span class="mw-page-title-main">Donald L. Patrick</span> American social scientist

Donald L. Patrick is a social scientist, academic, and an author. He is a Professor Emeritus of Health Systems and Population Health at the University of Washington, Director of Seattle Quality of Life Group, and Creator of the Biobehavioral Cancer Prevention and Control Training Program jointly with the Fred Hutchinson Cancer Center. He has served as the co-chair of the Cochrane Collaboration's Patient Reported Outcomes Methods Group. His research interests revolve around various aspects of public health which integrate the themes from fields such as psychological intervention, social stratification, public health, and quality of life. Much of his research works have focused on outcomes research on vulnerable populations, health disparities, and end-of-life-care.

References

  1. Frommer, Michael; Rubin, George; Lyle, David (1992). "The NSW Health Outcomes program". New South Wales Public Health Bulletin. 3 (12): 135. doi: 10.1071/NB92067 .
  2. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ (May 2008). "What is "quality of evidence" and why is it important to clinicians?". BMJ. 336 (7651): 995–8. doi:10.1136/bmj.39490.551019.BE. PMC   2364804 . PMID   18456631.
  3. Malterud K (August 2001). "The art and science of clinical knowledge: evidence beyond measures and numbers". Lancet. 358 (9279): 397–400. doi:10.1016/S0140-6736(01)05548-9. PMID   11502338. S2CID   2248745.
  4. Horn SD, Gassaway J (October 2007). "Practice-based evidence study design for comparative effectiveness research". Medical Care . 45 (10 Supl 2): S50–7. doi:10.1097/MLR.0b013e318070c07b. PMID   17909384. S2CID   18410257.
  5. Ryan JG (1 March 2004). "Practice-Based Research Networking for Growing the Evidence to Substantiate Primary Care Medicine". Annals of Family Medicine. 2 (2): 180–1. PMC   1466650 . PMID   15083861.
  6. Pincus T, Sokka T (March 2006). "Evidence-based practice and practice-based evidence". Nature Clinical Practice Rheumatology. 2 (3): 114–5. doi:10.1038/ncprheum0131. PMID   16932666. S2CID   20794434.
  7. Pawson R, Tilley N. Realistic Evaluation. London: Sage Publications Ltd; 1997
  8. 1 2 Callaly T, Hallebone EL (2001). "Introducing the routine use of outcomes measurement to mental health services". Australian Health Review. 24 (1): 43–50. doi: 10.1071/AH010043 . PMID   11357741.
  9. 1 2 Ohio Mental Health Datamart
  10. Iezzoni LI (15 June 1996). "100 apples divided by 15 red herrings: a cautionary tale from the mid-19th century on comparing hospital mortality rates". Annals of Internal Medicine. 124 (12): 1079–85. doi:10.7326/0003-4819-124-12-199606150-00009. PMID   8633823. S2CID   2934.
  11. Nightingale F. Notes on Hospitals. 3rded. London: Longman, Green, Longman, Roberts, and Green; 1863
  12. Codman EA. The Shoulder. Rupture of the supraspinatus tendon and other lesions in or about the subacromial bursa. Privately published 1934 Reprint 1965 Malabar, Florida: Kreiger;
  13. Kaska SC, Weinstein JN (March 1998). "Historical perspective. Ernest Amory Codman, 1869-1940. A pioneer of evidence-based medicine: the end result idea". Spine. 23 (5): 629–33. doi:10.1097/00007632-199803010-00019. PMID   9530796. S2CID   25646828.
  14. Codman EA. A study in hospital efficiency. As demonstrated by the case report of the first five years of a private hospital. Published privately 1817. Reprinted 1996 Joint Commission on Accreditation of Healthcare Organizations Oakbrook Terrace, IL, USA:
  15. 1 2 3 Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly 1966;44:166-206
  16. Oxman AD, Sackett DL, Chalmers I, Prescott TE (December 2005). "A surrealistic mega-analysis of redisorganization theories". Journal of the Royal Society of Medicine. 98 (12): 563–8. doi:10.1177/014107680509801223. PMC   1299350 . PMID   16319441.
  17. "Archived copy". Archived from the original on 2009-08-11. Retrieved 2009-08-11.{{cite web}}: CS1 maint: archived copy as title (link)
  18. "St George's Healthcare".
  19. Bridgewater B; Adult Cardiac Surgeons of North West England (March 2005). "Mortality data in adult cardiac surgery for named surgeons: retrospective examination of prospectively collected data on coronary artery surgery and aortic valve replacement". BMJ. 330 (7490): 506–10. doi:10.1136/bmj.330.7490.506. PMC   552809 . PMID   15746131.
  20. Murtagh, Fliss E. M.; McCrone, Paul; Higginson, Irene J.; Dzingina, Mendwas (2017-06-01). "Development of a Patient-Reported Palliative Care-Specific Health Classification System: The POS-E". The Patient: Patient-Centered Outcomes Research. 10 (3): 353–365. doi:10.1007/s40271-017-0224-1. ISSN   1178-1661. PMC   5422446 . PMID   28271387.
  21. Long, A; Jefferson, J (1999). "The significance of outcomes within European health sector reforms: towards the development of an outcomes culture". International Journal of Public Administration. 22 (3): 385–424. doi:10.1080/01900699908525389.
  22. NIMH Early Clinical Drug Evaluation PRB. Clinical global impressions. In: Guy W, editor. ECDEU Assessment manual for psychopharmacology, revised. US Department of Health and Human Services Public Health Service, Alcohol Drug Abuse and Mental Health Administration, NIMH Psychopharmacology Research Branch; 1976. p. 217-22
  23. De Lusignan S, Stephens PN, Adal N, Majeed A (2002). "Does Feedback Improve the Quality of Computerized Medical Records in Primary Care?". Journal of the American Medical Informatics Association. 9 (4): 395–401. doi:10.1197/jamia.M1023. PMC   346626 . PMID   12087120.
  24. Keogh, Bruce; Jones, Mark; Hooper, Tim; Au, John; Fabri, Brian M.; Grotte, Geir; Brooks, Nicholas; Grayson, Antony D.; Bridgewater, Ben (2007-06-01). "Has the publication of cardiac surgery outcome data been associated with changes in practice in northwest England: an analysis of 25 730 patients undergoing CABG surgery under 30 surgeons over eight years". Heart. 93 (6): 744–748. doi:10.1136/hrt.2006.106393. ISSN   1468-201X. PMC   1955202 . PMID   17237128.
  25. Stewart M (April 2009). "Service user and significant other versions of the Health of the Nation Outcome Scales". Australasian Psychiatry. 17 (2): 156–63. doi:10.1080/10398560802596116. PMID   19296275. S2CID   43644661.
  26. Stewart M. Making the HoNOS(CA) clinically useful: A strategy for making the HoNOS, HoNOSCA, and HoNOS65+ useful to the clinical team. 2nd Australasian Mental Health Outcomes Conference; 2008
  27. Nicholl, Jon; Brown, Celia A.; Lilford, Richard J. (2007-09-27). "Use of process measures to monitor the quality of clinical practice". BMJ. 335 (7621): 648–650. doi:10.1136/bmj.39317.641296.AD. ISSN   1468-5833. PMC   1995522 . PMID   17901516.
  28. Turner-Stokes, Lynne; Williams, Heather; Sephton, Keith; Rose, Hilary; Harris, Sarah; Thu, Aung (November 2012). "Engaging the hearts and minds of clinicians in outcome measurement – the UK rehabilitation outcomes collaborative approach". Disability and Rehabilitation. 34 (22): 1871–1879. doi:10.3109/09638288.2012.670033. ISSN   0963-8288. PMC   3477889 . PMID   22506959.
  29. Bilsker D, Goldner EM (November 2002). "Routine outcome measurement by mental health-care providers: is it worth doing?". Lancet. 360 (9346): 1689–90. doi:10.1016/S0140-6736(02)11610-2. PMID   12457807. S2CID   36926482.
  30. "Shared decision-making in medicine", Wikipedia, 2018-11-19, retrieved 2019-01-14
  31. www.kingsfund.org.uk(PDF) https://www.kingsfund.org.uk/sites/default/files/Muir-Gray.pdf . Retrieved 2019-01-14.{{cite web}}: Missing or empty |title= (help)
  32. Elwyn, Glyn; Frosch, Dominick; Thomson, Richard; Joseph-Williams, Natalie; Lloyd, Amy; Kinnersley, Paul; Cording, Emma; Tomson, Dave; Dodd, Carole (October 2012). "Shared Decision Making: A Model for Clinical Practice". Journal of General Internal Medicine. 27 (10): 1361–1367. doi:10.1007/s11606-012-2077-6. ISSN   0884-8734. PMC   3445676 . PMID   22618581.

See also