 |
 |
 |
|
| home | CME | help | feedback | subscriptions | archive |
|
© 2003 The Journal of Bone and Joint Surgery, Inc.
Economic Evaluationin Orthopaedics
Kevin J. Bozic, MD, MBA
• Rising health-care costs and increased constraints on health-care economic resources have led to growing interest in economic evaluation in health care. • Economic analysis provides a powerful tool for evaluation of health-care technologies and treatment strategies. • A working knowledge of health-care economic principles is essential for physicians to understand the benefits and limitations of economic analysis. • The lack of well-designed and methodologically sound economic studies in the orthopaedic literature is evidence that most orthopaedic surgeons are unfamiliar with basic health-care economic principles. • Orthopaedic surgeons should consider economic factors in addition to traditional outcomes measures when designing clinical trials and evaluating treatment options. Rising health-care costs have become an issue of paramount importance over the past two decades. Since 1965, the percentage of the United States gross domestic product spent on health care has increased from 5% to 13.4%, a figure that is expected to continue to rise to 15.9% by 2010 1 . Advances in medical technology have outpaced our ability to pay for them, raising questions regarding the true value of these technologies and their ability to deliver genuine health gains to the general population2. Currently, efficacy (the extent to which medical interventions achieve improvements in health under ideal circumstances 3 ) and effectiveness (the extent to which medical interventions achieve improvements in health in real practice settings 3 ) are the most common measures by which we evaluate health-care interventions. However, clinical decision algorithms and resource allocation decisions that are based solely on these factors are inadequate for determining which interventions are likely to yield the most benefit for a given amount of resources 4 . While efficacy and effectiveness are important parameters, they are no longer the only outcomes that clinicians consider when evaluating new health-care treatments and technologies. Economic evaluations such as cost-effectiveness and cost-utility analyses have been developed to complement clinical outcomes measures. They provide standardized, quantitative estimates of the likely cost per unit of health benefit achieved by a given treatment intervention 5 . This type of research has begun to play an increasingly important role in the evaluation of health-related technologies and treatment programs. Many clinical investigators have begun to recognize the importance of including economic analysis when designing prospective, randomized clinical trials to evaluate the success of a particular treatment. Furthermore, clinicians have become more aware of the economic implications of their practice patterns, and increased emphasis has been placed on fiscal responsibility in medical decision-making. The field of orthopaedic surgery has experienced tremendous growth and innovation over the past two decades. Orthopaedic science has provided us with many exciting new therapies, such as tissue engineering and gene therapy, with the potential to alter the clinical course of many orthopaedic conditions. However, in order to properly evaluate the true value of these technologies, clinicians must familiarize themselves with the basic tools of economic analysis. The purpose of this paper is to review the basic principles of health-care economics, with the goal of helping orthopaedic surgeons interpret the growing body of literature in this field.
Outcomes assessment is a powerful research tool that was designed to move beyond the traditional measures that are used to evaluate surgical results (e.g., hip and knee scores) "to help patients, payers, and providers make rational medical care-related choices based on better insight into the effect of these choices on the patient's life." 6 One of the primary goals of outcomes assessment is to provide patients and clinicians with access to relevant analyses that will assist them in making important health-related decisions 6 . Two other goals of outcomes assessment that are relevant to economic analysis are (1) to demonstrate the true value of a particular treatment or technology, and (2) to guide future efforts to continuously improve the quality of care 7 . A detailed description of the methods used to measure health outcomes is beyond the scope of this review. Interested readers are referred to the many comprehensive textbooks and articles that are devoted to this subject 8-11 . However, in order to interpret the literature on health-care economics, clinicians must familiarize themselves with the basic techniques that are used to measure health outcomes.
Health-Related Quality of Life
Quality-Adjusted Life Years The quality-adjusted life year (QALY) is one of the most commonly used measures of health utility in the medical literature. The quality-adjusted life year combines a quantitative measure (i.e., life years gained) with a qualitative measure (i.e., quality of life) to provide a single preference, or value-weighted number, that summarizes a patient's health-related quality of life 11 . By convention, the quality-adjusted life year is a measure of health outcome that assigns to each period of time a weight, ranging from 0 to 1, corresponding to the quality of life during that period. A weight of 1 corresponds to perfect health, and a weight of 0 corresponds to a health state judged to be equivalent to death 2 . Techniques used to measure the quality of life of a patient or a population are described below. The number of quality-adjusted life years, then, represents the number of healthy years of life that are valued equivalently to the actual health outcome. The disability-adjusted life year (DALY) is another measure of health utility, which was developed to assess the global burden of disease. Disability-adjusted life years are calculated by adjusting age-specific life expectancy for loss of healthy life due to disability. The value of a year of life at each age is weighted, as are decrements to health due to disability from specified diseases and injuries 2 .
Methods Used to Evaluate Health States Another method is the standard gamble technique 8,13 . With this technique, the patient is offered two treatment alternatives. Alternative one is a treatment with two possible outcomes: either the patient is returned to normal health and lives for a finite number of years (probability [p]) or the patient dies immediately. Alternative two is the certain outcome of chronic illness or disability (i) for a finite period of time. The probability (p) is varied until the subject is indifferent about which of the two alternatives is chosen, at which point the value for i is equal to that of p. One disadvantage of this method is that research has shown that people will almost always choose to remain in an undesirable, inferior health state than accept any substantial risk of death 14 . As a result, the utilities derived with this method are numerically higher than those derived with other techniques. An alternative method with which to determine health states is the rating scale technique 8 . With this technique, patients are asked to simply locate various health states on a linear scale from 0 to 1. One drawback is the tendency for most patients to rate their health as "average" compared with the health of their peers 15 .
Rosser Index Matrix
In some studies, researchers have used various health measurement questionnaires to directly measure disability and distress scores for calculation of Rosser Index Matrix scores 18-21 . Other investigators have indirectly derived disability and distress scores from Harris hip scores or other commonly used orthopaedic outcomes measures to generate Rosser Index Matrix scores 22-23 .
The primary goal of health-care economic analysis is to identify interventions that produce the greatest health benefit with the resources available 4 . Most health-care economic evaluations are used to compare health outcomes with the relative costs of the resources used to achieve those outcomes 24 . The breadth of outcomes considered varies according to the type of economic analysis performed 25 . Furthermore, the costs and benefits considered vary depending on the viewpoint adopted in the analysis. Thus, economic analyses vary in scope, perspective, applicability, and complexity. The four basic types of economic analysis that we will review are cost-minimization, cost-effectiveness, cost-utility, and cost-benefit analysis. Each of these approaches involves the systematic identification and valuation of the relevant costs and consequences of health-care interventions 26 . When combined with clinical trials that evaluate effectiveness and efficacy (defined above), they provide a decision-making framework that can be useful to both clinicians and health-care administrators. Economic analyses provide useful estimates of cost and health outcomes that assist clinicians in making ethically sound decisions regarding which interventions provide the maximum health benefit for a given cost. Once cost-effectiveness ratios are calculated from the societal perspective and placed in rank order, administrative decision-makers can use this information to fund the intervention with the lowest cost per quality-adjusted life year and continue to allocate scarce resources until the available funds are exhausted 2 .
Cost-Minimization or Cost-Identification Analysis While cost-minimization analyses may provide useful information by identifying all of the costs associated with a particular treatment, they can be used to compare treatments only when there is strong clinical evidence that patient outcomes are the same or similar 26 . If they are not, focusing solely on costs can lead to misleading results by ignoring important differences in outcomes.
Cost-Effectiveness Analysis An example of an appropriate cost-effectiveness study would be an evaluation of the cost per successful union in the treatment of Grade-III open tibial fractures. A study of two treatment options, intramedullary nailing without reaming and external fixation, might reveal that the former costs $25,000 per patient whereas the latter costs $20,000 per patient. At first glance, it appears that intramedullary nailing is a more expensive option for the treatment of Grade-III open tibial fractures. However, Shannon et al. recently reported that intramedullary nailing yielded a much lower rate of nonunion (15%) than did external fixation (42%) 29 . Thus, as shown in Table III , even though intramedullary nailing is more costly, it is more cost-effective for the treatment of Grade-III open tibial fractures because of the lower cost per successful union.
Cost-effectiveness studies are considered to be a more objective method of analyzing health-care costs because subjective factors such as patient preferences and the value of a particular treatment or health state to a patient are not considered. One advantage of this technique is that, with a common unit of outcome or effectiveness, different procedures can be compared and can be expressed in terms of cost per unit of outcome. However, cost-effectiveness analyses are not helpful for choosing between treatments that have different outcomes or for which the outcomes were measured with different techniques 24 . Therefore, this method cannot be used to combine reductions in morbidity and reductions in mortality into a single index to allow direct comparison between treatments that vary with regard to these two dimensions 26 .
Cost-Effectiveness Ratio The numerator of a cost-effectiveness ratio should reflect the difference in resource use between an intervention and the intervention with which it is being compared 31 . This can be referred to as the marginal or incremental cost of the intervention, as will be discussed below. By convention, the denominator of a cost-effectiveness ratio represents the change (or improvement) in health associated with an intervention. For example, in a study evaluating the use of thromboembolic prophylaxis after total joint replacement, the cost-effectiveness ratio would be reported as the cost of treatment per life saved (by preventing fatal pulmonary embolism).
Cost-Utility Analysis One obvious limitation of this technique is the difficulty of measuring a person's quality of life. Outcome measures such as Harris hip scores, Hospital for Special Surgery knee scores, and EuroQol scores 36 are commonly reported in the orthopaedic literature. Although these instruments are of considerable value for assessing functional outcome following a particular treatment (such as total hip or knee arthroplasty), they do not usually generate a single quality-of-life score. As such, they are not considered measures of health utility per se, and they cannot be used to compare outcomes between different treatments. In order to make such comparisons across treatment programs valid, results must be converted to a single generalizable measure of health utility, such as quality-adjusted life years 32 . The Rosser Index Matrix (described above) is one technique that can be used to make this conversion.
Cost-Benefit Analysis Although many health economists consider cost-benefit analysis to be the most sophisticated and theoretically sound form of economic evaluation, it is often difficult to accurately and reproducibly assign monetary values to health outcomes. One technique for assigning such values is to use wage rates as a so-called shadow price for the production gains resulting from people returning to work after successful treatment. An alternative method is to determine a patient's willingness to pay for a particular treatment outcome 26 . Clearly, any method that is chosen to measure the monetary value of a treatment outcome will provide only a rough estimate of the true benefit and will fail to capture the full value of the benefits gained from a particular health-care intervention.
Identifying and evaluating costs are important steps in any economic evaluation. Which costs are included in the analysis will vary on the basis of the time frame and perspective being considered in the study. Ideally, a thorough economic analysis measures direct, indirect, and intangible costs 5 . Direct medical costs include all costs that are directly relate to the intervention, including those for personnel, supplies, and the facility involved in the treatment. Most health-care economic studies in the orthopaedic literature have focused on only direct medical costs 37-41 . Direct nonmedical costs include costs borne by patients and their families in the course of treatment (e.g., for transportation or lodging). Indirect costs include costs associated with lost productivity, usually valued as lost wages or an imputed monetary value of time 5 . (Note that this should not be confused with the more traditional accounting definition of indirect costs, which refers to fixed overhead costs.) For determination of intangible costs, an attempt is made to assign a dollar value to reductions in quality of life associated with pain, suffering, and grief. Those costs are often included in the measurement of quality-adjusted life years 42 . It is important to consider not only the so-called upfront costs associated with an intervention, but also the downstream costs of resources that will be consumed in the future but are still attributable to the intervention 24 . For example, an economic analysis of total hip replacement should include projected costs associated with subsequent readmission for dislocation, infection, and revision surgery, as calculated on the basis of complication and revision rates reported in the literature.
Health-care costs can be very difficult to measure accurately. Therefore, it is essential that the authors of economic studies explicitly identify which costs are being counted and which were excluded from the study 43 . Both direct and indirect costs incurred by patients can be estimated by asking patients to keep track of their expenses in a cost diary. Many hospitals have implemented resource-based accounting systems to more accurately track direct and indirect costs by patient, procedure, department, or physician. Costs can be further categorized as fixed, variable, marginal, average, and total. Fixed costs are those incurred regardless of the level of productivity; they cannot be altered over the short term. For example, the mortgage or lease on an outpatient surgical center and the salaries of most operating-room personnel would not change if there was a 10% increase or decrease in surgical volume. Sunk costs are a subset of fixed costs that cannot be altered at all, even over the long term. For example, the costs involved in building an additional operating suite for an outpatient surgical center would be considered a sunk cost. If capacity dropped and the operating suite was no longer used, the costs that were already incurred could not be recovered and the operating room probably could not be used for another purpose. In general, sunk costs should not be considered in economic analyses because they cannot be changed or altered. In contrast, variable costs change in proportion to changes in the level of production. For example, the costs of most surgical supplies and of certain personnel who are paid on an hourly basis would be expected to vary in proportion to the volume of operations performed. Total costs are the sum of fixed and variable costs. Total costs should be considered in most economic evaluations, but this could vary depending on the perspective and/or purpose of the study. In strict economic terms, average total cost is defined as the total cost of production divided by the quantity of output, and marginal cost is defined as the increase in total cost that arises from one extra unit of production 44 . Average total costs can be further divided into average fixed costs (fixed costs divided by the quantity of output) and average variable costs (variable costs divided by the quantity of output) 44 . The difference between average total cost and marginal cost can be expressed mathematically as: average total cost = total cost/quantity, and marginal cost = (change in total cost)/(change in quantity) 44 .
In health-care economic studies, average costs are the total cost of a treatment program divided by the total number of patients treated, whereas marginal costs are the costs of treating one additional or one fewer patient 45 . It is important to note that analyses of marginal costs and those of average costs often provide markedly different results, and therefore the context of the study determines which measure is more appropriate for the analysis 43 . Average cost data are generally easier to obtain than marginal cost data, and therefore average costs are often used as a proxy for marginal costs in health-care economic evaluations. However, as Finkler and Schwartzben 46 pointed out, the use of average costs can lead to misleading conclusions regarding cost savings. In a study of protocol systems in pediatric emergency care, they demonstrated that while large reductions in usage of ancillary services resulted in similar reductions in average costs, they led to only small reductions in marginal costs, which are the more relevant measure of cost savings. The reason for this finding is that personnel and equipment costs are relatively fixed and insensitive to changes in volume unless those changes are large enough to justify reductions in staff or equipment. The type of analysis being performed dictates whether marginal or average cost measurement is more appropriate. For instance, for decisions regarding whether to expand or contract the use of a particular intervention or whether to switch from one treatment strategy to a new strategy, analysis of marginal costs (the incremental benefit obtained from an increment in cost) rather than average cost and benefit ratios should be used 47,48 . Alternatively, for decisions regarding whether to introduce a new program or technology de novo for a certain patient population, such as a preoperative education program for patients scheduled for total joint replacement and their families, it is more appropriate to assess the average cost per patient treated. In either case, the incremental costs (the difference in marginal or average costs compared with the base case) should be used when comparing treatment alternatives 47 . For instance, an evaluation of the cost-effectiveness of kyphoplasty to prevent the progressive collapse of vertebral insufficiency fractures should involve measurement of the incremental costs of kyphoplasty compared with that of a physical therapy and rehabilitation program with training in posture and body mechanics. Figure 2 provides an illustrative example of the difference between average fixed, average variable, average total, and marginal costs. In this example, a hospital administrator is interested in evaluating the costs associated with the purchase and use of a magnetic resonance imaging scanner. The initial cost to purchase the machine, $3,000,000, would be considered the fixed cost of the investment and would be the same no matter how many patients were treated. The variable costs of the project would include the costs for hourly workers (i.e., technicians), film, supplies, and so on. The variable costs would be expected to change in proportion to the volume of patients treated. Furthermore, variable costs often increase slightly as volume increases, as a result of coordination costs and other costs that are incurred in order to deliver a higher volume of service ( Fig. 2 ). Four notable features of these cost curves should be recognized: (1) average fixed costs decrease in proportion to the total number of procedures performed, (2) marginal and variable costs usually rise with the number of procedures performed, (3) the average total cost curve is u-shaped, and (4) the marginal cost curve crosses the average total cost curve at the point of minimum average total cost.
Opportunity Costs By measuring outcomes and costs associated with a particular health-care intervention, economic analyses show the trade-offs involved in choosing among interventions. In doing so, they help to define the opportunity cost of a treatment decision; i.e., the health benefits lost because the next-best alternative was not selected 2 . For instance, because of limited funds, a hospital might have to choose between building either an outpatient imaging facility or an on-site physical therapy clinic. An economic analysis could be performed to compare the financial impact of building the outpatient imaging facility compared with that of building the on-site physical therapy clinic. The true opportunity cost of building the imaging facility would be any profit or health benefit lost by not having the physical therapy clinic. Ideally, economic analyses should determine the value of all inputs in terms of their opportunity costs 27 . Opportunity costs represent the value that the resource could have produced if it had been spent for the next best available alternative 31 . However, direct measurement of opportunity costs is often difficult or even impossible. Therefore, for practical purposes, the market prices of health-care inputs are used in place of opportunity costs in most health-care economic evaluations.
Economic data can be analyzed from different perspectives, including that of the patient, the hospital, the payer, the physician, the surgical device manufacturer, or society as a whole. Investigators should report and justify the perspective they choose for their analysis so the reader can better interpret the results and evaluate the appropriateness of other methodological decisions made by the investigators 25 . The perspective chosen will also reflect the type of decisions that the analysis is intended to support and will determine which costs and health outcomes are included in the cost-effectiveness ratio and how those costs and outcomes are valued 4 . Economic analyses omit certain health outcomes and costs if they are not of interest to the decision-maker 2 . For example, a study of costs associated with work-related injuries undertaken by a health maintenance organization might not include data regarding when patients return to work following an injury. However, if the same study were undertaken from a governmental or societal perspective, the time when the patients returned to work would be an important parameter to consider. Results may vary considerably according to the perspective that is chosen 47 . For instance, early discharge from the hospital following lumbar laminectomy may be considered cost-effective from the hospital's perspective, but it may in fact prove to be more costly to a third-party payer and to society as a whole if the costs of home care and complications resulting from early discharge outweigh any cost-savings from reductions in the length of hospital stay. The fact that the two studies led to different conclusions regarding the cost-effectiveness of an early discharge program also illustrates why studies based on different perspectives are not comparable. In general, most comprehensive economic evaluations that are undertaken for the purpose of allocating limited health-care resources should be carried out from the broadest perspective, that of society as a whole 49 . However, useful information can also be obtained from studies with a narrower viewpoint, such as that of a physician group or a hospital.
Data on clinical outcomes can be obtained from a number of different sources, including randomized, controlled trials, observational studies, uncontrolled experiments, or descriptive series 31 . In most economic evaluations, effectiveness data (defined above) are preferred over efficacy data, since they are less likely to overestimate the treatment effect in actual practice 24 . Ideally, economic evaluations should be performed prospectively during a clinical trial so that relevant data on costs and effectiveness can be collected simultaneously 42,50 . As pointed out by Laupacis et al.: "The highest-quality estimates of costs include those derived from direct measurement of resources used by the competing strategies in a sample of the population that used those resources." 48 Briggs emphasized the importance of including economic variables in the original study design, rather than adding them to an existing clinical trial 51 , because cost variables generally have a higher variance than clinical outcomes. Therefore, economic evaluations require greater sample sizes than do corresponding clinical comparisons. When an economic analysis is added to an existing clinical trial, the results of both the cost analysis and the cost-effectiveness analysis will generally be underpowered, thereby calling into question the validity and usefulness of the results 51 .
Cost-to-Charge Ratios Some hospitals have recently begun using sophisticated knowledge-based software programs in an attempt to track clinical, financial, and patient satisfaction variables (Eclipsys, Boca Raton, Florida). These systems attribute direct and indirect costs to individual patients and procedures on the basis of the resources that were used. However, investigators must pay attention to how costs are allocated by these software algorithms, as many hospital cost accounting systems use average rather than marginal cost data 30 . As described above, use of average costs rather than marginal costs in a comparison of treatment options can lead to inappropriate conclusions. Although it is ideal to prospectively collect cost and effectiveness data at the same time 42 , this can be both time-consuming and expensive. Therefore, most economic analyses rely on the existing literature for effectiveness data. Researchers from the Panel on Cost-Effectiveness in Health and Medicine have recommended that investigators select outcome and effectiveness data from the best designed and least biased sources that are relevant to both the question and the population under study 31 . However, there are often trade-offs between the internal validity of data from such sources as randomized trials and their external validity (or effectiveness) in actual practice 31 . In other words, economic evaluations are more valid if effectiveness data reflect normal clinical practice as closely as possible 25 . If no data are available from any of these sources, economic studies may have to rely on assumptions about clinical evidence 50 . Since results can vary widely on the basis of these assumptions, sensitivity analysis (described below) should be carried out to test the impact of a range of different assumptions on the results and conclusions of the study.
The time horizon of a health-care economic evaluation is the period of time for which the costs and outcomes are measured 2 . The time horizon should be specified and justified as being appropriate for the clinical condition being studied and the expected costs and benefits of the treatment program 49 . For instance, in a study comparing the costs associated with open and endoscopic carpal tunnel release, a relatively short time horizon might be appropriate to capture most of the relevant costs and benefits associated with the procedures. In contrast, a study to evaluate the cost-effectiveness of lumbar spinal fusion in patients receiving Workers' Compensation would require a longer time horizon, to account for the potential need for future interventions and their associated direct (medical) costs and indirect costs (lost wages and so on). Depending on the time horizon that is chosen, costs and benefits should be discounted appropriately, as will be discussed below. Other time-sensitive issues that should be considered include improvements in technology and overall societal well-being that occur over time as well as the so-called learning-curve effect that follows the introduction of a new technology 55 . Organizational behavior research has shown that there is a settling-in period following the introduction of a new technology, during which time costs are higher and results are more variablev. For instance, the cost of arthroscopic shoulder stabilization may be lower in both absolute and real terms in 2002 than they were in 1997. However, advances in technology and surgeon familiarity with these techniques must be considered when costs and benefits are compared between these two time-frames.
Discounting refers to valuing future costs and benefits in present-day monetary units 56 . In health care, benefits and costs often accrue over different periods of time and for different durations of time 30 . Therefore, when economic evaluations are performed, both the costs and the benefits associated with a specific intervention should be discounted back to a common point in time, in order to facilitate direct comparison between treatment regimens. Discounting is a critical step in any economic evaluation. For example, most of the costs of bracing programs for idiopathic scoliosis are incurred at the time that they are implemented. However, their benefits, in terms of preventing operations and pulmonary complications, are usually realized much later in life. Additional costs may also be averted in future years. In order to make a valid comparison between the costs and benefits of such a program, current and future costs and future benefits must be adjusted back to their present values 30 . The practice of discounting health-care costs to accurately reflect the so-called time value of money has long been accepted by health-care economists. In modern economies, people pay interest when they borrow money and receive interest payments when they lend or save money. Thus, a dollar paid in the future is worth less than a dollar paid today. Consequently, in an analysis of health-care interventions, the costs or savings of which are distributed over many years, the practice of discounting is essential 2 . The concept of discounting health benefits, however, is more controversial, and some clinicians may be uncomfortable with it. Essentially, discounting health benefits implies that future health benefits are less valuable than present health benefits. However, if the discount rate chosen for health benefits is lower than that for costs, every health-care intervention could be made to appear more economically attractive simply by delaying its implementation 5 . Therefore, most health-care economists recommend applying the same discount rate (usually 5%) to both the costs and the benefits of health care. Health-care economists have long debated about the appropriate discount rate for health-care interventions 57 . The conventional practice in health-care economic studies is to discount both costs and benefits at a rate of 5% 25 . However, the discount rate should be specified and justified, with the perspective of the analysis, time preference, and opportunity costs taken into consideration 57 .
Sensitivity Analysis Readers and users of health-care economic analyses need information on the degree to which the conclusions of a study might change with changes in the assumptions or input variables. Sensitivity analysis is an important tool that can be used to accomplish this goal 31 . Sensitivity analysis involves changing one or more of the variables under consideration and examining the effect of the change or changes on the results of the analysis 56 . This technique is widely used in economic evaluations to assess how the strength and validity of the conclusions of a study vary with changes in the underlying assumptions and estimates 30 . Sensitivity analysis should be applied to the range of potential values included in the confidence intervals for critical variables that could affect the conclusions of the study 25 . By performing sensitivity analyses, investigators can assess whether a result or conclusion is valid and remains stable over a range of plausible assumptions or whether it hinges on the accuracy of a particular assumption 30 . For instance, investigators comparing the cost utility of limb salvage with that of early amputation for Grade III-B tibial fractures should consider a wide range of values for failure of limb salvage in order to determine the sensitivity of the results to this parameter. Another way to apply this technique is to determine the break-even point—that is, the point when the results of one option are equivalent to those of another. For example, Daellenbach et al. created a sophisticated economic model and performed extensive sensitivity analyses to calculate the gains in longevity that would have to be achieved by performing total hip arthroplasties without cement to justify their additional cost compared with that of performing total hip arthroplasties with cement 35 . As shown in Table IV , the number of years by which cementless prostheses would have to outlast cemented prostheses in order to justify their additional cost ranges from four to seventeen, depending on the age of the patient, the assumed life of the cemented prosthesis, and the predicted cost of revision surgery.
Despite the importance of considering uncertainty when performing economic analyses, surprisingly few health-care economic studies provide an adequate sensitivity analysis. In a review of all health-care economic evaluations published in 1992, Briggs and Sculpher found that only 39% (thirty-six) of ninety-two studies provided an adequate sensitivity analysis to deal with issues of uncertainty 58 .
The goal of an economic evaluation is often to compare two treatment options in order to select the one that provides the maximum health benefit for a given increment of cost 59 . O'Brien et al. described a simple 3 x 3 matrix to categorize studies on the basis of whether the new treatment costs more, the same, or less than the control and whether it has more, the same, or less effectiveness or benefits ( Fig. 3 ) 60 . In category 1, the new treatment is both less costly and more effective than the control, so the new treatment is said to be dominant over the control. For example, Patterson and Scott reported that intramedullary nailing of the femur is dominant over simple traction for the treatment of most femoral shaft fractures, since it is less costly and results in a higher rate of union and a lower complication rate 61 . Category 2 represents strong dominance to reject the new therapy because costs are higher and effectiveness is lower than that of the control. An example of this can be found in a French study, published in 1983, in which meniscectomy by arthrotomy (open meniscectomy) was demonstrated to be more expensive, less effective, and associated with a higher complication rate compared with arthroscopic meniscectomy 62 . Categories 3, 4, 5, and 6 represent cases of weak dominance; only one dimension (either cost or effectiveness) of the treatment is dominant, and the other measure is equivalent to that of the control 60 .
Whenever a result falls into a category of dominance, a clear-cut cost-effectiveness recommendation can be made on the basis of the evaluation. However, if the result falls into a nondominance cell (7, 8, or 9), additional analysis is needed before a formal treatment recommendation can be made. The most common nondominance circumstance is category 7, in which the new therapy offers additional effectiveness but at an increased cost 60 . Schnitzer's comparison of cyclooxygenase 2-specific inhibitors with traditional nonsteroidal anti-inflammatory drugs for patients with osteoarthritis and a history of gastritis would fall into this category 63 . In such cases, it would be useful to calculate the incremental cost-effectiveness ratios of the new therapy, as discussed above. In addition to understanding the results of an economic analysis, clinicians should assess whether the conclusions of the study are applicable to their own practice setting. This involves evaluating the demographics of the patient population being studied, the practice patterns used in the study (and how they might differ from their own), the level of resource consumption used in the analysis, and the relative costs of health-care inputs 48,60 . If any or all of these variables differ substantially from the physician's own practice, the conclusions of the study may not be applicable to his or her practice setting.
Given that many ethical and value judgments may underlie health-care economic analyses, results of these studies are the subject of much controversy and debate. As noted above, the reliability of the sources of both the cost and the health outcomes data is a major determinant of the quality of an economic study. In a review of problems associated with modeling in economic evaluation, Sheldon pointed out that "the use of ever more sophisticated models to conduct [health-care economic] evaluations without improving the quality of the information inputs will not increase the reliability of the results." 64 Precise cost and effectiveness data are often difficult to obtain, and as a result many assumptions are made or implied on the basis of the existing literature. These assumptions can often have a profound impact on the conclusions of the study, and therefore they should be tested by sensitivity analysis. Many researchers and clinicians have been critical of the use of health-care utility measures such as quality-adjusted life years as a guideline for resource allocation or clinical decision-making 8,10 . Quality of life, by definition, is a subjective measure that can vary widely among patients, populations, and practitioners. Several authors 8,12 have noted that measurements of quality-adjusted life years have flaws related to variability in individual appreciation of life values. La Puma and Lawlor noted that measuring quality-adjusted life years requires researchers to make many ethical assumptions regarding what defines quality of life, equity and efficiency, and individual and societal preferences 10 . Furthermore, decisions based on quality-adjusted life years (by convention) direct care away from older patients or patients who have a poor quality of life because their care would yield few quality-adjusted life years at a relatively high cost. Finally, many clinicians have no frame of reference for interpreting the meaning of quantitative values for quality-adjusted life years. Some investigators have suggested that using threshold values for small, medium, and large gains in health-related quality of life due to treatment would be more informative and useful than reporting absolute values for quality-adjusted life years 11 . Use of cost-effectiveness league tables, which compare and rank health-care interventions on the basis of their cost-effectiveness or cost-utility, has been suggested as a method of allocating health-care resources in the United Kingdom and other places where health-care rationing is common 65 ( Table II ). A basic assumption of this technique is that health-care programs should be funded in the order in which they appear on a cost-effectiveness league table. However, Mason et al. pointed out that caution should be used when interpreting the results of such tables 65 . Given that most of the data from these types of tables are acquired from different sources, the ranking of interventions does not take into account the quality of the data or the methodology used in each study. Furthermore, depending on the time of origin and geographic location of each report, there may be changes in technology, price, clinical practice, and access to health-care resources that make direct comparisons inappropriate or even impossible. Investigators should refrain from mak |
Top
Introduction
Outcomes Assessment
