 |
 |
 |
|
| home | CME | help | feedback | subscriptions | archive |
|
© 1999 The Journal of Bone and Joint Surgery, Inc. Short Musculoskeletal Function Assessment Questionnaire: Validity, Reliability, and Responsiveness*  , SEATTLE, WASHINGTONInvestigation performed at Harborview Medical Center, Seattle
Background: A short questionnaire on functional status was designed for use in community-based outcome studies and in the management of individual patients who have musculoskeletal disease. As most musculoskeletal care is delivered in community practices, short, validated instruments are necessary to perform clinical studies on the effectiveness of treatment in this setting. Methods: A forty-six-item questionnaire was created as an extension of the work to develop the longer, 101-item Musculoskeletal Function Assessment (MFA) questionnaire. The Short Musculoskeletal Function Assessment (SMFA) questionnaire consists of the dysfunction index, which has thirty-four items for the assessment of patient function, and the bother index, which has twelve items for the assessment of how much patients are bothered by functional problems. The SMFA questionnaire was evaluated for reliability, validity, and responsiveness in a population of 420 patients who had a musculoskeletal disease or injury. Results: The SMFA questionnaire demonstrated excellent internal consistency and stability, with most values greater than 0.90. Content validity for the dysfunction and bother indexes was supported with very little skew (less than 1.00), few ceiling effects (less than 5 percent), and no floor effects. Convergent validity was supported with significant correlations between the SMFA dysfunction and bother indexes and the physicians' ratings of patient function (for example, activities of daily living, recreational and leisure activities, and emotional function [rho  0.40]) and standard clinical measures (for example, grip strength and walking speed [r 0.40]). Convergent and discriminant construct validity of the SMFA indexes were demonstrated (p < 0.01) in comparisons with clinical, demographic, Short Form-36 (SF-36), and life-change data. The responsiveness of the SMFA questionnaire to change over time was demonstrated with standardized response means ranging from moderate (0.76) to large (-1.14) for patients who had changes in health status.
Conclusions: The SMFA questionnaire may be used for clinical assessments of the impact of treatment in groups of patients who have musculoskeletal disease or injury. It also may be used in clinical settings to provide reliable and valid assessments of the health status of an individual patient.
The present study describes the development of the Short Musculoskeletal Function Assessment (SMFA), a two-part, forty-six-item, self-reported health-status questionnaire, which can be completed in about ten minutes. The SMFA is designed to detect differences in the functional status of patients who have a broad range of musculoskeletal disorders that are commonly seen in community practices. It also allows patients to evaluate how bothered they are by functional problems. The purpose of a functional assessment tool is to provide a standardized measure of the actual physical limitations of the patient. This tool then can be used to compare the patient with herself or himself over time as well as with other patients who have similar musculoskeletal disorders. Patient function also can be compared with function in the general population. In a well planned clinical experiment, functional assessment tools can be used to evaluate the effectiveness of treatments as well as health-care policies. When there is one general assessment tool that is applicable to all forms of musculoskeletal disease, the administrative burden of making the assessment is greatly reduced. Office staff need to learn only one set of procedures, not multiple sets according to the type of disease or injury. The SMFA questionnaire is based on the longer, 101-item Musculoskeletal Function Assessment (MFA) questionnaire, which also was developed by our team. The MFA questionnaire has been shown to be reliable, valid, and responsive12,44. It has demonstrated validity and responsiveness that are equal to or better than those of other disease-specific and generic health-status instruments45. Because of the length and detail of the MFA questionnaire, it has been most useful in research rather than in clinical settings7. Thus, we developed the SMFA questionnaire as a shorter alternative to the MFA questionnaire. In the present study, we describe the reliability, validity, and responsiveness of the SMFA questionnaire in a sample of patients who had musculoskeletal injuries and disorders.
Short Musculoskeletal Function Assessment Questionnaire
The scores on the dysfunction index and the bother index are calculated by summing the responses to the items and then transforming the scores so that they range from zero to 100, with use of the formula70: ([actual raw score - lowest possible raw score]/possible range of raw score) x 100. Higher scores indicate poorer function. For the dysfunction index, unanswered items within a category are replaced with the individual's mean score for that category, as long as more than 50 percent of the items in that category have been answered70. Substitution with the mean is not appropriate for the bother index as each item addresses a unique functional area. The forty-six items selected for the SMFA questionnaire were identified from analyses of baseline and follow-up data on 327 patients who had responded to the MFA questionnaire12,41. Items that were retained met three criteria: (1) they were determined by the research team to be clinically and conceptually important28,69; (2) they were stable, with percent agreement statistics from test-retest data that were greater than 0.80 and kappa values that were greater than 0.7028,65; and (3) they were moderately endorsed (patients responded with the item that was true for them) at baseline and follow-up evaluations, with exclusion of items that had very low endorsement (10 percent or less, with some exceptions to provide items for seriously disabled patients), very high endorsement (80 percent or more), or missing endorsements (5 percent or more)25. The retained items represent what we consider to be the shortest form that is feasible while maintaining statistical integrity. Another change from the MFA was the replacement of that questionnaire's dichotomous yes-or-no response format with a 5-point Likert-type response format. We believed this to be necessary because the SMFA has fewer items but was designed to be evaluative as well as discriminative. Thus, we thought it necessary to increase the variability in the scores by providing more response choices17,20,28. We also added two new composite questions that combine the MFA questions concerning housework. Finally, the categories in the SMFA questionnaire were assessed with use of principal component factor analytic procedures with baseline data. A pilot evaluation was carried out to assess the SMFA for readability, clarity, ease of use, and appropriateness with a sample of twenty-six patients from a university medical center and a community orthopaedic practice.
Characteristics of the Patients Patients were selected from eighteen sites across the United States. Four sites were university-based clinics, seven were private orthopaedic clinics not located in the state of Washington, and seven were private orthopaedic clinics located in the state of Washington.
Study Protocol The SMFA was self-administered at two points in time. The time-frames were selected on the basis of previous research on patients who had an acute fracture or soft-tissue injury33,39-41,48,54, a repetitive-motion disorder53,58, osteoarthritis, or rheumatoid arthritis13,73. A questionnaire was mailed to the 211 patients who had an acute fracture or soft-tissue injury at three and six months after the injury. The 209 patients who had a repetitive-motion disorder, osteoarthritis, or rheumatoid arthritis completed the survey at the time of the enrollment visit and again three months later.
Analyses
Reliability Internal consistency, defined as the comparability of items within the SMFA questionnaire, was assessed with use of Cronbach's alpha with baseline and follow-up data to verify the makeup of the categories. We expected correlations and alpha values to be greater than 0.70, the standard for adequate reliability for questionnaire development and group comparisons24,57.
Validity Content validity is present when "the domain of the instrument is appropriate relative to its intended use."59 Questionnaires that demonstrate content validity should have few missing responses, use the full range of scores with little skew (the amount of asymmetry in the distribution of scores), and have few ceiling (best possible score) or floor (poorest possible score) effects. On the SMFA questionnaire, a ceiling score of 0 points is the best possible score. The questionnaire is unable to measure any additional improvement in function, even though patients may still improve. At the other end of the scale, a floor score of 100 points is the worst possible score that the SMFA can measure but, again, there is no limit to patients' actual deterioration in function. We examined content validity for the dysfunction and bother indexes across all clinical groups with use of data collected at the three-month mark for the group that had a traumatic injury and at the time of the first office visit for the other groups (baseline). We expected less than 5 percent of the responses to be missing, the score ranges to include most values (that is, from 0 to 100 points), the distributions of the scores to have skew values of less than 1.00 (symmetrical distribution of the data—skew values between -3.00 and +3.00 are considered reflective of a normal distribution), and less than 5 percent of the patients to have a ceiling score of 0 points (best function) or a floor score of 100 points (poorest function). Because of the functional characteristics associated with clinical groups, we expected more limited score ranges (50 to 60 percent of available scores) and more skewed distributions (a skew value of less than 3.00)65 when examining SMFA scores by clinical groups. Because of the severity of illness in the patients who had rheumatoid arthritis, we expected higher mean scores and more limited score ranges for this clinical group. Construct validity is present when there is a relationship between the questionnaire and various hypotheses. It is possible to demonstrate either convergent construct validity or discriminant construct validity. Construct validity is also demonstrated when "the health measure relates to other measures in ways consistent with plausible hypotheses."65 These other measures may include clinical measures of patient function1,2,9,49,61,67,72, demographic data (for example, age, gender, education, race, marital status, income, health insurance, disability compensation, or legal action)4,27,29,31,37,39,50,52,60,63,64,66, or other health-status questionnaires70. Convergent construct validity must correlate the scales being measured in a positive fashion. Also, similar measures must correlate significantly with the new health-status questionnaire to demonstrate convergent construct validity36. Discriminant construct validity must correlate the scales being measured in a negative fashion. In addition, groups that are known to differ from each other on the basis of their group membership should differ significantly from each other with regard to the scores of the new health-status questionnaire to demonstrate discriminant construct validity37. The convergent construct validity of the SMFA was measured against four separate studies: ratings of patient function by an orthopaedist, clinical measures of patient function, the Short Form-36 (SF-36) subscales, and the demographic characteristics of the patients. For the first evaluation of validity, the orthopaedic surgeons subjectively assessed the functional status of the patients with use of a rating form developed for validation of the MFA12. Ratings were completed for 263 patients who had visits that corresponded with the baseline administration of the SMFA (Table I). The orthopaedists rated patient function with respect to the mobility of the upper and lower extremities, the ability to carry out the activities of daily living as well as recreational and leisure activities, and emotional coping and adjustment. Scores were assigned on the basis of an 11-point scale, with 0 points indicating the best function and 10 points, the poorest function.
For the second evaluation of validity, the physician or a member of the clinical staff carried out several clinical measurements during visits that corresponded with the administration of the baseline questionnaire. These measurements were of self-selected walking speed and grip strength or range of motion, or both, of the affected extremity. Ninety patients who had an injury or disorder of the lower extremity performed a test to determine their self-selected walking speeds at locations that had either a twenty-five or a fifty-meter walking course (Table I). In order to accommodate the variation in distances at the locations, speeds were standardized as meters per second. The walking speed was considered to be poor if it was more than one standard deviation slower than reported norms, adjusted for age and gender30,35,43,62. The grip strength of forty-nine patients who had an injury or disorder of the upper extremity was measured with use of the Jamar dynamometer (J. A. Preston, Jackson, Mississippi), according to the recommendations of the American Society of Hand Therapists47 (Table I). Grip strength that was more than one standard deviation less than normal, adjusted for age and gender, was rated as poor46,47. The range of motion of the affected extremity of patients who had a single injury or disorder was measured, with use of a goniometer, according to the recommendations of the American Academy of Orthopaedic Surgeons15 (Table I). Range-of-motion scores were reported as continuous variables and were also categorized as normal, functional, or poor with use of reported estimates defining normal and minimum functional ranges for the joint measured56. Values for flexion and extension were combined to create a single score. For the third evaluation of validity, all patients completed the SF-36 questionnaire. This generic, self-administered health-status questionnaire has thirty-six multiple-choice items grouped into eight subscales: physical functioning, role limitations due to physical health problems (physical role functioning), bodily pain, vitality, general health, social functioning, role limitations due to emotional problems (emotional role functioning), and mental health70. This test was administered as a way to assess convergent construct validity—that is, to use as a so-called yardstick with which to compare changes in the scores on the SMFA. In order to facilitate the validity analyses, SF-36 scales were reversed to correspond with SMFA scores (the higher the SF-36 scale value, the greater the dysfunction of the patient). For the fourth evaluation of validity, the patients answered questions about race, gender, age, education, marital status, health insurance, employment, income, disability compensation, legal action changes in life due to illness, and health status. These data demonstrated that the study group was a heterogeneous population of patients. It also allowed us to test validity against factors such as employment status and legal action, which are thought to have a potential impact on recovery for some individuals12.
Physicians' ratings of patient function, clinical measures of patient function, and SF-36 subscales were analyzed with use of correlations. Pearson's correlation (r) measures were used with the clinical measures and SF-36 subscales. Spearman's rho correlations, with a significance level of rs Discriminant construct validity was examined, with use of baseline data, for patients who differed by overall health status and by life changes because of illness. The perception of overall health status was assessed with the SF-36 item: "In general, would you say your health is excellent/very good/good/fair/poor?" We expected that patients who had different levels of health status would have significantly different scores on the SMFA indexes. We tested this hypothesis with analysis of variance (p < 0.01) and Scheffé's tests for pairwise comparisons (p < 0.05). Life changes due to illness were assessed with the item: "How much has your injury or arthritis changed your life—not at all/a little/somewhat/quite a bit/completely?" This item was used to evaluate, with use of receiver operating characteristic analysis5, the ability of the SMFA to discriminate between patients who had and those who did not have functional problems. (The principles and methods of receiver operating characteristic analysis have been previously described45.) With a receiver operating characteristic curve, the patients who were correctly identified by the question as having the disease were plotted on the y axis (sensitivity) and the patients who were incorrectly classified as having the disease were plotted on the x axis (1 - specificity). Patients were classified as not having a functional problem if they answered "not at all" or "a little" to the life-change question. Patients were classified as having a functional problem if they answered "quite a bit" or "completely" to the life-change question. We expected the specificity, sensitivity, and areas under the curve for the SMFA indexes at baseline to be greater than 0.70. A value of 1.00 for the area under the curve indicates that patients have been correctly classified, with the level of their dysfunction equal to their response to the life-change question.
Responsiveness Additionally, responsiveness was assessed with use of standardized response mean statistics. The standardized response mean is calculated as the mean change in score divided by the standard deviation of individuals' changes in score. On the basis of the recommendation by Guyatt et al.20 that the data from subjects with stable responses be used as the denominator of the standardized response mean, we used the standard deviation of the changes in the scores for the patients who reported that their health was "about the same." With use of the standards established by Cohen for small (more than 0.2), moderate (more than 0.5), and large (more than 0.8) effects8, we expected standardized response means to range from small to large for patients who reported that their health had changed.
Characteristics of the Patients
The ninety-four patients who did not complete a second survey were similar to the 420 respondents in terms of gender, number of comorbidities, number of disorders, procedures or treatments (for example, operative procedures, medications, physician visits, and physical therapy), severity or type of acute fracture or soft-tissue injury, severity of repetitive-motion disorder, and number of involved joints for patients who had osteoarthritis or rheumatoid arthritis. However, the nonrespondents were significantly (p < 0.05) younger than the respondents (44.70 ± 14.96 years [mean and standard deviation] compared with 48.93 ± 15.99 years), and they had fewer complications (0.01 ± 0.10 compared with 0.05 ± 0.22) and fewer bilateral injuries (1.25 ± 0.45 compared with 1.72 ± 0.45). In addition, the nonrespondents who had osteoarthritis or rheumatoid arthritis had less severe disease ratings than did the respondents who had those disorders (1.67 ± 0.69 compared with 2.66 ± 0.96)6. (The severity of the osteoarthritis or rheumatoid arthritis was determined on the basis of data in the medical record, specifically the type of medications prescribed, a history of operative procedures, and the number of involved joints.)
Reliability
Content Validity The content validity of the SMFA, examined according to clinical group, was supported, with some exceptions (Table III). All clinical groups, with the exception of patients who had rheumatoid arthritis, used 50 percent or more of the response scale. None of the distributions were skewed more than 3.065, and none of the clinical groups received a floor SMFA score (that is, 100, the poorest score for function). Ceiling effects, with more than 5 percent of the patients receiving a score of 0 points (no dysfunction), occurred only on the bother index for the patients who had an acute fracture or soft-tissue injury about the spine, 6.3 percent of whom had a ceiling score. As expected, patients who had rheumatoid arthritis had the highest mean score and the smallest range of scores on the SMFA indexes.
Convergent Construct Validity
Discriminant Construct Validity Receiver operating characteristic analyses10,22,23,42,51,55,68,71 of the SMFA indexes resulted in values of greater than 0.70. For the 319 responses on the dysfunction index, the values for sensitivity and specificity were 0.90 and 0.84, respectively, and the area under the curve was 0.94. For the 308 responses on the bother index, the values for sensitivity and specificity were 0.87 and 0.84, respectively, and the area under the curve was 0.94.
Responsiveness
Our findings support the reliability, validity, and responsiveness of the Short Musculoskeletal Function Assessment (SMFA) in a sample of patients who had musculoskeletal disorders. The SMFA has the versatility to function as a measure of a patient's change over time, as a measure of a group of patients who have a similar disease, and as a measure of the direct impact of functional limitation on individual patients. In the area of reliability, the dysfunction index demonstrated values for stability and internal consistency that warrant its use in the evaluation of health status in groups of patients and for making clinical decisions about individuals38. Similar to the dysfunction index, the bother index demonstrated values for internal consistency that make it appropriate for use with groups of patients and with individual patients. The stability value of the bother index (0.88) was just slightly less than the criteria of 0.90 for individual applications, suggesting that it may not be stable enough for use in guiding clinical decisions for individual patients. In the area of validity, the SMFA indexes demonstrated good content validity (that is, the scores spanned the full range of values, with few floor or ceiling effects and with distributions showing little skew). Convergent construct validity, assessed with physicians' ratings of patient function, clinical measures of patient function, SF-36 subscales, and demographic characteristics of the patients, was supported with few exceptions. Discriminant construct validity, assessed for groups of patients who had different levels of health status, was supported without exception. Although physicians' ratings of mobility of the upper and lower extremities were not found to be significantly related to the SMFA indexes, these ratings were significantly and positively correlated with specific SMFA categories. Ratings of mobility of the upper extremities were shown to be significantly related to the category of arm and hand function (rho = 0.57), and ratings of mobility of the lower extremities were found to be significantly related to the mobility category (rho = 0.56). In short, because the scores on the SMFA indexes are summary scores that describe function in a number of areas, it is not surprising that assessments of specific functional areas are not as strongly related to the summary SMFA indexes as they are to specific categories. The failure to find a relationship between the SMFA indexes and range-of-motion measures is somewhat more difficult to explain, although it has been reported by other investigators who have used range-of-motion measures as criteria for assessing health-status measures6,21,26. Patients may assess range of motion as a dichotomous variable ("I have enough motion to do what I need to do" or "I do not have enough motion"), whereas orthopaedists think of range of motion as a continuous variable. There are problems associated with finding so-called gold standards that are truly equivalent with the constructs measured by the health-status instruments. Criteria chosen for validation of health-status instruments are often complementary rather than substitutive6,9,32, and this lack of equivalence may explain the failure to find significant relationships. Additional validation studies with other patient samples may clarify this finding as well as the finding that the SMFA indexes did not vary significantly with regard to patient age, gender, or marital status. Lastly, the SMFA demonstrated very good responsiveness. Patients who reported a change in functional status displayed significant changes on the SMFA dysfunction and bother indexes (p < 0.01). These changes, evaluated with a standardized measure, ranged from moderate to large. The present study is limited by a number of factors. First, because of small numbers in some of our clinical groups (such as patients who had rheumatoid arthritis, acute fracture or soft-tissue injury with involvement of both upper and lower extremities, or acute or chronic conditions involving the spine), additional testing is needed to confirm the stability and generalizability of our findings. Second, the time-frame used to test the SMFA was narrow. We do not know how well the SMFA will perform for patients with acute problems who may have more dysfunction because of immobilization or how well it will perform for patients who are seen more than six months after injury. Third, we do not know how useful the SMFA is in the management of an individual patient. In the current study, we assessed only groups of patients over time. There are a number of issues to consider when choosing between the longer MFA questionnaire and the shorter SMFA questionnaire. For investigators who are interested in more detailed functional information and whose patients may have more time to complete the questionnaire, the long-form MFA may be useful, as it provides more categories and, in some areas, a fuller range of scores. For clinicians who are interested in a short questionnaire that can be completed at the clinic visit and that provides an overall assessment of major areas of patient function, the shorter SMFA may be the better choice. As a measure of patient change over time, the SMFA can be used during follow-up examinations to demonstrate individual differences in function. This change can be measured by individual question, category, or total score or by all of these. As a measure of disease in patients, the SMFA can be used to allow large groups of clinicians to successfully pool disease-specific data in order to provide a better measure of the severity of the disease or the effectiveness of treatment. The bother index of the SMFA can be used to assess the impact of limitation of function on the individual. A patient can endorse only the question, "How often does your leg lock or give-way?" with "some of the time," noting no other dysfunction in the questionnaire; for the initial evaluation with the SMFA, this patient would have a moderate individual question score and a low category and total score. However, because the patient is constantly worried about whether or not a given activity might cause the knee to lock, he or she might have chosen to answer "extremely bothered" for two items in the bother index—that is, use of the legs and problems with recreational activities. This flexibility in the use of the SMFA adds to the broad spectrum of applications for which it may be appropriate. In this situation, the patient is able to indicate that there is a moderately frequent occurrence of one specific dysfunction, with no other functional problems, but this one dysfunction is extremely bothersome in two facets of life. The usefulness of the SMFA in the management of individual patients is currently being studied. As both the thirty-four functional items comprising the dysfunction index and the twelve items comprising the bother index demonstrated validity, reliability, and responsiveness, a clinician may choose either index, depending on his or her needs. For example, if a physician is most interested in an inventory of the kinds of activities that are difficult for the patient to perform, the dysfunction index might be preferable to the bother index. The dysfunction index may also be suitable for tracking and monitoring patients, as it has shown reliability and stability that is adequate for individual applications. If the physician is most interested in how troubled the patient is by his or her current functional limitations, the bother index may be a better choice. Also, because there are fewer items on the bother index and these items are broader and more comprehensive, the bother index items may be more easily completed during a clinic visit. Whether they are used separately or in combination, the SMFA indexes constitute a promising short-form, self-reported questionnaire that is quickly completed and may be used in clinical settings to provide valid, reliable, and responsive assessments of health status for patients who have musculoskeletal disorders.
NOTE: The authors thank the following individuals and clinics for their participation in the present study: Clem Jones, M.D., San Francisco, California; Mark Meier, M.D., Orthopaedic Associates, Boise, Idaho; Kevin Maddox, Hugh Selznick, M.D., and Sheila Cellan, Pocatello Orthopaedics and Sports Medicine Institute, Pocatello, Idaho; David Brokaw, M.D., and Timothy Weber, M.D., Orthopaedics Indianapolis, Indianapolis, Indiana; James Hughes, M.D., and Philip Kregor, M.D., University of Mississippi, Jackson, Mississippi; Robert Snider, M.D., and Sheri Dodd, Orthopaedic Surgeons, P.S.C., Billings, Montana; Tom Sculco, M.D., and Tracie Dugas, Hospital for Special Surgery, New York, N.Y.; David Teague, M.D., University of Oklahoma, Oklahoma City, Oklahoma; David Noall, M.D., and Lily Nigo, Portland Orthopaedic Clinic, P.C., Portland, Oregon; Michael Christie, M.D., and Melanie Capps, Vanderbilt University, Nashville, Tennessee; Steven Ratcliffe, M.D., Northwest Orthopaedic Physicians, P.S., Bellevue, Washington; Michael Thorpe, M.D., Bellingham, Washington; David Bishop, M.D., Enunclaw, Washington; John Hendrickson, M.D., Martin Tullus, M.D., and Susan Cero, M.D., Valley Orthopaedic Associates, Renton, Washington; James Crutcher, M.D., and Martin Mankey, M.D., Orthopaedic Physician Associates, Seattle, Washington; Jon Keeve, M.D., and Leanne Mason, Northwest Orthopaedic and Fracture Clinic, Spokane, Washington; and Edward Farrar, M.D., and Deborah DeChand, Wenatchee Orthopaedics, Wenatchee, Washington.
*No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article. Funds were received in total or partial support of the research or clinical study presented in this article. The funding sources were the National Institute of Child Health and Human Development, the American Academy of Orthopaedic Surgeons, and the Orthopaedic Trauma Association.
  CiteULike  Connotea  Del.icio.us  Facebook  Technorati  Twitter What's this?
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Abstract
Introduction
| home | CME | help | feedback | subscriptions | archive | site index |
|
Stanford University Libraries' HighWire Press (TM) assists in the publication of JBJS Online. |