This Article Abstract Full Text (PDF) Free Letters to the Editor: Submit a response Alert me when this article is cited Alert me when Letters to the Editor are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My File Cabinet Download to citation manager Rights and Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by L'INSALATA, J. C. Articles by PETERSON, M. G. E. Search for Related Content PubMed PubMed Citation Articles by L'INSALATA, J. C. Articles by PETERSON, M. G. E. Social Bookmarking                   What's this?

A Self-Administered Questionnaire for Assessment of Symptoms and Function of the Shoulder*

JOHN C. L'INSALATA, M.D., RUSSELL F. WARREN, M.D., STEVEN B. COHEN, B.A., DAVID W. ALTCHEK, M.D. and MARGARET G. E. PETERSON, PH.D., NEW YORK, N.Y.

Investigation performed at the Sports Medicine and Shoulder Service and the Multipurpose Arthritis Center, The Hospital for Special Surgery, Affiliated with The New York Hospital—Cornell University Medical College, New York City

	Abstract

Top Abstract Introduction Materials and Methods Results Discussion References

 
A self-administered questionnaire was designed to assess the severity of symptoms related to and the functional status of the shoulder. It includes domains of global assessment, pain, daily activities, recreational and athletic activities, work, satisfaction, and areas for improvement. Each domain is graded separately and is weighted to arrive at the total score. The over-all scale and each domain were prospectively tested for validity, reliability, and responsiveness to clinical change. One hundred patients who were seen for evaluation of the shoulder were enrolled in the study. The validity of the scale was demonstrated by moderate-to-high correlation of the domains and individual questions of the Shoulder Rating Questionnaire with those of the Arthritis Impact Measurement Scales 2. Validity was supported further by significant correlation of the scores in each domain with the level of satisfaction in that domain and by significantly lower scores in domains that patients selected as areas important for improvement. The over-all scale and each domain were internally consistent (Cronbach alpha, 0.71 to 0.90). Reproducibility was evaluated by repeated administration of the questionnaire after a mean of three days to forty patients whose condition was clinically stable. Reproducibility of the over-all questionnaire and individual domains was excellent (Spearman-Brown index, 0.94 to 0.98). Individual questions were reproducible, with a weighted kappa value of more than 0.7 for each. Responsiveness was evaluated by comparison of the preoperative and postoperative scores of thirty patients who had a satisfactory result one year after an operation on the shoulder. The over-all Shoulder Rating Questionnaire and each domain were responsive to clinical change as demonstrated by favorable standardized response means (range, 1.1 to 1.9) and indices of responsiveness (range, 1.1 to 2.0). Similar analysis performed for individual diagnostic groups supported the validity, reliability, and responsiveness of the questionnaire in each group. The self-administered shoulder questionnaire was found to be valid, reliable, and responsive to clinical change. These qualities should make it a useful instrument for the prospective assessment of the outcome of treatment of disorders related to the shoulder.

	Introduction

Top Abstract Introduction Materials and Methods Results Discussion References

 
Several scoring systems have been used to evaluate the results of treatment about the shoulder^1,10,22. Points are allocated on the basis of objective criteria derived from physical examination and subjective criteria determined during an interview with the patient. Both of these methods may introduce observer bias. Also, certain so-called objective criteria may introduce substantial error because of poor interobserver or intraobserver reliability^9,15,23,25 and may be limited in their ability to measure improvements in health and quality of life^9,20.

Current shoulder evaluation forms were designed to detect clinical changes after treatment but have not been standardized or tested routinely for validity, reliability, and responsiveness. Recent work has demonstrated moderate correlation among questionnaires for the assessment of shoulder function, with regard to the ability of each to discriminate between levels of self-rated severity of disability related to the shoulder but not between levels of over-all well-being¹. Other studies have demonstrated poor correlation among clinical evaluation forms for the shoulder, making valid comparisons between study groups impossible^18,24.

Recently, there has been a growing interest in outcome studies in which valid and reliable methods are used to determine the effects of treatment on the health and the quality of life of the patient. Self-administered questionnaires have been shown to be effective in this regard^12,16,19,20. However, questionnaires validated for assessment of general health status may not be specific enough to provide an accurate, comprehensive description of symptoms and function of an individual joint^1,12. This concern has led to the recent development and validation of outcome assessment questionnaires for total hip arthroplasty and carpal tunnel syndrome^12,16.

The purpose of this paper is to present a self-administered questionnaire designed to assess symptoms and function of the shoulder and to report the results of a prospective evaluation of its validity, reliability, and responsiveness to clinical change.

	Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion References

 

Development of the Questionnaire
A preliminary questionnaire was developed and was completed by thirty patients who were being managed for disorders related to the shoulder. A subset of these patients was interviewed, and each question was assessed for clinical relevance, relative importance, and ease of completion and grading. This allowed modifications to be made to produce the revised questionnaire that was prospectively assessed. After this assessment, questions that had poor reliability, substantially reduced the total or subset internal consistency, or contributed little to the clinical sensitivity of the over-all instrument were eliminated to produce the current questionnaire. The questions that were eliminated are considered later in this article.

The Shoulder Rating Questionnaire includes six separately scored domains: global assessment, pain, daily activities, recreational and athletic activities, work, and satisfaction (Table I). A final, non-graded domain allows the patient to select two areas in which he or she believes improvement is most important (Table I).

Each of the other scored domains consists of a series of multiple-choice questions with five selections scored from 1 (poorest) to 5 (best). Each domain is scored separately by averaging the scores of the completed questions and multiplying by two. Thus, the possible score for each domain ranges from 2 (poorest) to 10 (best).

The pain domain consists of four questions that assess the severity of pain at rest (Question 2) and during activities (Question 3), the frequency of pain that interferes with sleep (Question 4), and the frequency of severe pain (Question 5).

The daily activities domain consists of six questions, including one that requires a general assessment of the limitation of daily activities (Question 6) and a series of questions that assess difficulty with typical daily activities, such as putting on or removing a pullover shirt, combing hair, reaching shelves above the head, scratching or washing the lower back, and carrying groceries (Questions 7 to 11).

The recreational and athletic activities domain consists of three questions. One asks for a general assessment of limitation during recreational and athletic activities (Question 12), another requires an assessment of the degree of difficulty in throwing a ball overhand or serving in tennis (Question 13), and the third allows the patient to select an activity that he or she particularly enjoys and to assess his or her limitation in that activity (Question 14).

The work domain includes a non-graded question that categorizes the form of work (Question 15) and four graded questions that assess the frequency of inability to do any work (Question 16), inability to work efficiently (Question 17), and the need to work a shorter day (Question 18) or to change the manner in which usual work is performed (Question 19).

The satisfaction domain (Question 20) consists of a single question that asks the patient to grade his or her over-all satisfaction from poor to excellent. This domain is not included in the total score but rather is scored and presented separately.

Finally, the importance domain (Question 21) allows the patient to rank the two areas in which he or she most desires improvement. These are rated 1, for most important, and 2, for second most important. This does not contribute to the total score but can be used with the scores of the individual domains to determine if substantial improvement has occurred in the areas most important to the patient or to individualize the weighting method used to determine the over-all score.

A suggested weighting system for the calculation of a total score was developed after consultation with several shoulder surgeons and patients regarding the relative importance of each of the domains. The maximum score was 15 points for global assessment (domain score multiplied by 1.5; score range, 0 to 15 points), 40 points for pain (domain score multiplied by four; score range, 8 to 40 points), 20 points for daily activities (domain score multiplied by two; score range, 4 to 20 points), 15 points for recreational and athletic activities (domain score multiplied by 1.5; score range, 3 to 15 points), and 10 points for work (domain score multiplied by one; score range, 2 to 10 points). Therefore, the total possible score ranged from 17 to 100 points.

Design of the Study
One hundred patients who were seen for evaluation of the shoulder at the offices of the two senior ones of us were enrolled in the study. There were seventy-three male and twenty-seven female patients. The dominant shoulder was involved in sixty-four patients and the non-dominant shoulder, in thirty-six. The mean age was forty years (range, fifteen to seventy-one years). Eleven patients were less than twenty years old, thirty-eight were twenty to thirty-nine years old, thirty-seven were forty to fifty-nine years old, and fourteen were at least sixty years old. Seventy-four patients identified their main form of work as paid work; four, as housework; and twelve, as schoolwork. Three patients were unemployed, three were disabled, and four were retired. The diagnosis was impingement syndrome in thirty-four patients, glenohumeral instability in thirty-three, a complete tear of the rotator cuff in seventeen, osteoarthrosis of the glenohumeral joint in six, adhesive capsulitis in five, osteoarthrosis of the acromioclavicular joint in three, and unknown in two.

Each patient completed the Shoulder Rating Questionnaire and the relevant domains of the Arthritis Impact Measurement Scales 2 at the initial evaluation. The Arthritis Impact Measurement Scales 2 is a self-administered health-status questionnaire designed for use by patients who have arthritis. It is reliable and validated¹⁹. The domains included from the Arthritis Impact Measurement Scales 2 were over-all impact, pain, function of the arm, work, and satisfaction. The score for each domain is normalized to a range of 0 to 10 points, with 0 being the best possible score and 10 being the worst. Forty patients whose condition was clinically stable repeated the Shoulder Rating Questionnaire a mean of three days (median, two days; range, one to seven days) after the initial questionnaire.

The patients were followed prospectively, and those who had an operation on the shoulder were reassessed one year postoperatively. Of fifty-six patients who had an operative procedure, the first thirty patients who had a satisfactory result, as determined on the basis of an interview and a physical examination, repeated the Shoulder Rating Questionnaire at the one-year evaluation.

Validity
Validity is an index of how well a test measures what it is supposed to measure. Criterion validity is the correlation of a scale with a valid; accepted; and, ideally, universally acknowledged measure of the trait or disorder under study. Criterion validity was assessed by correlation of the individual questions and domains of the Shoulder Rating Questionnaire with the comparable questions and domains of the Arthritis Impact Measurement Scales 2 with use of the Spearman rank correlation.

When there is no universally acknowledged measure of the attribute under study, construct validity is generally sought to demonstrate the validity of a scale or test. To evaluate construct validity, a theoretical construct is invoked between the attribute under study and other attributes that are expected to be related. The actual relationship between these attributes is then measured. If the relationship is as expected, the measure and the construct are correct and support the construct validity of the scale or test. Because there is arguably no universally acknowledged measure of disability related to the shoulder, construct validity was sought to support the over-all validity of the scale. The initial questionnaire allowed the patients to rate their over-all satisfaction and their satisfaction with each domain category (pain, daily function, recreational and athletic function, and work). As a measure of construct validity, satisfaction in each area was tested for correlation with the corresponding domain score with use of Spearman correlation. It was expected that satisfaction would be positively correlated with the domain score. A second estimate of construct validity was derived from the extent to which the domain scores of patients who indicated a domain as an important area for improvement were significantly worse than the scores of patients who did not indicate that domain as an important area for improvement. Finally, the interrelationship of the questions within each domain were compared with use of Spearman correlation.

Reliability
Reliability is a measure of consistency or degree of dependability. It can be divided into two major classes. Internal consistency, which is a measure of equivalence, is the ability of a scale to measure a single coherent concept. Reproducibility, or test-retest reliability, which is a measure of stability, is the ability of a scale to give the same results when administered on separate occasions⁶.

Internal consistency was assessed by calculating the Cronbach coefficient alpha for the entire scale and for each individual domain for the 100-patient cohort. An alpha of 1.0 represents perfect internal consistency, 0.90 is considered excellent, 0.80 is considered good, and 0.70 is considered acceptable^8,19.

Reproducibility was evaluated in the group of forty clinically stable patients who repeated the Shoulder Rating Questionnaire after a mean of three days. The reproducibility of the entire scale and of each domain that was composed of more than one question (pain, daily activities, recreational and athletic activities, and work) was assessed with use of the Spearman-Brown test-retest reliability test. Correlation of the initial and retest scores of the entire scale and of each domain was measured with the Spearman rank correlation coefficient and was used as an additional measure of reproducibility.

The reproducibility of the individual questions was calculated with use of the weighted kappa statistic with individual weights defined by the equation 1 - ([i - j]/[k - 1])², where i and j are the corresponding row and column numbers and k is the number of possible selections per question (in this case, five)⁴. The kappa value measures the proportion of the observed agreement that exceeds the agreement that is expected by chance alone. A kappa value of 1.0 represents perfect agreement. A kappa value of more than 0.75 indicates excellent agreement. Questions with a weighted kappa of less than 0.7 were eliminated.

Responsiveness
Responsiveness, or sensitivity, is the ability to detect clinical change. Preoperative and postoperative scores were compared with use of the paired Student t test for the first thirty patients who had a satisfactory result one year after an operation on the shoulder. Within-patient change was calculated for each domain and for the total score with the weighting method described earlier.

Sensitivity to clinical change for each domain and for the entire scale was expressed as the standardized response mean, as described by Liang et al. This is calculated as the mean difference between the preoperative and postoperative scores divided by the standard deviation of this difference, coinciding with one form of the effect size index proposed by Cohen⁵. A value of 0.2 is considered small; a value of 0.5, moderate; and a value of more than 0.8, large⁵. Instruments with a greater standardized response mean are considered more sensitive and require smaller sample sizes when used in studies comparing the results of different types of treatments. Pain response standardized response means of 0.6 to 0.75 have been found after medical treatment of rheumatoid arthritis, and standardized response means of 0.4 to 1.4 have been reported with use of validated health-status questionnaires after hip and knee arthroplasty¹⁷.

On the basis of the distribution and mean improvement in the total score and in each domain score for patients who had a satisfactory outcome after an operation on the shoulder as well as the distribution and mean difference in the scores of the test-retest questioning, the minimum clinically important improvement was estimated for the total score and for the domain scores. The index of responsiveness, as defined by Guyatt et al., was calculated. This is defined as the ratio of the minimum clinically important difference to the square root of twice the mean square error of stable patients¹¹. The greater the index of responsiveness for a given scale, the smaller the sample size needed for a clinical trial in which the scale is used to assess the outcome. The formulae for calculation of sample size for independent groups (number of patients per group) and related groups (number of paired observations) are 2([Z + Zß]/index of responsiveness)² and ([Z + Zß]/index of responsiveness)², respectively, where is the probability of erroneously concluding that there is a difference and ß is the probability of erroneously concluding that there is not a difference¹¹.

Diagnostic Groups
The patients were divided into four groups on the basis of diagnosis. Group 1 included the thirty-three patients who had glenohumeral instability; group 2, the thirty-four patients who had impingement syndrome; group 3, the seventeen patients who had a complete tear of the rotator cuff; and group 4, the six patients who had osteoarthrosis of the glenohumeral joint, the five who had adhesive capsulitis, and the three who had osteoarthrosis of the acromioclavicular joint. The two patients for whom the diagnosis was not known were not included in this part of the study. Validity, reliability, and responsiveness analyses were done separately for each diagnostic group with use of the same methods employed for the analysis of the over-all cohort. A pattern of consistent results was sought as additional evidence of validity, reliability, and responsiveness.

Questions That Were Eliminated
Several questions were eliminated from the initial questionnaire. A question that assessed the frequency of pain in the shoulder that limited activities was omitted because of poor reproducibility (weighted kappa statistic, 0.49). A question that asked the patient to describe the strength of the shoulder was eliminated because it reduced the correlation of the daily activities domain with the Arthritis Impact Measurement Scales 2 function domain and also reduced the internal consistency and reproducibility of the domain. A question regarding the degree of limitation in a second recreational or athletic activity selected by the patient was eliminated as it reduced the internal consistency and the responsiveness of the recreational and athletic activity domain.

Questions that evaluated patient satisfaction in each of the domain areas were included in the initial questionnaire as an internal standard for assessment of convergent construct validity. They were eliminated from the final questionnaire, however, because they increased the complexity and length of the questionnaire and reduced the reproducibility.

Two questions regarding instability of the shoulder were eliminated because of poor correlation with each of the other domains and an inability to demonstrate domain validity clearly. Furthermore, instability was thought to be a causal factor of dysfunction rather than an actual measure of dysfunction. The questionnaire remained responsive for patients who had a diagnosis of instability, with an over-all standardized response mean of 1.38.

	Results

Top Abstract Introduction Materials and Methods Results Discussion References

 

Validity
There was a high degree of correlation between the comparable domains of the Shoulder Rating Questionnaire and the Arthritis Impact Measurement Scales 2 (Table II). The normalized scores for the domains of the Arthritis Impact Measurement Scales 2 decrease with improved outcome, whereas the scores for the domains of the Shoulder Rating Questionnaire increase with improved outcome. Thus, a negative correlation was expected for each domain.

The satisfaction scores for each domain (eliminated from the final questionnaire) correlated well with each of the respective domain scores. The correlation coefficients were 0.6 for the pain domain score and satisfaction with pain, 0.66 for daily activities, 0.62 for recreational and athletic activities, and 0.7 for work. The correlation between over-all satisfaction and the global assessment score was 0.5. Each of these was significant (p < 0.00001). While supporting the construct validity for the scale, this finding suggests that the satisfaction score is measuring a somewhat distinct component of health status.

As a second measure of convergent construct validity, the scores of patients who selected a particular domain as an important area for improvement were compared with the scores of patients who did not select that area as important. For example, the pain domain scores of patients who selected pain as an important area for improvement were compared with those of patients who did not select pain as an important area for improvement. Significant differences were found for the domains of pain, daily activities, and recreational and athletic activities (Table III). A trend was noted toward reduced scores for patients who selected work as an important area for improvement compared with those who did not (Table III). This did not reach significance despite the greatest differential in scores of all of the domains because only seven patients selected work as an area that was important for improvement.

Reliability
The internal consistency (measured with the Cronbach coefficient alpha) of the over-all final questionnaire was 0.86. The internal consistency was 0.71 for pain, 0.90 for daily activities, 0.77 for recreational and athletic activities, and 0.83 for work. The Cronbach alpha exceeded the 0.70 acceptable threshold for the over-all questionnaire and each domain.

Reproducibility (measured with the Spearman rank correlation coefficient) was very good, with r values ranging from 0.81 (satisfaction) to 0.96 (work and over-all scale) (p < 0.0001 for each) (Table IV). Spearman-Brown test-retest analysis was excellent, with coefficient values ranging from 0.94 (recreational and athletic activities) to 0.98 (work and over-all scale) (Table IV).

Responsiveness
There was significant improvement in the postoperative scores in each domain and in the over-all scale (p < 0.0001) (Table V). Responsiveness as measured by the standardized response mean was excellent for each domain and the over-all scale, with each substantially above the 0.8 threshold, which was considered large. The scores at the one-year evaluation had improved for 100 per cent (all thirty) of the patients in relation to the preoperative scores for the over-all scale, for 90 per cent (twenty-seven of thirty patients) for global assessment, for 93 per cent (twenty-eight of thirty) for pain, for 90 per cent (twenty-seven of thirty) for daily activities, for 97 per cent (twenty-nine of thirty) for recreational and athletic activities, for 67 per cent (eighteen of twenty-seven) for work, and for 86 per cent (twenty-five of twenty-nine) for satisfaction.

Similar analysis was performed for the individual domains. The minimum clinically important difference was estimated to be 2 points for each domain. At one year after an operation on the shoulder, the score for 62 per cent of the domains had improved by at least 2 points and twenty-seven (90 per cent) of the thirty patients had improvement of at least 2 points in two domains or more, not including the satisfaction domain. For the forty clinically stable patients (test-retest analysis), 5.5 per cent of the domains had a change in score of 2 points while only one (3 per cent) of the forty patients had a 2-point change in the score of two domains. Thus, improvement of at least 2 points in two domains or more provides a threshold for clinically important over-all change independent of domain weighting with an estimated error rate of 0.025 and an estimated ß error rate of 0.10 (Table V).

Diagnostic Groups
The Spearman rank correlation coefficients for comparison of the domain scores for pain, daily activities, and work on the Shoulder Rating Questionnaire with the comparable domain scores on the Arthritis Impact Measurement Scales 2 for each diagnostic group ranged from 0.62 to 0.94. The internal consistency of the over-all scale (as measured with the Cronbach coefficient alpha) ranged from 0.85 to 0.90 for the four groups. The test-retest analysis for the four groups yielded Spearman-Brown coefficients ranging from 0.97 to 0.99 for the over-all scale. Responsiveness assessment for the four groups demonstrated an over-all scale standardized response mean of 1.4 for group 1, 2.2 for group 2, 1.3 for group 3, and 4.1 for group 4. The standardized response mean value of each domain exceeded the 0.8 large threshold for each group except for the global domain of group 3 and the work domain of group 4, for which the values were in the moderate range (0.56 and 0.66, respectively). This supports the validity, reliability, and responsiveness of the Shoulder Rating Questionnaire for each of the diagnostic groups.

	Discussion

Top Abstract Introduction Materials and Methods Results Discussion References

 
The importance of improving the validity of orthopaedic clinical research, including the standardization of definitions and outcome criteria, has been emphasized previously^2,9. While important progress has been made in this area, additional work is necessary^12,16,22. Currently, there is a broad consensus that well designed studies of outcome are needed to demonstrate the ability of orthopaedic procedures to improve the health status and the quality of life of patients. These studies should not rely exclusively on intermediate outcomes and performance indicators, such as range of motion or lack of loosening of the prosthesis, to measure health outcomes⁹. Outcome measures should address areas of principal concern to patients, including an improved sense of well-being, relief of pain, functional improvement, and improved economic status^7,9. By definition, outcomes research is prospective and should make use of methods of assessment that are valid, reliable, and responsive to clinical change^13,16.

Several methods have been used to evaluate the results of treatment of disorders related to the shoulder^1,10,22. It is generally agreed that the development of a standard method of assessment of function of the shoulder is important; however, the ideal mechanism for this remains controversial¹⁰. Questionnaires that assess function of the shoulder have been found to have moderate-to-good correlation with one another¹, while scoring systems that include a clinical examination with which to arrive at a score have been shown to correlate poorly with one another^18,24. To our knowledge, there are no published reports demonstrating the reliability and responsiveness of these systems. Thus, a reliable, validated self-administered questionnaire used together with traditional methods of clinical assessment may provide a standardized means for comparison of data while eliminating potential error due to observer bias and interobserver variability. Other advantages of self-administered questionnaires are that they require a minimum investment of professional time, are relatively inexpensive, and potentially improve the rate of follow-up of clinical studies as they can be completed by mail or telephone.

The essential elements of a health-status index for use in outcomes research have been well defined^16,21. First, the index should be simple and practical; the time and costs related to collection of the data should be minor, and the index should include a limited number of comprehensible scales that are easy to score and interpret. Second, it should be dependable; the scales must meet accepted standards of validity, reproducibility, and internal consistency. Finally, the index should be useful; it must be responsive to clinical change and should be applicable to the population being studied but independent of a specific diagnosis or treatment.

The Shoulder Rating Questionnaire meets each of these criteria. It is self-administered and can usually be completed in five to ten minutes. Each domain is scored by averaging the individual scores for each question without a need for confusing normalization scales. The most tedious portion of the process is scoring the questionnaire and entering the data into a database, but we are currently eliminating this portion with the use of touch-screen computers located in patient waiting areas. Patients complete the computerized questionnaire, and the data are immediately entered into the database. With current improvements in technology, systems such as this or computer-ready data forms will become readily available.

The criterion validity of the Shoulder Rating Questionnaire was demonstrated by statistical correlation with comparable domains of the Arthritis Impact Measurement Scales 2, a valid and reliable health-status index¹⁹. Additional evidence of validity was based on estimates of convergent construct validity. As would be expected, the domain scores were significantly correlated with the level of satisfaction, and the scores were significantly lower in domains selected as important areas for improvement. Furthermore, comparable questions within each domain were moderately to highly correlated with one another.

The over-all Shoulder Rating Questionnaire had very good internal consistency, with a Cronbach alpha of 0.86. The internal consistency for individual domains was acceptable to excellent. The reproducibility of the questionnaire was evaluated for the over-all scale, each domain, and individual questions and was very good to excellent in each case, exceeding the internal reliability reported for several objective measures, including manual motor-strength testing²⁵, the range of motion of the shoulder²³, and radiographic assessment of osteoarthrosis¹⁴. In the present study, the test-retest interval was relatively short (a mean of three days) in order to limit the possibility of true clinical change falsely reducing the measured reproducibility of the questionnaire. This may, however, increase the chance of a memory effect, falsely improving the measured reproducibility of the questionnaire.

Our goal in the assessment of the responsiveness of the Shoulder Rating Questionnaire was to determine whether the questionnaire could detect clinical change when it had indeed occurred. As such, only patients who had a satisfactory clinical result postoperatively were evaluated. The questionnaire was able to detect significant improvements in the total score and in each domain score, verifying its responsiveness. This approach differs from that of a clinical study, in which the goal would be to evaluate the results of a particular treatment. All patients would be evaluated after treatment, with the knowledge that the Shoulder Rating Questionnaire is able to detect clinically important change if it has occurred.

The responsiveness of the Shoulder Rating Questionnaire, as measured by the standardized response mean, compares favorably with the values reported for several health-status questionnaires. The standardized response mean for the over-all scale was 1.9, whereas values for the individual domains ranged from 1.1 to 1.8. Standardized response means of 0.39 to 1.41 have been reported after total hip and knee arthroplasty¹⁷, while values of 0.82 and 1.4 were reported for the scales of a validated assessment questionnaire for carpal tunnel syndrome¹⁶.

Guyatt et al. described an index of responsiveness that relates the variability of scores in clinically stable patients to the clinically important difference in scores of patients who have had treatment. The advantage of this index is that it allows relatively simple calculation of sample-size requirements for investigations that use the questionnaire, an essential part of study design^9,11. The index of responsiveness for the Shoulder Rating Questionnaire was 1.58, which compares favorably with the index of 1.09 reported for the hip-rating questionnaire¹². With use of the index of responsiveness, it was determined that approximately nine patients would be needed to find a significant difference between the postoperative score and the preoperative score if there was a difference of 12 points or more. This demonstrates the sensitivity of the current instrument. For studies that compare the results of two different treatments or that compare a treatment group with a control group, differences of fewer than 12 points in the final scores between the two groups may be clinically important because the control group reduces the potential of chance improvements in score. In such a case, a new index of responsiveness can be calculated with use of the mean square error for clinically stable patients that we have reported and the selected minimum clinically important difference for an individual study. For example, if we select a minimum clinically important difference of 5 points for comparison of the results of two treatments for a specific disorder related to the shoulder, the new index of responsiveness would be 5/ 2 x 29 = 0.66. With use of the formula described by Guyatt et al., approximately fifty patients would be needed in each treatment group to demonstrate significance between the two treatment groups with an alpha of 0.05 (two-tailed) and a power of 80 per cent.

The Shoulder Rating Questionnaire appears to be applicable to the broad range of disorders related to the shoulder. We divided the patients into four diagnostic groups and demonstrated validity, reliability, and responsiveness of the scale in each. Additional research needs to be done, with use of larger cohorts of patients, to assess better the effectiveness of the scale for specific diagnoses.

Finally, the use of relative weighting of each domain to arrive at a final score deserves discussion. The effects of variation in relative weighting of domains has been shown to produce substantial differences in over-all scores after total hip arthroplasty³. We developed a suggested weighting system on the basis of information obtained during interviews with shoulder surgeons and patients regarding the relative importance of each area. We calculated the standardized response mean and analyzed the distribution and mean change of scores to determine the minimum clinically important change and the index of responsiveness. We performed a similar analysis with use of nine alternate weighting methods, and we found that variation in the weighting methods had minimum effect on the responsiveness of the questionnaire. Thus, the relative weighting of the Shoulder Rating Questionnaire could be customized to the population of patients that is being studied while the validity, reliability, and responsiveness of the instrument are maintained. Relative weighting could be based on the diagnosis or it could be individualized for each patient on the basis of their responses to the importance domain. In order to maintain a standard scale that allows comparison of different studies, the mean score and change in score for each domain should also be reported, as these values are independent of the relative weighting.

As the resources available for medical care become more restricted, greater emphasis will be placed on distinguishing treatments that are both medically effective and cost-effective. This requires a standardized method for assessment of patient-oriented results of care with use of techniques that are valid, reliable, and responsive to clinical change. The Shoulder Rating Questionnaire meets these criteria. The questionnaire should be used together with a professional clinical assessment and, when indicated, radiographic evaluation to provide a complete assessment of the outcome related to the shoulder. We hope that the Shoulder Rating Questionnaire will be a useful tool for the evaluation of the outcome of treatment of disorders related to the shoulder.

NOTE: The authors thank Mary E. Charlson, M.D., for insights and suggestions and Lois Horowitz, R.N., Nancy Abbate, R.N., and Donna L'Insalata, R.N., for assistance in collection of the data.

	Footnotes

 

*No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article. Funds were received in total or partial support of the research or clinical study presented in this article. The funding source was National Institutes of Health Multipurpose Arthritis Center Program Grant 5P60-AR38520.

Orthopaedic Surgical Consultant, P.C., 9921 4th Avenue, Brooklyn, New York 11209.

The Hospital for Special Surgery, 535 East 70th Street, New York, N.Y. 10021.

	References

Top Abstract Introduction Materials and Methods Results Discussion References

 

1. Beaton, D. E., and Richards, R. R.: Measuring function of the shoulder. A cross-sectional comparison of five questionnaires. J. Bone and Joint Surg., 78-A: 882-890, June 1996.[Abstract/Free Full Text]
2. Burstein, A. H., and Cohen, J.: Editorial. Measurements in the conduct of research. J. Bone and Joint Surg., 75-A: 319-320, March 1993.[Free Full Text]
3. Callaghan, J. J.; Dysart, S. H.; Savory, C. F.; and Hopkinson, W. J.: Assessing the results of hip replacement. A comparison of five different rating systems. J. Bone and Joint Surg., 72-B(6): 1008-1009, 1990.
4. Cohen, J.: Weighted kappa. Nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull., 70: 213-220, 1968.[Medline]
5. Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. New York, Academic Press, 1977.
6. Cronbach, L. J.: Coefficient alpha and the internal structure of tests. Psychometrika, 16: 297-334, 1951.
7. Epstein, A. M.: The outcomes movement—will it get us where we want to go?. New England J. Med., 323: 266-270, 1990.[Medline]
8. Feinstein, A. R.: Clinimetrics. New Haven, Connecticut, Yale University Press, 1987.
9. Gartland, J. J.: Orthopaedic clinical research. Deficiencies in experimental design and determinations of outcome. J. Bone and Joint Surg., 70-A: 1357-1364, Oct. 1988.[Abstract/Free Full Text]
10. Gerber, C.: Integrated scoring systems for the functional assessment of the shoulder. In The Shoulder: A Balance of Mobility and Stability, pp. 531-550. Edited by F. A. Matsen, III, F. H. Fu, and R. J. Hawkins. Rosemont, Illinois, The American Academy of Orthopaedic Surgeons, 1993.
11. Guyatt, G.; Walter, S.; and Norman, G.: Measuring change over time: assessing the usefulness of evaluative instruments. J. Chronic Dis., 40: 171-178, 1987.[Medline]
12. Johanson, N. A.; Charlson, M. E.; Szatrowski, T. P.; and Ranawat, C. S.: A self-administered hip-rating questionnaire for the assessment of outcome after total hip replacement. J. Bone and Joint Surg., 74-A: 587-597, April 1992.[Abstract/Free Full Text]
13. Keller, R. B.: How outcomes research should be done. In The Shoulder: A Balance of Mobility and Stability, pp. 487-499. Edited by F. A. Matsen, III, F. H. Fu, and R. J. Hawkins. Rosemont, Illinois, The American Academy of Orthopaedic Surgeons, 1993.
14. Koran, L. M.: The reliability of clinical methods, data and judgments. New England J. Med., 293: 642-646, 1975.[Medline]
15. Lea, R. D., and Gerhardt, J. J.: Current concepts review. Range-of-motion measurements. J. Bone and Joint Surg., 77-A: 784-798, May 1995.[Free Full Text]
16. Levine, D. W.; Simmons, B. P.; Koris, M. J.; Daltroy, L. H.; Hohl, G. G.; Fossel, A. H.; and Katz, J. N.: A self-administered questionnaire for the assessment of severity of symptoms and functional status in carpal tunnel syndrome. J. Bone and Joint Surg., 75-A: 1585-1592, Nov. 1993.[Abstract/Free Full Text]
17. Liang, M. H.; Fossel, A. H.; and Larson, M. G.: Comparisons of five health status instruments for orthopedic evaluation. Med. Care, 28: 632-642, 1990.[Medline]
18. Lirette, R.; Morin, F.; and Kinnard, P.: The difficulties in assessment of results of anterior acromioplasty. Clin. Orthop., 278: 14-16, 1992.
19. Meenan, R. F.; Mason, J. H.; Anderson, J. J.; Guccione, A. A.; and Kazis, L. E.: AIMS2. The content and properties of a revised and expanded Arthritis Impact Measurement Scales Health Status Questionnaire. Arthrit. and Rheumat., 35: 1-10, 1992.
20. Meenan, R. F.; Anderson, J. J.; Kazis, L. E.; Egger, M. J.; Altz-Smith, M.; Samuelson, C. O., Jr.; Willkens, R. F.; Solsky, M. A.; Hayes, S. P.; Blocka, K. L.; Weinstein, A.; Guttadauria, M.; Kaplan, S. B.; and Klippel, J.: Outcome assessment in clinical trials. Evidence for the sensitivity of a health status measure. Arthrit. and Rheumat., 27: 1344-1352, 1984.
21. Miller, J. E.: Guidelines for selecting a health status index: suggested criteria. In Health Status Indexes, pp. 243-251. Edited by R. L. Berg. Chicago, Hospital Research and Education Trust, 1973.
22. Richards, R. R.; An, K. N.; Bigliani, L. U.; Friedman, R. J.; Gartsman, G. M.; Gristina, A. G.; Iannotti, J. P.; Mow, V. C.; Sidles, J. A.; and Zuckerman, J. D.: A standardized method for the assessment of shoulder function. J. Shoulder and Elbow Surg., 3: 347-352, 1994.
23. Rodkey, W. G.; Noble, J. S.; and Hintermeister, R. A.: Laboratory methods of evaluating the shoulder: strength, range of motion, and stability. In The Shoulder: A Balance of Mobility and Stability, pp. 551-577. Edited by F. A. Matsen, III, F. H. Fu, and R. J. Hawkins. Rosemont, Illinois, The American Academy of Orthopaedic Surgeons, 1993.
24. Romeo, A. A.; Bach, B. R., Jr.; and O'Halloran, K. L.: Scoring systems for shoulder conditions. Am. J. Sports Med., 24: 472-476, 1996.[Abstract/Free Full Text]
25. Sapega, A. A.: Current concepts review. Muscle performance evaluation in orthopaedic practice. J. Bone and Joint Surg., 72-A: 1562-1574, Dec. 1990.[Free Full Text]

CiteULike

Connotea

Del.icio.us

Facebook

Technorati

Twitter

This article has been cited by other articles:

				J. Agel and R. F. LaPrade Assessment of Differences Between the Modified Cincinnati and International Knee Documentation Committee Patient Outcome Scores: A Prospective Study Am. J. Sports Med., November 1, 2009; 37(11): 2151 - 2157. [Abstract] [Full Text] [PDF]

				X. Jia, S. A. Petersen, A. H. Khosravi, V. Almareddi, V. Pannirselvam, and E. G. McFarland Examination of the Shoulder: The Past, the Present, and the Future J. Bone Joint Surg. Am., November 1, 2009; 91(Supplement_6): 10 - 18. [Full Text] [PDF]

				L. Eriksson, B. Lindstrom, G. Gard, and J. Lysholm Physiotherapy at a distance: a controlled study of rehabilitation at home after a shoulder joint operation J Telemed Telecare, July 1, 2009; 15(5): 215 - 220. [Abstract] [Full Text] [PDF]

				K. M. Baumgarten, A. F. Vidal, and R. W. Wright Rotator Cuff Repair Rehabilitation: A Level I and II Systematic Review Sports Health: A Multidisciplinary Approach, March 1, 2009; 1(2): 125 - 130. [Abstract] [Full Text] [PDF]

				U. G. Longo, F. Franceschi, M. Loppini, N. Maffulli, and V. Denaro Rating systems for evaluation of the elbow Br. Med. Bull., September 1, 2008; 87(1): 131 - 161. [Abstract] [Full Text] [PDF]

				N. G. Mohtadi, R. M. Hollinshead, T. M. Sasyniuk, J. A. Fletcher, D. S. Chan, and F. X. Li A Randomized Clinical Trial Comparing Open to Arthroscopic Acromioplasty With Mini-Open Rotator Cuff Repair for Full-Thickness Rotator Cuff Tears: Disease-Specific Quality of Life Outcome at an Average 2-Year Follow-Up Am. J. Sports Med., June 1, 2008; 36(6): 1043 - 1051. [Abstract] [Full Text] [PDF]

				A. Paul, M. Lewis, J. Saklatvala, I. McCall, M. Shadforth, P. Croft, and E. Hay Cervical spine magnetic resonance imaging in primary care consulters with shoulder pain: a case control study Ann Rheum Dis, October 1, 2007; 66(10): 1363 - 1368. [Abstract] [Full Text] [PDF]

				M. Sandstrom and L. Lundin-Olsson Development and evaluation of a new questionnaire for rating perceived participation1 Clinical Rehabilitation, September 1, 2007; 21(9): 833 - 845. [Abstract] [PDF]

				J. E. Voos, A. D. Pearle, C. J. Mattern, F. A. Cordasco, A. A. Allen, and R. F. Warren Outcomes of Combined Arthroscopic Rotator Cuff and Labral Repair Am. J. Sports Med., July 1, 2007; 35(7): 1174 - 1179. [Abstract] [Full Text] [PDF]

				S. H. Coleman, D. B. Cohen, M. C. Drakos, A. A. Allen, R. J. Williams, S. J. O'Brien, D. W. Altchek, and R. F. Warren Arthroscopic Repair of Type II Superior Labral Anterior Posterior Lesions With and Without Acromioplasty: A Clinical Analysis of 50 Patients Am. J. Sports Med., May 1, 2007; 35(5): 749 - 753. [Abstract] [Full Text] [PDF]

				J. R. Walton, N. K. Bowman, Y. Khatib, J. Linklater, and G. A.C. Murrell Restore Orthobiologic Implant: Not Recommended for Augmentation of Rotator Cuff Repairs J. Bone Joint Surg. Am., April 1, 2007; 89(4): 786 - 791. [Abstract] [Full Text] [PDF]

				D. A Nawoczenski, J. M Ritter-Soronen, C. M Wilson, B. A Howe, and P. M Ludewig Clinical Trial of Exercise for Shoulder Pain in Chronic Spinal Injury Physical Therapy, December 1, 2006; 86(12): 1604 - 1618. [Abstract] [Full Text] [PDF]

				K. Anderson, M. Boothby, D. Aschenbrener, and M. van Holsbeeck Outcome and Structural Integrity After Arthroscopic Rotator Cuff Repair Using 2 Rows of Fixation: Minimum 2-Year Follow-Up Am. J. Sports Med., December 1, 2006; 34(12): 1899 - 1905. [Abstract] [Full Text] [PDF]

				J. S. Dines, S. Fealy, E. J. Strauss, A. Allen, E. V. Craig, R. F. Warren, and D. M. Dines Outcomes Analysis of Revision Total Shoulder Replacement J. Bone Joint Surg. Am., July 1, 2006; 88(7): 1494 - 1500. [Abstract] [Full Text] [PDF]

				A. M. Smith, S. A. Barnes, J. W. Sperling, C. M. Farrell, J. D. Cummings, and R. H. Cofield Patient and Physician-Assessed Shoulder Function After Arthroplasty J. Bone Joint Surg. Am., March 1, 2006; 88(3): 508 - 513. [Abstract] [Full Text] [PDF]

				H. M Vermeulen, P. M Rozing, W. R Obermann, S. le Cessie, and T. P. Vliet Vlieland Comparison of High-Grade and Low-Grade Mobilization Techniques in the Management of Adhesive Capsulitis of the Shoulder: Randomized Controlled Trial Physical Therapy, March 1, 2006; 86(3): 355 - 368. [Abstract] [Full Text] [PDF]

				M. S. Kocher, M. P. Horan, K. K. Briggs, T. R. Richardson, J. O'Holleran, and R. J. Hawkins Reliability, Validity, and Responsiveness of the American Shoulder and Elbow Surgeons Subjective Shoulder Scale in Patients with Shoulder Instability, Rotator Cuff Disease, and Glenohumeral Arthritis J. Bone Joint Surg. Am., September 1, 2005; 87(9): 2006 - 2011. [Abstract] [Full Text] [PDF]

				P. J Rundquist and P. M Ludewig Correlation of 3-Dimensional Shoulder Kinematics to Function in Subjects With Idiopathic Loss of Shoulder Range of Motion Physical Therapy, July 1, 2005; 85(7): 636 - 647. [Abstract] [Full Text] [PDF]

				S. Chen, P. S. Haen, J. Walton, and G. A. C. Murrell The Effects of Thermal Capsular Shrinkage on the Outcomes of Arthroscopic Stabilization for Primary Anterior Shoulder Instability Am. J. Sports Med., May 1, 2005; 33(5): 705 - 711. [Abstract] [Full Text] [PDF]

				P. Harvie, T. C. B. Pollard, R. J. Chennagiri, and A. J. Carr The use of outcome scores in surgery of the shoulder J Bone Joint Surg Br, February 1, 2005; 87-B(2): 151 - 154. [Full Text] [PDF]

				J. D. O'Holleran, M. S. Kocher, M. P. Horan, K. K. Briggs, and R. J. Hawkins Determinants of Patient Satisfaction with Outcome After Rotator Cuff Surgery J. Bone Joint Surg. Am., January 1, 2005; 87(1): 121 - 126. [Abstract] [Full Text] [PDF]

				P. Sathyamoorthy, G. J. Kemp, A. Rawal, V. Rayner, and S. P. Frostick Development and validation of an elbow score Rheumatology, November 1, 2004; 43(11): 1434 - 1440. [Abstract] [Full Text] [PDF]

				A Paul, M Lewis, M F Shadforth, P R Croft, D A W M van der Windt, and E M Hay A comparison of four shoulder-specific questionnaires in primary care Ann Rheum Dis, October 1, 2004; 63(10): 1293 - 1299. [Abstract] [Full Text] [PDF]

				J. D. Placzek, S. C. Lukens, S. Badalanmenti, P. J. Roubal, D. C. Freeman, K. M. Walleman, A. Parrot, and J. M. Wiater Shoulder Outcome Measures: A Comparison of 6 Functional Tests Am. J. Sports Med., July 1, 2004; 32(5): 1270 - 1277. [Abstract] [Full Text] [PDF]

				J. A. Paoloni, R. C. Appleyard, J. Nelson, and G. A.C. Murrell Topical Glyceryl Trinitrate Treatment of Chronic Noninsertional Achilles Tendinopathy. A Randomized, Double-Blind, Placebo-Controlled Trial J. Bone Joint Surg. Am., May 1, 2004; 86(5): 916 - 922. [Abstract] [Full Text] [PDF]

				S D M Bot, C B Terwee, D A W M van der Windt, L M Bouter, J Dekker, and H C W de Vet Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature Ann Rheum Dis, April 1, 2004; 63(4): 335 - 341. [Abstract] [Full Text] [PDF]

				P M Ludewig and J D Borstad Effects of a home exercise programme on shoulder pain and functional status in construction workers Occup. Environ. Med., November 1, 2003; 60(11): 841 - 849. [Abstract] [Full Text] [PDF]

				R. J. Williams III, S. Strickland, M. Cohen, D. W. Altchek, and R. F. Warren Arthroscopic Repair for Traumatic Posterior Shoulder Instability Am. J. Sports Med., March 1, 2003; 31(2): 203 - 209. [Abstract] [Full Text] [PDF]

				C. R. Bottoni, J. H. Wilckens, T. M. DeBerardino, J.-C. G. D'Alleyrand, R. C. Rooney, J. K. Harpstrite, and R. A. Arciero A Prospective, Randomized Evaluation of Arthroscopic Stabilization Versus Nonoperative Treatment in Patients with Acute, Traumatic, First-Time Shoulder Dislocations Am. J. Sports Med., July 1, 2002; 30(4): 576 - 580. [Abstract] [Full Text] [PDF]

				F. A. Cordasco, M. Backer, E. V. Craig, D. Klein, and R. F. Warren The Partial-Thickness Rotator Cuff Tear: Is Acromioplasty Without Repair Sufficient? Am. J. Sports Med., March 1, 2002; 30(2): 257 - 260. [Abstract] [Full Text] [PDF]

				K. Anderson, R. F. Warren, D. W. Altchek, E. V. Craig, and S. J. O'Brien Risk Factors for Early Failure after Thermal Capsulorrhaphy Am. J. Sports Med., January 1, 2002; 30(1): 103 - 107. [Abstract] [Full Text] [PDF]

				R. G. Marx, E. C. Jones, A. A. Allen, D. W. Altchek, S. J. O'Brien, S. A. Rodeo, R. J. Williams, R. F. Warren, and T. L. Wickiewicz Reliability, Validity, and Responsiveness of Four Knee Outcome Scales for Athletic Patients J. Bone Joint Surg. Am., October 1, 2001; 83(10): 1459 - 1469. [Abstract] [Full Text] [PDF]

				P. CROFT Measuring up to shoulder pain Ann Rheum Dis, February 1, 1998; 57(2): 65 - 66. [Full Text]

This Article Abstract Full Text (PDF) Free Letters to the Editor: Submit a response Alert me when this article is cited Alert me when Letters to the Editor are posted Alert me if a correction is posted Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My File Cabinet Download to citation manager Rights and Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by L'INSALATA, J. C. Articles by PETERSON, M. G. E. Search for Related Content PubMed PubMed Citation Articles by L'INSALATA, J. C. Articles by PETERSON, M. G. E. Social Bookmarking                   What's this?

Stanford University Libraries' HighWire Press (TM) assists in the publication of JBJS Online.
Copyright © 1997 by The Journal of Bone and Joint Surgery, Inc. Online ISSN: 1535-1386.
The Journal of Bone and Joint Surgery®, JBJS®, JB&JS®, and eJBJS® are registered in the U.S. Patent and Trademark Office.