 |
 |
 |
|
| home | CME | help | feedback | subscriptions | archive |
|
© 1997 The Journal of Bone and Joint Surgery, Inc. Publication Patterns of Papers Presented at the Annual Meeting of The American Academy of Orthopaedic Surgeons* , LOS ANGELES, CALIFORNIAInvestigation performed at the Department of Orthopaedic Surgery, University of California at Los Angeles, Los Angeles
Publication in a peer-reviewed journal is the ultimate lasting acknowledgment of an author's research results. Numerous clinical and basic-science papers are presented at the Annual Meeting of The American Academy of Orthopaedic Surgeons, but not all of these presentations culminate in publication in a peer-reviewed journal. In an effort to determine the percentages and patterns of publication, every presentation from the 1990, 1991, and 1992 meetings (a total of 1465 presentations) was reviewed with use of a Melvyl Medline Plus computer search during the summer of 1996. The presentations were classified on the basis of subspecialty. The over-all rate of publication by August 1996 was 46 per cent (668 of 1465), and the rates were similar for the 1990, 1991, and 1992 meetings. The average time to publication was twenty months, and most publications appeared within three years after presentation, with a plateau thereafter. Basic-science presentations had the highest rate of publication: 64 per cent (eighteen of twenty-eight presentations). The rates of publication by clinical subspecialty ranged from 39 per cent (twenty-eight of seventy-two presentations in the hand subspecialty and eighty-two of 210 presentations in the trauma subspecialty) to 53 per cent (110 of 206 presentations in the spine subspecialty). There was no significant difference among these groups (p = 0.611, chi-square test). The American volume of The Journal of Bone and Joint Surgery published the most papers (24 per cent; 161 of the 668 publications), followed by Clinical Orthopaedics and Related Research (16 per cent; 106 of 668). The over-all rate of publication of presentations at the Annual Meeting of The American Academy of Orthopaedic Surgeons is comparable with reported rates in other medical specialties.
The ultimate goal of any scientific query is to explain, defend, and share research methodology and results with other scientists. The two most common modes of expression of new scientific data are oral presentations at meetings and written publications in scientific journals. Publication in a peer-reviewed journal is generally considered the most lasting, retrievable, and validated culmination of the scientific process7,8,10,11. The purpose of the present study was to determine the rate of publication of papers in peer-reviewed journals after oral presentation at the Annual Meeting of The American Academy of Orthopaedic Surgeons and also to define the patterns of publication for different orthopaedic subspecialties.
All 1465 presentations listed in the Final Programs of the Annual Meetings of The American Academy of Orthopaedic Surgeons for 1990, 1991, and 1992 were entered into the database. The abstracts were divided into nine different subspecialty groups on the basis of the title. The content of approximately 10 per cent of the abstracts was read because the abstract fell into a general category in The American Academy of Orthopaedic Surgeons Final Program and the title suggested that it could pertain to more than one subspecialty. There were eight clinical subspecialty groups: hand, pediatrics, sports, trauma, spine, arthroplasty, oncology, and foot and ankle. The remaining group, basic science, encompassed any anatomical, biomechanical, or biological topic that related to orthopaedics in general more than to any subspecialty. The division of abstracts into clinical subspecialties was in some measure subjective as it was based on generally accepted categorizations often found in the literature as well as on the grouping of abstracts in the Final Program. For instance, the group of abstracts considered for the hand subspecialty included ones dealing with congenital deformities of the hand as well as some on traumatic injuries of the hand, such as fractures of the scaphoid and the phalanges. The abstracts in the spine subspecialty were those dealing with most conditions relating to the spine, including spinal disorders in children and adolescents and traumatic injuries of the spine. Most presentations relating to fractures, with the exception of fractures of the hand or spine, were included in the trauma category. The primary rationale for this categorization was the division into specialty groups as listed in the Final Program. For instance, although a presentation on the treatment of unstable thoracolumbar fractures theoretically could be included in either the trauma or the spine subspecialty, it was categorized in the latter since it was listed in one of the spine sections in the Final Program. The tumor section encompassed all abstracts relating to neoplasms or chronic infection in any anatomical location. Any paper with clear methodology relating to anatomy, biomechanics, or molecular biology that applied to orthopaedics in general was listed in the basic-science category. After division into subspecialties, a computer search for each abstract was done with the Melvyl Medline Plus database in order to ascertain if it had been published in a peer-reviewed journal (from January 1990 through August 1996 for the presentations given at the 1990 and 1991 meetings and from January 1992 through August 1996 for those given at the 1992 meeting). The Medline system is divided into five-year periods, with the exception of 1990 and 1991, which are included in one database. The database for 1990 and 1991 and that for 1992 to the present were searched for presentations given in 1990 and 1991; the database for 1992 to the present also was searched for presentations given in 1992. Abstracts were initially searched for by key words in the title. If that did not reveal a publication, then a search was initiated with use of the last name of every author of the presentation cross-referenced with broad key words from the abstract, such as hip arthroplasty, until a match was found or it was ascertained that there was not a match. In many cases, the title and authors of a publication revealed by a search differed somewhat from those of the presentation that was being searched for. Sometimes there were several potential matches with the title of the presentation, and it was frequently necessary to compare several published abstracts with the abstract of the presentation from the Final Program to determine if any of the former were nearly identical to the latter with regard to the experimental protocol and the number of patients. If the protocols did not match almost exactly, then the paper was not considered a publication of the presentation, even if there were some similarities between the two. We did not investigate the possibility of dual publication. If there was a nearly identical match between the methodology in the presentation and that in the publication, then that paper was listed as the published version of the presentation; we did not endeavor to search further for other publications. In some instances, the data in a publication appeared to have similarities to presented data and may have been indirectly generated from the research described in the original presentation. There were also times when it was possible that authors had used the data presented at the Annual Meeting in combination with other data to create a single, larger manuscript. As we were unable to exclude this possibility with a retrospective search of Medline, we did not include a publication in our data pool unless there was a nearly identical match between the methodology and the number of patients in that publication and those in the presentation. A database of published and unpublished presentations was then prepared. The time from presentation to publication was determined to the nearest month in order to construct a detailed table and a graphical representation of the publications in each year after the 1990, 1991, and 1992 Annual Meetings of The American Academy of Orthopaedic Surgeons. If the abstract was published in the same month as the meeting or any time before the meeting, a value of zero months was assigned. Negative values were not used.
There was a total of 1465 presentations at the Annual Meetings of The American Academy of Orthopaedic Surgeons in 1990, 1991, and 1992. From these 1465 presentations, 668 papers were published in peer-reviewed journals by August 1996, as determined by our search of Medline. This resulted in an over-all rate of publication of 46 per cent. Two hundred and thirty-nine (46 per cent) of the 523 presentations from the 1990 meeting, 228 (46 per cent) of the 501 presentations from the 1991 meeting, and 201 (46 per cent) of the 441 presentations from the 1992 meeting were eventually published (Table I). There was a gradual increase in the total number of published presentations with each month after the meeting until a plateau was reached approximately three years after the time of presentation (Fig. 1). The absolute values and the shape of the curves were quite similar from year to year.
Thirty (5 per cent) of the 668 published papers, and thus thirty (2 per cent) of the 1465 presentations, were published before the meeting at which the presentation was given. Of the 1465 presentations, 221 (15 per cent) were published within one year; 427 (29 per cent), by two years; 579 (40 per cent), by three years; and 648 (44 per cent), by four years. By five years after the meeting, 463 (45 per cent) of the 1024 presentations from the 1990 and 1991 meetings had been published; by six years after the meeting, 239 (46 per cent) of the 523 publications from the 1990 meeting had been published (Table I). The highest over-all rate of publication was in the basic-science category: eighteen (64 per cent) of the twenty-eight presentations were published. The next-highest rates were in the spine subspecialty (110 [53 per cent] of 206 presentations) and the foot-and-ankle subspecialty (sixteen [52 per cent] of thirty-one presentations). The lowest rates were in the hand subspecialty (twenty-eight [39 per cent] of seventy-two presentations) and the trauma subspecialty (eighty-two [39 per cent] of 210 presentations). However, a chi-square test revealed that any difference between groups was probably due to chance alone (p = 0.611) (Table II).
The American volume of The Journal of Bone and Joint Surgery published the most presentations: 161 (24 per cent) of 668. Clinical Orthopaedics and Related Research published the second highest number of presentations: 106 (16 per cent) of 668 (Table III).
Whereas the presentation of a paper at a national meeting is a well accepted and timely mode for conveying the conclusions of research, the publication of such data in a peer-reviewed journal represents a more permanent, more accessible, and validated culmination of the scientific process7,8,10,11. The ultimate rate of publication of 46 per cent for presentations from the Annual Meeting of The American Academy of Orthopaedic Surgeons is comparable with that for other scientific disciplines2-15. Scherer et al. recently conducted a large meta-analysis of the rates of publication after oral presentations described in eleven studies from different medical fields. They found that 50 per cent (1198) of the 2391 oral presentations regarding studies with at least two years of follow-up had been published. The highest rate of publication, 78 per cent (154 of 197 presentations), was noted for the oncology literature; however, this included publications in non-peer-reviewed journals3. The rate decreased to 58 per cent (115 of 197 presentations) when only peer-reviewed journals were included3. The rate of publication in peer-reviewed journals for different fields ranged from 32 per cent (122 of 379 presentations) as noted in one anesthesiology report13 to 66 per cent (sixty-one of ninety-three presentations) for ophthalmological presentations based on randomized clinical trials14 (Table IV).
The reasons why an oral presentation is not published are not always clear. However, when queried, authors have stated that a lack of time and the fact that pursuit of publication had a low priority were more common reasons than was rejection of a submitted paper3,4,6. The present study had several limitations. One is the potential that any Medline-type search might overlook a study when the authors or title has changed markedly. Also, although the purpose and spirit of the Annual Meeting of The American Academy of Orthopaedic Surgeons is to propel new and unpublished ideas, thirty of the 668 papers had been published before the meeting. Our search was limited primarily to papers published on or after the date of presentation; we searched back only one month before the meeting. Therefore, a small number of presentations could have been missed if they were published well before the meeting. Also, some papers presented at the Annual Meeting may have been published in journals not included in the Medline system. It is likely that even more papers were of publishable quality but were not published because of a lack of concordance between the authors during the final preparation of the manuscript or because publication was not pursued vigorously. In our study, no attempt was made to contact authors of presentations that were not published; therefore, we cannot comment on the precise reasons why presentations were not published. We chose to restrict our study to publications listed in the Medline Plus database (Index Medicus) of peer-reviewed journals. This methodology has been described previously and allows a comparison with the rates of publication in other disciplines7,8. An orthopaedist who misses the presentation of a paper at the Annual Meeting of The American Academy of Orthopaedic Surgeons has approximately a 50 per cent chance of retrieving that information from a peer-reviewed journal within the next three years. However, it is also true that approximately half of the material presented at the Annual Meetings will never be permanently recorded in the Medline-indexed scientific literature. The fact that nearly half of the papers presented at the Annual Meeting of The American Academy of Orthopaedic Surgeons are eventually published in peer-reviewed journals validates the program committee's selection process and the amount of time at the meeting allocated for presentation of original work. The fact that the rate of publication for papers presented at these meetings is mid-range compared with rates from other medical disciplines and remained identical for three consecutive years suggests also that there are sufficient journal resources to record permanently much of the new scientific data presented orally at the Annual Meeting. An extremely low rate of publication would imply that many abstracts are accepted for oral presentation (a low acceptance threshold for presentation) and that relatively few papers are accepted for publication (a high threshold for publication). Therefore, we believe that a low rate of publication would suggest, in some combination, that the meetings are too long or too frequent or that insufficient journal space is available for the number of abstracts accepted for oral presentation. An extremely high rate of publication has the opposite implications. It could mean that the over-all quality of scientific research in a particular field is very high. It might also imply that only a small number of the highest-quality abstracts are accepted for oral presentation (a high threshold for presentation) because relatively few presentations in that field were selected for the national meeting. In contrast, it could mean that the journals that accept publications in a particular field are not as discriminating with regard to the quality of papers that they accepted (a low threshold for publication). If the acceptance threshold for oral presentation is too low compared with the threshold for publication, then many conceptions and interpreted conclusions may circulate in the scientific community without eventual validation by publication in a peer-reviewed journal. If, however, the acceptance threshold for oral presentation is too high, then many promising new ideas, which may eventually come to fruition in the form of a published manuscript, will not be presented orally at a national meeting at their outset. Thus, ongoing research at other centers that may have been spawned from the original presentation of the new ideas might not occur until years later, when the manuscript is published. Reported rates of publication have varied widely among medical specialties2,3,6-14 (Table IV). These rates have varied not only from field to field but also from one meeting to the next within a specialty14. The reasons for these differences are not completely clear. One possibility is that there are inherent differences in the designs of the studies being presented at a given meeting. Meetings that include only basic work in molecular biology or only randomized clinical trials would seem to demand more time and more comprehensive data from authors who submit abstracts than do meetings that accept abstracts of any study design, including retrospective chart reviews. For instance, Scherer et al. reported a rate of publication of 66 per cent (sixty-one) of ninety-three ophthalmology abstracts presented at a meeting devoted strictly to randomized clinical trials. However, Chalmers et al. reported a rate of publication of only 36 per cent (sixty-four) of 176 presentations from perinatology meetings also limited to randomized clinical trials. The Uniform Requirements for Manuscripts Submitted to Biomedical Journals1, first published in 1979, strongly discourage the referencing of abstracts of oral presentations in scientific papers. Several authors have supported this policy because of the high percentage of oral presentations that never mature into peer-reviewed articles9-12,14. Papers presented at national meetings, and their abstracts, should primarily act as the catalyst for new ideas and interest, but abstracts of oral presentations should not be credited with scientific value equivalent to that of published papers that have survived the rigors of peer review. Abstracts of oral presentations should be cited in scientific writing only if it is explicitly noted in the text that they are unpublished data.
NOTE: The authors gratefully acknowledge the assistance of Frederick Dorey, Ph.D., for statistical analysis and preparation of the data.
*No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article. No funds were received in support of this study.
  CiteULike  Connotea  Del.icio.us  Facebook  Technorati  Twitter What's this?
This article has been cited by other articles:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top
Abstract
Introduction
| home | CME | help | feedback | subscriptions | archive | site index |
|
Stanford University Libraries' HighWire Press (TM) assists in the publication of JBJS Online. |