Reporting quality of submissions to the National Conferences on Electronic Learning in Medical Education: implications from Iranian research performance

Reporting quality of research on medical education has come under scrutiny in recent years in wake of evidence obtained from relevant studies (1-4). Main concern has been raised is the insufficiency of provided details for unbiased judgment of the readers to critically appraise the scientific literature. Abstracts are windows into their corresponding scientific processes and must be written in ways that attract interested readers’ attention to learn more about the study and its results.  But poor reporting quality of published abstracts may distract readers from careful reading of research evidence or in a worst case may mislead scientists. Despite development and administration of standard research reporting guidelines (5-7) we are still witnessing suboptimal research reports in our academia (8, 9) internationally.  Main objective of this study was to evaluate the extent and quality of the submitted abstracts to the 3rd and 4th National Conference on Electronic Learning in Medical Education which were held on 17-18 February 2010 in Mashad and 15-17 February 2011 in Tabriz, Iran. Our special focus was on the accuracy of reporting quality as well as the validity of conclusions.

METHODS

A stratified random sample of abstracts (n= 188) representing quantitative and review (narrative and systematic) studies (excluding qualitative studies) were selected among a total of 366 accepted submissions to the 3rd (N= 181) and 4th (N= 185) National Conference on E-learning in Medical Education in Iran. We assessed the quality of the selected abstracts based on the criteria explained by Reed et al (9) to determine the quality of experimental, quasi-experimental, and observational studies. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (6) were also considered to assess abstracts from systematic review studies. External validity criteria were checked for abstracts from primary studies based on the reported sampling method, sample size and response rate to dispose probability of sampling bias and hence generalisibilty of a study finding to the wider population. Internal validity criteria were also verified based on the provided numeric results of the studied outcome variables with their precision to endorse reliability of the study results. Explanation of the main aim of study, name of databases searched, study selection criteria, quality assessment procedures in the retrieved studies, providing summary results and also giving conclusion were also criteria to assess the abstracts from review studies. Authors independently examined the abstracts for all these quality criteria and any disagreement was resolved by discussion until final consensus was reached.

RESULTS

Name of the statistical software to analyse data was given in the 47 abstracts (42.0%) from primary studies but in 5 abstracts (20.8%) it was given without explaining the statistical analysis method.

Amongst abstracts representing findings of narrative reviews (n=74) in 11 cases (14.9%) main aim of the review was not mentioned as indicated in Table 2.

 Among the abstracts from narrative reviews 12 abstract (17.9%) provided conclusion without explaining the study results. In the only one abstract which summarised findings of a systematic review name of databases searched, search strategy, studies selection criteria and their quality assessment procedures were not mentioned.

DISCUSSION

This study results revealed that 74.3% of abstracts from primary studies and 99.0% of abstracts from review studies lack all essential elements for informative reporting. Amongst the reviewed abstracts important information necessary to make judgment about their accuracy were absent frequently and this severely limits referees to critically appraise the submitted abstracts. Our study indicated varied reporting quality of submitted abstracts and the gaps wesuffer to have a more robust national research performance in the field of medical education. About 40% of submitted abstracts reported to introduce findings of narrative reviews but current debates (10-13) around their credibility in the hierarchy of research evidence merits revision in the current flow of decisions on submitted abstracts in the national conferences’ scientific committees. In the absence of internationally agreed standards to report study findings (14-18) reliance on less rigorous research reports can be embarrassing.  

To the best of our knowledge this is the first study in Iran which focuses on the quality issue of the submitted abstracts to the national medical education conferences. To understand the rationale behind the study procedures and conclusions in the reviewed abstracts it would be highly appealing for us to have access to the full text of corresponding studies, thereby to enhance our judgment about the quality of research evidence. A number of abstracts were written in a prospective way to indicate what will be done by author(s) in future. They should not be accepted as it is purely because their title is astonishing. Nonetheless, the title of a sizable number of abstracts were repetitive such as studies on learners satisfaction after applying blended educational methods in medical education instead of traditional teaching methods or  investigations on acquisition of knowledge and skills immediately after implementation of a educational programme on E-learning. Such a quality pattern among the submitted abstract to a scientific event is not unique for Iran and almost similar insufficiencies were also reported in other national and international conferences (19-22). We have not assessed the title of reviewed abstracts for multiple presentations but it has been shown that it is not unfamiliar in other countries (23-28). Overall quality of a research report will not be judged only upon its reporting quality but also based on its contribution to the scientific world. So to

improve our national research productivity we recommend working on infrastructural prerequisites needed for prevention of research misconducts in this field. Although we have selected a representative sample of abstracts submitted to only two national conferences and hence the findings may not be assumed to extend beyond these conferences and to all performed research in this field and also other medical fields, but these findings may reflect pitfalls in our research methodology educational efforts and embrace national level challenges we face to ensure evidence based research outcome. Such a conclusion is inline with the constraints were reported for countries across the whole Asian continent (29).

Due to the impact findings of a good quality research on different aspects of medical teaching-learning processes may have on the attainment of graduates from academic institutions and therefore on the enhancement of health care quality (30), our national research performance in the field of medical education merit further investigation and highly recommended.

ACKNOWLEDGEMENT

Authors would like to thank Iranian researchers who submitted their works to the 3rd and 4th National Conference on E.learning in Medical Education.

Conflict of interest: There are not any conflicts of interest regarding the submission and publication of the article and its potential implications. The authors agree to transfer the copyright of this article to the Future of Medical Education Journal and not to publish the manuscript elsewhere in any other language without the consent of the Journal.

Funding and support: This study was supported by Tabriz University of Medical Sciences.