Exploring Learners’ Viewpoint regarding Training Program of Health Care Providers Based on CIPP Model

Document Type: Original Article

Authors

1 MSc student, Department of Medical Education, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

2 Department of Medical Education, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

3 Department of Epidemiology & Biostatistics, School of Public Health, Sabzevar University of Medical Sciences, Sabzevar, Iran

Abstract

Background: Aiming to explore learners’ viewpoint regarding training program of health care providers based on CIPP model, this research was conducted on health care providers in the city of Sabzevar in 2017.
Methods: This cross-sectional study was carried out on 139 health care providers working in the health centers of Sabzevar. The data were collected through a researcher-made questionnaire containing demographic characteristics and evaluation questions of training program of health care providers in four domains: context, input, process, and output.  The three modes of undesirable, slightly desirable and desirable were measured in each domain. 
Results: Through investigating learners’ viewpoint, the results of the research indicated that the situation of context and output of training program of health care providers were highly desirable, while the situation of input and process of the program were somewhat desirable. Overall, the results showed that the highest score among the four factors of the CIPP model belonged to the context factor with the mean score of 39.46 + 8.53 and the product factor with the mean score of 28.61 + 7.36 reflected the lowest score.
Conclusion: The final results of the exploring learners’ viewpoint of training program of health care providers indicate a range of slightly desirable to desirable situation for the program. To achieve a highly desirable level, it is incumbent upon the officials to reinforce the important factors such as teaching and assessment methods to their instructors. In addition, creating a definite plan for costs and developing the accessibility of welfare facilities seem to be crucial. Finally, it is worth planning more accurately in order to increase learners’ contribution while undertaking ongoing evaluation to receive feedback for revising the teaching program.

Keywords

Main Subjects


Introduction

The nations’ welfare and their progress are measured by their access to some utilities and servicing systems. In this regard, the healthcare system is considered one of the main factors of such measurement (1). The ideal society depicted in the vision of the year 2025 is a society with the highest health levels. Based on the Iranian government's commitments and the Ministry of Health and Medical Education, the Health Deputy in its offering plan to the parliament, entitled generating a fundamental amendment in the health system, established the health amendment initiative which relies on presenting the basic-service health package by the health care providers. A health care provider is a person with an associate’s degree or bachelor’s degree in the fields of family health, public health, nursing, midwifery and disease control (in the case of men) who is turned into a multidisciplinary individual called a "health care provider " after passing the defined training program successfully (2).

The training program of health care providers includes 147 hours of verbal training organized by the Health Deputy of the University of Medical Sciences (3). In the training schedule, one of the most important issues is quality, which entails the true identification of its strengths and weaknesses, so that, by relying on it, making the correct use of opportunities and the improvement of organizational performance would be achieved (4). This is reached by establishing an efficient system to evaluate the impact of the training program (5), which enables the managers to supervise and monitor the processes in order to identify and recover the challenges (6).

To examine the impact of training programs, different evaluation models have been proposed. Worthen and Sanders referred to more than 50 models. Given the subject and the statistical population of the study, the CIPP model was chosen to create a comprehensive framework for the evaluation of the training program performance and to provide accurate and reliable information for managers to make strategic decisions (7). CIPP is an acronym, derived from the first letter of the words Context, Input, Process, and Product (8). Reviewing the research background shows that training centers have always considered the evaluation problem as a challenging issue. For instance, the results of the study conducted by Scheerens and Bosker indicated that the CIPP model, due to its capability to evaluate the performance of training centers from different dimensions, helped the components of the structure set aside each other with a particular harmony, so that, using its consequence, proper programs could be prepared and delivered (9). Agrawel recommended the CIPP model because of generating a significant relationship between the improvement of processes and the effectiveness of the trainings. He believed that this model can help the educational managers turn human resources into a strategic capital (10).

For probing into the impact of training programs using the CIPP model, studies have been conducted in Iran. For example, Tezakori et al., evaluating the PhD program of nursing in Iran using the CIPP model, pointed out that the program’s specification and the content of the courses is in accordance with the philosophy and objectives of the nursing field, but major problems lie within the process and its implementation, and the domain of product is influenced by poorly running of the process (11).

A study aiming to evaluate the Family Medicine Scheme in Marivan by Izadi et al. showed that the domain of context, process and product of the scheme is slightly desirable (12). The results of the study by Mahram et al. in evaluating the achievement rate of undergraduate students of nursing to educational objectives indicated that 63 percent of students and 80 percent of matrons considered the achievement rate (product) slightly desirable (13). The results of the research conducted by Alimohammadi et al. (8) in evaluating the Rafsanjan Faculty of Medicine revealed the desirability of the context, input, and process; yet the product of the school was measured slightly desirable. Zandvanian (14) in the evaluation of teacher-training centers of Khuzestan province reported that the context, input and process of these centers are slightly desirable whereas the product reflects a highly desirable status.

Training health care providers at Health Training Center in the Health Deputy of Sabzevar University of Medical Sciences began at 2016, while health care providers offered services in the health centers. According to monitoring by health experts and health trainers on the services offered by health care providers, the current study was designed to explore learners’ viewpoint of training program of health care providers, given the results of monitoring and perceiving the need, which is emerging from need assessments, so that the results can propose an appropriate model for improving the training programs and increasing the efficiency of health care providers in offering services to the target groups.

 Methods

This cross-sectional study was conducted as a survey in the Health Deputy of Sabzevar University of Medical Sciences. The statistical population included all health care providers working in the health centers of Sabzevar who attended the training programs of health care providers during the years 2016 and 2017. 139 (out of 146) individuals contributed to the study. Due to the limited community in this research, the whole community was explored via census.

To collect the fieldwork data of this study, a researcher-made questionnaire consisting two parts was applied.  The first part of the questionnaire consisted of the personal details of health care providers including gender, age, marital status, and period of service. Its second part consisted of 47 closed-ended questions, which evaluated the viewpoint of health care providers about the domains of context, input, process, and output of the program using a five-degree likert scale from very high to very low with a maximum score of 4 and a minimum score of 0. This questionnaire was extracted from a checklist designed by Stufflebeam in 2002 for the evaluation of training programs according to the CIPP model and the conducted studies in Iran. In this questionnaire, the environmental conditions of learners are considered in its context. Educational content and facilities and educational objectives are considered as its input. The training program (the capability of the program in providing course goals), its suitability to the needs of learners (creating an acceptable scientific, skillful, and attitudinal base) and the performance of education are determined as process. Finally, the rate of achieving goals of the health care providers’ program in fulfilling their professional tasks is understood as output.

To calculate the validity of the questionnaire, the face and content validity were taken into account. Face validity of the questionnaire was investigated based on writing, wording, and the logical and exciting appearance of the questionnaire as well as the experts’ opinion in the stage of judgment. In order to investigate the significance of each term, item impact method was considered. In this regard, the comments of eight faculty members and senior experts of the university were collected and considered. To determine the reliability of the questionnaire, Cronbach's alpha coefficient was employed, where the amount of this coefficient was calculated 0.90, which indicates a good reliability for the questionnaire used in this research. Following that, data were analyzed using descriptive statistics, mean, percentage frequency, standard deviation, and software Stata version 14.

Results

According to the results of this study, the demographic characteristics of the samples including gender, marital status, degree and major, period of service, and age were observed. Of 139 health care providers participating in the study, 86 percent were women with an average age of 35 years, 82 percent were married, and the majority had an associate’s degree or bachelor's degree in the field of public health. (Tables 1 and 2).

Table 1. Frequency distribution of demographic characteristics of research samples

Variable

Number (%)

Gender

Female

120 (86.33)

Male

19 (13.67)

Marital Status

Married

115 (82.73)

Single

24 (17.27)

 

Qualification

Diploma

2 (1.44)

Associate’s degree

63 (45.32)

Bachelor’s degree

70 (50.36)

Master’s degree or higher

4 (2.88)

 

Field of Study

Midwifery and nursing

40 (28.78)

Public health (Specialization: family health)

79 (56.83)

Public health (Specialization: disease control)

20 (14.39)

Table 2. Frequency distribution of age and period of service among research samples

Variable

Minimum

Maximum

Mean(SD)

Age

22

56

35.64 (6.66)

Period of service (month)

3

360

110.51 (8.56)

Furthermore, the findings of this study were evaluated based on four main factors of the CIPP model, named context, input, process, and product (Tables 3, 4, 5, and 6). In these tables, 47 indices have been scrutinized. In the context of the program 13 items, in the input 12 items, in the process 12 items, and in the output 10 items using descriptive statistical tests were evaluated in three desirable, relatively favorable, and undesirable levels.

 For the context of the program, the relevance of context of the training program to the health care providers’ job requirements was the highest level of desirability with 45.32 percent, whereas the health care providers' reluctance to continue their training courses in its current form was measured 43.88 percent, which was the lowest level of desirability (Table 3). In terms of the input of the program, the satisfaction of health care providers with the method of training was the highest level of desirability with 42.19 percent, while the dissatisfaction of health care providers with the level of servicing was considered the lowest, which was 51.08 percent (Table 4). Health care providers’ needs to pass the training programs was measured by the domain of process, with 63.31 percent as the highest rate of desirability, and health care providers’ participation in the training program with the aim of promotion and reward was evaluated 64.75 percent as the lowest rate of desirability (Table 5). Ultimately, in terms of the product of the program, leading the implementation of training programs of health care providers to make the staff active and vibrant in the workplace was measured 32.37 percent as the highest level of desirability. By contrast, the lack of research and study into the effects and consequences of the plan on the health development before starting the training program was 39.57 percent which was the lowest level of desirability from learners' point of view (Table 6).

Table 3. Indices of context for the training program of health care providers

Context evaluation

Undesirable

Slightly Desirable

Desirable

Number (%)

Number (%)

Number (%)

Considering a mission in planning the training programs of health care providers

24(27.17)

64(46.04)

51(36.69)

Considering objectives in planning the training programs of health care providers

16(11.51)

61(43.88)

62(44.60)

Creating equal opportunity approach in planning the training programs of health care providers

35(25.18)

60(43.17)

44(31.65)

Providing required conditions for evaluating content in the planning of training programs

38(27.34)

59(42.45)

42(30.22)

Matching the content of the training programs held with global situations, changes, and new technologies

45(32.37)

62(44.60)

32(23.02)

Desirability of subjects taught in the training programs

42(30.22)

55(39.57)

42(30.22)

Relevance of content of the training programs to the health care providers’ job requirements

25(17.99)

51(36.69)

63(45.32)

Considering local and regional health needs in planning the training programs of health care providers

45(32.37)

68(48.92)

26(18.71)

fulfilling the objectives set by the organizers of the training programs

42(30.22)

67(48.20)

30(21.58)

tendency to continue the training programs of health care providers in the current form

61(43.88)

41(29.50)

37(26.62)

Rate of failure at work in case of absence in the training programs

24(27.17)

54(38.85)

61(43.88)

Usability and transferability of knowledge and attitude or skills of the training programs to other situations

27(19.42)

58(41.73)

54(38.85)

The amount of difficulty in achieving the objectives and development of the organization in case of absence in the training programs

41(29.50)

45(32.37)

53(38.13)

 

Table 4. Indices of input for the training program of health care providers

Input evaluation

Undesirable

Slightly Desirable

Desirable

Number (%)

Number (%)

Number (%)

Rate of meeting the expectations for the implementation of the training program of health care providers

45(32.37)

69(49.64)

25(17.99)

The implementation of the training program as anticipated

54(38.85)

49(35.25)

36(25.90)

Barriers and limitations in organizing and implementing the training program of health care providers

27(19.42)

53(38.13)

59(42.45)

Deficiency and failure in organizing and implementing the training program of health care providers

24(27.17)

49(35.25)

66(47.48)

Level of satisfaction of health care providers with the implementation of the training program of health care providers

55(39.57)

57(41.01)

27(42.19)

Rate of consistency between teaching methods of instructors with the context and objectives of the training programs

27(19.42)

68(48.92)

44(31.65)

Suitability of required facilities for teaching topics and presenting the context of the program

49(35.25)

71(51.08)

19(13.67)

Level of satisfaction with facilities and equipment of the training program of health care providers

62(44.60)

55(39.57)

22(15.83)

Level of satisfaction with facilities and equipment of the one’s workplace

67(48.20)

50(35.97)

22(15.83)

Level of access to the required service packages

38(27.34)

56(40.29)

45(32.37)

Level of satisfaction with the transportation facilities to the training venue

53(38.13)

47(33.81)

39(28.06)

Level of satisfaction with the reception status at the time of the training program

71(51.08)

54(38.85)

14(10.07)

 

Table 5. Indices of process for the training program of health care providers

Process evaluation

Undesirable

Slightly Desirable

Desirable

Number (%)

Number (%)

Number (%)

Suitability of financial facilities and budgeting for the implementation of the training program of health care providers

66(47.48)

56(40.29)

17(12.23)

Clarity of policies and strategies for achieving the objectives of the health system development by program organizers

35(25.18)

71(51.08)

33(23.74)

Suitability of time for the implementation of the training program of health care providers

51(36.69)

55(39.57)

33(23.74)

Determining and declaring the necessary activities for organizing the programs as well as presenting an agenda

30(21.58)

67(48.20)

42(30.22)

Required rate for health care providers to attend the training programs

17(12.23)

34(24.46)

88(63.31)

Level of welcome and motivation by the staff to attend the training programs of health care providers

43(30.94)

54(38.85)

42(30.22)

Having the aim of promotion or reward to attend the training programs of health care providers

90(64.75)

36(25.90)

13(9.35)

Level of satisfaction with the method and the implementation process of the training programs of health care providers

51(36.69)

61(43.88)

27(19.42)

Level of interaction between official experts and health care providers

47(33.81)

53(38.13)

39(28.06)

Level of satisfaction with the planning of the training programs of health care providers

49(35.25)

59(42.45)

31(22.30)

Level of satisfaction with the teaching-learning strategies of the training programs

45(32.37)

60(43.17)

34(24.46)

Level of satisfaction with the amount and method of counseling offered by instructors

37(26.62)

56(40.29)

46(33.09)

 

Table 6. Indices of product for the training program of health care providers

Product evaluation

Undesirable

Slightly Desirable

Desirable

Number (%)

Number (%)

Number (%)

Increasing creativity in the workplace by attending the training programs

48(34.53)

58(41.73)

33(23.74)

Level of achievement of policymakers and planners of the programs to the objectives set by the health system

45(32.37)

62(44.60)

32(23.02)

Rate of evaluation of the positive and negative effects  of the training programs of health care providers

37(26.62)

65(46.76)

37(26.62)

Precise and operational definition of the products and efficiency of the training programs

37(26.62)

79(56.83)

23(16.55)

Determining valid and scientific criteria for measuring the efficiency of training health care providers

42(30.22)

65(46.76)

32(23.02)

Desirability of efficiency for activities and training programs of health care providers

39(28.06)

68(48.92)

32(23.02)

Measuring and evaluating the product of the training programs of health care providers to date

40(28.78)

59(42.45)

40(28.78)

Researching the effects and consequences of the scheme before organizing the training programs of health care providers

55(39.57)

58(41.73)

26(18.71)

Level of assisting the health facilitation in the training programs

39(28.06)

56(40.29)

44(31.65)

Making the staff active and vibrant in the workplace after attending the training program

47(33.81)

47(33.81)

45(32.37)

 

From health care providers’ viewpoint, the highest mean score belongs to context (39.46±8.53) and the lowest mean score (28.61±7.36) is related to product (Table 7). Context with 51.80 per cent has the highest level of desirability, while input with 28.78 per cent has the lowest level of desirability. (Table8).

Table 7. Mean, standard deviation, minimum and maximum of domains from health care providers’ viewpoint in Sabzevar

Domain

Mean(SD)

Minimum

Maximum

Context

39.46(8.53)

16

60

Input

33.82(5.85)

19

52

Process

34.30(7.78)

14

55

Product

28.61(7.36)

10

42

Table 8. Learners' viewpoint about the domains of the training programs of health care providers in Sabzevar

Domain

Undesirable

Slightly Desirable

Desirable

 

Number (%)

Number (%)

Number (%)

Context

12(8.63)

55(39.57)

72(51.80)

 

Input

8(5.76)

91(65.47)

40(28.78)

 

Process

15(10.79)

74(53.24)

50(35.97)

 

Product

22(15.83)

56(40.29)

61(43.88)

 

 

Discussion                                                                                                                                          

In this study, four domains of context, input, process and output training program of health care providers of Sabzevar University of Medical Sciences in the educational year 95-96 was evaluated from learners’ point of view. The results of this study showed that in general, from learners’ viewpoint, the educational context created in the training program of health care providers has been desirable. This finding was consistent with the results of studies undertaken by Rezapour et al. (15), Alimohammadi et al. (8), Tezakori et al (11), Akpur et al. (16), and Ortiz et al. (17). Evaluating the training context is carried out to provide a logical context for the determination of educational objectives and includes analytical activities to identify elements in the educational environment in order to identify the problems, needs, and available opportunities as well as to design the programs (8). Regarding this issue, the Health Training Center at Sabzevar University of Medical Sciences has been trying to identify the educational needs of health care providers for the commencement of the training programs to provide the necessary means. For instance, the Health Training Center, aiding from the Health Deputy of Sabzevar University of Medical Sciences, prepared, disseminated, and installed posters to publicize the goals and mission of health care providers in the health centers of Sabzevar. Furthermore, conducting the need assessment in the health centers across the city at the beginning of each year aiding from the training unit of the Health Deputy of Sabzevar University of Medical Sciences, local and regional needs can be identified and considered in the planning of verbal and virtual training programs of health care providers.

The analysis of the results of this research in the domain of input showed that the health care providers' viewpoint about the training inputs indicates their relative desirability, which is consistent with the results of the studies by Alimohammadi et al. (8), Saadat et al. (18), Riahi et al. (4), and Izadi et al. (12). Health care providers were not satisfied with the facilities and equipment available at the workplace and during the training program. It is worth mentioning that by purchasing and supplying the computers needed in the centers, the problem of computer shortages was largely overcome; nonetheless, upgrading other equipment in the centers should be surely taken into account.

The results of the study done by Rezapour et al. (15) indicated the desirability of input in the training program of Ardakan University. They explained this finding as the effectiveness of the programs and visions of the university to fulfill the educational goals. However, in their study, students were rather satisfied with the facilities and equipment, yet wanted their upgrade, which in turn was consistent with the results of the current study.

The third element of the CIPP model that should be considered in evaluation is the curriculum procedures. Teaching-learning procedures are the most technical component of the curriculum, since they mediate the implementation of context and objectives. Therefore, their evaluation is of the utmost importance. According to the results of this research, the process of the program reflects a slightly desirable level, which is consistent with results of the studies done by Saadat et al. (18), Riahi et al. (4), and Mohebbi et al. (19). Using modern teaching methods such as e-learning could yield a growing trend in the rise of interaction of health care providers with instructors and official experts, as well as bringing more attention to the role of learners in the process of learning.

Finally, the analysis of the results of this study demonstrates the desirable level for product of the program; however, comparing the domains, it proves that the product of the program has the lowest mean score of all. This finding was consistent with the results of the studies conducted by Alimohammadi et al. (8), Riahi et al. (4), Izadi et al. (12) and Saadat et al. (18). The most important products of a training program are its learners whose competencies can show the success of the program. Therefore, it is suggested that learners' status, in terms of different competencies, becomes constantly evaluated after the completion of the program and within different periods of time. To do this, the establishment of a specific system inside a training program seems essential for constant communication with learners in addition to following their status using accurate and up-to-date information. This task has been made somewhat possible in the health care provider scheme by monitoring the care delivered by health care providers via the SIB system, its verification by official experts, and evaluation of the client satisfaction through SMS. Using this experience can help the design and implementation of the new training programs or alteration and revision of the current programs.

At the end, it should be stated that in general, the impact of the program is related to a particular set or case and differs from other studies. Each training program has its own characteristics, thus it is not possible to compare and discuss the case in a common fashion in other studies (20).

This research has faced with some limitations that should be taken into account when generalizing the results. The first limitation was the lack of consistency between trained health care providers, some of whom were newly recruited while some had several years of experience. This problem brought limitations for examining the impact of the training program, particularly its product. Another limitation is that the data emerged from this study only reflect the views of the health care providers working in the urban centers of Sabzevar University of Medical Sciences. Consequently, the results should be interpreted and generalized considering this limitation.

Findings of this research reveals that the program was slightly desirable in terms of the input and the process, while the domains of context and product reflect a desirable level. Given the fact that officials of this scheme request educating experienced and efficient health care providers in the field of health services, it is necessary to motivate health care providers to gain knowledge, offer facilities and educational equipment, prepare and distribute new and different service packages in the centers offering service, revise the training program of health care providers to fulfill the objectives of the health care provider scheme.

Ethical considerations 

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.

Acknowledgments

The authors would like to thank the Research Deputy of Mashhad University of Medical Sciences and also the Health Deputy of Sabzevar University of Medical Sciences for their valuable contributions.

Financial Support:

This paper is extracted from a student dissertation for a Master's degree in community-oriented medical education in the health system (code: 961065) of Mashhad University of Medical Sciences, which has been approved by the Ethical Committee (IR.MUMS.fm.REC.1397.31) and is sponsored by the Research Deputy of Mashhad University of Medical Sciences.

Conflict of interest

The authors declare that there is no conflict of interest.

1.            Ebadi Fard Azar F, Haghani H, Hashemi F. Monitoring the national plan family physician and referral system in the city of Robat Karim hospital. Journal of Hospital 2009;8(5):7-12. Persian.

2.            Sayari AA. Program execution instructions on marginal areas of cities and towns over 20 thousand. Ministary of Health and treatment guidelines. 2017;4:1-3. Persian.

3.            Jafari AG, Mohammadpur F, Alam Tabriz A. Effective strategies for empowerment of experts staff in health deputy of Shahrekord University of Medical Sciences, Iran, 2005. Journal of Shahrekord University of Medical Sciences 2008;10 (2): 66-71. Persian.

4.            Riahi F, Mohajer Y, Abbaspour A. Assessment of educational process of payam noor university in Behshahr with emphasis on CIPP model. [Msc thesis]. Behshahr: Payam Noor University; 2012. Persian.

5.            Yarmohammadi H. Fundamentals and principles of curriculum planning. Tehran book memorial. 2014;13:174. Persian.

6.            Hadavand S, Rahimi R, Darabi M. Validity Measurement of Cipp Assessment Model in Assessing the Instruction Centers Function: A Case Study on National Iranian Petrochemical Company. Research on Educational Leadership and Management 2014;1(1):119-36. Persian. 

7.            Nikkhah M, Sharif M, Nasr A, TALEBI H. A feasibility study of using evaluation indicators of higher education in evaluation of the post-graduate curriculum, based on the CIPP-model. Biennial Journal of Management and Planning in Educational Systems 2012; 4 (7): 100-132. Persian.

8.            Alimohammadi T, Rezaeian M, Bakhshi H, VaziriNejad R. The evaluation of the Medical School Faculty of Rafsanjan University of Medical Sciences based on the CIPP model in 2010. Journal of Rafsanjan University of Medical Sciences 2013;12(3):205-18. Persian.

9.            Scheerens J, Bosker R. The foundations of educational effectiveness. Oxford: Pergamon; 1997.

10.          Agrawel S. Competency based CIPP model: An integrative perspective. Indian Journal of Industrial Relations. 2008;8(3):8-12.

11.          Tezakori Z, Mazaheri E, Namnebat M, Torabizadeh K, Fathi S, Ebrahimibil F. The Evaluation of PhD Nursing in Iran (Using the CIPP Evaluation Model). J Faculty Nursing & Midwifery. 2010;12:44-51. Persian.

12.          Izadi F, Delawar A, Mohajer Y. Evaluation of Marivan Family Physician Plan through Evaluation Model CIPP. [Msc thesis]. Tehran: Allameh Tabatabaei University; 2011. Persian.

13.          Mahram B, Vahidi M, Mojali M, Hosseini Shaheidi L. Achievement to the educational goals of undergraduate nursing students from view point of students and head nurses. Journal of Nursing Education. 2013;1(2):29-35. Persian.

14.          Zandavanian NA. A comprehensive evaluation of Khuzestan province teacher training centers, folowing CIPP model. Journal of Educational Sciences and Scycology 2006; 13(2)137-62. Persian.

15.          Rezapour Mirsaleh Y, Atri Ardakani SA, Behjati Ardakani F. Evaluating the educational performance of Ardakan University based on CIPP model. Higher Education Letter 2017;9(36):7-30. Persian.

16.          Akpur U, Alci B, Karatas H. Evaluation of the Curriculum of English Preparatory Classes at Yildiz Technical University Using CIPP Model. Educational Research and Reviews 2016;11(7):466-73.

17.          Ortiz Q, Isabel D, Muñoz Marín JH, Sierra Ospina N. CIPP Model-based Ccntext evaluation on an English language reading comprehension course at a Public University in Colombia. Íkala, Revista de Lenguaje y Cultura. 2015;20(3):293-314.

18.          Saadat M, Mansoorian MR, Basiri M, Khosravan S. The evaluation of bachelor of nursing’s educational program in Gonabad University of Medical Sciences based on the CIPP model in 2015. [Msc Thesis]. Gonabad: Faculty of Nursing; 2016. Persian.

19.          Mohebbi N, Yarmohammadian MH. Develop evaluation indicators of Health Information Technology Course at Master's degree in selected Medical Sciences Universities. Health Information Management 2013;10(4(serial No 32)):558-70. Persian.

20.          Seyf AA. Educational assessment, evaluation and measurement. Tehran: Dowran publication. 2003:392-401. Persian.