Annual Assessment of Doctoral Students

Effective Fall 2013 the Graduate College requires doctoral programs to conduct an annual assessment of their doctoral students.

An analysis of assessments in programs reveals some commonality mixed with program specific assessment items. It is clear that “one size does not fit all.” For example, in some disciplines (e.g., the STEM disciplines), doctorate students may only have an adviser after two (or more) years due to other requirements such as course work. Further, admission to doctorate candidacy (passing the Ph.D. Preliminary Exam) occurs at different times in different disciplines. For uniformity and the sake of good practice, the policy requires annual assessment of all doctoral students. In addition, care has been taken to ensure that the policy guards against potential conflicts of interest by requiring that the assessment of doctoral students contains an independent element.

POLICY¹

Programs must conduct annual academic progress reviews for all enrolled doctoral students at least once every academic year, beginning in the students’ first year. Students who are registered under a doctoral program code are considered doctoral students for the purpose of this policy.  Annual assessments are encouraged, though not required, for master’s students.

All annual assessments should include:

1. A student self-assessment of academic progress, and an opportunity for the student to provide evidence of his/her progress.
2. An assessment of the student’s academic progress from his/her doctorate adviser (if he/she has one).
3. A written review prepared by an individual or group different from the adviser (e.g., the Director of Graduate Studies (DGS), Department Head/Chair, or group of faculty) that will focus on the student’s degree progress, including completion of milestones and student strengths and weaknesses. A copy of the review, that includes items (1) and (2), must be provided to the student.
4. A timely opportunity for the student to discuss this review in person with the DGS (and adviser, when appropriate), if requested by the student. In the event that the student’s adviser is the DGS a suitable third party (e.g. the department chair/head or other senior professor) should lead the discussion.
5. An opportunity for written student feedback to the formal review.
6. All of the above are to be retained in the student’s academic file in the program/department.

The requirements listed above represent minimum actions, and programs may further develop reviews to incorporate additional items.

GUIDELINES

• The student’s self-assessment may consist of an updated list of publication, presentations, academic honors and fellowships, etc. The student may include academic goals for the upcoming year, and in the first year or two the self-assessment may mainly consist of plans and goals.
• In early years, typically before research has begun, the assessment meeting should focus on advising, bringing into the discussion the student’s preliminary research topics, so that specific background courses or prerequisites can be tailored to the individual student early enough so as not to delay research in later years (e.g. specific statistics courses that may be needed for the intended area of research, above what may be required; etc.).
• Although assessment in the early years has an advising focus, deficiencies or weaknesses that have been uncovered should be addressed so that the student can take corrective action.
• Students who have been assessed and are still not meeting required standards should be mentored out of the program within the first few years, unless the program is attempting to work on deficiencies and provide the student with specific requirements, with stated deadlines, for the student to correct weaknesses.

IMPLEMENTATION

The policy was effective for the Academic Year 2013-14, for all registered doctoral students.

SAMPLE ASSESSMENTS

Note:  These examples are meant as information and guidance, and may not, individually, contain all elements required of the UIC Graduate College policy.  This is true even for the examples from UIC programs.  In addition, the programs represented may have changed the forms/processes represented since they were collected.

• UIC Philosophy
• UIC Anthropology
• UIC Anatomy & Cell Biology
• UIC Business Administration
• UIC Sociology
• UIC Urban Planning & Policy

[Examples from other institutions, as well as UIC examples updated to reflect the new assessment policy will be added. Check back for updates.]

BACKGROUND

During the Spring Semester 2011, the Graduate College surveyed doctoral students and Directors of Graduate Studies (DGS) about current annual assessment practices in their programs. There were several motivations for conducting this survey, including continued issues related to the mentor/student relationship.  The results of a similar UIUC survey²showed a lack of agreement between students and DGSs as to whether certain programs were conducting annual reviews.  In a focus group the Graduate College held on assessment after the survey was completed, and in various recent discussions, students clearly and explicitly indicated a lack of clarity in regards to expectations within their programs, and a desire for early and annual assessments.

The following are results from the UIC survey and resultant focus group that informed a recommendation to institute an annual assessment policy:

• All 2,593 UIC doctoral students (based on Spring 2011 registration) were sent the link to the student survey. 41% at least partially completed the survey, and 33% completed the survey in its entirety. In response to the last question of the student survey, over 400 doctoral students wanted to participate in a focus group (which is over 50% of those who completed the survey).
• Due to scheduling, timing and other constraints, the Graduate College was able to conduct a focus group session on one date only. On April 20th, almost 50 students participated in a focus group session, despite being near the end of the semester.
• While 67% of DGSs indicated that their programs conducted annual assessments of their doctoral students, only 41% of students were certain that their program conducted such reviews.
• In some programs where the DGS indicated that there was a review process, the aggregate response of students indicated that there was not one.
• Many doctoral programs at UIC do not have, or the students are not aware of, annual assessments.
• Generally, in many programs, students responded that they do not know their status in the program in terms of satisfactory progress (it is assumed that no news is good news, etc.).
• For most programs with annual reviews, there was significant disagreement among students and DGSs about whether results were communicated to students. The only two programs where there was 100% agreement are also the two programs known to the Graduate College to have highly effective assessment procedures.
• Students who agreed that they were being reviewed were more likely to agree that they had a good understanding of program expectations.
• Students overwhelming want to know about expectations and want feedback on performance to allow for growth and to correct deficiencies. For the students in programs that do not currently have an annual assessment (i. e. students responded their program does not have an annual assessment), 74% of the students answered that an annual assessment would be beneficial.

BENEFITS

1.  From the University of Illinois at Urbana-Champaign Graduate College website:

• “Clarity of expectations” can reduce attrition and increase completion“Our findings suggest that the GEI (Graduate Education Initiative, a 10-year Mellon Foundation project) reduced attrition rates and improved graduation rates primarily through the routes of improving clarity of expectations and encouraging students to finish their dissertations as quickly as possible”—Ronald G. Ehrenberg, et al., “Inside the Black Box of Doctoral Education”“Research suggests that formal annual evaluations tend to increase the numbers of students who complete, and are therefore preferable to sporadic and informal reviews . . . Some form of progress-tracking, annually or each semester, allows the student and supervisor to meet and establish objectives for the year. This ensures that both students and advisors be held accountable for timely progress and for constructive feedback”—Council of Graduate Schools, “Ph.D. Completion and Attrition”
• Reviews may prevent more serious problems from developing later“Pre-empting conflict is possible only if we make the implicit explicit. We need to overcome information that is incorrect or not universally shared or understood; the lack of commonly shared and agreed-upon expectations; the lack of specific expectations between individual faculty and individual students regarding possible areas of conflict and their resolution”—John P. Beck, Michigan State University
• Reviews can help departments see their programs from the student’s perspective“It would be valuable for departments to regularly engage enrolled students in conversation about their goals and their experiences in the program, and about how the department and faculty might help students”—Chris M. Golde, “Should I Stay or Should I Go?”
• Regular reviews can be an indicator of program qualityThe new National Research Council assessment of research doctorate programs will feature a student survey of admitted-to-candidacy students in five disciplines (Chemical Engineering, Economics, English, Microbiology, and Physics). The first question under “Program Environment” asks “Does your program provide an annual or more frequent assessment of your progress?” and the second, “Do you receive timely feedback on your research?”2.  In the words of UIC students from the survey conducted by the Graduate College, the benefits of annual assessments are:
• “Helps me stay connected to my degree plan”
• “It motivates students since they know that the department cares about them and is aware of our progress. In that sense, we don´t feel isolated.”
• “Preparing the professional development plan helps me to evaluate my own progress and keeps my documentation (CV, etc.) updated.”
• “Provides an opportunity to handle criticism.”
• “Important milestones of my program are discussed during the review – the approximate timing of getting certain things accomplished.”

¹Appoved at the Graduate College Executive Committee meeting, January 27, 2012.
²The Graduate College at UIUC provided the survey to UIC and the Graduate College at UIC made minor modifications. UIC’s Graduate College would like to acknowledge and thank UIUC’s Graduate College for aiding with this policy and process.