Developing Rubrics
Developing Rubrics
Developing Rubrics
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 2
Using Rubrics to Assess Student Learning Outcomes
at the Program Level
What are Rubrics?
A rubric is a scoring guide with criteria for evaluating students’ work in direct relation to one or more of the
program’s learning outcomes and a rating scale indicating differing levels of performance.
Rubrics are:
• Used to examine how well students have met learning outcomes rather than
how well they perform compared to their peers.
• Typically include specific, observable, and measurable descriptors that
define expectations at each level of performance for each criterion.
They can be used to assess student performance in course assignments, exams, practica and internships, research
papers, portfolios, group projects, public presentations, and many other types of work. The overall scores and any
subscales developed to measure more specific elements of performance are easily aggregated for further analysis at
the program level.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 3
• Checklists are dichotomous (e.g. Yes/No; Met/Not Met) and easy to use; however, they may not
provide substantive information regarding the performance levels.
• Rating scales identify a range of performance without specific descriptors for each performance level
(e.g. Exceeds Expectation/Meets Expectation/Below Expectation). They may not provide sufficient
information for raters to score consistently and the feedback to students may not be specific enough for
revision or future growth.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 4
• Analytic scales articulate specific levels of performance for each individual criterion. Scoring using
analytic scales is typically more consistent and specific areas of growth can be identified. The analytic scales
take more time to develop.
Challenges –
• Take time and effort to develop
• Need to include descriptions of specific criteria, and preferably exemplars of student work to
benchmark different levels of performance
• May be a less precise measure because of the broad range of performance within each specified level
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 5
How is a Rubric Created and Used in Assessing Program-Level Student
Learning Outcomes?
1. First, review each of the program’s student learning outcomes. How to Create and Use a Rubric
Where in the program do students have opportunities to Review SLOs
demonstrate that they have achieved each of the outcomes? Identify Student Work
Capstone courses, senior or upper level courses, research or Establish Criteria
internship experiences, comprehensive exams, dissertation
defenses, etc.? Identify Scale
2. Once you have mapped the outcomes to the courses, exams, Determine Range and
Target
Conduct Pilot
and other activities within the program, what specific
assignments or means of demonstrating skill can serve as a Develop Sampling Plan
source of student work that can be assessed in relation to the Aggregate Scores
learning outcome?
3. Establish the criteria. What are the performance dimensions associated with the learning outcome? What are
the critical components of the student performance that you need to capture as evidence of learning when
assessing the work?
4. Identify the scale. What is the appropriate scale for measuring each student’s performance on these
dimensions?
• Holistic scales – provide an overall evaluation; appropriate for assessment that does not require specific
feedback;
• Checklist – appropriate for assessment criteria that can be addressed using a dichotomous scale
(e.g. Yes/No);
• Rating scales – provide feedback on the performance level; appropriate for assessment that does not
require specific description of each performance level;
• Analytic scales – provides detailed description of each performance level.
5. Determine the range of performance levels and the program’s target. The target is the average performance
or percentage of students who achieved a certain score target that the program aspires to or considers to be
a minimum threshold for success in achieving the learning outcome.
• The number of performance levels may vary. Many people start with a 3-point scale (e.g. Exceeds
Expectation/Meet Expectation/Below Expectation), 4-point scale (e.g. Outstanding/Good/ Average/Poor),
or 5-point scale (e.g. Advanced/Proficient/Developing/Emerging/Beginning)
7. Develop your sampling plan for selecting work to be assessed with the rubric.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 6
8. Aggregate the rating scores across the entire sample. Compare the results to the program’s target for
performance on that learning outcome.
After implementing the rubric, continue to review the findings. In addition to proving a consistent method for
assessing student work, rubrics can identify opportunities for program improvement. Trends uncovered through
aggregated rubric scores can determine areas where students need additional instruction or support, as well as
inform changes in the curriculum or how content is taught.
Question: Do rubrics have to be created and used in assessing all student work to measure achievement of
expected learning outcomes?
Answer: No. There are a number of different methods of assessing program-level student learning
outcomes. However, there are advantages to using rubrics, including greater reliability and consistency in
rating performance across the program, the ease with which scores can be aggregated across the program
to do more sophisticated analysis, and the ability to examine different dimensions of performance as
opposed to just an overall grade or score.
Question: Why can’t we just use average course grades to assess student achievement of learning outcomes?
Answer: Accreditors do not recommend using course grades to assess learning outcomes. They argue that
course grades, especially at the undergraduate level, often include class participation, attendance, and other
behavior not directly related to the learning outcomes. Further, average final grades provide no insights into
relative strengths and weaknesses of students across the various components of the outcome. Carnegie
Mellon University’s website offers a useful description of the differences between course grades and
assessment of program -level learning outcomes. Instead, assess a specific assignment within one or more courses
that maps to the learning outcome of interest.
Question: Is there ever a situation in which we can use average course grades as evidence of achieving program-
level learning outcomes?
Answer: This may be possible at the graduate level if you can document that that 100% of the course grade
was based on the quality of a paper, research project, or final exam that directly measured the learning
outcome. But if a graduate program has a comprehensive or qualifying exam or dissertation proposal that
measures the knowledge covered in the required core courses, why not just use the performances on those
milestones as evidence of achievement of the outcome? Reporting on course grades would be redundant.
Question: We mapped our undergraduate level student learning outcome for research skills to the final paper in
our capstone course, which reports the procedures and results of their semester-long project. The course
instructor graded the papers measuring competency in each of the stages of the research process. Do we need to
have additional faculty independently re-evaluate the papers using a formal rubric?
Answer: No. When there is such a close fit between the learning outcome and the work being used to
assess it, the scores or ratings originally assigned by the instructor can probably be used as evidence as to
whether students are achieving the outcome. Most of the time, no one is more qualified to evaluate the
work than the person who taught the course. For the department’s annual Student Learning Outcomes
Assessment Report, the instructor could prepare a brief report for the DUS or undergraduate studies
committee that includes a description of the assignment, a copy of the rubric or grading scheme used, the
results of the analysis of student performance, observations about strengths and weaknesses, and
recommendations for improving student learning related to the research skills outcome.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 7
Question: To assess our learning outcome related to critical thinking skills, we plan to sample final papers from a set
of senior-level courses in our major and have them rated by two faculty members with a rubric designed to measure
critical thinking. Some faculty have criticized this plan, saying that it is a poor use of time to re-grade student work
and that averaging final course grades across the set of courses should provide sufficient data for this assessment
exercise. In addition, concerns were expressed that having other faculty not associated with the course re-grade
another faculty member’s papers sends a message of distrust and raises questions about their academic freedom to
make judgments about student performance in their own courses. How do I explain this?
Answer: The papers will be used for a secondary analysis to assess whether students are achieving a single
program-level outcome. This is not the same as having the papers completely re-graded by independent
raters using the same criteria as the original instructor. The instructors no doubt evaluated the papers with
additional expectations unique to their particular courses, such as specific subject matter knowledge. They
might have taken critical thinking skills into account when grading the papers, but unless common criteria for
this competency were used across instructors and assignments, aggregating the course grades will not
provide very reliable information about whether seniors have achieved this specific skill.
In addition, it is important for all involved to understand that the assessment of learning outcomes is
student-focused; it is not used to evaluate instructors. And, there is no reason to exclude the original course
instructors from rating the papers for evidence of critical thinking skills, although it is a good idea have them
rate papers that they have not already read for another purpose.
In selecting the courses and papers for use in this exercise, the raters will need to ensure that the different
assignments provided comparable opportunities for students to demonstrate critical thinking skills. Raters
should meet to reach a common understanding of the rubric and how it is to be applied, and to test the
consistency of their ratings.
Examples of Rubrics
Examples of rubrics used by other institutions to evaluate different kinds of student work are provided in the
Appendix. They can be used as models, or you can copy and adapt them for your own purposes. However, please
be sure to acknowledge the source (listed under each matrix) of each instrument if you distribute it.
Another great source for rubrics is the American Association of Colleges and Universities’ (AAC&U) Valid Assessment
of Learning in Undergraduate Education (VALUE) project. This set of 16 rubrics includes the following that are
relevant for measuring a number of learning outcomes across majors:
The VALUE rubrics, which are being used by hundreds of institutions, are free and can be downloaded from AAC&U’s
website: http://www.aacu.org/value/rubrics .
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 8
Contact Information
For more information about assessment resources, including rubrics, please go to:
http://oira.unc.edu/institutional-effectiveness/assessment-resources/
Please feel free to contact the Office of Institutional Research & Assessment for help:
Dr. Bryant Hutson, Interim Director of Assessment
[email protected]
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 9
Appendix A
Sample Rubrics | Papers
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 10
Rubric for a Research Paper in the Humanities
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 11
Rubric for a Paper in the Social and Behavioral Sciences
Exemplary Accomplished Developing Beginning Unacceptable
Focus • Relevant research is • Relevant research is • Paper addresses the relevant • Paper does not adequately • Relevant research not
thoroughly and completely adequately addressed. research generally satisfactorily, address the relevant discussed.
discussed. • Most of the key concepts though explanations and research. • Key concepts either not
• Key concepts are are identified and elaborations may be imprecise. • Very few key concepts are identified or are identified
identified and operationally defined. • Most of the key concepts are identified and clearly in a confusing manner.
operationally defined. • Contains some irrelevant identified and operationally defined. • Contains irrelevant
• Objective stance information but does not defined, though the definitions may • Paper contains too much information.
maintained throughout detract from focus. be imprecise. irrelevant information that • Very frequent subjective
document. • Objective stance mostly • Contains some irrelevant seriously detracts from remarks.
maintained. information but does not focus.
significantly detract from focus. • Frequent subjective
• Objectivity mostly maintained, intrusions.
though occasional subjective
remarks occur.
Organization • Organization and • Organization and • Organization and development of • Organization and • Organization and
and development of content is development of content is content is adequate. development of content is development lacking.
Development logical and is well- logical with minimal errors. • Contents not well developed. insufficient. • Content and central ideas
developed. Content and central ideas • Hypotheses/Aims/Objectives are • Content is not developed. are not developed.
• Hypotheses/Aims/Objectiv developed. stated, but may lack precision and • Hypotheses/Aims/ • Hypotheses/Aims/Objectiv
es are clearly formulated • Hypotheses/Aims/ clarity. Objectives are poorly es are not stated and do
and articulated. Objectives are adequately • Conclusion is stated, but lacks stated and do not become not become evident until
• Conclusion is fully and formulated and articulated. precision and clarity. evident until the end. the end.
clearly articulated. • Conclusion is adequately • Conclusion is poorly • Conclusion is not stated.
and clearly articulated. stated, and is faulty • Paper has no logic and
• Paper has many problems paragraphs are random
with structure. and lack explanation.
Style • Sophisticated and varied • Frequently varied sentence • Occasional variation in sentence • No variation in sentence • No variation in sentence
sentence structure and structure and length with structure and length. structure – very simple structure – very simple and
length. rare style or structure • Generally, pragmatically adequate, syntax, short and simple frequently erroneous
• Objective, efficient errors. though some informal style may sentences. syntax.
academic language. • Mostly objective, efficient appear. • Style frequently • Very short sentences.
academic language. pragmatically inadequate • Style mostly pragmatically
with excess informal inadequate with excess
language. informal language.
Research • Flawless APA format and • Mostly appropriate APA • Some lapses in APA conformity. • Frequent lapses in APA • Paper mostly APA
document design. section headings and • Paper sections occasionally conformity. inconsistent.
• Unfailingly correct in-text • Subheadings. Infrequent inconsistent with APA. • Paper sections frequently • Widespread errors in in-
citations. errors in in-text citations. • Some errors in in-text citations and • APA inconsistent. text citations and in
• Flawless reference • Occasional errors in in reference section. • Frequent errors in in-text reference section.
section. reference section. • Complete absence of plagiarism. citations and in reference • Evidence of plagiarism.
• Complete absence of • Complete absence of section.
plagiarism. plagiarism. • Complete absence of
plagiarism.
(Adapted from Texas A&M University)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 12
Rubric for Grading an Essay Exam
Criteria & Points Missing or Serious Below Meets Expectations Excellent Work Points Earned
Problems Expectations
Assigned
0 1 2 3
Relevance of The essay did not Answer is Answer is brief with Answer is complete;
answer to the answer the question. incomplete. insufficient detail. sufficient detail
question Excessive Unrelated issues provided to support
discussion of were introduced assertions; answer
unrelated issues and/or minor errors focuses only on issues
and/or significant in content. related to the question;
errors in content. factually correct.
Thoroughness of None of the relevant Serious gaps in the Most of the basic Deals fully with the
answer details were basic details details are included entire question.
included. needed. but some are
missing.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 13
Appendix B
Sample Rubrics | Presentations
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 14
Scoring Rubric for General Oral Presentation
Category Scoring Criteria Total Score
Points
The type of presentation is appropriate for the topic and audience. 5
Organization
Information is presented in a logical sequence. 5
(15 points)
Presentation appropriately cites requisite number of references. 5
Introduction is attention-getting, lays out the problem well, and establishes a framework 5
for the rest of the presentation.
Technical terms are well-defined in language appropriate for the target audience. 5
Appropriate amount of material is prepared, and points made reflect well their relative 10
importance.
There is an obvious conclusion summarizing the presentation. 5
Speaker maintains good eye contact with the audience and is appropriately animated 5
(e.g., gestures, moving around, etc.).
Speaker uses a clear, audible voice. 5
Presentation Delivery is poised, controlled, and smooth. 5
(40 points) Good language skills and pronunciation are used. 5
Visual aids are well prepared, informative, effective, and not distracting. 5
Length of presentation is within the assigned time limits. 5
Information was well communicated. 10
Score Total Points 100
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 15
Poster Presentation Peer Scoring Rubric
4. Analysis/Results: 0 1 2 3
a) Figures/tables used appropriately and clearly to present the
data
0 1 2 3
b) Findings are presented clearly and accurately
c) Analysis is well explained and appropriately applied 0 1 2 3
5. Conclusion/Discussion:
a) Addressed study’s problem/question 0 1 2 3
b) Conclusions are sufficiently supported by results 0 1 2 3
c) Results are placed into broader framework 0 1 2 3
d) Importance of findings is addressed 0 1 2 3
6. Presentation Overall:
a) Effective overall aesthetic/organization of poster 0 1 2 3
b) Flow of information is logical and facilitates understanding 0 1 2 3
c) Presenter summarized study clearly 0 1 2 3
d) Presenter answered questions well 0 1 2 3
e) Length of poster summary (by presenter) was appropriate 0 1 2 3
f) Study is innovative and has potential to contribute to the field 0 1 2 3
Total / 66
Comments:
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 16
Appendix C
Sample Rubrics | Arts and Humanities
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 17
Rubric for Visual Arts
Exceeding Expectations Meeting Expectations Approaching Expectations Not Meeting Expectations
Portfolio/ The student demonstrates The student demonstrates The student demonstrates The student demonstrates
Performance deep understanding of the solid understanding of the basic understanding of the limited understanding of the
principles and elements principles and elements principles and elements principles and elements used
used in the art form under used in the art form under used in the art form under in the art under study, and has
study, and demonstrates study, and demonstrates study, and demonstrates difficulty demonstrating
creativity with the medium creativity with the medium creativity with the medium creativity with the medium
chosen. chosen. chosen. chosen.
The work produced will The work produced will The work produced will The work produced will be of
demonstrate high quality, demonstrate good quality, demonstrate developing sub-standard quality, and be
and be presented in a and be presented in a quality, and be presented in presented in a nonprofessional
professional manner. somewhat professional an acceptable manner. manner.
manner.
Craftsmanship The student will demonstrate The students will demonstrate The student will demonstrate The student demonstrates
and working methods, and knowledge of some new demonstrate some insights demonstrate new insights and
some historic knowledge of insights and working methods, and working methods, and working methods, and has
the uses and development of and some historic knowledge some historic knowledge of insufficient historic knowledge
the medium chosen are of the uses and development the uses and development of of the uses and development
essential. of the medium chosen. the medium chosen. of the medium chosen
Interpretation/ Students will demonstrate an Students will demonstrate Students will demonstrate a Students will demonstrate
Analysis exemplary ability to analyze proficient ability to analyze and developing ability to analyze deficiencies in their ability to
and interpret the art form interpret the art form and interpret the art form analyze and interpret the art
under study. understudy. understudy. form understudy.
The student will demonstrate The student will demonstrate a The student will demonstrate The student will demonstrate
an excellent understanding of solid understanding of this art an understanding of this art an insufficient understanding
this art form. form. form at a beginning level. of this art form at any level.
(Adapted from Otis School of Arts and Design)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 18
Rubric for Digital Media Project
Exceeding Expectations Meeting Expectations Approaching Not Meeting Expectations
Expectations
Organization Organizational pattern Organizational pattern Organizational pattern Organizational pattern
(specific introduction and (specific introduction and (specific introduction and (specific introduction and
conclusion, sequenced conclusion, sequenced conclusion, sequenced conclusion, sequenced
material within the body, material within the body, material within the body, material within the body,
and transitions) is clearly and transitions) is clearly and transitions) is and transitions) is not
and consistently observable. intermittently observable. observable.
observable.
Content Uses appropriate, relevant, Uses appropriate, relevant, Uses appropriate and Uses appropriate and
Development and compelling content to and compelling content to relevant content to develop relevant content to develop
illustrate mastery of the explore ideas within the and explore ideas through simple ideas in some parts
subject. context of the assignment. most of the work. of the work.
Storyboard or Illustrates the presentation Includes presentation Thumbnail sketches are not Documents incomplete and
organizing structure with thumbnail structure with thumbnail always clearly marked. unclear.
document sketches of each scene. sketches of each scene. Descriptions of scenes,
Notes showing transitions, Notes showing transitions, transitions, etc. are
effects, etc., are clearly effects, etc. are presented. incomplete or difficult to
planned follow.
Technical Project is edited with only Project is edited with only Project is edited in places. Project is edited poorly with
Elements high quality shots and quality shots and sounds Transitions are noticeable distracting pacing and
(audio, lighting, sounds remaining. remaining. Pacing and and detract from viewing. timing. Clarity is impacted
video, timing, Transitions are smooth and timing are appropriate and Pacing and timing are by poor technical control.
etc) project’s timing is enhance clarity. choppy.
appropriate and enhances
clarity.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 19
Appendix D
Sample Rubrics | Science and Math
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 20
Rubric for Conducting an Experiment in the Lab
Exemplary Competent Needs Work
Materials All materials needed are present and entered All materials needed are present, but not all are All materials needed are not present and
on the lab report. The materials are entered on the lab report, or some materials are are not entered on the lab report. The
appropriate for the procedure. The student is absent and must be obtained during the materials are not all appropriate for the
not wasteful of the materials. procedure. The materials are appropriate for the procedure or there are some major
procedure. omissions.
Procedure The procedure is well designed and allows The procedure could be more efficiently The procedure does not allow control of all
control of all variables selected. All stages of designed, but it allows control of all variables variables selected. Many stages of the
the procedure are entered on the lab report. selected. Most stages of the procedure are procedure are not entered on the lab report.
entered on the lab report.
Courtesy While conducting the procedure, the student While conducting the procedure, the student is While conducting the procedure, the student
and Safety is tidy, respectful of others, mindful of safety, mostly tidy, sometimes respectful of others, is untidy, not respectful of others, not
and leaves the area clean. sometimes mindful of safety, and leaves the area mindful of safety, and leaves the area
clean only after being reminded. messy even after being reminded.
Purpose Research question and hypothesis are stated Research question and hypothesis are stated, Research question and hypothesis are not
clearly, and the relationship between the two but one or both are not as clear as they might be, stated clearly, and the relationship between
is clear. The variables are selected. or the relationship between the two is unclear. the two is unclear or absent. The variables
The variables are selected. are not selected.
Data Raw data, including units, are recorded in a Raw data, including units, are recorded although Raw data, including units, are not recorded
Collection way that is appropriate and clear. The title of not as clearly or appropriately as they might be. in a way that is appropriate and clear. The
the data table is included. The title of the data table is included. title of the data table is not included.
Data Data are presented in ways (charts, tables, Data are presented in ways (charts, tables, Data are presented in ways (charts, tables,
Analysis graphs) that best facilitate understanding and graphs) that can be understood and interpreted, graphs) that are very unclear. Error analysis
interpretation. Error analysis is included. although not as clearly as they might be. Error is not included.
analysis is included.
Evaluation The results are fully interpreted and compared The results are interpreted and compared with The results are not interpreted in a logical
of with literature values. The limitations and literature values, but not as fully as they might way or compared with literature values. The
Experiment weaknesses are discussed and suggestions be. The limitations and weaknesses are limitations and weaknesses are not
are made as to how to limit or eliminate them. discussed, but few or no suggestions are made discussed, nor are suggestions made as to
as to how to limit or eliminate them. how to limit or eliminate them.
(Adapted from Stevens, D.D. & Levi, A.J., (2005). Introduction to Rubrics. Sterling, VA: Stylus)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 21
Evaluation Rubric for Undergraduate Research Project in the Sciences
Exemplary Acceptable Unacceptable
Statement of the The student has independently identified and The student has made independent The question under study is poorly
Problem/Hypothesis developed a research question/hypothesis that contributions and development to a specified and/or is completely
provides a contribution to the scientific literature in the general idea or project suggested by specified by the faculty advisor with
research area. faculty advisor. no development or contribution by the
student.
Role of Theory The experiment is a novel test of one or more current The experiment tests one or more The experiment is unrelated or
theories, or the experiment tests an important set of current theories, or seeks to document misconstrues current theory and is a
novel phenomena. Relevant theory is clearly and expand understanding of phenomena poor extension of the empirical
correctly described so that the contribution of the described in the empirical literature. literature.
experiment is clear.
Development of Idea Logical, testable prediction(s) are identified and tested Logical, testable prediction(s) are The logic underlying the experiment
in the first experiment. One or more follow on identified and tested in a single is incorrect, badly explained, or
experiments are conducted to expand theoretical experiment. missing entirely.
conclusions or rule out alternative explanations.
The design of the experiment is novel. Independent Appropriate independent and dependent Inappropriate independent and/or
and dependent variable(s) have been identified and variable(s) are used. Adequate care has dependent variable(s) are used.
Experimental Design possible confounding factors are controlled. been taken to control possible Limited effort has been taken to
confounding factors. control possible confounding factors.
Analysis and Presentation The data analysis technique is sophisticated and The data analysis technique is The data analysis technique is
of Data appropriate for data collected, informative with respect appropriate for the data collected and inappropriate and/or incorrectly
to the question being studied. Data is appropriately correctly computed. Data is appropriately computed.
reported and displayed so that relevant findings are reported and displayed so that relevant
apparent. findings are obvious. Data displays are incorrect, sloppy, or
difficult to interpret.
Interpretation of Results The conclusions drawn are appropriate given the data The conclusions drawn are appropriate Conclusions are inappropriate given
and analyses conducted. Alternative interpretations given the data and analyses conducted. the data. Obvious alternative
are developed into follow-on experiments to further Alternative interpretations are considered interpretations are omitted.
limit conclusions. and either convincingly rejected or used
as the basis for further research
suggestions.
(Adapted from Brown University)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 22
Science Lab Report Evaluation Rubric
This analytic rubric is used to verify specific tasks performed when producing a lab report. The rubric permits students to self-assess as well as receive feedback from the
instructor.
Category Scoring Criteria Weight Student Instructor
Evaluation Evaluation
Lab The question to be answered during the lab is stated. 5
Introduction Research references used to prepare for the lab are listed. 5
15 points The hypothesis clearly shows it is based on research and not just speculation. 5
Procedures are written as part of pre-lab preparation and clearly state the plan for the 5
experiment. If adjustments are made during the lab, those changes are noted as they occur.
Procedures
All procedures are followed in appropriate order. 5
15 points
Specific formulas for chemicals used or equations for reactions that occur during the lab, 5
when required, are shown on the procedures side of the lab sheet.
Observations Results that occur during a procedure are clearly recorded. 5
Measurements, when required, are recorded as observations, using proper units. 5
15 points
Calculations, when required, are clearly shown on the observation side of the lab sheet. 5
Reasoning for the lab design is summarized, listing any facts or assumptions on which the lab 5
is based.
Conclusion The essential data gathered during the lab is summarized 5
25 points
Essential data from the lab is used to answer the lab question. 5
Aspects of the lab most likely responsible for measurable experimental error are identified 10
The report is neatly printed in ink, with no visible corrections. 10
Presentation The report is written in such a way that others could accurately duplicate the experiment and 5
compare their data.
25 points
There is a clear diagram of the essential apparatus used in the experiment drawn in the 10
largest available white space on the front of the lab report sheet.
Lab Safety No group members were cited for safety violations during the lab period. 5
5 points
Score Total Points 100
Self-Evaluation Students are expected to honestly evaluate their own work. If the difference between the student evaluation and the teacher evaluation is
more than 10 points, 5 points will be deducted from the teacher's score when the grade is recorded.
Deadline Lab reports are due at the beginning of class the day after lab. Reports will be accepted at the beginning of class the second day after lab
for 3/4 credit. No credit will be given after this time.
(Adapted from California State University)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 23
Evaluation Rubric for Mathematical Proofs
Exemplary Proficient Acceptable Unacceptable
The proof uses accurate and Notation and terminology are Most, but not all, the notation Notation and/or terminology is
Use of appropriate mathematical correctly used, but there may and terminology is used frequently misused. The writer
notation and terminology. be instances where the accurately. Errors are may use personal rather than
Mathematical Symbolic notation is used where discourse would benefit from identifiable and correctable by standard notation.
Notation it clearly simplifies the discourse, either more or less use of a reader of experience similar
and avoided when English will symbols versus English to the author.
better serve the reader.
Relevant definitions appear The proof accurately Some relevant definitions Several relevant definitions
where needed to guide the invokes all needed are missing or misstated, are missing or incorrectly
Use of Definitions logical flow. definitions, though they but the proof is otherwise stated, compromising the
may appear other than understandable. argument beyond repair.
precisely where needed.
The proof is well-organized The author generally The proof is well- The proof contains several
and clear, without inclusion avoids digressions, but organized but includes extraneous steps which lead
Concise Writing of irrelevant definitions or may repeat some ideas in extraneous steps, to a confused organization.
theorems. Spelling and an unnecessary way. definitions, theorems, or
grammar are correct. unnecessary repetition.
The proof accurately Reference to necessary Some theorems Reference to prior theorems
references necessary prior prior theorems is necessary to the is generally lacking, or the
Reference to theorems, with explicit complete, but may be deductions are used theorems in question are
Earlier Theorems statements or names somewhat vague. correctly, but others are stated inaccurately.
missing, misused, or
stated inaccurately.
A clear, complete, and The chain of deductive One or more intermediate The hypothesis or conclusion
properly ordered chain of steps is complete and deductive steps are is missing or incorrectly
deductive steps leads from correctly ordered. missing or unclear, but stated. The stated chain of
Logical Flow the hypothesis to the the correctness of the deductions does not lead to
conclusion. The proof proof is not compromised. the stated conclusion.
moves seamlessly between
symbolic notation and
standard English.
(Adapted from Dartmouth College)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 24
Appendix E
Sample Rubrics | Graduate Work
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 25
Evaluation Rubric for Thesis in Linguistics
High Pass Pass Low Pass Fail
Statement of the • Very well written. • Clearly written. • Provides a general • Shows a fundamental lack
Problem • Articulates a concise and • Presents interesting discussion of the of understanding of the
hypothesis and relevant problem. Poorly written,
interesting hypothesis hypothesis and describes issues, but does not incomplete, and lacks
about a significant its importance. discuss its broader structure.
empirical linguistic significance.
problem and its broad
significance.
Grounding in the • Places the work within a larger • Provides a meaningful • Cites most of the key • Fails to cite important,
Literature context. summary of the literature and literature. Lacks critical relevant literature.
• Appropriately integrates relevant builds a case for the research. analysis and synthesis. • Does not clearly relate the
material. literature to the student’s
• Shows deep understanding of contribution.
the significance of the research. • Misinterprets the literature.
Methods and • Uses original or creative use of • Uses existing linguistic • Demonstrates competent use • Uses incorrect methodology.
Data Sources linguistic analysis methods. analysis methodology well. of existing methods. • Data are not handled
Data are novel and appropriate Data are appropriate to the • Collected data allow an
• • appropriately.
for the study. study design. adequate test of the
hypothesis.
Results • Meaningful results obtained from • Well executed. • Analyses are executed • Misanalyzes data or
sophisticated data analyses. • Shows good correctly, but additional fails to analyze
• Analyses map back to the analyses may have relevant data.
understanding of the yielded further insights.
hypotheses. analytical methods. • Results do not follow from
• Discusses the limitations of the • Provides good arguments the analysis and mistakes
are made in interpretation.
analysis. for or against the
hypotheses.
Discussion/ • Places the study in a larger • States what was done and • Summarizes and repeats • Insufficient or incoherent
Conclusion theoretical context. identifies its significance and what was found. Does not discussion of results.
• Informs our understanding of the limitations. discuss the significance or • Shows lack of understanding
limitations of the research.
nature of language. of linguistic theories.
Overall • Original and significant. • Informative, clearly written, • Demonstrates competence. • Poorly written; does not
and well-organized. understand basic concepts.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 26
Rubric for a Dissertation
Exemplary Scholarship Developed Scholarship Developing Scholarship Emerging Scholarship
Abstract • Clear and concise • Organized well • The abstract has an • Introduction to the
• States the problem, • States the research introduction to the findings. problem or findings
missing or not
findings, methodology, problem, findings, • Statement of the problem, developed in a clear
and significance. methodology, and findings, methodology, way.
significance. and/or significance may
need some more • Findings,
organizational work. methodology, and/or
significance not well
organized.
Research • The question or thesis is • The question or thesis is • The question or thesis may • The question or thesis
Question or original and significant in original and clear in its be original but its needs more
Thesis its potential to address potential contribution. significance to the field is development to make
critical issues within the not well supported. the case that it is
respective field. interesting or
important.
Literature • Mastery of and creative • An insightful review that • Provides an analysis of • The literature review is
Review and critical engagement integrates relevant previous findings. incomplete and does
with relevant literature in literature. • Adequate coverage of not include some of
the field. • Demonstrates that the the important
relevant literature but weak
• Demonstrates the gap in references related to
student can use the connection with their the field and subject of
the literature relevant to literature to discuss research question or
the study.
their study and makes a scholarly trends, develop thesis.
compelling argument that hypotheses, and identify • Relevance of the
the candidate’s research the gap in literature their presented literature to
will address the gap. work will address. the research question
unclear.
Theoretical • Works with multiple • Current theories are • Current theories are • Theoretical framework
Frameworks demonstrably relevant, connected to and connected to but provide a is missing, unclear, or
complementary and provide a clear weak framework for the misunderstood.
competing theories framework for the research.
• Addresses how their work research. • Little or no discussion of
will contribute to, support, • Discusses the impact the impact their research
or change established their work may have on may have on existing
theory. existing theories. theories.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 27
Methods and • Study design manifests a • Creative methodology • Choice of methodology • Uses a methodology
Approaches deep understanding of the and study design. and study design shows and/or data that does
field. • Study biases and/or basic competence but not lend itself well to
• Iteratively explores ethical lacks originality. the study of the
limitations within the • Some consideration of question.
questions raised by the study clearly discussed.
data or theoretical • Ethical issues are ethical issues. • Shows lack of
analysis. • Limited discussion of study awareness of
• Discusses the limitations considered limitations within the
appropriately. biases and/or limitations
study design.
of the study design and within the study design.
potential bias. • Clear discussion of • Connection between the • Ethical issues of
• Clear connection between connection between research are not
methodology and data methodology and the data considered.
methodology and data analysis. analysis underdeveloped.
analysis. • No clear connection
between the
methodology and the
data analysis.
Analysis and • Analysis is rigorous, • Analysis is complete and • The analysis has a weak • The analysis is
Interpretation nuanced, and transparent. well-connected to the connection back to theory. incomplete or poorly
• Findings are connected to research question and • A more thorough analysis implemented.
theoretical framework. • The findings are not
research question and • Validity of the findings should be considered. supported by the
theoretical foundations. • Validity of the findings is analysis.
• A rigorous discussion of are addressed
thoroughly. addressed but may lack a
the validity of the findings thorough approach.
is presented and linked to
previous work in the field.
Conclusions • Provides a focused • Conclusions are well- • Summarizes the results • Summary of results is
discussion of conclusions, presented and insightful and provides a general unclear or absent.
situating them in the • Presents a compelling discussion in reference to • The connection
literature. the literature.
argument as to how their between the findings
• Advances the field and study addresses a gap • There is limited discussion and data may not be
raises new questions. in the literature. of the gap in the literature established in a
• Makes a compelling and their study addresses. convincing way.
interesting argument as to • Little or no
the importance of their interpretation is
findings. provided or the
interpretation may not
fit the findings.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 28
Writing and • Writing is precise and • Writing is grammatically • Writing is somewhat • Tone does not exhibit
Scholarly clear. correct, precise, and developed and an understanding of
Voice • Lexicon of the field is clear. professional. the academic writing
• Lexicon of the field is • The lexicon of the field is genre.
clearly explained and
defined. clearly explained and understood and used • The candidate appears
• Scholarly style and format defined. properly. to not have a
• Scholarly style and • Spelling, punctuation, command of the field’s
are accurately used. lexicon.
format are accurately grammar, and formatting
• The candidate’s ‘voice’ is used. generally meet program • Errors of spelling,
heard and yields a and institutional standards. punctuation or
definitive understanding of formatting may be
the issues being present.
discussed.
Oral Defense • Masterfully defends • Competently defends • Answers questions, but • Does not adequately
research by providing research by providing may lack insight. defend research.
clear and insightful very helpful answers to • Frequently shows a need • Frequently shows a
answers to questions. questions.
for deeper reflection on need for deeper
• Uses presentation • May occasionally minor points. reflection on key
resources as a guide. manifest need for further • Relies too much on points.
reflection on minor
points. presentation materials. • Reads the material
from presentation to
• Uses presentation make the report and is
resources as a guide. clearly not comfortable
with the topic.
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 29
Thesis Proposal Rubric
Fully Met (3) Met (2) Partially Met (1) Not Met (0)
Proposal Effectively and insightfully develops a Develops a set of testable and Develops hypotheses. Hypotheses are not testable or
Overview set of testable, supportable and supportable hypotheses. justifiable.
impactful study hypotheses.
Justification for The introduction section provides a The introduction section provides a The proposal provides weak Very little support for the
Hypotheses cogent overview of conceptual and logical overview of conceptual and support for study hypotheses. conceptual and theoretical
theoretical issues related to the study theoretical issues related to the relevant to the study hypotheses
hypotheses. study hypotheses. was provided.
Supporting Provides clearly appropriate evidence to Provides adequate evidence to Provides inappropriate or Provides little or no evidence to
Evidence support position. support position. insufficient evidence to support support position
position.
Review of Sophisticated integration, synthesis, Provides a meaningful summary of Fails to cite important or relevant Provides little or no relevant
Relevant and critique of literature from related the literature. Shows understanding scholarship. Misinterprets research scholarship.
Research fields. Places work within larger context. of relevant literature findings.
Focus The proposal is well organized and has The proposal has an organizational The proposal is somewhat focused The document lacks focus or
a tight and cohesive focus that is structure and the focus is clear or has minor drifts in the focus. contains major drifts in focus
integrated throughout the document. throughout.
Methodology Identifies appropriate methodologies Identifies appropriate Identifies appropriate The methodologies described
and research techniques (e.g., justifies methodologies and research methodologies and research are either not suited or poorly
the sample, procedures, and techniques but some details are techniques but many details are suited to test hypotheses. The
measures). Data analytic plan is missing or vague. missing or vague. The methodology is under-developed
suitable to test study hypotheses. methodology is largely incomplete. and/or is not feasible.
Provides appropriate justification for
controls. Project is feasible
Clarity and The proposal is well written and ideas The proposal effectively The proposal communicates ideas The proposal is poorly written
Organization are well developed and explained. communicates ideas. Some adequately. Many sections lack and confusing.
sections lack clarity. clarity.
References and Properly and explicitly cited. Reference Properly cited. May have a few The proposal has several instances The proposal lacks proper
Citations list matches citations instances in which proper citations of improper use of citations. citations or includes no citations.
are missing. Contains several statements
without appropriately citing.
(Adapted from the University of New Orleans)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 30
Appendix F
Sample Rubrics | Miscellaneous
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 31
Undergraduate Internship Rubric
Criteria Exemplary Proficient Acceptable Emerging Insufficient
Prompt and • Exhibits perfect • Is late or absent no more • Is late or absent no more • Is late or absent no • Is late or absent four or
Regular attendance than once during the than twice during the more than three times more times during the
Attendance • Is continually on-time internship internship during the internship internship.
Disposition • Inspires positive • Initiates positive • Displays positive • Displays positive • Does not display positive
interactions among interactions with others interactions with others interactions with some interactions with people in
others in the • Responds in a mature • Responds in a mature people in this setting this setting.
workplace manner to resolve manner to resolve • Attempts to resolve • Is unable to resolve
• Responds in a mature conflict conflict conflict in a mature conflict in a mature
manner to prevent or manner manner
resolve conflict
Responsive to • Actively seeks out and • Seeks constructive • Accepts constructive • Attempts to accept • Does not accept
Feedback applies constructive feedback feedback constructive feedback constructive feedback
feedback
Professionalism • Conducts self in a • Conducts self in manner • Conducts self in • Conducts self in • Does not conduct self in
professional manner that brings praise from expected professional acceptable manner an acceptable manner
that brings praise to other employees manner some of the time
the organization
Organization • Keeps workspace and • Keeps workspace and • Keeps workspace and • Takes steps to keep • Does not keep workspace
projects well- projects well-organized projects somewhat workspace and and projects organized
organized • Follows organizational organized projects organized • Does not follow
• Always follows policies and procedures • Follows organizational • Attempts to follow organizational policies and
organizational policies consistently policies and procedures organizational policies procedures
and procedures most of the time and procedures
Reliability & • Can be relied on to • Can be relied on to • Can be relied on to • Can be relied on to • Cannot be relied on to
Responsibility perform job tasks at all perform job tasks nearly perform job tasks most of perform job tasks some perform job tasks
times all times the time of the time
Initiative • Displays motivation in • Displays motivation in • Displays motivation in • Displays motivation in • Does not display
all interactions nearly all interactions most interactions some interactions motivation in interactions
• Creates solutions to • Creates solutions to • Creates solutions to • Occasionally creates • Unwilling or unable to
problems with no problems with little problems when solutions to problems create solutions to
instruction instruction instructed when instructed problems when instructed
Communication • Always responds to • Always responds to • Always responds to • Typically responds to • Fails to respond to
with Supervisor supervisor supervisor supervisor supervisor initiated supervisor communication
communication communication communication communication
• Initiates • Often initiates
communication communication
Fulfills • Consistently exceeds • Sometimes exceeds • Meets expectations and • Meets some • Does not meet
Expectations expectations and expectations and goals goals set by supervisor expectations and goals expectations and goals set
goals set by set by supervisor and and organization set by supervisor and by supervisor and
supervisor and organization organization organization
organization
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 32
A Rubric for Rubrics
Criteria Unacceptable Developing Acceptable
Selection and Clarity of Criteria Criteria being assessed are unclear, Criteria being assessed can be All criteria are clear, distinct, and
have significant overlap, or are not identified, but not all are clearly derived from appropriate standards
derived from appropriate standards differentiated or derived from for task and subject area.
for task and subject area. appropriate standard for task and
subject area.
Distinction Between Levels Little or no distinction can be made Some distinction between levels is Each level is distinct and
between levels of achievement. clear, but may be too narrow or too progresses in a clear and logical
wide. order.
Quality of Writing Writing is not understandable to all Writing is mostly understandable to all Writing is understandable to all
users of rubric, including students. users of rubric, including students. users of rubric, including students.
Has vague and unclear language Some language by be confusing among Has clear, specific language that
which makes it difficult for raters to raters. helps raters reliable agree on a
agree on a score. score.
Reliability of Scoring Cross-scoring among faculty and/or Cross-sharing by faculty and/or Cross-scoring of assignments
students often results in significant students occasionally creates results in consistent agreement
differences. inconsistent results. among scorers.
Use of Rubric to Communicate Rubric is not shared with students. Rubric is shared with students when the Rubric serves as a primary
Expectations and Guide Students task is completed and is only used for reference point as students begin
evaluation of student work. their work, for discussion and
guidance, as well we evaluation of
student work.
Engagement of Students in Using Students are not engaged in either Students are offered the rubric and use Students discuss the design of the
Rubrics development or use of the rubrics. it for self-assessment. rubric and offer feedback/input and
are responsible for use of rubrics in
peer and/or self-evaluation
(Adapted from a model developed by Dr. Bonnie Mullinix, TLT Group)
USING RUBRICS TO ASSESS STUDENT LEARNING OUTCOMES AT THE PROGRAM LEVEL PAGE 33