Design and evaluation of a robust rubric based on generic skills linguistic

Abstract

Teachers create their own rubric to evaluate learning, but sometimes it is not certain that it was the appropriate design. In this article it is proposed to apply the same rubric and verify its robustness, given a case of oral presentation of engineering students. The presentation of a topic is one of the most complete ways to evaluate the knowledge and skills of the student body. This research is semi-experimental with a pretest-posttest design, the treatment was the application of the rubric to a group of students. The presentations made by students without rubrics [pretest] (that indicate what is evaluated) are contrasted against presentations made by students where they do know what aspects are evaluated in their presentation and what score is awarded [posttest]. 23 speakers were evaluated in two sessions: session 1, the group that was not informed about the rubric; and session 2, the same group that was trained to use the rubric. Validity and reliability are verified using the leniency bias test (49 evaluators and evaluators were assigned, randomly grouped into 5-judge juries) and weighting (different weight was assigned for pretest or posttest). In all cases, the evaluators applied their evaluation on a list of items to evaluate. The results show that the students who adhere to the rubrics are better evaluated. The proposed rubric was reliable and validated.