Reliability of a Concept Inventory to Determine the Level of Students in Statics

Abstract

Basic core courses in engineering are often taught and assessed only through procedural approaches, neglecting the conceptual grounds of the subject. In an effort to change that, a Concept Inventory test was applied to 195 students at Eafit University enrolled in Statics. All of the students took the test inside in the campus facilities and were monitored during the whole session to make sure they did not share information. The students took a Computer-Based Test of the Concept Inventory. Three new items were added to the original test of 27 questions, in order to improve the reliability of two specific groups of concepts. For the test was found a mean of 9.7 and a standard deviation of 5.28. For the overall test, the Cronbach’s alpha coefficient was 0.81. The psychometric analyses show that all of the items in the test, except one, present an appropriate fit for discrimination and difficulty parameters of the test. Item 26 was deleted when performing the analysis of sub-scales reliability because presented abnormal value for difficulty. For the cluster of Equilibrium, it was seen that deleting this item resulted in an increase of the alpha from 0.2 to 0.36. For the sub-scale of Free-Body Diagram was seen a decrease in the reliability when two new items were added to it. All of the other sub-scales presented good reliability, most of them, above 0.5. The item-person map shows that the overall latent trait of students is lower than the overall test difficulty, meaning that most of the students found the test difficult. One of the conclusion to be drawn from the results is that the students subjected mainly to procedural approaches in teaching, do not perform well in conceptual tests. The results support the claim that Statics can be seen as a collection of concepts that can be clustered in independent groups for teaching.