Toward developing a valid and reliable assessment of adults' nature of engineering views

Abstract

Background

Pre-college engineering education reform documents aim to help students develop engineering literacy. Helping students develop sophisticated epistemological views about engineering makes a significant contribution toward developing engineering-literate citizenry. Despite these widely recognized benefits, research on students' and teachers' epistemological views about engineering, commonly termed nature of engineering (NOE) views, has remained stagnant due to the absence of an open-ended questionnaire whose collected evidence of validity indicates its appropriateness for assessing students' and teachers' NOE understanding.

Purpose

The purpose of this study is threefold: (a) describing the development of a new open-ended instrument, the Views of Nature of Engineering Questionnaire version B (VNOE-B), (b) providing evidence in support of the VNOE-B validity and reliability, and (c) discussing the implications of the newly developed instrument for future research.

Design/Method

We aimed to provide evidence for the content and face validity of the VNOE-B by seeking input from a panel of science/engineering educators. Evidence for the construct validity was determined by analyzing the open-ended questionnaire and semistructured interview responses provided by pre-service and in-service teachers (considered novices in this field) and experts (engineers and engineering educators).

Results

Our findings indicate that the VNOE-B is consistently successful in differentiating between experts' and novices' NOE views. All engineering experts held sophisticated NOE views across all NOE views under study, whereas most novices held partially informed or uninformed NOE views.

Conclusion

Evidence of validity supports the appropriateness of the VNOE-B as an open-ended instrument for assessing adults' NOE views. Its application may guide teacher educators in determining the efficacy of teacher professional development in meeting the aim of improving in-service teachers' NOE understanding. Additionally, VNOE-B may potentially inform the success of pre-service teacher training.