Early Post-Secondary Student Performance of Adversarial Thinking

Abstract

Motivation. “Adversarial thinking” (at) is viewed as a central idea in cybersecurity. We believe a similar idea carries over into other critical areas as well, such as understanding the perils of social networks and machine learning. Objectives. What kinds of at can we expect of early post-secondary computing students? In particular, can they meaningfully analyze computing systems that are well beyond their technical ken? Is their analysis limited to only a social or only a technical space? Method. In an introductory post-secondary course, we study student responses to questions designed to exercise at, broadly defined. To do this we develop a rubric that provides insight into desirable content. Results. We find that these students are fairly strong at at. They are regularly able to adopt an adversarial or empathetic viewpoint and analyze quite sophisticated systems. Most of all, they can meaningfully do so (a) outside an explicit cybersecurity context, (b) even from an introductory level, and (c) well before they understand well the key technologies under evaluation. On the other hand, we also find several instances where students do not explore systems as much as they could, and fail to reference other material they know, which could be evidence of lack of transfer. In addition, our rubric would benefit from refinement that would enable a more sophisticated analysis of student responses. Discussion. Our work provides a baseline evaluation of what we can expect from students. It suggests that at can be introduced early in the curriculum, and in contexts outside computer security.