Paper: Are All Wrong FCI Answers Equivalent

A paper that an undergraduate researcher and I were reading this week is this:

” Are All Wrong FCI Answers Equivalent?” by Helena Dedic, Steven Rosenfield, and Nathaniel Lasry, published in the 2010 Proceeding of the Physics Education Research Conference.

Essentially the paper examines patterns of student responses to the 4 Newton’s law questions in the FCI in order to identify different classes of students. Using a statistical method called Latent Markov Chain Modeling, they group students together based on how they respond to the four questions–not just in terms of correctness, but which answer they pick specifically. Their analysis identified 7 groups of students. One surprising thing from their analysis is that the groups form a natural hierarchy.

Basically, the authors looked at how students transition from one group to another, by calculating the probability of transitions occurring between classes. The cool result is that there is a strong directional bias to the transitions. In other words, there exists an ordering of the groups (e.g., C1-C7), such that transitions are always toward C1. After instruction, students are likely to either stay put or move up the hierarchy, but they are very unlikely to move down the hierarchy.

What’s even more interesting is that the classes do not form developmental stages. This means that C7-C1 is not a progression of learning. Certain forward transitions do not happen, or are very unlikely to happen.

It’s been helpful for me to think of the situation quantum mechanically. There are discrete cognitive states which can be measured, and there are certain probabilities of transitioning into different cognitive states; some transitions are very unlikely to happen and can perhaps even be forbidden. The fact that they fall in a hierarchy also reminds me of quantum mechanics, in that we can order QM states by there energy levels, and this ordering implies a bias in how perturbed and unperturbed systems such as the hydrogen atom will transition.

What I especially like about this paper is the meaning they give to each of the classes (in terms of student schemas), and how those interpretations build on much of what we know about student thinking about Newton’s 3rd Law. Each of the classes is generated from the data, and are more complexly defined than what I write below, but the gist is this

Class 1: They get all the questions right, essentially. This class is intended to represent a class that is highly likely to be a Newtonian Thinker about these Questions

Class 2: This class essentially gets the all the questions right except for the the question about the Newton’s third law when two objects are speeding up together. The interpretation given here is that these students are thinking of the Newton’s third law pairs within a Net Force (or competing forces) schema.  ** In my teaching physics class, we had long arguments about this question **

Class 3: What I can make of this class, is that the students essentially get either 1 question wrong (but not the speeding up question), or two questions wrong (but not in a pattern similar to Class 4 or Class 5). I wasn’t really quite sure what to make of this class, exactly. Students had about 65-75% of answering any question correctly, implying that there is an assortment of 1-2 answers wrong. But the Modal answer for each question is the correct answer, where as Class 2 the Modal answer for one of the question is wrong.

Class 4/5:  These two classes are almost the same. They both answer consistently that the more “dominant” object exerts more force. Class 4 says that when maintaining speed neither object is more dominant (so forces are same); whereas Class 5 says that the one of the object is more dominant (forces not same).

Class 6: I think this less strongly defined (with lots of scatter), but I gathered they included students who said that objects can be obstacles that are merely in the way without exerting a force.

Some cool comments about transition probabilities: I think a really cool gem in this paper is that there is a fairly low transition probability from Class 2 to Class 1. In other words, students who start in Class 2 only have a 37% of transitioning up to Class 1. This is in contrast with Class 3, where students have a 65% of transition up to Class 1 (and low probability of going to Class 2). This means that the cognitive state of thinking about Newton’s 3rd Law Pairs in terms of Net Force is a fairly stable state. This mean that being high in hierarchy doesn’t necessarily imply you’ll ever get to the top! It’s like there a trajectory of learning that gets you nearly to the top, but never quite there. Other things are this: Students with a dominance schema (class 4 and 5) have a high probability of basically staying put, and also true of Class 6, for which students are 58% likely to stay put.

It’s kind of weird, but only Class 3 makes significant movement to Class 1. Classes 2, 4, 5, and 6 have limited mobility, at least within the time frames they are looking.

Anyway, I’m curious what other people think, and what they see in their FCI data.