After my first semester here, I gave the break down of my teaching evaluations. Here I’m at it again.

More importantly, I’m going to follow up this post with another post that is more reflective on this past year has gone (in terms of teaching) and how I see this year’s experience informing goals for improvement next year. But I wanted to get the data down first.

So, here it is. Below is a summary of my teaching evaluations for both my inquiry and physics courses–course that I taught both fall and spring semester. In each category, I show the average rating for inquiry, physics, and the department. In parentheses, I show the change in my ratings from last semester to this semester. In every category, the ratings either improve or stay the same. With respect to department averages, my evaluation for my physics class are near ceiling. In inquiry, there are two categories that I am still below departmental averages, but both showed significant improvement.

Presentation:

Inquiry 4.7  (+0.3 from fall to spring semester)

Physics: 4.9  (+0.0)

Department: 4.2

Organization:

Inquiry: 4.3  (+0.4)

Physics: 4.8 (+0.0)

Department: 4.1

Assignments/ Grading

Inquiry: 4.3 (+0.4)

Physics:  4.9 (+0.1)

Department: 4.4

Scholarly Approach

Inquiry: 4.2 (+0.2)

Physics: 4.8  (+0.1)

Department: 4.2

Student Interactions:

Inquiry: 4.6 (+0.1)

Physics: 4.9 (+0.0)

Department: 4.0

Motivating:

Inquiry: 4.6 (+0.4)

Physics:  4.7 (+0.0)

Department: 4.1

Effectiveness/Worth

Inquiry: 3.5 (+0.5)

Physics: 4.6  (+0.1)

Department: 3.8

From “Lessons From the Physics-Education Reform Effort” by R.R Hake

As previously indicated, the data of Fig. 1 show that seven of the IE courses (717 students) achieved <g>’s close to those of the T courses. Five of those made extensive use of high-tech microcomputer-based labs (Thornton and Sokoloff 1990, 1998). Case histories of the seven low-<g> courses (Hake 1998b) suggest that implementation problems occurred. Another example of the apparent failure of IE/high-tech methods has been described by Cummings et al. (1999). They considered a standard physics Studio Course at Rensselaer in which group work and computer use had been introduced as components of in-class instruction, the classrooms appeared to be interactive, and students seemed to be engaged in their own learning. Their measurement of <g>’s using the FCI and the Force Motion Concept Evaluation (Thornton & Sokoloff 1998) yielded values close to those characteristic of T courses (Hake 1998a,b,c). Cummings et al. suggest that the low <g> of the standard Rensselaer studio course may have been due to the fact that “the activities used in the studio classroom are predominately ‘traditional’ activities adapted to fit the studio environment and incorporate the use of computers.” Thus the apparent “interactivity” was a product of traditional methods (supported by high technology), not published IE methods developed by physics-education researchers and based on the insights of cognitive scientists and/or outstanding classroom teachers, as for the survey courses.

This quote and similar ones from Hake have been on my mind these past few weeks as undergraduate students I am working with are grappling with the question, “Why are our learning gains on the FCI lowered than expected and desired?” This question emerges out of our learning about the FCI, analyzing our local data, and comparing to other research and outcomes that have used the FCI.

So, in our courses, we use peer instruction methods, collaborative-problem solving with whiteboards, and computer facilitated learning. Our class is mostly “flipped” (where students read lecture at home and practice problems in class). Most of it takes place in a studio setting. It sounds and often looks very interactive. On the other hand, all the content students are interacting with– from the textbook, to the laboratory activities, to the questions and problems students work on in groups–is home-grown. For better or worse, these home-grown materials would probably be characterized as “traditional activities adapted to fit the studio environment and incorporate the use of computers.”

As a group of researchers, we are approaching our question in a variety of different ways:

(1) One students hopes to ask instructors in our department to take the FCI, not by answering how they would answer, but by choosing the answer they think would be the most common incorrect answer chosen by students, and to estimate the percentage of students answering each questions correctly after the course. He hopes to compare instructor expectations to reality, in order to answer question like, “How knowledgeable are instructors of specific content difficulties students have and how aware are they of the prevalence of those difficulties in our courses?”  We have been reading a lot of pedagogical content knowledge.

(2) Another student is interested in examining student learning in relationship to our home-grown textbook. Does the book explicitly address specific difficulties we know about from our own data and research in physics education? Does the book implicitly reinforce any difficulties? Does it provide opportunities for developing conceptual understanding as well as problem solving? He is also interested in questions like, “Do students actually read the text? How much? How deeply? In what do they engage with text? What do they actually take away from the reading the text, and how does that play out in relationship to classroom instruction?” … We have been reading a lot about self-explanation, preparation for future leraning, refutation texts, and the influence of prior knowledge (e.g., misconceptions) on reading comprehension.

(3) Another student is interested in examining structural factors of instruction that might be contributing to lower than expected FCI gains, including

• Student background and academic preparation
• The prevalence of exam questions that probe (and hold students accountable) for conceptual understanding
• The quality of apprenticeship and training that undergraduate TAs and new faculty receive for teaching using interactive engagement methods
• The strategies that instructor use to motivate and cultivate a classroom culture in which IE methods are taken seriously.

We have been reading papers about the kinds of background that correlate with FCI scores, as well as papers about programs that have successful or unsuccessful implementations of reform-physics-curricula.

This work, for better or worse, treads on a sensitive arena–a close examination of ourself.  The fact that this work is being carried out by students, I think, could be perceived as making this endeavor even more sensitive, but in an odd way it makes it authentic. All of these students are really interested in improving instruction here, doing research that is valid but also relevant to local stakeholders. They have no axe to grind or hidden agenda. We are also just genuinely intrigued by the puzzle, and curious to pursue its potential solutions. Some of that solution, will not doubt in my mind, need to be geared toward improving the curriculum at the content-level–the content as embedded in all the tasks we ask students to engage with, from the text, to the labs, to the questions and problems they work on. Some of that solution will no doubt be about getting our department on board with the continued renewal of that content based on assessment, feedback, analysis, and ongoing revisions. In that sense, the work we are beginning serves a launching point for what will need to become an ongoing endeavor.

I’m not as happy with this… not as coherently packaged as first, trying to do too much, etc. But here it is.

Question 1:

In class we discussed that 60 mph means that you travel 60 miles in each and every 1 hour of travel. What does it mean for a car to be traveling at 60 mph if it hasn’t yet gone for an hour? What does the “60” mean, for example when you’ve only traveled for 30 minutes, or 15 minutes, or 1 minute?

Question 2:

You and a friend are driving along the highway at 60 mph, exactly 9 miles away from your exit. Your friend announces, “It should take about 9 minutes to travel those 9 miles, because every mile takes one minute.”

How do you think your friend arrived at this conclusion? Is he right? Explain.

Question 3:

Your friend is back at it again, keeping track of where you are one the highway, but this time he jots down the mile-marker every ten minutes.

Are you guys driving with constant speed, increasing speed, or decreasing speed? Explain how you can tell from the data above.

Question 4:

What’s your best estimate for where your car will be at 7:00am? Explain how you came to your answer and why it makes sense to you.

Question 5:

During your drive, you and your friend start arguing about whose car is better at speeding up. Your car can go from 0-60 mph in 4 seconds. Your friend’s car that can go from 0-80 mph in 5 seconds.

Whose car would you say is better at speeding up? Explain how you reached your conclusion.

I’m drafting my pre-class reading quizzes for the summer session. Here is day one. Please comment.

Question 1:

Imagine you are tutoring an 8th grade student in physics. You notice that the student is a little confused about the difference between what is meant by 3:00 pm and what is meant by 3 hours. How would you explain to this student the difference between 3:00 pm and 3 hours?

Question 2:

You are still tutoring this same student, and notice the student is having a similar problem, except now they are confused about the difference between being at mile-marker 72 and having traveled 72 miles. How would you explain to them the difference between being at mile-marker 72 and having traveled 72 miles?

Question 3:

Here is a physics problem! Imagine you are driving a car along a long straight. Your friend in the passenger seat is recording where you are every hour of the trip.

Explain why your friend might reach the conclusion that the speed of the car at 3:00 PM was 24 miles/hour.

Then explain why that answer is probably not correct.

Question 4:

You might have noticed that the table above does not have information about where the car was at 5:00PM. This information is missing because your friend was asleep. What do you think is a good estimate for the mile-marker of the car at 5:00PM? Explain how you determined your answer and why you think this is a good estimate.

 Question 5:

Here is one final question to consider. How much total distance did you and your friend travel between 2:00PM and 5:00PM? Why is this answer different than your answer to question 4?

A paper that an undergraduate researcher and I were reading this week is this:

” Are All Wrong FCI Answers Equivalent?” by Helena Dedic, Steven Rosenfield, and Nathaniel Lasry, published in the 2010 Proceeding of the Physics Education Research Conference.

Essentially the paper examines patterns of student responses to the 4 Newton’s law questions in the FCI in order to identify different classes of students. Using a statistical method called Latent Markov Chain Modeling, they group students together based on how they respond to the four questions–not just in terms of correctness, but which answer they pick specifically. Their analysis identified 7 groups of students. One surprising thing from their analysis is that the groups form a natural hierarchy.

Basically, the authors looked at how students transition from one group to another, by calculating the probability of transitions occurring between classes. The cool result is that there is a strong directional bias to the transitions. In other words, there exists an ordering of the groups (e.g., C1-C7), such that transitions are always toward C1. After instruction, students are likely to either stay put or move up the hierarchy, but they are very unlikely to move down the hierarchy.

What’s even more interesting is that the classes do not form developmental stages. This means that C7-C1 is not a progression of learning. Certain forward transitions do not happen, or are very unlikely to happen.

It’s been helpful for me to think of the situation quantum mechanically. There are discrete cognitive states which can be measured, and there are certain probabilities of transitioning into different cognitive states; some transitions are very unlikely to happen and can perhaps even be forbidden. The fact that they fall in a hierarchy also reminds me of quantum mechanics, in that we can order QM states by there energy levels, and this ordering implies a bias in how perturbed and unperturbed systems such as the hydrogen atom will transition.

What I especially like about this paper is the meaning they give to each of the classes (in terms of student schemas), and how those interpretations build on much of what we know about student thinking about Newton’s 3rd Law. Each of the classes is generated from the data, and are more complexly defined than what I write below, but the gist is this

Class 1: They get all the questions right, essentially. This class is intended to represent a class that is highly likely to be a Newtonian Thinker about these Questions

Class 2: This class essentially gets the all the questions right except for the the question about the Newton’s third law when two objects are speeding up together. The interpretation given here is that these students are thinking of the Newton’s third law pairs within a Net Force (or competing forces) schema.  ** In my teaching physics class, we had long arguments about this question **

Class 3: What I can make of this class, is that the students essentially get either 1 question wrong (but not the speeding up question), or two questions wrong (but not in a pattern similar to Class 4 or Class 5). I wasn’t really quite sure what to make of this class, exactly. Students had about 65-75% of answering any question correctly, implying that there is an assortment of 1-2 answers wrong. But the Modal answer for each question is the correct answer, where as Class 2 the Modal answer for one of the question is wrong.

Class 4/5:  These two classes are almost the same. They both answer consistently that the more “dominant” object exerts more force. Class 4 says that when maintaining speed neither object is more dominant (so forces are same); whereas Class 5 says that the one of the object is more dominant (forces not same).

Class 6: I think this less strongly defined (with lots of scatter), but I gathered they included students who said that objects can be obstacles that are merely in the way without exerting a force.

Some cool comments about transition probabilities: I think a really cool gem in this paper is that there is a fairly low transition probability from Class 2 to Class 1. In other words, students who start in Class 2 only have a 37% of transitioning up to Class 1. This is in contrast with Class 3, where students have a 65% of transition up to Class 1 (and low probability of going to Class 2). This means that the cognitive state of thinking about Newton’s 3rd Law Pairs in terms of Net Force is a fairly stable state. This mean that being high in hierarchy doesn’t necessarily imply you’ll ever get to the top! It’s like there a trajectory of learning that gets you nearly to the top, but never quite there. Other things are this: Students with a dominance schema (class 4 and 5) have a high probability of basically staying put, and also true of Class 6, for which students are 58% likely to stay put.

It’s kind of weird, but only Class 3 makes significant movement to Class 1. Classes 2, 4, 5, and 6 have limited mobility, at least within the time frames they are looking.

Anyway, I’m curious what other people think, and what they see in their FCI data.

I’ve been thinking about teacher preparation, and keep coming back to Duckworth:

“First, teachers themselves must learn in the way that the children in their classes will be learning… Second, the teachers work with one or two children at a time so they can observe them closely enough to realize what is involved for the children. Last,  it seems valuable for teachers to see films or live demonstrations of a class of children learning in this way, so that they can begin to think that it really is possible to run their class in such a way. The fourth aspect is of a slightly different nature. Except for the rare teacher who will take this leap all on his or her own on the basis of a single course and some written teachers’ guides, most teachers need the support of at least some nearby co-workers who are trying to do the same thing, and with whom they can share notes. An even better help is the presence of an experienced teacher to whom they can go with questions and problems.”

From the Essay ‘The Having of Wonderful Ideas’ by Eleanor Duckworth

One question I asked a lot of the future physics teachers during their oral assessments was to discuss the following expression in terms of what it could mean (i.e., the story it tells) and if it seems to be a valid statement or not.

Δx = vf Δt – 1/2 a (Δt)²

Follow up question were targeted at asking students to identify correspondences between various terms and features of graphs.

With some students, this led to a conversation about all three of equations below–how they were similar and different etc, and how they corresponded to features of graphs.

Δx = vi Δt + 1/2 a (Δt)²

Δx = vavg Δt

Δx = vf Δt – 1/2 a (Δt)²

The growth that I’ve seen in many of the students is this. At the beginning of the semester, most students would have either said that the equation was wrong by pointing out that the wrong velocity and the minus sign, or for some, their only way of determining its validity would have been to substitute another equation into that equation until it looked like an equation from a textbook that they already knew. Thus, by the authority of the textbook, the statement must be true as well.

Most students by the end of the semester were able to look at this brand new equation–one they have never seen or considered before–and tell both a verbal story and a graphical story that explains and validates the equation without need for algebraic substitutions. I am impressed.

In other areas, growth has come in fits and starts, and there were many relapses. Many students during oral assessments this last week exhibited one of more of the following difficulties:

• Confusing instantaneous velocity with constant or average velocity (e.g., claiming that an object thrown upward with a speed of 20 m/s goes 20 meter in the first second)
• Confusing change in velocity for average velocity–the two are subtly related in constant acceleration case. Interestingly, this mistake came up both mathematically and conceptually for the same student in the same assessment.
• Confusing acceleration, velocity, and displacement–claiming that when an object falls 10m, it will take 1s to do so, and be up to 10 m/s of speed.
• Confusing change in speed with change in velocity–claiming that a bouncy ball experiences no change in velocity upon rebounding off the floor.

To be fair, these relapses often came up within much more complex physical situations and tasks that were more cognitively demanding–bouncing balls, oral assessments, deriving a result, etc. It’s good to know that, distinguishing these concepts, is not stable and automatic for them. It requires specific cognitive attention to maintaining the distinctions. They can do it, but are prone to making these mistakes when there are many balls to juggle all at once. I discussed with students how the mistakes they are prone to making here are similar to the mistakes we encountered all semester in looking at student work and how they are similar to the ones they are going to encounter in the classroom, so it is best to understand these mistakes well by making them yourself in as many different situations as possible, and doing the hard work of finding your way out of them.

Doing oral assessments for standards makes the assessments spontaneously grow into interesting conversations. Sometimes, what I thought I was initially assessing became something else entirely. Sometimes during assessments, I would respond to wrong things student said by saying, “OK. Tell me about that again, and then tell me why what you just said can’t be right.” Some students would be able to recover, and others would need more time, and I’d give it to them. Other times, I would send a student off, saying, “There’s a mistake in here. I want you to go figure out what the mistake is. When you come back, I don’t just want to know what the right answer is, I want to know why you made that mistake and why it’s a big deal.”

It’s been interesting to watch the varied resiliency of students in the face of my refusal to accept their work as proficient. The only times I know I was doing a less-than-good job was when they would come back and say, “Is this what you want?” Fortunately, it wasn’t often, but it did happen a few times. Most of the times they would come back saying, “OK. I got it now,” or, “I still don’t get this, can we talk about it some more?”

Next year, I’m putting a 2-week rule on my standards, so that students have to turn in an initial draft within 2 weeks. I had many students flying in last minute trying to work through all the standards.

An email I received today:
First of all let me introduce myself… I am just finishing up my first year as a Physics teacher. I have come across a situation that I am a little bit stuck on trying to explain. I have been going through Electricity and Magnetism. Well, I should say that I have already taught this, but am now reviewing for the final exam, and I am coming back up to explaining what causes magnetism. I just don’t feel like I explained it clearly enough. When I try to explain Orbital magnets as well as Spin magnets I seem to struggle being clear in my explanation; how would you go about trying to explain this to High school seniors? If you could offer an explanation I would appreciate it. I am still trying to wrap my head around being clear in my explanations. I did not go to school for Physics; I did take a couple of classes and that qualified me to teach it. I want to be the best teacher I can be so that is why I am asking for some help. Once again I appreciate you taking the time to answer my questions.

My *way too long* of a response:

Thanks for the email. Congratulations on making your way through your first year teaching physics!

You’ve really opened a big issue, not only about magnetism, but about the nature of explanation and learning. My first suggestion in making progress through the big issue would be to watch and mull over this commentary by Richard Feynman, which is both about the nature of explanation and the nature of magnetic forces: http://www.youtube.com/watch?v=MO0r930Sn_8

I’d love to talk about the video. For me, the biggest issue in teaching is not having the best explanation, but in understanding the ways in which my students are going to come to learn new things from the things they already know. That means I need to know a lot about how they already think, understand things, and even how they come to settle upon things as being true or not. Only in this way can I know what the best explanation might look like for them. And at the end of the day, we have to pursue that explanation together.

Magnetism is notoriously funny. Some teaching physics friends of mine refer to “magnetism” as “turtles all the way down”. It’s a joke, but also true. Big magnets are made up of mid-size magnets, and those mid-size magnets are made of magnets that are tiny… and those made up of ones even tinier…and those magnets are made up of particles that are magnetic, those are made up of sub atomic particles that are magnetic. Starts to sound ridiculous, that our best explanation of magnets is that magnets are made up of magnets… but that’s all we have really.

Of course, when tiny magnets work together, they can produce magnetic effects that are larger, or when not working together, they produce tiny or no magnetic effects. Why are those tiny things magnetic? I don’t really have good explanations for those things. But understanding that there are magnetic things, and that they can push and pull and twist on each other, and that they can arrange themselves in ways that amplify those pushes and pulls is pretty much everything to know. Then the devil’s in the details of figuring out what arrangements lead to what kind of effects. The really specific details are the tough work of undergraduate and graduate physics.

If you watched the video, you’ll see that Feyman’s point is that also that magnetism is not meant to be explained -it is meant to do the explaining. That’s a stance that he is taking–magnetism is a powerful idea to explain other things. The story of how we have come to accept magnetism as a true and worthwhile idea is really curious.  I know I probably haven’t satisfied your question, but it’s the best I can do.

What would you have written?