One thing I really like about standards-based assessment is that learning to do it is much like riding a bicycle:

#1 When you see someone else doing it, it looks like a lot of fun. But it also looks scary. Because you can see right away that so many things could go wrong. The combination of fun and scary makes it compelling.

#2 When you aren’t doing it yet, the only people you see doing it are mostly people you look up to–your big brother, your mom. The fact that these people are doing it makes it compelling.

#3 Getting started is the hardest thing, but then your off and learning; because once you are going, you learn by the mere act of doing. The bike keeps moving; the re-assessments keep coming.

It’s really this third item that I want to talk about. One of things that I really like about doing standards-based assessment is that amount of things I am learning through doing it. There are several reasons for this. First, I am looking at 32-50 individual pieces of assessment per day that I’m NOT grading. I’m looking at the student work and deciding “Yes ” or “No”. Looking at the student work through the lens of, “Have they demonstrated understanding here?” is way more generative of learning for me than, “How many points should I give (or take away) here?” Because of research, I am more practiced at looking at student work and asking questions like, “What is the student thinking here?”, “How did this make sense to them?”, “What about this context made this response so likely?” And while these questions are important, they are different than asking if the student work should count as evidence of understanding. With standards-based assessment, I get lots of practice at this new skill.

Second, I have to re-write assessments constantly that will hopefully assess one isolated skill again and again–a quiz that hopefully cannot be answered correctly by just memorizing something from past quiz. I end up having to write about 5-10 different assessments for each standard.  There are a lot of constraints in writing these assessments:  keeping it a valid measurement of skill under question; keeping it somewhat isolated to that skill; making it different enough from previous quizzes; making sure it can be done in a short time frame; etc, etc. I like the fact that I now feel comfortable writing quizzes off the top of my head. While I occasionally make less-than-superb decisions in writing assessments, I am much better now than I was 6 months ago. I rarely write an assessment that students will get right without understanding. I feel comfortable knowing how to make any assessment slightly easier or slightly harder. Sure, sometimes I goof up. But that’s sort of my point. I learn by making those goofs and thinking on what went wrong with that assessment. How were students able to answer correctly without really getting it? Why did no one answer that one correct? Sometimes it means that I wrote the quiz bad; other times it means there’s some new issue about student learning of that skill that I’d never really thought through before.

My advice to anyone thinking about doing standards-based assessment is to do it, even if a little bit. While I think there may be people who say you have to do JUMP full into SBG to make it work, I’m not sure that’s true. Last semester I dipped my toe into it, replacing reading quizzes with standards-focused quizzes. Students valued it, and I learned from doing. This summer I expanded it. My course is still not fully standards-based. Students get credit for doing pre-class reading questions (“graded” on effort). Individual students labs are graded on a rubric, but a part of their lab grade is now based on lab-skill standards. I’ve made my rubric aligned with the standards, but the lab write-up still gets a grade that can’t be changed; while the lab standard part of the grade can be learned at any time. I still have exams; they are just tightly aligned with standards.

There are lots of good reasons to do standards-based assessment, but one reason should be the opportunity it provides YOU as the teacher to learn to become a better teacher. The practice itself is generative of better practice.

Here’s one thing that’s been going well in my summer physics class that I don’t want to forget about.

Warm-ups that Emphasize Mathematical Strategies

Let’s say later in the day, I plan on discussing and having students practice using a particular approach to solving constant acceleration problems–using the average velocity to figure out how far an object goes. For this, I’m going to need students to be able to have a strategy for finding the velocity that’s in the middle of the initial and the final velocity.

The warm up for class goes like. What number is directly in the middle of 10 and 20? Share you strategy for how you figure it out? How can you check to make sure that number you found is precisely in the middle?

Now try out your strategy for the numbers 320 and 20. Check to make sure that number is precisely in the middle. I keep all the conversation focused on the strategies and the checking strategies.

Next part of the warm-up involves trying our strategy for situations we might not be sure will work–non whole-numbers and negative numbers. Use the numbers – 32 and +12. Use the numbers 9.2 and 2.8.

Another example was strategies for finding the area of trapezoid. We talked strategy. Later in the day, we spent time talking about and practicing use the area under a velocity vs. time to find displacement.

There are a couple of things to note:

• I want students to feel like they invented or at least own the strategies. For this reason, problems with multiple strategies or strategies for checking will be most useful.
• I think it works best when you start with something intuitive (what’s in between 10 and 20?) before moving to the less intuitive (9.2 and 2.8).
• That said, building up to the abstract is made possible by emphasizing the strategies (not answers) and strategies for checking. So you as teacher have to emphasize the strategies students are using. Give them names if need be. The idea is to formalize the intuitive strategies.
• I found it useful to play up the possibility that the strategies might not work  in new situations even when I knew they were going to (e.g., Do we think this strategy will work with negative numbers? Or will we need a new strategy? Everyone try out the strategy first, and then check to see if it works.”  This will be important later, because later students will likely try to find average velocity later in ways that won’t work (e.g., averaging two constant speeds that were maintained for different times). Since you’ve already built up the notion that strategies may only have limited usefulness, students are poised and ready. When it happens (and it will if you provide them an opportunity), you can say, “Cool, you just discovered a situation where our strategy doesn’t work? … How did you do it? How is this situation different that the ones where it does work? Why didn’t it work? I wonder if there is another strategy that would work… etc.).
• In practice, I have been explicitly having students strategize with math problems that will involve same numbers in our physics problem. So our trapezoid problem had the same dimensions as the one in the physics problem. The initial and final velocities in our physics problem were the same as one of our math problems. I’m not sure if this is necessary, but it seems to have some benefit. A few students notice right away as you are setting up or working on the problem. They go, “Oh, this is just like the trapezoid we saw earlier.”  I think it makes those students feel insightful and, for others, it  makes the problem a little less intimidating. I suppose it may even reduce cognitive load.

I imagine a lot of people do things like this. I certainly have, but there’s also a sense in which I haven’t. I’ve done it before, but I haven’t necessarily done all the things necessary to pull it off: designing good problems that build on each other and toward physics, managing the discussion to focus on strategies not answers, and celebrating accidental discoveries of finding out that a strategy didn’t work in a certain context.

The voice of a real teacher:

“It was uncomfortable and sometimes I couldn’t tell if I was ‘doing it right.’  “

“I spent a lot of sleepless Sunday nights, worrying that I wasn’t good enough to pull this off and that I’d mess up my students’ minds, or at least their careers.”

“I’m still amazed at how much more confident and less anxious I felt two years ago, when my teaching was demonstrably weaker. “

“It’s been an interesting lesson to me about why not all teachers are chomping at the bit to dive into this pool.”

I’ve made it an important goal of mine to meet Mylène in person. Seriously.

I’m teaching a summer physics course. One thing I’m doing differently this year is having students do at-home experiments with friends or family. Part of their reporting back involves having to share the ideas of their friend or family member. Here are a few quotes from students discussing what happened when they dropped a book, a piece of paper, and crumpled up piece of paper. Just for context, we discussed this situation on the very first day of class, but we haven’t resolved any of it, and won’t for at least another week. We haven’t even talked about forces yet.

“My friend basically said that the book would for sure hit the ground before the paper did in both of the instances. She thought that because the book is obviously heavier and either way the paper is lighter whether it is flat or in a ball. I would explain the book and paper hitting the ground by saying that when the paper is in a ball then the air isn’t playing any part on the paper. The book and paper have the same gravitational pull.”

“I asked my roommate also in college at MTSU to watch the experiment and give me her ideas on what she thought would happen. Before dropping the flat sheet of paper versus the book she stated that the book would drop first because it was heavier. For the next stage of balling up the paper she stated that the book would still hit the ground first, also that the paper would be closer because we decreased the surface area. She was shocked that they hit at the same time we even tried the experiment with different books to see if the weight of the book would make a difference. As far as my self I am still amazed by this trick. I feel that my friends theory would still hold true for the book versus the flat sheet of paper. I feel that when the paper is in a ball that it has something to do with it being more aerodynamic but other than that it still puzzles and amazes me.”

“I don’t have any roommates or anyone nearby to run this experiment with, but i know most of my friends would have the same reaction as me with the exception of one friend who is in physics right now too. i have no idea how i would explain them hitting the ground at the same time. i guess because the gravitational pull on an object is constant and has nothing to do with the weight of the object, so all you would have to consider between two falling objects is the air resistance or any other resistances on the way down to the ground, and since the uncrumpled paper is far more susceptible to the air force it gets slowed down until it is crumpled into a ball.”

“I did this to my dad. He thought the book dropped first because the paper had air resistance which made it drag. When I crumbled up the piece of paper, he thought that they would hit the floor at the same time which did indeed happen. He said they hit at the same time because the paper has less air resistance all crumbled up. I think the crumbled paper and the book hit the floor at the same time due to not only less air resistance on the paper, but the gravitational pull on both the objects being the same, 9.8 m/s^2 (free-fall)”

“My sister said the book would hit first because it is heavier. When I asked her what would happen with the crumpled up the paper she changed her mind about her first response. She said that the book would hit first the first time because it has less air resistance. I agreed with her about the air resistance being the reason why the book hit first when the paper is flat”

“My mom thought that the book landed first because it was heavier than the piece of paper. Then I asked her what she thought would happen if I crumbled the paper up and she said the same thing. I then explained to her that the gravitational pull remains constant.”

“They said the book falls first because it is heavier than the paper therefore it falls first. when i crumpled the paper they said the book would still hit the ground first because the book is still heavier than the paper. I would explain why the book and paper hit the ground at the same time by explaining that gravity is the same on the book and the paper but since the paper is crumpled up there is less surface for the air to hit and force it to just float down so it falls straight down.”

“Their prediction was that when the paper was crumpled, it might wall slightly faster than it had before, but not as fast as the book, as the paper “became more directed” and “gained more mass when crumpled” …explaining why the book and paper hit the ground at the same time is really not very simple without an understanding of air as a substance. With the book and paper having the same surface area, the book had more weight (mass attraction to earth) and therefore accelerated longer before it could reach a terminal velocity, or equilibrium of resistance to gravitational force. The paper however achieved terminal velocity quickly as its low mass and as such its gravitational attraction, was offset faster. however when the paper was crumpled it lost surface area reducing the force of air on the paper, so even though the paper had the same mass as before, it takes a longer time to reach terminal velocity and accelerates for the most part with the book. ”

“I showed my friend that the book fell first in the first part and she said that it is because the book weighed more. She said that the same thing would happen even if the paper was crumpled up, the book that weighs more would fall first. After seeing that they both hit at the same time she explained that the paper was more aerodynamic the second time and allowed it to fly through the air at the same time as the book. I believe they hit the ground at the same time because the surface area compared to the weight of the object is a factor of resistance. Now that the large surface area is eliminated on the light piece of paper, it could fall at the same velocity as the book.”

What I’m loving about this is that now I have moms, dads, boyfriends, friends, brothers, sisters, roommates in my classroom. Their ideas reaching in and tugging at the idea space our classroom. Our next at-home experiment is the running key-drop. Stay tuned.

In a post about the value of teaching Newton’s 3rd Law from the get go, Greg Jacobs discusses how he deals with possibly accomplished students who complain having to label forces a certain way,  which may seem laborious to students who “get it”.

So when the bright kid says, “Mr. Lipshutz, I know this, why are you making me waste my time writing silly extra words on my homework,” how do I react?  I start by wondering why it’s such a big deal — if he knows what to write, why is it so horrible to take a moment to prove that to me?  I might appeal to the idea that I want all of our problems to look similar, so that the class can help each other more easily.  I might be transparent about my pedagogy, giving an impromptu Newton’s third law lecture to show the benefit.

 In the end, if the student pushes my patience, the answer is, “Because you’ll lose points if you don’t.  You may drop the class if you think this requirement is overly onerous.

I think the issue here for me is that such a response represents to me a subtle confusion among three things: evidence of understanding, evidence of misunderstanding, and lack of evidence for either.  And I think this is where standards-based grading may come to the rescue. In a grading system where you take away points, evidence of misunderstanding and lack of evidence for understanding are both punishable offenses.  Standards-based grading, however, focuses our attention to confirming evidence of understanding.

I was trying to think about what Kelly O’Shea might do in this grading predicament. Kelly, I think, would grade such a skill with a “-“, meaning that the student work provides no meaningful data concerning this students’ understanding of the skill or concept. The students isn’t punished for not labeling things the way you want them to; they simply can’t be given credit for understanding things for which they have provided no evidence. Maybe they will show that evidence later by labeling forces the way you want; or maybe they will show you evidence of understanding in a different way.

What do you all think?

What Jason Thinks: http://alwaysformative.blogspot.com/2012/06/burden-of-proof.html

I found myself reading the following exchanges about calls for politicians to take standardized tests

http://dianeravitch.net/2012/06/01/a-solution-to-the-testing-mania/

http://blog.mrmeyer.com/?p=14184

http://function-of-time.blogspot.com/2012/06/untitled.html

You might find that back-and-forth interesting…

But if you really want to read something, read this:

http://grantwiggins.wordpress.com/2012/05/22/on-accountability-part-2-how-to-do-it-right/

After my first semester here, I gave the break down of my teaching evaluations. Here I’m at it again.

More importantly, I’m going to follow up this post with another post that is more reflective on this past year has gone (in terms of teaching) and how I see this year’s experience informing goals for improvement next year. But I wanted to get the data down first.

So, here it is. Below is a summary of my teaching evaluations for both my inquiry and physics courses–course that I taught both fall and spring semester. In each category, I show the average rating for inquiry, physics, and the department. In parentheses, I show the change in my ratings from last semester to this semester. In every category, the ratings either improve or stay the same. With respect to department averages, my evaluation for my physics class are near ceiling. In inquiry, there are two categories that I am still below departmental averages, but both showed significant improvement.

Presentation:

Inquiry 4.7  (+0.3 from fall to spring semester)

Physics: 4.9  (+0.0)

Department: 4.2

Organization:

Inquiry: 4.3  (+0.4)

Physics: 4.8 (+0.0)

Department: 4.1

Assignments/ Grading

Inquiry: 4.3 (+0.4)

Physics:  4.9 (+0.1)

Department: 4.4

Scholarly Approach

Inquiry: 4.2 (+0.2)

Physics: 4.8  (+0.1)

Department: 4.2

Student Interactions:

Inquiry: 4.6 (+0.1)

Physics: 4.9 (+0.0)

Department: 4.0

Motivating:

Inquiry: 4.6 (+0.4)

Physics:  4.7 (+0.0)

Department: 4.1

Effectiveness/Worth

Inquiry: 3.5 (+0.5)

Physics: 4.6  (+0.1)

Department: 3.8

From “Lessons From the Physics-Education Reform Effort” by R.R Hake

As previously indicated, the data of Fig. 1 show that seven of the IE courses (717 students) achieved <g>’s close to those of the T courses. Five of those made extensive use of high-tech microcomputer-based labs (Thornton and Sokoloff 1990, 1998). Case histories of the seven low-<g> courses (Hake 1998b) suggest that implementation problems occurred. Another example of the apparent failure of IE/high-tech methods has been described by Cummings et al. (1999). They considered a standard physics Studio Course at Rensselaer in which group work and computer use had been introduced as components of in-class instruction, the classrooms appeared to be interactive, and students seemed to be engaged in their own learning. Their measurement of <g>’s using the FCI and the Force Motion Concept Evaluation (Thornton & Sokoloff 1998) yielded values close to those characteristic of T courses (Hake 1998a,b,c). Cummings et al. suggest that the low <g> of the standard Rensselaer studio course may have been due to the fact that “the activities used in the studio classroom are predominately ‘traditional’ activities adapted to fit the studio environment and incorporate the use of computers.” Thus the apparent “interactivity” was a product of traditional methods (supported by high technology), not published IE methods developed by physics-education researchers and based on the insights of cognitive scientists and/or outstanding classroom teachers, as for the survey courses.

This quote and similar ones from Hake have been on my mind these past few weeks as undergraduate students I am working with are grappling with the question, “Why are our learning gains on the FCI lowered than expected and desired?” This question emerges out of our learning about the FCI, analyzing our local data, and comparing to other research and outcomes that have used the FCI.

So, in our courses, we use peer instruction methods, collaborative-problem solving with whiteboards, and computer facilitated learning. Our class is mostly “flipped” (where students read lecture at home and practice problems in class). Most of it takes place in a studio setting. It sounds and often looks very interactive. On the other hand, all the content students are interacting with– from the textbook, to the laboratory activities, to the questions and problems students work on in groups–is home-grown. For better or worse, these home-grown materials would probably be characterized as “traditional activities adapted to fit the studio environment and incorporate the use of computers.”

As a group of researchers, we are approaching our question in a variety of different ways:

(1) One students hopes to ask instructors in our department to take the FCI, not by answering how they would answer, but by choosing the answer they think would be the most common incorrect answer chosen by students, and to estimate the percentage of students answering each questions correctly after the course. He hopes to compare instructor expectations to reality, in order to answer question like, “How knowledgeable are instructors of specific content difficulties students have and how aware are they of the prevalence of those difficulties in our courses?”  We have been reading a lot of pedagogical content knowledge.

(2) Another student is interested in examining student learning in relationship to our home-grown textbook. Does the book explicitly address specific difficulties we know about from our own data and research in physics education? Does the book implicitly reinforce any difficulties? Does it provide opportunities for developing conceptual understanding as well as problem solving? He is also interested in questions like, “Do students actually read the text? How much? How deeply? In what do they engage with text? What do they actually take away from the reading the text, and how does that play out in relationship to classroom instruction?” … We have been reading a lot about self-explanation, preparation for future leraning, refutation texts, and the influence of prior knowledge (e.g., misconceptions) on reading comprehension.

(3) Another student is interested in examining structural factors of instruction that might be contributing to lower than expected FCI gains, including

• Student background and academic preparation
• The prevalence of exam questions that probe (and hold students accountable) for conceptual understanding
• The quality of apprenticeship and training that undergraduate TAs and new faculty receive for teaching using interactive engagement methods
• The strategies that instructor use to motivate and cultivate a classroom culture in which IE methods are taken seriously.

We have been reading papers about the kinds of background that correlate with FCI scores, as well as papers about programs that have successful or unsuccessful implementations of reform-physics-curricula.

This work, for better or worse, treads on a sensitive arena–a close examination of ourself.  The fact that this work is being carried out by students, I think, could be perceived as making this endeavor even more sensitive, but in an odd way it makes it authentic. All of these students are really interested in improving instruction here, doing research that is valid but also relevant to local stakeholders. They have no axe to grind or hidden agenda. We are also just genuinely intrigued by the puzzle, and curious to pursue its potential solutions. Some of that solution, will not doubt in my mind, need to be geared toward improving the curriculum at the content-level–the content as embedded in all the tasks we ask students to engage with, from the text, to the labs, to the questions and problems they work on. Some of that solution will no doubt be about getting our department on board with the continued renewal of that content based on assessment, feedback, analysis, and ongoing revisions. In that sense, the work we are beginning serves a launching point for what will need to become an ongoing endeavor.