Assessing and Grading with Answer Pairs

This pair of questions was on the multiple-choice test my students took:

A rock is thrown vertically upward, slowing as it rises until it reaches it high position, where it stops momentarily before falling back to the ground. Take the positive y-direction to be upwards.

1. Immediately after the the rock is thrown, its y-component of acceleration is _______

A. Negative

B. Zero

C. Positive

D. Not enough information to tell

2. Just before the rock hits the ground,  its y-component of acceleration is _______

A. Negative

B. Zero

C. Positive

D. Not enough information to tell

For all 250 some students,

30% of students answered negative for both (NN): To me, the answer pair might serve as a proxy for understanding acceleration as a change in velocity and an understanding of signing conventions for vector quantities.

25% answered negative on the way up and positive on the way down (NP): To me, answer pair might serve as a proxy for understanding acceleration as change in speed, with student thinking that negative signs mean slowing down and positive signs means speed up.

30% answered positive on the way up and negative on the way down (PN): This pair serves as a proxy for not having disentangled velocity (speed) and acceleration

15% for all other combinations: This serves as a collection of students who either misunderstood the question or have other confusions about concepts.

How were these questions graded?

As is, each questions was graded independently. This makes it so that answering negative on either question gets you one answer correct. To me, this make no sense, because students’ answer to any one question is meaningless in terms of what they might know about acceleration. From the MC-question alone, we have evidence to support a claim that about 30% of students understand the concept fully, but 55% of them are getting some points. Additionally, we have evidence to support a claim that 25% understand the concept partially, but 55% got partial credit.

How might I grade this question?

I might grade this in terms of the pair combinations, and give more points to the student who answered NP than the student who answered PN. My reason for this would be that thinking of acceleration as denoting a change in speed is further along than someone who still has to figure out that acceleration is different from velocity (or speed).

The other option I would consider treating the two questions as one question, and only give credit for NN pair combinations, and give no credit for any other answer pairs.

Whether I did one or the other would depend on what I was trying to assess precisely about their understanding of acceleration.

What do you think?

Addressing Student Concerns:

Today, we took a break from our inquiry into light to do a mini-inquiry into motion and to inquire into childrens’ thinking about motion by watching and discussing a video-case study. We also took some time to examine and discuss some of National Standards and AAAS Benchmarks about what 2nd and 5th graders are expected to be able to do and understand about scientific inquiry. Most of that went pretty well, although we were all a bit zonked out after looking at a bunch of standards. Wednesday, we are back at light for just another day or two. I think we’ll all be happy to wrap up some of the loose ends and have a fresh start at something new.

After class, I got to read over their anonymous feedback about how class is going for them.  The range of things students bring up is quite broad, but the big three complaints are:

(i) Concerns about getting an A in my class and frustration at my Capstone A grading policy

(ii) Not getting the big picture of what we are doing and what they are expected to learn (hopefully I addressed some of this today)

(iii) Annoyance and frustration toward students who have a negative attitude about class

There were a range of other complaints including, boredom, not wanting to work in groups, wishing we were covering more topics, wanting to know the answers, complaining about the workload, being sick of light as a topic.

On Wednesday, I want to re-explain my Capstone policy and try to frame it in a more productive manner. I also want to talk about the exam that is coming up to reinforce what this class is about in the big picture. On the exam, students will be asked to do what we have been doing a lot of, namely

Explain their thinking about some physical situation, using words and diagrams to come to a prediction.

Make some observations and reconcile their prior thinking

Construct plausible alternative predictions and respond to them

Situate their explanations and ideas within emerging classroom ideas.

They’ll be assessed on what I’ve been assessing them on- clearly articulating your thinking, constructing arguments and counter-arguments, reconciling one’s ideas with new observations, etc.

Lastly, on Wednesday, I want to convey to them my (re)commitment to creating a prepared learning environment that can be fun and engaging. I do think class should be intellectually fun in ways that instill curiosity, puzzlement, wonder, and awe. I’m going to leave complaint #3 alone and just remind us all that at the beginning of the year, we all agreed that it was (as a class) we have responsibility to come to come to class with a good attitude and an open mind. I’ll work harder at making class engaging, they’ll work harder at allowing themselves to become engaged.

Note to my future self about the first day

Brian,

I’ve been thinking a lot about my inquiry class this weekend. At some point I remembered what I actually did on the very first day of class the last time I taught an inquiry course for elementary school teachers. Instead of jumping into a month long inquiry into a topic, our class did an hour long inquiry into one physical situation and then we watched a video of children talking about the very same situation. In retrospect, this was a better decisions than I knew at the time.

First, it likely helped my students frame the science learning (I was going to ask them to do) in terms of their future careers as teachers. In particular, it sent a strong message about the potential connections that exist between their inquiries and children’s inquiries. I also, unknowingly, picked a topic and situation that adults and children tend to think the same about. Note that this isn’t always the case. Kindergarteners ideas about magnets, for example, tend to be centered around magnets being “energized”; whereas adults tend to think about magnets in terms of electric charges. Both,as we know, not how scientists think about it. But, importantly, there are many topics where children and adults tend to think the same. In the particular situation I asked them to reason about, it is likely the case that 4 or 5 ideas will always come up, and those will be the exact same 4 or 5 ideas that come up in the video. The video  shows children having these ideas, sharing these ideas, and listening and responding to each others’ ideas in a sophisticated way. Thus, what we do on the first day (and what the children do) serves as a model for what I want them to do over long periods of time during a sustained inquiry into a single science topic.

Another big advantage is that we get to talk about the fact that they were able to easily make sense of what the children were doing and saying, and that this might have been because they had just spent an hour thinking, listening, and sharing their own ideas about the situation. This way, I get to let a mini lesson and video make the case that they will better understand their children by doing science based on their own ideas and not simply memorizing canonical science understandings. That way, when we are 4 weeks deep into a muck of ideas about a topic like light and shadow, I can remind them why we are doing this. We can even take a break to inquire again into childrens’ thinking.

Next time you teach this course, come back an re-read this post. It has lots of good ideas, at least I think. Best of luck next time.

Brian

2nd Draft of Inquiry Standards

I decided to rewrite and organize my ideas about some possible standards for my inquiry course. Right now I’m only focused one strand:

A.  Creating, Supporting, and Developing My Own Ideas

1. Articulating Ideas
Generating Ideas
A1.1  I think about my own experiences in the world (looking for possible connections), as a source for ideas about how the world might work
A1.2 I ask questions about how the world works and discuss my own curiosities about the world
Expressing Ideas
A1.3 Given time to reflect and write, I can articulate my own initial thinking and ideas about phenomena in clear and specific ways
A1.4 I express my ideas and thinking through diverse representations that enhance or clarify my thoughts (e.g., drawings, sketches, diagrams, graphs)
2. Supporting Ideas

Evidence-based Support
A2.1 I consider supporting evidence or examples when making claims or presenting an idea.
A2.2 I go beyond citing evidence by detailing the rationale that explains why I think evidence supports claims and ideas.
Theory-based Support
A2.3 I can explain how different ideas that I have seem to ‘fit’ together in a logical way.
A2.4 I attend to the implications of my own ideas and articulate how those implications lead me to make certain conclusions or predictions
3. Sorting through Ideas

Theoretical Progress
A3.1 I look for inconsistencies among the various ideas that I have and I can compare and contrast competing ideas I have
A3.2 I seek to resolve inconsistencies among the ideas I have …
Empirical Progress
A3.3 I seek ways to put my ideas to the test through new careful observations and/or experimental design
A3.4 In the face of evidence that run counter to my ideas, I return to my ideas to ask questions and reconsider my thinking.

4. Maturing Ideas
Explanation
A3.1 I can provide explanations that tell a gapless story which detail exactly how specific conditions give rise to certain outcomes
A3.2 I can use diagrams to carefully work out the implications of ideas and use such diagrams to explain phenomena
Awareness
A3.3 I can explain the history of previously held ideas and the reasons for abandoning them
A 3.4 I am aware of my own level of commitment to ideas and can explain why in terms of evidence, arguments, & theory.
Anyway, here’s to progress in my thinking about this. I have a long way to go.

Today’s Sarcasm: Re-writing the Intro Physics Book

Chapter 1: Vermin and Ribbits

Here are two important things you need to know. There are ribbits and vermin. Ribbits tell you how much. Vermins tell you how much and what direction. Through the whole semester, you will need to think about the difference between ribbits and vermin.

Chapter 2: Yolks and Whites

Here is an equation

Y = Q/ R … if I ask you for Average Yolk, please find Q, find R, and plug those numbers in for Y. Q and R are ribbits, so Average yolks are ribbits, too. Please note that Q = Q1 + Q2 + Q3… and R = R1+R2+R3+…

Here is another equation

W = (q1-q2) / R … if I ask you for Average Whites, please find q1, q2, and R and plug those into the equation for W. (q1-q1) is a vermin, so W is also a vermin.

Don’t confuse Yolks and Whites. This would be a very easy mistake to make. Remember that yolk is a ribbit, which is always positive, while Whites are called vermin, which could be positive or negative. The positive and negative tell you the direction of the vermin.

Chapter 3: Changing Whites

There are four equations relevant to this understanding changing whites.

Equation 1: q2 = q1 + wi u + 1/2 s u²

Equation 2: wf = wi + s u

Equation 3: wf² = wi² + 2 u (q2-q1)

Equation 4: q2 = q1 + (wi +wf)/2 * u

We call “s” the snappy-juice, and wi and wf the starting and final whites. Don’t confuse starting whites and final whites with average whites. Average white is an only average, but starting whites and final whites are instantaneous. Get it? Great.

If you notice, there are six different letters above. q2, q1, wi, wf, u, and s. These are the six quantities you are going to have to always think about and list. If you notice, each of the above equations is missing at least one of these six quantities.

Equation 1 is missing wf

Equation 2 is missing all q’s

Equation 3 is missing u

Equation 4 is missing s.

This makes life really easy. Anytime I ask you for something, just list the six quantities and write next to it what I’ve told you. If I didn’t tell you something, list that letter as well, but put a question mark next to it. Here’s what you should do next: find the equation that’s missing the letter you don’t know but I haven’t asked you for. Picking that equation will ensure you will have no trouble finding what I’ve asked. Please then manipulate that equation to isolate the quantity you don’t know but I’ve told you to find, and then plug in the values I’ve told you. Listen carefully  to what I’ve told you will be really important. If you don’t, how could you possibly pick the right equation? Get it? Good.

Footnote: Listing knowns and unknowns is an expert problem-solving strategy. It’s what all experts do. Ask an expert. If you ask them what they do, they will certainly say, “I list what I know and don’t know, before I start thinking or proceeding to do anything.” This is why we are training you to do this. Even if you never have to learn physics again, you will know how experts solve problems. This will be really beneficial to you in your life.

Chapter 4: A special case of changing whites

OK. When I say a problem is about funfare, you should know that the vermin of s is b = 42. B is merely the ribbit of the vermin. B, the vermin, always points down because the flabber due to the eagle is down.  B, the ribbit of the s vermin, is always +42, and it would be wrong to think of b as -42. Rather sometimes s will = -b and sometimes +b, but b is always 42. You will decide whether s = +b or s = -b, on your choice of whether up or down is the positive direction for vermin. Get it? Good.

Now you know how to solve funfare problems: just do the same thing you did in chapter 3, except you know that the ribbit of s is 42.

Footnote: You may be tempted to think that b is not always 42. That would be wrong, all funfare problems have b = 42.

Frustrations and Rebellions: Why didn’t I listen to Joss?

On top of a bad day of teaching last Wednesday, I also had to manage a mini-rebellion from some students in inquiry class:

Several students are unhappy about a few things, including

(1) That I use a Capstone Project for an A. In my course, a 100% for notebooks, tests, homework, and participation is the same as an 83%. It gives you a B and makes you eligible to get an A by doing an independent project.

I see it this way: You can bomb 1/6 of my course and still get an A. This puts in lots if wiggle room for mistakes, for missed assignments, struggling early on, etc.

Students see it this way: If I get an 100%, I still can’t get an A.

I see it this way:  Independent projects are optional. You can pass this course without doing one.

Students see it this way: To get an A, the teacher makes us do things that are optional.

(2) That this is an inquiry course and its different from other parallel classes. In my course, we cover three topics, so we spend a lot of time thinking, discussing, sharing, doing investigations, whiteboarding, etc. In other parallel courses, students cover a new topic each day or each week. Students simply aren’t used to classes like this and it doesn’t help my case when other classes are different. I understand that learning in a new way can be frustrating, especially when you’ve mastered the routine of (1) taking notes, (2) doing homework, (3) passing exams. There are definitely some students who are very frustrated, and I think they are frustrated for different reasons. Some are frustrated because they don’t know where we are going as a class and if we are really learning and making progress. I know that they are learning and that we are making progress, but it’s true that I don’t know exactly where we are going. Other students are frustrated because they wish they were in the other class, where it fit within their comfortable model of what a science class should be.

(3) That we are spending too much time on science In other parallel courses, instructors enrich the course by having students write up lessons plans and having students share them with each other. I am choosing to enrich my course by talking about the nature of science more and by watching videos of children doing science. I would say my course is 70% inquiry into science, 20% inquiry into the nature of science, and 10% inquiry into children’s thinking about science. Other courses are perhaps more like 60% activities related to science and 40% activities related to teaching science. Students are unhappy that the other courses are about teaching science,  where they get to collect, make, and share lesson plans; whereas my course we mostly do science.

I understand their frustrations. I also understand that not every student feels the way these students do. Many students have told me that they really enjoy the course. The Joss in my head, however, is reminding me that I didn’t spend enough time in the beginning of the course selling it–explaining why and convincing students that this way of teaching is in their best interest. So next week, it’s time to talk a little more about why I grade the way I do and why I am running the class the way I am.

First Draft: SGSI Standards?

What would it look like if I tried to do standards-based grading in my inquiry course? That’s a question I’ve been thinking about a lot. Here’s a quick rough draft of some more precise learning goals.

I. Your Own Thinking

Creating Ideas

I reflect upon my own experiences and explore my own thinking as sources for possible ideas about how the world works.

I articulate my own thinking and ideas about phenomena in clear and specific ways that help others to understand them.

I ask questions about how the world works and document those questions as they arise

I express my ideas and thinking through drawings, sketches, diagrams, graphs, etc.

Developing Ideas

I seek out evidence to support claims that I make, and use evidence or examples to support or refute claims.

I go beyond just citing evidence by providing the rationale that explains why evidence either supports or refutes claims.

I attend to the implications of my own ideas and articulate how those implications lead me to make certain conclusions or predictions

In the face of evidence that run counter to my ideas, I return to my ideas to ask questions and reconsider my thinking.

I develop explanations that tell a gapless story that detail exactly how specific conditions give rise to certain outcomes.

Monitoring Ideas

I return to and follow up on questions I have had and continue to have

I monitor how my ideas and thinking change over time and compare and contrast ideas I had at different times

I look for inconsistencies among the various ideas I have and compare and contrast competing ideas I have

I explain things that are confusing to me in ways that help others understand exactly where my confusion lies.

II. Others’ Thinking

I listen to my peers as source for ideas about how the world works.

When I don’t understand someone else’s idea, I inquire either by asking questions, trying to summarize what I thought they said, etc.

When I understand others’ ideas, I show my understanding of those ideas by writing or talking about them

When I understand another person’ ideas, I follow the logical implications of those idea (even when I disagree)

When I disagree with an idea, I provide a critical perspective toward this idea by constructing counter-arguments, citing contradictory evidence, or finding the flaw in some reasoning or premise.

I construct plausible counter-arguments by attending  to others’ ideas or thinking through their implications

I respond to counter-arguments by attending to the argument itself–perhaps by attending to some inconsistency in the reasoning or the faultiness of some premise.

III. Accountability to Community Norms

I can think and reason in ways that are consistent with our class’ foothold ideas

When my thinking departs from our class’ foothold ideas, I recognize this to be the case and point it out.

Things left out so far:

Distinguishing Observations vs. Inferences?

Designing an experiment? Tinkering?

Careful construction of diagrams?

Organization of writing?

Constructing clear definitions?

Clarity of writing?

Reflections on a Bad Day of Teaching

On Monday, I wrote about some of the theories and puzzles about light that my class is making contact with. I was excited about their ideas and excited about their puzzles. In this post here, I want to tell you about the mistake I made in moving forward. I had decided that my goal for class should be to help them bridge the gap between their ideas and their puzzles. My idea was to give them a question that would act as a stepping stone.

First, it  is safe to say that didn’t go all that well. Not at all. Students really struggled to understand what I was asking them to do. They struggled to apply some of the ideas they or others had come up with. They struggled to work out their implications. They didn’t see why I was asking them to do this. In general, the day was also just a combination of boring or frustrating–not just for them, but for me as well.

No point lamenting over a bad day. So, the question is why? I came up with three categories of things I failed to do: anticipate, implement, and, finally, maintain the big picture I have for this course, which is student generated scientific inquiry.

How I failed in anticipation:

I totally underestimated the difficulty of the task, both in general and for my students.

I totally underestimated how we (as a class) understood the ideas I was asking them to apply. These ideas were brand spanking new.

I underestimated their propensity to carefully construct diagrams, which would be necessary for carrying out the task.

I underestimated the difficulty we would have in agreeing about what we observe.

How I failed in implementation:

I let my own exhaustion be at forefront of my classroom demeanor. I had worked at 14 hour day the day before and was both mentally and physically fatigued. I was certainly not at my best in terms of of patience, listening, and empathy. Rather, I came off as tired, disinterested, and impatient. I do think that good demeanor can go a long way in diffusing a troubled lesson, but I was making a bad lessons worse.

Second, I forgot that my whole plan was to talk about careful diagramming AFTER the activity. Instead, I became a nit-picker during the activity. I pressed too hard and obnoxiously with several student groups. I felt like I was nagging students, which is not where I want to be. By not sticking to my plan, I let the worst of my classroom demeanor snowball into frustration for everyone.

How I ultimately failed in vision:

Ultimately, the failures above are trumped by the two big failures:

My activity was inherently about deception. In my plan, I wasn’t going to tell them that they had really specific and strong theoretical ideas, which I thought they could try to use to explain the puzzles they are interested in. Instead, I concocted some secret plan to sneak them from where they are to where I wanted to be. To me, deception in teaching is something to be avoided; and I fell for the trap, not only in the moment (which happens), but by design (which is scary). Designing deception into my curriculum is far far from where I want to be.

Lastly,the activity  inherently took agency and authority away from my students. Instead of letting them continue to pursue their great ideas, questions, and puzzles, I strangled them in an activity that I thought was more worthwhile than their own curiosities. And I was wrong to do so, and wrong about the activity. If I really thought there ideas and puzzles were so great (which I do), why didn’t I let them run with them until they really needed me to step in.

So what next?

I don’t believe I did any permanent harm, but there is certainly some repair that’s going to be needed. Fortunately, I have a 5 days to think this over.

What am I going to do now? Nachos and beer.

Inquiry into Inquiry

As I’ve mentioned before, MTSU is a UTeach replication site, named MTeach. Among many things, MTeach offers two one-credit courses called “Step 1” and “Step 2”, where students get to dip their toes in the water of math and science teaching. In “step 1”, student pairs get to make several observations and co-teach several inquiry lessons in a local school as they are learning about inquiry teaching. What is nice is that they get to do so with the same teacher and classroom throughout the semester.

In “Step 1”, teachers choose from a list of maybe 30-40 possible lessons that the student pair will have to learn and then tweak to fit their class. Each pair gets several opportunities to practice teach their lessons and get feedback before going off to the school. With over  a hundred students in MTeach, their are many students to help. In the past week, I’ve had a chance to talk to several step1 and step 2 students about their lessons.

Their lessons are structured using 5E’s: Engage, explore, explain, elaborate, and evaluate. So far, my concern with the 5E’s is they seem (for students) to have little to do with student thinking and learning, so they easily get interpreted as “Get students’ attention with something cool”, “Get students to do something”, “Tell them what they should have learned”, “Have them practice what you told them”, “Give them a quiz”.

Here are what my steps might be:

(1) Inquire: Do something that will help you find the boundaries between what your students know and what they don’t know.

(2) Perplex: Usher students to that boundary in a way that engages both their ideas and curiosity

(3) Explore: Provide a structure for students to explore that boundary in a way relevant to your learning goal

(4) Listen: Listen and watch what your students do; structure an opportunity for students to share and discuss what they’ve found

(5) Connect: Help students make connections between their ideas and the formal concepts they are making contact with

(6) Solidify: Help students permanently stake out this new territory through elaboration, practice, extension, reflection, etc.

IPELCS is not as snazzy as the 5E’s, but at least it is about things we know are important, including formative assessment, prior knowledge, zones of proximal development, epistemic agency and authorship in the classroom, disciplinary ways of understanding, scaffolding, and meta-cognition.

Dont’ get me wrong. I am  absolutely certain that any attempts to teach “IPELCS” would be reinterpreted by many students as “give them a pre-quiz”, “show them an exciting demo”, “let them play with the demo”, “tell them what they demo is about”, and “give them a practice worksheet”. I’m not suggesting that we go out and teach students IPELCS, because, ultimately, the problem with trying to teach anything like IPELCS or the 5E’s comes from not following its own advice. The issue of how to best engage students’ ideas about inquiry teaching and to usher them to the boundary of their own understanding is not a simple one.

Instructional goals, instincts, and decisions

This post is about instructional decisions in my inquiry course, which I am trying to be more explicit about.

If you read my last post, my class has some theories about what light does when it reaches different surfaces. They also have identified some challenging puzzles involving light on surfaces. Their theories are tentative and their puzzles are complex. I’ve decided that it’s my job to help them bridge the gap between some of our theories and some of the phenomena we’d like to explain.

My specific plan of action is based on at least  two notions I’d like to make explicit:

I think my students need some practice “holding a specific theory in mind” and working out its implications. So far, we’ve been doing a very different kind of scientific work–trying to sort through a swamp of our own ideas and experiences to reach some understanding of what’s going on. Along with this work, we’ve been changing our minds a lot, because we keep having new observations or ideas to contend with. To me, it’s about time we start “trying on some ideas” or at least “committing to some ideas just long enough to see where they lead us”.

Conceptually, I also think they need to be nudged toward the vicinity of “source-object-receiver” models. I have been trying to push various groups in this direction during conversations here and there, but so far, no one seems to be concerned with it. Our ideas are about where light goes, and much less with who can see what from where. This is going to be pretty important if we are going to get anywhere with our puzzles, which specifically concern what different observers see.

In summary, my goal is to narrow the gap between theories and puzzles. We have a lot of good theory and very interesting and revealnt puzzles. I also have two instructional instincts: these students need to make contact with a new kind of “scientific practice” (following implications) and a new “conceptual idea” (source, object, receiver). I also have some other goals, which function as constraints. For example, I want to honor and value the class as author’s of ideas and nudge them in directions that at least seem continuous with what we’ve been doing so far.

Anyway, here is my first draft, for what we’ll be doing on Wednesday. They have some predictions to make based on below, some things to discuss, and then we’ll be moving forward with some observations. My final goal for the day will be to pin down some “community rules” for drawing diagrams, and maybe have a conversation about the difference between a sketch (which might get across the gist) and a diagram (which could be used to make a specific prediction). Anyway, what do you think?

Blog at WordPress.com.

Up ↑