Skip to content

A paper for the next year or two: The Long Hard Slog of Improving Student Learning Outcomes

June 2, 2018

A paper (or more likely a letter) that we will get around to writing at some point is the narrative of the long arc of how we went about making changes to our algebra-based physics course.

The short version of the story is that over the course of a decade, the department implemented many popular research-based physics teaching strategies with no measurable effect on student learning gains as measured by the FCI. Our normalized learning gains held strongly in the 0.20–0.25 range (with no individual class ever scoring above ~0.3), despite efforts to use research based teaching strategies such as

  • Collaborative problem solving (white-boarding)
  • Peer Instruction (clicker questions), and
  • Learning Assistant led Tutorials

In response, a more comprehensive curricular overhaul was developed over a 2-3 year period. Early piloted versions with developers and select instructors achieved normalized gains in the 0.40 – 0.60 range. Our first year of full implementation (with ~10 different instructors across 20 different sections) achieved an average normalized gain of 0.46.

The perhaps interesting / challenging part of the story is that pretty much the exact same teaching strategies listed above (for the older course) also describe the new course structure. From a bird’s eye view, nothing about the course changed. Perhaps not surprising, but worth unpacking, is how improved learning outcomes were maybe achieved through a combination of efforts to (1) develop/adapt instruction to local conditions, (2) design for curricular coherence, (3) and provide on-the-job professional development.

Our story I think has relevance to broader physics education community for a few reasons:

1. Our experience with several failed efforts to improve learning outcomes through popular research-based strategies is probably common just under reported. I do not take this to mean that these strategies are “ineffective”, but it probably means that we don’t have a comprehensive picture of what factors allow them to be effective. More nuance is needed to understand what actually works and how, when, why.

2. The population of students we work with is understudied in physics education research. (See here). How this does or doesn’t contribute to failed efforts and successes is important to start to address.

3. We did this across the entire department, not just for select classes and instructors. I don’t know how common this is, nor how much of the research has been done in this context. There are likely different threads here related to implementation, buy-in, and professional development.

4. Given the overall sameness of the course structure, it could be valuable just to unpack more of the details of what was done, and to construct plausible stories of what details may have mattered and why. When Hake published his original 6,000 survey, they spent some time documenting and discussing cases where IE methods were used but high learning gains were not achieved. Having a documented case of the journey from an ineffective IE course to more effective could contribute to better understanding.

All of that would be too ambitious for a paper about our curriculum development efforts, but I think it helps frame why I think trying to document and report about our experience is worthwhile.

Update: here are some links to what our curriculum overhaul looked and felt like:

  • Four thoughts on “how daycare helps me grow as a classroom teacher”

    May 28, 2018

    I’ve written about this before, but I’m still growing a lot as a teacher due to my experiences with daycare. Some of the things I’ve been thinking about recently are the following:

    1. Just how much of a priority it needs to be to work toward the goal of everyone truly enjoying being together. There is so much that goes into this work, but I’m more and more convinced that this is critical. Of course, this isn’t enough, but it’s an amazing catalyst and buffer. It’s often easy to get an overall picture of how this is going during official breaks–do students socialize?– and down time waiting for other groups to finish up (are they risking ways to stay engaged, or feeling relaxed, or feeling resentful and restless).

    2. Reframing how I see “setting up”, “tearing down”, “cleaning up” the classroom from a regrettable chore to important detailed-oriented work of preparing the environment for learning. The space and arrangement of tools need to be curated and maintained. Sure, sometimes it feels like a chore, but keep my focus on big importance that chore serves helps.

    3. Cultivating a working mindset and bodily feeling that minimizes macro and micro messages of annoyance, disappointment, and frustration. A colleague of mine describes this as “students feeling like you are on their side”. Traditional schooling dialogue can easily trap us into expressing disappointment.

    4. The importance of “re-seeing” troublesome behavior as relating to unmet needs and/or skills not yet taught. We have a chart in our kitchen that helps to identify many behaviors through alternative lenses, which is really helpful. I feel like I could use one for the physics classroom. This means I should probably write one up. 😜

    Circuit Design Challenge Lab

    April 20, 2018

    Last week students were given three resistors and had to design a circuit to draw 320 mW when connected across 8V , and then predict the power output at each resistor, and then take measurements using ammeters and voltmeters to compare to.

    These cut outs helped a ton

    April 19, 2018

    Stumbling toward the finish line …

    April 18, 2018

    I’m still doing ok, but it’s been harder and harder this semester to be my best at teaching, partially because I’m just so physically, cognitively and emotionally tired. Some days I feel fine, but I’m also struggling to make decisions effectively, especially in real time. Even with just speaking, I’m having bouts of aphasia that come and go. I’ve said this before, but I find it hard to be focused on both curriculum and instruction. For me instruction requires careful attention to ongoing events. When my mind is spending too much energy keeping track of curricular aspects, there is just not sufficient resources allocated to attend to what’s going on. Furthermore, so much time spent organizing curriculum leaves with too little time to think carefully through execution and to rehearse key points and transitions. I like to rehearse as much as I can, honestly, and I’m getting very little rehearsal time.

    Finally, I often spend a lot of time outside of class thinking about individual students and whole class dynamics. I email students who have missed class. I notice and strategize about groups that have got into a rut. I think crucially about specific issues of motivation, confidence, engagement, and plan accordingly. Many of their issues get put on the back burner when there’s too much curricular stuff I’m minding to.

    Finally, some of it is emotional fatigue. I care about students, but sustained acts of caring require some emotional well to draw from. I’m not quite finding enough ways to fill my bucket as quickly as it gets depleted. Because of this, I’m prone to disengage a bit in class. While I would always circulate around during clicker questions, I’ve noticed more and more I hang back. It’s like there’s an activation barrier I can’t quite always get over.

    This semester will soon be over, and I will take some time to recharge and refocus. I’ll also be attending a few conferences this summer, so I’m hoping to make the most of those to reconnect with colleagues and friend, and to re-energize.

    Losing steam, but otherwise ok

    April 4, 2018

    I’m definitely losing steam at the end of the semester here. I don’t so much feel burnout like I did at the beginning of the academic year, which is good, but I do feel more fatigued. My brain feels so so ready to stop thinking and get into some routine work. I look forward to the end of semester when I can get busy doing work around the house.

    This summer I’ll be attending AAPT and PERC, which will be nice, because it’s been 3 years since I last attended. It will be nice to see colleagues and friends, and also to just let research ideas wash over me. I’ll still need to spend a decent fraction of my summer revising curriculum, but it will be nice to balance curriculum and research, not curriculum and teaching.

    Various Images from Physics II

    April 2, 2018