Increased Conceptual Understanding Through Peer Instruction in Engineering Dynamics

Article Review 3 – Lori Sowa

Contemplating 21st Century skills and gaming theory through the videos and writings of John Seely Brown, I jotted down a number of “big ideas” to research: embracing change, learning to join, power of play, not having a defined endpoint, tinkering, demand-based learning.   I searched through a number of articles on the benefits of video games, passion, obsession, and even addiction – but kept returning to the idea of peer instruction, which I think is central in Brown’s theories. My dissertation research is forming around a number of faculty who are using a flipped classroom approach to STEM courses at the freshman and sophomore undergraduate level.   The main question becomes – how can we best structure the in-class activities to promote deep learning?   The goal is to have students learning from each other, but how can we structure the class to promote this?

In Teaching Engineering Dynamics by use of Peer Instruction Supported by an Audience Response System, Schmidt (2011) describes a study where he implements Mazur’s peer instruction (PI) method in two engineering dynamics courses at a University in Denmark.   A third course, taught using the same methods but without the PI discussion questions, served as a control.   The author (who was also the instructor for all three courses) used a number of exams (a pre-test of engineering knowledge, final exams, and the cohort’s mathematics exam scores) and a class survey at the end of the course to look at learning gains and students’ dispositions related to the teaching style.   The questions on the final examination were broken into two categories: traditional problem solving and conceptual understanding.   The author found that scores on the traditional problem solving portion did not vary significantly among the groups, but that the two classes that used PI scored better on the conceptual questions.

The study was overall well-conducted, as the author has a reasonable control group and made an effort to tease out the level of preparedness of the students in the study.   However, one variable that was not controlled for was the language in which the course was taught.   The program of study was highly international, so two of the sections (the control and one of the experimental groups) were taught in English, while the third course was taught in Danish.   The latter group scored better all around, which could potentially be due to the course being taught in their native tongue. In addition, I would have liked to have seen some more detailed questions on the student survey at the end of the course.   For example, one of the questions read “Give an assessment of your own preparation for classes”.   Since the author encouraged students to read ahead in the text, and their doing so would have likely influenced their performance in the PI activities, a more specific question such as “I prepared for class by reading the assigned sections” and using a Likert-scale rating system to indicate always, sometimes, rarely, etc. may have provided better data.   One of the common issues with flipped classrooms is the students’ lack of preparation before class, so it would be nice to quantify this (to the extent you can actually rely on this data).

In my mind the most impressive result of the study was the increase in the number of correct responses after PI discussions and before instructor intervention.   Figure 1 from Schmidt’s article (p. 418) shows the percentage of correct answers increased in almost all cases from the students’ initial response (the x-axis below, before PI) to their response after discussion with peers (y-axis, after PI), many times quite substantially.

Scmidt PI

(apologies – the figure is much more clear in the original text)

Another important aspect of student learning using PI is the awareness by students that they do, indeed, make mistakes and have conceptual misunderstandings. In regards to the students who engaged in PI rating their own understanding of the material lower relative to the control group (who had a higher level of confidence in their grasp of the subject matter), the author provides this explanation:

It is believed that the discrepancy between the students’ assessment of their own outcome and the examination score is related to the quality of the clicker method to expose misunderstandings among students.   By taking part in PI-teaching, the student faces the fact that he/she makes quite a lot of mistakes when interpreting new methods and ideas.   Thus, the student gets the impression that the knowledge gained is not as profound as the student receiving traditional lectures feels regarding his or her outcome: at a traditional lecture it is tempting for the student to be fully satisfied with all the lecturer’s nice explanations! (p. 421)

There is an outpouring of research that shows that making mistakes and experiencing failure are truly important in the learning process.   Many times, students are afraid of failure, and this can inhibit their ability to learn.   Schmidt (2011) also states that “…the goal was a safe study environment where the student had no reason to fear giving a wrong answer… [in] this way, it is believed that the most ‘honest’ answers and the best measure of the students’ conceptual understanding as possible were obtained” (p. 416). In searching for an article to reference the importance of not being afraid of failure, I came across this powerful TED talk video with a focus on peer instruction and learning from mistakes.

Once again, the idea of students (novices?) learning from other students proves beneficial to the overall learning process.   While this instructional method was used in a face-to-face scenario, the method could be adapted to an online medium.

Schmidt, B. (2011). Teaching engineering dynamics by use of peer instruction supported by an audience response system. European Journal Of Engineering Education, 36(5), 413-423.

5 thoughts on “Increased Conceptual Understanding Through Peer Instruction in Engineering Dynamics

  1. Bob

    Failure… what do we mean by this? We throw it around glibly anymore. Failure can be devastating. Failure can crush souls and motivation. We know that business failure has huge consequences. We know that ecological failure is perhaps irreversible and catastrophic. What are we talking about when we celebrate failure?

    I have decided that I want to challenge myself every time I drop the f-bomb.

    “I have not failed. I’ve just found 10,000 ways that won’t work.” – Thomas A. Edison

    So, let me pose this riddle: my teenage son gets an x-box for Christmas. He borrows Call of Duty from his friends and immediately connects to the online player versus player instances. He then proceeds to die thousands of times in order to learn how to master game play. We fretted, well actually my wife, about the obsessive focus on game play. Hours and hours spent in perfecting micro-motor skills it would seem.
    Is it possible that what was being learned in all that failure was something else? Resilience, attention span, systematic experimentation, and I don’t know what all else, perhaps a fondness for challenge and learning itself.
    So what is it about simulated and unlimited failure? It is small cost, high repetition, so are those attributes of “good” failure? I’m assuming that we are making a distinction between good and bad failure. Good failure must be what we are looking for when we wish it on the learners we have responsibility for. What are the attributes of “good failure”?

    Reply
    1. Lori

      Great point, Bob – since when did failure become a good thing? Certainly we need to be careful about encouraging crushing, debilitating failure. And as you point out I think the real goal is to achieve the important traits you outlined: resilience, attention span, systematic experimentation.

      I was a kid who always seemed to get the right answer, without having to work very hard at it. Lucky me, in a myriad of ways, but I also became very performance-oriented, and was really disappointed when I got something wrong. I was quick to label myself as: I am really good at these things, and not good at these other things (such as drawing artistically, writing eloquently, etc.), and that’s the way it is. I actively avoided subjects and experiences that were outside of my comfort zone, in part because I fed so much off of the positive comments “you are so smart!”, which we are now realizing should be more along the lines of “you worked so hard to accomplish that!” I almost changed my major during my freshman year in college because there were concepts in chemistry that were hard for me to readily understand – apparently I was no longer “good” at chemistry.

      So I guess when I think about not being afraid to fail, I think about an educational setting (and society) where we are encouraged to step outside of our comfort zone, and aren’t afraid to answer a question for fear of being wrong. And really maybe what it comes down to is mindset. I haven’t yet read Carol Dweck’s book MindSet https://www.mindsetonline.com/ , but have attended conferences where her central tenet was discussed. She argues there are two basic mindsets we can have – a fixed mindset or a growth mindset. A fixed mindset basically limits us to what we perceive that we are “good” at, where a growth mindset is achieved when you truly believe that you can do anything if you work hard enough for it. While I think it is wise for all of us to know and accept our strengths and weaknesses, there is a fine line between accepting who we are in a positive way, and labeling ourselves in a much too restrictive way. I still believe there are limits to this, but as a success story I think about one of my current students who came back to school later in life, who was never “a math person”, who has worked so hard and is finally seeing success. She described a recent experience with her homework: a word problem, the bain of many students, which she typically would just skip. This time she decided to try it, had to google the terminology to figure out what ‘cubed’ really meant, and persevered through the problem. She said she almost fell off her chair when she checked the answer and got it right. Now, she has a different mindset when she approaches a difficult problem.

      Reply
      1. Owen

        Lori – several fantastic revelations here. Your description of yourself at an early age reminds me much of my daughter. I think she’s much the same.

        Learning how to persevere when outside one’s comfort zone – that is a huge hurdle for those who gain confidence early.

        All of this seems so clear in little pieces, I wonder when/if we’ll put it all together in a holistic vision and create a system that all learners can benefit by? … Moment of dreaming there.

        Reply
  2. Owen

    Wow. Failure. One of my favorite topics.

    I have a friend and colleague who teaches Computer Science and is also a pretty serious poker player. People are always asking him to teach them poker. After 20 years, he’s come to the conclusion that people only learn when they play him for money. If it is just for fun, they don’t learn. If failure hurts, they pay attention, they get nervous, they get stressed, and they feel elation at success.

    At the same time, we all know that if failure hurts too much, it can be crushing.
    How much do we make the formative moment sting?

    I wonder if learner investment is part of the answer? The more invested the learner, the more pain can be tolerated (pain in a metaphorical sense).

    Reply
  3. Alda

    How interesting that the peer instruction was such a boost for the learning process. Reminds me of one of John Seely Brown’s science lectures that we watched on YouTube. I just read in another article that there are studies showing peer instruction to be as useful as instructor feedback in some situations. Given that many students are novices, and social pressure can be strong, it still surprises me somewhat that peer instruction can be so effective. What about the potential for groupthink? Because there is also research out there telling us that committees aren’t always a good idea, and people may make worse decisions as a group than if they had worked by themselves. I’ve seen one-on-one peer feedback work well; I use it in my speech classes so students have an outline buddy who helps them proofread and revise. But I also teach group speech classes and have seen some group projects crash and burn because the louder group members talked the quieter group members into going along with some bad decisions.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *