MONA - Matematik- og Naturfagsdidaktik is a Danish journal for research in STEM education and their editorial office asked me to comment on an (paywalled) article on the use of clicker in STEM education at the University of Southern Denmark. Sort of a post-publication peer review, which is a nice idea. This blogpost is a draft.
My background so you know where I am coming from
I have taught chemistry courses since 1996 both in the USA and in Denmark. I used the standard lecture model (always blackboard, never powerpoint) until about 3 years ago when I switched to peer instruction. I use the Socrative.com site where students vote using their smartphones or laptops.
The study and the course
The paper presents an analysis of a questionnaire on the use of clickers given to students in a first-year biology course that must be taken by most science majors at the University of Southern Denmark. The enrollment was 290, 184 took the exam, and 58 filled out the questionnaire. In addition, the paper presents quotes from 4 interviews with 10 students in their first (6), third (1), and fifth year (3). I don't think all these student where enrolled in the course.
The instruction consisted typically of a 2 x 45 min lecture with 3-4 peer instruction questions in total. The questions focussed on either recall of facts or testing of conceptual understanding (more on this below). There was usually one round of voting with discussion before the vote.
Some observations on the study
The study states "... good teachers can improve their teaching with clickers while bad teachers don't necessarily improve their teaching just by using clickers." I think that is very true, so it is worth keeping in mind that the questionnaire offers feedback on one particular teacher and one particular way of using clickers in one particular course. More on this below.
Related to this point: in my experience it is often the "good" lecturers who are most resistant to trying clicker questions and other actively learning techniques. This is unfortunate because there is not data to indicate that students learn more from engaging lectures - they are just less bored. As Richard Feynman wrote in the preface to his famous lecture notes:
The interviewees mention that "the clicker questions ... should test understanding and not recollection of facts and be of such difficulty that the individual student will submit both wrong and right answers." So I was a little disheartened to read that the clicker questions included recollection of facts. In fact the question highlighted in the article (see below) as a conceptual understanding questions is actually a factual recall question that you can answer in, literally, 15 seconds using Google.
The goal in writing peer instruction questions is to write questions that ca half the students answer incorrectly on the first vote, followed by a discussion and a second vote because this leads to the best discussion and ensures that the majority of students are challenged. So I was a little disheartened to read that the students only vote once on most questions. The article states that on average 59.7% voted correctly on the questions, so it looks as though students would have benefitted from more discussion with each other.
I was happy to see that only 60% liked to see the result of the votes. I never show the results of the vote. If the vote is nearly unanimous then showing the vote results is uninformative and takes time. If the vote is split than the results bias the revote.
I would very much have liked to see a question related to the pacing included in the questionnaire. The biggest critique I go when I first started using peer instruction was that the whole process was too slow. Now my advice is better too fast than too slow.
Were the article not behind a paywall and written in Danish you would now point out to me that despite all my reservations of the way clicker questions are used in the course the study showed that the 98% of the students think clicker questions should be used in the course in future years. I would argue that after 20-30 minutes the students would welcome almost any break from being lectured to - the teacher's cell phone ringing or a knock on the door by a lost visitor - and, if asked, would enthusiastically recommend that the practice be continued.
The big question?
The study ends with, but does directly address, the "big question": do students learn more? This question is often re-phrased as "do more students pass the course" and a recent meta study indicates that introduction of active learning techniques can decrease the failure rate from 33.8% to 21.8% in STEM courses. While this is a great way to sell the approach it is worth recalling why peer instruction was invented to begin with.
In the early 1980's - the good old days of lecturing before students could check Facebook on their cell phones - an astute physics professor developed a set of very simple conceptual questions related to the classical physics of force (the so-called Force Concept Inventory). Careful studies involving thousands of students at several different universities showed that passing a first year physics course did very little to improved these students' very poor conceptual understanding of basic Newtonian mechanics. This was also true to courses taught by brilliant award winning lecturers.
When physics professor Eric Mazur read about this study in he late 1980's his first thought was "not my students" - motivated Harvard pre-med students who did consistently well on his exams and gave his lectures rave reviews. But as he discovered his students conceptual understanding was equally poor and their ability to solve relatively complex physics problems on the exam was a result of memorization. As a result Mazur invented the peer instruction approach where the "lecture" period was reserved to focus on the conceptual understanding, which is hard to address with homework problems - not to improve test scores.
This is why it is so important to ask good conceptual clicker questions: it is the only time they develop and are tested on their conceptual understanding of the subject. If the clicker questions are merely recall questions or mini homework problems the underlying problem (rote memorization leading to superficial understanding that is quickly forgotten) are not addressed even for the majority of the students who pass the course.
My background so you know where I am coming from
I have taught chemistry courses since 1996 both in the USA and in Denmark. I used the standard lecture model (always blackboard, never powerpoint) until about 3 years ago when I switched to peer instruction. I use the Socrative.com site where students vote using their smartphones or laptops.
The study and the course
The paper presents an analysis of a questionnaire on the use of clickers given to students in a first-year biology course that must be taken by most science majors at the University of Southern Denmark. The enrollment was 290, 184 took the exam, and 58 filled out the questionnaire. In addition, the paper presents quotes from 4 interviews with 10 students in their first (6), third (1), and fifth year (3). I don't think all these student where enrolled in the course.
The instruction consisted typically of a 2 x 45 min lecture with 3-4 peer instruction questions in total. The questions focussed on either recall of facts or testing of conceptual understanding (more on this below). There was usually one round of voting with discussion before the vote.
Some observations on the study
The study states "... good teachers can improve their teaching with clickers while bad teachers don't necessarily improve their teaching just by using clickers." I think that is very true, so it is worth keeping in mind that the questionnaire offers feedback on one particular teacher and one particular way of using clickers in one particular course. More on this below.
Related to this point: in my experience it is often the "good" lecturers who are most resistant to trying clicker questions and other actively learning techniques. This is unfortunate because there is not data to indicate that students learn more from engaging lectures - they are just less bored. As Richard Feynman wrote in the preface to his famous lecture notes:
I don't think I did very well by the students. When I look at the way the majority of the students handled the problems on the examinations, I think that the system is a failure. ... It's impossible to learn very much by simply sitting in a lecture, or even by simply doing problems that are assigned.The course is still mostly lecturing. The interviewees mention that one of the advantages of clicker questions is "People who are half asleep wake up and participate and everyone in the classroom is talking." To me that begs the question why put them to sleep in the first place? The "traditional" peer instruction/clicker approach is that the students prepare at home and then the entire "lecture" period is used for questions. This is what I do.
The interviewees mention that "the clicker questions ... should test understanding and not recollection of facts and be of such difficulty that the individual student will submit both wrong and right answers." So I was a little disheartened to read that the clicker questions included recollection of facts. In fact the question highlighted in the article (see below) as a conceptual understanding questions is actually a factual recall question that you can answer in, literally, 15 seconds using Google.
The goal in writing peer instruction questions is to write questions that ca half the students answer incorrectly on the first vote, followed by a discussion and a second vote because this leads to the best discussion and ensures that the majority of students are challenged. So I was a little disheartened to read that the students only vote once on most questions. The article states that on average 59.7% voted correctly on the questions, so it looks as though students would have benefitted from more discussion with each other.
I was happy to see that only 60% liked to see the result of the votes. I never show the results of the vote. If the vote is nearly unanimous then showing the vote results is uninformative and takes time. If the vote is split than the results bias the revote.
I would very much have liked to see a question related to the pacing included in the questionnaire. The biggest critique I go when I first started using peer instruction was that the whole process was too slow. Now my advice is better too fast than too slow.
Were the article not behind a paywall and written in Danish you would now point out to me that despite all my reservations of the way clicker questions are used in the course the study showed that the 98% of the students think clicker questions should be used in the course in future years. I would argue that after 20-30 minutes the students would welcome almost any break from being lectured to - the teacher's cell phone ringing or a knock on the door by a lost visitor - and, if asked, would enthusiastically recommend that the practice be continued.
The big question?
The study ends with, but does directly address, the "big question": do students learn more? This question is often re-phrased as "do more students pass the course" and a recent meta study indicates that introduction of active learning techniques can decrease the failure rate from 33.8% to 21.8% in STEM courses. While this is a great way to sell the approach it is worth recalling why peer instruction was invented to begin with.
In the early 1980's - the good old days of lecturing before students could check Facebook on their cell phones - an astute physics professor developed a set of very simple conceptual questions related to the classical physics of force (the so-called Force Concept Inventory). Careful studies involving thousands of students at several different universities showed that passing a first year physics course did very little to improved these students' very poor conceptual understanding of basic Newtonian mechanics. This was also true to courses taught by brilliant award winning lecturers.
When physics professor Eric Mazur read about this study in he late 1980's his first thought was "not my students" - motivated Harvard pre-med students who did consistently well on his exams and gave his lectures rave reviews. But as he discovered his students conceptual understanding was equally poor and their ability to solve relatively complex physics problems on the exam was a result of memorization. As a result Mazur invented the peer instruction approach where the "lecture" period was reserved to focus on the conceptual understanding, which is hard to address with homework problems - not to improve test scores.
This is why it is so important to ask good conceptual clicker questions: it is the only time they develop and are tested on their conceptual understanding of the subject. If the clicker questions are merely recall questions or mini homework problems the underlying problem (rote memorization leading to superficial understanding that is quickly forgotten) are not addressed even for the majority of the students who pass the course.
No comments:
Post a Comment