September 22, 2014
Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies (2010) provides 50 independent effects for consideration. The meta-analysis describes each facet of the study and provides detailed metrics. I applaud the attempt to make sense of online studies measuring the effects on student learning; however, I find myself questioning the validity. For example the meta-analysis outlines in exhibit 1. Conceptual Framework for Online Learning (p. 5) the comparison of like and enhanced assignments between the differing learning modalities. It indicates that some opportunities for learning were provided in one context but not in the comparison assignment. For example: “Live, one-way webcast of online lecture course with limited learner control (e.g., students proceed through materials in set sequence)’ is compared to “Viewing webcasts to supplement in-class learning.’ It appears one set of students are instructed to proceed through the learning materials while another is provided a choice to use the materials. In my opinion, if we do not measure the same course assignments then how is it possible to have a valid understanding of the student learning outcomes?
In the Early Childhood Education department at UAF faculty have developed face to face, eCampus, and audio courses to meet the needs of various learning styles and course accessibility. We measure each course learning objectives with a mix of common assignments and teacher selected assignments. Each course has a least one common assigned which includes the same assignment instructions, rubric and student feedback format. We can measure student learning across all modalities in this one learning outcomes using the common assignment. However, I do not see how we can do the same for the teacher selected assignments, since each teacher determines what assignment is used and how it is measured. I could not compare the effectiveness of the student learning outcome with mine since we do not have a common way to measure. To me it would be like measuring apples to oranges. However, individual teachers can compare their teacher-determined assignment from year to year to understand the effectiveness of the assignment, instruction and student supports. As a whole, I am not convinced this particular meta-analysis can make a valid determination of the effectiveness of online compared to face to face courses. I suppose I need to see that we are comparing apples to apples.
However, I consider the ongoing conversation concerning the differing approaches to online learning interesting. What exactly do we mean by online learning? According to this study blended learning seems to edge-out other forms of learning online and purely face-to-face.
“In fact, the learning outcomes for students in purely online conditions and those for students in purely face-to face conditions were statistically equivalent. An important issue to keep in mind in reviewing these findings is that many studies did not attempt to equate (a) all the curriculum materials, (b) aspects of pedagogy and (c) learning time in the treatment and control conditions. Indeed, some authors asserted that it would be impossible to have done so. Hence, the observed advantage for blended learning conditions is not necessarily rooted in the media used per se and may reflect differences in content, pedagogy and learning time.’ (p. 17)
This particular statement aligns with what I believe takes the best of both the online and face to face world- combining approaches that are best for students learning styles. Some students confess during advising appointments that they do not like the online forum of learning and benefit more so from face to face contact with peers and teachers. They often cite their learning style necessitates a face to face interface for learning. Some students argue that flexibility of schedule is more important than meeting the needs of their particular learning style. They do the work online because it is most convenient for them, rather than take a modality they prefer. Then there are students that feel most comfortable taking online courses, yet still complain they feel isolated. When I added blended elements to my eCampus courses, student feedback was more positive about them not feeling so isolated.
To me, meeting the student learning outcomes is not all there is to the learning experience for students. Another wondering I had about the meta-analysis is comparison of student accomplishment. Did students learn more because they had higher grades? Did students learn more because it fit their learning style better? Will students maintain the learning? I have earned high grades in all my courses since starting college many years ago, but can I remember the course content? Which content do I remember? For me, the most memorable learning opportunities involved the interdependent relationships between me and the teacher and my peers when I was able to construct my own knowledge in a social context. Sometimes that included online learning communities, but most often in face to face settings. The point of learning is not to learn something once; it is to use the information in an application for future learning or performance. I would argue those students who are able to use their preferred learning modalities and styles are more capable of application and retainment.
U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies, Washington, D.C., 2010.