Unit 2: Week 1- Article Critique

Unit 2 Week 1: Article Critique

AnneMarie Mattacchione

September 20, 2014

The article I selected for this assignment focuses on the effectiveness of assessment of an online learning course. The course was developed and implemented by the UK Open University, Milton Keynes, UK. The course focused on web site development and implementation. The assessment considered the content and design of the final project web site. The analysis did not include a test-based option. At the time of the study, expectations of web-based courses included the use of web-based technologies specifically.

Weller indicated that “Web-based courses differ from traditional face-to-face lectures and print-based distance learning courses, in the manner in which the technologies influence the pedagogy, content, interactions and overall student experience. Bates (1995) states that there is a tendency with new technologies to attempt to mimic the traditional classroom face-to-face method of teaching, or to add new technologies onto an existing model. This is particularly the case with the assessment component of a course. If one accepts that assessment must be designed to react to the course pedagogy, aims and objectives, then the changes wrought in these by the shift to a web-based environment should necessarily be reflected in the assessment.’  (Weller, 2010, p. 110) This perspective is a well discussed issue prevalent to online course development compared to the development of face to face courses. Careful considerations concerning the nuances of online learning are encouraged. Such nuances include intentional examination of measurement of learning objective and the building of existing knowledge with new knowledge.

The article cautions course developers to carefully follow university procedures for authentic and measureable assessment in line with policy and practice for all courses despite modality.

The course enrollment totaled 850 students listing many as late returners to education and first time university attendees. The course was developed initially as a test of sorts to work out the bugs for future course offerings. Several adjustments were made to future version of the course based on the findings of the assessment during the first iteration.

Findings included the following:

Be careful not to weigh group assessments but rather weigh individual performance within the group process. For this particular course Weller states “Students often feel they are ‘punished’ if the group does not work well, or others do not contribute. By assessing the analysis of group-work students could still score highly even if their group did not work cohesively.’ The philosophy for this course included “a strong group-work emphasis, particularly at the start of the course. This embodied the underlying course philosophy that the Internet is about two-way communication and not just a delivery mechanism.’ (Weller, 2010, p. 111)

Ensure that the technology used works for all students. If we are measuring student’s use of technology, they must be able to have the proper software and hardware to enable meeting the course assignments expectation. Some of the students in the study were unable to turn in completed projects due to problem HTML editors. Experience with the technology is a pre-requisite to taking online courses. While this is a typical procedure for online courses, how can instructors be sure that students are competent before the course begins? Usually, competency of this nature is not assessed until the first part of the course when students find they have trouble with site navigation or connectivity; as is sometimes the case for rural areas, especially in Alaska.

Plagiarism, a potential issue in all courses, even more so with online learning. Weller indicates “An issue of great concern to many with regards to web-based assessment is that of plagiarism.’ (Weller, 2010, p. 115) Weller goes on to emphasize a stair-step approach to developing summative assessments by asking students to turn in some of the project in draft form prior to completion for analysis. This approach was implemented after the initial study and reduced the possibility of plagiarism substantially.

Qualitative data from the students suggest that an experience with this type of assessment was positive over-all. However, there is no data that reveals a comparison to traditional testing. It is my conclusion that the data cannot be as reliable and valid as needed to make a determination between the two differing types of assessment. I would be curious as to how both content knowledge and application of concepts were assessed. Does a student-developed web-site assess both subject knowledge and application? How does the faculty determine what students know? Most would agree that web-site development is a fair way to assess application of concepts. I found the article did not include commentary on both.

I found the article useful as a reminder that it is important to consider fairness in assessment application, fairness in expectations for completion of assignments, fairness of student work ethics and fairness in terms of measuring application and course knowledge. I have the benefit of reflecting on the outcomes and recommendation of this article that will directly affect my work with online courses this year. Many of our assessments are not test-based assessments. I intent to share the findings of this article at our next faculty self-assessment process leading us toward accreditation.

Works Cited

Weller, M. (2010). Assessment Issues on a Web-Based Course. Assessment & Evaluation in Higher Education 27:2, , 109-116.

2 thoughts on “Unit 2: Week 1- Article Critique

  1. Owen

    Sounds like a good find, and I like that you found the article valuable enough to share in the faculty self-assessment process. This paper is also neat in featuring content from the Open University of which I am a fan. 🙂

    On the subject of fairness, what is fair? Are we as educators held to different standards than the “real world”? Working teams in the non-academic world are held to achievement standards based on their group’s success or failure – not on their individual contributions. Is this fair? Are our concerns as educators different?

    What would you do differently if you were designing a similar study inquiring about the effectiveness of online assessment?

    Nice review, AnneMarie.

    Reply
  2. kgebauer

    AnneMarie,

    I like that you chose an article on assessment and how it relates to the online learning environment. In one of my first special education classes I had an instructor teach us that fair doesn’t mean the same. It isn’t fair to treat online assessment practices the same as a traditional classroom assessment practices, although there will be considerable overlap it will still be treated differently.

    Your question “Does a student-developed web-site assess both subject knowledge and application?” the answer in my opinion is typically a yes because it is project based assessment, which is all about show casing what you have learned as a student. In my experience is usually accompanied by a rubric that breaks the project down to demonstrate that the student has learned the content and applied it successfully. Whether this is a fair assessment in an online learning environment, I think so, but I am sure some would disagree and want students to take a test. The only thing I would add to the assessment is a reflective or evaluative component, which is important to any assessment no matter the environment. This is what encourages deep and critical thinking and helps learning continue. A test has a finality to it that makes the impression you showed you have learned it now you are done. Adding a reflective or evaluative component to a learning experience helps create lifelong learners and builds a sense of a next time.

    I believe group work can be assessed fairly if it is done right. It made me think of my experience with literature circles. Every student is assigned a job to complete and is graded individually on it. But the students come together as a group to discuss and get a grade as a group for discussion as well as an individual grade on their self-evaluation. This may seem like over kill in assessment, but it teaches that each student is responsible and held accountable for their share of the work and if they don’t do their part it can impact the entire group as well. In this way the grading doesn’t hurt the students who did the work because the group grade isn’t weighed as heavily and is usually participation based grade. I know this is nothing like the group work mentioned, but I think the simple underlying ideas of the assessment could be applied successfully to online learning.

    Good luck sharing what you learned and thanks for sharing with us.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *