Novices and Experts Navigating the Evaluation Learning Space

Benander (2009) compares novices and experts within the dynamic of student-teacher. This is a highly relatable paradigm since we’ve all been learners at some point. The difference in experience between novice and expert can also be observed in groups of workers with varying years of experience. I think there are parallels in the business world where veteran coworkers become mentors or “teachers” of the new workers.

Two years ago, I started pursuing a career as an “evaluator” and have been gaining experience through training, classes, and collaboration with agencies. The article by Cox (2004) about learning communities brought to mind the fact that I am part of a Facebook group for “novices” called Graduate Student and New Evaluator Topical Interest Group (GSNE TIG). I also have had many opportunties to meet with “expert” evaluators during the annual conference of the American Evaluation Association. Having an online community definitely helps combat the isolation that Cox (2004) mentions, especially for folks working in Alaska who may not have as many opportunities to network face-to-face with experts.

Often the teacher/learner dynamic in a conference setting is evident when novice evaluators attend “expert panels” to hear from folks who have been doing evaluation for decades. I’m sure those of you in other professional pursuits can relate to the idea of going to a conference and hoping to learn from and meet the “rock stars” of your field. Beyond what we can read in the guidebooks and manuals the rock stars have written, we can learn an immense amount from expert stories and tales of caution and success.

One of the reasons we gain so much from listening to experts is because they truly do have a different way of thinking and knowing the information of our field. As Benander (2009) notes, “novices and experts have different strategies for negotiating problems” and “experts have a different orientation…to learning about their subject matter” (p. 37). Listening to the advice and stories of experts helps novices grow in their abilities. But when you’re an expert, some things have become automatic or second-nature and you may forget how difficult it was at first for you to learn that skill. Thus, the author’s advice for experts to reflect on “what it’s like” for novices is definitely something to take to heart.

Benander (2009) also describes how “experiential learning is not just for students but can be a valuable tool to reflect on teaching and learning” because it helps you see the emotional side of learning and the differences in people’s learning styles. This reminds me of when I participated in an experiential evaluation class, where we had to choose a community organization and help them complete an evaluation of some aspect of their service. It was very beneficial to go through that experience and be both a learner (getting to know the organization, its history and people) and offer my expertise as to how best to construct surveys to gather the data they were looking for. I tried to constantly keep in mind the community partner’s perspective; while my focus might be “getting good data,” they are also concerned with “how is this going to be used” and “how might this change us.”

The experiential course also gave me some perspective outside of my everyday work. I consult with faculty at the Cooperative Extension Service on how to evaluate their outreach programming. Many new agents have not surveyed their workshop participants for changes in knowledge and behavior before. I was able to reflect on what I learned from working with another campus organization and carry over some of that knowledge to help solve problems for Cooperative Extension.

The experiential course I took part in was also a two-sequence course with a “cohort” of graduate students. Cox (2004) notes that “the community formed by a student cohort plays a key role in achieving better student learning outcomes for students in SLCs…” (p. 7). I definitely did feel a sense of community in that class, which surprised me because I was not part of the actual overall program that was offering the class (community psychology) and was merely trying to expand my professional knowledge. But spending two semesters in a row with the same fellow students helped us feel like a cohesive group, and paired projects helped us get to know each other and share ideas more freely.

On the teaching side, Bernander (2009) also recommends that teachers “use the student view of the electronic interface” of whatever content management system they’ve chosen (p. 39). As an evaluator buidling surveys, it is definitely critical to preview the survey and look at it just as the public would. It is even better if you have the time to try the electronic view from several different browsers, including a smartphone given today’s tech-savvy and on-the-go audiences.

Trying to be cutting-edge isn’t always a boon, though. I wanted to make a survey less boring and more “interactive” so I threw in a couple questions that were not multiple choice. One major detriment to that dataset was the fact that many users found the ranking question confusing because it asked them to “drop and drag” answer choices to reorder them. Many commented that they didn’t find it intuitive and would rather have a straightforward set of boxes where they typed in their rankings. That’s the sort of experience where perspective-taking may have headed it off, but at least through reflection I am learning from it now.

In closing, the authors mentioned here make compelling arguments for the value of experiential learning and learning communities for BOTH teachers and students. I believe this can also be applied to novices and experts in other similar dynamics, like new and veteran evaluators. Whether it’s in a formal college classroom or at an annual conference, we can all benefit from hands-on practice, focused reflection, and continued contact with others who share our learning goals.

References

Benander, R. (2009). Experiential learning in the scholarship of teaching and learning. Journal of  the Scholarship of Teaching and Learning, 9(2), 36—41.

Cox, M. D. (2004). Introduction to faculty learning communities. New Directions For Teaching & Learning, 2004(97), 5-23.

GSNE TIG (2014). In Facebook [Groups]. Retrieved Sept. 19, 2014, from https://www.facebook.com/groups/182695741744559/

3 thoughts on “Novices and Experts Navigating the Evaluation Learning Space

  1. Bob

    “This reminds me of when I participated in an experiential evaluation class, where we had to choose a community organization and help them complete an evaluation of some aspect of their service. It was very beneficial to go through that experience and be both a learner (getting to know the organization, its history and people) and offer my expertise as to how best to construct surveys to gather the data they were looking for. I tried to constantly keep in mind the community partner’s perspective; while my focus might be “getting good data,” they are also concerned with “how is this going to be used” and “how might this change us.””

    I hope you will continue to share from this experience. I too have found this intersection of classroom/real world to be fascinating. I took a number of courses in business and IT at the local b-school. This represented a left-turn in my intellectual trajectory coming out of pure academic inquiry into applied and practical solution driven questions. Rather than hypothetical or case study I have shifted most of my projects in coursework to this kind of project — significantly more satisfying. Enough about me — back to your writing.

    How self-conscious of yourself as a leader were you in that situation? How much work did you do on negotiation, and communication skills in this situation? Were you able to help these organizations make connections to other groups, local government, education institutions? I hear you speaking about community within the class cohort did this extend to the people involved in the particular projects? I ask because the class cohort is all gone now except in your memory — but the organizations are still moving ahead with their missions and visions.

    Reply
  2. Owen

    Hi Alda,

    Nice work!

    You raise many good points Alda and I think Bob raises some good questions as well.

    I think you illuminated some of the differences between expert and novice, and the value for instructors to be mindful of the novice experience. I liked you anecdote about your “drag and drop” answer choices. I face these issues fairly often – what appears intuitive to the expert isn’t always intuitive to the novice.

    I look forward to hearing more of your experiences as evaluator and student of evaluation.

    -Owen

    Reply
  3. kgebauer

    As mentioned above you have a wealth of experience that will be beneficial to this class cohort. I especially liked this quote, “But when you’re an expert, some things have become automatic or second-nature and you may forget how difficult it was at first for you to learn that skill. Thus, the author’s advice for experts to reflect on “what it’s like” for novices is definitely something to take to heart.” This tendency to become automatic and run on autopilot happens a lot in the K-12 world with expert teachers. They know what works and if a novice comes to them for advice they have an automatic answer or a piece of material to give you. They also tend to assume you will know what to do with the information. That is why evaluation, observation, and mentoring are important because they help keep us all in the “novice like frame of mind”, meaning that we remember there is always something new to learn or revisit. I try to give an evaluation at the end of big units to my students, so that they can grade me and give me their perspective on their learning experience through the unit. It is always eye-opening and helps motivate me to look for new ways to improve my teaching to help the learning process along. I also have been evaluated by the experts many times through my two internships and it definitely keeps you on your toes. This evaluation is a constant reminder that learning is a process and not all about mastery. You learn quickly that mastery is only a small part of learning and not the end all.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *