Every fall, we hosted our annual student/industry-networking program event bringing together 40 employers and alumni to share their experiences with over 200 students. Year after year, at the end of the evening, students were given a standard five question Likert-scale survey. We invested a significant portion of our budget and personnel hours on this premier event, and all we really knew after its conclusion, was seven students “liked” the program. #Fail. We discovered that while a well-developed survey can capture student learning, our surveys were only measuring satisfaction levels. Recognizing our inadequate survey design and the presence of survey fatigue among students (and us!), it was time to try something different. We decided to challenge ourselves to assess students for an entire year without the use of surveys.
Motivated by this challenge, we started incorporating the New Learning approach into all career development programs for an entire year; we utilized no surveys, only the New Learning approach. We began all our programs by asking the students to record on a carbon copy half-sheet anything they deemed “new” to them and any additional questions they experienced throughout the workshop/program/etc. Written on the top of the carbon-copy form was the same statement, “Please tell us about any new learning you have experienced or any new questions that have occurred to you as a result of this program.” This prompt provided students the freedom to document their learning in their own words and at their own pace throughout the event.
By incorporating the New Learning method of assessment into all data collection methods throughout the year, we finally discovered what our students were learning—and what they were missing entirely. Adjusting our assessment methods away from strictly quantitative data gave us greater breadth and depth of information. We gained greater understanding of the success of the event, the amount of exposure to new information, and more profound feedback on areas of needed changes or additions to our future programming. And through this reflection we learned four key things:
1. Students learn at a variety of points during a program or workshop and limiting our collection to the end of a workshop may not capture all that is happening throughout our time.
2. We must move our students to action, and not just engagement.
3. It was the presentation method that needed to be refined, and not the content of our programs.
4. Trying something different, and not relying on surveys entirely drives creativity and innovation.
What began with low response rate and a desire to eliminate surveys for a year, ended with a better appreciation of assessment, including surveys. We did not start this process as assessment experts, nor find ourselves as such now; but, something powerful happens when we stop and reflect on what our assessments are really measuring, leading to our own learning as educators.
Interested in learning more about New Learning? Check out these three references:
Haley, K.J., Wiessner, C.A., & Robinson, E.E. (2009). Encountering new information and perspectives: Constructing knowledge in conference contexts. The Journal of Continuing Higher Education, 57, 72-82.
Chapman, D.D., Wiessner, C.A., Hatcher, T., & Storberg-Walker, J. (2007). New learning: A different way of approaching conference evaluation. Knowledge Management Research and Practice, 5, 261-270.
Wiessner, C.A., & Gonzalez Sullivan, L. (2007). New learning: Constructing knowledge in leadership training programs. Community College Review, 35, 88-113.
Ringing in the new year with no particular theme, this month is a grab bag, where contributors can share any topic of interest. Because nobody puts baby in a corner.
This post is part of our #SACareer series, addressing careers in student affairs, careers outside of student affairs, and the work of career services professionals. Read more about the series in Jake Nelko’s intro post. Each post is a contribution by a member or friend of the Commission for Career Services from ACPA. Our organization exists to benefit the careers of career services professionals, student affairs professionals, and anyone supporting students in the career endeavors. For more information about how to get involved with the Commission for Career Services or the #SACareer blog series, contact Cristina Lawson at firstname.lastname@example.org.
> BONUS <
Podcast With Kedrick Nicholas on Assessment of Student Programming