We hear all the time that we need to make a conscious effort to reduce survey fatigue, that a survey should not be the end-all be-all for assessment. #SAchat recently offered a Twitter chat on how to prevent survey fatigue and a common theme was utilizing other assessment methods. I am fairly new to the world of Student Affairs Assessment, being in my role for nearly two years now overseeing assessment for the division. Recently, I was responsible for conducting institution-wide focus groups (11 in all), and from that experience I can now understand why many are so fast to jump on the survey bandwagon without considering other options for assessment. We all know when conducting qualitative assessment that it is very time consuming, but you really don’t realize how much time is involved until you are in the thick of it.
First of all, the planning stage includes looking at a great many details including recruiting participants, developing the questions, finding appropriate locations, creating the protocol, ordering food, etc. Sometimes it is difficult being the only person in charge of assessment for an entire division, and I am very appreciative of my Assessment Team. We call ourselves the A-Team and have a picture of the TV show’s cast members on each of our agendas. I was looking at the 11 focus groups that I planned to facilitate and determined that it was fine for me to ask for help. Each member of the A-Team served as moderators for the focus group and also served as co-moderators.
I was worried what the participation of students would be like in the focus groups. And much to my surprise, the students were very open and honest regarding the questions that we asked. Students were grateful that we cared enough about their experiences and that we wanted to make improvements based on their responses. I had one student say to a colleague of mine, “You guys actually care about what we have to say.” The focus groups ran over a period of three weeks.
Just when you think you have the difficult step completed (moderating the focus groups) it comes time for the transcription and the final report. I didn’t have a concept of how long it took to transcribe a 90 minute focus group. Needless to say that I had over 100 pages of transcription in the end, and it took me hours. Once you have the transcription complete, the analysis begins. This includes reading over the full transcription, developing a coding system based on common themes, and then organizing it into a final report.
It may sound like I am not in favor of qualitative assessment because of the time commitment. In actuality, I would highly recommend it. There is a tremendous amount of work involved in this type of assessment, but the information that you can discover is quite extraordinary. It’s a great way to determine the why from the surveys. In this case, these focus groups were the next step in assessing student satisfaction on campus, and we have learned a great deal more about the students’ experiences and their needs. Plus, it shows students that we do want to hear from them. It also demonstrates how important their feedback and involvement is in helping us move the institution further. Don’t be afraid of all the work involved in qualitative assessment because the end result is worth it!