"Real Artists Ship"

Colin Johnson’s blog

Archive for November, 2012

Getting Meso-scale Information from People

Thursday, November 8th, 2012

The National Student Survey (NSS) has been running for a number of years now, and, in common with similar exercises carried out by many organisations, uses two methods for getting feedback from students about their experience at university. The first of these are Likert scales asking for a numerical rating on a number of questions. For example, students might be asked to rate the quality of the library service on a scale from 1-5. These numbers are then averaged, at a course or institutional level, and these averages compared to other courses or institutions in the sector. The second is to ask for free text comments, e.g. positive experiences and suggestions for improvements.

These capture two kinds of information, each with disadvantages. The numerical questions capture macro-scale data about where problems are; but, the information is not at sufficient detail to inform action. Merely knowing that, say, the library is weak doesn’t inform us about what the concerns are: is this a problem with the availability of books, the opening hours, the quality of the space, or what? However, the free-text comments measure things at a micro-scale. Each comment represents the individual viewpoint of an individual student, which might be a shared concern of the whole cohort or might be a personal, unshared concern.

What we need for action is meso-scale information, information about the broad-brush details of what a large proportion of the student body are interested in. One way that we have found useful in getting this sort of meso-scale information is to use the results from the NSS in focus groups of students. By discussing the results with students, we can ask questions about why they think a particular low or high score on a numerical question got that way, and what might be done to improve it. Similarly, with the micro-level free text questions, we can ask whether a particular response is a common concern of a number of students, or which free-text responses stand out as being general concerns. Using the NSS results to provoke discussion within such a focus group seems to give a better quality of discussion to these points; it gives a starting point, a “focus” if you like, to the group’s discussion.