How many is enough?
One of the promises that we make at 1000 Feathers is to be committed - we invest in long-term relationships and make a commitment to trust the process. In practice, we view ourselves as working with our clients, not for them. This means that we work to become an extension of our partners team, committing to relationships we develop before, during, and after the journey of transformation. An important byproduct of this commitment is that our partners trust us and know they can ask us important questions, no matter how big or small, silly or serious it might seem to someone else.
Recently I received two iterations of a similar question, one from a partner and one from a current client and thought it would be interesting to share my responses given the different contexts. So, when you are collecting data, how many is enough?
Partner #1. An email found its way to me asking for the “magic number” of interviews necessary for aggregating data, assuming the stakeholders comprised a representative group.
Here's how I responded.
You've come to the right person, because as a psychologist I always have the answer: it depends! Kidding, well, it does somewhat depend, and I do have some answers that will help you.
Generally speaking, when doing interviews within qualitative investigations, we know that we have done our job, that no more data needs to be collected, when we have reached saturation. On the one hand, setting up a sound methodology should get us to saturation, without focusing on the number of interviews (because we're done when we're done). One example of a sound methodology is using information power. Now, that's all well and good in theory but not always practical, so on the other hand there has been some research that tells us that when we are using grounded theory, we should generally include between 20 and 30 interviews and when we are using single case studies we should include 15 to 30 interviews. Both of these will vary greatly (as little as 5 and as much as 50) when the scope of the research question is contained versus broad and when social, political, cultural, and unique factors are taken into consideration. In other words, 20-30 and 15-30 are guides and the distinct research question and context will determine the "magic number".
Partner #2. During the distribution of a recent organizational culture survey, a client asked, “What is an expected [survey completion] rate?”
Here's how I responded.
Like any good psychologist, the answer is, it depends! Typical survey response rates vary from 10-30% with ‘very good’ response rates at or around 50%. In this case, it might be helpful to ask, what are you hoping to do with the data? And how many people do you need to hear from to be able to do those things?
In this case where you sent the survey to a defined group of recipients (staff) and likely want to use the survey to help inform your work, you’d likely want to hear from a majority of your organization. For some, a majority might be over half (i.e., response rate >50%). For others, that might mean nearly all (response rate >90%). And of course, there’s everywhere in between.
So, what can we take away from this?
1. I’m a psychologist and the answer is always, “it depends”.
2. Nuance matters. Collecting 25 (or 50 or 100) interviews, surveys, etc. for one project will not be or mean the same thing as collecting 25 responses for another project. Context matters when determining the optimal way and number to gather information.
3. Quality questions matter. It is important that we clearly articulate what it is that we are interested in knowing more about (i.e., our broad area of interest or inquiry). For example, asking Where do you live? is not the same as asking In what city do you live? or Tell me about your current housing situation.
4. Action matters. There’s no point in gathering information if we are going to let it sit on a shelf. We always want to make sure that we understand our nuance and have quality questions so that we can take action once we have our information.
That’s how I responded. If you have gotten the “How many is enough?” question before, I’d love to hear how you responded.
