July 30, 2025
Jerry Arbittier
After years in this business, I’ve watched some sampling plans fail. And here’s the thing: it’s usually not because of some complex error or technological failure, but before we get into why that happens, let’s take a step back; what is a sampling plan? Although there is no standard definition carved in stone, here’s what a sampling plan really is: you set your goals, figure out your pace, and commit to a hard quota—that’s the total number of participants you absolutely promise to deliver. No wiggle room. Even if you only need one complete to hit quota, I have seen clients refuse to close for over a week until we finally recruited the last one.
Why do Sampling Plans Fail?
1. The Incidence Rate Reality Check
What is an incidence rate: An incidence rate is the percentage of individuals within a target population who qualify for a specific study based on pre-defined criteria, and this is where most people get burned. Typically, when developing a sampling plan, you estimate an incidence but typically the estimate is based on many unknown facts. When the study is in the field, many times the estimate of the incidence is nowhere near the reality. Obviously this is a problem but what dramatically increases the problem is that people do not check the incidence at the start of the fielding process or more likely are not willing to address it until it is too late. It is imperative that the incidence be checked after 1-2 days in the field and that the sampling plan be updated to address the issue. In healthcare research, the relatively small universe sizes often limited to specific patient populations, rare conditions, or specialized medical professionals make incidence rates a much more significant factor than in consumer research. Unlike broader consumer studies, where large, diverse populations allow for easier recruitment and segmentation, healthcare studies must contend with low incidence and hard-to-reach audiences. This means even a slight variation in incidence can dramatically impact feasibility, timelines, and costs, requiring more strategic planning and specialized recruitment approaches to ensure reliable, representative insights.
2. Interview Length:
Tell me the interview is 25 minutes, and it turns out to be 33? We’re stopping everything. I’m reworking invitations, retelling doctors the real time commitment, and we’ve just lost two days while I fix what should have been right from the start. Doctors especially—they don’t have patience for surprises about their time and will complain relentlessly to the panel companies if the length of interview advertised in the invitation is not reality..
3. Daily Tracking: No Excuses
This is probably the biggest reason things fall apart. People look at progress once a week. Yes, once a week! That’s when you are setting yourself up for failure. For example, when you sell a project and say you need 100 participants with a 50% incidence rate, due by July 15th, you need to watch this every day. If we’re not hitting our numbers by day two or three, you know we’ve got a problem, and you need to start doing something about it. You also may need to be willing to discuss this touchy topic with both the salesperson at your company and the end client and that sometimes is scary.
That’s what separates the pros from the amateurs.
4. Qualifying criteria set too high
Research teams sometimes set qualification criteria too high, thinking they can just re-invite people if incidence is lower than provided in the bid request. But in reality, less than half of those re-invited will come back to take the survey. This slows things down and can affect the quality of the results. It’s better to start with realistic criteria that balance the study goals with the challenges of reaching specific healthcare audiences.
How can you create a plan that follows through?
1. Incentives
I’ve seen response rates jump from 30% to 50% by going from $150 to $300 an hour. You can get much larger quotas if you’re willing to pay for it. Incentives need to be properly aligned with the time required and the audience you are going for. Some audiences need stronger incentives than others, especially in healthcare.
2. Give Project Managers Real Authority
If your project manager needs three approvals to raise an honorarium by 20 bucks, you’ve already lost. The project managers know what to do, to get the project completed on time, and satisfy the client. But your team needs the authority to make those calls without endless bureaucracy or set rules on allowable profit margins. Project managers get scared to report bad news, so they sit there hoping things will turn around. They’re afraid to tell their boss they need to triple the honorarium, even when the specs are exactly what they’re supposed to be and we’re falling behind for no clear reason.
3. Communication That Actually Happens
Everyone on your team needs to be in the loop. When everyone works as an ecosystem, that’s when a project succeeds, as we can present a united front towards the clients. For example, when a project is in trouble, the project manager should know about the stakeholders, and communicate it immediately. Usually, the sales team, and the project teams work in silos, leading to miscommunication, or something almost worse – No communication.
The Bottom Line
The best sampling plans don’t just manage data; they manage people, expectations, and the inevitable chaos that comes with any research project. They’re built on honest communication, realistic projections, and teams that have the authority to solve problems as they arise.
In the end, the best sampling plans are those that acknowledge uncertainty while preparing for it, that set ambitious goals while remaining grounded in reality, and that prioritize both efficiency and quality in pursuit of genuinely valuable insights. Though difficult, when we consistently apply what works, and deliberately avoid what doesn’t, Perfection is Possible, and that is what we believe at AOPS.