|
|
Technology from Database Systems Corp. lets you develop IVR survey applications using our interactive voice response IVR solutions. Surveys can be initiated by outbound phone calls or can be a response to callers. Using our PACER and WIZARD phone systems with the Smart Message Dialer and survey software, we can call your survey prospects and play a highly focused and custom greeting. We then can give your survey audience the option to take your survey or even talk with a representative, leave a voice message, hear additional information, or simply decline to participate in the survey. The survey can accept touchphone response or can record each question response for later analysis.
To view more information regarding our automated phone applications, please visit our Automatic Phone Survey solution web page.
The following is an article relating to call survey techniques and products and services in our business.
Best Practices of Mail and Phone Surveys
Page 4
From: American Business Media
Selecting Sample Members
Sample members must be selected with known probability using a random selection procedure.
It's not necessary that each subgroup of a sample have the same probability of selection, only that
the researcher knows the probability.
This principle rules out many possible sampling approaches:
- Samples of "experts" or "industry leaders" (unless the results purport to represent only those
sampled, not a larger group)
- Quota samples, where new names are continually contacted until there are enough responses in
categories such as age, sex, or location
- Convenience samples such as surveys taken at a local mall. Convenience samples fail to
represent larger populations of shoppers because they are influenced by location, time of day,
and the state of the economy.
Bias can enter the sample in other ways. If you choose from a transaction list of purchasers when
you want to survey the purchaser population, the probability of selection will be higher (to an
unknown extent) for frequent purchasers than for infrequent purchasers. If you want to model
firm purchasing intentions but your sample comes from a list of magazine recipients, the largest
firms will be over-represented to an unknown degree because, on average, they employ more
magazine recipients than smaller firms do.
The laws of statistics require a random selection procedure, or a near equivalent such as
"systematic" or "nth-name" process, to select the sample from the frame. People are demonstrably
poor at "random" selection, so hand-picking a survey sample is likely to introduce some form of
bias. Don't forget the extent to which sample members themselves choose whether or not to
participate. Self-selection means that the sample is no longer random. The research probably
represents only three minority segments of the population:
- Those who are unusually upbeat about the survey subject or the survey sponsor
- Those with an axe to grind
- People with a lot of time on their hands, or a taste for completing surveys
These factors came together in Madison, Wisconsin, in a local TV station's phone-in poll on
lowering the drinking age. More than five times the usual number of calls was received. The
results were 78% pro, 22% con-of course, this was the home of the University of Wisconsin.
Best Sample Size
The results of a methodologically sound survey will represent the population of interest with
known accuracy. If you want to understand the theoretical foundations, keep reading; otherwise,
skip to the next section for less technical matters.
A mathematical law called the Central Limit Theorem allows the calculation of the extent to
which the real value of a variable in the population will differ from the estimated value developed
from the sample.
This is most often expressed as the famous "margin of error"–for instance, "survey results are
subject to a margin of error of ±4% at the 95% confidence level." This statement says that we
expect the true population value to be within 4 percentage points either way 95 times out of 100.
It's also true that the laws of probability tell us that, five times in a hundred, the population value
will be outside that range.
Both parts of the statement are important. The higher the confidence level, the higher the plus-orminus
factor (and vice versa). The "margin of error" is a simplification. It is usually understood to
mean "this is the maximum sampling error to which percentages will be subjected." Values
closest to 50% have the highest level of error. Those closer to 1% or 99% have much less
imprecision associated with them.
Except for the very smallest groups (populations under about 5,000), the primary determinant of
the margin of error for percentages is the tabulated sample size. This principle is somewhat
surprising, but it is the rule that allows Gallup and other polling firms to use about 1,200
interviews to represent the views of 260 million Americans.
Sampling error decreases with the square of the tabulated sample size. For a survey returning 150
responses, the margin of error will be ±8% at the 95% confidence level. Four times as many
responses (600) would be needed to cut the error level in half ±4%.
So the answer to the question "how large should the sample be?" is another question: "How
precise do you want the results to be?" If you know that you can accept ±4% at the 95%
confidence level, you also know that a tabulated sample of 600 is needed. You can work
backward from that figure, and your expected response rate, to determine how many survey
instruments will have to be mailed.
There are four more factors in the sample size decision. The first is the nature of the variables you
are measuring. The margin of error applies to percentages or proportions, However, the error
associated with statistics such as means or standard deviations depends on the variability of those
qualities within the population, not just the sample size. If you need to measure values like total
sales volume, average expenditures by category, or current salary, your sample size will probably
have to increase.
The second consideration is whether crosstabs or other segment-level analyzes will be required.
Sampling error considerations also apply to subgroups from a sample. A survey whose overall
results are statistically valid (e.g., 600 responses for a ±4% margin of error) may be invalid when
a segment of the population is broken out for closer examination. For instance, 37 responses from
recent buyers might be subject to a ±16% margin of error). One way to avoid this is to choose a
stratified sample, with a higher probability of selection for the group(s) of interest. This kind of
oversampling provides more tabulated responses for those key segments. The margin of error is
reduced, but it must be balanced by statistical weighting of the results (discussed below).
The third question is whether these results will be trended against prior data. If so, a larger sample
size will probably be required. If the first survey recorded a 50% value (±5%) and the second
survey recorded a 57% value on the same measure (again ±5% margin of error), the difference
between the two quantities is not statistically significant at the specified confident level. For
instance, the true value could have been constant at 55%, and fallen with the margin of error both
times.
Finally, the "face validity" factor sometimes influences sample sizes. This is a technical concept;
for this White Paper, it's enough to say that having a sample that's large enough for non-experts to
trust it can be more important than the quantifiable characteristics of the statistics themselves.
Next Page
Page.
[1]
[2]
[3]
[4]
[5]
[6]
[7]
|
