Chapter Ten
Questionnaire
Design
Chapter Objectives
• Discuss questionnaire design as a series of
interrelated steps
• List and describe the different question forms and
discuss when and how they can be used most
effectively
• Recognize and reword problem questions.
• State the suggested guidelines for proper question
sequencing
• Discuss the importance of questionnaire appearance
and layout
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 2
Chapter Objectives (Cont’d)
• Explain why pretesting the questionnaire can be
critical
• Describe the design benefits inherent in
computerized and online questionnaires
• List the features of a good cover letter
• Create appropriate openers for personal and
telephone interviews
• Discuss the questionnaire guidelines that would be
relevant in the design of an observation form
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 3
Circuit City – Bizrate Customer
Satisfaction Study
• Shopzilla.com is a division of Biz-Rate.com
that collects data from previous shoppers at a
particular online merchant
• Shopzilla reports satisfaction ratings for a
variety of online retailers so that shoppers
can compare merchants across 15 service
dimensions
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 4
Circuit City – Bizrate Customer
Satisfaction Study (Cont’d)
• Circuit City has extended this survey to its instore customers through the use of checkout
receipts that invite shoppers to the Biz-Rate
survey site
• The shoppers are them asked to rate Circuit
City on ten shopping dimensions
• This data can be used by both shoppers and
Circuit City to improve customer satisfaction
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 5
Shopzilla.com
Courtesy of Shopzilla
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 6
Questionnaire
• A set of questions designed to generate the
data necessary for accomplishing a research
project's objectives
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 7
Complexity of Questionnaire Design
• Consider the following question that appeared in a mail
questionnaire from an presumably experienced researcher
• Do you consider the many marketing research texts adequate
for most of your business majors at the undergraduate level ?
_____ Yes
____ No If no; briefly,why not?
• What do you think? Easy or difficult to answer?
• What exactly do the words many, adequate, and most refer to in
this question?
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 8
Questionnaire's Impact on Data Accuracy
• It must communicate to the respondent what
the researcher is asking
• It must communicate to the researcher what
the respondent has said
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 9
Sources of Distortion
• A sloppy questionnaire can lead to a great
deal of distortion
• The entry of an intermediary, or the
interviewer, into the communication channel
can also distort the questionnaire
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 10
Exhibit 10.1 Questionnaire: Link Between
Researcher and Respondents
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 11
Questionnaire Design Elements
• Does each question have the most appropriate form
(e.g., structured versus nonstructured)?
• Is each question relevant and properly worded to
obtain meaningful, valid responses?
• Is the sequencing of the questions likely to introduce
any bias?
• Are the layout and appearance of the questionnaire
conducive to accurate and easy data collection?
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 12
Exhibit 10.2 The Questionnaire
Design Process
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 13
Question Form
• Nonstructured questions
– Open-ended
• Structured questions
– Fixed-response
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 14
Nonstructured Questions
• Consider the following illustrative questions
How old are you?___________________
What do you like most about owning your own
home? _____________
Will you please describe your thoughts about a
person who shoplifts items from a grocery
store to keep from going hungry?
_____________________________________
_____________________________________
____________
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 15
Structured Questions
• In which of the following categories does your
age fall?
_______ Less than 18
_______ 18 to 30
_______ 31 to 45
_______ 46 to 60
_______ Over 60
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 16
Structured Questions:
Dichotomous Questions
• Do you smoke cigarettes?
___Yes ___No
• Have you ever watched CNN News?
___Yes ___No
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 17
Multiple-Category Questions
• In your opinion, which product category is the most
suited for making purchases on the Internet?
____Automobiles
____Books
____Electronic Items (TVs, video cameras, VCR and
DVD players, etc)
____Videos, CDs
____Computer-related items
____Travel (airlines, car rental, hotel reservations, etc.)
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 18
Multiple-Category Questions (Cont’d)
• What do you like about Bank of America’s
online banking service?
____24 hour service
____Transaction privacy
____No need to travel – do it from home
____Paying bills online– no need to mail bills
____Other ____(please specify)
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 19
Response Category Sequence
• Approximately how many long-distance
telephone calls do you make per week?
______ 0 to 1 call
______ 2 to 3 calls
______ 4 to 5 calls
______ 6 to 7 calls
______ More than 7 calls
• The response categories in this question
follow a natural sequence
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 20
Response Category Content
• Response choices should be
– Collectively exhaustive
• taken together
– Mutually exclusive
• not overlap
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 21
Response Category
• The response choices are collectively exhaustive but
not mutually exclusive
• On the average, how many cans of cola do you drink
per week
____0 to 3 cans
____3 to 6 cans
____More than 6 cans
• Changing the 3 in the second category to a 4 will
make the response choices mutually exclusive
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 22
Response Category (Cont’d)
• What do you like about Bank of America’s online
banking service?
____24 hour service
____Transaction privacy
____No need to travel – do it from home
____Paying bills online – no need to mail bills
____Other ____(please specify)
• What is wrong with this question ?
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 23
Response Category (Cont’d)
• Modify the question to read, "Which of the
following do you like most about Bank of
America’s Online Banking Service?"
• With this revision each respondent would, or
should, check only one response category
• What do you like about Bank of America’s
Online Banking Service? (Please check as
many categories as apply.)
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 24
Number of Response Categories
• Multiple-category questions fall between
dichotomous questions & open-ended
questions
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 25
Question Relevance and Wording
Yes
Can the research
objective(s) be
fulfilled without
asking this
question?
question should be deleted
Copyright © Houghton Mifflin Company. All rights reserved.
No
The question may still be
a candidate for revision
or deletion and must be
further examined
10 | 26
Can the Respondent
Answer the Question?
• In your opinion, how many students in your
marketing courses will become potentially
successful marketing managers?
Ten percent
______
Twenty percent ______
Thirty percent ______
______percent ______
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 27
Can the Respondent
Answer the Question? (Cont’d)
• Consider the following question taken from a
questionnaire used by a marketing research
firm in conducting a face-to-face survey of
female household heads
– How much has your family spent in the last 12
months on cookware? $ _______
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 28
Writing a Questionnaire for
International Markets
• Caution
– Use of the same word can have different
meanings in different cultures
• Word Equivalence
– In Japan, China and non-English speaking
European countries
• “Very Satisfied” means “Somewhat Satisfied”
• “Very Satisfied” should be replaced by “Totally
Satisfied”
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 29
Will the Respondent
Answer the Question?
• Questions about personal financial matters or
sexual behavior are examples of sensitive
questions
• Words used in the questions should not be
ambiguous to the respondents
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 30
Double-Barreled Questions
• Suppose the following question is to be used in conducting a
survey of the general public
– Do you feel firms today are concerned about their
employees and customers?
_____ Yes
_____ No
_____ No opinion
• A “no” response can be interpreted three different ways
– The respondent feels firms are concerned about neither
employees nor customers
– The respondent feels firms are concerned about employees
but not customers
– The respondent feels firms are not concerned about
employees although they are concerned about customers
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 31
Avoiding Double-Barreled Questions
• This revision is usually accomplished by breaking
one question into several questions
• Thus our illustrative question can be reworded as two
separate questions
– Do you feel firms today are concerned about their
employees?
___ Yes ___ No
___ No opinion
– Do you feel firms today are concerned about their
customers?
___ Yes ___ No
___ No opinion
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 32
Leading Questions
• Don't you think offshore drilling for oil is
environmentally unsound?
___Yes
___ No
___ No opinion
• Do you think the quality of products on the
market today is as high as it used to be ten
years ago?
___ Yes
___ No
Copyright © Houghton Mifflin Company. All rights reserved.
___ No opinion
10 | 33
Avoiding Leading Questions
• The two leading questions we saw earlier can
be modified for neutrality as follows
– What is your feeling about the environmental
impact of offshore drilling for oil?
___ Offshore drilling is environmentally sound
___ Offshore drilling is environmentally unsound
___ No opinion
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 34
Avoiding Leading Questions (Cont’d)
• How do you feel about the quality of products on the
market today compared with the quality of products
on the market ten years ago?
___Product quality is better now than ten years ago.
___Product quality is worse now than ten years ago.
___Product quality is the same now as ten years ago.
___ No opinion.
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 35
One-Sided Questions
• Do you feel firms today are concerned about
their employees?
___Yes
___ No
___ No opinion
• Would you agree or disagree that the
deregulation of the airline industry has
benefited customers?
___Yes
___ No
Copyright © Houghton Mifflin Company. All rights reserved.
___ No opinion
10 | 36
Avoiding One-sided Questions
• Do you feel firms today are concerned about their
employees?
___Concerned
___ Unconcerned
___ No opinion
• Do you feel the deregulation of the airline industry
has benefited consumers, has had no impact on
consumers, or has hurt consumers?
___Has benefited customers
___Has had no impact on customers
___Has hurt customers
___No opinion
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 37
Unbalanced Questions
• How important is price to you in buying a new
car?
___More important than any other factor
___Extremely important
___Important
___Somewhat important
___Unimportant
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 38
Balanced Questions
• How important is price to you in buying a new
car?
___Very important
___Relatively important
___Neither important nor unimportant
___Relatively unimportant
___Very unimportant
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 39
Avoiding Questions with Implicit
Assumptions
• Are you favorable, indifferent, or unfavorable toward
a 10% increase in city taxes?
___Favorable
___Indifferent ___ Unfavorable
• The above question can be improved as
– Are you favorable, indifferent, or unfavorable toward a
10% increase in city taxes to repair potholes in the
city's streets?
___Favorable
___Indifferent ___Unfavorable
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 40
Avoiding Questions with Implicit
Assumptions (Cont’d)
• Do you drink coffee while watching football, baseball,
and so on?
____ Yes
____ No
• The above question can be reworded as
– During which of the following activities do you drink
coffee?(Check as many as apply.)
___ Attending ballgames
___Attending sporting events other than ballgames
___Watching ballgames on TV
___Watching sporting events other than ballgames on TV
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 41
Guarding Against Errors
• How often do you eat eggs for breakfast?
_____ Frequently
_____ Occasionally
_____ Rarely
_____ Never
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 42
Guarding Against Errors (Cont’d)
• To guard errors due to misinterpretations, the
previous question can be revised as
– On average, how many days per week do you
eat eggs for breakfast?
___ Every day
___ 5 or 6 days
___ 3 or 4 days
___ 1or 2 days
___ Less than 1 day per week
___ Never eat eggs for breakfast
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 43
Complex Questions
• In which of the following do you typically invest your
liquid assets?
___Insured accounts
___Stock market
___Insured accounts and stock market
___Other accounts
• Of the total number of miles you drove during the
past month, approximately what percentage was
traveled in driving to and from work?
___Percent
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 44
How Does USA Today Avoid
Biases in Polling?
• Question posed to Jim Norman in Hackensack, New
Jersey
• “With all of the little idiosyncrasies available to
pollsters to potentially skew the answers of the
respondents, e.g., voice tone, making statements vs.
asking questions (a.k.a. push polling), or the
historical success of polling where the last choice
offered automatically becomes the preferred
answer,with all of these potential manipulations, how
does a polling unit like yours avoid ANY appearance
of partisanship or impropriety?”
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 45
How Does USA Today Avoid
Biases in Polling? (Cont’d)
• Jim Norman Responds
– “Voice Tone: There can be a wide range in the
professionalism of the interviewers from polling outfit to
polling outfit, but the best outfits use people who are
TRUE professionals. With them voice tone is not an
issue.”
– “Making statements vs. asking questions: If you sat
through just one of our often excruciating conference
calls to determine question wording, you would know
how much the people involved are concerned that we
wring out any bias in our questions. It still can
happen,but I think it’s rare.”
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 46
How Does USA Today Avoid
Biases in Polling? (Cont’d)
• Jim Norman Responds (Cont’d)
– “Last choice offered automatically becomes
the preferred answer: we rotate the
responses, so that each possible response is
the last one offered to an equal number of
respondents.”
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 47
Sequencing of Questions
• Position of Demographic and Sensitive
Questions
• Arrangement of Related Questions
• Funnel and Inverted-Funnel Sequences
• Skip Patterns
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 48
Position of Demographic and
Sensitive Questions
• Respondents' personal or demographic
characteristics, i.e., age, education level, or
income are placed at the end of the
questionnaire
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 49
Arrangement of Related Questions
• Cluster questions that focus on the same topic
– In a survey of business executives
• Views on the economy
• Competition
• Employee turnover
• Grouping questions into meaningful clusters can
increase respondents' ease in answering the
questions and reduce the chance of response errors
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 50
Funnel Sequences
• Which of the following types of TV shows do
you watch? (Check all categories that apply.)
____ News shows ____ Quiz shows____ Game
shows
• Which of the following types of TV shows do
you like the most? (Check as many
categories as apply.)
____ News shows ____ Quiz shows____ Game
shows
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 51
Funnel Sequences (Cont’d)
• Which of the following specific shows did you
watch during the past seven days? (Check
"yes" or "no" for each show listed.)
Deal or No Deal
Wheel of Fortune
CBS Evening News
Copyright © Houghton Mifflin Company. All rights reserved.
___ Yes
___ Yes
___ Yes
___ No
___ No
___ No
10 | 52
Inverted-Funnel Sequences
• A survey of consumers to measure the image
they have of Netgrocer.com
• What do you think of the specific attributes of
Netgrocer.com?
___ Website
___ Products
___ Customer Service
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 53
Skip Patterns
• Proper sequencing of questionnaire items is
essential to avoid complicated skip patterns
that may confuse interviewers and
respondents
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 54
Question Sequence Needing Improvement
• 11. Do you own or rent your current place of
residence?
Own
Rent
• 12. How long have you lived in this state?
Less than one year
One year to less than 5 years
5 years or more
• 13. If you have lived outside this state, in which state
did you live immediately before moving here?
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 55
Question Sequence Needing Improvement
(Cont’d)
• 14. If you have lived in this state 5 years or more and
if you currently rent your residence, do you intend to
buy a home within the next two years?
Yes
No
Don’t know
• 15. How long have you lived in your current place of
residence?
Less than one year
One year to less than 5 years
5 years or more
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 56
Improved Question Sequence
• 11. Have you always lived in this state?
Yes
No
(Go to Question 12)
(Go to Question 11a)
• 11a. In which state did you live immediately
before moving into this state?
• 12. How long have you lived in this state?
Less than one year
One year to less than 5 years
5 years or more
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 57
Improved Question Sequence (Cont’d)
• 13. Do you own or rent the place where you live?
Own (Go to Question 15)
Rent (Go to Question 14)
• 14. Do you intend to buy a home within the next two
years?
Yes
No
Don’t know
• 15. How long have you lived in your current place of
residence?
Less than one year
One year to less than 5 years
5 years or more
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 58
Questionnaire Appearance And Layout
• Version 1
– How old are you?
___Less than 18 ___18 to 25 ___26 to 40 ___Over 40
• Version 2
– How old are you?
___Less than 18
___18 to 25
___26 to 40
___Over 40
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 59
International Marketing Research
• First Step
– A professional who is fluent in both the languages
should translate the original questionnaire
• Second Step
– A different translator must translate the local language
questionnaire back to the original version
• Third Step
– Researchers should compare the original version and
the translated original version to ensure that they are
identical in meaning
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 60
International Marketing Research (Cont’d)
• Fourth Step
– The local language version should also be
checked by the marketing research
professionals in the country of interest
• Fifth Step
– The local language version should be
pretested to make sure that the questionnaire
serves the intended purpose
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 61
Pretesting
• Pretesting is administering a questionnaire to
a limited number of potential respondents and
other individuals capable of pointing out
design flaws
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 62
Questionnaires for Computerized and
Online Interviewing
–
–
–
–
–
Randomizing response choices
Checking for response consistency
Incorporating complex skip patterns
Personalization
Ability to draw questions from computer
libraries
– Adding “new” response categories
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 63
Randomizing Response Choices
• The computer can be programmed to
randomize the order of presentation of the
categories separately for each respondent for
multiple-category questions in which category
sequence may influence response choice
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 64
Checking for Response Category
• The computer can be programmed to check
for consistency between the response given
to a question at hand and responses given to
certain key questions asked earlier
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 65
Incorporating Complex Skip Patterns
• Computerized online interviewing can handle
very complex skip patterns
• The computer can check as many “if then”
statements and previous responses as
necessary,decide which question should be
asked next and pose that question on the
computer screen instantaneously
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 66
Personalization
• When a respondent’s name is entered into
the computer keyboard at the start of an
interview, that name can be automatically
inserted into key questions and instructions
throughout the questionnaire to provide a
degree of personalization that would be very
time consuming to achieve in noncomputerized interviewing
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 67
Ability to Draw Questions from
Computer Libraries
• A questionnaire writer simply clicks and drags
a standard questionnaire from an online
library and modifies it as necessary to create
a desired questionnaire
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 68
Adding New Response Categories
• In a multiple-category question with an “other”
category, when a prespecified number of
respondents provide the same open-ended
response to the “other” category, that
response can be automatically converted into
an explicit check off category by adding it to
the set of prespecified categories
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 69
Designing Cover Letters For Mail
Questionnaires
• Primary purpose
– To win the cooperation of respondents
• A good cover letter should be
– Concise
– Objective
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 70
Exhibit 10.5 Illustrative
Cover Letter and Key
Features
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 71
Openers for Personal and
Telephone Interviews
• Features
– An appropriate salutation, such as "Good
morning! I am Ralph Johnston with Marketing
Research Associates Company.”
– A brief statement about the project and its
purpose, such as "We are conducting a survey
of heads of household regarding their feelings
about modern home appliances.”
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 72
Openers for Personal and
Telephone Interviews (Cont’d)
• Features (Cont’d)
– An indication of how long the interview might
last, such as "This survey should take no more
than ten minutes.”
– A polite request for permission to conduct the
interview, such as "May I please talk to the
male or female head of your household for a
few minutes?"
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 73
Designing Observation Forms
• Guidelines
– Wording
– Sequencing
– Layout
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 74
Exhibit 10.6 Observation Form for Recording Characteristics and
Behavior of Customers Stopping at a Special Display
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 75
Creating Online Questionnaires
• Creating an online questionnaire is simplified
by a growing number of websites that allow
researchers to design and launch their survey
from one site
• SurveyZ.com is an example of one such site
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 76
SurveyZ
• SurveyZ allows users to register for a free
trial and sample the functionality of the
website
• There are also a number of sample surveys
posted for users to view
• The site is a powerful tool that allows anyone
to create a professional looking surveys very
quickly
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 77
Using SurveyZ
• Surveyz! The Ultimate Survey Software
Solution - Online Survey Software - Customer
Satisfaction Survey - Employee Survey - Web
Survey Software
• Follow the screen shots in your text to explore
the website
Copyright © Houghton Mifflin Company. All rights reserved.
10 | 78
Descargar

Chapter 10 Questionnaire Design