Idaho Business Review Article - Chavez launches survey service - by Brad Carlson 10/23/2006...

Todd Chavez has found his niche helping others fill theirs.

This summer he founded Chavez Survey Research, which focuses on Internet-based methods of helping companies research their companies and customers.

The owner is a longtime believer in the power of the Internet. He headed marketing for the old Micron Internet Services from January 1995 to 2000.

“It was a lot like the Wild West,” Chavez said. “It’s amazing how that has changed.”

Chavez moved to Micron Technology, and worked first as Web marketing manager from 2001 to ’03, then as organizational communication planning manager from 2004 until late June.

A week after leaving Micron, Chavez incorporated his survey research company and got his first client — Micron. After being approved as a Micron vendor, Chavez started what would be a seven-week project for Micron’s information services department.

Surveys have always interested Chavez, 47. At the University of Idaho he did a comprehensive survey project, which led to a master’s degree in resource recreation and tourism. He also conducted online surveys while working for Micron Technology.

His niche is helping companies and organizations make the most of their Web-based survey research efforts. His company conducts comprehensive “end-to-end” surveys or helps with particular components of a project.

He uses a professional-level survey engine that requires an annual fee. But an engine typically provides only a small piece of the larger project, he said.

“For example, even if a customer intends to conduct a survey using an online tool like Survey Monkey, I strongly suggest consulting with a firm like mine to develop a valid question list and effective questionnaire design,” Chavez said.

Chavez contracts with statisticians as needed, and with Chavez Writing and Editing, a six-year-old firm that his wife, Natalie, operates.

Chavez said the market for Internet-based survey research is expanding.

“There is a growth opportunity there, not only for myself but for other research companies, and that will allow me to specialize,” he said.

There are Internet-based providers based elsewhere, and companies like Clearwater Research and Northwest Research — both in Boise — offering a full suite of research services and modes.
One of Chavez’s specialties is helping clients with question wording and sequence, which can increase the response rate and the value of the information, he said.

Good surveys, regardless of modes used, remain grounded in strong research and statistical principles, Chavez said.

A high-quality survey can benefit organizations of all sizes — a company might systematically ask customers what they like and don’t like, or need and don’t need.

Compared to traditional survey methods, an Internet-based approach can offer greater speed and accuracy, a lower cost to the surveying entity per respondent, and ease of participation, he said.

“It’s just as valuable for smaller companies as it would be for Micron or HP,” Chavez said.

Find the Idaho Business Review on the Web at www.idahobusiness.net for Idaho's best business news and information.

Writing Survey Questions: More Then Meets the Eye...





I recently reviewed three online surveys - one from a government office, another from a radio station, and a third from a university. Each was a well-intentioned effort to learn more about their respective audiences' opinions. Unfortunately, each was also critically flawed by the same fundamental problem: Question construction errors. Before a single response was received, the usefulness of each survey was in doubt.

What's most disconcerting is the likelihood that decisions will be made, and actions taken, based on poor information. Each organization would have been better served by avoiding the rush to push their survey out, slowing down, and taking a step back to carefully consider how they were asking the questions.

Let's look at an example of how a question can be revised to provide better information.

Example 1

Do you think our local news coverage has any "slant" or "bias?"

• Very liberal
• Somewhat liberal
• Fair and balanced
• Somewhat conservative
• Very conservative

Question analysis

The question seems straight-forward enough, and the response categories appear balanced. Yet confusion exists between the question and the response categories. The question prepares the respondent that news coverage "bias" is going to be measured, but the response categories introduce the terms liberal, fair, balanced and conservative.

Furthermore, an assumption is made that the question and response categories are referring to degree of bias (liberal or conservative) in a political sense.

The discontinuity between the question and response categories tax the respondent, increasing the likelihood of inaccurate measurement as well as survey abandonment.

In addition, and to my knowledge, the audience to which the survey invitation was sent consists largely of financial contributors. In which response category do you think most contributors would place themselves? Fair and balanced, perhaps?

Question revision

A better approach is to ask two questions, using question skipping logic, to obtain better information:

Do you think our local news coverage is politically biased? (Select one)

• Yes
• No

Skipping logic built into the survey would then present a follow-up question to the respondents selecting "Yes" to the bias question.


To what degree do you feel our local news coverage is politically liberal or conservative? (Select one)

• Extremely liberal
• Very liberal
• Somewhat liberal
• Somewhat conservative
• Very conservative
• Extremely conservative

In this case, a six-point scale is used because there's no need for a neutral response category. By the very nature of the first question, we know that a respondent answering "Yes" is going to pick a side (liberal or conservative) in the follow-up question. The follow-up question is simply measuring the degree of liberal or conservative.


Example 2

What are your favorite sources of local news?







Question analysis

There are at least four problems with this question and its associated response categories.

First, it's implied, but never stated, that the respondent is supposed to select a response for each news source presented. Inadequate instructions may result in the unintentional skipping of response categories, thereby reducing the amount of useable data.

Second, the scale items are incongruent. Often, Sometimes and Never are measures of frequency, but the item "One of my favorites" is not. Clearly, this poses a problem when interpreting the results.

Third, it's implied that the respondent will be able to correctly select the local NBC, ABC and CBS news programs without the aid of the much promoted local affiliate call signs.

Fourth, the "Other" response category is fine in itself, but the lack of a way for the respondent to then identify the news source is unfortunate.


Question revision

The revised question frames the frequency items to provide more useful data and provides instructions to increase complete responses. In addition, call signs were added to the response categories to help the respondent correctly identify local news on network affiliates and, finally, an option is presented to allow a respondent to identify a local news source used but not included on the list.


In an average week, how often do you use each of these sources for local news? (Select a response for each news source)







If you selected an "Other" source of local news, please provide the name of the source here:




These are just two examples demonstrating the importance of properly constructed questions and response categories. The key to gathering good survey intelligence begins with taking time to compose the best question list possible. I encourage you to invite a survey research specialist to help review and revised your question list the next time you begin a survey project. A small investment in professional assistance will pay huge dividends in the value of the intelligence you gain.

Thanks for reading...

Todd Chavez
Chavez Survey Research, Inc.
www.chavezsurveyresearch.com
(208) 429-8945

Surveying: Do it yourself or use a professional?...

When it comes to knowing what you need to know about your customers, employees or your membership, survey research is the most efficient way shed new light on the landscape in which you work. How well you perform survey research will determine just how brightly the light will shine.

Today, the Internet provides an exciting alternative in conducting survey research in your hands. New online tools are readily available and make surveying a snap! What could be better?

Actually, a lot could be better. An online survey tool is like any other tool - In the wrong hands, even with the best of intentions, it can damage. Why is this the case? Performed well, survey research is a skill that delivers accurate and reliable information. Performed poorly, it delivers inaccurate and unreliable information. When the survey report hits your desk, how do you, as the decision maker, know what you're getting? Making decisions based on good information leads to success.

The best way to avoid the "do it yourself surveying" pitfall is to engage research professionals to guide you through the surveying process - The results will be worth the effort. And the cost? Probably not as much as you think, and certainly not as much as the cost of making decisions based on bad information.

So when can you safely perform your own surveys?

Ask yourself these questions:

1. Am I getting a quick "pulse" about a single topic by asking just a few questions?
2. Is my survey target audience small? (A population of 50 or less)
3. Is my audience likely to respond to a survey coming from me?
4. Am I using the survey results simply to verify a decision I'm making based on reliable information I have from other sources?
5. Am I comfortable interpreting the results? (Typically frequency tables and free text comments)

If you can answer, "Yes" to each of these questions, you're probably OK to proceed without help. If your answer is "No" to any of the questions, you should consider taking advantage of a professional's experience.

A professional's experience is very beneficial when your survey project is characterized by any one of these considerations:

• You seek information about more than one topic
• You will ask more than, about, five questions
• Your survey population is larger than 50 people
• Your survey population is often thought of as being segmented by geography, demographics, or organizational structure
  
• You need to ask different questions to subsets of the population based on how they respond to other questions
• You need to be absolutely sure that the questions and their response categories are worded and sequenced perfectly
• Your using the survey results as a primary source of information on which to base key decisions
• You need to be able to compare responses from subsets of the population with one another
• You need to identify the results that are the most important—the statistically significant "gems" of information
• You need to identify trends over time
• You need to be able to interpret free text comments in a defendable, unbiased manner

The bottom line?

In a few instances, performing your own survey research is going to be fine. Yet I think you'll find the quality and credibility you gain from using a professional will deliver much more.

As a parting note, there is one item for which I always recommend using a survey research professional - Questionnaire Composition and Design! The process of creating a series of perfect questions, whether it's five or 50, is far more complicated than generally thought. Question wording and the assigning of appropriate response categories is a science in itself. Placing them in the right order and creating a layout that aids in navigating the survey is equally as challenging. S adly, I review too many surveys that are dysfunctional from the very beginning due to poorly worded questions.

Thanks for reading...

Best regards,

Todd Chavez
Chavez Survey Research, Inc.
www.chavezsurveyresearch.com
(208) 429-8945