• Introduction
• Objectives
• Accessing the Survey
• Organizing Surveys and Results
• Creating Surveys
• Changing the Survey Scale
• Survey Intervals
• Entering Data
• Survey Scoring
• Viewing Survey Results
• Reports
• Combining Organization Data
• MCS Utilities

Appendix A: Survey Development A-1

Appendix B: Definitions of Statistical Terms B-1

INTRODUCTION

The Guide For Measuring Customer Satisfaction (MCS) is an easy way to track what customers think about the service they receive. The software provides a sample survey and questions. You can use the sample as an idea source for questions you create for your survey.

You can use MCS to:

Create your own customer survey(s)
Print and duplicate the sample customer survey
Quickly enter survey results
View the results graphically
Receive a printout of survey results

OBJECTIVES

The MCS User's Guide will show you how to:

• Access the MCS software
• Set up directories for your data
• Use the default survey and create your own survey
• Print the survey
• Enter data
• Obtain printouts and graphs of data
• Understand the results

It is recommended that you walk through the manual, following the numbered steps as practice, prior to collecting data. The practice allows you to create 2 surveys - each illustrating various aspects of the software. The guided practice should take about 1 and 1/2 hours to complete.

ACCESSING THE SURVEY

Access the Survey

1. From the DoD Performance Assessment Guide's initial screen, choose module 3, Measures of Customer Satisfaction.
2. The MCS program screen appears.
3. Press <Enter>, the MCS Main Menu appears.

The Main Menu

The Main Menu consists of three elements:

• Menu bar
• File information
• Function key selections

ORGANIZING SURVEYS AND RESULTS

You can use the MCS software to obtain information about:

Customer satisfaction
Both customer satisfaction and customer-defined importance of service attributes

For example, a survey may report that customers are not greeted with a smile (satisfaction). The same survey may also report that a friendly smile is not important when receiving help (importance). This information helps you focus on the service areas that need the most attention. (In this case a smile is not important, and not as critical to "fix.")

You can create different surveys or use the same survey for more than one element of your organization. Whichever way you choose to utilize MCS, you will need to create a system for organizing your surveys and results.

MCS provides a convenient way for you to store your survey questions and answer data. This is done by setting up directories within the Survey Directory on your computer. For practice purposes, you will be setting up 2 new directories and creating 2 new surveys.

Setting Up Directories

1. On the Main Menu (from Survey), select New File Set. The message: Do you want to change to an existing directory? appears.
2. Type N to create a new directory. The Disk Drive Specification window appears:

Enter the hard disk for a new data set:
Enter the directory for a new set of data:

(If the directory does not exist, it will be created.)

3. Enter a directory name, then press <Enter> and complete the entries as specified below.
4. The Organization Name prompt appears.
5. Type the organization to which this survey is to be administered, then press <Enter>. The Survey Name prompt appears.
6. Type a name that uniquely identifies this survey, (for practice, type in Survey 1), then press <Enter>. The directory is created. Follow the prompt and press any key to continue. The Customer Satisfaction Survey Main Menu appears.
7. Practice by creating another directory. Name the survey, Survey 2.

CREATING SURVEYS

You can create surveys using the default (sample) survey as a guide, or start from scratch with a list of questions you've created. In either case you must first create, then access, the directory in which the survey and its data are to be stored.

Creating directories was described above. To begin creating a survey, you must first select the directory in which it is to be saved.

Selecting Directories

1. On the MCS Main Menu, select New File Set. The prompt "Do you want to change to an existing directory?" appears.
2. Type Y. The list of directories you've created appears.
3. Select the directory in which the Default Survey is stored, then press <Enter>. The directory you selected is displayed in the lower left corner of the MCS Main Menu screen.

Creating a Survey using the Default (Sample)

You can create a survey using the sample stored on your computer. Before creating a survey it may be helpful to view the sample; it may also help to print the sample so you can easily review and modify the sample questions.

View the Default Survey

1. On the MCS Main Menu select View Survey. The Sample Survey appears.
2. Scroll through the survey to review the questions. Press <Esc> to return to the MCS Main Menu.

Print the Sample Survey

1. On the MCS Main Menu select Print Hard Copy Survey. The prepare printer prompt appears.
2. Follow the prompts to print the survey or return to the Main Menu.

Editing the Sample Survey

Now that you've reviewed and printed the sample survey, you can use it as a basis for your own survey. The MCS program makes it easy for you by providing an edit capability for the sample survey. You can create your own survey by editing the sample and saving it to a directory you create.

1. Switch back to a File Set you created earlier to store Survey 1, then...

Edit the Sample Survey

1. On the MCS Main Menu select Create/Edit Survey. A prompt appears asking if you wish to use questions from a previously created survey or the sample survey.
2. Type Y to select a previously created survey. The list of surveys appears.
3. Select the survey you wish to use (for practice, select Default), then press <Enter>. The questions screen appears.

Create a survey using the following input.

You can use the MCS keystrokes listed on the following page to create the introduction.

1. Press <F8> Intro, then change the introduction as follows:

Please help us improve our service to you. Circle the number that most closely represents how we're doing.

2. Press <F10> Save. The Questions screen appears.

Edit the Questions

1. Select the first question then press <F2> Edit. The cursor appears on the question line. To edit, type over the existing question. Use the MCS keystrokes listed above to delete the text to the right of the cursor.
2. Type: Were you greeted with a helpful attitude when you placed your service order? Press <Enter>.
3. Leave questions 2 and 3 unchanged. Select the fourth question and type: Was parking adequate?
4. Delete questions 5 through 10. Highlight the question, press the <Ctrl> key and <Y> key simultaneously. Then, press <Enter>.

NOTE: See Appendix A for tips about how to construct survey questions.

Edit the Closing

1. On the Survey Questions screen, press <F9> Close. The Closure window appears.
2. Edit the closing statement: We'd appreciate your ideas on how we can better serve you. Please write them below:.
3. Press <F10> Save. The survey is saved and the Questions screen appears. The survey questions, introduction, and closing statement for Survey 1 are now saved to your computer in the directory you specified at the beginning of this practice.
4. Press <F10> again to return to the Main Menu.
5. From the Main Menu, select Utilities.
6. Select Set Maximum Number of Questions. Type 4 and press <Enter>. Every time you create/edit a survey, use this option to define the number of questions.
7. From the Utilities menu, select Re-index Files. This option ensures the integrity of the files after a change to a survey has been made.

Creating a Survey Using Your Own Questions

1. Choose the File Set which stores Survey 2.
2. Select Set Maximum Number of Questions. Type 18 and press <Enter>. Before you begin to create a second survey (and any additional surveys), reset the maximum number of questions to the default, 18.
3. On the MCS Main Menu select Create/Edit Survey. A prompt appears asking if you wish to use questions from a previously created survey or the sample survey.
4. Type N to start with a blank survey screen.
5. Type in the questions shown on the screen. Change the importance column to No by using the arrow keys to highlight the field and pressing the space bar to select No.
6. Press <F10> Save to save the survey.
7. From Utilities, select Set Maximum Number of Questions. The number of questions window appears. Enter 3 and press <Enter>. Select Re-index Files.

CHANGING THE SURVEY SCALE

When you use the sample survey to build a survey, the system provides a survey scale. The scale ranges from (5) Outstanding to (1) Poor. Although the number of scale anchors (5) cannot be changed, there may be instances where the questions you create require different scale anchor names. For Survey 2, to edit the scale anchors, follow these steps.

1. From Survey, select Select the Survey Scale. The Scales Examples screen appears.
2. Press <F2> to edit the first scale anchor. The cursor appears on the line. (For Survey 2, use the Strongly Agree - Strongly Disagree scale).
3. Type the scale rating that corresponds to the number, then press <Enter>. The highlight bar moves to the next field.
4. Press <Enter> to move the highlight bar to the next number on the scale list.
5. Repeat steps 2 through 4 until the scale is updated. Press <F10>. The scale is saved and the Main Menu appears.

NOTE: The scale anchor names you choose appear once at the top of the survey. If you wish to use different scale anchor names for some questions, you can further customize your printed survey. The survey is stored in an ASCII text file (survey.txt) which can be edited in any word processing program. This file can be found in the directory you created for survey 2. The number of scale anchors (5), however, cannot be changed since all data analyses are based on a 5-point scale.

SURVEY INTERVALS

The software allows you to look at survey results on a weekly, monthly, quarterly, and yearly basis. Before you begin collecting survey results, you should choose the smallest period of time you want to examine. Choose an interval that allows you to collect around 30 people's responses. If your organization is very large and you are able to collect 30 surveys each week, you might choose weekly intervals. Monthly intervals are appropriate for most organizations. Quarterly intervals are most appropriate for very small organizations.

When you enter survey data, your computer automatically "dates" the data using today's date. If you enter data as surveys are received, you will never have to worry about overriding the computer dating. However, for most offices, this will not be feasible. Typically, surveys should be collected and stored separately by the time interval chosen above as most appropriate for your organization. Then, when someone is available to enter the survey data, he/she can "date" the surveys using an option available in MCS.

Dating Surveys

You can specify the time interval by changing the system date. This will help you group your response entries according to the interval you wish to set. For practice we're going to establish a monthly interval. Access the Main Menu, choose File Set containing Survey 2, then:

1. From Utilities, select Set New Computer System Date. The System Date window appears.
2. Enter the date: 01/01/93. The System Date window closes and the date appears at the lower right of the Main Menu screen.

ENTERING DATA

1. From the Data Entry menu select Enter Survey Data. The Data Entry screen appears.
2. Enter the data shown on the screen in the response column (Respondent 1).
3. Press <F10> to save answers.
4. Choose Y to continue.
5. Enter the following data for respondents 2 and 3.

Question	Respondent 2	Respondent 3
1. Our service was prompt.	1	2
2. Our personnel is friendly.	3	1
3. The quality of the information was excellent.	1	2

6. Use Set New Computer System Data to change the date to 02/01/93 and enter the following data:

Question	Resp 1	Resp 2	Resp 3
1. Our service was prompt.	3	3	9
2. Our personnel is friendly.	2	4	4
3. The quality of the information was excellent.	3	3	3

7. Repeat the process for 03/01/93 data.

Question	Resp 1	Resp 2	Resp 3
1. Our service was prompt.	4	5	5
2. Our personnel is friendly.	5	4	5
3. The quality of the information was excellent.	4	4	4

Entering Data for Survey 1

Access the File Set storing Survey 1. Enter the following data for the specified time period. Notice there is only 1 time period but 9 respondents. Also notice that you must enter the importance scores for this data as well.

Time Period: Current Date

Question	Resp 1		Resp 2		Resp 3
	Resp	Imp	Resp	Imp		Resp	Imp
1. Were you greeted with a helpful attitude when you placed your service order?	5	2	4	1		5	1
2. Were you given clear information?	2	4	1	5		2	4
3. Did you receive assistance with your inquiries?	4	5	4	3		5	4
4. Was parking adequate?	1	2	2	2		1	2

Question	Resp 4		Resp 5		Resp 6
	Resp	Imp	Resp	Imp	Resp	Imp
1. Were you greeted with a helpful attitude when you placed your service order?	4	2	5	1	4	2
2. Were you given clear information?	2	4	3	5	1	4
3. Did you receive assistance with your inquiries?	5	5	4	5	9	9
4. Was parking adequate?	2	2	1	2	1	2

Question	Resp 7		Resp 8		Resp 9
	Resp	Imp	Resp	Imp	Resp	Imp
1. Were you greeted with a helpful attitude when you placed your service order?	4	2	5	1	4	1
2. Were you given clear information?	1	5	2	4	2	5
3. Did you receive assistance with your inquiries?	4	4	5	5	5	5
4. Was parking adequate?	9	9	1	2	2	2

SURVEY SCORING

Once you enter responses for an interval (or intervals), you need to score the data. Survey scoring consists of three statistical elements:

Mean Score	The average score across all respondents for each question and for the survey as a whole.
Standard Deviation	An expression of variability about the mean. A large standard deviation indicates that people do not agree about the question. Some people answered with a score of 1 and others with a score of 5. People may be receiving inconsistent service. A small standard deviation (1.00 or lower) indicates that people are agreeing about the statement. The service is probably consistent.
Control Limits	Control limits are calculated based on the first interval's mean and standard deviation. They are located at three standard deviations above and below the mean. Any subsequent data points above or below these limits indicate a statistically significant improvement (above) or decline (below) in service.

MCS provides graphs and reports that help you interpret scores and understand what the data represents.

To Score a Survey

1. Access the directory in which the survey is stored. (For this lesson, access Survey 2 File Set.)
2. From Data Entry, select Consolidate Scores. The Intervals menu appears.
3. Specify the interval in which you would like to see the data.

Regardless of how you enter responses (weekly, monthly, daily, etc.), you can select an interval that corresponds to the way you would like to view the data. For example, if you enter responses weekly, you can specify a monthly interval . MCS gathers the responses entered every four weeks and presents the data in monthly segments.

VIEWING SURVEY RESULTS

Once the scores are calculated, MCS provides results viewing options. You must select a view option in order to review survey information.

You can use MCS to create three results view options:

Graphs
Character-based graphs
Reports

MCS provides a variety of graph options for your survey data. You can view data for:

A specific question
The entire survey
Satisfaction measures only
Satisfaction and importance measures

Graphs (Graphics Interface) - Satisfaction Measures Only

1. From the Main Menu, select Results. The Results menu appears.
2. Select Graphs (Graphics Interface). The list of questions appears.
3. Select the question for which you wish to view a graph, or select Entire Survey to see a composite of all survey questions, then press <Enter>. For practice, select Entire Survey. The graph should look similar to the this screen.

Character-Based Graphs - Satisfaction Measures Only

1. From the Main Menu, select Results. The Results menu appears.
2. Select Character-Based Graphs. The Question Selection list appears.
3. Select the question for which you wish to view a graph, or select Entire Survey to see a composite graph for all survey questions. For practice, select Question 1.
4. Press <Esc>. The Main Menu appears.

Graphs (Graphics Interface) - Satisfaction and Importance Measures

1. Access Survey 1 File Set.
2. From Data Entry, select Consolidate Scores. The Intervals menu appears.
3. Select Monthly intervals. The starting date window appears.
4. Type the current date, then press <Enter>.
5. Press <Esc>. The Main Menu appears.
6. From the Main Menu, select Results. The Results menu appears.
7. Select Graphs (Graphics Interface). The list of questions appears.
8. Select the question for which you wish to view a graph, or select Entire Survey to see a composite of all survey questions, then press <Enter>. For practice, select Entire Survey.
9. Since you included importance scores for this survey, a prompt appears. Indicate if you wish to include importance. Select an option (for practice, select Yes), then press <Enter>.

REPORTS

You can use the reports option to produce a printout of your survey results. When you select this option the report provides scores for each question in each interval.

1. Our service was prompt.

Interval	Mean	n	Deviation	Low	Hi
Jan93	1.67	3	0.58	1.00	3.40
Feb93	3.00	2	0.00
Mar93	4.67	3	0.58		X

Interval	Mean	n	Deviation	Low	Hi
Jan93	1.67	3	1.15	1.00	5.00
Feb93	3.33	3	1.15
Mar93	4.67	3	0.58

3. The quality of the information was excellent.

Interval	Mean	n	Deviation	Low	Hi
Jan93	1.50	2	0.71	1.00	3.62
Feb93	3.00	3	0.00
Mar93	4.00	3	0.00		X

4. Entire Survey

Interval	Mean	n	Deviation	Low	Hi
Jan93	1.63	3	0.10	1.33	1.92
Feb93	3.13	3	0.44		X
Mar93	4.44	3	0.19		X

1. Access Survey 2 File Set.
2. From Main Menu, select Results.
3. From Results, select Reports. The Printer Prompt appears.
4. Press Y to print the report. The report is sent to the printer.

The printouts show the mean score, number of respondents, and standard deviation for each time interval. The low and high scores noted next to the first interval of data indicate the control limits which were plotted on the character-based graphs. For subsequent intervals, any means which lie below the lower control limit, or above the upper control limit, are noted by an "X". For example, for Question 1, the March mean (4.67) is above the upper control limit of 3.40. Thus, an X appears under the "Hi" column. Any means receiving an "X" indicate a significant departure (either improvement or deterioration) from the initial time interval data.

You can combine organization data between different data sets. This option allows you to review the "big picture" for the entire organization. For example, suppose you have 2 similar, but separate, organizations. You would like to use the same survey for both organizations but look at the data separately. You would set up 2 different directories, each using the same survey, and store the results in the respective directories. Later you may want to look at the overall results for both organizations. Use the following procedures to combine results from 2 or more organizations. Remember, the same survey must be used in order to combine results!

Export Data

1. Access the file set for one organization.
2. From Utilities, select Export Data to Floppy. The Drive Selection window appears.
3. Select the floppy drive, then press <Enter>. The Insert Diskette prompt appears.
4. Insert a formatted floppy diskette, then press <Enter>. The file set is copied to the diskette.
5. Press any key. The Main Menu appears.

Import Data

1. On the Main Menu, create a New File Set to store your combined data. Make sure to insert the survey from organization 1 into the new file set.
2. From Utilities, select Import Data From Floppy. The Drive Selection list appears.
3. Select the floppy drive on which the exported data is stored.
4. Insert the floppy with the data. Follow the prompts until the responses are imported into the directory.
5. Access the second organization's file set.
6. From Utilities, select Export Data to Floppy. Follow the prompts until the data is exported.
7. Access the file set to which the data is to be imported.
8. From Utilities, select Import Data from Floppy. Follow the prompts until the data is imported.

Combine Scores

1. From Data Entry, select Consolidate Scores.
2. From Results, create graphs and reports.

MCS UTILITIES

Set New Computer System Date

Use this option to set a new system date.

Edit the Organizational/Survey Name

Select to change the organization and or survey name.

Clear Answer File

Deletes all responses from the currently selected survey directory/file set. MAKE BACK-UPS before using this option. Use the export option to copy data onto a floppy diskette.

View/Print Sample List

Select to review a list of questions provided as guidance for survey development.

Export Data to Floppy

Copies survey and response data from the currently selected directory to a floppy diskette.

Import Data from Floppy

Copies survey and response data from a floppy diskette into the currently selected directory.

Printer Selection

Use this option to specify the printer on which survey reports and graphs are to be printed.

Printer Port Selection

Use this option to specify the connection port on your computer to which the selected printer (above) is connected.

Set Confidence Limits

If you are knowledgeable about statistics, you can control the confidence limits. The default limits are three standard deviations above and below the mean. You may choose one or two standard deviations for the limits. This will "tighten" the spread and result in more data points lying outside the limits.

Set Maximum Number of Questions

Select to specify the maximum number of question items you are including on your survey. The system defaults to 18. If you want more or less, you must use this option to specify the maximum number of questions for the survey. Use this option AFTER

you have created/edited the survey.

Re-Index Files

Use this option after you have changed the maximum number of questions. Also, should program execution be interrupted, select this option to reorganize the files, then again execute the program.

APPENDIX A: SURVEY DEVELOPMENT

Start with the sample questions. Select from these and rewrite them if appropriate. You will probably want to write some of your own questions. As a first step, write questions without agonizing over every detail.

Ask yourself the following questions before a rewrite.

1. Will the words be uniformly understood? In general, use words that are part of the commonly shared vocabulary of the customers. For example:
a. (poor) Rate the proficiencies of the personnel.
b. (better) Personnel are knowledgeable.

2. Do the questions contain abbreviations or unconventional phrases? Avoid these to the extent possible, unless they are understood by everyone and are the common way of referring to something. For example:
a. (poor) Rate our walk-in desk.
b. (better) Personnel at our front desk are friendly.

3. Are the questions too vague? Survey items should be clear and unambiguous. If they aren't, the outcome is difficult to interpret. Make sure you ask something that can truly be measured. For example:
a. (poor) This housing office should change its procedures.

b. (better) Did you receive the information you needed?


4. Are the questions too precise? Sometimes the attempt to avoid vagueness results in items being too precise and customers may be unable to answer them. For example:
a. (poor) Each time I visit the Housing Office, the waiting line is long.
b. (better) Generally, the waiting line in the Housing Office is long.

5. Are the questions biased? Biased questions influence the customer to respond in a manner that does not correctly reflect his/her opinion. For example:
a. (poor) How much do you like our Housing Office?
b. (better) Would you recommend our Housing Office to a friend?

6. Are the questions objectionable? Usually this problem can be overcome by asking the question in a less direct way. For example:
a. (poor) Are you living with someone?
b. (better) How many people, including yourself, are in your household?

7. Are the questions double-barreled? Two separate questions are sometimes combined into one. The customer is forced to give a single response and this, of course, would be ambiguous. For example:
a. (poor) The Housing Office is attractive and well-maintained.
b. (better) The Housing Office is attractive.

8. Are the answer choices mutually exclusive? The answer categories must be mutually exclusive and the respondent should not feel forced to choose more than one. For example:
a. (poor) Scale range: 1, 2-5, 5-9, 9-13, 13 or over
b. (better) Scale range: 0, 1-5, 6-10, 11-15, 16 or over

9. Are the answer choices mutually exhaustive? The response categories provided should be exhaustive. They should include all the possible responses that might be expected. For example:
a. (poor) Scale range: 1-5, 6-10, 11-15, 16-20
b. (better) Scale range: 0, 1-5, 6-10, 11-15, 16 or over


---

STEP THREE

Show the questions to some co-workers and some customers. Get their reactions as to what the questions mean to them. Revise as needed.

---

For more detailed information about writing questions, see:

Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York, NY: John Wiley & Sons, Inc.

Hayes, R. E. (1992). Measuring customers satisfaction: Development and use of questionnaires. Milwaukee, WI: ASQC Quality Press.

---

APPENDIX B: DEFINITIONS OF STATISTICAL TERMS

Mean - The mean (arithmetic average) is the sum of the values of all responses divided by the number of responses.

Standard Deviation - A standard deviation is the square root of the variance of a distribution. The variance is computed by summing the squared differences from the mean for all responses and then dividing by 1 less than the number of respondents. If all responses are identical, there is no variance and the standard deviation would be 0. The larger the standard deviation, the more spread out, or less agreement, there exists among responses.

Control Limits - Control limits are calculated using the standard deviation and mean.


Upper Control Limit = mean + (3 times the standard deviation)
Lower Control Limit = mean - (3 times the standard deviation)

The software uses the first interval of data that you have chosen for consolidating scores. The control limits are calculated using the mean and standard deviation of the first interval of data.

By looking at the means of the next 11 intervals of data and examining whether they lie above or below the control limits, you can determine whether you are significantly improving (or falling behind) on the level of service you are providing your customers.

---