Performance Assessment Guide
MODULE 3

Guide For Measuring Customer Satisfaction User's Guide


TABLE OF CONTENTS

Appendix A: Survey Development A-1

Appendix B: Definitions of Statistical Terms B-1


INTRODUCTION

The Guide For Measuring Customer Satisfaction (MCS) is an easy way to track what customers think about the service they receive. The software provides a sample survey and questions. You can use the sample as an idea source for questions you create for your survey.

You can use MCS to:


OBJECTIVES

The MCS User's Guide will show you how to:


It is recommended that you walk through the manual, following the numbered steps as practice, prior to collecting data. The practice allows you to create 2 surveys - each illustrating various aspects of the software. The guided practice should take about 1 and 1/2 hours to complete.


ACCESSING THE SURVEY

Access the Survey

1. From the DoD Performance Assessment Guide's initial screen, choose module 3, Measures of Customer Satisfaction.
2. The MCS program screen appears.
3. Press <Enter>, the MCS Main Menu appears.

The Main Menu

The Main Menu consists of three elements:


ORGANIZING SURVEYS AND RESULTS

You can use the MCS software to obtain information about:

Customer satisfaction measures the level of satisfaction that customers report about the service you provide. Customer-defined important service attributes measure the importance (to the customer) of the types of services you provide.

For example, a survey may report that customers are not greeted with a smile (satisfaction). The same survey may also report that a friendly smile is not important when receiving help (importance). This information helps you focus on the service areas that need the most attention. (In this case a smile is not important, and not as critical to "fix.")

You can create different surveys or use the same survey for more than one element of your organization. Whichever way you choose to utilize MCS, you will need to create a system for organizing your surveys and results.

MCS provides a convenient way for you to store your survey questions and answer data. This is done by setting up directories within the Survey Directory on your computer. For practice purposes, you will be setting up 2 new directories and creating 2 new surveys.

Setting Up Directories

1. On the Main Menu (from Survey), select New File Set. The message: Do you want to change to an existing directory? appears.
2. Type N to create a new directory. The Disk Drive Specification window appears:


Enter the hard disk for a new data set:
Enter the directory for a new set of data:

3. Enter a directory name, then press <Enter> and complete the entries as specified below.
4. The Organization Name prompt appears.
5. Type the organization to which this survey is to be administered, then press <Enter>. The Survey Name prompt appears.
6. Type a name that uniquely identifies this survey, (for practice, type in Survey 1), then press <Enter>. The directory is created. Follow the prompt and press any key to continue. The Customer Satisfaction Survey Main Menu appears.
7. Practice by creating another directory. Name the survey, Survey 2.


CREATING SURVEYS

You can create surveys using the default (sample) survey as a guide, or start from scratch with a list of questions you've created. In either case you must first create, then access, the directory in which the survey and its data are to be stored.

Creating directories was described above. To begin creating a survey, you must first select the directory in which it is to be saved.

Selecting Directories

1. On the MCS Main Menu, select New File Set. The prompt "Do you want to change to an existing directory?" appears.
2. Type Y. The list of directories you've created appears.
3. Select the directory in which the Default Survey is stored, then press <Enter>. The directory you selected is displayed in the lower left corner of the MCS Main Menu screen.

Creating a Survey using the Default (Sample)

You can create a survey using the sample stored on your computer. Before creating a survey it may be helpful to view the sample; it may also help to print the sample so you can easily review and modify the sample questions.

View the Default Survey

1. On the MCS Main Menu select View Survey. The Sample Survey appears.
2. Scroll through the survey to review the questions. Press <Esc> to return to the MCS Main Menu.

Print the Sample Survey

1. On the MCS Main Menu select Print Hard Copy Survey. The prepare printer prompt appears.
2. Follow the prompts to print the survey or return to the Main Menu.

Editing the Sample Survey

Now that you've reviewed and printed the sample survey, you can use it as a basis for your own survey. The MCS program makes it easy for you by providing an edit capability for the sample survey. You can create your own survey by editing the sample and saving it to a directory you create.

1. Switch back to a File Set you created earlier to store Survey 1, then...

Edit the Sample Survey

1. On the MCS Main Menu select Create/Edit Survey. A prompt appears asking if you wish to use questions from a previously created survey or the sample survey.
2. Type Y to select a previously created survey. The list of surveys appears.
3. Select the survey you wish to use (for practice, select Default), then press <Enter>. The questions screen appears.

Create a survey using the following input.

You can use the MCS keystrokes listed on the following page to create the introduction.

1. Press <F8> Intro, then change the introduction as follows:

2. Press <F10> Save. The Questions screen appears.

Keystrokes to Edit Introduction and Closing Statements

FeatureKeystroke
Abort Edit/Return Original<Esc>
Up one line<Up Arrow>
Down one line<Down Arrow>
Left one character<Left Arrow>
Right one character<Right Arrow>
Left one word<Ctrl>/<Left Arrow>
Right one word<Ctrl>/<Right Arrow>
Beginning of line<Home>
End of line<End>
Beginning of window<Ctrl>/<Home>
End of window<Ctrl>/<End>
Previous window<PgUp>
Next Window<PgDn>
Beginning of memo<Ctrl>/<PgUp>
End of memo<Ctrl>/<PgDn>
Insert/move four spaces<Tab>
Beginning of line<Enter>
Delete line<Ctrl>/Y
Delete character left<Backspace>
Delete character at cursor<Del>
Insert mode on/off<Insert>

Edit the Questions

1. Select the first question then press <F2> Edit. The cursor appears on the question line. To edit, type over the existing question. Use the MCS keystrokes listed above to delete the text to the right of the cursor.
2. Type: Were you greeted with a helpful attitude when you placed your service order? Press <Enter>.
3. Leave questions 2 and 3 unchanged. Select the fourth question and type: Was parking adequate?
4. Delete questions 5 through 10. Highlight the question, press the <Ctrl> key and <Y> key simultaneously. Then, press <Enter>.


NOTE: See Appendix A for tips about how to construct survey questions.


Keystrokes to Edit Question Fields

FeatureKeystroke
Record Entry, go to next field<Enter>
Restore field to original values<Ctrl>/<U>
Abort edit/return original<Esc>
Left one character<Left Arrow>
Right one cha Right one character<Right Arrow>
Beginning of field<Home>
End of field<End>
Delete word right<Ctrl>/<T>
Delete to end of field<Ctrl>/<Y>

MCS Function Keys

Function KeyFeature
F1: [Help]View help screen
F2: [Edit]Field edit mode
F7: [Selct]Insert question from sample list
F8: [Intro]Edit introductory statements
F9: [Close]Edit closing statements
F10: [Save]Save changes
Esc: [Quit]Abort edit/return original

Edit the Closing

1. On the Survey Questions screen, press <F9> Close. The Closure window appears.
2. Edit the closing statement: We'd appreciate your ideas on how we can better serve you. Please write them below:.
3. Press <F10> Save. The survey is saved and the Questions screen appears. The survey questions, introduction, and closing statement for Survey 1 are now saved to your computer in the directory you specified at the beginning of this practice.
4. Press <F10> again to return to the Main Menu.
5. From the Main Menu, select Utilities.
6. Select Set Maximum Number of Questions. Type 4 and press <Enter>. Every time you create/edit a survey, use this option to define the number of questions.
7. From the Utilities menu, select Re-index Files. This option ensures the integrity of the files after a change to a survey has been made.

Creating a Survey Using Your Own Questions

1. Choose the File Set which stores Survey 2.
2. Select Set Maximum Number of Questions. Type 18 and press <Enter>. Before you begin to create a second survey (and any additional surveys), reset the maximum number of questions to the default, 18.
3. On the MCS Main Menu select Create/Edit Survey. A prompt appears asking if you wish to use questions from a previously created survey or the sample survey.
4. Type N to start with a blank survey screen.
5. Type in the questions shown on the screen. Change the importance column to No by using the arrow keys to highlight the field and pressing the space bar to select No.
6. Press <F10> Save to save the survey.
7. From Utilities, select Set Maximum Number of Questions. The number of questions window appears. Enter 3 and press <Enter>. Select Re-index Files.


CHANGING THE SURVEY SCALE

When you use the sample survey to build a survey, the system provides a survey scale. The scale ranges from (5) Outstanding to (1) Poor. Although the number of scale anchors (5) cannot be changed, there may be instances where the questions you create require different scale anchor names. For Survey 2, to edit the scale anchors, follow these steps.

1. From Survey, select Select the Survey Scale. The Scales Examples screen appears.
2. Press <F2> to edit the first scale anchor. The cursor appears on the line. (For Survey 2, use the Strongly Agree - Strongly Disagree scale).
3. Type the scale rating that corresponds to the number, then press <Enter>. The highlight bar moves to the next field.
4. Press <Enter> to move the highlight bar to the next number on the scale list.
5. Repeat steps 2 through 4 until the scale is updated. Press <F10>. The scale is saved and the Main Menu appears.


NOTE: The scale anchor names you choose appear once at the top of the survey. If you wish to use different scale anchor names for some questions, you can further customize your printed survey. The survey is stored in an ASCII text file (survey.txt) which can be edited in any word processing program. This file can be found in the directory you created for survey 2. The number of scale anchors (5), however, cannot be changed since all data analyses are based on a 5-point scale.


SURVEY INTERVALS

The software allows you to look at survey results on a weekly, monthly, quarterly, and yearly basis. Before you begin collecting survey results, you should choose the smallest period of time you want to examine. Choose an interval that allows you to collect around 30 people's responses. If your organization is very large and you are able to collect 30 surveys each week, you might choose weekly intervals. Monthly intervals are appropriate for most organizations. Quarterly intervals are most appropriate for very small organizations.

When you enter survey data, your computer automatically "dates" the data using today's date. If you enter data as surveys are received, you will never have to worry about overriding the computer dating. However, for most offices, this will not be feasible. Typically, surveys should be collected and stored separately by the time interval chosen above as most appropriate for your organization. Then, when someone is available to enter the survey data, he/she can "date" the surveys using an option available in MCS.

Dating Surveys

You can specify the time interval by changing the system date. This will help you group your response entries according to the interval you wish to set. For practice we're going to establish a monthly interval. Access the Main Menu, choose File Set containing Survey 2, then:

1. From Utilities, select Set New Computer System Date. The System Date window appears.
2. Enter the date: 01/01/93. The System Date window closes and the date appears at the lower right of the Main Menu screen.


ENTERING DATA

1. From the Data Entry menu select Enter Survey Data. The Data Entry screen appears.
2. Enter the data shown on the screen in the response column (Respondent 1).
3. Press <F10> to save answers.
4. Choose Y to continue.
5. Enter the following data for respondents 2 and 3.

Question
Respondent 2
Respondent 3
1. Our service was prompt.
1
2
2. Our personnel is friendly.
3
1
3. The quality of the information was excellent.
1
2

6. Use Set New Computer System Data to change the date to 02/01/93 and enter the following data:

Question
Resp 1
Resp 2
Resp 3
1. Our service was prompt.
3
3
9
2. Our personnel is friendly.
2
4
4
3. The quality of the information was excellent.
3
3
3

7. Repeat the process for 03/01/93 data.

Question
Resp 1
Resp 2
Resp 3
1. Our service was prompt.
4
5
5
2. Our personnel is friendly.
5
4
5
3. The quality of the information was excellent.
4
4
4

Entering Data for Survey 1

Access the File Set storing Survey 1. Enter the following data for the specified time period. Notice there is only 1 time period but 9 respondents. Also notice that you must enter the importance scores for this data as well.

Time Period: Current Date

Question
Resp 1
Resp 2
Resp 3
RespImp RespImpResp Imp
1. Were you greeted with a helpful attitude when you placed your service order?
5
2
4
1
5
1
2. Were you given clear information?
2
4
1
5
2
4
3. Did you receive assistance with your inquiries?
4
5
4
3
5
4
4. Was parking adequate?
1
2
2
2
1
2

Question
Resp 4
Resp 5
Resp 6
RespImp RespImpResp Imp
1. Were you greeted with a helpful attitude when you placed your service order?
4
2
5
1
4
2
2. Were you given clear information?
2
4
3
5
1
4
3. Did you receive assistance with your inquiries?
5
5
4
5
9
9
4. Was parking adequate?
2
2
1
2
1
2

Question
Resp 7
Resp 8
Resp 9
RespImp RespImpResp Imp
1. Were you greeted with a helpful attitude when you placed your service order?
4
2
5
1
4
1
2. Were you given clear information?
1
5
2
4
2
5
3. Did you receive assistance with your inquiries?
4
4
5
5
5
5
4. Was parking adequate?
9
9
1
2
2
2


SURVEY SCORING

Once you enter responses for an interval (or intervals), you need to score the data. Survey scoring consists of three statistical elements:

Mean ScoreThe average score across all respondents for each question and for the survey as a whole.
Standard DeviationAn expression of variability about the mean. A large standard deviation indicates that people do not agree about the question. Some people answered with a score of 1 and others with a score of 5. People may be receiving inconsistent service. A small standard deviation (1.00 or lower) indicates that people are agreeing about the statement. The service is probably consistent.
Control LimitsControl limits are calculated based on the first interval's mean and standard deviation. They are located at three standard deviations above and below the mean. Any subsequent data points above or below these limits indicate a statistically significant improvement (above) or decline (below) in service.

MCS provides graphs and reports that help you interpret scores and understand what the data represents.


NOTE: See Appendix B for definitions of statistical terms.

To Score a Survey

1. Access the directory in which the survey is stored. (For this lesson, access Survey 2 File Set.)
2. From Data Entry, select Consolidate Scores. The Intervals menu appears.
3. Specify the interval in which you would like to see the data.


For practice, select Monthly intervals. The starting date window appears.
4. Type 01/01/93, then press <Enter>. The scores are calculated for 12 intervals beginning with the date you specified.
5. Press <Esc>. The Main Menu appears.

VIEWING SURVEY RESULTS

Once the scores are calculated, MCS provides results viewing options. You must select a view option in order to review survey information.

You can use MCS to create three results view options:

Viewing Graphs

MCS provides a variety of graph options for your survey data. You can view data for:

Graphs (Graphics Interface) - Satisfaction Measures Only

1. From the Main Menu, select Results. The Results menu appears.
2. Select Graphs (Graphics Interface). The list of questions appears.
3. Select the question for which you wish to view a graph, or select Entire Survey to see a composite of all survey questions, then press <Enter>. For practice, select Entire Survey. The graph should look similar to the this screen.

Character-Based Graphs - Satisfaction Measures Only

1. From the Main Menu, select Results. The Results menu appears.
2. Select Character-Based Graphs. The Question Selection list appears.
3. Select the question for which you wish to view a graph, or select Entire Survey to see a composite graph for all survey questions. For practice, select Question 1.
4. Press <Esc>. The Main Menu appears.

Graphs (Graphics Interface) - Satisfaction and Importance Measures

1. Access Survey 1 File Set.
2. From Data Entry, select Consolidate Scores. The Intervals menu appears.
3. Select Monthly intervals. The starting date window appears.
4. Type the current date, then press <Enter>.
5. Press <Esc>. The Main Menu appears.
6. From the Main Menu, select Results. The Results menu appears.
7. Select Graphs (Graphics Interface). The list of questions appears.
8. Select the question for which you wish to view a graph, or select Entire Survey to see a composite of all survey questions, then press <Enter>. For practice, select Entire Survey.
9. Since you included importance scores for this survey, a prompt appears. Indicate if you wish to include importance. Select an option (for practice, select Yes), then press <Enter>.


REPORTS

You can use the reports option to produce a printout of your survey results. When you select this option the report provides scores for each question in each interval.


1. Our service was prompt.

2. Our personnel were friendly.

3. The quality of the information was excellent.

4. Entire Survey


1. Access Survey 2 File Set.
2. From Main Menu, select Results.
3. From Results, select Reports. The Printer Prompt appears.
4. Press Y to print the report. The report is sent to the printer.


COMBINING ORGANIZATION DATA

You can combine organization data between different data sets. This option allows you to review the "big picture" for the entire organization. For example, suppose you have 2 similar, but separate, organizations. You would like to use the same survey for both organizations but look at the data separately. You would set up 2 different directories, each using the same survey, and store the results in the respective directories. Later you may want to look at the overall results for both organizations. Use the following procedures to combine results from 2 or more organizations. Remember, the same survey must be used in order to combine results!

Export Data

1. Access the file set for one organization.
2. From Utilities, select Export Data to Floppy. The Drive Selection window appears.
3. Select the floppy drive, then press <Enter>. The Insert Diskette prompt appears.
4. Insert a formatted floppy diskette, then press <Enter>. The file set is copied to the diskette.
5. Press any key. The Main Menu appears.

Import Data

1. On the Main Menu, create a New File Set to store your combined data. Make sure to insert the survey from organization 1 into the new file set.
2. From Utilities, select Import Data From Floppy. The Drive Selection list appears.
3. Select the floppy drive on which the exported data is stored.
4. Insert the floppy with the data. Follow the prompts until the responses are imported into the directory.
5. Access the second organization's file set.
6. From Utilities, select Export Data to Floppy. Follow the prompts until the data is exported.
7. Access the file set to which the data is to be imported.
8. From Utilities, select Import Data from Floppy. Follow the prompts until the data is imported.

Combine Scores

1. From Data Entry, select Consolidate Scores.
2. From Results, create graphs and reports.


MCS UTILITIES

Set New Computer System Date

Use this option to set a new system date.

Edit the Organizational/Survey Name

Select to change the organization and or survey name.

Clear Answer File

Deletes all responses from the currently selected survey directory/file set. MAKE BACK-UPS before using this option. Use the export option to copy data onto a floppy diskette.

View/Print Sample List

Select to review a list of questions provided as guidance for survey development.

Export Data to Floppy

Copies survey and response data from the currently selected directory to a floppy diskette.

Import Data from Floppy

Copies survey and response data from a floppy diskette into the currently selected directory.

Printer Selection

Use this option to specify the printer on which survey reports and graphs are to be printed.

Printer Port Selection

Use this option to specify the connection port on your computer to which the selected printer (above) is connected.

Set Confidence Limits

If you are knowledgeable about statistics, you can control the confidence limits. The default limits are three standard deviations above and below the mean. You may choose one or two standard deviations for the limits. This will "tighten" the spread and result in more data points lying outside the limits.

Set Maximum Number of Questions

Select to specify the maximum number of question items you are including on your survey. The system defaults to 18. If you want more or less, you must use this option to specify the maximum number of questions for the survey. Use this option AFTER

you have created/edited the survey.

Re-Index Files

Use this option after you have changed the maximum number of questions. Also, should program execution be interrupted, select this option to reorganize the files, then again execute the program.


APPENDIX A: SURVEY DEVELOPMENT


STEP ONE

Start with the sample questions. Select from these and rewrite them if appropriate. You will probably want to write some of your own questions. As a first step, write questions without agonizing over every detail.


STEP TWO

Ask yourself the following questions before a rewrite.