You may decide that a scannable form is not appropriate for your intended audience or program. Or perhaps you were unable to find a scan form that meets your needs. Whatever the reason, conducting evaluations not using scan forms is valid and appropriate in a lot of situations. In fact, for much of Extension’s existence, it was the only way to conduct evaluations.
Be aware that by not using scan forms, all parts of the evaluation are done by you. However, you have total freedom to design and implement the evaluation to best meet your needs, and Organization Development is here to advise and assist at every step of the process. Here are some of the suggested basic steps you need to perform, along with some suggested resources:
Determine a Data Collection Strategy
From who are you collecting data?
- All participants (census)
- Representative sample
Decide whether you need to collect both pre and post program data from participants, or just post program only data. Three common approaches includes:
- Traditional pre-post (pre data collected before the program; post data collected after the program)
- Retrospective post (both pre and post data collected after the program)
- Post-only (post data collected after the program)
Select an Appropriate Data Collection Tool which may include:
Survey Methods
- In-person surveys using non-scan forms or audience response systems (clickers)
- Mail surveys using non-scan forms
- Phone surveys
- Web surveys
Non-Survey Methods
- Observation
- Individual measurement
- Records review
- In-depth interviews
- Focus groups
- Case study
Suggested Resources:
- Qualtrics for web surveys
Develop Your Questionnaire (if applicable)
Developing a quality survey can be a formidable task. Make it as short, simple, and straightforward as possible while still collecting the information you need. Have multiple individuals review the survey for length, clarity, and ease-of-use, and pilot test it.
- Questionnaire Design (Extension Publication E-227)
- Example questionnaires used in the past
Enter Your Data (if applicable)
Unless your data are captured automatically via a data entry app, web survey, or audience response system, you’ll need to enter the data you collected via non-scan paper forms. Develop a coding scheme (i.e., no=0; yes =1) for each question and enter the data in tabular fashion. Excel is ideal for this task. The first row should be your column (question) labels. Then enter data – one row per survey.
Tabulate and Analyze your Data Using an Analysis Tool
Once entered, you’ll need to tabulate and analyze your data. Tasks associated with this include data manipulation, filtering, descriptive statistics, and/or inferential statistics. Depending on the complexity of your data and analysis needs, an add-in tool within Excel may be sufficient or a more comprehensive data analysis package like SAS or SPSS may be required for more advanced procedures.
Suggested Resources:
- EZAnalyze add-in to Excel
- SAS or SPSS available via the Software Evaluation & Loan Library at TAMU
Interpret Your Results
This task involves identifying and collecting pieces of information from the raw output produced by an analysis tool.
Suggested Resources:
Telling Your Story
Telling your story involves taking pieces of information collected from raw output into a concise, straightforward, and well-organized written or oral summary report for stakeholders that addresses “relevance, response, and results.”
Suggested Resources: