Program Evaluation Part 4- Clientele Change

As discussed in previous blogs, Extension educators develop programs that are designed to result in some type of clientele change. Extension educators should ask themselves some fundamental questions related to determining the appropriate clientele change level as follows:

• What level of clientele change should be the goal of a program?
• How do programs achieve the desired level of clientele change?
• How do we know if the program was successful in achieving the stated objective?

In addition to the questions above, Extension educators should consider the following factors:

• What will be the most beneficial level to ultimately achieve?
• Is the need for clientele to have new knowledge to make more informed decisions?
• Is the attitude of the clientele the major attribute that needs to be changed by the program?
• Is there a need for the program efforts to result in clientele change in behavior or adoption of best practices or technology?

The answers to these questions are an integral part of the program development process and should be answered when issues are being analyzed. An example might be that a program area committee or task force identifies herbicide-resistant weeds in crop production as a major issue. When analyzing this issue, the Extension educator and the members of the program area committee or task force must determine the following

• Is the goal of the program is to raise awareness of the problem?
• Is the goal to Increase knowledge of producers?
• Is the goal to determine if producers intend to adopt best practices?
• Have producers adopted a best practice associated with weed management?

The manner that a program is designed and subsequently evaluated would be substantially different depending on the answer to these questions.

It is evident that knowing the appropriate level of client change to expect is paramount to an effective program design and will lead to the evaluation strategy best suited to measure the impact that the program has achieved. In the Extension publication “Keys to Education That Works”, Boleman, Cummings, and Pope (2005) outline these primary levels of intended clientele change for Extension programs to target:

Learning

Knowledge – learned information or accepted advice, including varying levels of comprehension.

Skills – individuals’ mental and physical abilities to use new and/or alternative processes.

Attitudes/Beliefs – individuals’ beliefs, opinions, feelings, and perspectives.

Application

 Behavior Change – changes or benefits for participants (Behavior change typically happens after learning change, i.e., after participants have developed a skill, they may change behaviors if they deem it necessary to do so.).

Best Practice – participant decides to adopt a new practice in place of a current one.

New Technology – participant adopts an innovation.

Deciding which of these outcome levels to select is one of the most challenging choices Extension educators face. Before choosing, it is important to analyze the six levels of change. Ask yourself, where do you want audience members to be by the end of their educational program? Key to answering this question, the Extension educator must know the target audience’s level of knowledge, behavior or skill when the program starts.  Not knowing the target audience’s level in these areas may necessitate having a needs assessment before planning the program to establish benchmarks that can later be compared. Since the first step toward change is knowledge, for audiences with no background in an issue, Extension educators should start by providing knowledge, and then move to skill development or behavior change. However, if an audience already has a knowledge base in the issue being addressed, it may make more sense to target audience change to a level higher than knowledge, selecting instead behavior change or best practice adoption (Boleman, et. al., 2005).

At the time that the intended client change is determined, the appropriate strategy to evaluate the program should become evident. If the intended client change is knowledge-based, then an evaluation instrument should be developed to measure an increase in knowledge, or possibly a change in attitude. However, if the intended change is in the behavior of clientele, a knowledge-based evaluation instrument would fall far short of capturing the true impact of the program. Understanding the differences between the levels of intended change, and the appropriate educational methods to initiate the intended level is an important skill for the Extension Educator to understand and acquire. Working closely with Regional Program Leaders and the Organizational Development Unit to make certain the planned program could achieve the stated goals and objectives is an excellent way for newer Agents to gain an understanding of the process.

In future Next Step to Success blogs, we will discuss how to interpret data collected from evaluations.

References

 Boleman, C., Cummings, S. & Pope. P. (2005). Keys to education that works: Texas Cooperative Extension’s program development model, Texas Cooperative Extension, College Station, Texas. Publication #345.