Program Evaluations: Research or Non-research

By Donna Peters

The Human Research Protection Program website provides an interactive module for researchers to determine if their research project needs IRB review and approval.

The first question in the module asks:
Program evaluation – Is the work you are planning:

• Intended to learn how well a particular program is working
• AND is not intended to add to generalizable knowledge?
  

There are three common approaches and/or terminology used when evaluating programs:

• Program Evaluations
• Quality Improvement Programs (QI)
• Quality Assessment Programs (QA)
  

Most program evaluations, QI or QA studies interact (people contact) or involve an intervention (i.e. survey) with human subjects. This meets the partial definition of human subjects research (45 CFR 46.102) because what you plan to do involves people. To determine if your research intent needs IRB approval comes down to the issue and definition of generalizable knowledge, the second part of the definition. If the intent for the data you plan to collect on a program is to be shared with a broader audience than the internal program staff, then what you are planning to do will be considered generalizable knowledge because what you learn goes beyond the scope of the program. You intend to share the information you learn with others. An IRB proposal will be needed before you conduct your program evaluation. On the other hand, if the program evaluation information that is collected will stay within the confines of the program and program staff, then no IRB review is necessary.

Example 1: The owner of a family-owned jewelry store is asking customers to complete a survey. The survey questions ask what the customer likes about the store and dislikes about the store. There may be questions on the frequency of new displays or the prices of items. There is a question that asks for customer feedback they feel is important for the owner to know. The survey asks for no names. Completed surveys can either be dropped in a box on the counter or mailed back to the store in the stamped envelope provided. The owner of the jewelry store plans to review and analyze the survey answers in order to learn more about the thoughts and concerns of customers. The owner hopes to take the survey answers to improve his customer satisfaction ratings and business sales.

Does this project need IRB review? No, there is no intent for generalizable knowledge even though there is interaction with human subjects. The jewelry store owner will be the only one who knows the results of the survey analysis.

Example 2: An established program involves multiple sessions/courses. The program staff wishes to evaluate the program effectiveness. The staff members have identified some variables and attributes they wish to study. They also want to contact the attendees in a few months to ask them if they are using the information they learned from the program. They plan to review their program using the information from the program attendees. Their intent is also to share the information they learn with other programs across the country, especially if it can be helpful. An instrument to measure the variables is designed and validated. Individuals who attend each session/course will be asked to complete the measurement.

Does this project need IRB review? Yes, even though the results of the program evaluation aim to identify the program effectiveness, the program staff intend to share what they learn with other programs. An IRB review and approval is needed.

Donna Peters is manager of Texas Tech's Human Research Protection Program