Protecting Against Protocol Shift: Endless and Exhausting Vigilance

By Marianne Evola

In the daily life of a laboratory manager we encounter regular sighs, eye rolls, embarrassed scampering, and conversations going silent when we walk into the labs. We get called control freaks behind our backs, and sometimes to our face, often in jest but with a hint of reproach. All of this comes from the critical need to maintain the consistency and integrity of our research protocols. The role of lab control freak is one that every lab manager needs to own, regardless of their status. From principle investigator to senior undergrad, once you are put in charge of managing lab personnel, it becomes your responsibility to guard the research protocols from slight but progressive variation because if left unchecked, the protocol will evolve into something unrecognizable with too many variables left uncontrolled. Protocol shift happens with every minor short-cut and/or modification, and ultimately, if not corrected, progressive variation is introduced into experiments so that results are no longer comparable with previous studies. Furthermore, protocol shift can markedly impact the reproducibility of research data.

The above description of protocol shift sounds nefarious, doesn't it? And, the above description of lab manager sounds like they are serving as prison guards. Perhaps the description is overstated. But, when you are correcting rudimentary controls, repeatedly, day after day, you begin to feel like a research sentry, ever watchful. And there are times when it feels like your entire team is working against the integrity of the research protocols.

The truth is, most protocol shift is not nefarious. Rather, the inattention to detail that progresses to protocol shift is often a result of inexperience and/or ignorance. Lab managers are required to engage in unrelenting and repetitive monitoring, correcting and education of the research team. The endless vigilance requires great patience and persistence. In fact, I often thought that it would be much easier to serve as an unforgiving lab guard than it was to sincerely engage in the constant and repetitive education of my team. It could be exhausting to muster the necessary patience to yet again educate new team members, or to remind experienced team members, why every minor protocol detail was an essential part of the experiment. The truth is, I never encountered a single student researcher that was dismissive of protocol. If I had, I would not have continued to work with them. Rather, new members of the team often did not realize why seemingly minor protocol details were an essential part of the project. In addition, experienced team members become so habituated to the work that they can become dismissive of seemingly minor protocol details so that rather than recognizing their importance, they come to view them as unnecessary "busy work" that impairs their productivity. Thus, the entire team needs to be regularly reminded to adhere to the minor details of each experimental protocol.

A research protocol can easily become distorted if a manager does not sufficiently communicate with each team member. As such, it is critical that managers be present in the lab to directly monitor and educate their team members. However, at the same time, it is critical for lab managers to provide their team with opportunities to learn how to be managers by allowing them to supervise subordinates. In other word, it is not enough to be a good supervisor, you need to learn how to supervise the supervisor. Or as I like to put it, welcome to the confusing world of personnel management. Most larger labs work on a hierarchy. The principle investigator trains/supervises the postdoc, who trains/supervises the grad students, whom train/supervise the undergrads. The problem is, how do you, as a principle investigator or postdoc, ensure that your immediate subordinates appropriately train/supervise their team. Experienced researchers are aware that research problems can arise when procedural steps are inadvertently omitted as information moves down the information hierarchy. Omissions are not purposeful, rather, when personnel conduct experiments daily, they often follow steps without consciously attending to each detail. Rather when we do our jobs we chunk multiple smaller steps into larger steps. That does not mean that your experienced team members are omitting those steps themselves, but rather the small steps are so automated that they do not think about the small steps and can easily omit them when they train new team members.

I like to think about the challenge of training a research protocol down a status hierarchy like playing the telephone game that we all played at some point in childhood. A group of children sit in a circle and the first child whispers something to the person sitting next to them. Generally, the whispered phrase was a familiar line of cultural jargon or the line from a nursery rhyme. Then each child whispers to the next till the message reaches the last child and they announce the message that they received. The familiarity of the first whispered message never mattered, because by the time the message reached the ears of the last person in the circle, it was always unrecognizable from the first whispered message. If the goal was for each individual to whisper the same message, how did the message get so jumbled? Well, all adults know how and why the message got jumbled. Whispering from ear to ear in a room full of active and distracting children is not the most efficient means of communicating a message. Laughing and noise naturally makes it difficult to hear a whispered message. Then, there were probably a few mischievous playmates that purposely distorted the message to make the game more fun. The point of this analogy is that regardless of whether the distortion coming from innocent fun or mischief, it was the form of whispered communication down a line, that allowed the message to become distorted. If the first whisperer had whispered the original message to each child in the line, the message would have likely reached each individual and the message would never have been distorted but the game would not have been much fun. Research may be fun, but it is not a game and we don't want our message to be distorted. So, supervisors need to check down the hierarchy to make sure that each team member has learned and is adhering to the defined protocol. This can be done by monitoring and interacting with the entire team, or quizzing individuals on protocol during lab meetings. Regardless of the strategy, it is essential that senior members of the team assess whether protocols are shifting.

I remember one of the first challenges that I ran into as a new graduate student while conducting my first independent experiment. I had to transport my animals down the hall from the vivarium to the laboratory. As such, I needed to stack the animal cages onto a cart. While shadowing senior team members, I had been shown many times how to stack the wire cages onto stainless-steel pans that were lined with cardboard and then drape the animals with a large piece of cloth to transport them down the hallway. However, on my first morning working on my own, I could not find the stainless-steel pans, the cardboard liners nor the cloth drapes and there was no one around to ask. You see, as a new member of the team, I could not get access to the research equipment in the middle of the day during the favorite time slots. The favored time slots were reserved by students with seniority. In contrast, my first experiments had to begin at 7am so that I could finish my work before the senior team members arrived in the lab around 9am. This also meant that no one else was in the lab when I began my early morning experiments. As a result, on my first day of independence, I showed up in the lab and could not find any of the equipment that was necessary for transporting animals. The equipment was not in the colony room, which is where it had always been when I had shadowed a senior student. No one had told me that the pans, liners and drapes were switched out every day. The last experimenter of the day discarded the equipment to be washed. The first experimenter of the day grabbed fresh equipment from storage. As such, on my very first day of independence, I had no idea where to get the equipment that I needed to begin my experiments. No one had intentionally misdirected me on protocol, but no one had thought to show me where the equipment could be found, because in the middle of the day, while I was shadowing senior students, the equipment was in the middle of the colony room.

What types of minor protocol shifts and omissions do lab managers need to regularly review? Lab managers need to assess their protocols on their own to identify small details that could be overlooked or bypassed because research is so variable. As such, I cannot speak for every research protocol, I only have experience with my own, but hopefully sharing some of my frustrations will stimulate others to think about their own research. Our lab worked with animals, so many of our research controls were designed to minimize the stress on animals, minimize human error and equipment failures. One of the protocol steps that I had to constantly address was the need to drape animals when transporting them from room to room. Students would conveniently "forget" when cloth drapes were not readily available. Searching for a cloth drape was interpreted by students to be an unnecessary delay when they were simply moving the animals a short distance down the hallway. After all, from the student's perspectives, the animals took this short trip every day, thus there was nothing novel about it. However, students did not realize that animals were not merely draped for the journey. Rather, animals were draped because hallways are not controlled environments and as such, experimenters cannot control what animals will encounter during transport. In hallways, experimenters and animals could pass researchers, other animals, students passing through, custodians or workmen, etc. Animals are draped because it is an uncontrolled environment. Arguably, drapes do not block unusual odors or sounds but draping animals for transport served to minimize the impact of uncontrolled stimuli which can be stressful and which can impact animal behavior and physiology.

Another protocol issue that I regularly had to address was one that was designed to ensure that individual research animals did not get mixed up. Our research often monitored the behavior of a single animal, within a group, for changes over time. We specifically worked with white rats, which to a human, often look markedly alike. Ok, they are largely identical. Therefore, procedures that ensure the individual identity of animals are critical to maintaining the integrity of the research and data. There are a few options, but our lab utilized the least invasive option. We would mark a rat's tail with colored markers. The procedure was reliable, minimally invasive, not harmful and provided a clearly visible identifier for each animal. However, gradually, the colored mark would fade over several weeks and the tails would have to be re-colored. Students needed to be constantly reminded to re-color the tails. After several weeks of working with a small group of rats, students would insist that they "knew" which rat was which, based on the size and behavior of each individual animal. And arguably, students were telling the truth. Students knew their animals very well. When you work with a small group of animals every day, you know which are sweet and cuddly, which like to be aggressive and nip, which squeak every time you pick them up. In other words, students became very attached to and familiar with their animals. As such, it was easy for them to rationalize their decision to bypass the tail-marking step in the protocol. However, after years working in the lab, I had addressed more than a couple incidents where confusion over data associated with individual rats made a difference between significant or insignificant experimental results. Re-coloring the tails was a small but critical protocol step and students needed to regularly be reminded to take the time to appropriately ID their animals.

Researchers define a very precise research procedure and each step has a logical and specific reason for being included in the protocol. Some steps are obviously critical research steps that are essential to the experiments. Adhering to these big obvious steps is easy for the research team. However, many smaller steps, are often not perceived of as critical. It is an essential role of lab managers, at all levels of the hierarchy, to persistently educate and remind students, subordinates and sometimes themselves that even minor protocol steps are important facets of the research. Consistency is imperative. If the team is not persistently corrected on oversights, the protocol could markedly shift so that the data is no longer comparable with historical data from the research group, and/or the modified protocol may no longer be recognizable to the principle investigator. Smaller procedural steps are often critical for collecting reliable and reproducible data. Admittedly, there is fatigue associated with repeated review and discussion about protocol, but lab managers need to persevere. Protocol consistency is critical for producing reliable and reproducible research.

 

Marianne Evola is the director of the Human Research Protection Program in the Office of Research & Innovation. She is a monthly contributor to Scholarly Messenger.

For more articles about Responsible Conduct of Research (RCR), click here >>