Best Practices vs. the “Wild West Nature of Basic Science"

By Marianne Evola

What are Best Practices? I predominantly think of Responsible Research Conduct as utilizing best research practices to enhance the integrity of research. As such, I believe that developing good research skills is the best defense against making poor research decisions that could result in sloppy research or worse, research misconduct. However, I recently read an excellent article in the Scientist entitled, "The Great Big Clean-Up," that made me ponder how best practices are defined and who defines them. As I got toward the end of the article, the authors asserted that lack of formalized best practices or the "wild west nature of basic-science" (Grens, 2015) may have contributed to a large scale contamination problem (described below). When I read that, my mind began to buzz with a conceivable conflict between defined best practices as opposed to the creativity and independent thinking associated with insightful research. To what extent do we want to regulate the "wild west nature of basic-science"? Furthermore, I thought a bit more about happy accidents vs. tragic accidents that resulted from poor research practices. Finally, are defined best practices at odds with the creativity that can lead to scientific breakthroughs that stem from thinking outside the box and how do you train students to balance these potentially opposing motivations so that they can have a productive research career?

The article mentioned above, which I highly recommend, addressed large scale problems with contaminated cell lines in the biomedical sciences, specifically the RGC-5 cell line used in vision research (Grens, 2015). The RGC-5 cells were thought to be from the rat retina but after the publication of almost 200 articles, all assuming that they were working on a rat retinal cell line, it was discovered that the cell line was actually from the mouse and it was a cone photoreceptor cell line. It was an embarrassing scandal in the vision research community and led to the top vision research journals deciding that they would not publish any further submissions on the RGC-5 cell line. In addition, a couple of journals made it policy to require all future authors to independently validate the authenticity of their cell lines beyond any documentation that was provided by suppliers.

Upon reading this, I thought a great deal about the trust that we placed in vendors and collaborators when they provide us with research supplies. In my research, we often purchased genetically modified mice and conducted research on these animals, fully trusting that their genome had been confirmed by the vendor or collaborator. We had to trust our supplier because as behavioral researchers, we did not have the capability to test the genome nor did we have a budget for contracting the work to others. We similarly trusted that when we solicited supplies or drugs from vendors or colleagues that we were being provided the compounds that we had requested. It was not naiveté that caused us to trust that vendors provided us with the requested compounds/animals, rather it was a simple matter of practicality. We had a budget that we had to meet and arguably, it would be a waste of resources to verify the authenticity of every animal/compound that we purchased. In contrast, however, perhaps it may be prudent to occasionally adhere to the words of former President Ronald Reagan, "Trust but verify". I've often presented on the virtue of trust that is embraced by much of the research community and how engaging in misconduct is a violation of that trust that can never be undone. I began to wonder if I should have incorporated the second virtue of verification, to ensure that our trust is well founded. As I pondered, I found myself waffling (political pun intended).

As I thought about this, I realized that verification is part of the skepticism of a well-trained researcher. Yes, we are trained to trust, but we are also trained to be skeptical of everything that we read. And more importantly, we are trained to be skeptical of our own data and interpretation. We are aware of the inherent bias that comes from being human, and we do our best to work against our own biases as we strive to contribute objective truth to our disciplines. The waste that was associated with the RGC-5 scandal is regrettable and yes, it was embarrassing to the discipline. But ultimately, the virtue of skepticism in the form of verification corrected the literature and revealed a major flaw in the community. So, was this scandal an example of poor research practice, practical use of resources, or the powerful self-correcting nature of research?

Although we all strive to develop and engage in solid and consistent research practices, we have also been amused and often inspired by stories of accidental discovery in science. We are impressed by colleagues that think beyond accepted theories and/or the practices of one's chosen field of research. I would not go so far to say that innovators are rule breakers, rather I see them as rule changers. So, are rule changers violators of best practices or the best examples of innovation associated with the "wild-west of basic research" (Grens, 2015)?

Beyond science, I think that the world at large loves legends associated with accidental discovery, whether they are true, partially true or even fabricated. We love the legends of the "wild west", we love stories of innovation and discovery. In contrast, it is harder to love stories of sweat and tears research and development. As a child of Detroit, I am familiar with the legend associated with Vernors ginger ale. I refer to it as a legend because it has been demonstrated to be untrue. Even so, true to my Detroit roots, I like the legend more than the truth. Many people in Texas, or the rest of the world, have never heard of Vernors, but as you travel northeast, toward the Great Lakes, you will find Vernors stacked on grocery store shelves next to Coca-Cola and Pepsi. So powerful is the legend of Vernors that most people from Michigan will assert that not only is Vernors a tasty beverage, it also has curative effects and alleviates the symptoms of colds, flu and upset stomach. No one in Michigan survived childhood illness without treatment with Vernors and to this day, even with working knowledge of placebo effects, I keep Vernors stored in my pantry as a critical component of my home remedies kit for when I am not feeling well. In fact, when I moved to Texas, I barely survived my first bout with flu due to my inability to find Vernors ginger ale at the supermarket.

The legend of the accidental discovery of Vernors begins with the Civil War and a pharmacist named James Vernor, who was trying to replicate the ginger ale sold in Ireland, his ancestral home. While experimenting with different combinations of ingredients the Civil War broke out and James Vernor was called to service. After spending several years serving in the war, he returned home to Detroit to discover that he had left one of his concoctions sitting in an oak barrel for years. As the concoction aged it had combined with flavors from the oak barrel and had created something completely unique and completely by accident. Years later, his son outed him and revealed that the legend was also completely invented by his father, who truthfully had developed the Vernors product after the war. But as a native Detroiter, I still prefer the legend and I still incorporate Vernors into my treatment and recovery from all ailments.
In contrast, the discovery of penicillin is in fact one of the most notorious cases of serendipitous discovery and unlike my Vernors tale, it is actually true and in fact, did contribute to my recovery from numerous childhood illnesses. Dr. Alexander Fleming kept a messy lab bench and commonly kept his lab windows open. Upon returning from to the lab from vacation, he discovered that mold had contaminated his petri dishes and interestingly, the mold had interfered with the growth of staphylococci. Often, this is the only part of the story that is attributed to the development of penicillin. However, this accidental observation was only the triggering event that led to the development of penicillin. It was the rigorous scientific follow-up work by Drs. Howard Florey, Ernest Chain & Norman Heatley that took the initial accidental observation and fully developed penicillin into a potent and widely available drug that forever changed the biomedical world. Arguably, Dr. Fleming was not engaging in best practices when his samples were initially contaminated by mold. Regardless, his observation skills, examination of the samples, and accurate reporting of his observations triggered the development of penicillin. Furthermore, the rigorous work of Drs. Florey, Chain & Heatley to build on those initial observations were critical for making the discovery accessible for widespread medical application. Regardless, Inall of these scientists contributed to one of the greatest discoveries and contributions to modern medicine, even though their lab practices markedly differed from one another. And certainly all of them differed from standard practices in modern day laboratories. It's true, some accidents have triggered discovery, but not without the discipline and rigorous research that comes from defined research practices and solid methodology.

So obviously, I'm not recommending that anyone overtly defy the rules of their respective lab or discipline or engage in negligent research. Neglect leading to contamination is not generally a productive research practice as demonstrated by the RGC-5 scandal. Engaging in overtly poor research practice hoping for accidental discovery could be catastrophic or at the very least, get you into serious trouble with your research advisor or supervisors. Furthermore, I think that there are rules that we all accept to be an inherent part of the scientific process. Scientists should never engage in practices that could put themselves or their research team in jeopardy. Nor, should they ever fabricate or falsify data because these practices do not contribute to the evolving truth of science but rather only contribute fiction and damage the "trust" that is imperative to the research community. Furthermore, scientists should never plagiarize because that is stealing from your colleagues. Research practices that fall outside of the standard accepted practices in one's discipline should be highly scrutinized for inappropriate interpretation or even unethical decisions.

However, in contrast, it may be good research practice to question the standard practices or methodology of your research team, as long as you do not violate restrictions that overtly waste resources or defy your advisor so that you get yourself into trouble. It is important to define best practices but it is prudent to question them and to contribute to the evolution of those practices. For example, all research groups have standard practices that are often defined by the senior member of the team. When I was a graduate student, our lab placed strong emphasis on developing strong and reliable behavioral control before we proceeded with any experimental manipulations. We utilized animal behavior as a very effective tool. However, living things are very complex and as such, behavior can be altered by many factors such as stress, noise, odor, etc. As such, we would spend weeks and even months developing a highly consistent behavior and a highly stable environment so that we minimized extraneous factors that could impact the behavior of our research animals. And largely, at that time, I thought that this was standard practice in our discipline. However, during my master's thesis defense, a member of my committee questioned why I would spend so much time establishing such a strong baseline of behavioral control. She asserted that rudimentary control was sufficient to demonstrate a statistically significant change in observable behavior and I had wasted considerable time.

At that time, as a junior researcher, I was stunned by the question, because I'd been trained on the importance establishing highly reliable control on behavior to maximize our trust that behavioral changes could be attributed to our experimental manipulation/s. Much later I realized that there were a large variety of standards or best practices that were defined by different labs and even projects within our own lab. I remember a fellow student that would get highly frustrated with normal speaking voices in our lab because she was utilizing a different behavioral tool and hers was much more sensitive to any extraneous noise. Her research required a different standard of control. We all have rationale for the standards that we utilize. However, it is appropriate to question those standards so that we continue to learn and our research continues to improve. Furthermore, variable methodology between research groups within a discipline provides strong evidence for robust research results. In other words, if an experimental question produces consistent results while tested with different research standards and practices, the results are likely robust or "true".

So, after much pondering about the opposing forces of defined best practices and the "wild west of innovative basic science" (Grens, 2015), I've determined that like many controversies in life, one is not superior to the other. Rather, we should strive for balance. You cannot be innovative if you are not willing to change the rules. Yet, without rules, research has no structure and will degenerate to chaos or worse, unethical behavior or seriously dangerous working conditions. As such, here are my thoughts on how to strive for balance.

1. Senior members of the team should define the "Best Practices" or methodology. To best ensure that everyone is well versed in defined best practices, a lab manual should be constructed that defines methodology. Many labs have safety manuals, yet they do not have manuals describing best practices. Clearly define step-by-step instructions and incorporate your rationale. However, realize that this is an evolving document because your team will discover research questions that are not consistent with your defined methodology. Make sure that every member of your team has access to the written methodology and is knowledgeable on the contents and rationale. Encourage your team to question the best practices when they encounter inconsistencies between their ideas and the written document so that they include you in decisions to alter methods that they deem inappropriate for addressing their research questions. Encourage them to include senior members of the team when they need to alter methodology but discourage outright defiance of lab standards.
   
   
2. Encourage everyone on your team to develop a strong back bone so that they are willing to challenge lab standards that impair their ability to address new and creative research questions. Make sure that senior members of the team are willing to consider alternate methods and standards to better address research questions. Define who is permitted to make changes to best practices strategies in response to innovative research questions. Encourage all members of the team to contribute their insight to innovative ideas. Generally the team, especially senior members of the team, can contribute multiple perspective that have not been considered by junior researchers with creative minds. Teach all members of the team to objectively critique the methods and rationale of colleagues with an open mind so that they are willing and able to learn from the defined best practices of colleagues and competitors and improve your own practices. Make sure any deviations from methodology are documented and included in publication.
   
   
3. Teach your team to develop a "thick skin" so that they are willing to accept the critiques of others and acknowledge their own weaknesses so that they can learn and improve the practices of the research team. This lesson often involves developing the ability to separate professional criticism from personal attacks. I've been told that I am pretty good at this and it is tougher than I realize. Furthermore, many of my colleagues do not find these types of challenges to be as entertaining as I do. Many researchers are so personally entrenched in their research that they have difficulty separating professional critiques from personal attacks. I don't know when or how I began to find assertive academic debate to be stimulating and amusing but I love to be intellectually challenged or even corrected during a presentation. I'm not sure whether detached or personal entrenchment is a more productive approach to research. Perhaps, once again, balance is the best approach. However, from my perspective, a personal/professional separation enables you to objectify and learn from critique rather than resist intellectual input into your ideas. As such, I'd be happy to alter my position with appropriate input.

Marianne Evola is senior administrator in the Responsible Research area of the Office of Research & Innovation. She is a monthly contributor to Scholarly Messenger.

---

For more articles about Responsible Conduct of Research (RCR), click here >>