Stay out of court and save millions with WIP limits!

Last time, I wrote about the concept of technical debt for scientific work. The most important example of this is poor lab notebook practices—It’s the Mother of All Technical Debt. Cutting corners on notebook keeping is easy, saves time, and doesn’t cause immediate problems. What’s not to like? For starters, how about any of the following:

• Your work won’t be repeatable by others. That could be a company employee you’ll never meet, but it could also be your future self.
• There's a risk of having the company’s patent invalided in the event of a court challenge. If that happens, millions in profits go out the window — along with your job!
• Being unable to write an adequate patent or journal publication because key experimental details are unclear or missing.
• Opportunities for learning and synergy with colleagues past, present, and future simply vanish because the SECI knowledge spiral is interrupted.

Here’s a pithy restatement of that from Adam Savage of Mythbusters fame: “Remember kids, the only difference between science and screwing around is writing it down.”

Clearly there’s much more downside risk than upside gain. But if that’s true, why do so many otherwise stellar researchers perpetually struggle with this issue?

A Slippery Problem

There’s two main reasons for this perpetual bugbear. First, keeping a good lab notebook (LNB) really is a nuisance. Most of the notebook’s utility is for other people in the murky future. There’s little short term benefit to the researcher despite all those huge downside risks in the long term. Under these circumstances, it’s human nature to cut a few corners. We tend to discount the impact of future events, no matter how negative. Today’s work seems much more important because it’s in front of us right now. The inhabitants of a distant future won’t help you get things done today.

Secondly, there’s always pressure to be as productive as possible. Filling your fume hood with as many reactions as it can hold seems like a good way to do this—until it comes time to finish those experiments and write them up. Take a look at the figure below. The balls at the top represent our work in progress, and the balls at the bottom are finished tasks. It’s easy to see that our capacity to start tasks easily outstrips our ability to finish them.

This is why we say stop starting and start finishing. You can become more productive by widening the spout at the end of the funnel, but that's pretty difficult. Cramming more items into the top is much easier, but it still isn’t the way to go. That creates excessive multitasking which hobbles productivity and diverts attention from what really matters.

Only Human

In their book Switch, Chip and Dan Heath relate a story from W. Edwards Demming about a company plagued by a rash of small fires in its factory. Most of them didn't cause much damage, but eventually one surely would. The company president's solution was to send an all-hands letter urging employees to please set fewer fires. I’m guessing it didn’t help. Nobody actually wants to set fires in their workplace, so the real problem was not with the people, but their environment. Change the environment and behavior will follow. 
Poor LNB practices are just like those fires. Most of them are small and cause minimal damage. While the chances are very low that any random fire will grow out of control, it's only a matter of time before one of them will. For lab notebooks, that means a conflagration of liability, headache, and lost time. Just like wearing your seat belt, it’s not the odds but the stakes that matter.

Unfortunately, what’s the usual approach used to prod scientists into keeping a better notebook? More of the same: Send an all-hands email asking everyone to please keep a better notebook. It’s not effective because the real problem was overlooked. Begging and pleading won’t change human nature. Neither will a carrot and stick approach. But what about the researchers’ work environment? Could something be influencing them to neglect their notebooks? Could it be tweaked for the better?

Change your WIP, change your life

Limiting your work in progress (WIP) is a pillar of Agile practices, but in the absence of an Agile mindset, most people operate as if WIP limits don’t exist. Like it or not, we all have one even if we're not aware of it. If we exceed our natural WIP limit, we get less work done because costs of task switching strangle our time and attention. It leads to a downward spiral of time scarcity and poor decision making. See Sendhil Mullainathan and Eldar Shafir’s excellent Scarcity for further reading. But how do WIP limits encourage better notebook keeping practices? The answer is that they don’t work directly on the problem, but rather on the environment. Appropriate WIP limits reduce the sense of overwhelm and time scarcity, encouraging researchers to focus on important but not urgent tasks like good LNB practices.

Appropriate WIP limit acts like a flow regulator for your work. Now our funnel looks like this:

As C. S. Lewis said, “It’s not the load that breaks you down. It’s the way you carry it.” Because of this, it’s ironic that productivity tends to increase under these conditions. In time, the scientists discover the biggest winners are themselves. Their own well-written notebook entries from the past help them with current work, including composition of journal articles that boost their reputation and career. These benefits don’t just add up over time — they compound! Researchers truly own the solution and become enthusiastic about it. No longer is keeping a good notebook a nuisance to be avoided. Not when they have proof that it’s a sound investment in themselves. And don’t forget about the company itself. Executives and the legal department can breathe a sigh of relief too.

Conclusion

Technical debt will always be a problem, but it makes sense to go after the biggest items first. Good notebooks are the cornerstone of good science. The risks are too great to ignore, but we can’t ignore human nature in our search for an effective solution. It’s tempting to conclude that there’s a trade-off between productivity and quality, but good WIP limits are a tweak to our work environment that lets us have both. Try it out for yourself and you won’t be disappointed!

Technical Debt in Drug Discovery

Almost every resource on Agile practices such as scrum and kanban is aimed at software developers. There are good reasons for that, but there’s no reason why those practices can’t be used for other types of knowledge work like drug discovery. Knowledge work is knowledge work, regardless of the substrate. It’s an iterative cycle of design-make-test-analyze. Software people write code at a computer and tinker with it until it’s a usable app; People like me make molecules in a lab and tinker with them until it’s a usable drug.

If you read through a book on managing software development projects, it’s remarkable what you can learn if you just swap of the names of individual roles (programmer, tester, project manager, etc.) for other names (associate scientist, principal investigator, group leader, etc.). Aside from unfamiliar titles and roles, there are other unfamiliar terms. Some are specific to the software industry, but some are more general. One term in particular describes the shortcuts and uncorrected errors that come back to haunt you later. That’s called technical debt. For software people, that’s usually sloppy, undocumented, non-standardized code. It's hard to follow. It’s bloated, irregular, and buggy. Sure, it’s faster to write bad code, and if it works well enough you can deploy it to your customers. But it’s risky because it introduces unnecessary fragility that can cause enormous headaches later on should something go wrong.

Since the process of knowledge work is very similar across many industries, I started to think about what technical debt means to medicinal chemists like me. Just like in software development, medchem technical debt is invisible and silent, a booby trap of our own making waiting to be sprung. And just like software development, accruing technical can speed along the initial stages of a project, but months or years later the awful truth inevitably surfaces. The necessary re-work could be minor, but perhaps the entire project is put in jeopardy. There could even be billions of dollars at stake!

Below is my partial list of medchem technical debt items sorted into broad categories. Note that these apply to traditional small molecule projects and not necessarily new modalities such as stapled peptides, millamolecular compounds, or antibody-drug conjugates. Note also that many of these items are interrelated. For example, over-reliance on a small subset of synthetic tools can lead to long, inelegant, unscaleable synthetic routes. Another example is how the over-optimization of selected properties can lead to flagrant violations of good drug design principles, especially molecular weight, hydrogen bond donor/acceptor count, and solubility.

Planning

• Inadequate literature searches lead to missed opportunities to find easier synthetic routes and head off potential intellectual property issues
• Long, inelegant, unscaleable synthetic routes are often the result of the above point, and create further problems
• Compounds with known structural liabilities such as PAINS, PrATs, and other problematic functional groups
• Chemotypes known to be problematic within your company or research group
• Flagrant violations of good drug design principles such as cLogP, molecular weight, polar surface area, hydrogen bond donor/acceptor count
• Over-reliance on a small subset of synthetic reactions is addressed in many journal articles. For an example, see this reference.

Synthesis

• Poor laboratory notebook keeping practices are potentially the most costly in terms of lost time and frustration. In the worst case, it could cost billions in sales if when a blockbuster drug's patents are challenged in court.

• Poor lab technique gives a false negative result on synthetic viability
• Purification issues leave open the possibility of impurities that skew assay results

Testing & Analysis

• Unrecognized assay drift will lead you to believe things about your compounds that just aren’t true
• Ignoring the error limits of assays gives the false impression that one compound is better or worse than another when in reality they’re not distinguishable from one another
• Over-optimization of selected properties (e.g., potency and selectivity) at the expense of everything else is a parochial viewpoint that ignores the complex interconnectedness of the drug discovery process
• Check out Garrett Hardin’s Filters Against Folly and Henry Hazlitt’s Economics in One Lesson for the same message in other areas of research

There are many more that aren’t in this list. You can probably come up with a bunch yourself. If you’re not a medicinal chemist yourself, try thinking of examples of technical debt in your particular area of knowledge work. If you are a medicinal chemist, you can go deeper down the rabbit hole by reading this Drug Discovery Today article from scientists at AstraZeneca.

Of course there’s always a balance to be struck between speed and quality. Research is complex, messy, and uncertain. Although some technical debt items are non-negotiable, others have some ‘wiggle room’ as long as you’re aware of the possible consequences and have a plan to fix it later (assuming the problems are fixable to begin with). Edmond Lau makes the same point in his new book The Effective Engineer, another guide on software development that transfers easily to drug discovery.

So there you have it. If you’re taking some time off over the holidays, why not read through a software development book or two? I think you’ll find many striking parallels that transfer easily to your own work.