Smarter L&D Evaluation: Run Incidental Experiments

In a recent Harvard Business Review blog Your Company Is Full of Good Experiments (You Just Have to Recognize Them) Hauser and Luca alerted us to take advantage of incidental experiments just laying around your business. When you want to “try out” or “test” a new product, process, or solution why not set up an incidental experiment. You are probably doing it anyway. Leisa runs a project with the old project management approach and John runs a project with a new approach then you compare the effectiveness of both approaches. Did one approach deliver better results and come in early, under budget, and with fewer iterations?

In years past (many years, but who’s counting), when I would consult with L&D functions they would baulk and gasp and whine and whimper when I would suggest setting up an experimental study to determine the effectiveness of a learning interventions. Honestly, many of them would not call me back. I would hear “the doctor wants to make this more complex than necessary”, “this will take forever”, “we don’t even have access to any organizational data”, and “we could never get budget approved for that!” Now, who is baulking and gasping? They all want to do it and they all have a budget! Just a nerd ahead of her time I guess.

From an L&D perspective, incidental experiments are usually embedded in your design anyway. When you have staggered program rollouts (i.e., Team A starts in January and Team B starts in March and Team C starts in July...) you have embedded control groups. In January, you have a group who is receiving the treatment (Team A) and groups that are not receiving the treatment (control groups) Teams B and C. Thus, you can compare Team A’s (treatment) performance with Team B and C’s (control) performance in January. If you need to you can make program modification for March when Team B starts the program. With a staggered rollout everyone is receiving the training benefit just not all at once so you can optimize the program. If you do not have staggered program rollout, you need to build it into the program design so you can conduct incidental experiments.

Do not forget the entire purpose of incidental experiments are to afford data-driven decision making. Take advantage of your incidental experiments just laying around and do some Smarter People Planning!

Are you currently running any incidental experiments? Do you have any planned? What are your barriers to conducting incidental experiments? Comment below to start the conversation.