Data’s Delicate Balance

Recently I was contacted about some previous research I did with Dr. Karl Kapp on the numbers it takes to develop one hour of instruction. This is not uncommon, the information, though dated is valuable. And, yes, before I get into the main topic – Karl and I are working on those numbers again for you!

What was interesting about this communication was the additional inquiry into an article released by Forbes on reading rates of adults.

Though the request was for (hopefully) more data surrounding the latter article, it prompted a completely (unsolicited) discussion (I think maybe I scared them off honestly) on data and its impact on project planning and solution design.

So here is a summary of my current thoughts on this topic . . .

What the Forbes Article Says To Me
The scholar in me wants to go and read the Staples research to evaluate the validity and reliability and also, the ability to generalize to a larger audience, however I will stick with what the author at Forbes wrote because it really resounded with a conversation I recently had with students. (**Please note, the scholar in me is also revamping the original research Karl and I did – so calm down folks who would rather distract from the main discussion.**)

When thinking about solution design what the Forbes article indicates to me is – we (the average human “we”) don’t have time in the day to read, read at effective rates and lastly and most importantly – comprehend all we could from what we read.

So foremost, when we develop training the point is to have the learner gain full or the fullest comprehension they can. We use strategies like chunking or sequencing to help breakdown materials to assist in that comprehension. So complexity of information factors here.

But, also, we use other attributes and factors in attempt to provide an opportunity for maximum comprehension. Learning characteristics that are intrinsic and extrinsic – what motivates the learner, what is the average reading level of our target audience, preferences for being engaged with the content, etc.

What Am I Trying to Say?
This peers inquiries and the article made me think about general design practices and why data/articles like the one presented can lend to misconceptions about how we can design and design effectively. I am in no way insinuating that perhaps organizations are attempting to create training that is based solely on this information though.

However, a novice instructional designer might – as they may not be well versed in examining data to drive decisions. They may also struggle with how to utilize research to inform decisions about design solutions.

Even I caught myself reading the article thinking about how many words were on a screen and what rules of thumb had I used in the past to gauge how many screens given how much time. I also pondered if this information would assist me in doing that better.

Then, of course, the ID consultant mindset kicked in with all the “yeah but” statements. “Yeah but what if the screen has engagements like rollovers?” “Yeah but what if the screen is a set of bullet points with audio?” and you get the point.

Not to mention there has been no factoring on the planning side of things yet. For example, the authoring skills and experience of the developer. If you read Karl and mine’s linked article you can see project estimates in the instructional design field are thwarted by many factors that we can only guess at when we create our initial numbers.

Final Thoughts
Data can be a fantastic way to support decisions we make in planning, budgeting, and subsequently creating our educational products. However, data should be used liberally or in a holistic manner to mitigate skewed justifications for our choices.

As I always tell my students, we are the learners advocate. If the data is not helping us help the folks sitting in those seats taking the training we created then we have to reconsider its value in how it shapes the solution.

What kind of data does your organization use to drive training and talent development?