Measurable Business Impact: Qualitative and Quantitative

At Briljent we define our success through the lens of our clients: Our success is your success. We position our core values, our Non-Negotiable Items, in service of keeping the “Briljent way of doing business” that our clients have come to know and appreciate for more than 25 years.

We revel in anecdotes and testimonies, but if you are in the professional services business like us, you know that measurable business impacts require both qualitative and quantitative evaluation. That is why it is a priority for us to measure the impact of our solutions on our clients’ business goals.

Getting data to find factors that influence the effectiveness of our services is no easy task. Unlike a checklist, measuring the true effectiveness of professional services requires special expertise. Thankfully, we have just such an expert on staff! In this series of posts, we summarize the key insights from our Learning Evaluation subject matter expert, Anthony Randolph, PhD through his dissertation, Human Resources Department: Professionals’ Experience Utilizing Data Analytics in the Training Evaluation Process.

In this first of three posts (parts two and three), we introduce Anthony and survey the limitations of incorporating data analytics into training evaluation. In the second post, readers will learn about the two research questions Anthony formulated to answer in his doctoral research, and we share a theoretical basis for his research. Finally, in the final post in the series, we share Anthony’s answer to the research questions and apply the theoretical work to everyday practice in Learning and Human Resources teams.

Anthony Randolph, PhD: An In-House Learning Evaluation Expert

Anthony Randolph is a man of many talents! At Briljent, he brings his expertise in the development of computer-based training, instructional system design, analysis, curriculum development, animation, interactive graphics, info graphics, HTML, graphic design, user interface design, video/film production, and instruction.

Whew! See what we mean?!

After earning his undergraduate degree from Indiana University in Studio Art: Graphic Design, Anthony earned his master’s degree from the Academy of Art University in Computer Art & Animation. While attending graduate school in the evenings, he worked full-time at Oracle Corporation developing web-based training for their online Oracle University, and later, Anthony was a Senior Visual Designer for an e-learning company, Digital Think, Inc. In this role, Anthony designed custom courses for such corporate clients as Avaya, Siebel, ADT, Cisco, Tenet Health Care, Gallop, Circuit City, KPMG Consulting, Aspect, and McDonald’s.

If those big names did not capture your attention, during his graduate-level studies, Anthony had the opportunity to work in the film industry at one of the leading animation and post-production companies in the world, Industrial Light & Magic, a division of George Lucas Films LTD, now Walt Disney Company.

If you wonder what could top credentials like these, what about a second master’s degree? Anthony received his second master’s from California State University in Instructional Technology Leadership. He has over 10 years of experience in learning and instruction, having served as a college adjunct instructor, full college instructor, secondary teacher, and instructional designer. More recently, Anthony obtained his PhD in Technology Leadership and Innovation from Purdue University in December 2021.

Data Analytics in Training Evaluation: The Gap

Anthony shared the spark that motivated his advanced study in Learning Evaluation: “While pursuing my master’s studies in Learning and Instructional Development in 2014, a specific class I took in Instructional Technology caught my interest in the state of technology around learning.”

Anthony saw that technology is quickly creating new pathways to apply data analytics to HR work: “[R]ecent technological innovations have presented opportunities for [HR professionals] to obtain this data, and consequently, apply new approaches for the training evaluation process.” We see this key observation as central to Anthony’s dissertation, Human Resources Development: Professionals’ Experience Utilizing Data Analytics in the Training Evaluation Process.

Anthony took care to define common terms that are as ubiquitous as their definitions are squishy. Just what is meant by “evaluation” and “analytics”? Anthony tells us, “[E]valuation is a process that consists of assessing, reviewing, analyzing, and judging the importance of information gathered,” and “the term ‘analytics’ will be used to refer to the process of converting data into useful information to improve individual and organizational performance.”

To put these together, when Anthony speaks of training evaluation by analytics, he means to communicate that HR professionals have a key role to play in judging the importance of training outcomes through converting data to useful information that improves individual and organizational performance.

While the potential for HR professionals to harness evaluation through analytics is great, Anthony notes that existing literature focuses on the promotion of analytics in the organization versus the practical application of analytics by HR professionals.

In short, despite the popularity of analytics, HR professionals face a research gap in practical application. Naming and closing this gap are Anthony’s main goals. In his own words, Anthony writes, “findings from this study will provide training departments and HRD professionals recommendations for expanded job role and/or function descriptions, as well as best practices for incorporating data analytics in the training evaluation process.”

What obstacles do you perceive in effective learning evaluation? We would love to hear in the comments! If you’re ready for more, head over to the second part in the series, “Mind the Gap: A Theoretical Look at Analytics in Learning Evaluation.”