Gone are the days of the checklist compliance program, where what mattered was the existence (or not) of key program elements. These days, we focus on “effectiveness,” neatly summarized by the US Department of Justice’s three fundamental questions in their Evaluation of Corporate Compliance Programs guidance: is the compliance program well designed; it is adequately resourced and empowered to function effectively; and does it work in practice. In short: Does the program work…is it effective…how do we know? Enter the rise of benchmarking, and the tools to support it.
It's human nature, I suppose. We constantly compare ourselves to others, so why not our organizations and programs, too? Makes sense, and yet—the term “blind benchmarking” was coined for a reason. Benchmarking can be a productive and motivating tool for measuring compliance program success, but it should never be the only tool in the tool shed. So where is benchmarking relevant, and what are its limits? Let’s dig in.
What is compliance benchmarking, and how does it help E&C programs?
Benchmarking is the practice of comparing business processes and performance metrics to industry bests and best practices from other companies. The DOJ’s “fundamental questions” provide a helpful framing for common ethics and compliance benchmarks—those commonly used as well as those that are desired.
- Program design: Industry comparisons are helpful here, with organizations seeking metrics like number of hotline/helpline cases per 1,000 employees; total length of E&C training per employee per year; what topics employees are trained on; and when to translate materials into a given language.
- Program resourcing and empowerment: In this area, organization size might be a more relevant category than industry, although the latter still applies, particularly for organizations in highly regulated industries. Ethics and compliance leaders want to know things like “What is the typical program budget for an organization of my size?” and “How many compliance FTEs?” Governance structures and reporting lines are also key areas of inquiry.
- Program effectiveness [“work(ing) in practice”]: Common metrics here include course completion stats, learner knowledge and retention (e.g., via “knowledge checks” as part of or following training experiences), and measures of ethical culture (e.g., comfort speaking up, perceptions of organizational justice, ethical role modeling by leaders, etc.).
Tracking these metrics in and of themselves provides valuable insight into program appropriateness, reach, and impact. Comparing performance against similar institutions (size, footprint, industry, etc.) can further demonstrate effectiveness to leaders and “justify” the investment the organization is making in ethics and compliance, as well as make a case for additional or different investment and strategic direction.
Benefits of benchmarking ethics and compliance programs
Ultimately, benchmarking ethics and compliance programs provides context. We don’t—or shouldn’t—operate in a vacuum, and understanding how other programs run can help open our eyes to approaches and best practices we might not otherwise have considered on our own. Benchmarking, or perhaps a healthy spirit of competition, can help foster innovation in an organization and act as a “marker” on their journey/transformation.
Comparing your E&C program to others in your industry—and around the world—can also help you set standards for measuring success and offer a more accurate view of how your program is progressing. Benchmarking can improve learner outcomes when you combine industry-level data with insights into your specific culture sentiment, course resonance, curriculums, and course outcomes.
Finally, let’s not forget the power of internal benchmarking: establishing your baseline and then measuring progress on a longitudinal scale.
Compliance benchmarking is one piece of the puzzle
Like all things ethics and compliance, benchmarking is not one size fits all. The DOJ acknowledges this in its guidance, stating, “We recognize that each company's risk profile and solutions to reduce its risks warrant particularized evaluation (emphasis added). Accordingly, we make a reasonable, individualized determination in each case that considers various factors including, but not limited to, the company’s size, industry, geographic footprint, regulatory landscape, and other factors, both internal and external to the company’s operations, that might impact its compliance program.”
The DOJ also states that the starting point for any ethics and compliance program should be its risk assessment. Even organizations in the same industry may have different risks. For example, LRN evaluated the maturity and effectiveness of two programs in the same technology manufacturing industry segment. And yet our recommendations for each were different because they were exposed to different risks due to their export chain: one company was a major exporter to the Chinese tech company Huawei, which was sanctioned by the US government in 2019, whereas the other was not. Thus, comparing policies and procedures around trade control, for example, would not have been appropriate.
A benchmark data point quite often represents the average of whatever is being measured. And, average isn’t always good! Case in point: LRN assessed nearly 150 codes of conduct from the top-listed companies in the US, UK, France, and Germany and found that 70% of the codes had a reading level of over 9.5—higher than what is typically recommended for content to be accessible to a broad population.
How to benchmark your ethics and compliance program effectively
Benchmarking, when used appropriately, can be a helpful tool as compliance programs seek to continuously improve. But, they aren’t the only answer. Here are some points to consider:
- Ensure the benchmark is valid. You’ll rarely be able to compare apples to apples (see: different risk profile), but you should at least be comparing apples to oranges and not broccoli. The more similar to your organization—size, industry, geographic footprint, makeup of workforce—the more relevant and applicable a benchmark will be. Recency also matters in validating benchmarks. You don't want to measure program effectiveness using outdated publications or metrics.
- Look to industry peers, but modify based on your own risk assessment. Your company has its own unique risks that must be managed. Just because industry peers commonly train their employees on certain topics, does not necessarily mean you should; you might operate in a different geography, for example, or have a different supply chain.
- Consider your program goals. Is the benchmark where you want to be? We often hear from our client partners that they don’t want to just meet the benchmark; they set a higher bar, and want to be leading the trends, not following them. We see this regularly in our work helping companies assess and understand their ethical culture: the benchmark might be helpful in year one, but after that the focus is more on outperforming the internal benchmark. The industry benchmark grows less important over time.
- Use benchmarks in constellation with other data sources. Benchmarks are just one source of truth, not the single source of truth. In seeking to understand impact and effectiveness, try to get a holistic picture. For example, combine metrics on training topics with employee feedback on the relevance and applicability to their roles, as well as their satisfaction and enjoyment of the experience overall.
The key takeaway
So, to benchmark or not to benchmark? Like great improv, the answer is “yes, and—” (Yes, benchmark. And, identify how it fits in your larger strategy to measure program effectiveness.). We know how valuable benchmarking can be to understanding ethics and compliance program effectiveness, which is why we’ve developed a new analytics and benchmarking tool called Catalyst Reveal. It pulls various sources of data (training metrics, learner sentiment, and ethical culture measures) together into one place, and allows program administrators to see their own progress over time, drill down into different population segments, and benchmark against other organizations (our filters include industry, number of employees, and annual revenue).
However, when the benchmark isn’t relevant or applicable for whatever reason, that’s OK: it’s easy to turn it off, and just focus on yourself. To learn more about benchmarking compliance programs, visit our Catalyst Reveal page.
