Ethics and compliance programs have more data available to them than ever before. Capturing data on program activity and impact can be a powerful tool to assess whether your ethics and compliance program works in practice. Many companies also find value in benchmarking their programs against industry norms and in response to risks. That said, data’s nascency in the E&C space means that many compliance professionals are still struggling to understand what to do with all this data—and how to know what it is telling them.
So, how are organizations incorporating data into their E&C programs, and how is the E&C industry leveraging data as a means of understanding program impact and effectiveness? LRN’s own Emily Miner, director of Advisory Services, and Derek Clune, product manager, dug into these questions at this year’s PrivSec Global event. Their webinar discussion, “Leveraging Data in Your Ethics and Compliance Programs,” covered the following topics:
- What types of ethics and compliance data are organizations collecting?
- Who can access and use an organization’s compliance data?
- How does compliance data collection account for privacy, security, and regulatory concerns?
- What methods are ethics and compliance teams using to interpret—and act on—data?
What types of ethics and compliance data are organizations collecting?
The use of data in the E&C industry is fairly new. Oftentimes, Miner explained, the main types of compliance data that organizations collect fall into one of three categories:
Basic data: Data that measures activity—course completions, hotline usage, certification and disclosure submissions, financial or expense audits/monitoring—but not impact. While it’s important to track these metrics, they alone tell E&C professionals a limited story about their compliance programs. Basic data sets can’t confirm whether people retained what they learned in training or what they read in the company code of conduct. This type of data also leaves out actions like reporting concerns directly to a manager. In short, Miner explained, basic data sets are usually lagging indicators—meaning, they can only tell compliance professionals about what has already happened with respect to their E&C program.
Advanced data: This type of data digs deeper into understanding the impact of program activity and is consequently more actionable for compliance professionals. Miner described four types of advanced data:
- Knowledge: This data unpacks retention of information, not just completion of a training course. Examples include test and quiz scores, test out and tracking, and awareness (of reporting channels, policies, code of conduct, E&C resources).
- Behavior: This data looks at how employees are using and engaging with E&C resources and campaigns. Examples include clicks, visits, and time spent on resources; open rates on email communications; course completions; topic visits to code of conduct (alone and by geography, time, etc.); and search terms on company policies and procedures.
- Perception/Sentiment: This data is used to evolve and improve upon E&C initiatives such as training, communications, and overall ethical culture. Examples include learner course evaluations to gain feedback on training content/format, employee surveys on the relevance of E&C training and communications, and roundtables and focus groups to understand your organizational culture and uncover any risks in certain locations, departments, or employee subgroups.
- Risks: This data identifies and tracks indicators of misconduct and other risks over time. Examples include disclosures; reporting, investigation, and disciplinary action status metrics; and risk assessments of ethical culture and conduct, such as performance under pressure and comfort levels with speaking up to share concerns.
Miner noted that some organizations leverage additional advanced data metrics to understand and track risk, compliance, and ethical conduct. “We see some of our clients analyze contracts, financial spending habits around specific business units or regions. Some also looking at HR data such as rates of turnover and promotion.”
Predictive data: This type of data leverages machine learning and AI to spot risks and prevent misconduct or unethical behavior from happening, rather than being reactive to breaches of conduct.
Who can access and use an organization’s compliance data?
A key question from the discussion asked about who from the organization should have access to the results of compliance data analytics. Clune noted that a best practice he often sees in the E&C space is designating a core group of compliance professionals who have access to an aggregate of all company data sources, which they can then analyze—via manual manipulation or using an analytics tool—moving forward to better identify and manage risks.
That said, Clune also noted that making compliance data available to others within an organization helps create additional stakeholders and ethics and compliance champions. For instance, some organizations will give middle managers access to basic completion data so they can follow-up directly with their reports and help drive overall training and disclosure completions. Though the details of execution depend on the organization, this can be a more effective strategy than deploying a top-down, corporate compliance email communication.
Clune also emphasized the growing interest among senior stakeholders (such as executive teams and boards of directors) into performance metrics. When compliance teams share information on overall performance analysis—including high-level takeaways on knowledge retention, benchmarking, and year-over-year comparisons—it further underscores the case for executive buy-in on E&C programs. "We find that the more you can democratize knowledge and data within organizational restrictions,” said Clune, “the better your effect on the ethics and compliance strategy and impact.”
Miner agreed, adding common examples of ethics and compliance program KPIs that compliance professionals report to executive teams or boards. “We often see topics like training-related data—any trends or year-over-year knowledge of completions and effectiveness—as well as reporting, investigation, and disciplinary status. Also, any type of regional specific initiatives, policy rollouts, or new campaigns.”
How does compliance data collection account for privacy, security, and regulatory concerns?
Understandably, webinar attendees has questions about whether this type of data mining aligns with certain data privacy laws (e.g. GDPR, CCPA) and security requirements related to employee monitoring activities and disclosures.
Clune shared that what he typically sees from organizations is that their E&C courses are all mandatory, so there is often at least one unique identifier for each employee in the data. “[It’s usually] a random sequence of numbers so you can see at a high level what’s going on,” he said, “without being able to drill down to the individual or business unit.”
Disclosures, on the other hand, do require knowing the employee who submitted them. “If someone says they witnessed harassment in the workplace, we need to know who that individual is and we need to follow up on that,” said Clune. “If it’s a board service conflict of interest, a gift of over $1000, a contract to a government official, all those types of things require knowing who the individual is.” But that doesn’t mean compliance teams can’t exercise discretion. “In those cases, we limit where that data is going and who is responsible for reviewing that data.”
Miner noted that when her team conducts research on ethical corporate culture—which often involves surveying employees—they include upfront communication that only aggregated results will be shared in some minimum group size thresholds and that all data is de-identified.
Miner also spoke about the importance of accounting for regulatory guidance when collecting compliance data. “Both the UK Serious Fraud Office and the US Department of Justice have guidance materials that address two main points about leveraging data in compliance programs,” she explained. “One is that it's not a paper exercise—meaning, we need to be understanding program activities as not just a checklist of elements in place, but how effective they are. The second is that all activities should be grounded in the organization’s specific risks. Even companies within the same industry could have completely different risks based on their geographic footprint or supply chain.”
What methods are ethics and compliance teams using to interpret—and act on—data?
Miner and Clune talked about how more and more organizations are looking to compliance technologies that can visualize, interpret, and provide actionable data without requiring much manual manipulation. “Where the industry is trending is leveraging a predictive model,” said Clune. “It really makes ethics and compliance professionals’ lives easier when the tools can work for them in a prescriptive and predictive manner.”
The two explained how using AI and machine learning capabilities to analyze compliance data can help with predictive analytics and prescriptive actions to take to enhance program effectiveness. For example: Integrating test score data with financial spending habits and geographic footprint to deliver tailored communications about gifts and entertainment training and disclosures. “At LRN,” said Clune, “we’ve been focused on developing a business intelligence tool to aggregate all compliance data into a single source of truth, visualize it in a way that’s easy to digest, and provide prescriptive actions to take based on the data itself.” He shared three key components of LRN’s compliance analytics and benchmarking tool, Catalyst Reveal.
Measuring course performance: Purpose-built dashboards to provide key insights into how learners are performing on E&C content
- Completion metrics
- Time spent, test out, profiler performance
- Knowledge check performance
- Self-assessment question performance
- Benchmarking
- YoY comparisons
Measuring learner satisfaction: Direct feedback from learners on E&C training content
- Measurement of key course KPIs
- AI/ML-generated learner sentiment analysis
- Filtering by industry, company size, and revenue to compare against peers
Measuring company culture: Measure, compare, and track culture trends within the organization and against its peers
- Key insights into culture dimensions
- Export to share with senior leaders and colleagues
- Benchmark against peers
“We have developed a ton of different metrics within Catalyst Reveal that pull from E&C program content to create more effective measurement tracking and year-over-year analysis,” Clune explained. “Based on the regulators’ guidance mentioned earlier, our tool helps provide that continuous improvement loop of ethics and compliance programs.”
The key takeaway
As the evolution of leveraging data in E&C continues to expand, understanding how to use data effectively to determine program effectiveness and spot risk is crucial for compliance professionals. In addition to tracking emerging trends and ongoing updates to data privacy laws and regulatory guidance, E&C teams need the right tool to collect, share, analyze, and act on their data. To learn more about LRN’s benchmarking and analytics tool, check out the Catalyst Reveal product page at LRN.com.
