Truth, Lies and Surveys: A New Methodology to Assess Culture

By Russell Reynolds Associates

Several years ago, the chair of a large global financial institution – we will call him Bill – received some concerning confidential information about potential control breakdowns and ethics violations in certain parts of the organization, which would put the institution at risk of significant regulatory penalties, as well as reputational damage. Bill brought in an outside firm to run an employee survey and analyze the situation. Reassuringly, the survey results showed the risks of such violations to be very limited, a finding which the HR department confirmed.

However, confidential concerns continued to stream in. In response, Bill commissioned another firm that applied a new methodology, known as randomized response, to repeat the assessment. The survey did not ask employees to report their own experiences with violations directly, but used an indirect method to allow employees to report them in an untraceable way.

The eye-opening results of the advanced analysis validated the confidential allegations that had been sent to the board, revealing systemic problems within certain business units due to behavior that was both unethical and illegal. No matter how good the financial institution’s culture might be in some areas, it was failing in others.

Unfortunately, insight came too late for Bill to address, as regulators caught wind of the bad behavior. Both the CEO and Bill were soon out of their roles, and the organization faced significant fines and reputational damage.

We often work with CEOs and board leaders who want to know if they have good cultures. They have created mission statements, held town halls, and visited factory floors to define what culture should be and personally reinforce it.

They have run engagement surveys to make sure they are canvassing every corner of the company, asking employees to agree or disagree with questions like “I am proud to work at this company” and “We care about our customers.”

Often, though, it remains unclear how well these efforts are working, and what risks may be lurking beneath the surface. In a recent study of nearly 700 US companies, researchers at MIT Sloan School of Management and London Business School found little to no correlation between stated company values and employee perceptions of the organization.¹ Newspaper headlines confirm this lack of alignment, routinely showcasing the gaps between stated corporate values and employee behavior in the form of corporate fraud, regulatory infractions, safety violations, and discriminatory incidents. These issues underscore how complex and multi-faceted culture can be – and how essential it is to have a clear view of employee beliefs as well as actions.

To help leaders better understand what is happening in their organizations, this paper addresses why current culture data often lacks credibility and reveals a new methodology with greater transparency and contextual data.

To help leaders better understand their organizations, this paper:

1. Explains why current culture data often lacks credibility
2. Reveals new methods for producing high definition views of culture
3. Provides a framework to see data in context

Why most culture data is not credible

No single perspective can paint a full picture of an organization’s culture. Instead, culture is the combination of both leaders’ values (tone at the top) and employees’ beliefs, behaviors and perceptions (echo from the bottom).

Most leaders have a clear vision of what they want their culture to be and go to great lengths to define and demonstrate it, making tone at the top relatively easy to gauge. Yet as our story about Bill illustrates, gaining an accurate understanding of the employee perspective is nuanced and challenging. Standard surveys that ask employees how strongly they agree or disagree with statements about their workplace often fail to uncover deep-seated issues.

Standard culture surveys have three shortcomings:

Few people believe the promises of anonymity that accompany these surveys. As a result, they answer as if their identity was known, which incentivizes them to make themselves look good. They are unlikely to respond honestly to questions on sensitive topics such as ethics, safety or commitment to diversity for fear of reprisal.
Loyalty is a powerful force. Most people have a natural inclination to protect those they work closely with, and report more positively on them as well.
External pressure from managers can come into play, particularly if the manager’s compensation is linked to survey results.

In combination, these factors tend to give management and boards an overly rosy picture of the strengths and risks in their organization’s culture. Without deeper insights, problematic behaviors and attitudes are difficult to address.

For example, in the 2011 Mississippi General Election, the results of a public vote on banning abortion demonstrates the wide variation between what people say and what they do. Despite having strong support in pre-election polls, the bill ultimately failed by a wide margin.² Princeton University conducted a research study via phone interviews with voters comparing their aggregate answers against county-level voting data. The results were stunning: While public records showed that 65 percent of voters in the 19 counties surveyed had voted “no,” only 30 percent of the 2655 voters surveyed by phone admitted to doing so, with another 20 percent refusing to answer the question.

These challenges can make culture seem elusive and immeasurable, however, it is not. Innovations in survey methods and advanced statistical approaches are creating new and better ways to calibrate culture. A new methodology, referred to as a culture MRI, has emerged which involves asking sensitive questions in a protected manner, making behaviors untraceable to specific individuals but providing greater insights across groups.

Assessing culture objectively

Most leaders seek the answer to “Do we have a good culture?” The answer lies in understanding the broader context of what a good culture looks like within a given organization. In our experience, there is no single definition of a good culture. Instead, leaders should be asking: “Do we have a mature culture?”

A mature culture can look very different from organization to organization, but in every case, it is a culture marked by three features: alignment between culture and strategy, alignment between leaders and employees, and alignment across business units and regions. This alignment is what leads to a high-performance orientation across the organization, on both financial and non-financial measures.

A culture MRI shows leaders where things currently stand within their organization. These data points, when mapped against a cultural maturity framework, will depict the level of maturity of various factors across multiple dimensions.

CULTURAL MATURITY MATRIX

In mature organizations, there is little variation in culture perception or experience between different seniority levels or across different business groups.

MATURE CULTURE

Strong alignment between leaders and employees
Low variation across organization

APPROACHING MATURITY

Moderate alignment between leaders and employees, and across organization

DANGER ZONE

Weak alignment between leaders and employees
High variation across organization

While there is no true endpoint to cultural maturity – culture needs to constantly adapt to evolving situations – this framework helps clarify where organizations should be heading, and which gaps they must close to get there. If employees have uniform perceptions of the organization’s culture, but it is at odds with that of leadership, leaders will want to better understand what is happening on the front lines across the organization. Conversely, if different groups of employees have vastly different perceptions of culture, leaders will want to focus their efforts on the groups with the most variance.

Seeing through walls: a new approach to measuring and managing culture

In an ideal world, leaders would understand what employees truly believe instead of what they think management wants to hear, and to accurately understand what behaviors they’re seeing at work. They would be able to see cultural hotspots, and pinpoint precisely where the breakdowns are occurring. With this information, they could build thoughtful strategies to address concerns, moving the entire organization toward tighter alignment, lower risk, and better performance.

A number of companies we work with have recently adopted advanced tactics, using the methodology that is embedded in Russell Reynolds Associates’ new culture analytics approach. Developed by behavioral economists, political scientists and neuroscientists, this approach gets closer to employees’ deep attitudes and behaviors through innovation in survey design as well as new ways of analyzing responses.

A key feature of Russell Reynolds Associates’ culture analytics approach is that employees never have to respond directly to questions about potentially incriminating or embarrassing behavior. Instead, sensitive statements are bundled with neutral, odds-based statements (such as dice rolls or relatives’ birthdates) and employees simply indicate whether any of the statements are true. Known as randomized response, this survey design makes it impossible for anyone to know the respondent’s true attitudes or behaviors, while providing enough information to calculate precise group-level estimates of behavior.

KEY FEATURES OF RRA’S CULTURE ANALYTICS APPROACH

PSYCHOLOGICAL SAFETY – A brief tutorial at the start of the survey helps respondents understand how and why their answers will not be traceable to them individually.

RANDOMIZED RESPONSE – Sensitive statements are bundled with neutral, odds-based statements; respondents can indicate how many statements are true without specifying which ones.

ADVANCED STATISTICAL TECHNIQUES – Sophisticated algorithms translate survey data into actionable information using probabilities to determine group-level estimates.

THE FULL PICTURE – Results offer a complete, multidimensional “cultural MRI” revealing the degree of alignment among leadership values, employee values, and lived experiences, as well as the degree of alignment across business or demographic groups.

Toward a higher definition view of culture

Taking a closer look at culture often means bearing difficult news, as leaders learn that compliance with desired norms is a bigger challenge than they may have believed. Yet failing to acknowledge these gaps is ultimately more painful and potentially destructive than proactively uncovering them. In fact, leadership is often the key enabler (or obstacle) to a mature culture, and the benefits it can produce. Leaders who are not afraid to confront hard truths are essential to building resilient cultures that sustain high performance over time.

We recommend these critical steps as starting points to gaining this authentic understanding:

CLARIFY THE STRATEGY-CULTURE CONNECTION:

As a leadership team, it is essential to align on the organization’s strategic direction and the culture required to enable it. This starts by getting crystal clear on the “cultural truths” that need to exist and being explicit on how they link to strategic imperatives.

INVEST IN THE RIGHT CULTURAL DIAGNOSTICS:

To get a clear and accurate picture of an organization’s existing culture, leaders must ensure they have the right assessment tools and methods. These tools should not only use advanced techniques and statistical analyses, they should also reflect the unique language of your organization so that they are seen as relevant to your people.

LEVERAGE DATA FOR FOCUSED ACTION:

Organizations often try to do too much with the insights they get on culture. Keeping strategic needs in mind, avoid the tendency to do too much action planning and stay focused on a few organizationwide and surgical interventions in the areas that matter most.

CREATE A FEEDBACK LOOP FOR CONTINUOUS GROWTH:

Culture is constantly evolving, whether leaders want it to or not. The ability to regularly assess and refocus interventions is nearly as important as getting accurate information in the first place.

With a better handle on culture, boards are able to mitigate risk to the organization, executive teams are able to ensure that strategy becomes reality, and human capital leaders have a powerful tool to attract and retain the talent that best fits the organization’s ideals. The right culture starts at the top, but its benefits elevate the entire organization.

For sensitive topics, randomized response provides a deeper and truer level of insight than standard, direct methods can glean. In the study of Mississippi voter behavior mentioned earlier, the Princeton researchers also tried a variety of indirect survey methods to see if they would elicit more truthful answers than asking people directly about how they voted on a controversial issue. While all of the indirect methods produced more accurate results than direct questions, randomized response was the clear winner: of the nearly 1000 people surveyed with this method, 65 percent were estimated to have voted against the abortion ban; exactly the number reflected by public voting data. Similar results hold true in studies of Olympic athletes asked about doping, or drug use.³

Leaders can use this state-of-the-art culture analytics approach to measure employees’ alignment with a variety of corporate priorities, from inclusion to financial integrity to sustainability (see case study below on how Exxon used it to get an honest assessment of safety risks). The insights it surfaces then come together in what we call a “culture MRI,” or a visual representation of the degrees of alignment across different aspects of the enterprise, allowing leaders to quickly pinpoint both where and why problems may be emerging.

CULTURE MRI SHOWS MULTIPLE LAYERS OF RESULTS ACROSS FUNCTIONS AND REGIONS

I am appropriately recognized for my contributions

From this visual, it quickly becomes clear that the sales function in the Americas has strength in this topic area, while R&D in the Americas and in APAC have work to do. To address these “hotspots,” the organization might consider having sales leaders share best practices with R&D leaders.

CASE STUDY: Safety culture at ExxonMobil

SITUATION: Leaders at ExxonMobil wanted to understand the extent to which a group of construction contractors might be skipping safety rules. However, since workers can be penalized or fired for failing to follow all safety rules, few would want to admit to this behavior, even if they practiced it.

A traditional survey method which asked employees directly if they had skipped safety rules showed that very few workers – just 1.3 percent – were engaging in this behavior, which was ostensibly a good sign. When leaders applied our additional and more sophisticated techniques, however, it became clear that the problem was far worse.

Our survey asked employees directly about their behavior, but added in a virtual dice roll that allowed respondents to answer in a way that would make their insights untraceable to any individual. With this advanced technique, ExxonMobil found that actually more than a quarter of the construction contractors surveyed were skipping safety protocols, and that the rates of other concerning violations were much higher than previously reported, as well.

RANDOMIZED RESPONSE IN ACTION AT EXXONMOBIL

STEP 1: ROLL YOUR VIRTUAL DICE AND REMEMBER THE NUMBER.

STEP 2: HOW MANY OF THE FOLLOWING STATEMENTS ARE TRUE?

To be honest, I think a lot of the safety procedures and programs at this work site are unnecessary
My virtual dice roll was 5 or 6

Either 0 or 2 statements are true

Only 1 statement is true

FINDINGS: By applying a culture MRI methodology and re-tallying results across the organization, the advanced algorithms that power this approach gave leaders a full, multi- dimensional picture of how the attitudes and behaviors varied within and among groups and levels. This “culture MRI” ultimately allowed them to see that cultural variation was closely connected to leadership at the general foreman level, creating a clear path for future interventions.

AUTHORS

ANTHONY ABBATIELLO leads Russell Reynolds Associates’ Leadership & Succession practice. He is based in New York.
JAMIE HECHINGER leads Russell Reynolds Associates’ Social Justice and Philanthropy practice. She is based in Washington, D.C.
ERIC WIMPFHEIMER leads Russell Reynolds Associates’ Leadership & Succession practice Knowledge team. He is based in New York.
SEAN DINEEN is a senior member of Russell Reynolds Associates’ Leadership & Succession practice. He is based in Boston.
DEE SYMONS is a senior member of the Russell Reynolds Associates’ Board & CEO Advisory Partners. She is based in London.