ESG Data, Ratings, and Investor Objectives

Sustainability reporting is growing fast. Of the largest 250 global companies by revenue, 96% published sustainability reports in 2020—three times the share in 1999.¹ Nearly a fifth of these 250 companies reported at least partially in line with Task Force on Climate-related Financial Disclosures (TCFD) recommendations. Companies that make up a majority of global market capitalization now self-report emissions data. When combined with model estimates, emissions data are now available for almost all public companies, as shown in Exhibit 1. This is great news for investors who want to allocate to companies based on their carbon footprints.

Regulation has further increased the amount of environmental, social, and governance (ESG) data reported by issuers. In the EU, the Non-Financial Reporting Directive (NFRD) came into force in 2014, mandating large companies publish information related to environmental and social matters. A proposal to extend the scope of disclosure and add audit requirements was adopted in 2021.² In the United Kingdom, TCFD-aligned disclosures are already required on a comply-or-explain basis for certain listed companies subject to high regulatory standards, and the government plans for them to become mandatory for many more types of companies by 2025.³ In the United States, the Securities and Exchange Commission (SEC) provided guidance regarding disclosure related to climate change in 2010⁴ and is now working on a comprehensive ESG disclosure framework.⁵

Nevertheless, interpreting corporate ESG reporting presents meaningful challenges. Sustainability reports may run a hundred pages long and substantially differ from one company to the next, and may not contain all the information that interests investors. Indeed, professional investors view issues like low data reliability, a lack of audit or assurance, low comparability across firms and over time, competing reporting standards, high costs of gathering information, infrequent disclosure, and too much unnecessary information as impediments to their use of ESG data (Amel-Zadeh and Serafeim 2017). With this backdrop, it is only natural to look for systematic ways of dealing with ESG data.

At first, ESG ratings may appear a promising tool to navigate this complexity. Providers of such ratings may look at hundreds of reported and estimated variables for a single company and boil them down into an overall ESG rating. Individual company ratings may then be aggregated into fund and index ratings or scores.

Thanks to their convenience, ESG ratings have grown in popularity in recent years. As of the end of the first quarter of 2021, close to $2 trillion were invested in more than 4,500 sustainable funds globally.⁶ Many of these funds rely on ESG ratings to make investment decisions, particularly passive funds replicating ESG-themed indices. For end-investors, ESG ratings appear to offer a simple way of determining whether a company or fund is more sustainable. For advisors, wealth managers, and asset owners, ESG ratings also offer a convenient label to show clients and beneficiaries that sustainability is considered in portfolios.

Regulators are more skeptical. In 2020, then-SEC Chairman Jay Clayton stated that he has “not seen circumstances where combining an analysis of E, S, and G together, across a broad range of companies, for example with a ‘rating’ or ‘score,’ particularly a single rating or score, would facilitate meaningful investment analysis that was not significantly over-inclusive and imprecise.”⁷ In January 2021, the European Securities and Markets Authority (ESMA) wrote that “the market for ESG ratings and other assessment tools is currently unregulated and unsupervised. When combined with increasing regulatory demands for consideration of ESG information, there are increased risks of greenwashing, capital misallocation, and products mis-selling.”⁸ In the UK, similar concerns have been expressed by the Financial Conduct Authority (FCA).⁹

Dimensional shares these concerns. In the next section, we present some of the shortcomings associated with ESG ratings. We conclude by suggesting a framework investors may want to consider when using ESG data and by presenting the approach we have taken.

Beauty is in the eye of the beholder. Often, so too is sustainability. ESG ratings providers frequently disagree on company ratings. Berg, Kölbel, and Rigobon (2020) estimate the correlation between the ESG scores of different ESG ratings providers to be only 0.54, and even lower when looking at the individual E, S, and G pillars. In comparison, the correlation in the credit ratings assigned by Moody’s and S&P is 0.99. A company may be identified as best-in-class by one provider and as average by another provider. These findings are consistent with Dimson, Marsh, and Staunton (2020). Exhibit 2, reproduced from Boffo and Patalano (2020), shows some examples of companies with a high level of disagreement.

Berg et al. further analyze the cause of these divergences and find it is attributed to three main drivers: difference in scope, difference in measurement, and difference in weights of the components of ESG ratings. In Exhibit 3, we show how these drivers play out in a simplified example.

• Difference in scope: When assessing the “S” pillar, Provider A focuses on gender equality, while Provider B focuses on health and safety. This difference could be due to the preferences and objectives of each rating provider. Alternatively, it may be due to a difference in their perception of data quality—Provider A may deem health and safety data unreliable and eliminate them from its score, and Provider B may feel the same about gender equality data.
  
  
• Difference in measurement: When assessing business ethics within the “G” pillar, the two providers focus on the same two issues—board independence and business ethics—but give different grades for business ethics. They could be measuring business ethics in a different way; for example, one may be rating quality of disclosures while the other is measuring the number of controversies, or they could be measuring the same thing in different ways. They could even be measuring the same thing in the same way but ranking the company against a different peer group—for example, Provider A may compare the company to peers in the same industry, while Provider B may compare it to the entire market.
  
  
• Difference in weights: Providers A and B attribute different weights to the E, S, and G pillars and to the issues within these pillars. Their ratings may have different objectives, or Providers A and B may have a different opinion on the materiality of each issue.
  
  
  

This simplified example illustrates the potential sources of dispersion across ESG ratings. In fact, most major ESG ratings seek to track 20 or 30 ESG issues and measure hundreds of individual indicators. The weights applied to individual issues also frequently vary by sector and company. The potential sources of divergence are vast, and since detailed methodologies and score attributions are generally not publicly available, understanding where discrepancies come from is far more difficult than in our simplified example.

This complexity means that ESG ratings may not be effective at achieving, and sometimes even work against, the sustainability objectives of certain investors. When looking at three ESG ratings providers, Boffo, Marshall, and Patalano (2020) find a low correlation between ESG scores and E pillar scores. They also find a positive correlation between the E pillar scores and carbon emissions for two out of the three providers they assessed, as carbon emissions was only one out of many environmental variables considered within their E pillar scores. In other words, a strong environmental rating was associated with emitting more, not less.

Given the subjectivity inherent in ESG ratings, we believe they should be viewed not as objective ratings, but as opinions—not unlike the buy/hold/sell opinions that have been issued by sell-side analysts for decades. When using ESG ratings from one provider to allocate assets, investors should be aware that other ratings providers may have dramatically different opinions and ratings. Investors should therefore strive to ensure that the ESG priorities of their chosen provider reflect their own ESG priorities, that the opinions of their chosen provider have a reasonable and adequate basis, and that they are supported by appropriate research and data.

This is particularly relevant for investors relying on ESG ratings to build portfolios. Indices based on ESG ratings may deviate significantly from natural market weights. For example, as of the end of March 2021, a popular global equity index designed to achieve strong ESG ratings assigned a weight of more than 12% to Microsoft—nearly four times its market capitalization weight—but excluded Apple and Alphabet.¹⁰ By contrast, in a competing ESG index from a different provider, all three companies were overweighted.¹¹ It is important that investors understand how these investment decisions are arrived at—however the opacity, complexity, and subjectivity of ESG ratings methodologies may make this difficult to achieve.

Separately, investors should also be on the lookout for changes in methodologies and restatements of past results. Several of the main ESG ratings providers have amended their methodology once or several times in the past few years, with varying degrees of transparency. The implications may be stronger than some realize. In “Rewriting History II: The (Un)predictable Past of ESG Ratings” (2020), Berg, Fabisik, and Sautner study one such change that occurred from 2018 to 2020. The new ratings were assigned retroactively by the ESG ratings provider, overwriting the ones that had been in place at the time. Berg et al. observe that the change materially improved the past performance of firms with high ESG scores: “Our estimates show that investing in firms with high initial E&S scores would not have led to economically or statistically significant performance gains. This is very different if we use the rewritten data: we now find a positive and statistically significant performance effect of the E&S score. However, this investment strategy would not have been available at the time the investment was made.” For investors, this is a potential look-ahead bias to be mindful of when using ESG ratings to study risk and returns.

What should investors do? Rather than rely on generic ESG ratings, we believe investors would be better served to identify which specific ESG issues are most important to them, understand the data used by their investment manager and how these data are used, and ask to see transparent reporting on specific ESG outcomes.

The starting point should be to define a clear sustainability objective. Examples of sustainability objectives may include reducing exposure to certain ESG risks, excluding companies involved in controversies, or tilting a portfolio toward companies believed to be more ethical. The broader the set of objectives, the more difficult it can be to manage the interactions among them. As we saw earlier, a “kitchen sink” approach that integrates dozens of variables may make it hard for investors to understand a portfolio’s allocations and may lead to unintended outcomes. We also believe investors should beware of combining funds and indices that rely on distinct ESG approaches, as it may add another layer of complexity.

At Dimensional, we have designed our sustainability strategies to have a clear focus on climate change, which we believe is the main sustainability challenge of our time. Climate science tells us that greenhouse gas (GHG) emissions are the primary driver of climate change, which is why our sustainability strategies focus heavily on reducing exposure to companies with significant GHG emissions or potential emissions in the form of fossil fuel reserves.

The next step is to assess the availability, quality, and objectivity of the data required to pursue this objective. Are the data reported by companies or estimated by third parties? If the latter, are they consistent from one data provider to the next? Not all ESG data are created equal, and certain disclosures are more reliable and robust than others. For example, companies tend to follow standards when reporting emissions, and the data can be cross-referenced and validated using multiple sources. By contrast, certain other environmental measures can be subjective and difficult to verify, and they may introduce unintended biases. For example, ESG data that rely on voluntary surveys may favor large companies with well-staffed reporting teams, regardless of their actual ESG performance (Drempetic, Klein, and Zwergel 2020).

The ability to provide transparent reporting should be another important consideration. With opaque methodologies and strategies pursuing multiple objectives, it can be difficult to understand whether a portfolio is delivering on its objectives. At Dimensional, our strategies allow investors to measure outcomes in a transparent and objective way. Exhibit 4 is an excerpt from the quarterly report we publish for all our sustainability strategies.

Finally, investors may want to assess the tradeoffs involved with their sustainability objectives. In principle, the integration of ESG represents a set of constraints to a portfolio. Such constraints may decrease diversification while increasing turnover and costs. On the other hand, expected returns may be affected by the incorporation of sustainability. While our research does not indicate a link between emissions and expected returns (Dai and Meyer-Brauns 2020), an ESG strategy that inadvertently tilts toward large, high relative price, low profitability companies would be expected to have lower expected returns than the market. When designing our sustainability strategies, we pay attention to maintaining sound investment principles, which includes maintaining diversification and exposure to the drivers of expected returns. Exhibit 5 shows the results achieved since the inception of our oldest sustainability strategies compared to their non-sustainability equivalents.

In summary, the fast growth in ESG data constitutes an opportunity for investors interested in sustainability. At the same time, this growth has introduced significant complexity that investors may find hard to navigate. In our view, the subjectivity and opacity of ESG ratings limit their usefulness to investors. Investors are best served by establishing their sustainability priorities themselves while being mindful of the tradeoffs involved. The Dimensional sustainability strategies, with their clear focus, robust methodology, and transparent reporting, may appeal to investors specifically interested in climate change.