Data quality and completeness is a major challenge in the ESG data market as, despite increasing data availability, significant gaps still exist. Unlike financial reporting, which has predetermined variables in a company’s balance sheet and income statement, there is no such uniform reporting mechanism in ESG reporting. A prime example is Scope 3 emissions data, especially for SMEs, where data vendors tend to rely on estimations and proxy data derived from industry averages and eventually tend to dilute its quality because they are not entirely accurate. In estimating financed emissions, most banks use a methodology drawn up by the Partnership for Carbon Accounting Financials (PCAF). Other approximation techniques for Scope 3 estimations are Input-Output Models and a Lifecycle Analysis approach.

Data standardisation is another area of critical need. ESG data vendors use corporate disclosures to back up their ratings and scores, which requires significant harmonization since different jurisdictions have different asks from corporates operating in their territories. As a result, this makes international comparisons complicated. For example, for safety-related data points, some companies disclose raw numbers (e.g., the number of incidents), and some disclose a relative number where the base varies from company to company (e.g., injuries per unit of hours/days worked). In contrast, others will provide a percentage scale (e.g., lost time incident frequency. Based on Evalueserve’s experience with ESG data, these differences can make it difficult for an ESG ratings agency to ensure consistency and comparability in the data used in generating an ESG score.

Inconsistent approaches and guidance by data providers are a widespread source of frustration. Firms place different levels of importance on the broad categories of E, S, and G, resulting in drastically different ratings from vendors of the same company. For example, an ESG ratings vendor might penalise or reward a company depending on whether the disclosure can fit into their specific methodologies. In cases where data is missing, providers typically apply different rules. Imagine a situation where Provider 1’s approach is to take the industry average as a proxy, but Provider 2’s approach is punitive, wherein they assume no value at all. Both service providers have rationales to support their methodologies, but this makes direct comparison difficult and faulty. Therefore, understanding the ins and outs of a prospective investment is crucial for decision-focused comparison.

Double Materiality is another highly debated concept in the ESG space where rating providers can offer divergent views. Double Materiality refers to how the information disclosed by a company can be material in terms of its implications for the company’s financial value and its impact on the world at large. For instance, one rating agency’s approach may seek to capture the company’s impacts on the environment and society when assessing its climate change disclosures. However, another provider may take a more limited view, looking solely at the potential impact on the shareholders and the impact on the company’s development, performance, and position.

The example of Chevron Corporation illustrates the potential for significant divergence in ESG ratings from different providers.² Sustainalytics gave Chevron an ESG ranking of 43.0 for 2022, placing it in the “severe/high risk” category. MSCI, on the other hand, gave Chevron a ranking of 4.2 to 5.7, placing it in the “average” band. These very different outcomes for the same company result from each provider’s specific approach and methodology. MSCI’s rating is influenced by its criteria, which prioritizes environmental factors over social and governance factors. Sustainalytics, on the other hand, considers a broader range of factors, with a more equal weighting across all three dimensions of ESG. This difference in methodology is reflected in the specific factors that each provider considers when evaluating Chevron. For example, MSCI recognizes Chevron’s efforts in disclosing environmental data, implementing emission reduction targets, and engaging in stakeholder engagement. Sustainalytics also considers these factors, but it also examines Chevron’s carbon intensity, oil spills, involvement in controversial projects, executive compensation, and lobbying activities.

This divergence in ESG ratings among different providers can create confusion for investors trying to assess a company’s ESG performance. Indeed, in practice, many investors need to subscribe to multiple ratings providers to ensure they have a well-rounded understanding of a company’s ESG performance.

Year-on-year score revisions present a further challenge where one sometimes finds notable deviations in a company’s ESG rating from one year to the next as offered by the same ratings provider. A study conducted in 2021, which reviewed the ESG scores provided by a major ESG ratings provider between 2018 and 2020, found that the same ESG dataset covering the same set of companies had varying scores at different times. Upon downloading and analyzing the data, not a single ESG score was the same when viewed at separate points in time. The variation was due to subsequent reconciliations by the provider in its scoring methodology between the two years being compared.³ This rewriting of historical data by providers is an ongoing process to improve the efficacy and relevance of their data and by streamlining their methodologies from time to time. Problematically, however, retrospective score rewrites can significantly change companies’ ESG scores, undermining their utility to investors making investment decisions based on this information.