Picture a nondescript meeting room in a nondescript but oversized office building belonging to a remarkably well-known investment bank. Now imagine five heads (and bodies) huddled together, appearing to be discussing a matter of great import. Then learn that these heads belong to two SA[1] Editors, two analysts, and one Head of Research discussing how to handle the publication of an important report that had the small matter of anti-money laundering in the title but no elaboration anywhere else. “Can’t believe we missed this, even our chatbot!”. He’s called Felix, by the way, the chatbot.
One of the SA-Editors, who likes to keep herself to herself, pipes up with a half-frustrated, half-triumphant “I didn’t!!!” (Three exclamation marks too many for an editor on any given day, but more so for her), that the report wasn’t even talking about anti-money laundering (AML) – something she had questioned during the first edit but which the analyst had missed correcting in the title. AML is a sensitive topic at the best of times but critically problematic if left hanging around like a stray cat, abandoned. “It was actually about asset-liability matching (ALM)”, she continues, “and the report was otherwise well-explained, well-thought-out, and well-written. Felix wouldn’t have caught the AML/ALM problem.”
Just a small flip of a letter or two (did I tell you about the whole adsorption/absorption fiasco in the Chemicals report? Maybe another day.), and you have the entire department in a tizzy. Of course, an erratum was published shortly thereafter with profuse ‘thank yous’ directed at aforementioned SA-Editor. But no one likes errata. No one likes to be felled by a measly letter. And while not everyone loves an editor or an SA (or anyone, for that matter), they understand the vital role they play in content review.
If you are an editor or an SA, you’ll have many such anecdotes, some tongue-in-cheek, some ironic, but, more often than not, critical - as most things tend to be at Financial Services (FS) firms that must work within the confines of stringent regulatory guidelines and communication that is becoming increasingly cross-border (and often misunderstood or used with malice). In this age of Artificial Intelligence, social media, and a global culture, we need to be hypervigilant about the way we talk, the way we write, the way we think, and the way we use the ever-expanding Pandora’s box of technology tools. We can’t continue to turn a blind eye to them, self-professed luddite or not, nor be blind to their faults. As we know all too well, one person’s comedy is another person’s tragedy.
An article by MIT Sloane School of Management (When AI Gets It Wrong: Addressing AI Hallucinations and Bias - MIT Sloan Teaching & Learning Technologies) says that GenAI has been “shown to produce images and text that display biases related to gender, race (Nicoletti & Bass, 2023), political affiliation (Heikkilä, 2023), and more”. Not only that, but they can also be wildly inaccurate. This is no one-off instance of machine gone rogue but happens so often as to have “earned their own moniker; we refer to them as “hallucinations”. A senior engineer at a firm that shall not be named said to me the other day, “it all looks pretty believable (and unbelievably pretty) but isn’t always accurate. So, anyone depending solely on these machines will miss something very big, very soon”.
As guardians of financial research – its data, facts, and language - editors and SAs will continue to play a crucial, non-optional role in investment banking/financial communication. We must all be judicious in the way we use technology and check and double-check the output. While many of the rules- and data-driven processes can be automated or even created (text summarization tools, for example), they would need an experienced editor/SA (and analyst) with a deep understanding of their domain, great attention to detail, and a discerning eye to spot errors that can escape the patterns AI has been taught to identify.
Evalueserve and the Art of (Research) Lifecycle Maintenance
As we’ve said before, some rule-based and data-driven processes are more suited to automation, making them significantly more efficient and error free, for example some aspects of writing (Authoring) and proofreading (Level 1 editing). At the same time, and more importantly, we recognize the role that language experts (editors and translators) play in finessing the final research product that investors consume.
At Evalueserve, we straddle the seemingly contradictory worlds of the human and the machine, thinking about innovative ways we can integrate the two in all our business lines. In FS, for example, we have specialized in professional language services for more than 20 years, facilitating effective communication through our multinational teams of language experts who edit and translate reports written by research authors from all over the world. With offices in 12 countries and more than 4,500 employees from diverse faiths and backgrounds, our cultural diversity offers multi-faceted benefits and perspectives to our clients, enabling timely, accurate, and professional project execution and insights across cultures. The diversity of our teams translates into diversity of perspectives – saying a business division is the ‘crown jewel’ of a company may not seem like anything to someone sitting in the location where the bank is headquartered, but it means a great deal to a regional investor reading the report.
Let us now turn our attention to the lifecycle of an equity research report, as an example, and analyze how we can combine the human with the machine in a less dependent and ultimately less risky way.
Our experience with professional language experts, continuous commitment to innovation, and experiments with AI suggest that AI tools can be brilliant for shorter, more time-sensitive reports where an SA/Compliance review is more important than editing and in some cases all you have time for. However, for longer, thematic pieces, all the parts of the report lifecycle gain in importance: writing, proofreading, editing, and SA/Compliance. And because the writing (or translation) that is aided by AI is more accurate grammatically, the writing and the proofreading take less time, leaving more time for the editor/SA to drill down to the nitty-gritties of the extra-grammatical and softer characteristics of language - inclusivity, compliance, logical flow, consistency, and clarity of the main theme or idea.
A crucial point here is that AI tools need to be deployed within self-contained and discrete client environments to significantly reduce the (incredibly high) regulatory risks associated with the FS domain – data breach, plagiarism, copyright infringement, etc. – while maintaining the uniqueness and sanctity of the financial firm’s research content.
Ever the Twain Shall Meet
On the one hand, Evalueserve has been encouraging its analysts to use its proprietary AI tools available on our Digital Intelligence & Robotics (DIR) platform, an in-house library which hosts a range of AI-enabled, generative solutions that produce text and summaries (calls/paraphrasing/bullets). This helps to streamline processes through micro-automations, reduce data errors, improve language accuracy, and increase efficiency.
On the other hand, although DIR has enabled legacy reports to be automatically generated, resulting in a certain degree of consistency, a high level of (grammatical) accuracy, and significant time-saving, Evalueserve’s trained and experienced editors review research with a deep understanding of both the structural elements of research reports (such as formatting, design, visual clarity, and logical flow) and the subtleties of language (e.g., context, domain-specific terminology, phraseology, style, and even irony).
At a deeper level still, an editor or a Supervisory Analyst scrutinizes the content for any breach of propriety, socio-political biases, cultural stereotypes, poor taste, offensive humor, and client-specific and regulatory compliance, ensuring coherence, clarity, and quality in every report. How about house style/guidelines, you ask. These style rules are frequently broken, making an editorial review non-optional. Ensuring that client-specific formatting, punctuation, and wording (house style rules) are used consistently requires a significant amount of time – and dare we say an editor’s patience – and rewards the clients’ research with a much more rigorous and nuanced review. As we are all aware, inaccurately written subject-specific terms, (off-color) humorous financial jargon, and even colloquialisms sometimes find their way into the documents we process. AI tools often fail to catch these, and analysts have to depend on editors to ensure that style errors or inconsistencies don’t slip through.
So, Who Does the Laundry: How AI Is Helping Evalueserve Move Up the Value Chain
Using GenAI to create research notes has been a game changer for Evalueserve. Our existing clients have taken note of our in-house capabilities through the demos that we have provided over the past year. We would like to make it clear that these tools have not yet been implemented within a client environment, but we have been continuously pushing for wider adoption at the Evalueserve level through extensive cross-center training programs. This should help us to consolidate our position in the industry as we move toward gaining a stronger foothold in critical SA roles while maintaining our solid, long-standing editorial position. As analysts start to increasingly use AI tools to generate notes, hours previously spent on the more mundane editorial tasks are eliminated, allowing our editors to take on more complex roles,
What Might Be Lost in Translation
Translation work, an important part of our language services, requires people with expertise both in a foreign language and in English. As most of the text is translated into English, there is a strong possibility that the final output contains native-language biases or are difficult to understand, as the writer is constantly having to think in one language and write in another. While AI tools can help us overcome this challenge by making reports more accurate grammatically, an editor-translator can identify the biases in global, cross-cultural research and add a unique flavor to each report tailored to the analyst’s writing style. Author-Analysts, too, can focus on data accuracy and adherence to regulatory guidelines without having to struggle to write in a language they aren’t comfortable with.
We have developed an in-house EVSGPT tool that along with summary generation also does some heavy lifting in translation. Since it uses AI models to perform these tasks, the output it produces is much better than traditional online applications such as Google Translator and Papago that often fail to understand the intended meaning where satire or pun has been used.
With Eyes Wide Open
Like most new things, there is excitement and fear around GenAI. We are all aware of the associated risks such as hallucination, bias, logical flaw, questions around ethics, copyright violations, plagiarism, data breach, misunderstanding context, and lack of checks and balances. But we can’t be spectators to this feverish unfolding of technological history even as regulation around GAI and its uses is gradually passed into law, focusing instead on GenAI’s complementary rewards – significant increases in efficiency, accuracy, and productivity.
This has all left me wondering how Felix would rewrite any old passage from Joyce’s Ulysses. <I shudder.> Let’s just be glad F is busy moping in some part of the Interweb right now ruing a missing letter in a word, which has dented his confidence slightly. He has been avoiding the SAs and editors for some time now, come to think of it.
Felix, you’re no patch on Messrs., Mmes., and Mses. Double-Helix…. Yet.
****For more terrible headers, please reach out to Felix.
[1] A Supervisory Analyst (SA) is an analyst who is (Series 16) qualified to approve research reports and has knowledge around rules and statutory provisions applicable to the preparation of those reports.
