Generative AI in Banking: Today, Tomorrow, and Beyond

There are few things hotter than AI today. Not only in banking but across industries, everyone is talking about how they can use AI. And much of the conversation is focused on one very specific subset of AI: generative AI, which catapulted into public awareness with the release of ChatGPT in 2022. 1 Since then, generative AI has found its way into companies and homes around the world, including through Microsoft’s Copilot, embedded in Microsoft 365. It has ignited conversations around the future of AI and the future of work. But questions remain about how to effectively leverage this technology, and the risks involved, especially in complex areas like financial services.

This report takes a close look at the generative AI opportunity in banking. It examines what this technology is and isn’t, how it is being used already, and how it may be used in the near and longer term. The goal is to provide a baseline for financial services executives who want to understand generative AI and how it is poised to impact the industry.

(Note: The following material is written with community institutions in mind and is not concerned with activities of the largest banks.)

A primer on AI

AI is not new. It also isn’t one technology. One type of AI, machine learning, has been around for a long time and is already used in banking across areas like anti-money laundering (AML), fraud detection and prevention, underwriting, analytics, biometrics, and automated document processing, for example. 2 What we are experiencing is a massive technological leap forward in the field of AI, driven in large part by the new frontier of generative AI. That, in turn, is leading to a surge of enthusiasm for AI technologies overall.

Our report is focused on generative AI specifically, but to understand this subset of AI, it is important to first acknowledge that there is a wider umbrella under which it falls. To the extent that it is useful, here is the definition of AI laid out in the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence: 3

“…a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. Artificial intelligence systems use machine- and human-based inputs to perceive real and virtual environments; abstract such perceptions into models through analysis in an automated manner; and use model inference to formulate options for information or action.”

As you can see, the definition is fairly broad. This is because AI is a wide field of study and encompasses a wide range of technologies. The takeaway, for our purposes, is that generative AI is one piece of a much larger pie. It is a revolutionary and exciting slice, but it is only one slice. And it will not be a catch-all for every problem inside of a financial institution (FI). As Maya Mikhailov, CEO and founder of SAVVI AI, explained in an interview, AI is a toolkit, and those tools can be applied in different ways to different problems based on what kind of solution is needed. Generative AI is one tool in that toolkit.

The generative AI opportunity

Generative AI — also known as gen AI — is primarily useful for analyzing and creating content. The key word when it comes to assessing what this technology can do is generative — it can quickly generate outputs of many formats from data inputs. It does this using large language models (LLMs), which are models that can analyze and generate human language. 4 Based on the question you ask it and the parameters you set, generative AI will give you a response formulated from the data it has ingested. Examples of LLMs include OpenAI’s GPT-4, Meta’s Llama, Google’s PaLM, and Anthropic’s Claude. McKinsey estimates the annual potential value of generative AI to the banking industry to be $200 billion to $340 billion, with a majority coming from increased productivity. 5

Most of that is likely to be concentrated among larger banks: Big banks have a massive advantage in deploying generative AI, not only because of their available resources but also their broader AI experience and foundations, and their ability to attract and develop talent, according to Theodora Lau, founder at Unconventional Ventures. As a result, she said, smaller FIs risk a wider delta with these larger competitors if they do not act.

One major problem, though, is that generative AI is not always accurate. In fact, research shows that it “hallucinates,” or draws inaccurate conclusions, between 1% and 30% of the time. 6 In particular, this makes it risky for FIs to leverage the technology for what is arguably its most obvious use case: customer service. In recent years, companies across industries have been experimenting with generative AI-powered chatbots that can reduce the number of customer requests needing to be handled by employees by providing a near-human-like conversational experience. 7 After all, ChatGPT itself is a chatbot. But generative AI’s current limitations create a major barrier for highly regulated industries like banking.

As a result, most of the use cases we are seeing currently are internal or maintain a human in the loop. Over time, as the technology matures, we expect this to shift. For now, though, institutions we talked with are taking the opportunity to lay foundations for generative AI within their organizations in a secure way that promotes the use of the technology, fosters education among employees, and prepares for its evolution.

Real-world applications today

Absent any defined AI regulatory framework, FIs are largely left to figure out how to navigate this new frontier without running afoul of existing regulations, especially in areas like fair lending and unfair, deceptive, or abusive acts or practices (UDAAP). Because of this, while there are a few exceptions,8 current use of the technology appears centered primarily on improving processes and helping organizations to run more efficiently. Here, we’ve included a limited number of cases where the technology is used in external applications but not to converse directly with customers or make any recommendations.

Use cases we’ve identified include:

Knowledge management

Knowledge management is the most common use case we’re seeing currently. This refers to the ability to use generative AI to create a “supercharged intranet” that helps employees better surface information and do their jobs more quickly and efficiently. In many cases, this looks like an internal chatbot. Microsoft’s Copilot, for example, can perform this function. However, a few more forward-thinking institutions are building on this capability to create bespoke chatbots or internal tools. This can be done in conjunction with existing partners, for instance, like Microsoft, or through purpose-built solutions.

SouthState Bank, for example, operates an internal chatbot, called Tate, that is built on ChatGPT in Microsoft’s Azure but references the bank’s own data. It has its own pre- and post-processing requirements that control how a person can query the model and how it is allowed to respond. Tate can reduce searches that used to take 10-12 minutes to 10 seconds, according to Chris Nichols, director of capital markets at SouthState Bank.

Another example is Maverick Bank, which is working with Hapax, a platform built to combine an FI’s data with external sources specific to financial services, such as the CBANC network. Brent Leslie, CEO at Maverick Bank, told CCG Catalyst that it makes more sense to spend time training a model than to spend time training people. “That way, you only have to train people to interact with a model, rather than training them on individual policies and procedures,” he said. The 60-employee bank, with about $300 million in assets, has 10-15 “superusers” piloting the platform.

The most obvious factor to consider here (and generally) is data security, because any model is only as good as the data it’s fed, and an FI needs to feel comfortable contributing as much useful data as possible to generate the best possible results. That data can include everything from policies and procedures to meeting notes. One way to handle data security is through a layer that prevents data ingested from training the larger model. So, for instance, no data that SouthState Bank uses to train Tate is used to train GPT-4; that is a function of its contract with Microsoft. Another option is to host everything separately — for example, according to Kevin Green, COO at Hapax, each instance of Hapax is deployed on its own single server for the FI to ensure all data is confined to the FI’s environment.

Another important consideration is education. Employees need to be trained on interacting with the technology, its limitations (i.e., hallucinations), and its value. Generally, answers to employee queries will come with reference data attached, but employees should know they need to verify information that they are being provided, and how to do that.

Content development

Content development is the bread and butter of generative AI. Its use in sales and marketing is a no-brainer, across industries. 9 However, there are a few other interesting ways the technology is being leveraged to create content in banking.

For instance, Citizens First Bank is using generative AI to write procedures. According to Heather Farwell, SVP of financial services at Citizens First Bank, it would usually take an employee 1-2 hours to go through and do screenshots, write procedures, go back over them, and get to a place where someone else could review. Just to do one procedure. That’s been reduced to 10-20 minutes. Now, the bank is expanding use of the technology to areas like summarizing reports, and it plans to create an AI steering committee to help identify other ways it can be leveraged within the institution.

Additionally, we identified the use of generative AI in learning and development initiatives. For example, LemonadeLXP is a learning platform that is leveraging the technology to help employees create training programs and courses as well as educational content for customers. Per Hawley Kane, director of product marketing at LemonadeLXP, the platform will look across all the content it is fed by the FI to create learning material, which is then approved by an employee. If one of the source files is altered for any given piece of content, the platform will flag the change and ask if you’d like to regenerate the content. On the employee side, it offers a chatbot that can be used to surface information.

Generative AI can also be used to create synthetic data or code, though FIs considering this application appear to be still largely in the exploratory phases.

Customer service (with guardrails)

This is the big one — and, for now, we would call this an internal use case for reasons mentioned earlier on in this report, namely, hallucinations. FIs are beginning to embrace generative AI for customer service, but we’re mostly seeing efforts to help improve the way employees interact with customers and to improve workflows.

The clearest example of this builds on the knowledge management use case to empower frontline employees. Customer interaction platform Glia, for instance, offers an AI assistant (whose responses can be built using generative AI but are pre-approved and pre-loaded by the FI) as well as a product called Agent Assist, which provides a summary of virtual assistant conversations to an agent and can surface helpful content in real time. The company also has a solution that autocompletes agents’ post-call work.

A slightly bolder example would be Casca, which is a loan origination platform that offers an AI-powered assistant, called Sarah. Sarah is designed to interact with customers throughout the loan application process to minimize drop-off. It can follow up when an applicant churns and answer questions to help them complete their application. Critically, all of Sarah’s messages are approved first by a human. Casca’s team or FI employees review and approve its messages 24/7. Additionally, the platform leverages generative AI on the underwriter side to analyze documents, compare them against internal policy, and help to inform their decision. According to Lukas Haffer, cofounder and CEO at Casca, one client saw its conversion rate increase from 8% to 81%.

Interface AI provides similar technology for the call center. As Srinivas Njay, founder and CEO at Interface AI, explained, the company is currently using generative AI to help virtual assistants understand queries as well as to respond, with its proprietary human in the loop technology. It can approve responses in as little as 3 seconds, he said.

Gen AI has tremendous potential in customer service, but it is still very early days. Even those building in the space agree that the tech is not quite ready for prime time yet. As Jake Tyler, product marketing, Glia Virtual Assistants at Glia, explained, “While there is unlikely to be an interaction on our platform not touched by our AI layer, generative AI is never interacting with customers directly because it will, at some point, hallucinate.”

The possibilities of tomorrow

We’re at the beginning of the beginning. In a nutshell, FIs are starting to move from talking about gen AI to carefully controlled, employee-led use cases. A next step is to allow the technology to provide information to customers directly. For the reasons we’ve outlined above, this will likely happen slowly for most FIs, and in phases. Per Maverick Bank’s Leslie, “If we were to allow generative AI to talk to a customer, it would only pull from certain responses or from certain content. And it will probably be that way until people grow more comfortable with generative AI. The last thing you want is something to go rogue. As a society, we need to get more comfortable.”

Based on our research, that last part is very important. Because, even as the technology matures, we still may not see its accuracy grow suitably infallible. What is likely to happen instead (or at least first) is that we, as a society, will grow more comfortable with the technology. Once that happens, combined with greater levels of accuracy, it will be easier to deploy with customers.

Beyond the use of gen AI to provide information to customers are use cases that involve it providing advice or taking action. These are much further down the line and bleed into an area called agentic AI, 10 which is beyond the scope of this report. It is the arrival of this frontier that will bring additional capabilities to the fore, such as the ability to improve onboarding processes and underwriting/access to credit. This will require serious consideration and risk management, especially around bias and explainability.

Strategic considerations

As FIs contemplate gen AI, there are a few important considerations to keep in mind:

• Start with the problem, not the solution. Looking for use cases for generative AI, or any technology, is one of the biggest mistakes you can make, according to Mikhailov at SAVVI AI. Instead, institutions should be looking at the problems they are trying to solve and making sure they understand the tools out there. That way, they will be able to apply the right tools to the right problems when relevant. Any use of technology should be in service to tangible business goals.
• Data security and organization. While there are ways to leverage generative AI (and other tools) without a data strategy early on, it is best to get your data in order to reap the greatest benefits. Additionally, as mentioned earlier, it is important to have a good understanding of what data you are feeding into any model, and to create a barrier between the organization’s environment and the larger LLM.



• Organizing in a way that allows AI to proliferate. This is not specific to generative AI; it applies to AI broadly, and it is about scale. One way we’ve seen FIs drive proliferation is by using pilot employees to train others. These employees are armed with knowledge, and used effectively, they can act as envoys for the technology throughout the FI. Another is to create a center of excellence or steering committee that is tasked with centralizing and championing AI efforts across the organization.
• Implications of potential regulation. Regulation is a major unknown. This is not at all specific to banking. The way the latest frontiers of AI are going to be regulated in the US and globally is still very much unclear. As a result, keeping on top of changes, communicating with regulators, and moving slowly and deliberately is key. For now, the most important thing is going to be staying in compliance with current regs.

Generative AI is very new, and very risky. But it is also more accessible than one might imagine. A key finding of this research is that some AI developers are keenly aware of the risks, and they are putting guardrails in place to mitigate those risks, with an emphasis on measured approaches to deployment. Pretty much every expert we spoke with had similar thinking: one step at a time, not only for users (FIs in this case) but also for builders.

Additionally, FIs with no experience with AI at all have the option to start with more established technologies and work their way up. This should be strategy dependent, as noted above, but getting foundations in place may be helpful when it makes sense. For example, Great Lakes Credit Union has a customer service virtual assistant, called Olive, that is built on a traditional AI subset called natural language processing (NLP), and it is now beginning to enable gen AI capabilities to improve Olive’s ability to understand queries. (All responses are still static.) As Elizabeth Osborne, the credit union’s COO explained, during business hours, 60-65% of calls are fully serviced by Olive, and outside of business hours, that’s 70-75%. The initial development of Olive, built by Interface AI, came as a result of Osborne’s mandate to improve contact center operations.

Ultimately, no matter the FI or where they are in their journey, the best thing leaders can do is to get educated. That might mean getting comfortable engaging with ChatGPT or talking with peers and experts, but the goal is to understand what the technology is and what it can do. The stage of evolution of AI we’re in is likely to be monumental for society. This isn’t embedded finance, or crypto, or another buzzy corner of fintech. It represents an industry-agnostic technological leap forward, one that everyone needs to be prepared for. Success will require growing fluent in a new field — just as we did with Google or the internet itself.

©CCG Catalyst 2025 – All Rights Reserved
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording or any information storage and retrieval system, without prior permission in writing from the publisher.

Read AI in Banking: Accessible Applications and Use Cases

Download a PDF of this article

 

Back