AI Governance Framework for Regulated Financial Services

Resource By Shakudo September 11, 2026 9 min read

Two colleagues reviewing notes across a conference table in a glass-walled meeting room, seen through the glass partition

A credit union CIO has been told to “harness AI.” Staff are already using ChatGPT, Perplexity, and Claude at their desks to summarize loan documents, draft board minutes, and answer customer questions. The compliance office has a question: who knows what data went where, and what happens when NCUA asks?

This guide walks through the AI governance structure a regulated financial institution needs to answer that question, and the specific controls that produce the evidence an NCUA, state, or HIPAA review expects.

What an AI governance framework actually covers in a financial services context

AI governance is the set of operating controls that determine who can use which AI tools, what data they can reference, what the system logs, and what happens when something goes wrong. It is not a policy document; it is a running set of controls that produces evidence continuously.

In a regulated financial institution, the governance scope is defined by the data classes the institution handles. A credit union touches PII (account numbers, names, SSNs), sometimes PHI (for credit unions offering health benefits), and transaction data that NCUA and state examiners expect to see inside a defined boundary. The governance structure must reflect that: each data class has an access rule, an egress rule, and an audit requirement.

Control layer	What it governs	What the regulator sees
Identity and access	Who can use AI, which tools, on which data	Access logs tied to user roles; the minimum necessary standard applied per data class
Data boundary	Where regulated data can and cannot go	Network egress controls; the statement that PII/PHI does not leave the institution’s infrastructure
Audit trail	What the system did, when, and who triggered it	Immutable logs with user, role, model version, timestamp; retention matching the institution’s schedule
Model and change control	Which models are approved, how they are updated	Model inventory, approval records, update logs; the supply-chain evidence a reviewer expects
Incident path	What happens when AI produces a bad output or a data event	Documented incident response, notification timelines aligned to NCUA and state requirements

The five layers are not optional and they are not independent. A model change that is not logged is an audit gap. An access rule that is not enforced is a policy, not a control. The governance structure works when each layer produces evidence the others can reference.

How an AI governance framework maps to the NIST AI RMF

Most regulated teams already use the NIST AI Risk Management Framework as the vocabulary for AI risk, even before they operationalize it. The NIST AI RMF defines four functions, and a governed deployment produces the evidence for each one as part of normal operation.

NIST AI RMF function	What the framework asks	What the governed deployment produces
Govern	Who owns AI risk, what is the policy, is accountability assigned?	Named AI ownership, a written access policy, and role-based controls enforced at the gateway on every request
Map	What AI systems, data, and use cases exist, and what could go wrong?	A model inventory, the data classes each system touches, and the risk context for each use case
Measure	Is the risk being monitored, and can you prove the controls worked?	An immutable audit trail of prompts, access decisions, and model versions, retained to the institution’s schedule
Manage	When a risk materializes, what is the response path?	A documented incident procedure aligned to NCUA and state notification timelines, with the log as the starting evidence

The framework gives the compliance team the language to describe the program to a board or an examiner. The governed deployment is what makes each function true rather than aspirational, because each one produces evidence the others can reference. The AI risk management framework a credit union builds is the NIST AI RMF with a running control underneath it, not a slide deck.

The uncontrolled AI problem in financial services

Before a governed deployment, the pattern in most credit unions and mid-size financial institutions looks like this:

Staff use public AI tools for document summarization, email drafting, and research. Each prompt that includes a customer name, account number, or transaction detail sends that PII to a third-party provider under that provider’s terms.
The IT team has no visibility into what data left the environment, how much, or to which provider.
The compliance office has a risk register entry for AI that says “monitored,” with no underlying evidence.
When NCUA or a state regulator asks for the AI governance documentation, the team assembles it from memory, which is not the same as producing it.

The cost is not a single incident; it is the accumulated gap between the policy the institution says it has and the evidence the institution can actually show.

How a governed AI deployment closes the gap

A governed AI deployment replaces the “monitored” risk register entry with a running control. The architecture has three layers:

The interface layer. Staff interact with AI through an internal application, such as OpenWebUI, that looks and feels like the tools they already use. The interface handles prompts, conversation history, and document upload. It does not enforce the policy; it presents it.

The governance layer. The AI Gateway sits between the interface and every model. It authenticates the user, applies the role-based access rule, inspects the prompt for sensitive data patterns, routes the request to the approved model, and logs the full interaction. It blocks egress outside the institution’s boundary. This is where the evidence is produced.

The model layer. The models run inside the institution’s infrastructure. They are open source, closed source, or a mix, selected by the institution’s use case and approved through the model change control process. The models do not call home; they do not send telemetry to a vendor. The institution controls the model lifecycle.

The three layers are independent. The institution can change the interface, add models, or update the access policy without rebuilding the others. The governance layer is the constant, and it is the layer a regulator asks about first.

What NCUA and state regulators look for in an AI governance program

NCUA does not have a standalone AI regulation. Exam expectations derive from the Risk Management program standards, the 12 Elements of a Sound Internal Controls System, and the information security standards applied to AI use. The specific evidence an examiner looks for:

AI in the risk register. The system has a named owner, a risk rating, and a documented review cycle. It is not a footnote; it is a line item.
Access control documentation. Role definitions, the data classes each role can access, and the access logs that prove the rule was enforced. The logs should cover the examiner’s lookback period, which for NCUA is typically seven years for transaction records.
Data boundary statement. A clear, testable statement that regulated data does not leave the institution’s infrastructure, supported by the network egress controls that make the statement true.
Audit trail retention. Immutable logs that cover prompts, retrievals, and generations, with user identity, role, model version, and timestamp. The retention schedule should match the institution’s records retention policy.
Model and vendor due diligence. For each model in use: where it came from, what it was trained on, what license governs it, and the approval record for its deployment. For closed-source models, the vendor agreement and the data handling terms.
Incident response path. A documented procedure for AI-specific incidents: a model produces an incorrect output that reaches a customer, a prompt contains PII that should have been blocked, a model update introduces a regression. The procedure should specify who is notified, within what timeframe, and what the escalation path is.

The examiner is not looking for perfection. They are looking for a system that produces the evidence continuously, so that on exam day the institution is showing them what already exists rather than assembling it under time pressure.

PII and PHI in an AI Governance Framework

Credit unions and financial services institutions handle PII in every customer interaction. Some also handle PHI when they administer health benefits or partner with a health services provider. The AI governance structure must distinguish between the two because the access rules are different.

PII access is broad: most staff roles need to reference customer data to do their job. The governance control is role-scoped, so a loan officer can access loan documents but not the board minutes. PHI access is narrower: the minimum necessary standard means a role that needs to summarize a claims document does not get the full patient record. The AI Gateway enforces this by inspecting the data class in each request and applying the access rule before the request reaches the model.

For a credit union that handles PHI, the governance structure also requires a business associate agreement with any vendor that touches PHI, including the AI vendor. A governed deployment where the models run inside the institution’s infrastructure simplifies this: there is no external vendor in the data path, so the BAA chain is shorter and the data boundary is the institution’s own network.

How governance maps to the NCUA exam cycle

NCUA examines credit unions on a cycle that depends on size and risk rating: every 36 months for smaller institutions, every 12 to 18 months for larger or higher-risk ones. Between exams, the relationship officer may request updates on significant technology changes.

The practical implication: AI governance documentation should be current at all times. An institution that can show a six-month audit trail, not a one-week pre-exam compilation, is in a fundamentally different position in a supervisory conversation. The governed deployment produces the trail as part of normal operation, so the documentation is always current.

A practical starting point for an institution building its AI governance program is the AI governance frameworks overview, which maps the major frameworks to specific control requirements. For the deployment architecture side, the regulated AI guide covers the boundary models and evidence requirements in more depth.

What to ask a vendor about AI governance

Five questions that separate a governance platform from a model hosting service:

Where does every AI request go, including telemetry and model updates? Require the network diagram and a written zero-egress statement for regulated data.
What is in the audit log? User identity, role, prompt, response, model version, timestamp, and the access decision. Not a summary; the full record.
How is the access policy enforced? At the gateway layer, on every request, with the decision logged. Not at the application layer, where a bypass is possible.
What happens when a model is updated? Is the update logged? Is there an approval step? Can the institution roll back? The answer should be yes to all three.
What does the institution own? The data, the logs, the access policy, and the model weights should all be under the institution’s control. A vendor that retains ownership of any of these is a data dependency, not a governance partner.

Bottom line

AI governance in a regulated financial institution is not a document; it is a running control that produces evidence continuously. The institution that can show a regulator six months of audit trail, access decisions, and data boundary enforcement, produced as part of normal operation, is in a fundamentally different position than the one that assembled the evidence in the three weeks before the exam. The architecture that produces that evidence is the one where the governance layer sits between the staff and the models, inside the institution’s own infrastructure, and logs every interaction it processes.

A working demonstration of the governed AI architecture is available at Shakudo’s demo.

Last verified: 2026-09-11

Why Regulated Financial Services Compliance and IT Leaders Choose Shakudo to power their governed AI deployments

Single identity and access layer for all AI tools

SSO and role-based access control apply to every model, every agent, and every data source in one policy, so staff see the right tools and nothing more. The access evidence an NCUA or state exam expects is produced on every request, not assembled for a review.
Audit trails that survive a supervisory exam

Every prompt, retrieval, and generation is logged with user identity, role, model version, and timestamp. The log is immutable and retained to the retention schedule the institution's risk function sets, which is the evidence pattern NCUA examiners and state regulators look for.
PII and PHI stay inside the compliance boundary

The AI Gateway inspects every AI request and response for sensitive data patterns. Customer account data, health records, and other protected classes are blocked or redacted before they reach a model, and the model runs inside the institution's own infrastructure so no regulated data egresses.
No vendor lock-in, no data dependency on a third party

Orchestration is model-agnostic, so the institution can switch between open and closed source models as regulations and capabilities evolve. The governance layer, the audit trail, and the access policy remain constant across model changes, which is what a regulator asks about first.

Talk to us

Frequently asked questions

What does AI governance mean for a credit union?

It means the credit union can show an NCUA examiner or state regulator that AI is deployed under a defined risk framework, with named ownership, role-based access, data boundary controls, and an immutable audit trail. The evidence is produced as part of normal operation, not assembled for a specific exam cycle.

How do NCUA examiners evaluate AI deployments?

NCUA examiners review AI under the existing risk management, information security, and operational risk standards. They look for the AI system in the risk register, access control documentation, data boundary evidence, and incident response procedures. The relationship-based exam model means documentation quality matters as much as its existence.

Can staff use consumer AI tools like ChatGPT for work?

Consumer AI tools send data to a third-party provider under that provider's terms, which is a data egress event for regulated institutions. A governed AI deployment gives staff the same convenience through an internal interface, but routes all traffic through a gateway that enforces the data boundary and records every interaction.

What is the difference between AI governance and AI risk management?

AI risk management identifies and scores the risks an AI deployment introduces, which is what the NIST AI RMF and the AI risk management framework are built to do. AI governance is the operating layer that enforces the controls: identity, access, data boundaries, audit logging, and model change management. A credit union needs both, and the governance layer is what produces the evidence an examiner needs to see.

How does a governed AI deployment satisfy HIPAA minimum necessary?

Role-based access control limits each user to the data class and fields their task requires. A compliance officer who needs to summarize a claim does not have access to the full patient record. The AI Gateway enforces this restriction on every request and logs which role accessed which data class, which is the minimum necessary evidence HIPAA requires.