AI Oversight: Nick Curum Identifies Four Key Questions for Boards

In a recent LinkedIn post, Nick Curum addresses the critical issue of artificial intelligence (AI) oversight within organizations, posing four fundamental questions that boards should be able to answer to ensure genuine governance. Curum emphasizes that while many organizations can address the initial questions about AI’s current role and decision-making power, a significant gap often emerges when probing the accountability structures.

Curum highlights the common challenges in establishing clear lines of responsibility, stating:

“What is AI allowed to decide today? Where is it influencing decisions right now? Who reviews a decision before it acts? What controls the output itself?”

The author points out that while organizations may have answers for what AI is permitted to do and how its output is controlled, the crucial areas of current influence and pre-action review often lead to hesitation.

The ‘Pause’ in AI Accountability

According to Nick Curum, this hesitation, or ‘pause,’ is a significant indicator of a potential lack of robust AI oversight. He cites survey data to underscore the widespread use of unapproved AI tools by senior employees.

Curum references a survey by Cybernews, which found that:

“93% of those at executive or senior-manager level said they use AI tools their employer has not approved.”

This statistic, Curum argues, is particularly concerning as it involves individuals in positions of leadership, not just entry-level staff. He implies that the responsibility for managing and approving these tools should ideally rest with those who are chairing meetings and making key decisions.

The Peril of Split Accountability

Further complicating the issue of AI governance, Curum discusses the diffusion of accountability. He references a survey conducted by the Bank of England and the FCA among UK financial firms, noting that:

“Most name three or more accountable persons or bodies for AI.”

While this indicates that accountability is not entirely absent, Curum argues that it is often excessively split. In his view, when accountability is distributed among multiple parties, it can lead to a situation where no single entity feels fully responsible, effectively meaning that no one is truly signing off on AI-driven decisions or their implications.

A Framework for AI Governance

Nick Curum proposes a visual framework that maps AI oversight through four distinct layers, starting from governance at the outermost layer and moving to the core system. Each layer, he explains, should have a designated owner, a specific control mechanism, and a clear signal to indicate failure. This structured approach, Curum suggests, is essential for effective risk management in the age of AI.

He encourages leaders to consider this framework before their next risk committee meeting, implying that clear ownership and control at each stage are vital to prevent the ‘decision nobody signs off’ scenario.

Curum’s insights provide a practical lens through which organizations can assess and strengthen their AI governance, ensuring that the rapid adoption of AI technologies does not outpace the necessary controls and accountability measures.