Having devoted the last 1.5 years to the domain of AI/ML Model Governance in a large multinational organisation, I would like to reflect on some key lessons I learned.

The list below is not meant to be exhaustive, but summarises some key points worth considering when establishing and/or supporting an enterprise-wide Model Governance program in a larger organisation.

  • Having AI Ethics Principles is only the first step. The most substantial phase of effort commences when operationalising those principles, ensuring their seamless integration into the organization’s daily processes. There is no one-size-fits-all solution when doing so, as a thorough understanding of the organisation (incl. dynamics, organizational structure, inscentives) is required for effective implementation. Ultimately, it’s the widespread adoption of these principles that makes a real difference.
  • Focus on change management. Implementing Model Governance within an organization necessitates a comprehensive approach to change management to ensure its success. Expectations that come with Model Governance will impact the day-to-day work of model developing, as well as model consuming teams. Some initial resistance and hesitation is expected, due to the implied adjustment of accustomed worthing rhythms. It can take years for a successful Model Governance program to be rolled out, and most importantly, adopted enterprise-wide. It is paramount to integrate change management into your strategic planning, and dedicate time to enhance the change management-related skills of the teams responsible for spearheading this transformative endeavor.
  • Focus on the value-add. Effective communication is key when it comes to establishing Model Governance within an organisation, particularly during its initial stages. Focus on sharing stories that showcase the value-add, and potential risks in case of non-adherence to the guidance provided. Allocating a dedicated resource to compelling storytelling as the program gets rolled out pays off.
  • Involve impacted stakeholders early on. Actively listen to your stakeholders, take their feedback into consideration, and iterate based on their feedback. Dedicate effort to comprehending existing ways of working and establish a user-centered approach when rolling out new processes. This should involve technical as well as non-technical end users. Defining shared OKRs ensures that everyone is working towards a common goal, turning Model Governance into a truly shared responsibility, instead of relying on a single centralised team to be successful.
  • Leadership buy-in. Having leadership buy-in is the foundation for the successful roll-out of Model Governance across the entire organisation. While support from practitioners is undeniably important, its impact can be constrained if the inscentives higher up are not aligned. Some degree of top-down pressure also ensures that processes are followed despite competing priorities.
  • Invest in organisation-wide upskilling. Topics like Model Governance and AI Ethics are new to many organisations, and have the potential to ultimately impact a wide range of job families within your organisation. Devote efforts to enhancing the collective skill set of your organization in these domains, focussing on value-add and how it impacts their individual roles. Make sure the training material is aligned to the unique characteristics of your organisation, taking model landscape, products, and data sources into account. This will build the foundation for additional use case specific advisory, should the need arise.
  • Focus on flexibility. As previously highlighted, the efficacy of any Model Governance program hinges on its widespread adoption. To increase adoption, it is important to show model developing and consuming teams how to embed Model Governance processes in their existing workflows. Embrace a hands-on approach, as long as there is a clear distinction of responsibilities. Essential to this approach is flexibility to give your stakeholders freedom to seamlessly embed the new requirements into their established ways of working. As the program matures, focus on automating processes to further streamline the work.
  • Start small, slowly build up over time. Consistency in adoption and advisory is important, which can only be ensured if you start small, and slowly increase the Model Governance expectations over time, as the organisation matures. Fostering this evolution demands a vision of where you’re hoping to be headed based on regulatory trends, customer demands, and more. Make this vision come true through a phased approached, ensuring that your aspirations materialize into tangible outcomes.
  • Stay up-to-date with the ever-evolving field of AI. As the field of AI evolves, exemplified by recent surges such as Generative AI, make sure governance-related teams have dedicated time for continuous upskilling. Additional guidance might be required to ensure existing Model Governance expectations work well with the latest trends.

Note that the views expressed here are solely my own and do not necessarily reflect the views of my current employer.

Banner photo from Steve Johnson on Unsplash