A slew of surveys conducted by various research organizations have indicated that nearly three out of four Artificial Intelligence projects fail to deliver the expected business value. While details about the projects used in the surveys are unclear, it is intriguingly still a high failure rate.
So what might be some possible reasons why AI projects fail? First, it is important to recognize that AI projects do not fully fit the mold of traditional software development lifecycle methodologies or Saas implementation projects, that many organizations are now well versed with. There are unique technical challenges to building an AI solution itself - such as data sourcing, model selection and technical evaluation. But just technical aspects of building an AI solution are not the whole story. The following are a few key reasons, which if not properly accounted for, can significantly contribute towards failure of AI projects.
The importance of narrowly focused and well-defined projects and an understanding of AI technology limitations
Vision and success factors need to be clearly defined for any project. The prominent reason cited (IDC Survey 2023) by organizations as a reason for failure of AI projects is that AI technologies did not perform as expected or promised. This was traced to poorly defined goals and hyped expectations of the technology. Organizations need to clearly define the business objective or process that AI can support, which includes realistic improvement targets or performance metrics. These can only come from a clear understanding of the general limitations of AI technology, whose performance can greatly vary as a result or combination of many factors such as training data quality, selected machine learning model characteristics or even how the model is prompted.
Root cause analysis of problems within AI solutions is generally far more difficult and tedious to perform as compared to traditional software solutions. As a result, AI projects carry higher return on investment risks, especially those focused on those influencing KPIs (increasing topline, decreasing operations cost, increasing customer experience, etc.). Therefore organizations should aim first for narrowly focused projects with well defined goals that are able to demonstrate meaningful outcomes, while minimizing risks.
Need for Data Strategy and Data Preparedness
Timely access to quality data in sufficient quantities is a crucial requirement for any AI based solution to function effectively. Surveys have shown that data gathering and preparation is the single most time consuming and effort intensive component of an AI project. Having a data strategy allows organizations to harness the power of data efficiently and effectively by implementing a cohesive plan to collect, prepare and manage data from their various business areas on an ongoing basis. It also allows project teams to have access to relevant and accurate organizational wide data in a timely and efficient manner, while ensuring regulatory and legal compliance at all levels.
For organizations lacking an overall organizational data strategy, acquisition of necessary data and its preparation for ongoing use in an AI solution can become a significant challenge for project teams. A data strategy that falls on the project team to manage can result in increased complexity of tasks to acquire, cleanse and govern the data, which in turn can lead to a high risk of delivery delay, or possibly even long term solution failure.
Planning and Investment in Infrastructure Resources
Infrastructure architecture remains a significant challenge for AI projects moving from concept to production. Unlike experimental AI projects to demonstrate feasibility, business support systems require stable yet long term flexible end to end environments for development, test and production. AI solutions can be compute intensive. Infrastructure selection decisions have a significant impact on data privacy and security considerations. Environments may need to be managed across multiple geographies for a global solution rollout, each with its own local rules and regulations for data privacy and security.
While there are now many infrastructure providers in the market, there is no one size fits all. There are many considerations that need to be carefully evaluated by the project team for infrastructure planning and investment. For example, should the infrastructure be on premise, off-premise (cloud) or some combination of both? Which infrastructure setup allows flexibility to connect new data sources, allows swapping machine learning models easily and maintains security and data privacy, while minimizing maintenance costs? How will the infrastructure scale as the solution grows? How will the AI solution be maintained in production so that it can easily adapt to the constantly changing needs of the business? What are the skills and resources required to maintain the infrastructure on an ongoing basis, and how easily can they be acquired? All of these require careful evaluation based on the goals and constraints of the project to reduce the risk of failure.
Focus on Change Management
With the advent of ChatGPT, there has been a lot of awareness of AI and its potential in many respects. As with any new technology, there is always a “build it and they will come” mentality. While AI has had impressionable success as a consumer product / service, the same excitement has not been forthcoming from end users in organizations. As per a recent Gallup poll, approximately 3 in 4 of those surveyed believe that AI will reduce jobs and they lack trust that businesses will use AI responsibly.
The potential of AI to ultimately replace workers in the critical business functions can push off users from using AI based systems as much as possible. AI systems also have a less than perfect track record of providing consistently accurate results, with most machine learning systems currently lacking transparency of how they arrive at those outputs. Without trust, users will be reluctant to adopt AI systems as part of their critical decision making process. Thus, a balance of transparency and trust needs to be achieved with the end users who will be expected to use these systems. The project team members should plan to involve users as early in developing the AI solution as much as possible to build their trust in the system, and incorporate education about AI in training sessions. Organizations should also draft up principles or guidelines of how AI systems will be incorporated as part of the business to further build trust and achieve a successful adoption rate among users.
Final Notes
AI is still a maturing technology with a significant potential to enhance business workflows and decision making for businesses. It is important for sponsors and project teams to remain cognizant of the unique characteristics of AI solutions when compared to standard custom developed software or Saas solutions, so these potentials are realized effectively. While there are numerous other reasons that can contribute to the failure of AI projects, planning to address some of these key areas early on can help reduce risks and lead to increased chances of success.
We can help with AI project planning and management. Contact Us to learn more.