Chris Hillman, International Data Science Director at data management company Teradata, has recently observed an increased focus on the costs of data science and AI teams as companies seek to demonstrate the benefits of their investments in new technologies.
However, he believes that data scientists are capable of building AI models at a technical level. And it is often the business partners who hinder the success of AI projects because they do not understand how AI models work or are unable to put the model recommendations into action.
“In the world of data science, everything is a technical problem and we solve it with technology,” Hillman explained. “But I firmly believe that many of the reasons these things don’t get into business processes are fundamentally a cultural, political or human problem – not a technical problem.”
Teradata’s experience building models for a number of international customers suggests that:
- To successfully advance and achieve projects, managers must be familiar with AI.
- Managers learn better from practical examples than from “Data Science 101” courses.
- Companies should conduct impact assessments before starting AI projects.
Culture, politics and people: hurdles to the success of AI projects
Hillman argues that the failure of AI projects is often due to economic interests:
- Distrust of the results of the AI model because they were not part of the process.
- It is not possible to translate model results into real processes and actions.
As long as the data is made available to a data science and AI team, Hillman said, the AI problem is not a technical one. Rather, difficulties often arise when business partners understand this technology and translate the AI results into business actions.
Managers should be involved in the AI development process
As long as the data is available, Hillman’s team can successfully train, test and evaluate AI models.
“We write the output of that model somewhere, and that’s the job done,” he said. “Production is running the model every month and putting something into a spreadsheet somewhere.”
However, this is exactly where an error can occur.
“It fails because the entrepreneurs have to be involved in the process,” Hillman added. “They have to take that score and decide, ‘What is the signal?’ If I say something has a 90% chance of being fraud, what does that actually mean?
SEE: Evidence of Australian innovation in the pursuit of generative AI
“If the signal is to block the payment and they decide to do that, someone has to do it. In many companies, that means at least three, if not four, teams have to be involved: the data engineers and data scientists, the business owners and the application developers.”
This can become a dysfunctional process where teams do not communicate effectively, AI has no impact on business processes, and AI does not create the desired value.
Entrepreneurs need to understand how AI models work
The rise of AI means all business leaders need to know how these models are built and work, Hillman said.
“They should understand the outcome because they have to manage the process,” he explained. “They are the ones who have to ask, ‘What does this mean for my customer or my business processes?'”
While technical knowledge of the algorithms is not essential, leaders should understand the basic mathematics of AI, such as the probability theory of AI models. Business partners need to understand why the accuracy of AI models deviates from expectations of traditional business intelligence reporting tools.
“If I went to the finance director with a report and he asked me how accurate it was and I said, ‘About 78% accurate,’ I would probably be thrown out,” Hillman said. “But if an AI model is 78% accurate, that’s good. If it’s more than 50% accurate, you’ve already won.”
“Some of our customers have made requirements and said, ‘We want this model and we want 100 percent accuracy with no false positives.’ And we had to tell them, ‘We can’t do that because it’s impossible.’ And if you get a model like that, you’ve done something wrong.”
Use cases: Effective tools for training managers in AI models
Hillman doesn’t believe business owners should take courses in “Data Science 101” that may be “useless” to them in practice. Instead, he said AI use cases can be used to demonstrate how AI models work much more effectively for business people.
“I think the use case-driven approach is definitely better for people on the business side because they can identify with it and then you can join the discussion,” he said.
Tips to make sure your AI project really runs
Hillman offered several recommendations for entrepreneurs to ensure their AI projects make the journey from idea and proof of concept to production:
Conduct an impact assessment
An impact assessment should be conducted beforehand. This assessment should include important considerations, such as why the company is pursuing the AI project and what business benefits it will bring.
“I very rarely see something like this in the original specifications,” Hillman noted.
Rather, impact assessment is often started when a project is already underway or after the technical work is completed. This can lead to projects being put on hold and not moving into production.
Choose the right use cases
Although Transformer models were gaining popularity before ChatGPT, the hype surrounding OpenAI’s introduction of the chatbot led to companies starting generative AI projects to stay relevant. This has led to some use case choices that may be incorrect.
SEE: 9 innovative use cases of AI in Australian businesses in 2024
Hillman often asks companies if they can “build a report instead,” as there are usually easier ways to achieve business goals than building an AI model. He said AI models are typically not adopted because an impact assessment is missing or the use case is wrong.
Do you have a strong corporate sponsor
AI projects are more successful when driven by a strong business sponsor. A business champion can ensure that the potential impact of an AI project is understood by other teams in the company and that they work together to implement AI data into processes.
“The IT department may have the budget for the technology and someone else may be in charge of the data and the security and privacy aspect, but the real impetus always has to come from the business side,” Hillman said.
