This three-part series of blogs will reflect on three major talking points from the September Big Data London conference.
The main grumble for many AI platform vendors at the conference was why their customers’ AI and ML projects aren’t making it into production. Although many organisations are starting to do some innovation leveraging AI, very few projects make it into production. AI platform vendors are seeing their technology deployed but, in some customer cases, only leveraged in relatively limited ways and not at scale.
Vendors report that a typical “good” customer will complete and put into production between 1% and 5% of the models they develop — suggesting tolerance of project failure needs to be relatively high when developing and implementing AI. However, there is a relatively large number of customers that are installing AI platforms and failing to get any models into production — a big concern for AI platform vendors.
Vendors are highly incentivised to see customers put models into production. From a revenue perspective, they can provide further implementation services, license infrastructure for inferencing or even license the IP for models themselves. Also, if a team with access to a platform can convert its efforts into a meaningful use case, then it can serve as an excellent advertisement to the rest of the business.
The most common issues cited by vendors is that IT remains hugely segmented from the wider organisation and not incentivised to support lines of business (LOBs) in productionising AI at scale. Unless organisations have a chief data officer in place, reporting directly to the CEO and tasked with driving AI and ML capabilities through the business, then it can often be incredibly challenging for LOB teams to get IT to manage the introduction of AI-enabled use cases.
Organisations need to wake up to the fact that if they want to see AI lead innovation, they need to ensure buy-in across the business, especially in IT.
Vendors report the difference between organisations that can successfully productionise AI and those that struggle to make it past proof-of-concept projects often comes down to a small change in internal management structure. Appointing an executive leader to fight the corner of the business’ AI proponents can be crucial to swinging the balance back in favour of delivering AI-related innovation.
Nearly all AI platform vendors report issues with customer projects failing to make it into production. However, due to their business models, some vendors typically see far higher rates of production for their models than others. For instance, DataRobot typically charges by the model (instead of per seat) or bundles models within an infrastructure package. As a result, its customers are typically already committed to introducing a particular use case and only leverage its technology once other crucial implementation and data challenges are addressed.
There are downsides to the managed approach taken by DataRobot, as many customers are looking for far greater freedom to iterate and experiment in a platform; they simply need to have the flexibility to produce what will be in effect redundant models without being charged for the IP.
AI platform vendors still need to work to better understand the crucial barriers preventing organisations from scaling AI. Business model innovation might not necessarily be the answer.
However, enterprises looking to leverage AI also need to better consider technical and organisational challenges that might stand in the way of them putting AI use cases into production. Simply purchasing an AI platform system and giving it to a captive group of data scientists and business users is often not the best recipe for success.
To learn more about our upcoming research, please contact Jack Vernon, or head over to https://www.idc.com/eu and drop your details in the form on the top right.