Data management software has been around since the 1960s. Data management is a fundamental enabler of almost all types of software application and so it’s given rise to major corporations like Oracle and provided huge revenue streams for Microsoft, IBM and others over the years.
So you’d think this was a pretty mature, dull market, right? The term data management will probably induce a yawn? Not so. Following waning of the excitement and hype around Big Data technologies, like Hadoop, we’re now seeing a resurgence of interest. This is shown clearly in the investment going into the market. September 2020 saw the biggest software IPO of all time: cloud data warehousing company Snowflake debuted at a valuation of $33 billion, which rose sharply to peak at over $110 billion later that year.
But that’s not all. Digital transformation and the pandemic have ensured that companies are looking for better ways to reach their customers, to optimise fragile supply chains and automate ever-trickier processes, using sophisticated analytics and machine learning. For this they need more, bigger and different data sets — for instance, linked social data.
The Big Data technologies of the last decade looked like a good route to fix that, but Big Data as originally cast brought its own problems — for instance managing data models, making it easily available and usable by all those who needed it. Large corporations or highly skilled start-ups could tackle the issues, but for the great majority, the ROI, the understanding or the effort required just wasn’t there.
Cloud technologies have brought new, simpler solutions to these very real problems and new impetus to the market. Attention has turned to companies that can offer what’s needed in the cloud context. And as a result VC funding is flooding in. In 2021 we’ve seen:
- Distributed database product developer Cockroach Labs raise $160 million in a new round of funding, bringing its valuation to $2 billion and funds raised to $355 million
- Graph database company TigerGraph raise $105 million in Series C funding, claimed to be the largest funding round in graph market history
- Rival graph database company Neo4J raise $325 million in Series F funding, bringing its valuation to over $2 billion and total funding to over $500 million
- Entity resolution vendor Quantexa (applied database solutions) raise $153 million, valuing it up to $900 million — up from $200–$300 million a year earlier
Every CIO and chief data officer is surely aware of the possibilities of the new data types and management tools as is evidenced by their providers’ strong growth. But capitalising on them isn’t so straightforward. IDC research indicates that only a small percentage (single digits) of data is ever analysed. And, we find that only 6% of European organisations have embedded analytics in all their key workflows and applications.
So to help ensure that new intelligent technologies are able to deliver real business value and are not consigned to the dustbin of failed data initiatives we see leading organisations creating a new, layered data architecture that brings together different data sources and types into a kind of virtual data lake; this is then presented through a meta-data layer, or access plane, to the applications and tools that need it. IDC is advocating that organisations need to extend this middle layer to become a data control plane that encompasses semantics in an intelligence layer, governance with a policy engine and a data engineering layer that brings the different data types together. Only then can companies leverage the vast wealth of data assets they are constantly creating (on average, business data is growing 50% year on year) in a way that will help them achieve that elusive goal: the data-driven organisation.
For more information about IDC’s research into business analytics and data strategies, industry trends, established and emerging use cases, and key players, please contact Philip Carnelley, or head over to https://www.idc.com/eu and drop your details in the form on the top right. More detail about the data control plane architecture is available on idc.com.