Benchmark Your Way to Big Data Success
As most organizations begin to prepare for a new operational reality where issues such as resiliency, agility, cost awareness and governance will be key driving factors, there is a renewed focus on the use of data to drive decision making and process improvement. In principle, data is a valuable asset, however, for many organizations collecting large amounts of data has turned it into a liability and organizations often struggle to balance the business benefits and technical challenges associate with their Big Data and Analytics initiatives.
For the past 2 years, IDC has played a key role in an EU-funded project to develop a set of business and technical metrics/KPIs to help organizations benchmark both the business outcomes and technical elements of their Big Data investments. The overall goal of the project, Databench (https://www.databench.eu/), is to provide European organizations with a framework and set of tools to integrate business and technical benchmarking approaches for Big Data technology investments.
As part of this, the team has surveyed over 700 European organizations across various industries and sizes on their Big Data investments, developed over 700 qualitative use cases across these industries, examined the progress of over 40 large-scale big-data technology projects, and carried out an in-depth analysis of specific use-cases with organizations such as Whirlpool, Intel, CaixaBank and the European Space Agency.
In general, those who have invested in Big Data technologies have achieved significant improvements in key business metrics such as revenue, costs and profits. While performance varies across industries and use cases, a 4% – 8% improvement in these metrics on average is what we are seeing. When viewed in terms of more qualitative metrics (e.g. customer satisfaction, service quality, business model innovation) we see many organizations achieving improvements in the range of 10% – 24% which is a major achievement.
Technical Benchmarking Should Not Be Ignored
However, these is also a clear warning on how failure to adequately address the technical elements of the initiative early on, can seriously reduce the overall effectiveness. Most organizations believe that technical benchmarking of their Big Data initiatives requires very specialized skills and is inherently expensive, so few have carried out extensive or accurate benchmarking. In this context, cloud solutions for Big Data tend to be preferred as they are deemed to provide easier access to a wider set of technologies. That said, these organizations acknowledge that the variety and complexity of available Big Data solutions bring with it several key risks:
- The risk of choosing a technology that over time, proves to be non-scalable either technologically or economically
- The risk of Cloud vendor lock-in and the risks associated with migration to another platform
- The risk of Cloud services being more expensive than expected, particularly when scaled-up, and technology expenses outweighing business benefits
Real-Life Cases Undermine The Complexity
During the course of the project, a number of real-life implementations were examined to gain better insight into the challenges of balancing business outcomes with technical requirements. Two of these cases, from the retail sector, provide a clear indication of just how difficult this balancing act can be:
Case 1 Intelligent Fulfillment: A retail player implemented a system using a complex AI and Machine Learning system (sales prediction) to optimize the assortment selection and automated fulfilment at an individual shop level. The system was piloted in one shop and delivered a 5% increase in margin, which was the equivalent of about €5 million annually. Despite this, the project is currently on hold due to the economics of scaling up the initiative. The system was run on Apache Spark in a commercial cloud environment, but when scaled across hundreds of product categories, shops and system runs, the total cost to achieve the optimized assortment outweighed the business benefits.
Case 2 Online Recommendation System: Another retail player developed a system to provide personalized recommendations at an individual customer level. The system was designed to increase margin through targeted upsell and cross-sell recommendations. When trialled, the system was seen to generate a 3%-4% improvement in margin. However, across all product lines, the data tables required to support the cross-sell recommendations became too large to be deployed on an on-premise hardware appliance. To address this, the retailer simplified the targeting to the client segment level rather than individual client level – but these less personalized recommendations were seen to generate 90% fewer clicks by customers on the products being recommended, resulting in lower upsell and cross-sell success.
To underline the benefit of effective technology benchmarking and how this can make a significant difference to the overall success of an initiative, for each of these use cases a comparison was made of the relative cost of the cheapest and most expensive optimal machine configuration with a major cloud provider. In both cases, the cheapest option offered savings of over 70%. This level of saving could change the dynamics of the business case and in some cases, make the difference between scaling-up and shelving the project.
Star Performers Think Differently
We have also identified over 30 organizations that we perceive as “Star Performers” – those organizations that have achieved above average results from their Big Data initiatives. What will likely be of most interest to CIOs and business managers is what sets a Star Performer apart from the rest and what do they do that others do less well or not at all. This is what we have found, Star Performers:
- are more likely to be large organizations
- are generally more mature in their Big Data usage and can often be seen as “early adopters”
- set more ambitious business goals and have higher tolerance for risk
- place more emphasis on achieving a wider range of business goals but are very focused on the goals that are chosen
- do not perform any better than the rest in terms of using Big Data and analytics for cost reduction, but perform better in terms of revenue and margin improvements
- are prepared to implement technically complex architectures for sharing data outside the company
- are much less likely to implement enterprise-wide data lakes, and much more likely to use streaming real-time data in combination with contextual data
Interested? Try The Self-Assessment
So if you are considering new investments in Big Data and Analytics and are puzzling over some or all of these issues, we have some great insights to share. Better still, try the Self Assessment Survey https://www.databench.eu/self-assessment-survey/ and see how you stack up against your peers.
Recession-Beating IT — Time for Managed Cloud Service Providers to Shine?
The world is in distress. The COVID-19 pandemic has shaken our lives, our economies, and our social fabric. The ICT market has not been immune. IDC expects the market in Western Europe to contract 3.3% in 2020, returning to moderate growth of 0.9% in 2021, according to the latest IDC Worldwide Black Book forecast.
The overall market may look a little bleak, but if you zoom in on individual technologies, solutions, and services, opportunities do present themselves. This is especially true in cloud, security, and managed services. Though growth is forecast to be lower than in 2019, managed cloud providers can still look ahead with some optimism.
IDC has been running a buyer sentiment survey across key European markets every two weeks since the start of the pandemic, providing insight into buyer sentiment around managed cloud services (MCS), how use of cloud and related services is changing, and what buyers expect from their providers. We also carried out an MCS survey with 150 buyers and decision makers in Europe to explore how COVID-19 was affecting their decision making, road maps, and plans around MCS. The survey results have been published on IDC.com, but we wanted to share some key takeaways in this blog.
How COVID-19 is affecting plans around managed cloud services (MCS)
COVID-19 has forced many organizations to quickly adjust operations to respond to the new business reality: 26% of IT buyers believe MCS help them to build a resilience layer to deal with these sudden changes.
In these times of business uncertainty, there is pressure to improve business agility, and cloud-based solutions can be the platform to meet these demands and support digital transformation. And of course, increased demand for cloud has spurred a corresponding increase in demand for MCS, creating opportunities for service providers.
- MCS are a powerful tool for customers responding to COVID-19. Managed cloud services have been helping to deliver critical business and technology requirements during the pandemic. Among Western European companies, 26% see managed services as a tool to build an additional layer of business resiliency to protect against sudden shocks, and they have benefited from MCS providers’ ability to meet increased demand due to COVID-19.
- Customers seek a wider array of services. Customers want MCS providers to expand their service lines — managed security for remote access, workload optimization, and cost optimization are top of the wish list. Demand for cloud consulting and support for cloud migration are also increasing. Customers will introduce new service providers if necessary to meet their needs.
- Relationships between MCS providers and their customers will change. Customers see the potential for strategic relationships with their MCS providers, but economic uncertainty will encourage them to renegotiate prices and ask for new, more favorable terms and conditions as relationships grow.
IT decision makers have had to deal not only with an immediate need to shift to remote work and operations, but also a need for increased security as new weak points and threats emerge. Workload optimization has become a priority as businesses adjust to quickly changing market conditions, and of course, cost efficiency is a priority in an economic slowdown and recessionary environment. Close to a quarter (24%) of companies in Western Europe have sought after-cloud-migration support from third-party providers to help them adjust operations in this new reality. Many MCS providers believe that market turmoil, such as the current pandemic and the business lockdown, increases interest in a more flexible and scalable IT platform — a great opportunity for MCS providers.
We have seen greater use of managed cloud services among European customers, with 4 in 10 companies increasing their use of MCS. Within a year, 43% will continue to do so.
What impact has COVID-19 had on your company’s use of managed cloud services? How it will change in the next 12 months?
More than a third of IT buyers will expand the number of services they use, with the strongest demand for multicloud management, next-gen security, data platforms, and cognitive capabilities. There will also continue to be strong demand for cost-optimization services. To find new services, European companies will look to change the number of providers they work with, with 45% of organizations saying they will increase their number of MCS providers. Providers need to be prepared for fierce competition as their customers work with a wide range of MCS providers from different backgrounds — from traditional technology vendors such as IBM and Fujitsu, to consulting companies such as Accenture, to telecommunication operators such as Orange and T-Mobile, and to specialized managed cloud service providers such as Nordcloud and Cloudreach.
Recommendations for managed cloud service providers
In this hyper-competitive environment, MCS providers should stay close to their current and potential clients and be ready not only to offer a broader, more comprehensive service portfolio, but also to be flexible as customers seek to renegotiate prices (25% of all respondents) and contract conditions (20%), and try to work more directly with cloud providers/hyperscalers (19%).
When deciding whether or not to work directly with a hyperscaler or one of their MCS provider partners, there are a few attributes that give managed cloud service providers the edge:
- The ability to build a long-term relationship with a client
- A willingness to engage in day-to-day problem resolution
- Being readily available to customers
These advantages must be part of the core value proposition — and are key for future business development.
At the end of June, our buyer sentiment survey showed that 80% of European companies plan to sustain or increase their spending on managed services in 2020. The opportunity is there — you just have to grab it.
The Role of Technology in Rebuilding European Business
IDC’s vision of the “future enterprise” is an organization that is completely digitally transformed. Such an organization underpins business processes with technology, is fueled by innovation, and is platform-enabled and ecosystem-centric. The COVID-19 pandemic underscores the importance of digital transformation in the eyes of CEOs across all industries. But with a recession on the horizon, CEOs face a tough call — to follow the same course of cost cutting as in previous recessions or to use technology to flatten their own organizations’ recessionary curve.
Leverage Technology to Transition to the Next Normal
Where Are Organizations on the Road to Recovery?
According to the latest IDC COVID-19 Impact Survey, conducted in June among 730 organizations in Europe, the largest pool of organizations — 37% — are still in the business continuity stage (Stage 1) of the journey to the “next normal.” 33% are still in the cost optimization stage (Stage 2), while 18% are focusing on building business resiliency to drive profitability during the pending recession. The remaining 15% have already started their recovery journey and are working toward growth in the next normal. (Note that we are frequently asked for a timeline for these stages, but the reality is that every geography, industry, and organization will follow its own trajectory.)
Stage 1: COVID-19 Crisis — Business Continuity Is the Leading Business Priority
Whether we’re dealing with the current pandemic or the next shock to the global market, hyper-volatility will increasingly become the norm. Business continuity plans are already being rewritten — and will need to become a lot more dynamic as they are constantly tested and innovated.
C-suite narrative: “We’re in crisis mode. We’ve had to rip up our business continuity plans and look at everything from scratch.”
Organizations in this stage are desperately looking for new ideas, emerging best practices, and greater input from technology partners. This is not a time to push the product. More importantly, the support and help that technology partners provide in this period will be the basis for genuine long-term partnerships.
Stage 2: Economic Slowdown — Cost Optimization Is the Leading Business Priority
In this stage, organizations are in cost optimization mode, with businesses looking to technologies that can either help them manage the economic hardships or generate financial outcomes for the current fiscal year. These organizations look for financing assistance and/or opex options to manage their cashflows and working capital. Their decisions are driven by the short term as they seek projects that provide productivity gains and savings in cost-to-create or -serve.
C-suite narrative: “We’re in cost-saving mode and we’re prioritizing projects and programs that can tide us over during this challenging period. We’re not interested in exploring bleeding-edge technology use cases or jumpstarting any transformation program.”
Organizations in this stage tend to prefer technology vendors, SIs, and service providers that are closer to them — in both proximity and cultural alignment — and provide cheaper offerings, flexi-pricing and payment terms, or modular offerings.
Stage 3: Recession — Business Resiliency Is the Core Driving Force
In this stage, organizations have overcome the initial phase of the COVID-19 crisis and have stabilized their financials. Their focus now is on building business resiliency — adapting to changing circumstances while maintaining their central purpose. The purpose or mission articulated in the company’s digital transformation will remain a primary focus even as remediation actions are taken to reduce costs or take advantage of pandemic-related revenue opportunities.
Many organizations will recognize an opportunity to “flatten the curve” or minimize the impact of the recession on their organization by leveraging technology. These organizations will also double down in technology investments to emerge, on the other side of the curve, more resilient, more digitally fit, and ready to capture a greater share of the new opportunities.
C-suite narrative: “We’re ready to move our organization to the next level of response. We understand the criticality of technology and are reprioritizing our focus and investments to compete differently post-COVID-19. Adaptability and agility are our main thrust.”
Organizations in this stage tend to prefer technology vendors with established presence and delivery capabilities in their geographies and with the financial muscle to ride through the crisis. Access to the vendor’s product/capability road map is also important as organizations here are preparing for longer-term growth.
Stage 4: Return to Growth — Targeted Investments Are the Key Business Driver
At this stage, economic activity is returning to pre-crisis levels and companies are looking to invest more aggressively, but with a focus on technologies that advance their digital capabilities. As they get back on the path to being relevant in the digital economy, they will look back at their efforts around resiliency to ensure they are more prepared for a similar shock in the future.
IDC anticipates that corporate boards and government bodies will have a better understanding of the value of technology and will prioritize digital initiatives.
C-suite narrative: “We consider periods of market volatility as opportunities that well-prepared organizations can embrace and exploit.”
This will be a critical stage for technology vendors as organizations look for a safe place to innovate. Vendors that are financially healthy, have scale, and have robust ecosystems will be preferred. Vendors will earn strategic partnerships at the board level that will shape post-recession spending and permanently impact market shares.
Stage 5: The Next Normal — The Future Enterprise
When organizations come out of the recession, they will be operating in a new type of economy — the next normal. This will be the beginning of the digital economy with some new twists as a result of COVID-19. CEOs who have been consistently investing in digital transformation will be running the future enterprises and employing a digital enterprise agenda that centers on nine items (see below).
The CEO Agenda for the Future Enterprise
C-suite narrative: “Operating as a digital enterprise is vital to the success of our business. We will continue to invest in technology as it is a competitive advantage.”
Strategic vendor relationships are established during Stage 5, and the fortunate vendors that have won those positions must prove they can innovate with speed and scale to maintain them. Resiliency will remain a key expectation, not just to mitigate against a potential repeat of the pandemic but to enable companies to quickly recalibrate to the changing demands of customers, citizens, and patients.
Conclusion
Organizations’ response to COVID-19 will shape the perception of their brand and reputation over the next decade. As part of this, there needs to be a rethink of the core principles of how business operates.
In the short term, technology has proven to be a key enabler that has kept businesses afloat during the crisis by enabling remote work, agile business, and external digital engagement. In the longer term, organizations will leverage technology to underpin every single process, initiative, or value chain as they journey to the future state — the future enterprise.
Quantum: Towards the Next Computing Revolution
Quantum computing has a very good chance of becoming the next revolution in the computing world. Even though we’re still far from the commercial maturity of real quantum computing applications, companies, universities, and research institutions worldwide are working hard to fill in the technical gaps to make those technologies ready for business.
This was clear during the Quantum.Tech Digital Week (June 15–19, 2020), the online edition of the Quantum.Tech European event that should have taken place in April in London. Quantum.Tech is one of the pivotal business-oriented events in the field of quantum applications, connecting people from IT companies, start-ups, academia, research institutions, and governments.
The event shines a light on worldwide quantum computing developments, highlighting the most recent technological advancements as well as the first insights and road maps for quantum applications experimentation and adoption in a business context. The event is also an opportunity for leading companies working on quantum computing — such as Google, Honeywell, IBM, and Rigetti — to present their latest research updates.
Quantum Technological Race Is Accelerating
It’s clear that the overall effort of IT companies in quantum research and development is increasing, and that the announcements of more and more performing prototypes are multiplying. Honeywell has announced the highest-performing quantum machine currently available, a trapped ion quantum computer with a “quantum volume” of 64. The quantum volume doesn’t represent the number of qubits (which is 6 for Honeywell’s machine) but the overall capabilities of a quantum computer in terms of execution of larger quantum circuits and hence the complexity of problems it can solve.
After the announcement of the so-called “quantum supremacy” achievement in 2019, Google introduced during the event the results of the largest quantum chemistry simulation and the largest quantum approximate optimization to date, as well as its new open-source framework for quantum machine learning, TensorFlow Quantum. Also, IBM introduced its latest advancements in Qiskit’s circuit library and algorithms, and how they are co-designing codes and hardware to improve the error correction process. Rigetti presented its efforts in quantum hardware scaling, and in particular in cryogenic systems with large qubit capacity and modular quantum processing unit (QPU).
If we consider the number of other IT companies (e.g., AWS, QC Ware), start-ups (e.g., Aliro, Oxford Quantum Computing), universities (e.g., University of Cologne, Imperial College London), and research institutions (U.K. Innovation & Research, Russian Quantum Center) that took part in the debate by presenting their technical achievements, we can see the impressive increase in contributions from all over the nascent quantum computing ecosystem.
Quantum Technologies and Business: Growing Interest from End-User Companies
Technology, of course, is just one side of the quantum computing coin. Though it’s still some way off, the proper experimentation and deployment of quantum applications in a business context will be ever more crucial for quantum computing success. In particular, industries and use cases suitable for quantum computing development should be identified and exploited right away to be ready for real business impact when the underlying technology reaches maturity. IT companies taking part in Quantum.Tech Digital Week are increasingly aware of this, and the event was also an opportunity to see some interesting examples of quantum experimentations across different verticals.
Applications mentioned throughout the week involve several end-user companies in materials science and chemistry (Covestro, BASF, Johnson Matthey), manufacturing (Airbus, BMW, Volkswagen, Rolls Royce), finance (Goldman Sachs, Barclays, Standard Chartered Bank, Morgan Stanley, ABN AMRO), healthcare (GSK, Roche), utilities and oil and gas (Total, E.ON), and telecommunications (Telecom Italia, BT). All the experimentations are still at an R&D level, but it’s already possible to see the main problems quantum technology is trying to solve there, namely optimization (especially in manufacturing), machine learning (finance), simulation (chemistry), and secure communication (telecommunications).
Quantum Computing: A Key Challenge for the Future of Technology
The road to effective quantum technologies is still long and full of obstacles, but the Quantum.Tech Digital Week sent an optimistic message for the future.
Besides purely technological challenges to be addressed (e.g., stability of the system, error correction), a key to the potentially revolutionary success of quantum computing will be the achievement of significant business impact for companies. For this to happen, IT companies and the whole European quantum ecosystem in general should consider the following recommendations:
- Partner with top research centers and start-ups worldwide to create efficient innovation ecosystems to reach the necessary R&D critical mass.
- Adopt a “hardware-agnostic” approach for applications development to avoid switching between different types of quantum technologies as they evolve, and develop a “hybrid quantum-classical“ approach, as quantum will likely support and not fully replace traditional computers.
- Identify and focus on the most promising industries and use cases, and adopt a business-oriented language to involve and make an increasing number of end-user companies aware of quantum’s potential.
- Focus on skills and quantum tech talent recruitment and, in the near future, on the importance of training real “quantum business consultants” to link the technology to the business needs of end-user companies.
For more on IDC’s European Quantum Computing Launchpad market research, you can listen to a free recording of my presentation Quantum Computing in Europe: The Road to Business Impact at Quantum.Tech Digital Week, or contact me directly at sperini@idc.com.