This month SQream, which specializes in data processing and analytics acceleration, introduced its 2024 State of Big Data Analytics Report, providing insight into the growing disconnect between the costs of enterprise projects (particularly due to the explosion of AI) and value being realized.
The company’s report looked to shine a spotlight on the need to change how companies handle huge volumes of data.
The amount of data created continues to grow at an exponential rate: In 2020 there were 64.2 ZB of data in the digital universe; in 2024 that figure will more than double to an estimated 147 ZB.
SQream, with offices in New York, London and Tel Aviv, is working to change the way organizations approach big data analytics with its GPU-patented solutions. The company enables businesses to unlock insights from their data with speed and efficiency.
These hallmarks (efficiency and speed) are increasingly of interest to technical leaders, with the surge in AI creating massive volumes of data, which in turn have resulted in IT costs to dramatically expand.
In its report SQream surveyed 300 senior data management professionals from US companies with at least $5 million annual spend on cloud and infrastructure.
“This survey underscores the widespread nature of these data management challenges for large enterprises,” said Deborah Leff, Chief Revenue Officer of SQream. “Leaders are increasingly recognizing the transformative power of GPU acceleration. The immense value of an order-of-magnitude performance leap is simply too valuable to be ignored in the race to become AI-driven.”
Shockingly (even with their material budgets), in the report 98% of companies surveyed still experienced ML project failures in 2023.
Added Matan Libis, VP Product at SQream, “To get ahead in the competitive future of AI, enterprises need to ensure that more big data projects reach the finish line.”
“Constant compromising, including on the size of data sets and complexity of queries, is a huge risk factor that corporate leaders need to address in order to effectively deliver on strategic goals,” concluded the executive.
The report, available for download here, includes the below insights, among others:
Most organizations experience analytics “bill shock”
Although billing cycles vary from company to company, when asked how often they experience bill shock, 71% of respondents reported they are surprised by the high costs of their cloud analytics bill fairly frequently, with 5% experiencing bill shock monthly, 25% bimonthly and 41% quarterly.
41% of companies report high costs as the leading challenge
As with data analytics, the cost-performance of ML projects is key to successful business predictions. However, given that in ML the more experimentation a company conducts, the better the final result – it is no surprise that 41% of companies consider the high costs involved in ML experimentation to be the primary challenge associated with ML and data analytics today.
98% of companies experienced ML project failures in 2023
The top contributing factor to project failures in 2023 was insufficient budget (29%), which is consistent with findings throughout the report. In addition to cost concerns, the other top contributing factors to project failures were poor data preparation (19%) and poor data cleansing (19%).
3 out of 4 executives are looking to add more GPUs in 2024
75% of those surveyed said that adding GPU instances to their analytics stack will have the most impact on their data analytics and AI/ML goals in 2024.
Close to half of the respondents admitted they compromise on the complexity of queries
48% of the respondents admitted to having compromised on the complexity of queries in an effort to manage and control analytics costs – especially in relation to cloud resources and compute loads. 92% of companies are actively working to “rightsize” cloud spend on analytics.
Article’s featured photo: SQream Chief Revenue Officer Deborah Leff