The 2023 State of AI Infrastructure Survey, commissioned by Run:ai, sheds light on the growing challenges faced by organizations in AI development. The survey, which was conducted by Global Surveyz Research and gathered responses from 450 industry professionals across the US and Western EU, reveals that infrastructure and compute, chosen by 54% and 43% of respondents respectively, are now the primary hurdles, surpassing data as the key challenge facing AI development. This marks a shift compared to last year’s survey by Run:ai, where the largest number of respondents – 61% cited data as their top challenge.
The survey also pointed to another shift over the past year as the number of organizations deploying less than half of their AI models in production increased from 77%, according to last year’s survey, to 88% this year. Weighted for average, just 37% of AI models make it into production.
The adoption of cloud services for AI infrastructure continues to rise, with 73% of surveyed organizations using cloud services. However, the survey found a significant challenge in accessing GPU compute, as only 28% of respondents reported having timely and sufficient access to compute power upon demand. This shortage of on-demand access leads to frequent GPU allocation issues for 89% of respondents who use a ticketing system.
“Despite being on the cloud, organizations are still facing limitations with unlocking the full potential of their data,” said Omri Geller, CEO of Run:ai. “This highlights the reality that cloud hasn’t delivered on its on-demand promise and the importance of building a robust and scalable infrastructure.”
The survey also found that as organizations scale and require more GPUs, they face a proliferation of third-party tools, making it increasingly complex to manage AI infrastructure and get the most out of it. 77% of respondents indicated they are using multiple third-party tools, making it difficult to get the right amount of compute to different workloads and end-users.
“Organizations must shift their focus from solely acquiring more data to ensuring they have the proper infrastructure in place to effectively process and utilize it,” added Geller.
Some other findings of the survey:
- 91% of companies are planning to increase their GPU capacity or other AI infrastructure by an average of 23% in the next 12 months. This shows that despite the uncertain economic climate, companies are still investing in AI due to the potential and value they see in it.
- 50% of companies plan to implement monitoring, observability, and explainability in the next 6-12 months to keep track of their AI models.
- The second and third priorities were model deployment and serving (44%) and orchestration and pipelines (34%). This indicates that companies are focused on bringing their AI models into production and streamlining the process to make it more efficient.
