Pure Storage recently achieved a significant design win with one of the top four hyperscalers. According to a company press release, this industry-first milestone marks the expansion of Pure Storage’s DirectFlash technology into large-scale environments currently dominated by hard disks.

Pure Storage describes this as the ‘beginning of a transformative shift in the industry’. Historically, hyperscalers have relied on commodity disk storage, representing nearly 70% of global hard disk drive purchases—an estimated 700 exabytes per year.

“This is the first-ever design win to provide flash for standard hyperscaler storage and it is the vanguard for flash storage providing all online storage in major hyperscale environments in the future. For reference, the hyperscale market is responsible today for 60% to 70% of all Hard Disk Drives purchased globally,” Pure Storage CEO Charlie Giancarlo said while announcing the hyperscaler win on the company’s third quarter FY25 earnings call. 

Pure Storage has been contending that by 2028, practically no new all-HDD storage systems will be sold for data center computing. In fact, HDD shipments have seen a sharp decline over the past decade. From a peak of over 651 million units in 2010, shipments dropped by nearly 71%, reaching 166 million in 2022, according to Statista.

The India Story

Ramanujam Komanduri, Country Manager for Pure Storage India, emphasized how the hyperscaler win is driving the transition to flash storage.

“Hyperscalers are possibly the largest data consumers, and they have been using commodity disk storage simply because there hasn’t been an alternative that offers comparable economics—until now. Recent improvements in flash media density, its ability to scale higher and lower power consumption have been game-changers in recent years. Our design win signals that hypercalers can manage extremely large data footprints with high-capacity flash storage at economics similar to what they have been used to in the past.”

He emphasized that the design win will add to the momentum for the replacement of hard disk drives (HDDs) in the market.  

India accounts for a significant share of the market, generating nearly one-fifth of the world’s data. “Flash storage is a more sustainable way for hyperscalers to store the growing volumes of data. With Pure’s Direct-to-Flash technology, we can provide hyperscalers better performance, reliability, and significant power and space savings compared to hard disks, all at a comparable or lower total cost of ownership,” Ramanujam maintains. 

The post The Disk-to-Flash transition: Pure Storage bets on a new era for Hyperscalers appeared first on CIO Dimension.

Authored by: Leela Krishna

Data has never been as important for companies as it is today. The potential impacts of loss of data privacy have led many countries to pass data regulations such as the General Data Protection Regulation (GDPR) in the EU or the recent Digital Personal Data Protection Act (DPDP Act) in India. These regulations require companies to take specific measures to protect customer data and facilitate requests from individuals, making compliance a key concern. 

The protection of customer data is a major business imperative for any organization, especially ones working with sensitive information like financial or health data. Data has long been the backbone of operations and business models of companies and is only becoming more crucial with the increasing deployment of machine learning models. Creating data protection protocols in compliance with these regulations will help assuage customer concerns and enable fintech companies to build a strong data foundation upon which they can create solutions. For the sake of simplicity, the measures that companies can take can broadly be grouped into two categories: security and general compliance.

Protection is the priority

Adopting robust security measures is now a non-negotiable that has been ratified into law. There are many ways in which data can be protected in the digital age, but a good first place to start is actually non-digital – adopting best practices on data governance, hygiene, disposing of the data and physical security. Organizations need to ensure that their data assets, and other critical infrastructure are safeguarded against unauthorized access, natural disasters, and other potential threats. By implementing strict access control systems, businesses can deter malicious actors from physically accessing sensitive data. And then the governance, supported by a strong and regular audit of the effectiveness of the controls is vital to ensure that vulnerabilities do not exist and or are remediated promptly.

Physical measures will only be the starting point for data protection. Businesses are becoming more and more interconnected with rising rates of cloud adoption across sectors, all while remote working becomes a common business practice. All these factors compound, exponentially increasing network vulnerabilities and making network security a priority for organizations. Creating and regularly updating the perimeter defenses such as firewalls, intrusion detection systems, and intrusion prevention systems to monitor and block any unauthorized or malicious traffic will be crucial to thwarting evolving cyber threats.

The conventional network/security controls can be taken one step further with the creation of Zero Trust architectures that have stringent verification protocols, regardless of whether the user or device is inside or outside the organization’s network. Zero Trust is becoming the norm for many organizations as it protects organizations from external and insider threats alike. 

While network security focuses on safeguarding the pathways by which data can be illegally accessed there are security measures that target the data itself. Such data security measures first require companies to map out all the data they own and process, whether it is customer data or proprietary data. This provides the foundation for additional measures like encryption at rest and in transit to limit the damage when access is compromised by malicious actors, and regular backups. Backups are an essential component of contingency plans in the event that data gets lost, corrupted or compromised.

Having robust internal data protocols and accesses also can create a rich layer for advanced machine learning based analytics and automated response mechanisms. Using security information and event management (SIEM) tools with a layer of analytics can allow the organization to monitor, detect and respond to threats in real time. Finally, to reiterate, no security measures will be effective if they are not complimented by commiserating audit process. And lastly, the most important aspect is the human intervention in the middle. Continuous training and awareness campaigns are what can avoid or minimize any and every possible impact of an incident.

Beyond network boundaries

While security measures are an essential part of data protection laws, most regulations cover a variety of other provisions concerning the implications of data processing activities. For example, organizations will have to think of compliance when setting up their data processing and storage infrastructures. Many data protection regulations, including the DPDP Act, create obligations on companies to respond to requests from data principals regarding the status of any personal data the company may be holding. Similarly, there are obligations to notify data principals if their data has been compromised via a data breach. All these measures require the setting up of necessary mechanisms to ensure that organizations are well placed to adhere to these provisions. This can be especially complicated for large companies that have third party vendors who store or process data on behalf of these companies. 

Organizations will also need to conduct privacy impact assessments to identify, assess, and mitigate privacy risks. Previously just an ideal practice for organizations to follow, Data Protection Impact Assessments (DPIAs) are now a requirement under the DPDP Act. These assessments allow companies to scrutinize the nature, scope, context, and purpose of their data processing activities to ensure compliance and protect the rights of data principals under the DPDP Act.

Keeping up to date

Data protection regulations are going to be the norm moving forward so ensuring compliance will be a crucial aspect of operations for all organizations. As the importance of data continues to grow, so does the responsibility of protecting it. 

A multi-faceted approach encompassing both robust security measures and stringent privacy compliance processes is the way forward. This includes clearly defining policies on classification, ensuring access on need basis, employing right set of standards and technology controls, aptly defining processes for customer data, vendor/third party assurance programs, proactively integrating the privacy in design, and getting executive support for the initiative while ensuring continuous improvement. In a world rife with cyber threats and regulatory complexities, this dual focus ensures not only business continuity and legal compliance but also the trust of customers and stakeholders.

The author is the IT Head of Broadridge India, a global fintech company. Broadridge provides the critical infrastructure that powers investing, corporate governance, and communications to enable better financial lives.

The post Data protection in the digital age: How FinTechs can dot the I’s and cross the T’s appeared first on CIO Dimension.

Snowflake and NVIDIA announced that they are partnering to provide businesses with a platform to create customized generative AI applications using their own proprietary data in the Snowflake Data Cloud. The announcement came today at Snowflake Summit 2023.

With NVIDIA NeMo platform for developing large language models (LLMs) and NVIDIA GPU-accelerated computing, Snowflake customers can use data in their Snowflake accounts to make custom LLMs for advanced generative AI services, including chatbots, search and summarization. Snowflake plans to host and run NeMo in the Data Cloud. 

“Snowflake’s partnership with NVIDIA will bring high performance machine learning and artificial intelligence to our vast volumes of proprietary and structured enterprise data, a new frontier to bringing unprecedented insights, predictions and prescriptions to the global world of business,” said Frank Slootman, chairman and CEO, Snowflake.

“Data is essential to creating generative AI applications that understand the complex operations and unique voice of every company,” said Jensen Huang, Founder and CEO, NVIDIA. “Together, NVIDIA and Snowflake will create an AI factory that helps enterprises turn their own valuable data into custom generative AI models to power groundbreaking new applications — right from the cloud platform that they use to run their businesses.”

The partnership will enable customers to use their proprietary data — which can range from hundreds of terabytes to petabytes of raw and curated business information — to create and fine-tune custom LLMs that power business-specific applications and services. 

The companies maintain that the integration of AI technology from Snowflake and NVIDIA will enable users to quickly build generative AI apps for a number of business use cases. For example, a healthcare insurance model could answer complex questions about what procedures are covered under various plans. A financial services model could share details about specific lending opportunities available to retail and business customers based on a variety of circumstances. 

Another advantage, according to Snowflake, is the ability to create these apps where the governed data already resides – a benefit that reduces cost and latency while maintaining the security of the data. 

The post NVIDIA’s NeMo comes to Snowflake Data Cloud appeared first on CIO Dimension.

Research and advisory firm Info-Tech Research Group has published its newest industry blueprint, Build an IT Risk Taxonomy. This data-backed resource aims to help organizations to become more flexible and agile to adapt to changing business conditions, according to the company. 

“Managing emerging risks such as climate risk, the impact of digital disruption on internal technology, and the greater use of third parties will require IT leaders to be more disciplined in how they manage and communicate material risks to the enterprise,” says Donna Bales, principal research director at Info-Tech Research Group.

“Establishing a hierarchical common language of IT risks through a taxonomy will facilitate true aggregation and integration of risks, enabling more effective decision making.”

Info-Tech’s blueprint highlights the challenges IT departments face in managing and addressing risks. These challenges include business leaders who are seeking to make informed decisions and expect timely and consistent risk reporting from IT. The constantly evolving threat landscape also adds complexity, requiring IT risk managers to balance the emerging threats while not losing sight of the current risks.

The firm’s research explains that developing a relevant and detailed IT risk taxonomy over time can be particularly challenging and that gaining acceptance and promoting accountability within the organization may pose further obstacles. However, involving business leaders and risk owners in the development of the IT risk taxonomy can enhance organizational acceptance and understanding.

In the new resource, Info-Tech advises that risk management must also mature as technology and digitization continue to advance. To strengthen operational and financial resiliency, organizations must move away from a siloed approach to IT risk management toward an integrated approach.

Without a common IT risk taxonomy, effective risk assessment and aggregation at the enterprise level is not possible. The firm’s blueprint outlines an IT risk taxonomy approach that can provide a common language to enable more efficient risk aggregation and interoperability between IT and the enterprise. The recommended approach includes the following three phases:

1. Understand Risk Management Fundamentals: IT departments must take a collaborative approach when developing an IT risk taxonomy to enable greater acceptance and understanding of accountability.
   
2. Set the Organization Up for Success: Risk managers must invest sufficient time in conducting a comprehensive analysis of the existing and future threat landscape when defining level 1 IT risks and consider the causal impact and complex linkages and intersections.
   
   
3. Structure an IT Risk Taxonomy: IT risk managers must recognize the dynamic nature of the threat landscape and acknowledge that an IT risk taxonomy is a living document that requires regular review and enhancement to ensure its ongoing relevance and effectiveness.

“This holistic, disciplined approach to risk management helps to promote a more sustainable risk culture across the organization while adding greater rigor at the IT control level,” explains Bales.

As the risk landscape continually evolves, there is greater pressure on the risk function to work collaboratively throughout the organization to strengthen operational resilience and minimize strategic, financial, and reputational impact. A successful risk taxonomy is forward-looking and codifies the most frequently used risk language across an organization.

The post IT must anticipate risks to adapt in age of disruption: Info-Tech Research Group appeared first on CIO Dimension.

For today’s digital businesses, the perimeter isn’t static. The perimeter rather consists of all the data and users that an organization holds. And, that perimeter is everywhere – across devices, networks, and clouds. Cyber incidents have, therefore, become an inevitable reality. A global report points out that, each organization faces an average of 1,248 attacks per week. 

In our latest episode of Innovation Talk series, we spoke to Maheswaran S, Country Manager, Varonis India, to understand the evolving threat landscape. 

“CISOs are taking a closer look at whether their existing controls are aligned with their organization’s attack surface. Most organizations have invested in assets and perimeter-centric controls, which is important. What is equally important is to have controls around data. Security must shift from being threat-centric to being user and data-centricl” says Maheswaran. 

Watch the full video here: 

As a security industry veteran with over two decades of experience, Maheswaran has been working with enterprises across the APAC region to guide their data protection strategies. 

He believes that data and users are the weakest link in the system. 

In an organization with a headcount of 10,000+, an average employee has access to over 12 million files while all she needs is a few thousand files. This has led to overexposure of data and increased attack surface, he adds. 

AI and automation technologies can greatly help in analyzing and reducing a company’s attack surface and blast radius. “Automation has a wide range of application across the security value chain, right from addressing the skills gap, creating awareness among employees to data security, incident management and so on,” says Maheswaran. He goes on to talk about effectively leveraging automation technologies and how CISOs can achieve better RoI from their cybersecurity investments.  

The post Innovation Talk:  Maheswaran S of Varonis on shifting to data & user-centric security  appeared first on CIO Dimension.

They say businesses thrive on data. The simple logic is, the more the data the better. But is that really the case? 

A recent study published by Oracle proves otherwise. This massive study titled “The Decision Dilemma” by Oracle and Seth Stephens-Davidowitz, New York Times bestselling author, surveyed 14,000 employees and business leaders across 17 countries. 

Some of its findings are quite a revelation. For example, 86% surveyed said that the volume of data is making decisions in their personal and professional lives much more complicated. So much so, that a thumbing 70% of the people admit that they have given up on making a decision because the data was too overwhelming. 

The decision dilemma, as expected, is reflecting equally strongly on business decision-making too. 85% of business leaders have suffered from decision distress—regretting, feeling guilty about, or questioning a decision they made in the past year.

That sounds like an archetypal leadership conflict.

But is the availability and accessibility of data helping these leaders make better and quicker decisions? But the contrary is true, with 91%

of business leaders reporting that the growing number of data sources has limited the success of their organizations. In fact, 73% even admit their lack of trust in data has stopped them from making any decision at all.          

A vicious cycle? 

The study indicates that these challenges lead to new set of problems within the decision-making process – making a decision first and then looking for data to support it, or putting the highest-paid person’s opinion ahead of data. 

This love-hate relationship with data is visible across the survey, with 97% of people wanting help from data and yet get frustrated due to the overload it. The study also specifically highlights the fact that a significant number of business leaders (87%) in India would prefer for all these difficulties to just go away and to have a robot make their decisions. 

“People are drowning in data,” said Seth Stephens-Davidowitz, data scientist and author of Everybody Lies and Don’t Trust Your Gut. “This study highlights how the overwhelming amount of inputs a person gets in their average day — internet searches, news alerts, unsolicited comments from friends — frequently add up to more information than the brain is configured to handle. People are tempted to throw out the confusing, and sometimes conflicting, data and just do what feels right. But this can be a big mistake. It has been proven over and over again that our instincts can lead us astray and the best decision-making is done with a proper understanding of the relevant data. Finding a way to get a handle on the stream of data at their fingertips, to help businesses distinguish between the signal and the noise, is a crucial first step.”

The study summarizes that technology plays a key role decluttering the data-driven decision making process to a large extent. “The message is overwhelmingly clear—our current approach to managing and analyzing data must change. Especially in the business world. The benefits of getting it right span from attracting talent in a difficult job market, to gaining new investors in a tough economic climate,” underlines the study. 

The post The classic dilemma of ‘too much data’  appeared first on CIO Dimension.

New market research led by independent consulting firm Boston Consulting Group (BCG) and sponsored by Red Hat and Starburst explores organizational challenges related to exponential growth in data volumes and rapid innovation across the data stack. The research indicates that organizations are facing a perfect storm of cost, complexity, and talent when it comes to supporting critical analytics and AI initiatives.

The study titled, “A New Architecture to Manage Data Costs and Complexity,”  highlights three main trends reshaping the data landscape

1) The volume and velocity of data are increasing;

2) Data use cases are becoming both more accessible, creating the growth of “citizen data scientists”; and

3) Technology advancements have shifted the pricing model.

According to the new report, these trends are creating challenges that have put immense pressure on today’s architectures:

• Today’s enterprise architectures are stretched thin, with more than 50% of data leaders saying architectural complexity is a significant pain point.
• Vendor proliferation across all data categories is a major issue. For larger companies with a more mature data stack, the total number of unique data vendors has nearly tripled in the last decade — up from about 50 to over 150 today.
• 56% of managers said managing data operating costs is a pain point, but they are still continuing to increase their investments in modernizing and building new data architectures.

Regarding the trend of data use cases becoming more accessible, Pranay Ahlawat, Partner and Associate Director at BCG, said, “Accessibility will continue to increase as data literacy and a basic understanding of programming languages such as SQL become more widespread among nontechnical employees. According to our research, almost three-quarters (73%) of survey respondents expect the number of non-technical consumers of data to increase in the next three years.” T

he research details the growth of ‘citizen data scientists’ and juxtaposes that growth with the increased sophistication of AI-related use cases. The gap between more advanced analytics use cases and technologies, and the analytics skill sets required, is currently limiting the business outcomes AI can drive.

“Only 54% of managers believe that their company’s AI initiatives create tangible business value.”

Regarding data storage and architecture complexity, Steven Huels Senior Director, Cloud Services for AI and Machine Learning at Red Hat, said, “The survey responses confirm that many enterprises are struggling with adapting to increasing data volumes across multi-cloud and edge while also maintaining legacy data architectures. This is compounded by increasing data privacy regulations, pressure on IT and data spend and a shortage of highly-skilled talent. Red Hat believes that the solution to managing these challenges will be to implement data architectures that are agile — built for today’s requirements with the flexibility to evolve quickly in the future.”

The research indicates that given the rapid growth of data and use-case volume, increasing complexity and the skyrocketing costs, more organizations are reaching a breaking point. For those willing to take this on, the report offered a few key lessons to keep in mind:

• Data architectures are evolving to be more federated and service-oriented: According to survey respondents, 68% of companies aspire to implement a more federated and distributed architectural paradigm (i.e., Data Mesh or Data Products) in the next three years.
• Pay close attention to overall data TCO. To keep costs under control, establish baseline spending and de-average customer segments to understand key drivers—such as people, data transfer and movement, data storage, and software. Drive shorter-term tactical cost improvements by exploring multiple approaches.
• Economics will drive architecture choices — open source and hyperscalers will continue to influence technology choices: BCG research shows that cloud and open source are expected to continue to play a significant role in the future of enterprise data architectures as enterprises aim to manage costs. In fact, BCG estimates that open source can reduce the total costs of the data stack by 15%-40% for some organizations.

The post Study highlights how organizations need to address spiraling data costs & complexity appeared first on CIO Dimension.

The 2023 State of AI Infrastructure Survey, commissioned by Run:ai, sheds light on the growing challenges faced by organizations in AI development. The survey, which was conducted by Global Surveyz Research and gathered responses from 450 industry professionals across the US and Western EU, reveals that infrastructure and compute, chosen by 54% and 43% of respondents respectively, are now the primary hurdles, surpassing data as the key challenge facing AI development. This marks a shift compared to last year’s survey by Run:ai, where the largest number of respondents – 61% cited data as their top challenge.

The survey also pointed to another shift over the past year as the number of organizations deploying less than half of their AI models in production increased from 77%, according to last year’s survey, to 88% this year. Weighted for average, just 37% of AI models make it into production.

The adoption of cloud services for AI infrastructure continues to rise, with 73% of surveyed organizations using cloud services. However, the survey found a significant challenge in accessing GPU compute, as only 28% of respondents reported having timely and sufficient access to compute power upon demand. This shortage of on-demand access leads to frequent GPU allocation issues for 89% of respondents who use a ticketing system.

“Despite being on the cloud, organizations are still facing limitations with unlocking the full potential of their data,” said Omri Geller, CEO of Run:ai. “This highlights the reality that cloud hasn’t delivered on its on-demand promise and the importance of building a robust and scalable infrastructure.”

The survey also found that as organizations scale and require more GPUs, they face a proliferation of third-party tools, making it increasingly complex to manage AI infrastructure and get the most out of it. 77% of respondents indicated they are using multiple third-party tools, making it difficult to get the right amount of compute to different workloads and end-users.

“Organizations must shift their focus from solely acquiring more data to ensuring they have the proper infrastructure in place to effectively process and utilize it,” added Geller.

Some other findings of the survey:

• 91% of companies are planning to increase their GPU capacity or other AI infrastructure by an average of 23% in the next 12 months. This shows that despite the uncertain economic climate, companies are still investing in AI due to the potential and value they see in it.

• 50% of companies plan to implement monitoring, observability, and explainability in the next 6-12 months to keep track of their AI models.

• The second and third priorities were model deployment and serving (44%) and orchestration and pipelines (34%). This indicates that companies are focused on bringing their AI models into production and streamlining the process to make it more efficient.

The post Data no longer the top challenge for AI projects; Infra & Compute are appeared first on CIO Dimension.