The Role of Data Storage in Accelerating Time-to-Insights

    Published on:

    Issues such as data quality and algorithm effectiveness further complicate the issue, but one factor that is often overlooked is storage. However, a modern storage infrastructure is more than just a “place to dump your data”; it’s a critical part of your analytics stack.

    Making good storage decisions is essential to providing rapid insight and supporting decision-making. In a recent discussion with Shawn Rosemarin, Pure Storage’s VP of R&D, Customer and Engineering, he discussed the technology’s impact on two of his hottest topics facing companies in this space, of course. We have delved into some of the challenges. Artificial intelligence (AI) and analytics.

    The role of data storage

    Today, businesses store data for completely different reasons than they did just a few decades ago. At the beginning of the digital revolution, information was typically stored for compliance and governance reasons, or simply to track past performance.

    Rosemarine tells me: “It’s only in the last few decades that we’ve started saying…what can we glean from past data…from what happened in the past to help us predict what will actually happen in the future? What can we collect, the future?”

    In recent years, this has led to significant advances in the ability to leverage data for business decision making. This has coincided with an explosion in the amount of data being generated by organizations, as well as an evolution in the technology available to capture and analyze that data.

    However, the question of where and how to store information has often been left on the back burner. I think people are often surprised to learn that the vast amount of data in the world is stored on squeaky mechanical disks and even squeaky tape storage. Data stored in this way is often difficult to access, both in terms of economic cost and, importantly, energy usage.

    “If you look at an all-flash data center, look at the benefits of disk and flash, and look at the current environment we’re in, you can free up more energy, power consumption, and human overhead. .I can focus more on energy savings, efficiency, and humanity on what I’m actually trying to do: solve AI and analytics challenges,” says Rosemarin.

    For example, in the field of drug discovery, time to insight is critical, not only for business reasons, but also because it can make a difference in human health and the fight against pandemics.

    One of Pure’s customers, McArthur Lab, processes millions of data points every day to find solutions to the growing threat of antimicrobial resistance. This includes tracking genes and mutations that can cause drug resistance in infectious diseases. Storage infrastructure migration The introduction of Pure Storage technology increases some analytical processes by 300 times, allowing researchers to quickly identify “superbugs” and evaluate potential treatments.

    In his speech, Rosemarin also highlighted the work his organization has done with Chungbuk Techno Park, a Korean innovation center specializing in incubating deep learning and machine learning solutions with local companies.

    We realized that we needed an AI-optimized solution to optimize and reduce the energy consumption of our data storage infrastructure, so we migrated our operations to Pure Storage infrastructure. This is directly Process data stored for AI workflows 2x faster.

    data quality

    We also discussed the biggest issue facing companies attempting to become data-driven: data quality challenges. This is often a major stumbling block when it comes to gaining insight.

    “Is that a date? Is that an era? Is that a real English word? “We got through it,” he claims, but the real challenge is “Did it actually happen? ” is to judge.

    As an example, he thinks of a doctor taking notes while examining a patient.

    “Physicians are very patient-focused. They are focused on providing medical care, so…notes may not show exactly what happened…patients may not know exactly what happened… You may not have told your doctor exactly what happened… How many drinks did you have this week? How many times did you go to the gym? ”

    As a result, he believes companies have often raced to get all their information into data lakes, resulting in a “data swamp.”

    If the data does not match reality, the effectiveness of the model is compromised. Developing ways to assess and mitigate data quality gaps should be high on the priority list for companies making this transformation.

    There are many ways storage infrastructure impacts data quality. Accessibility makes data validation and correction easier. It also has implications for governance. Built-in AI-powered tools ensure that information is stored correctly and in accordance with the law, and that all mandated security checks and measures are implemented.

    Further problems arise when the data storage infrastructure is not scalable. This easily creates silos and barriers between access to different data pools within an organization.


    I asked Rosemarin for tips for enterprises to ensure their storage infrastructure is up to the task of running today’s high-performance AI-driven analytics initiatives. One piece of advice was to “embrace simplicity and eliminate complexity.”

    “Whether you look at energy consumption or look at labor costs, it’s true that it takes more energy and more people to accomplish these projects…and infrastructure, especially storage. …not only the human overhead, but also the energy inefficiency of many traditional legacy storage systems.”

    When it comes to data management (and storage is no exception), simplification is almost always the right direction to go. The emergence of AI platforms and large-scale language models is rapidly democratizing the ability to benefit from data-driven insights. and generative AI.

    To take full advantage of this benefit, organizations must ensure that access to data is also democratized.

    “Quick-win” analytics initiatives can be launched and evaluated more quickly when the storage infrastructure is simplified. You can also redirect the time and money saved by migrating away from complex storage solutions to: Analytics and AI.

    Overall, adopting strategies to simplify your data storage infrastructure can be an effective way to reduce delays and bottlenecks that slow your time to insight.

    towards the future

    Flash storage now seems to be the norm everywhere except in data centers. It’s what’s in our phones, computers, appliances, and even cars.

    At the end of our chat, Rosemarine said: “The only place spinning disks still exist is in data centers, because it takes a lot of effort and effort to move spinning disks to flash.”

    It’s clear he sees this as a challenge that Pure can help customers overcome. And it must be done not only for their own future, but also for the future of the planet.

    “In the orbit we’re on, the Earth will run out of power,” he told me.

    “Once we get there, nuclear fission may help us. But we are already seeing countries telling public cloud operators that they can’t come in simply because they don’t have the available power to build data centers. We’re seeing London, parts of Ireland, and indeed the United States, Virginia, saying, “We can’t build any more data centers.”

    Faster, more efficient new forms of storage, including all-flash, will continue to play a role in easing businesses’ digital transformation journey.

    At the same time, it can also help reduce energy usage, potentially minimizing your organization’s environmental footprint. Whichever way you look at it, thinking more seriously about your storage infrastructure is a worthwhile endeavor.


    Leave a Reply

    Please enter your comment!
    Please enter your name here