Cyber ​​security is the biggest concern for anyone working in enterprise IT. Cyber ​​resilience is among the top priorities in nearly every CIO survey, and for good reason. The average cost of a data breach is around $4 million. What is even more astonishing is that, according to Pokemon Institute, 83% of organizations suffered more than one such breach. Such numbers can add up quickly.

So it’s no wonder the cybersecurity industry is booming. According to McKinsey, the cybersecurity market could be worth up to $10.5 trillion annually by 2025. Last year, it was worth between $200-400 billion, depending on whose statistics you read. I believe in these numbers. Last month I attended the RSA Conference and was one of about 45,000 attendees. Today, RSA is one of the biggest IT events in the world.

As the market for cybersecurity products explodes, we’re also starting to see innovative approaches to detecting threats embedded in infrastructure. A storage array, for example, should be able to detect data corruption. After all, storage is where your data lives. The challenge, however, is that the storage array often needs more context to know if the data is good or bad. This is because the storage device simply stores the data, it does not attempt to interpret it.

IBM has taken on the challenge of building ransomware threat detection directly into its FlashSystem storage solutions. The company uses an innovative approach that promises to detect ransomware and other corrupt data as it is being written to disk, all without needing to understand the contents of that data. IBM builds this functionality directly into its storage devices. IBM tells us that more is coming soon.

Anomaly detection using data behavior analysis

There are three basic approaches used to detect ransomware: detect it online, detect it using file signatures on servers and computers, or detect it by looking at the behavior of the data itself – most threat detection solutions on the market focus on the first two. The challenge with signature-based detection is that to be useful, you need to know the signature of each piece of malware. So the arms race is on.

IBM’s new anomaly detection capabilities focus on the behavior of the data itself without requiring specific knowledge of malware signatures. IBM uses a mathematical technique known as Shannon Entropy to detect highly random data, such as encrypted data often used by malware.

IBM scans the data as it arrives in the FlashSystem, calculating the Shannon entropy of the data as it enters the cache. The system alerts the storage administrator via IBM Storage Insights if an anomaly is detected. After that, it is up to the warehouse administrator to take corrective action. IBM Storage Insights allows the administrator to configure alert thresholds to fit any environment.

IBM’s Computational Storage approach

Real-time data scanning is computationally expensive. So to prevent its ransomware detection from affecting system performance, IBM only samples 1% of data written. It is statistically significant enough to detect anomalies, but it could be better.

IBM’s announcement of its new FlashSystem anomaly detection says that, sometime later this year, these capabilities will be extended to IBM’s “flash computing storage drives—FlashCore modules—to bring detection as close to the data as possible, further reducing time to detection.” ” So this is something that IBM has been thinking about for some time.

Last year when IBM introduced its latest third-generation FlashCore module, the I talked IBM Fellow and FlashSystem CTO Andy Walls on the technology. Andy describes FlashCore as a computer storage device. This means that the FlashCore module combines NAND flash, DRAM and MRAM for caching and an astonishing amount of compute to provide more functionality than a traditional SSD could. It also takes some of the computationally heavy work, such as compression, off the storage array and does that work on the drive itself. As a result, this is a very efficient and flexible architecture.

The computing part of this memory is ARM processor cores embedded in flexible and reprogrammable embedded FPGA. The primary purpose of this logic today is to manage the QLC flash module. In addition to QLC being enterprise-ready, the first generation of FlashCore also focused on compression. This is one of the most critical attributes of any enterprise data storage array. Compression affects efficiency and cost, a critical concern for anyone in IT.

Andy said that while IBM started with compression and continues to refine that capability, the goal is to offload and accelerate storage applications where it makes sense. It also promises to enable new and exciting capabilities that we haven’t seen yet in a storage array.

Looking forward, the FlashCore Module could help solve the problem of managing unstructured data, performing filtering, searching and scanning at the media level.

Additionally, Andy told me that the processor could potentially be used to deliver real-time statistics on the entropy changes of the data stored on the disk itself. That’s exactly what IBM is hinting could be coming later this year.

Analyst’s Takeaway

While IBM is the only storage vendor to build this level of real-time anomaly detection directly into its storage arrays,

Data protection and cyber resilience features are quickly becoming standard offerings in enterprise storage solutions. Immutable snapshots, for example, are now part of the solutions of almost all top storage vendors. Scanning images using entropy calculations similar to those used by IBM for anomalies is also becoming popular.

The challenge with the approaches used by most storage solutions is that discovery often happens after the fact. Once a recording is found to be corrupted, it may be too late. You have to work backwards in time to find good data to restore. IBM’s approach closes that gap, alerting the user almost instantly when an anomaly is detected. This happens long before the corrupted data is written to the snapshot. That’s exactly the kind of protection you want in your business.

I continue to be impressed by the innovations of IBM’s storage group. The company continues to offer what is perhaps the most advanced and innovative storage technology in the industry. New ransomware detection capabilities with real-time entropy analysis at the volume level only add to that innovation.