Data growth has taken the tech industry by storm – and there’s no sign of stopping it. The ubiquitousness of connected devices, applications, and social media platforms has ingrained itself into the lives of billions, accelerating the accumulation of Big Data at a breakneck pace. By 2020, it’s predicted that 1.7MB of data will be generated every second for every person on the planet. Multiply that by 7.7 billion, and Big Data may now seem like an inadequate description.
For many cyber experts, the advent of this exponential production of data, and the industry’s quick response to adapt to it, comes as no surprise. Moore’s Law, which dictates that the number of transistors on a microchip will double annually, has so far proven to be true, albeit some claim it’s coming to an end soon. Computing power and the size of microprocessors have a strong, yet inverse relationship — the more powerful the device, the smaller the chip is that is powering it. Spurred on by chip-making giants such as Intel, modern engineering is delivering nanochips with greater numbers of transistors in recent years. However, the scalability of these ever-shrinking computer chips is reaching a breaking point — threatening the conventional wisdom that Moore’s Law once represented. Earlier this year, Intel chief engineering officer Murthy Renduchintala expressed concerns over the company’s ability to cope with the practical challenges posed by Big Data.
“It’s no secret that Intel has struggled with 10 nanometers,” Renduchintala said. “And what I have found in discussions with many… is the perception that Intel’s process innovation has slowed down during this time.”
This slowdown in innovation was inevitable, as the size of any physically existing object will soon find its limitation. In response to this technological hurdle comes the cloud, which has increased in use and popularity over the last decade. The idea of ridding an organization of the need for on-prem servers, which are inherently limited by space, has garnered a natural appeal in the wake of Big Data. As this shift has taken place, older, more outdated IT security platforms and on-prem servers have not been able to adequately evolve alongside Big Data production or cloud-based storage methods, preventing stakeholders from leveraging data in a way that makes sense for them.
Both legacy SIEM and central log management tools are excellent data gathering and detection methods but fail to comprehensively or intuitively associate pieces of data together, making the gobs of information produced by a company effectively useless or too time-consuming to understand. Migrating to the cloud requires more than a transfer of data, but rather a complete re-evaluation of how data-producing software should be tracked and stored.
SOAR, alternatively, has been able to partially address this problem, but is still bound by a more traditional framework that was not originally built for the cloud. SOAR improves upon the shortcomings of older SIEM’s by integrating threat-hunting and vulnerability testing, allowing for intelligent predictions or evaluations of suspicious activity and automatically responding accordingly. The automatic response characteristic of SOAR can render it useless, however, when unnecessary shutdowns are provoked by its detection of perceived threats, making it an inefficient SIEM. Overall, these solutions have worked fairly well, but still lack intuitive programming that is both native to the cloud and designed to skillfully interact with all services in a given network.
The key to harnessing the power of the cloud is found in taking SOAR’s capabilities to a greater level of network integration, producing what Fluency re-names as Cloud Orchestration, Automation and Response, or simply COAR rather than SOAR. This type of cloud-based data storage solution allows for vast accumulations of Big Data to be tracked across multiple APIs simultaneously. In this capacity, COAR operates as a central log management system engineered for the cloud, rather than a work-around solution based on an old on-prem server’s network. This kind of next generation SIEM shift is substantial when considering how Big Data is revolutionizing how and why potential threats are detected. COAR is able to track 12 million events per second sustained and with absolute ease – illustrating the kind of power a cloud-based advanced analytics tool can generate.
Increased power and storage capacity aren’t the only drivers of cloud migration. Data compliance is of major concern as organizations worldwide continue to produce vast amounts of personally identifiable information (PII) at exponential levels. The right of a customer or member of an organization to be “forgotten” has achieved paramount importance in the development of privacy laws in EU member states, with the passing of GDPR being the most prominent example.
Across the pond, California and New York are set to follow suit with CCPA and SHIELD legislation placing limitations on Big Tech’s level of data ownership, in addition to the databases of countless other businesses and institutions. Compliance enforcement comes at a time when data growth is at an all-time high, exerting even greater pressure on businesses large and small to securely and efficiently store PII, let alone healthcare organizations with PHI. Cloud-based log management offers a storage alternative with built-in compliance features and greater storage capacity, providing an easier pathway to compliance rather than overhauling pre-existing systems.
Moore’s Law may now be close to obsolete, and the forces that drove it to extinction are only growing stronger. The question of whether additional transistors can be squeezed onto a microchip may continue to persist but is ultimately a conversation devoid of innovative or practical thinking. Cloud migration is fueled by this shift, creating a greater need for sophisticated cloud orchestration tools and threat detection methods that reach far beyond one-off notifications or automatic shutdowns. Ultimately, the cloud offers a significant advantage when done correctly, but requires a large amount of expertise that most companies don’t have — and will soon need to.
