Using big data analytics to combat cyber threats

Companies are looking for more comprehensive resources to defend themselves against malware and cyberattacks

Vast stores of data make inviting targets.

But with the widening adoption of Hadoop, R and in-memory processing, companies are starting to realize that big data analytics can help detect potential security threats.

Capgemini’s Stephen Harris says organizations are looking to big data frameworks to get clues to protect not only the data but also their reputation and brand.

“More organizations are looking to big data frameworks to analyze and visualize their data to get contextual indicators or clues to protect not only the data but also their reputation and their brand,” says Stephen Harris, North American big data lead at Capgemini.

Malware and cyberattacks have become increasing concerns among companies. Many of them struggle to predict and mitigate threats, which can spring and evolve quickly. Companies also have some concerns about the physical security of their infrastructure.

The result is that they are in search of more comprehensive resources to address these challenges. Big data analytics, particularly when they are coupled with machine learning, represent a logical solution because it allows companies to consider multiple threat scenarios and determine the best response.

“You have to have platforms that are able to mine and aggregate data from multiple sources, not just data inside your physical walls or inside your network or infrastructure,” says Harris. “Technologies like Hadoop or Caldera or Hortonworks allow you to pull all that data into a single location and then run algorithms using R and other tools to visualize a potential threat, whether IT security or physical threats.”

Companies currently have to use two separate tool sets to protect themselves from physical and cyber threats. However, Harris says that over the next year or so there should be an increasing number of integrated platforms based around big data.

“There is a level of maturation that will come because customers want to have a fully integrated security suite that will provide them with both, one that will use 3-D visualization and graphics to overlay a potential cybersecurity threat and physical security threat,” Harris says.

Even without integrated tools, IT can still take advantage of existing big data analytics to identify potential threats. But Harris warns that companies’ success will depend on the quality of their big data tools. “The accuracy of that prediction will heavily depend on the amount of information they are pulling into their environment and the accuracy of the algorithms they leverage,” he says.

Here are a few tips for getting the most out of big data security analytics:

Look beyond logs: The log data produced by traditional security products should be part of the analytics, but it is only a starting point.

“We still need firewalls, intrusion detection systems, endpoint security and other point-specialized security products to block the drive-by attacks against known exploits, but we need additional technology to recognize when someone has slipped by these defenses using unreported vulnerabilities or other methods,” says Seth Goldhammer, director of product management at LogRhythm Inc. “While log activity can provide the clues this has occurred, trying to apply big data technology to this could only exacerbate the issue helping to hide the clues indicating truly concerning behavior.”

Break the rules: “Traditional rule-based systems are not equipped to discover the most advanced threats, since they typically detect signature-based malware or attack scenarios,” says Idan Tendler, CEO and co-founder of Fortscale Security Ltd. “These systems usually analyze a limited number of security sensors in real time, without using historical data.”

Big-data security analytics can analyze a wide variety of data sources to detect the unexpected. Companies can set profiles of the expected behavior of individuals, equipment and systems, and then look for deviations. For example, if an employee’s workstation logs onto the network at 1 a.m., either that person can’t sleep, there is a physical intruder logging on, or some malware has activated the machine. Comparing that log-on data with the building security system, network activity, and the data and systems accessed can indicate which it is. IT staff members can never determine as many possible scenarios as analytics and artificial intelligence.

“Most security analysts lack the technical know-how to develop sophisticated cybersecurity analytics on top of big data platforms to meet their specific security needs,” says Tendler. “Machine-learning algorithms can automatically pinpoint suspicious behavior or discover new patterns, with no pre-defined rules, heuristics, signatures or thresholds.”

Keep the score: Firewalls, antivirus and intrusion detection systems can spit out the number of items detected and blocked, but not what gets through or the damage done.

“Track specific metrics for your incident response team on an ongoing basis such as average time to identify and remediate attacks, number of incidents responded to, number and cost of successful attacks, time/cost of unplanned remediation, and number of stolen identities,” says Matthew Gardiner, senior manager, RSA.

Correlate attacks and risks: Big data can help determine which threats are the most serious. IT can then set priorities for a response. “It is clear, given the number of breaches reported in the news, which traditional security solutions are not meeting with the types and volume of attacks in today’s threat landscape,” says Goldhammer. “From state-sponsored hacking, hacktivists, spyware, ransomware, organized cybercrime, and so on, it’s no longer a question of if you will be hacked, but when.”

Drew Robb
Originally from Scotland, Drew Robb has been a freelance writer for 20 years, specializing in IT, engineering and business. He is the author of Server Disk Management in a Windows Environment by CRC Press.
Drew Robb
Drew Robb
Tags: Data Center,IT Security,Technology