An Integrated Approach for Cyberthreat Monitoring Using Open-source Software

As cyberthreats evolve each day, detecting these threats is becoming more important. Recent studies show that the time between a breach occurring and being detected is, on average, 229 days.¹ Since 229 days is a long time, an average company will not respond to an attack in a timely manner and will not mitigate its effects if there is no extra effort used for detection. This number shows there is a lack of accurate cyberthreat monitoring for most companies, and it is mostly because necessary monitoring mechanisms are not placed correctly and/or do not work seamlessly. Additionally, most companies focus on prevention rather than detection. Since prevention methods for most advanced threats fail, the need for detection is becoming more important each day. There are also security investment cost concerns for most small and medium-sized businesses (SMBs). While a not-so-skilled attacker can easily hack a corporate IT infrastructure by using a US $500 exploit that is being sold in an underground market, the cost for preventing or detecting these attacks is not proportional with this low cost when a company chooses to buy and install commercial solutions.

For these types of needs, open-source software presents numerous possibilities since it has great community support and is cost-effective, especially for SMBs. With its advantages, a company may choose to build its security infrastructure using open-source solutions.

An average breach typically consists of seven main steps (figure 1), as modeled by Lockheed Martin and called the Cyber Kill Chain.² If organizations want to adequately detect attacks, these steps are important starting points to address necessary monitoring needs.

By using this Cyber Kill Chain abstraction, there is a chance to detect an adversary if necessary detection mechanisms are in place, executed and correlated correctly for each step. For example, if a network intrusion detection system (NIDS) is monitoring the active remote connecting IPs for possible command and control (C&C) activity using threat intelligence feeds, it can easily alert the security staff for needed blocking actions. Again, if a host-based intrusion detection system (HIDS) can monitor the host activities (e.g., integrity checking for critical system files), it can alert the security team when a malicious event occurs on the host.

Network Intrusion Detection System

An NIDS performs analysis of passing traffic on the entire subnet and matches the traffic that is passed on the subnet to the library of known attacks. By using it effectively, an NIDS can help an organization be alert for attack attempts at various steps of the Cyber Kill Chain model. For example, if there is malware using malicious URLs/IPs, the NIDS will catch it from network traffic using its signatures, relating to step 6. And if its vulnerability signature matches with current active traffic, this would be related to step 4.

Security Onion (SO) is a Linux distribution created for intrusion detection, network security monitoring and log management. It is based on Ubuntu GNU/Linux and contains well-known open-source network security software such as Snort, Suricata, Bro and Sguil in an integrated approach.³ Since they are integrated with scripts for ease of use, it is very easy to install and start to use via its graphical user interface (GUI). It has three install options: standalone, sensor and server. If one wants to install sensor and server onto the same machine, the standalone mode can be used. For large networks, distributed installation could be the right answer for easy maintenance and central management of distributed sensors using built-in SaltStack configuration management support.

While using SO, one must use either port mirroring or network tap hardware devices to mirror all network traffic to the SO sensor machines. After the process of installation and enabling necessary settings, NIDS software components will start to see and analyze traffic against threats using built-in threat signatures.

Effective placement of the sensors in the network is also an important consideration to get a clear and accurate view of the network.

Figure 2 is an example of an NIDS alert reporting window using the Sguil application.

SO also comes with a useful log search tool called enterprise log search and archive (ELSA). It is built on syslog-ng, MySQL and Sphinx. It provides an easy-to-use, web-based query interface similar to the well-known Splunk application. It also supports email alerting, scheduled queries and graphing. Historical events queries and statistical results can be gathered using ELSA.

One of the most notable features of SO is its packet capture capability using the netsniff-ng tool. When choosing to configure the packet capture feature, whenever an intrusion detection system (IDS) alarm is generated, one can easily see and analyze the packet captures of the related event for detailed analysis. Since capturing all traffic consumes a large amount of hard disk capacity, organizations should plan carefully before installing their system. Network bandwidth value and log retention practices can be used as starting points for these plans.

Host-based Intrusion Detection Systems

HIDS is an intrusion detection system that monitors and analyzes the internals of a computing system.

Different from NIDS, HIDS monitors for host-based activities. For example, it can monitor the integrity of critical files, network connections, system logs, local firewall status, rootkit detection, brute-force attempts to the system and more.

Using HIDS effectively can help an organization detect attack attempts in steps 5 and 7 in the Cyber Kill Chain. For example, step 5 uses the HIDS file integrity monitoring feature, which can detect whenever malware corrupts a system file or write itself to the registry and raise an alert.

One of the more well-known open-source HIDS projects is OSSEC⁴ (figure 3). It supports Windows, Linux, Mac, BSD, VMware ESX systems and more.

Its capabilities include centralized management, real-time and configurable alerts, agentless monitoring, and integration with commercial security information and event management (SIEM).

It is also easy to customize since it is open source. OSSEC can be customized for purposes such as USB device white-listing and software vulnerability scanning.

For deployment, an OSSEC server installation is needed. After this step, an agent can be installed on any host, and, given the agent key and IP information of the server, the agent will start to monitor the host it has installed and send the logs to the OSSEC server. This process can be automated for large deployments using methods such as Windows Management Instrumentation (WMI) and Puppet. There is also a project called Auto-OSSEC⁵ for easy deployment.

Honeypots

Many adversaries start their malicious activities by scanning external subnets and trying to exploit the weakest machine among an organization’s public-facing hosts. A honeypot can be used to trick the adversary and entice him/her to try to exploit it. While an attacker is attempting a breach, honeypots report the event to the central security monitoring servers and help defend the production infrastructures. When used effectively, honeypots can help organizations detect attack attempts in step 1 of the Cyber Kill Chain.

There is a Linux distribution called HoneyDrive, which is a bundle of honeypot software and is easy to use to get started. Another well-known open-source honeypot is Dionaea. It is a malware-capturing honeypot initially developed under The Honeynet Project’s 2009 Google Summer of Code (GSoC).⁶ Dionaea aims to trap malware exploiting vulnerabilities exposed by services offered over a network and, ultimately, to obtain a copy of the malware. It captures exploits offered over a network and stores details of these harmful events such as source IP, attack type and downloaded binary for later analysis. While an attacker is mounting an attack within this honeypot, the organization can launch a proactive defense using this information. By default, Dionaea supports Server Message Block (SMB), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Trivial File Transfer Protocol (TFTP), Microsoft SQL Server (MSSQL) and Session Initiation Protocol (SIP).

Figure 4 shows Dionaea-captured malware and related hashes. These hashes can be submitted to Virustotal.com for more detailed analysis.

Figure 5 shows the malware source IPs for blocking purposes.

A major concern for honeypots is their correct placement in a network. While a public-facing honeypot is good for external attacks, extra internal honeypots for detecting lateral movements are also an effective effort.

Integrating Open-source Software and Making It All Work

One of the greatest challenges in cybersecurity is managing all security efforts centrally and making them easy to use. When an organization has numerous log sources and security systems, monitoring and managing them becomes more complex. This is a significant challenge for intrusion detection efforts, since all security logs should be carefully analyzed. If one is not able to detect an intrusion within a reasonable time frame, it can lead the entire system into a precarious situation. Therefore, using detection services effectively and in a combined manner is important for a well-protected IT infrastructure.

For central monitoring and dashboard purposes, ElasticSearch, Logstash and Kibana (ELK)⁷ stack are well-known open-source solutions. They consist of three major components. ElasticSearch is a Lucene-based search server and it provides a distributed full-text search engine. Logstash is an easy-to-use log collection framework that works well with ElasticSearch. Kibana is the ultimate monitoring web user interface and helps visualize all the logs that come from Logstash and are indexed by ElasticSearch.

Using this stack, HIDS, NIDS and honeypot systems can send their data to ELK, and an analyst can correlate these data, create a dashboard for central monitoring and start taking quick actions (e.g., blocking attacker IPs using honeypot data, correlating HIDS and NIDS data to increase accuracy of a detected attack according to kill-chain abstraction). Unless using security data effectively, all the logging efforts are useless.

Conclusion

With today’s fast-growing cybersecurity needs, building an effective cyberdefense infrastructure is a big challenge for many organizations. Building a solid and accurate monitoring infrastructure will decrease the time to detect attacks since it will help gain the necessary insights from systems. A strong monitoring infrastructure will be able to correlate and use data accurately, enabling the security team to only work on important and accurate alarms.

This article provides an overview of open-source tools that can be used to deliver enhanced cyberthreat detection and defense to suit the resources of most cyberdefenders. In addition, these open-source software offerings provide significant flexibility and the benefit of a large support community. This can help to level the playing field for those tasked with guarding an organization and its “crown jewels.” On the other hand, to utilize flexibility and low-budget advantages of open-source security solutions, the security team in charge of installing these solutions should know what they are doing and enjoy the open source community and culture. But open source is also a risk for companies that have small security staffs. Especially in the long term, a product that is no longer supported must be managed by the organization, resulting in unique challenges.

Endnotes

¹ Mandiant, 2014 Threat Report, M-Trends, April 2014, http://dl.mandiant.com/EE/library/WP_M-Trends2014_140409.pdf
² Lockheed Martin, Cyber Kill Chain, http://cyber.lockheedmartin.com/solutions/cyber-kill-chain
³ Security Onion, http://blog.securityonion.net/
⁴ OSSEC, http://ossec.github.io/
⁵ Kennedy, D.; “Tool Release: Auto-OSSEC—Automated OSSEC Deployment,” Binary Defense Systems Update blog, 5 October 2015
⁶ The Honeynet Project, Google Summer of Code 2009, http://honeynet.org/gsoc2009
⁷ Sissel, J.; “An Introduction to the ELK Stack,” Elastic, http://www.elastic.co/webinars/introduction-elk-stack