Home Features Botnets and Machine Learning: A Story of ‘Hide and Seek’

Botnets and Machine Learning: A Story of ‘Hide and Seek’

by Brian Sims

Botnets have been a major issue in the cyber security industry. As techniques to stop them have come into play, so the bots have evolved ways of navigating their way around these solutions. Leonidas Plagakis explains why machine learning may be the industry’s best chance of realising a future-proof method of defence.

Malware authors have always been trying to update their software and evolve their techniques in order to take advantage of new technologies and bypass security measures. Botnets are a perfect example of how cyber criminals have managed to accomplish just that over the last decade. Their widespread and severe consequences have transformed botnets into one of the most significant and destructive threats in the cyber security landscape. They’re responsible for many large-scale and high-profile attacks.

Examples of attacks performed by botnets include Distributed Denial of Service (DDoS) episodes, personal or classified data theft, spam campaigns, cryptocurrency mining and fake news being spread across social media platforms. Moreover, there’s an exponential increase in attacks that result from Crime-as-a-Service offerings, which usually include botnets that are rented or sold to either individuals or groups lacking experience or technical skills, but who wish to perform nefarious activities.

It’s abundantly clear, then, that formulating and actioning security measures against botnets is crucial for an organisation’s well-being and the protection of vital and sensitive private data.

Command and Control mechanisms

Figure 1: Centralised Command and Control infrastructure (IRC, HTTP)

Figure 1: Centralised Command and Control infrastructure (IRC, HTTP)

One way in which to categorise botnets is by the technology they adopt for their Command and Control mechanism. In terms of Command and Control, the architecture of a botnet can be either centralised (see figure 1) or decentralised (see figure 2). In the first category, the bots communicate with one or more servers using the client-server model. The first generation of centralised botnets used IRC channels to communicate with the Command and Control server. However, due to the single-point-of-failure nature of centralised architectures, the criminals started developing botnets that were based on peer-to-peer (P2P) communications, thereby overcoming the problem of the previous generation of botnets.

Indeed, P2P botnets, having the advantage of resilience and robustness, formed an even greater threat to organisations, but they also have two major drawbacks. To begin with, their maintenance is very difficult because of the complexity of their deployment and development. Second, since there’s no longer a central Command and Control server, the herder might not have full control of the botnet any longer.

Figure 2: Decentralised Command and Control infrastructure (P2P)

Figure 2: Decentralised Command and Control infrastructure (P2P)

The solution adopted by malware authors was to return to the centralised architecture model. However, they didn’t use the IRC protocol for the communications between the herder and the bots. Rather, the HTTP protocol was used instead. The advantage and strength of this solution is that the HTTP protocol is commonly used by legitimate, non-malicious web applications and services. The attackers are able to embed their traffic in non-malicious, legitimate HTTP traffic and hide Command and Control commands among normal network activities. This gives HTTP-based botnets their great advantage which is their ability to remain hidden ‘under the radar’ and perform their nefarious operations undetected.

Many researchers have dedicated their efforts to the study and analysis of HTTP botnets and finding accurate ways in which to detect them. A large number of researchers approach the problem by employing behaviour-based detection techniques since the traditional signature-based systems are often easily bypassed by new generations of malware. More specifically, the analysis of network traffic and its characteristics (not necessarily the packets’ payload) can provide very insightful information as to whether a network flow or packet is benign or if it’s part of a botnet’s Command and Control mechanism, even in those cases where traffic is encrypted.

Examples of traffic characteristics that could prove useful are the flow duration, the total number of packets exchanged in a flow, the length of the first packet in a flow and the median of payload bytes per packet.

Key role for machine learning

Machine learning plays a key role in this approach as behaviour-based botnet detection systems are usually built using a classification model that’s trained on a dataset with specified features (ie a set of network characteristics in our case). This classification model is able to identify both efficiently and accurately malware-generated traffic when certain behaviour patterns are met.

Apart from classification, more machine learning tools (eg feature extraction) could be used in order to make our system as accurate and fast as possible. In general, novel attacks deployed by newer or more advanced versions of existing malware can be prevented using this approach as it’s based on malware signatures.

Unsurprisingly, attackers started looking for ways and techniques that would allow them to overcome detection systems’ progress and bypass behaviour-based detection. Adversarial machine learning is an emerging technique that, among others, could target and evade security systems using machine learning for dealing with malicious activities. Typically, its functionality is based on taking advantage of classifiers’ weaknesses. For example, there might be a spate of instances (ie flows/packets) that the classifier might not be able to describe well, so instances that belong to that space will be misclassified.

Another kind of attack that can be performed against systems based on machine learning is when adversaries attempt to attack the training phase of classification. That is, they try to inject adversarial training data to the classification model. This eventually leads to a model that labels malicious instances incorrectly as being non-malicious, thus increasing the number of false negatives and leaving the system vulnerable.

Obfuscation techniques used by attackers should also be taken into consideration when implementing detection systems based on behaviour. More specifically, attackers might attempt to convert the value of certain attributes and characteristics of network traffic flows that are indicative of malicious activity into values that are typical and normal for non-malicious flows, thereby evading security measures. Therefore, if the obfuscated features are used by the classification system, the malicious flows will have a greater chance of bypassing the detection system.

Security Best Practice

Best Practice for organisations in terms of security is to always be up-to-date with the current trends in the cyber threat landscape as it’s a field that changes constantly and radically. Machine learning has proven to be an extremely powerful ally in the battle against certain kinds of malware and, currently, seems to be the ideal method for keeping up with the evolution of threats, both in terms of detection accuracy and efficiency.

Of course, behaviour-based systems have the drawback of false positives, but the benefits of this approach are more than enough to ignore that disadvantage.

However, when employing behaviour-based systems, organisations should never overlook the complexity and difficulty of building such systems and the caveats that come with such a solution, some of them mentioned above (ie adversarial machine learning, obfuscation of features, etc).

Technical expertise, along with patience and the ability to gain insight, are probably the most important values professionals and organisations should be equipped with in order to successfully deploy and manage such complex systems that will help them adjust to today’s threat landscape and continue operating in a secure environment.

Leonidas Plagakis is a Security Engineer at RiverSafe

You may also like