Rising DDoS Threats
Distributed denial-of-service (DDoS) attacks have surged worldwide. These attacks can overwhelm target servers with massive amounts of internet traffic, rendering websites unavailable. Millions of such incidents occur each year, and both their number and scale continue to grow. About one third of website outages are caused by DDoS attacks. Micha Karpowicz, research director at Poland's national cybersecurity and artificial intelligence institute NASK, said, "The goal of DDoS attacks is to create chaos, disrupt organizations, and of course cause economic damage. These attacks are common because they are easy to mount." In a typical DDoS attack, criminals leverage many computers and online devices infected with specially designed malware.
Those devices can include Internet of Things devices such as kitchen appliances, security cameras, and thermostats, and their numbers are increasing daily. Today, the Internet of Things covers more than 10 billion devices, forming a large potential botnet. DDoS attacks usually follow one or both of two patterns. In volumetric attacks, infected devices generate enormous traffic directed at a target network, overloading it and preventing service to legitimate users. In application-layer attacks, requests force the server to perform expensive computations, which likewise overloads it.
How Attack Sources Are Masked
Malware can obscure the origin of requests, which makes attacks difficult to repel. Some services offer attack-for-hire functionality, making attacks easier to mount. Attackers are often disgruntled insiders, activists, commercial rivals, or state actors. Karpowicz noted that during pandemic lockdowns, NASK detected attacks originating from student networks aimed at disrupting online exams. Recently, Karpowicz developed mathematical methods to detect such malicious requests and then block or reroute them. He said, "When you face a problem from the real world, it gives you a lot of inspiration."
Dynamic Fingerprinting
Internet traffic, including requests to websites, is carried in packets. Each packet has a header that uniquely describes its size, source, and destination. Typical defenses inspect these headers to see whether they match a blacklist, then redirect bad packets to harmless destinations. The problem is that defenders must possess the blacklist in advance, and attackers often change headers to evade detection. By contrast, Karpowicz's approach performs real-time detection of traffic patterns. Imagine a series of traffic cameras at city intersections. If you watch a random selection of video clips and suddenly see a large cluster of blue convertibles appear at one location, that is suspicious. You would then observe those vehicles closely to learn more and could pull some of them aside. This approach turns network data into signal data and the task becomes separating different signal sources.
Karpowicz likens this to the "cocktail party problem," where you must pick out what the person speaking to you is saying amid many conversations. He applies linear algebra. He fills matrices with signals, namely packet headers. In the matrix, each row represents a particular type of packets over a time interval, defined by packet size, source, or another factor. In the traffic analogy, rows might represent blue cars, small vans, and vehicles from California. Values in matrix cells indicate metrics that may relate to attack detection. In one matrix a cell might contain a count of flags; in another it might represent bits per second. Karpowicz then decodes which combinations of attributes appear unusually often. Not just blue cars or convertibles, but blue convertibles.
A volumetric attack detected by the FLDX system. (Image credit: Michal Karpowicz)
Once these feature combinations are identified, he maps them back to network rules for firewalls. In the traffic analogy, if he knows a specific brand and model of blue car is responsible, he would create camera filters to find those vehicles immediately. He calls this linear algebra–based method factorization. A paper on the factorization method is under review. Karpowicz implemented the computations in MATLAB. He said, "Linear algebra algorithms are among the fastest known in science, which makes this approach highly efficient."
Traffic Interception and Control
Detecting malicious packets is only part of the problem; defenders must also control them. Karpowicz happened upon a solution by chance. About ten years ago, he worked on a project to improve the energy efficiency of network devices. One goal of that project was to predict traffic and redirect packet flows to routings capable of handling them. When he discussed the project with a colleague in network security, that colleague suggested the approach could be applied to counter DDoS attacks. Karpowicz said, "We kept refining the idea, and bit by bit a new technique took shape." In a paper published last year in the European Journal of Control, Karpowicz described the new technique called adaptive regulation. Typical network traffic controllers lack feedback on their performance and defend by blocking all traffic that might be related to an attack. That often overblocks and can prevent legitimate traffic. Karpowicz proposes controllers with feedback mechanisms so they know whether they are redirecting the correct packets. He said, "Network security presents me with major scientific research challenges, especially mathematically." Theory and practice complement each other. "That is what keeps me going. I have a hunch there is something valuable here and a problem that needs solving."
These detection and control systems form the basis of the FLDX service developed at NASK, which is now patented. It can detect an attack within 5 seconds and begin mitigation within 10 seconds.
Real-time DDoS detection by the FLDX system. (Image credit: Michal Karpowicz)
NASK has deployed FLDX across Poland using distributed virtual machine clusters to protect national networks. Karpowicz said, "What makes the overall solution unique is that it is not only a cybersecurity system but also a research platform. It lets you write your own detection and suppression algorithms in MATLAB. It also gives access to the traffic data we collect in MATLAB, and you can apply data processing, signal processing, and machine learning and artificial intelligence techniques to the samples we provide." Both the detection and adaptive regulation algorithms use machine learning, specifically a semi-supervised learning approach that does not require extensive manually labeled data to train. Karpowicz said, "We encode some expertise into the learning algorithms and then let them discover patterns on their own. Their job is to find what is happening in the world. They can find statistical regularities in large data sets, for example discovering clusters of a certain type of vehicle. This is one of the best use cases for machine learning."
Before deployment, it is not necessary to collect a large training dataset to tune the system. The system begins operating the moment it is installed. The detector uses a voting scheme to identify anomalous flows. It also optimizes matrices by solving systems of equations. Even when network behavior is nonlinear, linear equations can capture most patterns. Researchers and engineers can access NASK's datasets via MATLAB, and because many technical universities already use the dataset, that access is convenient. NASK's dataset is unique because it contains traffic data collected nationwide at a very high sampling rate.
FLDX visualizes different traffic streams. (Image credit: Michal Karpowicz)
NASK's research focuses on traffic dynamics, target vulnerabilities, and attack sources, frequency, and methods. They use FLDX to protect public and commercial clients, including schools and vaccination registration services. Gaining the trust of colleagues in network security proved to be a significant challenge. Karpowicz said, "In the network security engineering department, scientists need time to earn respect. You must show you understand what they do and that you are doing useful work." With FLDX, "you can go straight to the front line. That is the special aspect of NASK. The path from lab to technology and deployment is very short." Karpowicz is now collaborating with other institutions, including MIT and the University of Technology Sydney. He said, "I hope NASK becomes known globally for work in cybersecurity and artificial intelligence." The project has already shown a promising start.
