The impact of false positives on web application security scanners
Ferruh Mavituna is the CEO at Mavituna Security and the Product Architect of Netsparker. In this interview he discusses what impact false positives have on web application security scanners and what his team is doing to deliver false positive free scans.
What are false positives in web application security?
False positives are like false alarms; they occur when security software reports a vulnerability or security issue that in reality does not exist. In the web application industry false positives are frequently associated with web application security scanners, which are also known as web security scanners or web vulnerability scanners. False positives are also common in several other IT industries and they are always associated with automated tools.
Why web application security scanners typically generate false positives?
False positives in web application security scanners are typically caused by weak signature patterns used in the scanner’s vulnerability checks. For example when the web vulnerability scanner is analyzing the HTTP response and trying to determine if a vulnerability exists or not, it tries to match the content of the HTTP response to the predefined signature pattern. If some text in the HTTP response matches the signature pattern it means that the page in question is vulnerable.
If the web security scanner’s predefined signature pattern is weak, some legitimate text patterns in the HTTP response might trigger the web vulnerability check and the scanner will report a false positive.
What is the impact of false positives in a web application security scan?
The impact that false positives have on web security scans and penetration tests is very negative. To start off with, automated tools such as web application security scanners are used to eliminate the repetitive processes during a penetration test to leave the tester or penetration tester enough free time to concentrate on other tasks, such as identifying logical vulnerabilities. If the security scanner used is known to report a lot of false positives, the tester will be wasting his time manually verifying the scanner findings.
But it is not just about wasting time and slowing down productivity. A lot of false positives can also deter penetration testers which as a result might leave exploitable vulnerabilities undetected. For example the web vulnerability scanner you are using reported 100 SQL injection vulnerabilities on a particular target. You manually checked 55 of them and confirmed they are false positives. In an effort to improve productivity, save time and money and avoid repetitive drudgery work, at this stage you start taking more risks and questioning “What are the probabilities that the remaining 45 SQL injections are false positives?”
Typically at this stage the penetration tester would have already lost trust in the security scanner he is using and start omitting manual verification of vulnerability checks. By taking such an approach the risks of leaving exploitable vulnerabilities undetected and not remediated are very high. In web application security it is important to identify every web application vulnerability and security issue because a malicious attacker only needs to exploit one vulnerability to finish the job.
However the situation is even worse if the user is not an experienced web application security tester. Vulnerability scanners employ very advanced checks, send complicated attacks with different encodings to bypass blacklisting protection etc. So when inexperienced users try to reproduce the identified vulnerability they might fail to replay the exact same attack, or simply cannot manually exploit the identified vulnerability. Since half of the issues reported by the scanner were false positives, when a non seasoned user cannot manually confirm a vulnerability he or she tends to ignore it and mark it as false positive. This obviously diminishes the value of the scanner, and this is the big part of the web application security scanners who cried wolf too often.
What are you doing in Netsparker to eradicate false positives?
When we started designing Netsparker Web Application Security Scanner we wanted to build a tool that helps penetration testers find vulnerabilities without the need to verify detected vulnerabilities, by guaranteeing a false positive free scan.
How do we guarantee false positive free scans? It is the same with what penetration testers are doing manually. Try to exploit the identified vulnerability and if the exploitation is successful, then it is not a false positive. So we included an exploitation engine in Netsparker that automatically exploits detected vulnerabilities in a safe and read only way. If the vulnerability is exploitable Netsparker flags the vulnerability as a real vulnerability therefore penetration testers do not have to manually verify it.
And we didn’t stop there. To confirm that we are on the right track, and that the Netsparker exploitation engine works as advertised we made plenty of tests, i.e. security scans of live web applications and compared the findings to that of other well known scanners. We are proud to say that Netsparker was the only scanner that did not report any false positives, not to mention that it was also the scanner that detected most exploitable vulnerabilities. In fact the team reported several zero day vulnerabilities in open source web applications such as Joomla, MediaWiki and Twiki.
Apart from guaranteeing false positive free web security scans, are there any other advantages in having an exploitation engine in Netsparker?
There are several other advantages in having an exploitation engine in Netsparker.
For example, just like many other professions “showing” instead of “telling” works better. When you show the actual impact an exploited vulnerability might have to the management or to the developers, they will immediately understand how important the problem is. However they tend to ignore you when you only talk about vulnerabilities in a hypothetical level.
Secondly, developers can use the Netsparker exploitation engine to better understand how the vulnerability works. By learning more about detected vulnerabilities and the different ways they can be exploited, developers will get better at remediating them and also in writing more secure code in future projects.
Last but not least, since users do not need to verify the vulnerability scanner findings, web security scans can be performed by junior members of the team, thus senior members of the development team can focus on more important issues, such as fixing reported vulnerabilities.
What is the way forward for Netsparker and other web application security scanners?
Netsparker is already able to automatically exploit and verify the most commonly exploited vulnerabilities such as SQL injection, Cross-site scripting (XSS), Remote code execution, Remote file inclusion and many others. Therefore if Netsparker detects any of these vulnerabilities, rest assured that they are not false positives. Though there are some other vulnerabilities that at this stage Netsparker is still unable to verify automatically; therefore a lot of research is being done to discover new ways on how to automatically verify more types of web application vulnerabilities. And of course at the same time also find new ways to improve the existing verification checks.
As regards the other web application security scanners, I think all of us, including Netsparker should start focusing a bit more on finding new and innovative ways on how to verify our findings and avoid reporting false positives. Web vulnerability scanners are not what they used to be 5 years ago, they came a long way. Many of them can detect vulnerabilities that 5 years ago we didn’t think it was possible to detect using automated means. So from that aspect, all web security scanners as a whole improved. But by increasing automation, false positives also increased. So, what’s the point in increasing automation when penetration testers and web security experts still have to verify a scanner’s findings?
To me it seems as if most software vendors are looking in the wrong direction. I think it is a good to improve coverage and report more vulnerabilities, but the industry should listen to what the market wants; Identifying the most common web vulnerabilities without having to verify them. Else false positives will always be a stigma for automated web application security scanners.