Beyond Security - Automated Black Box Fuzzers
HelpSystems welcomes  PhishLabs  and  Digital Guardian.  

Regarding Fuzzers: Black box, security testing tools

Software security testing with fuzzers

Unlike most dynamic application security testing tools, black box fuzzers do not look for certain attack signatures or attempt to locate known vulnerabilities in products, but rather deliver the widest possible range of unexpected input in order to uncover new and unknown vulnerabilities in network products.

Fuzzing is used either during the development process (in the QA phase) to automatically discover the security holes that still remain in the product, allowing the developer to fix those vulnerabilities before the product is shipped, or as a product certification solution to test applications and devices before acceptance.

Beyond Security’s beSTORM is an automated fuzzer tool, programmed to make an exhaustive search of all possible input combinations in order to test the product for weaknesses, subtle as they may be. Of course, attempting to cover all theoretical input combinations to the program is not practically possible for any non-trivial product. Therefore beSTORM is equipped with prioritization algorithms to enable complete coverage of all inputs that are likely to ‘trigger’ a security hole, all within a reasonable time frame.

These algorithms stem from Beyond Security’s experience and knowledge, accumulated during years of performing manual product audits in search of security holes, and from the Company’s 14 years of research in documenting over 15,500 security vulnerabilities that are posted at – one of the largest security databases on the Internet.

From our research, we have concluded that although the number of possible inputs to any given application is almost infinite, it is usually possible to isolate certain inputs that are likely to lead to the exposure of a security hole. This is how most leading security researchers (including us) find security holes in products – by covering this application input-space. However, this is a very costly procedure, that requires expertise and large amounts of time. The main issue is that while a software development company may not want to spend the huge amount of time and effort needed to uncover such security holes, a smart 16 year-old kid with plenty of time on his hands and a freeware fuzzer tools will go through this procedure for the fame and glory that comes with finding such security holes.

By making this procedure automatic and removing the need for security expertise in running the tool, beSTORM fuzzer standardizes the security checking phase of product development.

Fuzzing goals

There is an obvious need in the information security industry for security testing tools that will enable early detection of vulnerabilities in products that might allow unauthorized intrusion.

One of the most common problems in today’s software market is the numerous attempts by hackers to try and break the security protection of applications. Every single day, new vulnerabilities are discovered and published. The damage caused by these vulnerabilities is extensive and includes harm to the vendor’s image; financial loss due to labor being diverted from other tasks to cope with the problem; and third-party damage caused by privileged information stored in the applications used to hurt the interests of the organization. Direct damages can be in the hundred thousands of dollars and indirect damages can be in the millions of dollars, depending on the sensitivity of the information being exposed.

One cause of vulnerabilities is that application developers are not experts in writing secure code that is resilient to hackers and QA departments can’t maintain an array of often unsupported protocol fuzzers.

There are hundreds of fuzzers written by hackers whose purpose is to break applications. Developers have had to buy and deploy dozens of these unsupported tools to check whether their products can be attacked. This testing becomes labor intensive and time consuming as more lines are written into the application.

Let’s look at an application of medium size with about half a million lines of code. A security specialist scanning over eight hundred lines of code per day to detect vulnerabilities will spend five hundred days to perform a complete assessment of the code. This has an estimated street price of a quarter of a million dollars. The responsibilities of the security specialist not only include spotting known security vulnerabilities, but also to identify new security vulnerabilities that were previously unknown. In addition, this process needs to be repeated whenever a new version of the product comes out, which can be several times a year.

To save company resources that are allocated to the detection and subsequent repair of security vulnerabilities, there is a clear need for an enterprise fuzzer which will test every application’s level of protection against vulnerabilities before it is released to the market.

Due to time constraints caused by the testing of security vulnerabilities in products, many vendors release their products prior to conducting a proper security audit, exposing their customers to potential vulnerabilities. This is becoming unacceptable to customers and they have stepped up their demand that vendors conduct at least some security testing of their products prior to purchase.

In those cases where a product was purchased by a customer and a security vulnerability is discovered, two major problems arise – the vendor has the responsibility to fix the problem and must also provide its customers with the appropriate means to protect themselves against the vulnerability. The customer will usually contact the vendor and demand that the vendor provide an appropriate solution to protect the customer from attack. As mentioned above, these types of vulnerabilities are discovered on a daily basis. As hackers get better at finding vulnerabilities, and the number of vulnerabilities steadily increase, the rise in awareness on the part of customers will inevitably cause them to demand higher standards of security from vendors’ products.

To summarize, there is an obvious need for an automated audit tool, a fuzzer, that provides detailed information on different types of vulnerabilities that is currently unavailable to the average programmer. The tool will conduct product audits, discover known and previously unknown vulnerabilities by doing a thorough test of all possible combinations, and allow for prioritization. All this, without consuming excessive amounts of man-hours or any other accompanying resources.

This is where beSTORM fuzzer enters the picture.

Functional black box fuzzer tool description

In order to interface with the application, beSTORM fuzzer will need to “speak” the “language” the application expects. This language is the network protocol.

beSTORM provides over 200 special testing modules to do this. These modules are programmed according to known protocol standards (HTTP, FTP, SMTP, IMAP, POP3, DNS, DHCP, VOIP). More modules can be programmed on request to accommodate the specific needs of the product being tested.

The beSTORM modules support both non-interactive protocols (such as HTTP, possibly the most common protocol in any Internet environment; used by web servers, communication products, and many others), and interactive protocols such as SMTP (used by mail servers, content filtering devices and anti-virus gateways).

Use of these modules provides two key benefits:

First, it simplifies the adjustments needed to test an off-the-shelf product, since the protocol standard enables building a beSTORM fuzzer module that will work for all HTTP-compatible products.

For example, most network devices use HTTP for configuration, to allow the customer to use a standard web browser to configure the device. Regardless of whether the network device uses a known web server for HTTP communication or if the programmers developed an HTTP-compatible application from scratch, the same beSTORM module can be used for testing. Note, however, that the fact that the HTTP protocol is used does not mean that the relevant beSTORM modules will not attempt attacks “outside” the HTTP protocol – many security holes appear when breaking the standard, and beSTORM will attempt to locate these holes. The definition of the protocol only means that beSTORM will focus on finding HTTP based holes first.

Second, it facilitates quantification of the level of security of the product. By systematically checking the application, we can indicate how secure a certain product really is, by measuring the number of checks we have done compared with other products in the same category. This enables an actual security certification of products, based on an objective and automated scaling – as opposed to today’s manual and error-prone evaluation.

The typical beSTORM user will run beSTORM with a specific module, based on the application being checked.

The user will then set beSTORM to monitor the application that is being audited, and provide the necessary details for the attack: IP address and port (if the attack is on a remote machine), and the protocol used.

The attack can be started and stopped, as well as paused and resumed. A status bar indicates the current test that is being done, and the percentage of the attack that has been completed.

Fuzzer tool performance

beSTORM is expected to find the majority of vulnerabilities that a manual test would reveal within the first 24 hours of automated testing. The full test is expected to take several days to several months – depending on the size and complexity of the application. Distributed capabilities enable to shorten this time considerably by sharing the task between multiple machines. In any event, the testing is completely automated and requires no manual intervention.

Technical specifications

  1. OS: Windows 10, Linux
  2. Memory: 4 GB
  3. CPU: i5

Fuzzer case studies

During the development of beSTORM, Beyond Security has made sure that the fuzzer met its expected goals by constantly taking vendor provided products, and running beSTORM against them. In this list of 13 products, each had at least one security vulnerability that prompted the vendor to release a patch to prevent their customers from being vulnerable to the discovered problem.

Contact us for more information on fuzzers, we’d be happy to help!

Talk to Us