How To Write Penetration Test Reports

There are thousands of books written about information security and pen testing. There are hundreds of hours of training courses that cover the penetration testing process. However, I would happily wager that less than ten percent of all the material out there is dedicated to reporting. This, when you consider that you probably spend 40-50% of the total duration of a pen test engagement actually writing the report, is quite alarming.

It’s not surprising though, teaching someone how to write a report just isn’t as sexy as describing how to craft the perfect buffer overflow, or pivot round a network using Metasploit. I totally get that, even learning how the TCP packet structure works for the nineteenth time sounds like a more interesting topic.

Why is a penetration test report so important?

Never forget, penetration testing is a scientific process, and like all scientific processes it should be repeatable by an independent party. If a client disagrees with the findings of a test, they have every right to ask for a second opinion from another tester. If your report doesn’t detail how you arrived at a conclusion, the second tester will have no idea how to repeat the steps you took to get there. This could lead to them offering a different conclusion, making you look a bit silly and worse still, leaving a potential vulnerability exposed to the world.

Bad: “Using a port scanner I detected an open TCP port”.

Better: “Using Nmap 5.50, a port scanner, I detected an open TCP port using the SYN scanning technique on a selected range of ports. The command line was: nmap –sS –p 7000-8000.”

The report is the tangible output of the testing process, and the only real evidence that a test actually took place. Chances are, senior management (who likely approved funding for the test) weren’t around when the testers came into the office, and even if they were, they probably didn’t pay a great deal of attention. So to them, the report is the only thing they have to go on when justifying the expense of the test. Having a penetration test performed isn’t like any other type of contract work. Once the contract is done there is no new system implemented, or no new pieces of code added to an application. Without the report, it’s very hard to explain to someone what exactly they’ve just paid for.

Who is the report for?

While the exact audience of the report will vary depending on the organization, it’s safe to assume that it will be viewed by at least three types of people.

Senior management, IT management and IT technical staff will all likely see the report, or at least part of it. All of these groups will want to get different snippets of information. Senior management simply doesn’t care, or doesn’t understand what it means if a payment server encrypts connections using SSL version two. All they want to know is the answer to one simple question “are we secure – yay or nay?”

IT management will be interested in the overall security of the organization, but will also want to make sure that their particular departments are not the cause of any major issues discovered during testing. I recall giving one particularly damming report to three IT managers. Upon reading it two of them turned very pale, while the third smiled and said “great, no database security issues then”.

What should the report contain?

Sometimes you’ll get lucky and the client will spell out exactly what they want to see in the report during the initial planning phase. This includes both content and layout. I’ve seen this happen to extreme levels of detail, such as what font size and line spacing settings should be used. However, more often than not, the client won’t know what they want and it’ll be your job to tell them.

• A Cover Sheet. This may seem obvious, but the details that should be included on the cover sheet can be less obvious. The name and logo of the testing company, as well as the name of the client should feature prominently. Any title given to the test such as “internal network scan” or “DMZ test” should also be up there, to avoid confusion when performing several tests for the same client. The date the test was performed should appear. If you perform the same tests on a quarterly basis this is very important, so that the client or the client’s auditor can tell whether or not their security posture is improving or getting worse over time. The cover sheet should also contain the document’s classification. Agree this with the client prior to testing; ask the m how they want the document protectively marked. A penetration test report is a commercially sensitive document and both you a nd the client will want to handle it as such.

• The Executive Summary. I’ve seen some that have gone on for three or four pages and read more like a Jane Austen novel than an abbreviated version of the report’s juicy bits. This needs to be less than a page. Don’t mention any specific tools, technologies or techniques used, they simply don’t care. All they need to know is what you did, “we performed a penetration test of servers belonging to X application”, and what happened, “we found some security problems in one of the payment servers”. What needs to happen next and why “you should tell someone to fix these problems and get us in to re-test the payment server, if you don’t you won’t be PCI compliant and you may get a fine”. The last line of the executive summary should alw ays be a conclusion that explicitly spells out whether or not the systems tested are secure or insecure, “overall we have found this system to be insecure”. It could even be just a single word.

  A bad way to end an executive summary:

  “In conclusion, we have found some areas where security policy is working well, but other areas where it isn’t being followed at all. This leads to some risk, but not a critical amount of risk.”

  
  

  A better way: “In conclusion, we have identified areas where security policy is not being adhered to, this introduces a risk to the organization and therefore we must declare the system as insecure.”
  

• Summary of Vulnerabilities. Group the vulnerabilities on a single page so that at a glance an IT manager can tell how much work needs to be done. You could use fancy graphics like tables or charts to make it clearer – but don’t overdo it. Vulnerabilities can be grouped by category (e.g. software issue, network device configuration, password policy), severity or CVSS score –the possibilities are endless. Just find something that works well and is easy to understand.
• Test Team Details. It is important to record the name of every tester involved in the testing process. This is not just so you and your colleagues can be hunted down should you break something. It’s a common courtesy to let a client know who has been on their network and provide a point of contact to discuss the report with. Some clients and testing companies also like to rotate the testers assigned to a particular set of tests. It’s always nice to cast a different set of eyes over a system. If you are performing a test for a UK government department under the CHECK scheme, including the name of the team leader and any team members is a mandatory requirement.

• List of the Tools Used. Include versions and a br ief description of the function. This goes back to repeatability. If anyone is going to accurately reproduce your test, they will need to know exactly which tools you used.

  

• A copy of the original scope of work. This will have been agreed in advance, but reprinting here for reference purposes is useful.

• The main body of the report. This is what it’s all about. The main body of the report should include details of all detected vulnerabilities, how you detected the vulnerability, clear technical expiations of how the vulnerability could be exploited, and the likelihood of exploitation. Whatever you do, make sure you write your own explanations, I’ve lost count of the number of reports that I’ve seen that are s imply copy and paste jobs from vulnerability scanner output. It makes my skin crawl; it’s unprofessional, often unclear and irrelevant. Detailed remediation advice should also be included. Nothing is more annoying to the person charged with fixing a problem than receiving flakey remediation advice. For example, “Disable SSL version 2 support” does not constitute remediation advice. Explain the exact steps required to disable SSL version 2 support on the platform in question. As interesting as reading how to disable SSL version 2 on Apache is, it’s not very useful if all your servers are running Microsoft IIS. Back up findings with links to references such as vendor security bulletins and CVE’s.

  

Final delivery : Just because its finished doesn’t mean you can switch off entirely. You still have to get the report out to the client, and you have to do so securely. Electronic distribution using public key cryptography is probably the best option, but not always possible. If symmetric encryption is to be used, a strong key should be used and must be transmitted out of band. Under no circumstances should a report be transmitted unencrypted. It all sounds like common sense, but all too often people fall down at the final hurdle.