At Nixu we notice at times that there's a disconnect between the two technical parties in a security assessment: the developer and the security specialist. In this blog post we want to write a guide for any non-security IT professional, or even for those who are well versed in security, about how to prepare for a security assessment done for your application or system.
Getting it right from the beginning - security what?
The obvious point where to start is the terminology: security assessment, security audit or penetration test (pentest)? We have heard all of these used to communicate the same idea: analyzing application's security. The gory details of the terminology aren't so obvious.
Technically, security audit means strict checking against some predefined and standardized level of security requirement, which can be for example OWASP ASVS (Application Security Verification Standard) that we often use at Nixu. To provide a nice puzzle for the developer, the term used by your manager, security company sales and security company specialists may differ wildly depending on their background. For clarity, in this text "security specialist" refers to the technical person from a security company conducting the assessment.
Penetration test is usually a security assessment where the pentester is given fairly free hands to break into organization's infrastructure, specific system or even to physical premises. In a pentest, report provided is usually shorter and gives detailed description of how (s)he succeeded, but varying amount of other observations. Security audit and security assessment usually have a narrow scope and the reports are lengthier, containing all findings in detailed manner. Security audit reports usually also include controls that were found to be in good order.
- Make sure you understand whether this is an actual penetration test, a security audit or a more general security assessment of your application.
- If there is a standard used, make sure to get and check the documentation of the standard. Otherwise it can be hard to understand what the security specialist is trying to tell you.
Pieces of the security assessment puzzle
No matter what kind of security assessment is done, it usually consists of few identifiable separate parts. These parts may be moving forward simultaneously or in an order defined by team schedules. When the assessment project is about to begin, these schedules will be agreed on.
These possible parts are:
- Threat modeling. The goal of threat modeling is to provide insight to key risks related to the application. The report may or may not contain business impact considerations about risks listed. Methods vary, STRIDE being the best known and probably the most used. In practice this means that some documentation is provided to the security specialist who usually also conducts workshops with the developers. Report may contain remediation recommendations or visualizations.
- Source code security review. When aspiring for the highest level of security, an application security assessment is done together with source code security review. The security specialist can then search for vulnerabilities straight from the code or verify black-box findings from there. Usually targets of this kind of scrutiny are more complex applications with tens or hundreds of thousands of lines of code. This means that review in almost all cases quite software-driven. Software in this case might mean glossy commercial code scanning tools, proprietary programs or just plain ol' grep. Code review is prone to false positives unless there is plenty of time available, but it does reveal more bugs than plain black-box testing.
- Port scanning. The most common staple of security work, port scanning, is nothing magical but only checking which ports can be seen as open from the Internet. You are better off checking this yourself before the assessment to reduce unnecessary emails and findings, because in most cases some unneeded development related ports have been forgotten open.
- Vulnerability scanning. When doing a vulnerability scanning, the security specialist is running a software that checks your server and its software for known vulnerabilities. Examples of this software are Nessus and its open source equivalent OpenVAS. This kind of software usually relies on banners and similar ways of probing for version number, which is why it does result in false positives sometimes.
- Application security assessment / audit. This is the part where the security specialist does usually both automated and hands-on security testing to your application. This security testing is done by using an intercepting proxy, like Burp Suite, to do a Man-in-the-Middle to capture the traffic sent by browser to the target site, and then inject malicious content to the intercepted traffic. A normal assessment typically requires from one to three weeks for an application. This applies to both web and mobile applications, though methods differ.
Technical details to remember when security assessment is about to begin
An exhaustive list of everything a developer or a system administrator should remember when a security assessment is either about to begin or is already ongoing is an impossible goal. Despite that we wanted to take the challenge to gather you the crucial small things you should remember. Remember that security specialists are also humans and may have several projects progressing simultaneously, which is why they might forget to ask you about some small thing - you want to do your part of this project that aims to make your application more secure.
Before the assessment
- Check what kind of documentation you have and ask if some of it could be helpful. Doing plain black-box security testing without any documentation rarely makes sense, if ever.
- Freeze all development and other testing in the environment where security assessment is being done. Doing changes while security assessment is ongoing may produce unreliable results or the security specialist could miss an important functionality.
- Take a look at the current content and data used in security testing. It should not ideally contain customer data, but similar dummy data. This test data, including user accounts security specialist is using, should clearly show real use cases of the application.
- Double-check that you have provided all needed and correct URLs and IP addresses.
- Notify all parties handling network administration that security testing is being done. Most major hosting providers require a permission which are usually processed within one week.
- Make sure to be available during the assessment time. Assume that you will get an urgent email about a vulnerability or your application having crashed during the assessment.
After the assessment
- Before deploying updates and fixes, be sure to ask that the assessment has been finished.
- As you will get the report soon and probably already know the date, reserve enough time with the team to go through with the findings.
- Depending on your development practices, you may need to remove some content from the service. Application assessments done with automated tools may create huge amounts of content, like user accounts, pages or messages.
- Remove firewall openings and similar unneeded accesses. Usually there will be a second assessment round to check the fixes for the vulnerabilities, so it may be easier to leave the testing accounts if there is some hassle included in provisioning them.
Security assessment reports
It will confuse and may even make you angry. How did the security guy dare to say that my application is totally insecure? Well, (s)he just did - if that seemed to be the bottom line after the assessment. Security assessment are based on both some known methodology, requirement levels and previous experience of the security specialist. Some of the requirements may not seem to make sense, at least at first. Some standards may propose multiple pre-emptive measures against a type of vulnerability - like requiring X-XSS-Protection header, Content Security Policy and strict input validation to prevent cross-site scripting attacks. Obviously often one well-thought mitigation is enough and two is good security practice.
When reading the findings in the security assessment report remember that you:
- Make sure you understand the problem that was observed. What is the actual impact? How was the problem or exploit produced?
- Think carefully and try to be honest with yourself: is this a problem or not? If it is, everything will usually go more smoothly if you just acknowledge it and fix it.
- Discuss all issues with the entire team to make sure that similar problem isn't present elsewhere in the application. The security specialist may have missed one input field that was similarly lacking CSRF or XSS protection.
- When fixing vulnerabilities, make short notes about the fix referring to the finding. This will make checking the fix more reliable and faster.
The problems that teams stumble into in security assessments are rarely about technicalities, but about communication. Some assessments for complex systems with tight schedules may be technically challenging, but can be done smoothly with good communication and planning. Usually you as a developer or system administrator can't give too much information about your application - the security specialist will discard the information that is unneeded. But it is easy to forget to tell about your company's holiday schedules or about the existence of aggressive web application firewall protecting the application. While this blog post definitely did not cover all technical details of assessment, it hopefully underlined importance of the two-way communication needed to ensure best results.