Application testing and the OWASP Top 10

Quite often, a customer will ask us to "test our application against the OWASP Top 10".  I’m going to start by saying that the OWASP Top 10 is a wonderful tool which has helped improve web application security globally since it first launched.  But although it’s a common request to test applications against it, I think it’s helpful to explain why it might not give you the security outcomes you want from a web application penetration test.

What is the OWASP Top 10?

Let’s start with what it is.  OWASP (the Open Web Application Security Project) develops and maintains the OWASP Top 10.  This is a report which outlines the top 10 most critical risks in relation to web application security. It is compiled by web application security experts across the world, and is driven by data provided by many, many security researchers and penetration testers.  Risks are ranked based on how frequently they appear in security test outputs, how serious they are and how impactful they might be.

The purpose of the OWASP Top 10 is to highlight security risks which are very common and very serious in applications so that awareness of these weaknesses and vulnerabilities is raised.  This helps to provide security and development teams with a well-founded steer about the types of risks they should be thinking about during their software development lifecycle, and the types of controls they should be putting in place for mitigation.

This is extremely useful for anybody who wants to develop secure applications, and for web application security in general.  The simplicity and accessibility of the OWASP Top 10 makes it a useful tool for engaging executive stakeholders and business decision makers. 

This simplicity also lends itself to being used to specify compliance requirements.

Where does this fit in to penetration testing?

It’s not unusual for our customers to pass down a requirement from their supply chain which mandates "all applications must be validated against the OWASP Top 10" or states “penetration testing must be conducted for the OWASP Top 10 vulnerabilities”.  Unfortunately, this is less useful than people think it is as an objective for an application security test – for several good reasons.

The first reason is that OWASP Top 10 is not intended as a guide against which to test – it is the result of the aggregated outcome of a thorough penetration testing methodology rather than a benchmark for it.  The OWASP Top 10 regularly changes based on the experiences and data provided by security professionals; it’s a reflection of vulnerabilities and weaknesses which are currently common – and the zeitgeist heavily influences this. 

The OWASP Top 10 will continue to evolve as developers use different technologies and approaches in the implementation of their applications.  A comprehensive application testing methodology, by comparison, will reliably challenge security assumptions and continually identify novel areas of weakness.  If you ask for an assessment to be focussed around a narrow selection of weaknesses - rather than an assessment which follows a rigorous methodology – you risk failing to identify either uncommon issues which could seriously impact your applications, or new security vulnerabilities which might end up being the Next Big Threat.

A second reason that asking for an “OWASP Top 10 assessment” may be less useful than you assume is that there are significant areas of the OWASP Top 10 which simply cannot be assessed via a penetration test methodology alone, or where internal audit is more appropriate.  For example:

• A08:2021 – Software and Data Integrity Failures – to assess this properly typically requires a full clear-box assessment of development pipelines and architecture which is not often possible in a pure penetration test.
• A09:2021 – Security Logging and Monitoring Failures – it’s very unusual that we have access to back-end logging and monitoring solutions as part of an application security test, and therefore logging and monitoring are simply not things that we can usually assess via a penetration test alone.  Instead, this would be an audit-based activity. 

Conclusion

Rather than asking “Please check all of the OWASP Top 10 vulnerabilities in our application”, think “please conduct a comprehensive penetration test of our application, and then identify whether any of the risks you found also appear in the OWASP Top 10”. And if you want comprehensive security assurance for your application, it’s important to understand the limitations of an application security testing methodology.  

Penetration testing alone is not the only control you need to think about – there are also a lot of checks and balances that need to be done internally to provide the right resilience for your applications because there are things that won’t be visible during a pentest.  Talk to us about what you need to achieve from a technical assurance exercise - and where there are limitations to the methodologies you are asking for, we can work together to achieve the outcomes you want.