Choosing the Best AppSec Tools: Advice from Experienced Engineers

In our latest webinar Gary Robinson and Martin Hewitt from Uleska gave us a fascinating and comprehensive look into how experienced security teams choose application security tools.

We’ll have a quick look at some of the points covered below, and you can get the full webinar on-demand over here. It’s jam-packed with advice you can put into practice straight away, and illustrative case studies to guide your thinking.

Today’s application security tool landscape

Gary describes it in one word: Broad. Well-known testing environments like Kali Linux have over six hundred tools wrapped into them, with hundreds of commercial and other open source tools out there. So how do we decide which ones to choose and which will be effective? “It’s about the right tools in the right order as you grow your application security programs”, Gary says.

Categories of Security tools

It’s helpful to look at the tools in terms of release cycle areas so you can determine exactly where application security tools will have an impact at a pipeline level:

• SAST: These are tools that check source code or binary to find flaws or vulnerabilities like SQL or Command injections
• IaC: A slightly newer set, these tools test your Terraform, cloud formation, dockerfiles and such to see if there are any problems with how the code is used
• SCA: These tools check third party libraries or dependencies you’re using for known problems and any necessary upgrades
• Containers: Tools aimed at locating and reporting on flaws within your images
• DAST: Tools that send traffic to running systems to find responses and point to any security flaws
• IAST: These act like a WAF and a SAST together to add a layer inside your programs that can spot and relay flaws
• Cloud and infrastructure: Tools that check for flaws like if your S3 buckets are lying open, what your IAM looks like from an infrastructure point of view, what systems you have running, and if any patches are needed

Gary notes that he’s seen many security programs which have to combine different aspects of each tool category to build up the required level of coverage for daily releases. And with things moving at such a speed, it cannot be a manual process.

Commercial, Open Source & Custom Application Security Tools

We’ve gone through the kinds of tools that can be plugged into each stage of the pipeline, but what classification of application tools exist? There are generally three kinds: Commercial, Open Source and Custom. And each comes with benefits and drawbacks. 

Commercial Tools

Commercial tools cover a wide range of security controls and are usually easier to use, but they come with a price tag that might be prohibitive for smaller outfits, or across all projects in larger companies. 

Open Source Tools

Open Source tools like OWASP and Kali Linux are free but generally come with less functionality and reporting abilities. 

Custom Tools

Custom tools are generally developed in-house and provide a level of niche coverage specific to business logic or product authorisation that off-the-shelf tools simply can’t. Still, again, they are restrictive in false-positive functionalities and recommendations and are hard to automate into an AppSec program. 

Determining which is right for your organisation combines internal aspects like culture and technical fit and external considerations like regulations.

How to Choose the Right Application Security Testing Tool

Regulations and Standards fit

The way we test has moved on leaps and bounds in the last few decades. In the past, we would have had penetration testing exercises conducted by manual teams or outsourced companies. They would run tests to check systems that we implicitly assumed would cover the whole security scope. In today’s release world, where automation is key and shifting left is crucial, this type of long-tail testing throws up security gaps. And therein lies risks.

What security do we care about?

You might be a mid- or enterprise-sized organisation having to adhere to specific regulations, or you’re a smaller outfit or SME selling into these larger companies, so you will have to adhere to them too. Companies have various standards they need to check, and by bringing them together into a list of controls, it can tell you what you need to check for down to the technical level. Standards like NIST or OWASP list hundreds of controls to test for. When you have a list, you understand the controls each tool tests for and can map the coverage your organisation needs.

For more on this, check out the full webinar here.

Plugging manual gaps with feedback loops

The goal is to get to a stage of plugging controls that are manually testing gaps with automated coverage. When you start seeing the same categories of results in automated testing, you can then add automated tools to address those fixes.

Technical aspects to determine fit

How your company is set up will rule in or rule out certain tools. There’s generally a 50/50 split of internal and external technical aspects informing security tool decisions. For example, based on your setup, do you require tools that run only in the cloud? What languages are you working in? If development teams are running in C# there's no point in running a tool that will only check a different language  as it’s not relevant to you. 

An outcome-first approach

The main thing is to ensure that application security testing isn’t tied to the tools you want to use or have used in the past. It must be built around the controls you need to test for and the programs you want to run.

Cultural fit to determine tool choice

No matter the size of the organisation and the amount of tools running, the program needs to work for every team. That’s why it’s crucial to think in terms of risk, not vulnerabilities.

Language matters

Risk conversations can help make a case for security because it ensures a complete understanding of the risk context across all teams. For example, you could state that you have to fix an XSS vulnerability, but project managers and other stakeholders won’t understand this or grasp the gravity of the situation. Stating that it’s a $30 million risk, on the other hand, is a fact hard to ignore.

Determining tool fit: Context drives decisions

The fact remains that many companies believe they have a certain level of coverage operating the tools they like, know or have used before. But coverage comes with joined-up thinking: you understand the controls, map them to tool technicality and know what works culturally for your organisation. For automating security effectiveness and for teams using and understanding security, context is key.

For lots more on how to determine the right security tools, watch the full webinar here. And for more on AppSec automation, check out our previous webinars and our blog, play around with a free Uleska account or reach out to us for a chat.