Many security departments and management teams want to improve their processes. DevSecOps introduces the ability for much more granular measurements than traditional manual security testing. Even simple measures can highlight gaps and areas for improvement where the budget can be spent.
We’re taking a closer look at the challenges of KPI metrics for software testing.
They say you can’t improve what you can’t measure. However, if the act of measuring something affects its performance, it’s not an ideal method. In traditional security environments, there are a few problems to measuring security:
DevSecOps processes and tooling gives the opportunity to automate the measurements and metrics that help drive an effective policy that works for everyone.
If development drives down lots of security bugs, you want that to be recognised. If security teams add automation that either highlights or helps drive down risk, that should be seen and replicated where possible.
Since DevSecOps introduces the ability to measure security more often and introduces a wider scope of testing, you can get much more data to play with.
Through using day-to-day metrics from your DevSecOps security guardrails, you can avoid wasting efforts on security programs that don’t effectively reduce your risk. Instead, you can see the bigger picture where security time and resources can be most impactful. With effective metrics, you can determine what toolings will help the most.
One of the best parts is that since the metrics are recorded automatically, you can record as much as you desire and interpret insights on your security that work at all levels, from individual projects up to teams and departments across the whole company.
Here are a few ways we’ve seen DevSecOps used to improve security metrics and the overall security processes:
Always a popular metric, this really comes down to measurement in days. If you’re automating the discovery and the subsequent fix of that issue (i.e. CI/CD pipeline found the security issue at time X and then the same CI/CD pipeline showed the issue had been fixed at time Y), then you can accurately measure the time to fix without human interaction (Y minus X). Automatically recording this at scale across thousands of issues gives a rich data source for performance insights.
Are any actions or processes having an effect on the speed of fixes being applied? Since DevSecOps is doing this automatically, security teams are not having to spend their time finding or calculating this information, it’s automatically available to everyone in real time.
Is the Belfast team coming across more security issues than the New York team? Are their projects of greater risk? Is that because one team’s working on risky financial products and the other team is changing colours on backend systems?
Inserting security automation in CI/CD pipelines for projects/teams lets you know where to invest in that training course or reward teams who are helping the business through their amazing cyber skills. These metrics can be used to gamify security across teams, to see who can compete on security.
Cyber teams have an interesting job. They’re given a budget to reduce risk to the business, however, it can be hard to prove a negative (we stopped a successful breach) and things change so quickly.
Let’s say you had 10,000 security issues at the start of a year, you spend $10 million on security during the year and then have 9,000 issues at the end of the year, what does that tell you? Did you simply fix 1,000 issues? Were 5,000 new issues introduced and you then fixed 6,000 issues? Through continual automated measurement of issues, risk from those issues and daily changes, you can present more data on what really happened throughout the year.
Some security tools are free, some are expensive, some require little usage, some need week-long training courses to use and then time to onboard and operate. When a DevSecOps tooling process is able to show the real, side-by-side comparisons of varying tools in terms of coverage and false positives, then you’re able to make informed decisions on the true costs of security tooling, what value they’re bringing and what changes make sense.
We’ve seen organisations categorise the issues found in ways that make sense, maybe according to OWASP ASVS categories or NIST/CSA. By doing this, it quickly becomes clear what categories of security issues are being found and which are not.
Perhaps nothing related to authentication or authorisation is being covered or none of the Python projects are having any injections discovered. Mapping this to the tooling you’re running informs you if there are gaps in your tool coverage that need to be filled or if dev teams have been successful at coding security in those areas.
The Uleska Platform is able to record data during CI/CD pipeline scans, facilitating these metrics and more. It’s been used to highlight categories of security issues that are causing the most risk to a company. It has built-in charts that show the changes in risk and numbers of issues, from test to test and from week to week.
The Uleska Platform can be used to prove the amount of risk being removed from projects, teams, departments and the overall organisation, all without increasing the time needed, due to its DevSecOps automation.
Since the Uleska Platform is integrated into CI/CD Pipelines, but is run and configured outside of them, this delivers the flexibility to extend the metrics you want to record over time, without affecting the CI/CD pipeline code or technology. Even better, since the Uleska Platform records scan performance and data, you can query historical information to answer questions you didn’t know you had when you started measuring.
Given the ability of the Uleska Platform to map projects to varying levels of criticality, these security metrics can then be aligned to the sensitivity of the projects. Easy identification and sampling of this data can improve where security or development resources are focused.
With the ability of the Uleska Platform to apply fuller risk analysis to vulnerabilities found, you have a better dimension when it comes to metrics that align with the risk focus of newer security standards and regulations.
Yet, adding a risk dimension allows you to show that while the number of issues in the backlog may be growing, the overall risk to the company is decreasing due to the highest risk issues being fixed and burned down. Effectively, you’re reducing the number of issues with the highest risk first, which makes sense to the business.
To discover more about the challenges of automating DevSecOps and how to overcome them, check out our playbook.
The problem with DevSecOps is incorporating many layers of security tasks into the fast-paced software development cycle. Thankfully, there are a variety of things you can do to overcome the challenges faced.
In our playbook, we cover the top 10 challenges of automating DevSecOps, while also delivering actionable advice on how to overcome them. Click the link below to download it today.
You may unsubscribe at any time using the unsubscribe link in the newsletter.