Secure software development often becomes a discussion topic among security and engineering leadership, as quite a few organizations struggle to build programs that can show measurable success across the full portfolio.
Along with the failure to set clear, measurable goals, there is often insufficient focus on shifting the culture, resulting in lackluster performance in building a secure software development program. Building a secure software practice is a strategic, multi-year initiative, and if not done right, most companies see it as too disruptive. It gets a lot of scrutiny if it is not producing tangible results. Leadership, in most cases, fundamentally cares about delivering secure software to customers; usually it is the security organization that fails to build a balance of investment and practice that is well integrated and aligned with engineering and business objectives, and is therefore unsustainable. It is not uncommon to see multi-year programs demonstrating little value in reducing overall cybersecurity risk, and not uncommon for leaders to question whether the current model is a worthwhile investment.
In this series we explore new ideas on optimizing secure software development programs for the best return on investment. The approach is based on multiple years of experience building Secure Software Development Lifecycle (SDL) programs and consultation with peers and industry experts. No single approach is the perfect solution; different approaches might work for different organizations, or for the same organization at different maturity levels. This post highlights the common mistakes, and the potential of moving security investment to the areas that have the most impact on delivering sustainable, measurable value.
First and foremost, the philosophy must be adopted that security should not pose an undue burden on the organization. To be effective and successful, security has to be a partnership; it should be seamless and frictionless, integrating into existing processes without significantly altering them. The security team must take the burden of security away from users and other teams.
Along with not adhering to the philosophy of partnership, here are the mistakes most commonly observed in secure software development programs.
1. Not setting a high-level goal for the SDL program
“If you cannot measure it, you cannot improve it.” — Peter Drucker
Most organizations fail to set a clear target for their secure software development program. A high-level target — the KPIs and KRIs — is the north star for building any program; it sets the overall direction. These top-level measurements should be meaningful, easy to understand, and equate to a measurable reduction in risk. Other targets may support the overall goal, but the top-level target should remain standalone and be the one thing that is consistently measured and reported. The one that resonates here is measuring the reduction in the number of exploitable vulnerabilities in external applications, expanded to internal applications as the program grows. Examples of what is not useful as a high-level target:
- All software developed goes through static and dynamic code analysis, and similar targets.
- All vulnerabilities of a certain criticality are identified and remediated within specific timelines.
- All applications are enrolled in the SDL program.
- All developers have completed secure software development training.
Except for the second, none of the above directly results in risk reduction; they are entirely programmatic. They show adoption, but very little in terms of improvement in code quality or practice. Perhaps they should be measured, but not as the top-level indicator of the program’s success in reducing risk. The second — remediating every vulnerability of a given criticality within a timeline — becomes too expensive to achieve over time.
2. Over-emphasis on security training as the key area of focus
Depending too much on security training for engineers, instead of on engineering practice and engineering culture, creates a false sense of success. Security training as delivered today is standalone; the only purpose it serves is awareness. Not entirely useless, but it loses its value over time. The skill development needed for secure coding happens in line with code development; it is not on the same timeline as the training, which makes the training ineffective and burdensome. On top of that, an engineering organization of any size sees enough staff turnover that training alone does not yield the desired return once the cost of the training, its management, and the engineering time spent on it are taken into account. Security training as delivered today should be used as an awareness tool, not for skill building.
3. Over-dependence on dynamic and static code analysis
Static and dynamic analysis, if not properly tuned, produces a large validation workload for security and engineering. Depending on the velocity of code produced, it is a distraction rather than a net value-add. It should be used to gauge the overall improvement in the security of the codebase, not as the regular method of identifying vulnerabilities that need remediation. Instead, the use of security libraries as standard engineering practice should be promoted. Analysis tooling is also becoming prohibitively expensive compared to its value.
4. Not enough investment in threat modeling and non-functional requirements
Along with threat modeling, non-functional requirements (NFRs) are key to achieving higher code velocity and lower resistance in delivering a more secure application. Organizations resist threat modeling because it is seen as bureaucratic and not agile; however, time and again, under-investment in security in the design phase leads to poor security architecture and poor design choices, which are hard to rectify later and result in security debt.
5. Not leveraging the engineering practice organization
Secure software development is about cultural change, and cultural change is hard — especially if you run two parallel cultural initiatives. In most organizations, engineering practice establishes how all engineering activity is performed, from source repositories and code management, through language support, to release management and deployment. That practice already exists. Leveraging it to advance security goals is the better long-term investment: it aligns with the engineering culture and reduces the overall burden.
6. Not leveraging all the technologies available
The purist view that all vulnerabilities should be fixed in code has been changing for a while. Fixing one-off vulnerabilities is akin to whacking a mole; it is expensive, it slows the velocity of code deployment, it is unrealistic, and it tremendously increases the engineering organization’s workload. Reducing exploitable vulnerabilities in code should leverage the full spectrum of technologies and capabilities: Content Security Policy, real-time application protection, web application firewalls, server and infrastructure hardening, and so on should all be part of the arsenal. Experience has proven that these technologies make an application more resilient to attack and can significantly reduce the impact on the application even if it is successfully compromised.
7. Not measuring success
This is perhaps similar to the first mistake: you cannot improve what you cannot measure. To drive a successful program, its progress and results must be measured and broadly shared within the organization. This generates healthy competition between teams and their leadership as they strive for better security. It is not about shaming; it is about being open, honest, and direct about where improvement is needed. An insecure application represents a real risk to the whole organization.
The follow-up post, Working your flanks, presents a modified version of the SDL process, designed from the bottom up to reduce exploitable vulnerabilities in code.