Using Semgrep OSS to Find OWASP Top 10 Weaknesses in PHP Applications: A Case Study

Given PHP’s continuous success, it remains an important task to ensure security in its applications. While code reviews are a common measure to catch bugs during development, they lack scalability, are error-prone, and time-consuming [2, 38, 41]. Thus, static analysis tools like Semgrep emerged to provide programmatic feedback on code. But static analyses often show low precision, which can jeopardize utility. In this case study, we investigate precision rates for Semgrep OSS for common web weaknesses from the OWASP Top 10 [35]. We explore method and tool limitations in weakness detection, OWASP classes, and Semgrep’s public PHP rule set. We apply the latter to 300 open source applications, invest 34 h in manual sample validation, and derive precision rates for each OWASP class. Our validation shows that the rules correctly detected weaknesses for seven OWASP classes with 86% precision, demonstrating the tool’s utility. Yet, we estimate that most findings are not exploitable (81%). Thus, there is still considerable assessment overhead for users. Our work further highlights that only a subset of weaknesses are detectable, as dimensions such as runtime context and insecure design remain hidden. Finally, we advise practitioners to not exclusively rely on public rules, as translating application-specific business logic and design choices may open up to the detection of previously uncovered weaknesses.