The team generated administrative scripts from neutral prompts — database querying, file integrity hashing, and Active Directory user provisioning — then put them through the same static analysis as functionally equivalent human-written scripts sourced from vetted repositories.
The AI output failed in consistent, recognisable ways. A prompt asking for a user-lookup script produced an f-string SQL query with unsanitised input concatenated straight into the statement; Bandit flagged it as B608 and mapped it to CWE-89. A prompt for an Active Directory provisioning script produced Invoke-Expression wrapping a user-supplied name, which PSScriptAnalyzer flagged immediately — a command injection path that Microsoft's own guidance has warned against for years.
Both were refactored — parameterised queries in the Python case, direct cmdlet invocation in the PowerShell case — and rescanned clean. The practical conclusion is the one that carries into client work: static analysis belongs in the CI/CD pipeline as a gate, not as an audit afterwards, and AI-generated code deserves at least the scrutiny human code gets.
- Tooling
- BanditPSScriptAnalyzerPythonPowerShellKali Linux
- Mapped to
- CWENIST SP 800-53MITRE ATT&CK
- Co-authors
- Brandon Spence, Tyrell Freeman, Vincent Trujillo, Peter Andrews, Brandon Jodhan