[QA] The case for security test scenarios

Sun Aug 10 15:53:38 UTC 2014

Hi Sherif, I agree with everything you put here! I put the groundwork for
this into the Security_for_developers/Architecture. New features should
have Selenium tests for security critical features, and I would really like
to setup exactly the scenario you proposed, having automated regression
testing with Zap. I've just lacked the time to do it yet.

+1 on static analysis too. I tested checkmarks earlier this year and their
php 5.3 support wasn't good enough to give good results on our codebase.
I'm trying to get coverty setup right now. Both are closed source so I
don't like using them. Facebook's pfff has a lot of potential, but doesn't
do security testing yet, and we don't have ocaml expertise to help them
implement it.

So yes, I would love help! Let's get in touch next week and see if we can
get you setup to help us implement the dynamic testing part. Feel free to
ping me of list or on irc.
On Aug 10, 2014 7:24 AM, "Sherif Mansour" <cherifmansour at gmail.com> wrote:

> As a security engineer I want to make sure that new/existing site features
> are tested for security bugs, and new changes to a web application do not
> break existing security controls.
>
> *What can be done?*
> There are several ways to approach this of course. One way to ensure
> feature coverage is to run a lot of the QA regression packs such a selenium
> scripts, through a web app security proxy which will then run security
> tests based on the requests it sees flowing through it.
>
> Additionally I recommend the creation of security test scenarios in order
> to test certain application logic. Simple example: If a user signs-out and
> clicks back he should not appear as signed in.
>
> *Are there tech teams working on this?*
> Yes several tech companies are using similar approaches and for
> inspiration I recommend that you take a look at the following work:
> http://www.continuumsecurity.net/bdd-intro.html
>
> *Is it open sourced?*
> Yes, and its on github: https://github.com/continuumsecurity/bdd-security
> The ZAP security proxy is also open sourced (and it:
> https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project
>
> *How does that work in practice:*
>
>    1. The continuous integration server (e.g. Jenkins) would run selenium
>    scripts, and set the forward proxy to the security web app proxy (such as
>    ZAP).
>    2. The CI server would then send api commands to the zap proxy to run
>    security scans based on the requests it will receive (and the scope can
>    also be set there as well).
>    3. The selenium scripts run and zap would run security tests off of
>    that.
>    4. Additionally there would be some selenium tests that are based on
>    security scenarios.
>    5. The security findings are reported, reviewed for false positive,
>    the priority is set, and bug is raised for developers to work on.
>    6. You can additionally get it to run Nessus as well to look at
>    infrastructure vulnerabilities (note Nessus is now closed source...but
>    openVAS is the opensource fork).
>
> *That are the interesting challenges that a team might see from this.*
>
> And an interesting conversation today with Chris McMahon and Nikolas
> Everett about this:
> Right now the environment is quite public...does WikiMedia want to show
> security issues in SDLC reported the same way as QA issues?
>
> Nikolas Everett pointed out that Wikimedia could make the security issues
> public and never release code unless these issues are addressed (i.e. fix
> the security issue or accept the risk).
>
> *Please note: *The results from the ZAP proxy (or whatever is decided
> on), would not show in the environment as is, because you would need to
> tell that system where to push the results to.
>
> *Is there anything else that can be done?*
> Yes, you could use a security based static code analyser during the SDLC,
> which I have had experience with, and the results were very promising.
> However it does require an on-boarding process to tune the analyzer to your
> code base. By that I mean take a look at the intial findings and create
> filter-sets to avoid false positives and issues you do not care about. In
> many cases my team had to go back to the guys who wrote the rule packs to
> improve them.
>
> *I would like to volunteer as a security resource for Wikimedia*
>
> Kind regards
> Sherif Mansour
> User:Kerberosmansour
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/qa/attachments/20140810/333f9a42/attachment.html>