Wednesday, September 26, 2007

Getting the customer to speak

Tate Hansen, over on Clearnet Security has a post about getting the customer to provide input as part of a penetration test. It surprised me for two reasons: 1) I didn't know that it wasn't done and 2) it's so obvious an issue.

I'm not saying that I don't believe that the condition exists. People (and therefore organizations) tend to take the path of least resistance, so if the penetration testers don't ask, the customer is not going to offer up the information.

My surprise is that the question just doesn't come up. It may be because I'm the type to take a packet sniffer to a CTF contest. (Yeah, one of those that thinks that CTF is a spectator sport.)(I have Don M. at ODU and S-14 (hiya Pete!) to thank for that "bad habit".) To me, the "What did you see?" question is just so obvious that it's a "must ask".

I can also see how organizations fall into the practice of not participating in their own penetration testing. It may have something to do with that other form of security testing called the vulnerability scan. It's usually performed more often and requires no input from the customer, except during the remediation phase, and that is usually an internal process (e.g., the CIO may have some "'splaining to do" to the CIO).

The Hansen/Ranum/McGraw reference to the "badness-o-meter" is a good one. If your pen-testers have anything other than "we don't know" at the top end of the scale, the data they're providing about your level of security may be suspect. Pen-testing is an inverted business-model. The best you can hope for is: "We don't know. We failed." A few things to keep in mind:

  • This doesn't mean that someone else doesn't already know
  • It also doesn't mean that they won't know tomorrow or the day after
  • To quote a semi-cliche: "Security is a process, not an end state." (Dr. M. E. Kabay, 1998)
  • By extension, a pen-test is a snapshot of that process, not of an end state