Abstract
Capturing behavioral data to assess users' IT security awareness is state of the art. However, recording the click rate on a company wide phishing test for IT security awareness measurement does not suffice. Perceivable artifacts, that the user might be exposed to during an attack, are manifold. We introduce a framework that allows capturing user's responses to such artifacts similar to phishing tests. A field study among 259 users shows, that the expected effect of a well-established IT security awareness intervention can be demonstrated using arbitrary artifacts. It also shows that this intervention may impair the probability of a user reporting the sighting of an artifact and therefore impair an organization's capability to detect such events and possibly decrease overall security.