Effective Ways to Test the Strength of Your Security Awareness Program

Now that your security awareness program is up and running, it’s time to prove ROI. You know that people are responding to the training and the culture of security is starting to grow, but how do you actually test your program’s strength and nail down solid metrics?

We’ve got four solid methods for showing that your program is winning at creating lasting behavior change, which contributes to ROI, which means you have a case to grow your program ... and maybe even start an ambassador program. It’s a win-win for everyone.

1. Set goals.

Before you launched your security awareness program, you should have established some solid metrics and benchmarks for success. Forgot to do that? It’s not too late. Most programs start out heavily focused on phishing training, because it just makes sense (we’ve all clicked on a rotten link), so many of your initial goals will focus on your phishing campaigns. Here are some metrics to look at:

  • How many employees are clicking on phishing campaign emails?
  • How many employees are reporting phishing campaign emails?
  • How many devices were infected per year prior to the program’s launch, and how many are currently infected?

When setting goals for your phishing campaign, consider this: 78 percent of people don’t click on a single phishing email, according to Verizon’s 2018 Data Breach Investigations Report (DBIR). You’re probably thinking that’s pretty impressive, but you’ve got to ask yourself what your company’s passing grade looks like.

If your goal is for 90 percent of people to never click on a phishing email and, instead, to report it, that’s great. But what about the other 10 percent and the repeat responders (the people who respond to three or more phishing simulations within a six- to 12-month rolling window)? Setting benchmarks and goals for your repeat responders so you can deliver the right training will be one of the most important undertakings of your entire program.

Learn the steps to build an amazing security awareness program in the guide >>

2. Review reporting habits.

Your phishing program should have two primary behavior-changing goals:

  1. Don't respond to simulated phishing emails.
  2. Report phishing emails right away.

However, even if you train your employees on how to report a phishing campaign, are they actually using the methods provided to them? Verizon’s 2018 DBIR showed that people report just 17 percent of incidents, usually through a reporting button in the email client or an email address (for example, bademail@company.com).

With average reporting numbers so low, make sure you’ve got a solid goal for reporting metrics, including:

  • How long did it take for people to report?
  • What percentage of people are reporting?
  • Do reporting numbers vary by department or team?
  • After how many campaigns did repeat responders start reporting?

And while you’re digging into reporting habits, take a look at these important metrics, too:

  • Are people reporting when their devices get infected with nefarious software?
  • Are employees reporting right away when they lose a device?
  • Are people reporting when a password or other login information is stolen?

3. Improve password and security practices.

Another goal of your program should be that employees are using stronger, longer passwords and installing security software on their devices. Are employees following best practices and using longer passwords (12-plus characters) instead of crafting impossible-to-remember complex passwords? Are employees using a password manager such as 1Password or LastPass?

Crafting a culture of security also means making sure your staffers know the dos and don’ts of working on public Wi-Fi and the importance of having anti-malware and -ransomware software on their computers and devices. These security habits should become second nature, and as habits change and get better, the strength of your program will become glowingly apparent.

4. Survey your staff.

Hearing directly from the people impacted by your security awareness program will always be the most powerful feedback you can get. Send out regular surveys with questions like these:

  • What topic did you learn about that you didn't know about before?
  • Has the program encouraged you to change any personal behavior (for example, social media security settings, reporting stuff, and so on)?
  • What topics would you like to learn more about?
  • What has been your favorite security topic to learn about so far?
  • If you could, would you share this training with family or friends?

Or better yet, have an open-door policy. Set up a Slack channel or another way for employees to provide feedback and ask questions of the security team (or your ambassador program, if you’ve got one) as they think of them. Between anecdotal and quantitative data, you’ll be able to get a solid handle on whether your program is working.

Launching a security awareness training program is a huge undertaking. Once you start seeing behavior change for the better and people start seeing why and how security is important, you’ll be inspired with new and interesting ways to grow your program.

Put your awareness program on track with steps to build and manage an amazing security awareness program in this guide