The Showdown Between Bug Bounty Programs and Penetration Testing

On March 22, 2018, Netflix started a “bug bounty” program that compensates hackers who report vulnerabilities to the company. This is something that the company has done for the past five years, but only in a restricted setting. Now that it’s opened the program to the public, it will have a large number of hackers looking through the site extensively.

This practice may seem a bit chaotic, but many people assert that paying strangers to hack your website is one of the most effective ways to secure it against potential threats. The question, however, is whether bug bounty programs are really more effective than having an in-house penetration testing team.

How Penetration Testing Works

bugbounty-keyboard

Penetration testing is a normal part of the development cycle that’s usually done before a product is released to the public. It involves a team of individuals, either outsourced or in-house, that attempt to “hack” the software or system that the company wants to release. They then report all vulnerabilities found on the platform, allowing developers to fix these problems before they become nuisances later on.

During penetration testing, the team typically follows a set procedure to uncover all possible vulnerabilities. This may involve using techniques that hackers typically use to infiltrate systems and software. What you end up with is a comprehensive list of critical areas in your software that most hackers would be able to subvert.

What Makes Bug Bounties So Attractive?

bugbounty-crowdsourcing

When you make a bug bounty program, you are basically telling the public that you’re willing to pay a set amount of money to anyone who manages to report a significant vulnerability to you. To run a successful bug bounty, you need to set a couple of ground rules so that people know what kind of behavior is unacceptable during such a quest.

Despite how counter-intuitive it may sound to have this kind of policy, bug bounties offer a certain number of advantages over traditional penetration testing:

  • Participants in the bounty are paid once a vulnerability is found, creating an incentive to do a thorough sweep of all the software. Penetration testing doesn’t present these incentives, since team members are paid regardless of how thorough they are.
  • Bounties give thousands of skilled hackers the opportunity to test their mettle, providing an incredible number of perspectives. Penetration testing teams tend to be restricted in size. Regardless of their skill, their perspective is limited.
  • Many bug bounty participants are skilled full-time professionals who participate in several different hunts at the same time.
  • Companies with huge “attack surfaces” (i.e. software that is very prone to breaches) can uncover bugs that were previously left out by their own teams.

Why Penetration Testing Is Still Relevant

bugbounty-penetrationtesting

Bug bounties may be great and all, but they don’t necessarily work for companies that do not have enormous communities. It’s the reason penetration testing is still a big phenomenon. If you’re a medical supply software company, for example, you might not get as many willing participants as, say, a video game studio with a community of tens of thousands of people.

Penetration testing still offers other advantages that might convince companies to forego the idea of bug bounties entirely:

  • You minimize the risk of your vulnerabilities being exposed to the public before you have a chance to fix them. Even if you set a rule against this in your bug bounty, people are bound to misinterpret it.
  • Outsourced penetration testing companies might offer certification that is important to your customers.
  • The quality of reporting is often much higher in penetration testing.
  • It’s useful in highly-regulated markets (such as payment processing and anything that handles bank/debit/credit card data).

Do you feel safer using Netflix because of its bug bounty program? Or would the company have been better off working with a penetration testing team? Tell us all about it in a comment!

2 comments

  1. To answer point by point would require a post longer than the article. But to summarize:
    No matter how small the user community is it is still bigger than your penetration team.
    Penetration teams tend to have an unintentional bias. Hackers do not.
    As you state, penetration teams follow a set procedure and use techniques that hackers typically use. Hackers are under no such constraints. A ‘set procedure’ is always behind times. How do you account for ‘atypical’ techniques?
    Penetration team(s) are under your control. When they find a flaw or a vulnerability, they WILL report it to you. With hackers, it’s a toss up whether they will report or whether they will use the knowledge for their own gain.
    Which group to use depends on the type of software/data is involved. If data and/or software is sensitive, a penetration team must do the work. If data and software are mundane, hackers will subject it to a more thorough testing.

    If there is one thing I learned from my many years in Q&A, it is that no matter how thoroughly we tested an application before releasing it into production, the users always found more bugs.

  2. Another advantage to the bounty programs—you can get some expert skills from folks who otherwise are unhireable (or at least someone your other employees would not want to be working anywhere near) be it from poor personal hygiene, offensive behavior, etc.
    Hey, the classic “guy in his Mom’s basement” needs some cash too.

Comments are closed.