The Showdown Between Bug Bounty Programs and Penetration Testing

On March 22, 2018, Netflix started a “bug bounty” program that compensates hackers who report vulnerabilities to the company. This is something that the company has done for the past five years, but only in a restricted setting. Now that it’s opened the program to the public, it will have a large number of hackers looking through the site extensively.

This practice may seem a bit chaotic, but many people assert that paying strangers to hack your website is one of the most effective ways to secure it against potential threats. The question, however, is whether bug bounty programs are really more effective than having an in-house penetration testing team.


Penetration testing is a normal part of the development cycle that’s usually done before a product is released to the public. It involves a team of individuals, either outsourced or in-house, that attempt to “hack” the software or system that the company wants to release. They then report all vulnerabilities found on the platform, allowing developers to fix these problems before they become nuisances later on.

During penetration testing, the team typically follows a set procedure to uncover all possible vulnerabilities. This may involve using techniques that hackers typically use to infiltrate systems and software. What you end up with is a comprehensive list of critical areas in your software that most hackers would be able to subvert.


When you make a bug bounty program, you are basically telling the public that you’re willing to pay a set amount of money to anyone who manages to report a significant vulnerability to you. To run a successful bug bounty, you need to set a couple of ground rules so that people know what kind of behavior is unacceptable during such a quest.

Despite how counter-intuitive it may sound to have this kind of policy, bug bounties offer a certain number of advantages over traditional penetration testing:

  • Participants in the bounty are paid once a vulnerability is found, creating an incentive to do a thorough sweep of all the software. Penetration testing doesn’t present these incentives, since team members are paid regardless of how thorough they are.
  • Bounties give thousands of skilled hackers the opportunity to test their mettle, providing an incredible number of perspectives. Penetration testing teams tend to be restricted in size. Regardless of their skill, their perspective is limited.
  • Many bug bounty participants are skilled full-time professionals who participate in several different hunts at the same time.
  • Companies with huge “attack surfaces” (i.e. software that is very prone to breaches) can uncover bugs that were previously left out by their own teams.


Bug bounties┬ámay be great and all, but they don’t necessarily work for companies that do not have enormous communities. It’s the reason penetration testing is still a big phenomenon. If you’re a medical supply software company, for example, you might not get as many willing participants as, say, a video game studio with a community of tens of thousands of people.

Penetration testing still offers other advantages that might convince companies to forego the idea of bug bounties entirely:

  • You minimize the risk of your vulnerabilities being exposed to the public before you have a chance to fix them. Even if you set a rule against this in your bug bounty, people are bound to misinterpret it.
  • Outsourced penetration testing companies might offer certification that is important to your customers.
  • The quality of reporting is often much higher in penetration testing.
  • It’s useful in highly-regulated markets (such as payment processing and anything that handles bank/debit/credit card data).

Do you feel safer using Netflix because of its bug bounty program? Or would the company have been better off working with a penetration testing team? Tell us all about it in a comment!

Leave a Reply

Yeah! You've decided to leave a comment. That's fantastic! Check out our comment policy here. Let's have a personal and meaningful conversation.