Pentest or automated scanning: which do you need?

What is a pentest?

A pentest (short for penetration test) is a controlled attack on your application. An ethical hacker tries, with your permission, to break in the way a real attacker would: log in as someone else, read data that is not theirs, tamper with payments or take over the server. Not to cause damage, but to show what is possible.

You get a report listing every vulnerability found, how severe it is, how the tester exploited it and what you need to do to close it. That report is the real product of a pentest: it is what customers, partners and insurers want to see.

Important to know: a pentest is a snapshot. It describes your app on the day of the test. Push new code the week after and the risk has changed, while the report says nothing about it.

Having a pentest done: who tests your app?

A pentest is carried out by an ethical hacker: a security specialist who attacks with your written permission, within limits you agree beforehand. Usually that is someone from a specialised security firm, sometimes a freelance tester. The difference from a real attacker is not the techniques, but the permission, the agreements and the report you receive afterwards.

You give the tester access to your app and your data, so trust matters as much as skill. Look for these points when choosing:

• Certifications of the testers themselves, such as OSCP, OSWE, CREST or eWPT. These require someone to have actually attacked systems in practice, not just passed a theory exam.
• A quality mark for the company. In the Netherlands that is mainly the CCV pentesting quality mark (Keurmerk Pentesten) of the Dutch Centre for Crime Prevention and Safety. It requires the pentesters to hold a recognised personal certificate, and an independent certification body (DEKRA, DigiTrust or Kiwa) audits the company every year. Many customers also ask for an ISO 27001 certified supplier.
• A fixed methodology, such as the OWASP Web Security Testing Guide or PTES. Then you know what was tested and you can repeat the test later.
• Clear agreements on confidentiality and your data: a non-disclosure agreement, and a data processing agreement if the tester can see personal data.
• Independence. Whoever built your app cannot credibly pentest it themselves. A report for customers or insurers has to come from an external party.

Types of pentest: black box pentest, grey box and white box

Pentests differ mainly in how much the tester knows about your app beforehand. That determines what they can find, how long it takes and therefore what it costs.

• Black box: the tester knows nothing and only gets the URL. This is closest to a real attack from the outside, but the tester spends a lot of time exploring and misses things that are only visible with an account.
• Grey box: the tester gets a normal user account and some explanation of how the app works. For most web apps this is the best choice: realistic, yet efficient enough to also test the logic behind the login.
• White box: the tester gets full access, including source code, database schema and configuration. The most thorough and the most expensive, mainly useful when you really want to be sure there are no flaws in the code.
• The scope differs too: a web application pentest targets your app and API, an infrastructure pentest targets servers and network, and a mobile pentest targets the iOS or Android app.

For an app built with Lovable, Cursor or Bolt, a grey box test of the web application is almost always the starting point. Also read where AI code security structurally goes wrong.

How a pentest works, step by step

For an average web app, expect three to ten testing days spread over two to four weeks including reporting. A retest usually follows a few weeks later.

• 1. Scope and agreements. You agree on what will and will not be tested, when, and how the tester will contact you if they find something serious. You sign an authorisation, because hacking without permission is a crime.
• 2. Reconnaissance. The tester maps your app: which pages, forms, API endpoints and external services exist, and which technology sits behind them.
• 3. Testing. Manually and with tools, the tester looks for vulnerabilities: leaked keys, missing authorisation, injections, weak session handling, open database rules. This is the bulk of the work.
• 4. Exploitation. Discovered holes are exploited in a controlled way to prove the real impact: can I reach other people's data, can I change payments, can I become an admin?
• 5. Report. You receive a document with the findings ranked by severity, with reproduction steps and concrete recommendations. Often with a management summary for non-technical readers.
• 6. Fix and retest. You resolve the findings and the tester verifies that the fixes work. Ask up front whether that retest is included, because without it you do not know whether your report is now clean.

What is in a pentest report?

The report is what you keep after a pentest, so it pays to know beforehand what belongs in it. A good pentest report contains:

• A management summary: in plain language, where your app stands, what the biggest risks are and what needs to happen first.
• Scope and methodology: what was and was not tested, in which period, with what kind of account and following which method.
• The findings, each with a severity (often as a CVSS score: critical, high, medium or low), the impact and evidence such as screenshots or the requests the tester sent.
• Reproduction steps, so that you, your developer or your AI tool can replay the issue and check it after the fix.
• A concrete recommendation per finding: what you need to change, not just what is wrong.
• After the retest: per finding, whether it has been fixed. That is the version you show to customers and insurers.

How long does it take? The report usually arrives within one to two weeks after the last testing day. Add the wait until a tester is available and there are often about two months between the first conversation and the final report. Serious holes should not wait for the report: agree that the tester reports them immediately.

Pentest or automated scan: what is the difference?

An automated scan runs on every change and checks everything that can be checked automatically: keys in your code, vulnerable packages, open database rules, missing auth checks, misconfiguration. It is fast, cheap and continuous, but it does not think about your specific app.

A pentester does exactly that. They notice that a user can change the order id in the URL and view someone else's order, or that your discount code can be redeemed twice. A scanner does not find such logic flaws, because they are not a known pattern.

• Scan: continuous, on every push, done in minutes, finds known mistakes and forgotten settings.
• Pentest: one-off, scheduled, days of work, finds logic flaws and attack chains in your app.
• Scan: suited to working with every day while you build.
• Pentest: suited as proof to the outside world and as a check right before or after an important launch.

They complement each other: the scan clears the low-hanging fruit, the pentest looks for what remains. When comparing security platforms, that is the difference that matters.

When to choose which

• Your app still changes daily: start with continuous scanning. Today's pentest is outdated next week.
• You are going live with real customer data or payments: plan a pentest, preferably a few weeks before launch so you have time to fix things.
• A customer, partner or insurer asks for proof: a pentest report is usually what they mean.
• You need to comply with a standard (ISO 27001, SOC 2, PCI DSS): they ask for periodic testing, and a pentest is the usual way to provide it.
• Budget is tight: clear the low-hanging fruit with scanning first, so you do not pay a tester for findings you could have caught yourself.

Is a pentest mandatory? NIS2 and compliance

No law literally says every app must have a pentest. But more and more rules require you to demonstrably test whether your security works, and a pentest is the usual proof.

• NIS2: the European cybersecurity directive, implemented in national law (in the Netherlands, the Cyberbeveiligingswet, in force since 15 August 2026). Organisations in sectors such as healthcare, energy, drinking water, transport, government and digital infrastructure must take appropriate measures and assess whether they work. If you are not covered by NIS2 yourself but supply software to an organisation that is, those requirements reach you through your customer anyway.
• GDPR: article 32 asks for a process for regularly testing, assessing and evaluating your security measures. If you process personal data, a pentest is a strong way to demonstrate that.
• DORA: banks, insurers and other financial entities must test their systems regularly. Entities designated by the supervisor run a so-called threat-led pentest at least every three years, including on functions outsourced to ICT suppliers. Those suppliers then take part in the test.
• Standards: PCI DSS requires a pentest every year and after every significant change if you process card data. The Dutch DigiD standard asks for periodic pentests and vulnerability scans, checked every year by an IT auditor. ISO 27001, SOC 2 and the Dutch healthcare standard NEN 7510 ask for security testing and vulnerability management; a pentest is not literally required there, but it is the usual way to demonstrate it.
• Contracts and tenders: customers and governments increasingly ask for a recent pentest report, and cyber insurers do so when you take out a policy.

How often should you have a pentest done?

The rule of thumb is at least once a year, plus after every major change: a new login method, payments, a new API, different hosting or a new way of handling personal data. PCI DSS prescribes exactly that, and many customers and insurers follow the same line.

An app you build with AI, however, changes far more often than once a year. With only an annual pentest, the code of the other eleven months goes untested. That is why you combine the two:

• Annually: the baseline for most web apps with customer data.
• Every six months or every quarter: with payments, health data or an app that changes often and substantially.
• One extra test: before an important launch or after a major rebuild.
• Continuously: an automated scan on every change, so new mistakes do not sit unnoticed between two pentests.

Pentest cost: what do you pay for a pentest?

Pentests are almost always priced per day. In the Netherlands a day rate for an experienced tester is typically between 1,000 and 1,500 euros. A small web application with a handful of screens and an API often takes three to five days, a larger app with multiple roles, integrations and payments eight to fifteen days.

So for a simple app, expect roughly 3,000 to 7,500 euros, and for a more complex app 10,000 euros or more. A black box test is not necessarily cheaper: the tester needs more time to explore. Automated scans cost a fraction of that and run continuously.

Ask every quote for four things: the exact scope, the number of testing days, the lead time, and whether a retest after fixes is included. That last point often decides whether the report is worth anything.

Prepare before you buy a pentest

Every finding the tester makes costs part of their time. If they find a leaked key and a table without database rules on day one, you pay expensive hours for mistakes a scanner would have found in two minutes. So clean up yourself first:

• Turn on row level security for every table and review the policies.
• Get keys out of your frontend and rotate anything that ever touched your git history.
• Update packages and resolve known vulnerabilities.
• Check that every server function really validates the caller.
• Know what data you store and who can reach it.

Use the OWASP top 10 as a guide to check whether your app is safe yourself, and go through the checklist for launching safely.