How to Hide Cookie Banners with Puppeteer

Let's face it, cookie consent banners are annoying, especially when you're trying to take clean screenshots with Puppeteer. In this guide, I'll show you how to automatically handle these pesky banners and get perfect screenshots every time.

Installing Required Dependencies

Before we dive into the code examples, you'll need to install the necessary packages:

# Install Puppeteer
npm install puppeteer

# Install AdBlocker for Puppeteer
npm install @cliqz/adblocker-puppeteer

# Install AutoConsent (optional, for Method 2)
npm install @duckduckgo/autoconsent

Method 1: Using AdBlocker Filter Lists

One of the most efficient ways to handle cookie banners is by leveraging existing filter lists designed for ad blockers. The @cliqz/adblocker-puppeteer package provides an excellent solution:

const puppeteer = require('puppeteer');
const { PuppeteerBlocker } = require('@cliqz/adblocker-puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    try {
        const page = await browser.newPage();

        // Initialize the blocker with cookie banner filter list
        const blocker = await PuppeteerBlocker.fromLists(fetch, [
            'https://secure.fanboy.co.nz/fanboy-cookiemonster.txt',
        ]);

        // Enable blocking in the page
        await blocker.enableBlockingInPage(page);

        await page.goto('https://example.com');
        await page.screenshot({ path: 'screenshot.png' });
    } catch (error) {
        console.error('Error:', error);
    } finally {
        await browser.close();
    }
})();

This approach is particularly effective because:

It uses community-maintained filter lists
It's regularly updated with new cookie banner patterns
It's lightweight and efficient
It works across multiple websites

Method 2: Using AutoConsent

For more complex scenarios, we can combine the filter list approach with AutoConsent, a tool specifically designed to handle cookie consent dialogs:

const puppeteer = require('puppeteer');
const { PuppeteerBlocker } = require('@cliqz/adblocker-puppeteer');
const autoconsent = require('@duckduckgo/autoconsent');

(async () => {
    const browser = await puppeteer.launch();
    try {
        const page = await browser.newPage();

        // Initialize blocker
        const blocker = await PuppeteerBlocker.fromLists(fetch, [
            'https://secure.fanboy.co.nz/fanboy-cookiemonster.txt',
        ]);
        await blocker.enableBlockingInPage(page);

        // Inject AutoConsent
        await page.evaluateOnNewDocument(autoconsent.script);

        await page.goto('https://example.com');
        await page.screenshot({ path: 'screenshot.png' });
    } catch (error) {
        console.error('Error:', error);
    } finally {
        await browser.close();
    }
})();

Method 3: Custom Cookie Banner Handler

For maximum control, you can implement a custom handler that combines multiple approaches:

const puppeteer = require('puppeteer');

async function handleCookieBanners(page) {
    // Common cookie banner selectors
    const selectors = [
        "[id*='cookie']",
        "[class*='cookie']",
        "[id*='consent']",
        "[class*='consent']",
        "[id*='gdpr']",
        "[class*='gdpr']",
        '.modal__overlay',
        '.cookie-banner',
        '.consent-banner',
    ];

    // Common accept button texts
    const acceptTexts = [
        'Accept',
        'Accept All',
        'Accept Cookies',
        'Allow All',
        'I agree',
        'Got it',
        'Continue',
    ];

    // Try to click accept buttons
    await page.evaluate((texts) => {
        const buttons = Array.from(
            document.querySelectorAll('button, a, [role="button"]')
        );
        const acceptButton = buttons.find((button) =>
            texts.includes(button.innerText.trim())
        );
        if (acceptButton) acceptButton.click();
    }, acceptTexts);

    // Hide any remaining banners
    await page.evaluate((selectors) => {
        document
            .querySelectorAll(selectors.join(', '))
            .forEach((el) => (el.style.display = 'none'));
    }, selectors);
}

(async () => {
    const browser = await puppeteer.launch();
    try {
        const page = await browser.newPage();
        await page.goto('https://example.com');

        // Handle cookie banners
        await handleCookieBanners(page);

        // Take screenshot
        await page.screenshot({ path: 'screenshot.png' });
    } catch (error) {
        console.error('Error:', error);
    } finally {
        await browser.close();
    }
})();

Best Practices

When implementing cookie banner handling, consider these best practices:

Wait for Page Load: Ensure the page is fully loaded before handling banners
Multiple Approaches: Combine different methods for better coverage
Error Handling: Implement proper error handling for failed banner removal
Testing: Test across different websites and banner types
Maintenance: Regularly update selectors and patterns

Alternative: Using CaptureKit Screenshot API

If you're looking for a more reliable solution without managing browser instances, consider using CaptureKit's Screenshot API:

curl "https://api.capturekit.dev/capture?url=https://example.com&remove_cookie_banners=true&access_key=YOUR_ACCESS_KEY"

Benefits of using CaptureKit:

Built-in cookie banner handling
No need to manage browser instances
Consistent results across different websites
Regular updates to handle new banner types

Why Cookie Banners Matter

Now that we've covered the solutions, let's understand why handling cookie banners is crucial for your screenshot automation:

Content Visibility: Banners often cover critical content, making your screenshots incomplete
Consistency: Different banner states can lead to inconsistent results across captures
Layout Issues: Banners can trigger different page layouts or behaviors
Professional Quality: Clean screenshots without banners look more professional
Automation Reliability: Banners can interfere with automated testing and monitoring

Conclusion

Handling cookie banners is crucial for producing clean, professional screenshots. By combining filter lists, AutoConsent, and custom handlers, you can effectively manage most cookie banner scenarios. For production use cases, consider using CaptureKit's API to eliminate the complexity of browser automation.

Happy capturing! 📸

If you found this post useful, you might also enjoy these related Puppeteer tutorials:

Jonathan Geiger @geiger01