How to bypass Cloudflare bot protection in selenium

We Are Going To Discuss About How to bypass Cloudflare bot protection in selenium. So lets Start this Python Article.

How to bypass Cloudflare bot protection in selenium

  1. How to solve How to bypass Cloudflare bot protection in selenium

    I had this problem a long time ago and I was able to solve it. Use the code below and enjoy 🙂
    options = webdriver.ChromeOptions() options.add_experimental_option("excludeSwitches", ["enable-automation"]) options.add_experimental_option('useAutomationExtension', False) options.add_argument("--disable-blink-features=AutomationControlled") driver = webdriver.Chrome(options=options, executable_path=r"webdriver\chromedriver.exe")
    ///////// EDIT ////////////////
    this way now is not working !

  2. How to bypass Cloudflare bot protection in selenium

    I had this problem a long time ago and I was able to solve it. Use the code below and enjoy 🙂
    options = webdriver.ChromeOptions() options.add_experimental_option("excludeSwitches", ["enable-automation"]) options.add_experimental_option('useAutomationExtension', False) options.add_argument("--disable-blink-features=AutomationControlled") driver = webdriver.Chrome(options=options, executable_path=r"webdriver\chromedriver.exe")
    ///////// EDIT ////////////////
    this way now is not working !

Solution 1

I had this problem a long time ago and I was able to solve it. Use the code below and enjoy 🙂

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(options=options, executable_path=r"webdriver\chromedriver.exe")

///////// EDIT ////////////////
this way now is not working !

Original Author ma ja Of This Content

Solution 2

As March 2022 :

Hi, I had the same problem when using headless Selenium on a Docker Linux image.

I solved it by creating a virtualdisplay right before calling the webdriver:

from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 800))  
display.start()

Don’t forget to install both pyvirtualdisplay and xvfb:
pip install pyvirtualdisplay and sudo apt-get install xvfb

And you must remove the “headless” option in ChromeDriver, here is the complete code I use :

    #Display in order to avoid CloudFare bot detection
    display = Display(visible=0, size=(800, 800))  
    display.start()
  
    options = webdriver.ChromeOptions()
    options.add_argument('--no-sandbox')
    options.add_argument('start-maximized')
    options.add_argument('enable-automation')
    options.add_argument('--disable-infobars')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument('--disable-browser-side-navigation')
    options.add_argument("--remote-debugging-port=9222")
    # options.add_argument("--headless")
    options.add_argument('--disable-gpu')
    options.add_argument("--log-level=3")
    driver = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=options)

Since it was working nicely without headless on my local computer, I figured emulate a real display might do the work aswell.
I do not really understand why, but from what I’ve understood, CloudFare tries to execute javascript code in order to confirm you’re not a bot. Having a emulated webpage display helps to do so.

Original Author Jules Civel Of This Content

Solution 3

A bit of a late response and no wonder why developers still face this issue repetitively. I am using Java v17 and Gradle v7.4.2.
My solution is related to the one explained above but the code is in Java.

@Before
public void setup() {
    WebDriverManager.chromedriver().setup();
    ChromeOptions options = new ChromeOptions();
    // Bypass Cloudflare checks
    options.setExperimentalOption("useAutomationExtension", false);
    options.addArguments("--disable-blink-features=AutomationControlled");
    driver = new ChromeDriver(options);
    driver.manage().window().maximize();
}

Please refer to the Selenium ChromeOptions documentation for further details.

Happy coding.

Original Author Spicy strike Of This Content

Solution 4

Use undetected_chromedriver pip package. This is very simple package for fake client.

import undetected_chromedriver


def init_webdriver():
    driver = undetected_chromedriver.Chrome()

    driver.get(url)

    content = driver.page_content

    driver.close()
    driver.quit()

Also you can run it in background

import undetected_chromedriver
from selenium import webdriver



def init_webdriver():
    options = webdriver.ChromeOptions()
    options.add_argument("--headless")
    driver = undetected_chromedriver.Chrome(options)

    driver.get(url)

    content = driver.page_content

    driver.close()
    driver.quit()

I tested it 28 Jun 2022. Works very well.

Original Author Waujito Of This Content

Conclusion

So This is all About This Tutorial. Hope This Tutorial Helped You. Thank You.

Also Read,

ittutorial team

I am an Information Technology Engineer. I have Completed my MCA And I have 4 Year Plus Experience, I am a web developer with knowledge of multiple back-end platforms Like PHP, Node.js, Python and frontend JavaScript frameworks Like Angular, React, and Vue.

Leave a Comment