Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

We Are Going To Discuss About Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection. So lets Start this Java Article.

Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

  1. Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

    execute_cdp_cmd(): With the availability of execute_cdp_cmd(cmd, cmd_args) command now you can easily execute google-chrome-devtools commands using Selenium. Using this feature you can modify the navigator.webdriver easily to prevent Selenium from getting detected.

  2. Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection

    execute_cdp_cmd(): With the availability of execute_cdp_cmd(cmd, cmd_args) command now you can easily execute google-chrome-devtools commands using Selenium. Using this feature you can modify the navigator.webdriver easily to prevent Selenium from getting detected.

Solution 1

First the update 1

execute_cdp_cmd(): With the availability of execute_cdp_cmd(cmd, cmd_args) command now you can easily execute commands using Selenium. Using this feature you can modify the navigator.webdriver easily to prevent Selenium from getting detected.


Preventing Detection 2

To prevent Selenium driven WebDriver getting detected a niche approach would include either/all of the below mentioned steps:

  • Adding the argument –disable-blink-features=AutomationControlled

    from selenium import webdriver
    
    options = webdriver.ChromeOptions() 
    options.add_argument('--disable-blink-features=AutomationControlled')
    driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
    driver.get("https://www.website.com")
    

You can find a relevant detailed discussion in Selenium can’t open a second page

  • Rotating the through execute_cdp_cmd() command as follows:

    #Setting up Chrome/83.0.4103.53 as useragent
    driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
    
  • Change the property value of the navigator for webdriver to undefined

    driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
    
  • Exclude the collection of enable-automation switches

    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    
  • Turn-off useAutomationExtension

    options.add_experimental_option('useAutomationExtension', False)
    

Sample Code 3

Clubbing up all the steps mentioned above and effective code block will be:

from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.53 Safari/537.36'})
print(driver.execute_script("return navigator.userAgent;"))
driver.get('https://www.httpbin.org/headers')

History

As per the W3C Editor’s Draft the current implementation strictly mentions:

The webdriver-active flag is set to true when the user agent is under remote control which is initially set to false.

Further,

Navigator includes NavigatorAutomationInformation;

It is to be noted that:

The NavigatorAutomationInformation interface should not be exposed on WorkerNavigator.

The NavigatorAutomationInformation interface is defined as:

interface mixin NavigatorAutomationInformation {
    readonly attribute boolean webdriver;
};

which returns true if webdriver-active flag is set, false otherwise.

Finally, the navigator.webdriver defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, so that alternate code paths can be triggered during automation.

Caution: Altering/tweaking the above mentioned parameters may block the navigation and get the WebDriver instance detected.


Update (6-Nov-2019)

As of the current implementation an ideal way to access a web page without getting detected would be to use the ChromeOptions() class to add a couple of arguments to:

  • Exclude the collection of enable-automation switches
  • Turn-off useAutomationExtension

through an instance of ChromeOptions as follows:

  • Java Example:

    System.setProperty("webdriver.chrome.driver", "C:\\Utility\\BrowserDrivers\\chromedriver.exe");
    ChromeOptions options = new ChromeOptions();
    options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
    options.setExperimentalOption("useAutomationExtension", false);
    WebDriver driver =  new ChromeDriver(options);
    driver.get("https://www.google.com/");
    
  • Python Example

    from selenium import webdriver
    
    options = webdriver.ChromeOptions()
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('useAutomationExtension', False)
    driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
    driver.get("https://www.google.com/")
    
  • Ruby Example

      options = Selenium::WebDriver::Chrome::Options.new
      options.add_argument("--disable-blink-features=AutomationControlled")
      driver = Selenium::WebDriver.for :chrome, options: options
    

Legends

1: Applies to Selenium’s Python clients only.

2: Applies to Selenium’s Python clients only.

3: Applies to Selenium’s Python clients only.

Original Author undetected Selenium Of This Content

Solution 2

ChromeDriver:

Finally discovered the simple solution for this with a simple flag! 🙂

--disable-blink-features=AutomationControlled

navigator.webdriver=true will no longer show up with that flag set.

For a list of things you can disable, check them out here

Original Author 010011100101 Of This Content

Solution 3

Do not use cdp command to change webdriver value as it will lead to inconsistency which later can be used to detect webdriver. Use the below code, this will remove any traces of webdriver.

options.add_argument("--disable-blink-features")
options.add_argument("--disable-blink-features=AutomationControlled")

Original Author Baki Billah Of This Content

Solution 4

Before (in browser console window):

> navigator.webdriver
true

Change (in selenium):

// C#
var options = new ChromeOptions();
options.AddExcludedArguments(new List<string>() { "enable-automation" });

// Python
options.add_experimental_option("excludeSwitches", ['enable-automation'])

After (in browser console window):

> navigator.webdriver
undefined

This will not work for version ChromeDriver 79.0.3945.16 and above. See the release notes here

Original Author petke Of This Content

Conclusion

So This is all About This Tutorial. Hope This Tutorial Helped You. Thank You.

Also Read,

Siddharth

I am an Information Technology Engineer. I have Completed my MCA And I have 4 Year Plus Experience, I am a web developer with knowledge of multiple back-end platforms Like PHP, Node.js, Python and frontend JavaScript frameworks Like Angular, React, and Vue.

Leave a Comment