3.0.0 (#180)

*3.0.0
added lots of features and bugfixes

- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
    ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
    people having troubles with mobile emulation and other bullshit,
    can try again now.
- fixed the logic where sometimes options did not
    respect the given values
- for headless (though still not supperted for undetectability),
    added some real cool features which need to be set in
    the options object):

    defaults:
    emulate_touch = True
    mock_permissions = True  # headless had notificationpermissions
                                setup in a distinguisable way.
    mock_chrome_global = False
    mock_canvas_fp = True  # patch fingerprint

EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.

WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)


* 2.2.2

* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
    happens in a writable folder, instead of the current working dir.

Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>

* tidy up

* uncomment block

* - support for specifying and reusing the user profile folder.
    if a user-data-dir is specified, that folder will NOT be
    deleted on exit.
    example:
        options.add_argument('--user-data-dir=c:\\temp')

- uses a platform specific app data folder to store driver instead
    of the current workdir.

- impoved headless mode. fixed detection by notification perms.

- eliminates the "restore tabs" notification at startup

- added methods find_elements_by_text and find_element_by_text

- updated docs (partly)

-known issues:
    - extensions not running. this is due to the inner workings
        of chromedriver. still working on this.
    - driver window is not always closing along with a program exit.
    - MacOS: startup nag notifications. might be solved by
        re(using) a profile directory.

- known stuff:
    - some specific use cases, network conditions or behaviour
      can cause being detected.

* Squashed commit of the following:

commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:22:39 2021 +0200

    readme change

commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:18:09 2021 +0200

    - make sure options cannot be reused as it will
        cause double and conflicting arguments to chrome



    - support for specifying and reusing the user profile folder.
        if a user-data-dir is specified, that folder will NOT be
        deleted on exit.
        example:
            options.add_argument('--user-data-dir=c:\\temp')

    - uses a platform specific app data folder to store driver instead
        of the current workdir.

    - impoved headless mode. fixed detection by notification perms.

    - eliminates the "restore tabs" notification at startup

    - added methods find_elements_by_text and find_element_by_text

    - updated docs (partly)

    -known issues:
        - extensions not running. this is due to the inner workings
            of chromedriver. still working on this.
        - driver window is not always closing along with a program exit.
        - MacOS: startup nag notifications. might be solved by
            re(using) a profile directory.

    - known stuff:
        - some specific use cases, network conditions or behaviour
          can cause being detected.

setup.py

@@ -36,6 +36,8 @@ setup(
     packages=["undetected_chromedriver"],
     install_requires=[
         "selenium",
+        "requests",
+        "websockets",
     ],
     url="https://github.com/ultrafunkamsterdam/undetected-chromedriver",
     license="GPL-3.0",

undetected_chromedriver/__init__.py

@@ -19,19 +19,19 @@ by UltrafunkAmsterdam (https://github.com/ultrafunkamsterdam)
 import io
 import logging
 import os
+import random
 import re
+import string
 import sys
 import zipfile
-import string
-import random
 from distutils.version import LooseVersion
 from urllib.request import urlopen, urlretrieve
 
-from selenium.webdriver import Chrome as _Chrome
-from selenium.webdriver import ChromeOptions as _ChromeOptions
+from selenium.webdriver import Chrome as _Chrome, ChromeOptions as _ChromeOptions
 
 logger = logging.getLogger(__name__)
-__version__ = "2.2.7"
+__version__ = "3.0.0"
+
 
 TARGET_VERSION = 0
 
@@ -72,7 +72,9 @@ class Chrome:
                                            : target[key]
                                        })
                                    });
-                               """
+                                    
+                                                            
+                        """
                     },
                 )
             return instance._orig_get(*args, **kwargs)
@@ -120,7 +122,6 @@ class ChromeOptions:
 
 
 class ChromeDriverManager(object):
-
     installed = False
     selenium_patched = False
     target_version = None

undetected_chromedriver/cdp.py

@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+# this module is part of undetected_chromedriver
+
+import json
+import logging
+from collections import Mapping, Sequence
+
+import requests
+import websockets
+
+log = logging.getLogger(__name__)
+
+
+class CDPObjectBase(dict):
+    def __init__(self, *a, **kw):
+        super().__init__(**kw)
+        for k in self:
+            if isinstance(self[k], Mapping):
+                self[k] = self.__class__(self[k])  # noqa
+            elif isinstance(self[k], Sequence) and not isinstance(
+                self[k], (str, bytes)
+            ):
+                self[k] = self[k].__class__(self.__class__(i) for i in self[k])
+            else:
+                self[k] = self[k]
+
+    def __repr__(self):
+        tpl = f"{self.__class__.__name__}(\n\t{{}}\n\t)"
+        return tpl.format("\n  ".join(f"{k} = {v}" for k, v in self.items()))
+
+
+class PageElement(CDPObjectBase):
+    pass
+
+
+class CDP:
+    log = logging.getLogger("CDP")
+
+    endpoints = {
+        "json": "/json",
+        "protocol": "/json/protocol",
+        "list": "/json/list",
+        "new": "/json/new?{url}",
+        "activate": "/json/activate/{id}",
+        "close": "/json/close/{id}",
+    }
+
+    def __init__(self, options: "ChromeOptions"):
+        self.server_addr = "http://{0}:{1}".format(*options.debugger_address.split(":"))
+
+        self._reqid = 0
+        self._session = requests.Session()
+        self._last_resp = None
+        self._last_json = None
+
+        resp = self.get(self.endpoints["json"])
+        self.sessionId = resp[0]["id"]
+        self.wsurl = resp[0]["webSocketDebuggerUrl"]
+
+    def tab_activate(self, id):
+        return self.post(self.endpoints["activate"].format(id=id))
+
+    def tab_list(self):
+        retval = self.post(self.endpoints["list"])
+        return [PageElement(o) for o in retval]
+
+    def tab_new(self, url):
+        return self.post(self.endpoints["new"].format(url=url))
+
+    def tab_close_last_opened(self):
+        sessions = self.tab_list()
+        opentabs = [s for s in sessions if s["type"] == "page"]
+        return self.post(self.endpoints["close"].format(id=opentabs[-1]["id"]))
+
+    async def send(self, method: str, params: dict):
+        self._reqid += 1
+        async with websockets.connect(self.wsurl) as ws:
+            await ws.send(
+                json.dumps({"method": method, "params": params, "id": self._reqid})
+            )
+            self._last_resp = await ws.recv()
+            self._last_json = json.loads(self._last_resp)
+            self.log.info(self._last_json)
+
+    def get(self, uri):
+        resp = self._session.get(self.server_addr + uri)
+        try:
+            self._last_resp = resp
+            self._last_json = resp.json()
+        except Exception:
+            return
+        else:
+            return self._last_json
+
+    def post(self, uri):
+        resp = self._session.post(self.server_addr + uri)
+        try:
+            self._last_resp = resp
+            self._last_json = resp.json()
+        except Exception:
+            return self._last_resp
+
+    @property
+    def last_json(self):
+        return self._last_json

undetected_chromedriver/options.py

@@ -0,0 +1,257 @@
+#!/usr/bin/env python3
+# this module is part of undetected_chromedriver
+
+import base64
+import os
+
+from selenium.webdriver.chrome.options import Options as _ChromeOptions
+from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
+
+
+class ChromeOptions(_ChromeOptions):
+    KEY = "goog:chromeOptions"
+
+    session = None
+    emulate_touch = True
+    mock_permissions = True
+    mock_chrome_global = False
+    mock_canvas_fp = True
+    _user_data_dir = None
+
+    def __init__(self):
+        super().__init__()
+        self._arguments = []
+        self._binary_location = ""
+        self._extension_files = []
+        self._extensions = []
+        self._experimental_options = {}
+        self._debugger_address = None
+        self._caps = self.default_capabilities
+        self.mobile_options = None
+        self.set_capability("pageLoadStrategy", "normal")
+
+    @property
+    def user_data_dir(self):
+        return self._user_data_dir
+
+    @user_data_dir.setter
+    def user_data_dir(self, path: str):
+        """
+        Sets the browser profile folder to use, or creates a new profile
+        at given <path>.
+
+        Parameters
+        ----------
+        path: str
+            the path to a chrome profile folder
+            if it does not exist, a new profile will be created at given location
+        """
+        apath = os.path.abspath(path)
+        self._user_data_dir = os.path.normpath(apath)
+
+    @property
+    def arguments(self):
+        """
+        :Returns: A list of arguments needed for the browser
+        """
+        return self._arguments
+
+    @property
+    def binary_location(self) -> str:
+        """
+        :Returns: The location of the binary, otherwise an empty string
+        """
+        return self._binary_location
+
+    @binary_location.setter
+    def binary_location(self, value: str):
+        """
+        Allows you to set where the chromium binary lives
+        :Args:
+         - value: path to the Chromium binary
+        """
+        self._binary_location = value
+
+    @property
+    def debugger_address(self) -> str:
+        """
+        :Returns: The address of the remote devtools instance
+        """
+        return self._debugger_address
+
+    @debugger_address.setter
+    def debugger_address(self, value: str):
+        """
+        Allows you to set the address of the remote devtools instance
+        that the ChromeDriver instance will try to connect to during an
+        active wait.
+        :Args:
+         - value: address of remote devtools instance if any (hostname[:port])
+        """
+        self._debugger_address = value
+
+    @property
+    def extensions(self):
+        """
+        :Returns: A list of encoded extensions that will be loaded
+        """
+        encoded_extensions = []
+        for ext in self._extension_files:
+            file_ = open(ext, "rb")
+            # Should not use base64.encodestring() which inserts newlines every
+            # 76 characters (per RFC 1521).  Chromedriver has to remove those
+            # unnecessary newlines before decoding, causing performance hit.
+            encoded_extensions.append(base64.b64encode(file_.read()).decode("UTF-8"))
+            file_.close()
+        return encoded_extensions + self._extensions
+
+    def add_extension(self, extension: str):
+        """
+        Adds the path to the extension to a list that will be used to extract it
+        to the ChromeDriver
+        :Args:
+         - extension: path to the \\*.crx file
+        """
+        if extension:
+            extension_to_add = os.path.abspath(os.path.expanduser(extension))
+            if os.path.exists(extension_to_add):
+                self._extension_files.append(extension_to_add)
+            else:
+                raise IOError("Path to the extension doesn't exist")
+        else:
+            raise ValueError("argument can not be null")
+
+    def add_encoded_extension(self, extension: str):
+        """
+        Adds Base64 encoded string with extension data to a list that will be used to extract it
+        to the ChromeDriver
+        :Args:
+         - extension: Base64 encoded string with extension data
+        """
+        if extension:
+            self._extensions.append(extension)
+        else:
+            raise ValueError("argument can not be null")
+
+    @property
+    def experimental_options(self) -> dict:
+        """
+        :Returns: A dictionary of experimental options for chromium
+        """
+        return self._experimental_options
+
+    def add_experimental_option(self, name: str, value: dict):
+        """
+        Adds an experimental option which is passed to chromium.
+        :Args:
+          name: The experimental option name.
+          value: The option value.
+        """
+        self._experimental_options[name] = value
+
+    @property
+    def headless(self) -> bool:
+        """
+        :Returns: True if the headless argument is set, else False
+        """
+        return "--headless" in self._arguments
+
+    @headless.setter
+    def headless(self, value: bool):
+        """
+        Sets the headless argument
+        :Args:
+          value: boolean value indicating to set the headless option
+        """
+        args = {"--headless"}
+        if value is True:
+            self._arguments.extend(args)
+        else:
+            self._arguments = list(set(self._arguments) - args)
+
+    @property
+    def page_load_strategy(self) -> str:
+        return self._caps["pageLoadStrategy"]
+
+    @page_load_strategy.setter
+    def page_load_strategy(self, strategy: str):
+        if strategy in ["normal", "eager", "none"]:
+            self.set_capability("pageLoadStrategy", strategy)
+        else:
+            raise ValueError(
+                "Strategy can only be one of the following: normal, eager, none"
+            )
+
+    @property
+    def capabilities(self):
+        return self._caps
+
+    def set_capability(self, name, value):
+        """ Sets a capability """
+        self._caps[name] = value
+
+    def to_capabilities(self) -> dict:
+        """
+        Creates a capabilities with all the options that have been set
+        :Returns: A dictionary with everything
+        """
+        caps = self._caps
+        chrome_options = self.experimental_options.copy()
+        if self.mobile_options:
+            chrome_options.update(self.mobile_options)
+        chrome_options["extensions"] = self.extensions
+        if self.binary_location:
+            chrome_options["binary"] = self.binary_location
+        chrome_options["args"] = self._arguments
+        if self.debugger_address:
+            chrome_options["debuggerAddress"] = self.debugger_address
+
+        caps[self.KEY] = chrome_options
+
+        return caps
+
+    def ignore_local_proxy_environment_variables(self):
+        """
+        By calling this you will ignore HTTP_PROXY and HTTPS_PROXY from being picked up and used.
+        """
+        self._ignore_local_proxy = True
+
+    @property
+    def default_capabilities(self) -> dict:
+        return DesiredCapabilities.CHROME.copy()
+
+    def enable_mobile(
+        self,
+        android_package: str = None,
+        android_activity: str = None,
+        device_serial: str = None,
+    ):
+        """
+        Enables mobile browser use for browsers that support it
+        :Args:
+            android_activity: The name of the android package to start
+        """
+        if not android_package:
+            raise AttributeError("android_package must be passed in")
+        self.mobile_options = {"androidPackage": android_package}
+        if android_activity:
+            self.mobile_options["androidActivity"] = android_activity
+        if device_serial:
+            self.mobile_options["androidDeviceSerial"] = device_serial
+
+    def add_argument(self, argument):
+        """
+        Adds an argument to the list
+        :Args:
+         - Sets the arguments
+        """
+        if argument:
+            self._arguments.append(argument)
+        else:
+            raise ValueError("argument can not be null")
+
+    @classmethod
+    def from_options(cls, options):
+        o = cls()
+        o.__dict__.update(options.__dict__)
+        return o

undetected_chromedriver/patcher.py

@@ -0,0 +1,222 @@
+#!/usr/bin/env python3
+# this module is part of undetected_chromedriver
+
+import io
+import logging
+import os
+import random
+import re
+import string
+import sys
+import zipfile
+from distutils.version import LooseVersion
+from urllib.request import urlopen, urlretrieve
+
+logger = logging.getLogger(__name__)
+
+IS_POSIX = sys.platform.startswith(("darwin", "cygwin", "linux"))
+
+
+class Patcher(object):
+    url_repo = "https://chromedriver.storage.googleapis.com"
+    zip_name = "chromedriver_%s.zip"
+    exe_name = "chromedriver%s"
+
+    platform = sys.platform
+    if platform.endswith("win32"):
+        zip_name %= "win32"
+        exe_name %= ".exe"
+    if platform.endswith("linux"):
+        zip_name %= "linux64"
+        exe_name %= ""
+    if platform.endswith("darwin"):
+        zip_name %= "mac64"
+        exe_name %= ""
+
+    if platform.endswith("win32"):
+        d = "~/appdata/roaming/undetected_chromedriver"
+    elif platform.startswith("linux"):
+        d = "~/.local/share/undetected_chromedriver"
+    elif platform.endswith("darwin"):
+        d = "~/Library/Application Support/undetected_chromedriver"
+    else:
+        d = "~/.undetected_chromedriver"
+    data_path = os.path.abspath(os.path.expanduser(d))
+
+    def __init__(self, executable_path=None, force=False, version_main: int = 0):
+        """
+
+        Args:
+            executable_path: None = automatic
+                             a full file path to the chromedriver executable
+            force: False
+                    terminate processes which are holding lock
+            version_main: 0 = auto
+                specify main chrome version (rounded, ex: 82)
+        """
+
+        self.force = force
+
+        if not executable_path:
+            executable_path = os.path.join(self.data_path, self.exe_name)
+
+        if not IS_POSIX:
+            if not executable_path[-4:] == ".exe":
+                executable_path += ".exe"
+
+        self.zip_path = os.path.join(self.data_path, self.zip_name)
+
+        self.executable_path = os.path.abspath(os.path.join(".", executable_path))
+
+        self.version_main = version_main
+        self.version_full = None
+
+    @classmethod
+    def auto(cls, executable_path=None, force=False):
+        """
+
+        Args:
+            force:
+
+        Returns:
+
+        """
+        i = cls(executable_path, force=force)
+        try:
+            os.unlink(i.executable_path)
+        except PermissionError:
+            if i.force:
+                cls.force_kill_instances(i.executable_path)
+                return i.auto(force=False)
+            try:
+                if i.is_binary_patched():
+                    # assumes already running AND patched
+                    return True
+            except PermissionError:
+                pass
+            # return False
+        except FileNotFoundError:
+            pass
+
+        release = i.fetch_release_number()
+        i.version_main = release.version[0]
+        i.version_full = release
+        i.unzip_package(i.fetch_package())
+        i.patch()
+        return i
+
+    def patch(self):
+        self.patch_exe()
+        return self.is_binary_patched()
+
+    def fetch_release_number(self):
+        """
+        Gets the latest major version available, or the latest major version of self.target_version if set explicitly.
+        :return: version string
+        :rtype: LooseVersion
+        """
+        path = "/latest_release"
+        if self.version_main:
+            path += f"_{self.version_main}"
+        path = path.upper()
+        logger.debug("getting release number from %s" % path)
+        return LooseVersion(urlopen(self.url_repo + path).read().decode())
+
+    def parse_exe_version(self):
+        with io.open(self.executable_path, "rb") as f:
+            for line in iter(lambda: f.readline(), b""):
+                match = re.search(br"platform_handle\x00content\x00([0-9.]*)", line)
+                if match:
+                    return LooseVersion(match[1].decode())
+
+    def fetch_package(self):
+        """
+        Downloads ChromeDriver from source
+
+        :return: path to downloaded file
+        """
+        u = "%s/%s/%s" % (self.url_repo, self.version_full.vstring, self.zip_name)
+        logger.debug("downloading from %s" % u)
+        # return urlretrieve(u, filename=self.data_path)[0]
+        return urlretrieve(u)[0]
+
+    def unzip_package(self, fp):
+        """
+        Does what it says
+
+        :return: path to unpacked executable
+        """
+        logger.debug("unzipping %s" % fp)
+        try:
+            os.unlink(self.zip_path)
+        except (FileNotFoundError, OSError):
+            pass
+
+        os.makedirs(self.data_path, mode=0o755, exist_ok=True)
+
+        with zipfile.ZipFile(fp, mode="r") as zf:
+            zf.extract(self.exe_name, os.path.dirname(self.executable_path))
+        os.remove(fp)
+        os.chmod(self.executable_path, 0o755)
+        return self.executable_path
+
+    @staticmethod
+    def force_kill_instances(exe_name):
+        """
+        kills running instances.
+        :param: executable name to kill, may be a path as well
+
+        :return: True on success else False
+        """
+        exe_name = os.path.basename(exe_name)
+        if IS_POSIX:
+            r = os.system("kill -f -9 $(pidof %s)" % exe_name)
+        else:
+            r = os.system("taskkill /f /im %s" % exe_name)
+        return not r
+
+    @staticmethod
+    def gen_random_cdc():
+        cdc = random.choices(string.ascii_lowercase, k=26)
+        cdc[-6:-4] = map(str.upper, cdc[-6:-4])
+        cdc[2] = cdc[0]
+        cdc[3] = "_"
+        return "".join(cdc).encode()
+
+    def is_binary_patched(self, executable_path=None):
+        """simple check if executable is patched.
+
+        :return: False if not patched, else True
+        """
+        executable_path = executable_path or self.executable_path
+        with io.open(executable_path, "rb") as fh:
+            for line in iter(lambda: fh.readline(), b""):
+                if b"cdc_" in line:
+                    return False
+            else:
+                return True
+
+    def patch_exe(self):
+        """
+        Patches the ChromeDriver binary
+
+        :return: False on failure, binary name on success
+        """
+        logger.info("patching driver executable %s" % self.executable_path)
+
+        linect = 0
+        replacement = self.gen_random_cdc()
+        with io.open(self.executable_path, "r+b") as fh:
+            for line in iter(lambda: fh.readline(), b""):
+                if b"cdc_" in line:
+                    fh.seek(-len(line), 1)
+                    newline = re.sub(b"cdc_.{22}", replacement, line)
+                    fh.write(newline)
+                    linect += 1
+            return linect
+
+    def __repr__(self):
+        return "{0:s}({1:s})".format(
+            self.__class__.__name__,
+            self.executable_path,
+        )

undetected_chromedriver/reactor.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+# this module is part of undetected_chromedriver
+
+import asyncio
+import json
+import logging
+import threading
+
+logger = logging.getLogger(__name__)
+
+
+class Reactor(threading.Thread):
+    def __init__(self, driver: "Chrome"):
+        super().__init__()
+
+        self.driver = driver
+        self.loop = asyncio.new_event_loop()
+
+        self.lock = threading.Lock()
+        self.event = threading.Event()
+        self.daemon = True
+        self.handlers = {}
+
+    def add_event_handler(self, method_name, callback: callable):
+        """
+
+        Parameters
+        ----------
+        event_name: str
+            example "Network.responseReceived"
+
+        callback: callable
+            callable which accepts 1 parameter: the message object dictionary
+
+        Returns
+        -------
+
+        """
+        with self.lock:
+            self.handlers[method_name.lower()] = callback
+
+    @property
+    def running(self):
+        return not self.event.is_set()
+
+    def run(self):
+        try:
+            asyncio.set_event_loop(self.loop)
+            self.loop.run_until_complete(self.listen())
+        except Exception as e:
+            logger.warning("Reactor.run() => %s", e)
+
+    async def listen(self):
+
+        while self.running:
+
+            await asyncio.sleep(0)
+
+            try:
+                with self.lock:
+                    log_entries = self.driver.get_log("performance")
+
+                for entry in log_entries:
+
+                    try:
+
+                        obj_serialized: str = entry.get("message")
+                        obj = json.loads(obj_serialized)
+                        message = obj.get("message")
+                        method = message.get("method")
+
+                        if "*" in self.handlers:
+                            await self.loop.run_in_executor(
+                                None, self.handlers["*"], message
+                            )
+                        elif method.lower() in self.handlers:
+                            await self.loop.run_in_executor(
+                                None, self.handlers[method.lower()], message
+                            )
+
+                        # print(type(message), message)
+                    except Exception as e:
+                        raise e from None
+
+            except Exception as e:
+                if "invalid session id" in str(e):
+                    pass
+                else:
+                    logging.debug("exception ignored :", e)

undetected_chromedriver/tests/fp.js

@@ -0,0 +1,319 @@
+(function (name, context, definition) {
+    if (typeof module !== 'undefined' && module.exports) {
+        module.exports = definition();
+    } else if (typeof define === 'function' && define.amd) {
+        define(definition);
+    } else {
+        context[name] = definition();
+    }
+})('Fingerprint', this, function () {
+    'use strict';
+
+    var Fingerprint = function (options) {
+        var nativeForEach, nativeMap;
+        nativeForEach = Array.prototype.forEach;
+        nativeMap = Array.prototype.map;
+
+        this.each = function (obj, iterator, context) {
+            if (obj === null) {
+                return;
+            }
+            if (nativeForEach && obj.forEach === nativeForEach) {
+                obj.forEach(iterator, context);
+            } else if (obj.length === +obj.length) {
+                for (var i = 0, l = obj.length; i < l; i++) {
+                    if (iterator.call(context, obj[i], i, obj) === {}) return;
+                }
+            } else {
+                for (var key in obj) {
+                    if (obj.hasOwnProperty(key)) {
+                        if (iterator.call(context, obj[key], key, obj) === {}) return;
+                    }
+                }
+            }
+        };
+
+        this.map = function (obj, iterator, context) {
+            var results = [];
+            // Not using strict equality so that this acts as a
+            // shortcut to checking for `null` and `undefined`.
+            if (obj == null) return results;
+            if (nativeMap && obj.map === nativeMap) return obj.map(iterator, context);
+            this.each(obj, function (value, index, list) {
+                results[results.length] = iterator.call(context, value, index, list);
+            });
+            return results;
+        };
+
+        if (typeof options == 'object') {
+            this.hasher = options.hasher;
+            this.screen_resolution = options.screen_resolution;
+            this.screen_orientation = options.screen_orientation;
+            this.canvas = options.canvas;
+            this.ie_activex = options.ie_activex;
+        } else if (typeof options == 'function') {
+            this.hasher = options;
+        }
+    };
+
+    Fingerprint.prototype = {
+        get: function () {
+            var keys = [];
+            keys.push(navigator.userAgent);
+            keys.push(navigator.language);
+            keys.push(screen.colorDepth);
+            if (this.screen_resolution) {
+                var resolution = this.getScreenResolution();
+                if (typeof resolution !== 'undefined') { // headless browsers, such as phantomjs
+                    keys.push(resolution.join('x'));
+                }
+            }
+            keys.push(new Date().getTimezoneOffset());
+            keys.push(this.hasSessionStorage());
+            keys.push(this.hasLocalStorage());
+            keys.push(this.hasIndexDb());
+            //body might not be defined at this point or removed programmatically
+            if (document.body) {
+                keys.push(typeof (document.body.addBehavior));
+            } else {
+                keys.push(typeof undefined);
+            }
+            keys.push(typeof (window.openDatabase));
+            keys.push(navigator.cpuClass);
+            keys.push(navigator.platform);
+            keys.push(navigator.doNotTrack);
+            keys.push(this.getPluginsString());
+            if (this.canvas && this.isCanvasSupported()) {
+                keys.push(this.getCanvasFingerprint());
+            }
+            if (this.hasher) {
+                return this.hasher(keys.join('###'), 31);
+            } else {
+                return this.murmurhash3_32_gc(keys.join('###'), 31);
+            }
+        },
+
+        /**
+         * JS Implementation of MurmurHash3 (r136) (as of May 20, 2011)
+         *
+         * @author Gary Court
+         * @see http://github.com/garycourt/murmurhash-js
+         * @author Austin Appleby
+         * @see http://sites.google.com/site/murmurhash/
+         *
+         * @param {string} key ASCII only
+         * @param {number} seed Positive integer only
+         * @return {number} 32-bit positive integer hash
+         */
+
+        murmurhash3_32_gc: function (key, seed) {
+            var remainder, bytes, h1, h1b, c1, c2, k1, i;
+
+            remainder = key.length & 3; // key.length % 4
+            bytes = key.length - remainder;
+            h1 = seed;
+            c1 = 0xcc9e2d51;
+            c2 = 0x1b873593;
+            i = 0;
+
+            while (i < bytes) {
+                k1 =
+                    ((key.charCodeAt(i) & 0xff)) |
+                    ((key.charCodeAt(++i) & 0xff) << 8) |
+                    ((key.charCodeAt(++i) & 0xff) << 16) |
+                    ((key.charCodeAt(++i) & 0xff) << 24);
+                ++i;
+
+                k1 = ((((k1 & 0xffff) * c1) + ((((k1 >>> 16) * c1) & 0xffff) << 16))) & 0xffffffff;
+                k1 = (k1 << 15) | (k1 >>> 17);
+                k1 = ((((k1 & 0xffff) * c2) + ((((k1 >>> 16) * c2) & 0xffff) << 16))) & 0xffffffff;
+
+                h1 ^= k1;
+                h1 = (h1 << 13) | (h1 >>> 19);
+                h1b = ((((h1 & 0xffff) * 5) + ((((h1 >>> 16) * 5) & 0xffff) << 16))) & 0xffffffff;
+                h1 = (((h1b & 0xffff) + 0x6b64) + ((((h1b >>> 16) + 0xe654) & 0xffff) << 16));
+            }
+
+            k1 = 0;
+
+            switch (remainder) {
+                case 3:
+                    k1 ^= (key.charCodeAt(i + 2) & 0xff) << 16;
+                case 2:
+                    k1 ^= (key.charCodeAt(i + 1) & 0xff) << 8;
+                case 1:
+                    k1 ^= (key.charCodeAt(i) & 0xff);
+
+                    k1 = (((k1 & 0xffff) * c1) + ((((k1 >>> 16) * c1) & 0xffff) << 16)) & 0xffffffff;
+                    k1 = (k1 << 15) | (k1 >>> 17);
+                    k1 = (((k1 & 0xffff) * c2) + ((((k1 >>> 16) * c2) & 0xffff) << 16)) & 0xffffffff;
+                    h1 ^= k1;
+            }
+
+            h1 ^= key.length;
+
+            h1 ^= h1 >>> 16;
+            h1 = (((h1 & 0xffff) * 0x85ebca6b) + ((((h1 >>> 16) * 0x85ebca6b) & 0xffff) << 16)) & 0xffffffff;
+            h1 ^= h1 >>> 13;
+            h1 = ((((h1 & 0xffff) * 0xc2b2ae35) + ((((h1 >>> 16) * 0xc2b2ae35) & 0xffff) << 16))) & 0xffffffff;
+            h1 ^= h1 >>> 16;
+
+            return h1 >>> 0;
+        },
+
+        // https://bugzilla.mozilla.org/show_bug.cgi?id=781447
+        hasLocalStorage: function () {
+            try {
+                return !!window.localStorage;
+            } catch (e) {
+                return true; // SecurityError when referencing it means it exists
+            }
+        },
+
+        hasSessionStorage: function () {
+            try {
+                return !!window.sessionStorage;
+            } catch (e) {
+                return true; // SecurityError when referencing it means it exists
+            }
+        },
+
+        hasIndexDb: function () {
+            try {
+                return !!window.indexedDB;
+            } catch (e) {
+                return true; // SecurityError when referencing it means it exists
+            }
+        },
+
+        isCanvasSupported: function () {
+            var elem = document.createElement('canvas');
+            return !!(elem.getContext && elem.getContext('2d'));
+        },
+
+        isIE: function () {
+            if (navigator.appName === 'Microsoft Internet Explorer') {
+                return true;
+            } else if (navigator.appName === 'Netscape' && /Trident/.test(navigator.userAgent)) {// IE 11
+                return true;
+            }
+            return false;
+        },
+
+        getPluginsString: function () {
+            if (this.isIE() && this.ie_activex) {
+                return this.getIEPluginsString();
+            } else {
+                return this.getRegularPluginsString();
+            }
+        },
+
+        getRegularPluginsString: function () {
+            return this.map(navigator.plugins, function (p) {
+                var mimeTypes = this.map(p, function (mt) {
+                    return [mt.type, mt.suffixes].join('~');
+                }).join(',');
+                return [p.name, p.description, mimeTypes].join('::');
+            }, this).join(';');
+        },
+
+        getIEPluginsString: function () {
+            if (window.ActiveXObject) {
+                var names = ['ShockwaveFlash.ShockwaveFlash',//flash plugin
+                    'AcroPDF.PDF', // Adobe PDF reader 7+
+                    'PDF.PdfCtrl', // Adobe PDF reader 6 and earlier, brrr
+                    'QuickTime.QuickTime', // QuickTime
+                    // 5 versions of real players
+                    'rmocx.RealPlayer G2 Control',
+                    'rmocx.RealPlayer G2 Control.1',
+                    'RealPlayer.RealPlayer(tm) ActiveX Control (32-bit)',
+                    'RealVideo.RealVideo(tm) ActiveX Control (32-bit)',
+                    'RealPlayer',
+                    'SWCtl.SWCtl', // ShockWave player
+                    'WMPlayer.OCX', // Windows media player
+                    'AgControl.AgControl', // Silverlight
+                    'Skype.Detection'];
+
+                // starting to detect plugins in IE
+                return this.map(names, function (name) {
+                    try {
+                        new ActiveXObject(name);
+                        return name;
+                    } catch (e) {
+                        return null;
+                    }
+                }).join(';');
+            } else {
+                return ""; // behavior prior version 0.5.0, not breaking backwards compat.
+            }
+        },
+
+        getScreenResolution: function () {
+            var resolution;
+            if (this.screen_orientation) {
+                resolution = (screen.height > screen.width) ? [screen.height, screen.width] : [screen.width, screen.height];
+            } else {
+                resolution = [screen.height, screen.width];
+            }
+            return resolution;
+        },
+
+        getCanvasFingerprint: function () {
+            var canvas = document.createElement('canvas');
+            var ctx = canvas.getContext('2d');
+            // https://www.browserleaks.com/canvas#how-does-it-work
+            var txt = 'http://valve.github.io';
+            ctx.textBaseline = "top";
+            ctx.font = "14px 'Arial'";
+            ctx.textBaseline = "alphabetic";
+            ctx.fillStyle = "#f60";
+            ctx.fillRect(125, 1, 62, 20);
+            ctx.fillStyle = "#069";
+            ctx.fillText(txt, 2, 15);
+            ctx.fillStyle = "rgba(102, 204, 0, 0.7)";
+            ctx.fillText(txt, 4, 17);
+            return canvas.toDataURL();
+        }
+    };
+    return Fingerprint;
+});
+
+
+new Fingerprint({canvas: true}).get();
+
+
+var inject = function () {
+    var overwrite = function (name) {
+        const OLD = HTMLCanvasElement.prototype[name];
+        Object.defineProperty(HTMLCanvasElement.prototype, name, {
+            "value": function () {
+                var shift = {
+                    'r': Math.floor(Math.random() * 10) - 5,
+                    'g': Math.floor(Math.random() * 10) - 5,
+                    'b': Math.floor(Math.random() * 10) - 5,
+                    'a': Math.floor(Math.random() * 10) - 5
+                };
+                var width = this.width, height = this.height, context = this.getContext("2d");
+                var imageData = context.getImageData(0, 0, width, height);
+                for (var i = 0; i < height; i++) {
+                    for (var j = 0; j < width; j++) {
+                        var n = ((i * (width * 4)) + (j * 4));
+                        imageData.data[n + 0] = imageData.data[n + 0] + shift.r;
+                        imageData.data[n + 1] = imageData.data[n + 1] + shift.g;
+                        imageData.data[n + 2] = imageData.data[n + 2] + shift.b;
+                        imageData.data[n + 3] = imageData.data[n + 3] + shift.a;
+                    }
+                }
+                context.putImageData(imageData, 0, 0);
+                return OLD.apply(this, arguments);
+            }
+        });
+    };
+    overwrite('toBlob');
+    overwrite('toDataURL');
+};
+inject();
+
+
+new Fingerprint({canvas: true}).get();
+

undetected_chromedriver/tests/test_undetected_chromedriver.py

@@ -1,16 +1,12 @@
 import logging
-import os
 import sys
 import time  # noqa
 
-from ..v2 import *
-
 logging.basicConfig(level=10)
 
 logger = logging.getLogger("TEST")
 logger.setLevel(20)
 
-
 JS_SERIALIZE_FUNCTION = """
 decycle=function(n,e){"use strict";var t=new WeakMap;return function n(o,r){var c,i;return void 0!==e&&(o=e(o)),"object"!=typeof o||null===o||o instanceof Boolean||o instanceof Date||o instanceof Number||o instanceof RegExp||o instanceof String?o:void 0!==(c=t.get(o))?{$ref:c}:(t.set(o,r),Array.isArray(o)?(i=[],o.forEach(function(e,t){i[t]=n(e,r+"["+t+"]")})):(i={},Object.keys(o).forEach(function(e){i[e]=n(o[e],r+"["+JSON.stringify(e)+"]")})),i)}(n,"$")};
 function replacer(t){try{if(Array.prototype.splice.call(t).length<100){let e={};for(let r in t)e[r]=t[r];return e}}catch(t){}}
@@ -42,9 +38,10 @@ def test_undetected_chromedriver():
     driver = uc.Chrome()
 
     with driver:
-        driver.get("https://coinfaucet.eu")
+
+        driver.get("https://nowsecure.nl")
     time.sleep(4)  # sleep only used for timing of screenshot
-    driver.save_screenshot("coinfaucet.eu.png")
+    driver.save_screenshot("nowsecure.nl.png")
 
     with driver:
         driver.get("https://cia.gov")

undetected_chromedriver/tests/test_undetected_chromedriver_funstuff.py

@@ -0,0 +1,67 @@
+import asyncio
+import logging
+
+import cv2
+
+import undetected_chromedriver.v2 as uc
+
+logging.basicConfig(level=10)
+
+just_some_urls = [
+    "https://bing.com",
+    "http://www.google.com",
+    "https://codepen.io",
+    "https://",
+]
+
+
+class ChromeDriverCV2Streamer:
+    def __init__(self, driver):
+        super().__init__()
+        self.driver = driver
+        self.display = None
+        self.event = asyncio.Event()
+        self.daemon = True
+
+    def stop(self):
+        self.event.set()
+
+    def start(self):
+        asyncio.ensure_future(self._start_capture_loop())
+
+    async def _start_capture_loop(self):
+        executor = None
+        self.display = cv2.namedWindow("display")
+        while not self.event.is_set():
+            await asyncio.sleep(0.25)
+            try:
+                success = await loop.run_in_executor(
+                    executor, self.driver.save_screenshot, "capture.tmp.png"
+                )
+                logging.getLogger().debug("got screenshot? %s", success)
+                frame = await loop.run_in_executor(
+                    executor, cv2.imread, "capture.tmp.png"
+                )
+                logging.getLogger().debug("frame: %s", frame)
+                await loop.run_in_executor(executor, cv2.imshow, "display", frame)
+                await loop.run_in_executor(executor, cv2.waitKey, 1)
+                logging.getLogger().debug("waited key success")
+            except Exception as e:
+                print(e)
+
+
+async def main():
+    opts = uc.ChromeOptions()
+    opts.headless = True
+    driver = uc.Chrome(options=opts)
+
+    streamer = ChromeDriverCV2Streamer(driver)
+    streamer.start()
+    for url in just_some_urls:
+        # with driver:
+        driver.get("https://nu.nl")
+        await asyncio.sleep(3)
+
+
+loop = asyncio.get_event_loop()
+loop.run_until_complete(main())

undetected_chromedriver/tests/test_undetected_chromedriver_with_cdp_events.py

@@ -0,0 +1,63 @@
+# coding: utf-8
+import logging
+import os
+import sys
+
+import undetected_chromedriver.v2 as uc
+
+# it's not required to enable logging for cdp events to work
+# but as this is a test, it's good too it all
+logging.basicConfig(level=10)
+logging.getLogger("urllib3").setLevel(logging.WARNING)
+logging.getLogger("selenium.webdriver.remote.remote_connection").setLevel(logging.WARN)
+
+driver = uc.Chrome(enable_cdp_events=True)
+
+# set the callback to Network.dataReceived to print (yeah not much original)
+driver.add_cdp_listener("Network.dataReceived", print)
+
+# example of executing regular cdp commands
+driver.execute_cdp_cmd("Network.getAllCookies", {})
+
+# okay another one
+driver.execute_cdp_cmd(
+    "Page.addScriptToEvaluateOnNewDocument",
+    {"source": """  alert('another new document')"""},
+)
+
+# set the callback for ALL events (this may slow down execution)
+# driver.add_cdp_listener('*', print)
+
+
+with driver:
+    driver.get("https://nowsecure.nl")
+driver.save_screenshot("nowsecure.nl.headfull.png")
+try:
+    os.system("nowsecure.nl.headfull.png")
+except:
+    pass
+
+driver.quit()
+
+opts = uc.ChromeOptions()
+opts.headless = True
+driver = uc.Chrome(enable_cdp_events=True, options=opts)
+
+# okay another one
+driver.execute_cdp_cmd(
+    "Page.addScriptToEvaluateOnNewDocument",
+    {"source": """  alert('another new document')"""},
+)
+
+driver.add_cdp_listener("*", print)
+
+with driver:
+    driver.get("https://nowsecure.nl")
+    driver.save_screenshot("nowsecure.nl.headfull.png")
+try:
+    os.system("nowsecure.nl.headfull.png")
+except:
+    pass
+
+while True:
+    sys.stdin.read()