undetected-chromedriver/undetected_chromedriver/cdp.py

113 lines
3.4 KiB
Python
Raw Normal View History

3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
#!/usr/bin/env python3
# this module is part of undetected_chromedriver
import json
import logging
2021-11-29 06:39:21 -07:00
from collections.abc import Mapping, Sequence
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
import requests
import websockets
log = logging.getLogger(__name__)
2021-06-02 05:46:23 -06:00
class CDPObject(dict):
def __init__(self, *a, **k):
super().__init__(*a, **k)
self.__dict__ = self
for k in self.__dict__:
if isinstance(self.__dict__[k], dict):
self.__dict__[k] = CDPObject(self.__dict__[k])
elif isinstance(self.__dict__[k], list):
for i in range(len(self.__dict__[k])):
if isinstance(self.__dict__[k][i], dict):
self.__dict__[k][i] = CDPObject(self)
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
def __repr__(self):
tpl = f"{self.__class__.__name__}(\n\t{{}}\n\t)"
return tpl.format("\n ".join(f"{k} = {v}" for k, v in self.items()))
2021-06-02 05:46:23 -06:00
class PageElement(CDPObject):
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
pass
class CDP:
log = logging.getLogger("CDP")
endpoints = CDPObject(
{
"json": "/json",
"protocol": "/json/protocol",
"list": "/json/list",
"new": "/json/new?{url}",
"activate": "/json/activate/{id}",
"close": "/json/close/{id}",
}
)
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
2021-06-02 05:46:23 -06:00
def __init__(self, options: "ChromeOptions"): # noqa
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
self.server_addr = "http://{0}:{1}".format(*options.debugger_address.split(":"))
self._reqid = 0
self._session = requests.Session()
self._last_resp = None
self._last_json = None
2021-06-02 05:46:23 -06:00
resp = self.get(self.endpoints.json) # noqa
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
self.sessionId = resp[0]["id"]
self.wsurl = resp[0]["webSocketDebuggerUrl"]
2021-06-02 05:46:23 -06:00
def tab_activate(self, id=None):
if not id:
active_tab = self.tab_list()[0]
2021-06-02 05:46:23 -06:00
id = active_tab.id # noqa
self.wsurl = active_tab.webSocketDebuggerUrl # noqa
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
return self.post(self.endpoints["activate"].format(id=id))
def tab_list(self):
2021-06-02 05:46:23 -06:00
retval = self.get(self.endpoints["list"])
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
return [PageElement(o) for o in retval]
def tab_new(self, url):
return self.post(self.endpoints["new"].format(url=url))
def tab_close_last_opened(self):
sessions = self.tab_list()
opentabs = [s for s in sessions if s["type"] == "page"]
return self.post(self.endpoints["close"].format(id=opentabs[-1]["id"]))
async def send(self, method: str, params: dict):
self._reqid += 1
async with websockets.connect(self.wsurl) as ws:
await ws.send(
json.dumps({"method": method, "params": params, "id": self._reqid})
)
self._last_resp = await ws.recv()
self._last_json = json.loads(self._last_resp)
self.log.info(self._last_json)
def get(self, uri):
resp = self._session.get(self.server_addr + uri)
try:
self._last_resp = resp
self._last_json = resp.json()
except Exception:
return
else:
return self._last_json
2021-06-02 05:46:23 -06:00
def post(self, uri, data: dict = None):
if not data:
data = {}
resp = self._session.post(self.server_addr + uri, json=data)
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
try:
self._last_resp = resp
self._last_json = resp.json()
except Exception:
return self._last_resp
@property
def last_json(self):
return self._last_json