undetected-chromedriver/undetected_chromedriver/reactor.py

100 lines
2.9 KiB
Python
Raw Normal View History

3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
#!/usr/bin/env python3
# this module is part of undetected_chromedriver
import asyncio
import json
import logging
import threading
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
logger = logging.getLogger(__name__)
class Reactor(threading.Thread):
def __init__(self, driver: "Chrome"):
super().__init__()
self.driver = driver
self.loop = asyncio.new_event_loop()
self.lock = threading.Lock()
self.event = threading.Event()
self.daemon = True
self.handlers = {}
def add_event_handler(self, method_name, callback: callable):
"""
Parameters
----------
event_name: str
example "Network.responseReceived"
callback: callable
callable which accepts 1 parameter: the message object dictionary
Returns
-------
"""
with self.lock:
self.handlers[method_name.lower()] = callback
@property
def running(self):
return not self.event.is_set()
def run(self):
try:
asyncio.set_event_loop(self.loop)
self.loop.run_until_complete(self.listen())
except Exception as e:
logger.warning("Reactor.run() => %s", e)
async def _wait_service_started(self):
while True:
with self.lock:
if (
2021-12-22 07:07:27 -07:00
getattr(self.driver, "service", None)
and getattr(self.driver.service, "process", None)
and self.driver.service.process.poll()
):
await asyncio.sleep(self.driver._delay or 0.25)
else:
break
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
async def listen(self):
while self.running:
await self._wait_service_started()
await asyncio.sleep(1)
3.0.0 (#180) *3.0.0 added lots of features and bugfixes - You can now subscribe to Chrome Devtools Protocol Events like networking. - splitted the project up in seperate modules now - fixed locale (accept-language) - you can enter your user-data-folder as property of ChromeOptions() now. - The ChromeOptions had a makeover, and i took the one from alpha 4, people having troubles with mobile emulation and other bullshit, can try again now. - fixed the logic where sometimes options did not respect the given values - for headless (though still not supperted for undetectability), added some real cool features which need to be set in the options object): defaults: emulate_touch = True mock_permissions = True # headless had notificationpermissions setup in a distinguisable way. mock_chrome_global = False mock_canvas_fp = True # patch fingerprint EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST. ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION. WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR ME IT WORKS) * 2.2.2 * fixed a number of bugs - specifying custom profile - specifying custom binary path - downloading, patching and storing now (if not explicity specified) happens in a writable folder, instead of the current working dir. Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github> * tidy up * uncomment block * - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected. * Squashed commit of the following: commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:22:39 2021 +0200 readme change commit f214dcf33f26f8b35616d7b61cf6dee656596c3f Author: ultrafunkamsterdam <info@blackhat-security.nl> Date: Fri Apr 30 18:18:09 2021 +0200 - make sure options cannot be reused as it will cause double and conflicting arguments to chrome - support for specifying and reusing the user profile folder. if a user-data-dir is specified, that folder will NOT be deleted on exit. example: options.add_argument('--user-data-dir=c:\\temp') - uses a platform specific app data folder to store driver instead of the current workdir. - impoved headless mode. fixed detection by notification perms. - eliminates the "restore tabs" notification at startup - added methods find_elements_by_text and find_element_by_text - updated docs (partly) -known issues: - extensions not running. this is due to the inner workings of chromedriver. still working on this. - driver window is not always closing along with a program exit. - MacOS: startup nag notifications. might be solved by re(using) a profile directory. - known stuff: - some specific use cases, network conditions or behaviour can cause being detected.
2021-05-24 02:26:02 -06:00
try:
with self.lock:
log_entries = self.driver.get_log("performance")
for entry in log_entries:
try:
obj_serialized: str = entry.get("message")
obj = json.loads(obj_serialized)
message = obj.get("message")
method = message.get("method")
if "*" in self.handlers:
await self.loop.run_in_executor(
None, self.handlers["*"], message
)
elif method.lower() in self.handlers:
await self.loop.run_in_executor(
None, self.handlers[method.lower()], message
)
# print(type(message), message)
except Exception as e:
raise e from None
except Exception as e:
if "invalid session id" in str(e):
pass
else:
logging.debug("exception ignored :", e)