3.0.0 (#180)
*3.0.0
added lots of features and bugfixes
- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
people having troubles with mobile emulation and other bullshit,
can try again now.
- fixed the logic where sometimes options did not
respect the given values
- for headless (though still not supperted for undetectability),
added some real cool features which need to be set in
the options object):
defaults:
emulate_touch = True
mock_permissions = True # headless had notificationpermissions
setup in a distinguisable way.
mock_chrome_global = False
mock_canvas_fp = True # patch fingerprint
EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.
WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)
* 2.2.2
* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
happens in a writable folder, instead of the current working dir.
Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>
* tidy up
* uncomment block
* - support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
* Squashed commit of the following:
commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:22:39 2021 +0200
readme change
commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:18:09 2021 +0200
- make sure options cannot be reused as it will
cause double and conflicting arguments to chrome
- support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
2021-05-24 02:26:02 -06:00
|
|
|
#!/usr/bin/env python3
|
|
|
|
# this module is part of undetected_chromedriver
|
|
|
|
|
|
|
|
import asyncio
|
|
|
|
import json
|
|
|
|
import logging
|
|
|
|
import threading
|
|
|
|
|
2023-02-04 14:02:46 -07:00
|
|
|
|
3.0.0 (#180)
*3.0.0
added lots of features and bugfixes
- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
people having troubles with mobile emulation and other bullshit,
can try again now.
- fixed the logic where sometimes options did not
respect the given values
- for headless (though still not supperted for undetectability),
added some real cool features which need to be set in
the options object):
defaults:
emulate_touch = True
mock_permissions = True # headless had notificationpermissions
setup in a distinguisable way.
mock_chrome_global = False
mock_canvas_fp = True # patch fingerprint
EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.
WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)
* 2.2.2
* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
happens in a writable folder, instead of the current working dir.
Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>
* tidy up
* uncomment block
* - support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
* Squashed commit of the following:
commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:22:39 2021 +0200
readme change
commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:18:09 2021 +0200
- make sure options cannot be reused as it will
cause double and conflicting arguments to chrome
- support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
2021-05-24 02:26:02 -06:00
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
|
|
|
|
|
|
class Reactor(threading.Thread):
|
|
|
|
def __init__(self, driver: "Chrome"):
|
|
|
|
super().__init__()
|
|
|
|
|
|
|
|
self.driver = driver
|
|
|
|
self.loop = asyncio.new_event_loop()
|
|
|
|
|
|
|
|
self.lock = threading.Lock()
|
|
|
|
self.event = threading.Event()
|
|
|
|
self.daemon = True
|
|
|
|
self.handlers = {}
|
|
|
|
|
|
|
|
def add_event_handler(self, method_name, callback: callable):
|
|
|
|
"""
|
|
|
|
|
|
|
|
Parameters
|
|
|
|
----------
|
|
|
|
event_name: str
|
|
|
|
example "Network.responseReceived"
|
|
|
|
|
|
|
|
callback: callable
|
|
|
|
callable which accepts 1 parameter: the message object dictionary
|
|
|
|
|
|
|
|
Returns
|
|
|
|
-------
|
|
|
|
|
|
|
|
"""
|
|
|
|
with self.lock:
|
|
|
|
self.handlers[method_name.lower()] = callback
|
|
|
|
|
|
|
|
@property
|
|
|
|
def running(self):
|
|
|
|
return not self.event.is_set()
|
|
|
|
|
|
|
|
def run(self):
|
|
|
|
try:
|
|
|
|
asyncio.set_event_loop(self.loop)
|
|
|
|
self.loop.run_until_complete(self.listen())
|
|
|
|
except Exception as e:
|
|
|
|
logger.warning("Reactor.run() => %s", e)
|
|
|
|
|
2021-12-15 21:53:41 -07:00
|
|
|
async def _wait_service_started(self):
|
|
|
|
while True:
|
|
|
|
with self.lock:
|
|
|
|
if (
|
2021-12-22 07:07:27 -07:00
|
|
|
getattr(self.driver, "service", None)
|
|
|
|
and getattr(self.driver.service, "process", None)
|
|
|
|
and self.driver.service.process.poll()
|
2021-12-15 21:53:41 -07:00
|
|
|
):
|
|
|
|
await asyncio.sleep(self.driver._delay or 0.25)
|
|
|
|
else:
|
|
|
|
break
|
|
|
|
|
3.0.0 (#180)
*3.0.0
added lots of features and bugfixes
- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
people having troubles with mobile emulation and other bullshit,
can try again now.
- fixed the logic where sometimes options did not
respect the given values
- for headless (though still not supperted for undetectability),
added some real cool features which need to be set in
the options object):
defaults:
emulate_touch = True
mock_permissions = True # headless had notificationpermissions
setup in a distinguisable way.
mock_chrome_global = False
mock_canvas_fp = True # patch fingerprint
EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.
WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)
* 2.2.2
* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
happens in a writable folder, instead of the current working dir.
Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>
* tidy up
* uncomment block
* - support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
* Squashed commit of the following:
commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:22:39 2021 +0200
readme change
commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:18:09 2021 +0200
- make sure options cannot be reused as it will
cause double and conflicting arguments to chrome
- support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
2021-05-24 02:26:02 -06:00
|
|
|
async def listen(self):
|
|
|
|
while self.running:
|
2021-12-15 21:53:41 -07:00
|
|
|
await self._wait_service_started()
|
|
|
|
await asyncio.sleep(1)
|
3.0.0 (#180)
*3.0.0
added lots of features and bugfixes
- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
people having troubles with mobile emulation and other bullshit,
can try again now.
- fixed the logic where sometimes options did not
respect the given values
- for headless (though still not supperted for undetectability),
added some real cool features which need to be set in
the options object):
defaults:
emulate_touch = True
mock_permissions = True # headless had notificationpermissions
setup in a distinguisable way.
mock_chrome_global = False
mock_canvas_fp = True # patch fingerprint
EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.
WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)
* 2.2.2
* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
happens in a writable folder, instead of the current working dir.
Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>
* tidy up
* uncomment block
* - support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
* Squashed commit of the following:
commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:22:39 2021 +0200
readme change
commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date: Fri Apr 30 18:18:09 2021 +0200
- make sure options cannot be reused as it will
cause double and conflicting arguments to chrome
- support for specifying and reusing the user profile folder.
if a user-data-dir is specified, that folder will NOT be
deleted on exit.
example:
options.add_argument('--user-data-dir=c:\\temp')
- uses a platform specific app data folder to store driver instead
of the current workdir.
- impoved headless mode. fixed detection by notification perms.
- eliminates the "restore tabs" notification at startup
- added methods find_elements_by_text and find_element_by_text
- updated docs (partly)
-known issues:
- extensions not running. this is due to the inner workings
of chromedriver. still working on this.
- driver window is not always closing along with a program exit.
- MacOS: startup nag notifications. might be solved by
re(using) a profile directory.
- known stuff:
- some specific use cases, network conditions or behaviour
can cause being detected.
2021-05-24 02:26:02 -06:00
|
|
|
|
|
|
|
try:
|
|
|
|
with self.lock:
|
|
|
|
log_entries = self.driver.get_log("performance")
|
|
|
|
|
|
|
|
for entry in log_entries:
|
|
|
|
try:
|
|
|
|
obj_serialized: str = entry.get("message")
|
|
|
|
obj = json.loads(obj_serialized)
|
|
|
|
message = obj.get("message")
|
|
|
|
method = message.get("method")
|
|
|
|
|
|
|
|
if "*" in self.handlers:
|
|
|
|
await self.loop.run_in_executor(
|
|
|
|
None, self.handlers["*"], message
|
|
|
|
)
|
|
|
|
elif method.lower() in self.handlers:
|
|
|
|
await self.loop.run_in_executor(
|
|
|
|
None, self.handlers[method.lower()], message
|
|
|
|
)
|
|
|
|
|
|
|
|
# print(type(message), message)
|
|
|
|
except Exception as e:
|
|
|
|
raise e from None
|
|
|
|
|
|
|
|
except Exception as e:
|
|
|
|
if "invalid session id" in str(e):
|
|
|
|
pass
|
|
|
|
else:
|
|
|
|
logging.debug("exception ignored :", e)
|