Commit Graph

66 Commits

Author SHA1 Message Date
Leon 5c0d2e4cb8
Update __init__.py 2022-03-14 00:37:12 +01:00
UltrafunkAmsterdam 7c25fff16e 3.1.5r2
changed the way how patcher works (for those using multiple sessions/processes).

    when not specifying a executable_path (the default, and recommended!), the filename
    gets randomized to <somehex>_chromedriver[.exe]. this should fix the issue for multiprocessing
    (although Chrome/driver itself has restrictions in this as well, see it using processhacker).
    As i told before, webdriver is a purely io-based operation which only sends and pulls data. multiprocessing/threading isn't going to help much. You'd better use asyncio.)

find_chrome_executable:
    added google-chrome-stable to the list, as some distro's have this name.

advanced_webelements:  bool, optional, default: False
        makes it easier to recognize elements like you know them from html/browser inspection, especially when working in an interactive environment

        default webelement repr:
        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>

        advanced webelement repr
        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

    note: when retrieving large amounts of elements ( example: find_elements_by_tag("*") ) and **print** them, it does take a little more time for all the repr's to fetch

Chrome() parameters

    driver_executable_path=None
     ( = executable_path )
    if you really need to specify your own chromedriver binary.

    (don't log issues when you are not using the default. the downloading per session happens for a reason. remember this is a detection-focussed fork)

    browser_executable_path=None
        ( = browser binary path )
    to specify your browser in case you use exotic locations instead of the more default install folders

    advanced_elements=False
        if set to True, webelements get a nicer REPR showing. this is very convenient when working
        interactively (like ipython for example).

        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

        instead of

        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>
2022-03-14 00:05:27 +01:00
UltrafunkAmsterdam b876db7e9a 3.15
changed the way how patcher works (for those using multiple sessions/processes).

    when not specifying a executable_path (the default, and recommended!), the filename
    gets randomized to <somehex>_chromedriver[.exe]. this should fix the issue for multiprocessing
    (although Chrome/driver itself has restrictions in this as well, see it using processhacker).
    As i told before, webdriver is a purely io-based operation which only sends and pulls data. multiprocessing/threading isn't going to help much. You'd better use asyncio.)

find_chrome_executable:
    added google-chrome-stable to the list, as some distro's have this name.

advanced_webelements:  bool, optional, default: False
        makes it easier to recognize elements like you know them from html/browser inspection, especially when working in an interactive environment

        default webelement repr:
        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>

        advanced webelement repr
        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

    note: when retrieving large amounts of elements ( example: find_elements_by_tag("*") ) and **print** them, it does take a little more time for all the repr's to fetch

Chrome() parameters

    driver_executable_path=None
     ( = executable_path )
    if you really need to specify your own chromedriver binary.

    (don't log issues when you are not using the default. the downloading per session happens for a reason. remember this is a detection-focussed fork)

    browser_executable_path=None
        ( = browser binary path )
    to specify your browser in case you use exotic locations instead of the more default install folders

    advanced_elements=False
        if set to True, webelements get a nicer REPR showing. this is very convenient when working
        interactively (like ipython for example).

        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

        instead of

        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>
2022-03-13 23:47:57 +01:00
UltrafunkAmsterdam a6cf33b0e2 3.15
changed the way how patcher works (for those using multiple sessions/processes).

    when not specifying a executable_path (the default, and recommended!), the filename
    gets randomized to <somehex>_chromedriver[.exe]. this should fix the issue for multiprocessing
    (although Chrome/driver itself has restrictions in this as well, see it using processhacker).
    As i told before, webdriver is a purely io-based operation which only sends and pulls data. multiprocessing/threading isn't going to help much. You'd better use asyncio.)

find_chrome_executable:
    added google-chrome-stable to the list, as some distro's have this name.

advanced_webelements:  bool, optional, default: False
        makes it easier to recognize elements like you know them from html/browser inspection, especially when working in an interactive environment

        default webelement repr:
        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>

        advanced webelement repr
        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

    note: when retrieving large amounts of elements ( example: find_elements_by_tag("*") ) and **print** them, it does take a little more time for all the repr's to fetch

Chrome() parameters

    driver_executable_path=None
     ( = executable_path )
    if you really need to specify your own chromedriver binary.

    (don't log issues when you are not using the default. the downloading per session happens for a reason. remember this is a detection-focussed fork)

    browser_executable_path=None
        ( = browser binary path )
    to specify your browser in case you use exotic locations instead of the more default install folders

    advanced_elements=False
        if set to True, webelements get a nicer REPR showing. this is very convenient when working
        interactively (like ipython for example).

        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

        instead of

        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>
2022-03-13 23:43:11 +01:00
UltrafunkAmsterdam 087fa8d732 Patcher:
changed the way how patcher works (for those using multiple sessions/processes).

    when not specifying a executable_path (the default, and recommended!), the filename
    gets randomized to <somehex>_chromedriver[.exe]. this should fix the issue for multiprocessing
    (although Chrome/driver itself has restrictions in this as well, see it using processhacker).
    As i told before, webdriver is a purely io-based operation which only sends and pulls data. multiprocessing/threading isn't going to help much. You'd better use asyncio.)

find_chrome_executable:
    added google-chrome-stable to the list, as some distro's have this name.

 advanced_webelements:  bool, optional, default: False
        makes it easier to recognize elements like you know them from html/browser inspection, especially when working in an interactive environment

        default webelement repr:
        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>

        advanced webelement repr
        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

    note: when retrieving large amounts of elements ( example: find_elements_by_tag("*") ) and **print** them, it does take a little more time for all the repr's to fetch

Chrome() parameters

    driver_executable_path=None
     ( = executable_path )
    if you really need to specify your own chromedriver binary.

    (don't log issues when you are not using the default. the downloading per session happens for a reason. remember this is a detection-focussed fork)

    browser_executable_path=None
        ( = browser binary path )
    to specify your browser in case you use exotic locations instead of the more default install folders

    advanced_elements=False
        if set to True, webelements get a nicer REPR showing. this is very convenient when working
        interactively (like ipython for example).

        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

        instead of

        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>
2022-03-13 23:22:41 +01:00
UltrafunkAmsterdam 2710213a7e Patcher:
changed the way how patcher works (for those using multiple sessions/processes).

    when not specifying a executable_path (the default, and recommended!), the filename
    gets randomized to <somehex>_chromedriver[.exe]. this should fix the issue for multiprocessing
    (although Chrome/driver itself has restrictions in this as well, see it using processhacker).
    As i told before, webdriver is a purely io-based operation which only sends and pulls data. multiprocessing/threading isn't going to help much. You'd better use asyncio.)

find_chrome_executable:
    added google-chrome-stable to the list, as some distro's have this name.

 advanced_webelements:  bool, optional, default: False
        makes it easier to recognize elements like you know them from html/browser inspection, especially when working in an interactive environment

        default webelement repr:
        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>

        advanced webelement repr
        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

    note: when retrieving large amounts of elements ( example: find_elements_by_tag("*") ) and **print** them, it does take a little more time for all the repr's to fetch

Chrome() parameters

    driver_executable_path=None
     ( = executable_path )
    if you really need to specify your own chromedriver binary.

    (don't log issues when you are not using the default. the downloading per session happens for a reason. remember this is a detection-focussed fork)

    browser_executable_path=None
        ( = browser binary path )
    to specify your browser in case you use exotic locations instead of the more default install folders

    advanced_elements=False
        if set to True, webelements get a nicer REPR showing. this is very convenient when working
        interactively (like ipython for example).

        <WebElement(<a class="mobile-show-inline-block mc-update-infos init-ok" href="#" id="main-cat-switcher-mobile">)>

        instead of

        <selenium.webdriver.remote.webelement.WebElement (session="85ff0f671512fa535630e71ee951b1f2", element="6357cb55-92c3-4c0f-9416-b174f9c1b8c4")>
2022-03-13 23:05:22 +01:00
Leon 1245e160c5
Update __init__.py 2021-12-24 14:51:15 +00:00
Leon 3bf4cdf7a9 3.1.2 - some 'bug' fixes 2021-12-24 15:39:16 +01:00
admin f9e9e77218 bug fix 2021-12-23 18:23:25 +01:00
Leon f7306fad3e
bugfix spawn 2021-12-23 17:10:04 +01:00
UltrafunkAmsterdam 154f7fcdb3 3.1.0! 2021-12-22 14:07:27 +00:00
Leon abac314741
removed executable_path in favor of browser_executable_path
This makes it easier , when needed, in edge cases , to specify your browser executable.
2021-12-21 16:42:09 +00:00
Leon 8a3870bd6d
removed "delay" from constructor, added user_data_dir
simplify specifying a custom user_data_dir by passing it directly to the constructor.
however if a user_data_dir is specified in the options object,  the one in options will take precedence.
2021-12-21 16:31:04 +00:00
UltrafunkAmsterdam b60820a600 3.1.0rc1
-----------
  this version is for test purposes only and contains breaking changes
  -----------

  - v2 is now the "main/default" module. usage:

        import undetected_chromedriver as uc
        driver = uc.Chrome()
        driver.get('https://nowsecure.nl')

  - The above is the README for this version. or use the regular instructions, but
    skip the `with` black magic and skip references to v2.
  - v1 moved to _compat for now.
  - fixed wrong dependancies
  - ~~~~ added "new" anti-detection mechanic ~~~~

  - the above ^^ makes all recent changes and additions obsolete
  - Removed ChromeOptions black magic to fix compatiblity issues

  - restored .get() to (near) original.
       - most changes from 3.0.4 to 3.0.6 are obsolete, as t
       - no `with` statements needed anymore, although it will still
         work for the sake of backward-compatibility.
       - no sleeps, stop-start-sessions, delays, or async cdp black magic!
       - this will solve a lot of other "issues" as well.
  - test success to date: 100%
  - just to mention it another time, since some people have hard time reading:
    headless is still WIP. Raising issues is needless
2021-12-16 05:53:41 +01:00
Leon c1d02484d9
Update __init__.py 2021-11-29 14:40:10 +01:00
UltrafunkAmsterdam 77a3c3020f ^ 2021-11-14 13:06:34 +01:00
UltrafunkAmsterdam 97288266bc 3.0.3 - fixed a bug where driver hangs long time on quit - and now passing executable_path explicitly now causes chromedriver to not redownload since some people have issues downloading 3mb but expect to build next-gen scrapers 2021-07-30 00:23:13 +02:00
UltrafunkAmsterdam 7ddbfd00d0 added v2 as namespace in __init__, and for v1 added ChromeoptionsV2, which is just v2.ChromeOptions 2021-06-27 13:59:20 +02:00
UltrafunkAmsterdam fd9b6d5fd7 create namespace v2 in __init__.py 2021-06-27 13:41:21 +02:00
Leon e598e1ca1b
Quickfix 3.0.1
3.0.1 bugfix (bigfux)
2021-06-02 13:46:23 +02:00
Leon ebafbe1db6
3.0.0 (#180)
*3.0.0
added lots of features and bugfixes

- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
    ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
    people having troubles with mobile emulation and other bullshit,
    can try again now.
- fixed the logic where sometimes options did not
    respect the given values
- for headless (though still not supperted for undetectability),
    added some real cool features which need to be set in
    the options object):

    defaults:
    emulate_touch = True
    mock_permissions = True  # headless had notificationpermissions
                                setup in a distinguisable way.
    mock_chrome_global = False
    mock_canvas_fp = True  # patch fingerprint

EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.

WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)


* 2.2.2

* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
    happens in a writable folder, instead of the current working dir.

Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>

* tidy up

* uncomment block

* - support for specifying and reusing the user profile folder.
    if a user-data-dir is specified, that folder will NOT be
    deleted on exit.
    example:
        options.add_argument('--user-data-dir=c:\\temp')

- uses a platform specific app data folder to store driver instead
    of the current workdir.

- impoved headless mode. fixed detection by notification perms.

- eliminates the "restore tabs" notification at startup

- added methods find_elements_by_text and find_element_by_text

- updated docs (partly)

-known issues:
    - extensions not running. this is due to the inner workings
        of chromedriver. still working on this.
    - driver window is not always closing along with a program exit.
    - MacOS: startup nag notifications. might be solved by
        re(using) a profile directory.

- known stuff:
    - some specific use cases, network conditions or behaviour
      can cause being detected.

* Squashed commit of the following:

commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:22:39 2021 +0200

    readme change

commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:18:09 2021 +0200

    - make sure options cannot be reused as it will
        cause double and conflicting arguments to chrome



    - support for specifying and reusing the user profile folder.
        if a user-data-dir is specified, that folder will NOT be
        deleted on exit.
        example:
            options.add_argument('--user-data-dir=c:\\temp')

    - uses a platform specific app data folder to store driver instead
        of the current workdir.

    - impoved headless mode. fixed detection by notification perms.

    - eliminates the "restore tabs" notification at startup

    - added methods find_elements_by_text and find_element_by_text

    - updated docs (partly)

    -known issues:
        - extensions not running. this is due to the inner workings
            of chromedriver. still working on this.
        - driver window is not always closing along with a program exit.
        - MacOS: startup nag notifications. might be solved by
            re(using) a profile directory.

    - known stuff:
        - some specific use cases, network conditions or behaviour
          can cause being detected.
2021-05-24 10:26:02 +02:00
Leon bc30d7623f
2.2.6 - hodl your breath (#161)
* 2.2.2

* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
    happens in a writable folder, instead of the current working dir.

Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>

* tidy up

* uncomment block

* - support for specifying and reusing the user profile folder.
    if a user-data-dir is specified, that folder will NOT be
    deleted on exit.
    example:
        options.add_argument('--user-data-dir=c:\\temp')

- uses a platform specific app data folder to store driver instead
    of the current workdir.

- impoved headless mode. fixed detection by notification perms.

- eliminates the "restore tabs" notification at startup

- added methods find_elements_by_text and find_element_by_text

- updated docs (partly)

-known issues:
    - extensions not running. this is due to the inner workings
        of chromedriver. still working on this.
    - driver window is not always closing along with a program exit.
    - MacOS: startup nag notifications. might be solved by
        re(using) a profile directory.

- known stuff:
    - some specific use cases, network conditions or behaviour
      can cause being detected.

* Squashed commit of the following:

commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:22:39 2021 +0200

    readme change

commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:18:09 2021 +0200

    - make sure options cannot be reused as it will
        cause double and conflicting arguments to chrome

commit cf059a638c
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Thu Apr 29 12:54:49 2021 +0200

    - support for specifying and reusing the user profile folder.
        if a user-data-dir is specified, that folder will NOT be
        deleted on exit.
        example:
            options.add_argument('--user-data-dir=c:\\temp')

    - uses a platform specific app data folder to store driver instead
        of the current workdir.

    - impoved headless mode. fixed detection by notification perms.

    - eliminates the "restore tabs" notification at startup

    - added methods find_elements_by_text and find_element_by_text

    - updated docs (partly)

    -known issues:
        - extensions not running. this is due to the inner workings
            of chromedriver. still working on this.
        - driver window is not always closing along with a program exit.
        - MacOS: startup nag notifications. might be solved by
            re(using) a profile directory.

    - known stuff:
        - some specific use cases, network conditions or behaviour
          can cause being detected.

commit b40d23c649
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Tue Apr 27 20:41:18 2021 +0200

    uncomment block

commit d99809c8c6
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Tue Apr 27 20:19:51 2021 +0200

    tidy up

* .

* 2.2.7

Co-authored-by: ultrafunkamsterdam <info@blackhat-security.nl>
2021-05-01 22:49:59 +02:00
unknown 2e2047dc91 2.1.0 2021-02-04 12:15:22 +01:00
UltrafunkAmsterdam a50362f9e8 - added a keyword argument to the Chrome constructor: `emulate_touch` which, when set to True will report presense of a touch(screen/device). This is mainly for bet365 detections. Otherwise i would not recommend setting it.
credits to @boganfoo for this excellent find!

- removed `enable_console_log`
- reverted back to storing chromedriver binary in current workdir after several reports of users entire project folder being deleted (sorry for that btw).
- some code cleanup
- added a fix for useragent in headless mode in v2 (which reported 'Headless'). still headless mode in v2 is under construction and not fully functional.
2021-02-02 08:58:00 +01:00
unknown b792eaf242 - removed enable_console_log
- reverted back to storing chromedriver binary in current workdir after several reports of users entire project folder being deleted (sorry for that btw).

- some code cleanup

- added a fix for useragent in headless mode in v2 (which reported 'Headless'). still headless mode in v2 is under construction and not fully functional.

- added a keyword argument to the Chrome constructor: 'emulate_touch' which, when set to True will report presense of a touch(screen/device). This is mainly for bet365 detections. Otherwise i would not recommend setting it.

 credits to @boganfoo for this excellent find!
2021-02-02 08:37:14 +01:00
Leon 6708019040
randomize cdc_string and silicenced output
changed the warning log messsage to level INFO for @Avnxs
randomize cdc string   @czoins
2020-12-19 15:49:34 +01:00
Leon d8ccac2ac5
fixed cloudflare detection 2020-11-25 12:43:24 +01:00
ultrafunkamsterdam 4ce47e7f83 1.5.0 2020-10-13 03:51:35 +02:00
ultrafunkamsterdam 8baa61990f working patch 2020-09-19 18:22:03 +02:00
ultrafunkamsterdam 17e1f0e584 Revert "working patch"
This reverts commit c0ee188636.
2020-09-19 18:15:41 +02:00
Goolen c0ee188636 working patch 2020-09-19 18:01:11 +02:00
Leon 4c5ecc8f67
Update __init__.py 2020-09-13 11:36:47 +02:00
ultrafunkamsterdam b7685af668 added to pip package index 2020-09-03 22:12:09 +02:00
Leon 6e0f49797d
Update __init__.py 2020-09-02 04:45:12 +02:00
Leon 8810c70e08
set default target to version 84 2020-08-01 15:08:18 +02:00
Leon dd89df1666
removed the "platform" key of Network.setUserAgentOverride
It did nothing but causing confusion :)
2020-06-15 18:46:07 +02:00
Leon fee707a054
Update __init__.py 2020-06-03 13:44:22 +02:00
UltrafunkAmsterdam 2d2401caa5 fixed the TARGET_VERSION variable. is now actually evaluated at execution time so the correct chromedriver for your version of chrome is being downloaded... + some refactorings 2020-05-19 20:55:13 +02:00
Leon 9cd00ce07f
fix small bug in TARGET_VERSION 2020-05-13 22:18:20 +02:00
Leon 3293fb347f
Update __init__.py 2020-05-12 19:56:49 +02:00
Leon f3fdf9b78e
Update __init__.py 2020-04-24 14:02:15 +02:00
Leon be40873506
Update __init__.py 2020-03-06 21:20:31 +01:00
Leon 4d253ef037
Update __init__.py 2020-03-03 15:30:48 +01:00
Leon e5c00010c4
Update __init__.py 2020-03-03 15:22:38 +01:00
Leon 0cf15a2531
Update __init__.py 2020-03-03 15:20:42 +01:00
Leon a44e52751b
Update __init__.py 2020-03-03 15:17:50 +01:00
Leon 71ea65eb67
fix compatibility for linux/mac 2020-03-03 15:03:49 +01:00
Leon 7526b41e6f
Update __init__.py 2020-03-03 14:57:09 +01:00
Leon ecc92cd109
fix compatibility linux/mac 2020-03-03 14:53:55 +01:00
Leon cbe27643d0
compatibility linux and mac 2020-03-03 14:51:10 +01:00