Commit Graph

70 Commits

Author SHA1 Message Date
Leon c1d02484d9
Update __init__.py 2021-11-29 14:40:10 +01:00
Leon a84c53f3d5
Update cdp.py 2021-11-29 14:39:21 +01:00
Leon 9e41928375
Update dprocess.py 2021-11-17 09:38:01 +01:00
UltrafunkAmsterdam ec49c0086b 3.0.4 2021-11-16 18:43:52 +01:00
UltrafunkAmsterdam 77a3c3020f ^ 2021-11-14 13:06:34 +01:00
UltrafunkAmsterdam 9f9bd66d79 ^ 2021-11-14 12:32:22 +01:00
UltrafunkAmsterdam 97288266bc 3.0.3 - fixed a bug where driver hangs long time on quit - and now passing executable_path explicitly now causes chromedriver to not redownload since some people have issues downloading 3mb but expect to build next-gen scrapers 2021-07-30 00:23:13 +02:00
UltrafunkAmsterdam 7ddbfd00d0 added v2 as namespace in __init__, and for v1 added ChromeoptionsV2, which is just v2.ChromeOptions 2021-06-27 13:59:20 +02:00
UltrafunkAmsterdam fd9b6d5fd7 create namespace v2 in __init__.py 2021-06-27 13:41:21 +02:00
Leon e598e1ca1b
Quickfix 3.0.1
3.0.1 bugfix (bigfux)
2021-06-02 13:46:23 +02:00
Leon 5a24c22598
create_new_process_group flag not present on linux.
could be removed anyway.
2021-05-25 11:42:27 +02:00
Leon ebafbe1db6
3.0.0 (#180)
*3.0.0
added lots of features and bugfixes

- You can now subscribe to Chrome Devtools Protocol Events like networking.
- splitted the project up in seperate modules now
- fixed locale (accept-language)
- you can enter your user-data-folder as property of
    ChromeOptions() now.
- The ChromeOptions had a makeover, and i took the one from alpha 4,
    people having troubles with mobile emulation and other bullshit,
    can try again now.
- fixed the logic where sometimes options did not
    respect the given values
- for headless (though still not supperted for undetectability),
    added some real cool features which need to be set in
    the options object):

    defaults:
    emulate_touch = True
    mock_permissions = True  # headless had notificationpermissions
                                setup in a distinguisable way.
    mock_chrome_global = False
    mock_canvas_fp = True  # patch fingerprint

EXTENSIONS ARE NOT SUPPORTED BY CHROME IN HEADLESS MODE
YET. IF YOU WANT TO USE THEM, CREATE A PROFILE AND INSTALL
EXTENSIONS BY USING A REGULAR CHROME SESSION FIRST.
ALSO LOGIN TO GMAIL WHILE YOU'RE ON A GENUINE SESSION.

WHEN FINISHED, COPY THE USERDATA FOLDER OF CHROME TO SOME KNOWN
LOCATION (and make maye 2 copies?). BY HAVING GMAIL LOGGED IN
FIXES ALSO THE UNSAFE BROWSER MESSAGE FROM GOOGLE (AT LEAST FOR
ME IT WORKS)


* 2.2.2

* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
    happens in a writable folder, instead of the current working dir.

Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>

* tidy up

* uncomment block

* - support for specifying and reusing the user profile folder.
    if a user-data-dir is specified, that folder will NOT be
    deleted on exit.
    example:
        options.add_argument('--user-data-dir=c:\\temp')

- uses a platform specific app data folder to store driver instead
    of the current workdir.

- impoved headless mode. fixed detection by notification perms.

- eliminates the "restore tabs" notification at startup

- added methods find_elements_by_text and find_element_by_text

- updated docs (partly)

-known issues:
    - extensions not running. this is due to the inner workings
        of chromedriver. still working on this.
    - driver window is not always closing along with a program exit.
    - MacOS: startup nag notifications. might be solved by
        re(using) a profile directory.

- known stuff:
    - some specific use cases, network conditions or behaviour
      can cause being detected.

* Squashed commit of the following:

commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:22:39 2021 +0200

    readme change

commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:18:09 2021 +0200

    - make sure options cannot be reused as it will
        cause double and conflicting arguments to chrome



    - support for specifying and reusing the user profile folder.
        if a user-data-dir is specified, that folder will NOT be
        deleted on exit.
        example:
            options.add_argument('--user-data-dir=c:\\temp')

    - uses a platform specific app data folder to store driver instead
        of the current workdir.

    - impoved headless mode. fixed detection by notification perms.

    - eliminates the "restore tabs" notification at startup

    - added methods find_elements_by_text and find_element_by_text

    - updated docs (partly)

    -known issues:
        - extensions not running. this is due to the inner workings
            of chromedriver. still working on this.
        - driver window is not always closing along with a program exit.
        - MacOS: startup nag notifications. might be solved by
            re(using) a profile directory.

    - known stuff:
        - some specific use cases, network conditions or behaviour
          can cause being detected.
2021-05-24 10:26:02 +02:00
Leon bc30d7623f
2.2.6 - hodl your breath (#161)
* 2.2.2

* fixed a number of bugs
- specifying custom profile
- specifying custom binary path
- downloading, patching and storing now (if not explicity specified)
    happens in a writable folder, instead of the current working dir.

Committer: UltrafunkAmsterdam <UltrafunkAmsterdam@github>

* tidy up

* uncomment block

* - support for specifying and reusing the user profile folder.
    if a user-data-dir is specified, that folder will NOT be
    deleted on exit.
    example:
        options.add_argument('--user-data-dir=c:\\temp')

- uses a platform specific app data folder to store driver instead
    of the current workdir.

- impoved headless mode. fixed detection by notification perms.

- eliminates the "restore tabs" notification at startup

- added methods find_elements_by_text and find_element_by_text

- updated docs (partly)

-known issues:
    - extensions not running. this is due to the inner workings
        of chromedriver. still working on this.
    - driver window is not always closing along with a program exit.
    - MacOS: startup nag notifications. might be solved by
        re(using) a profile directory.

- known stuff:
    - some specific use cases, network conditions or behaviour
      can cause being detected.

* Squashed commit of the following:

commit 7ce8e7a236cbee770cb117145d4bf6dc245b936a
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:22:39 2021 +0200

    readme change

commit f214dcf33f26f8b35616d7b61cf6dee656596c3f
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Fri Apr 30 18:18:09 2021 +0200

    - make sure options cannot be reused as it will
        cause double and conflicting arguments to chrome

commit cf059a638c
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Thu Apr 29 12:54:49 2021 +0200

    - support for specifying and reusing the user profile folder.
        if a user-data-dir is specified, that folder will NOT be
        deleted on exit.
        example:
            options.add_argument('--user-data-dir=c:\\temp')

    - uses a platform specific app data folder to store driver instead
        of the current workdir.

    - impoved headless mode. fixed detection by notification perms.

    - eliminates the "restore tabs" notification at startup

    - added methods find_elements_by_text and find_element_by_text

    - updated docs (partly)

    -known issues:
        - extensions not running. this is due to the inner workings
            of chromedriver. still working on this.
        - driver window is not always closing along with a program exit.
        - MacOS: startup nag notifications. might be solved by
            re(using) a profile directory.

    - known stuff:
        - some specific use cases, network conditions or behaviour
          can cause being detected.

commit b40d23c649
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Tue Apr 27 20:41:18 2021 +0200

    uncomment block

commit d99809c8c6
Author: ultrafunkamsterdam <info@blackhat-security.nl>
Date:   Tue Apr 27 20:19:51 2021 +0200

    tidy up

* .

* 2.2.7

Co-authored-by: ultrafunkamsterdam <info@blackhat-security.nl>
2021-05-01 22:49:59 +02:00
UltrafunkAmsterdam b3484a8b7e 2.2.1 Fixed issue where arguments were not passed along to chrome. Some small improvements for v2 headless, which is still detectable, but it's getting better...., 2021-03-25 10:23:32 +01:00
UltrafunkAmsterdam 853f299967 remove the subclass and use delegation to access Webdrivers methods. This way an exception somewhere along your program will still ensure gracefull shutdown afterwards and fixes some other issues as well because Webdriver is now bound to the object instead of only initialized within the __init__ haeving no references at all. 2021-03-20 05:28:42 +01:00
unknown b3014260ce fix invalid session id after using get_in or context manager style 2021-02-04 12:55:44 +01:00
unknown 2e2047dc91 2.1.0 2021-02-04 12:15:22 +01:00
Leon 0fa88770a2
fix closing browser window when using as context manager 2021-02-04 11:57:45 +01:00
Leon db2214071d
Update v2.py 2021-02-03 09:29:41 +01:00
Leon 38b3eb2ec8
Update v2.py 2021-02-02 09:30:42 +01:00
UltrafunkAmsterdam a50362f9e8 - added a keyword argument to the Chrome constructor: `emulate_touch` which, when set to True will report presense of a touch(screen/device). This is mainly for bet365 detections. Otherwise i would not recommend setting it.
credits to @boganfoo for this excellent find!

- removed `enable_console_log`
- reverted back to storing chromedriver binary in current workdir after several reports of users entire project folder being deleted (sorry for that btw).
- some code cleanup
- added a fix for useragent in headless mode in v2 (which reported 'Headless'). still headless mode in v2 is under construction and not fully functional.
2021-02-02 08:58:00 +01:00
unknown b792eaf242 - removed enable_console_log
- reverted back to storing chromedriver binary in current workdir after several reports of users entire project folder being deleted (sorry for that btw).

- some code cleanup

- added a fix for useragent in headless mode in v2 (which reported 'Headless'). still headless mode in v2 is under construction and not fully functional.

- added a keyword argument to the Chrome constructor: 'emulate_touch' which, when set to True will report presense of a touch(screen/device). This is mainly for bet365 detections. Otherwise i would not recommend setting it.

 credits to @boganfoo for this excellent find!
2021-02-02 08:37:14 +01:00
Leon 0263617a83
disable auto-cleanup if custom user-data-dir is used
- prevent custom data dir to be automatically removed.
- remove the driver executable cleanup function, it is deleted at initialization anyway so it was double trouble.
- reverted back to having chromedriver in the CWD instead of a temp folder

@untested 
todo: test
2021-01-29 03:30:01 +01:00
Leon 2ec225069f
Update v2.py 2021-01-17 21:05:40 +01:00
Leon 3ef151e048
Update v2.py 2021-01-17 20:48:36 +01:00
unknown c8f28b1a70 corrected find_chrome_executable to work in MacOS 2021-01-17 20:31:22 +01:00
Leon 645a686cfe
Update v2.py 2021-01-17 06:43:03 +01:00
Leon 4fa212f0c5
Update v2.py 2021-01-17 06:31:22 +01:00
unknown e6d3299cbf v2 option 2021-01-17 06:11:11 +01:00
Leon 6708019040
randomize cdc_string and silicenced output
changed the warning log messsage to level INFO for @Avnxs
randomize cdc string   @czoins
2020-12-19 15:49:34 +01:00
Leon d8ccac2ac5
fixed cloudflare detection 2020-11-25 12:43:24 +01:00
ultrafunkamsterdam 4ce47e7f83 1.5.0 2020-10-13 03:51:35 +02:00
ultrafunkamsterdam 8baa61990f working patch 2020-09-19 18:22:03 +02:00
ultrafunkamsterdam 17e1f0e584 Revert "working patch"
This reverts commit c0ee188636.
2020-09-19 18:15:41 +02:00
Goolen c0ee188636 working patch 2020-09-19 18:01:11 +02:00
Leon 4c5ecc8f67
Update __init__.py 2020-09-13 11:36:47 +02:00
ultrafunkamsterdam b7685af668 added to pip package index 2020-09-03 22:12:09 +02:00
Leon 6e0f49797d
Update __init__.py 2020-09-02 04:45:12 +02:00
Leon 8810c70e08
set default target to version 84 2020-08-01 15:08:18 +02:00
Leon dd89df1666
removed the "platform" key of Network.setUserAgentOverride
It did nothing but causing confusion :)
2020-06-15 18:46:07 +02:00
Leon fee707a054
Update __init__.py 2020-06-03 13:44:22 +02:00
UltrafunkAmsterdam 2d2401caa5 fixed the TARGET_VERSION variable. is now actually evaluated at execution time so the correct chromedriver for your version of chrome is being downloaded... + some refactorings 2020-05-19 20:55:13 +02:00
Leon 9cd00ce07f
fix small bug in TARGET_VERSION 2020-05-13 22:18:20 +02:00
Leon 3293fb347f
Update __init__.py 2020-05-12 19:56:49 +02:00
Leon f3fdf9b78e
Update __init__.py 2020-04-24 14:02:15 +02:00
Leon be40873506
Update __init__.py 2020-03-06 21:20:31 +01:00
Leon 4d253ef037
Update __init__.py 2020-03-03 15:30:48 +01:00
Leon e5c00010c4
Update __init__.py 2020-03-03 15:22:38 +01:00
Leon 0cf15a2531
Update __init__.py 2020-03-03 15:20:42 +01:00
Leon a44e52751b
Update __init__.py 2020-03-03 15:17:50 +01:00