Commit Graph

4445 Commits

Author SHA1 Message Date
Raymond Hill 3ff456a63d
Prefer CDNs when subscribing to a new list
Also, purging the cache of a list should not remove the
compiled version of that list so that it keep being seen
as cached, but obsolete.
2023-11-07 08:21:22 -05:00
Raymond Hill ae24032607
Add more `replace`-able types
Reference:
https://en.wikipedia.org/wiki/M3U#Internet_media_types
2023-11-07 07:53:37 -05:00
Raymond Hill 2360bc02f3
Lower maximum `Expires` value to 4h
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2899
2023-11-06 19:45:16 -05:00
Raymond Hill 8ed1ad9c9d
Properly reset needle length in unserialized buffer
Possible related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2925
2023-11-06 19:22:47 -05:00
Imre Kristoffer Eilertsen a2ab313ae2
Attempted to set flag codes for Chromium Windows users (#3906)
* Attempted to set flag codes for Chromium users

* Not sure how I missed MD.
2023-11-06 16:46:56 -05:00
Raymond Hill 11ebb736ca
Import translation work from https://crowdin.com/project/ublock 2023-11-06 12:16:48 -05:00
Raymond Hill 5d1618e1be
Reset potential `g` regex before use in `replace=` option
Related discussion:
https://github.com/uBlockOrigin/uAssets/issues/5184#issuecomment-1794058253
2023-11-06 07:16:41 -05:00
Raymond Hill 6417f54299
Apply response filtering according to mime type
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2833
2023-11-05 19:13:38 -05:00
Raymond Hill 1055203542
Avoid creating a session if there are no jobs 2023-11-05 14:54:59 -05:00
Raymond Hill ce835bb300
Remove extraneous increment in charset sniffing code
Related commit:
7c3e060c01

Related feedback:
7c3e060c01 (commitcomment-131759551)
2023-11-05 14:19:06 -05:00
Raymond Hill b69b0b5fb9
Add more mimes as valid target for response filtering
Related feedback:
7c3e060c01 (commitcomment-131754750)

application/vnd.apple.mpegurl
Reference:
https://www.iana.org/assignments/media-types/application/vnd.apple.mpegurl

application/vnd.api+json
Reference:
https://www.iana.org/assignments/media-types/application/vnd.api+json
2023-11-05 12:50:03 -05:00
Raymond Hill d7c99b46e6
Have `urltransform=` use the same syntax as `replace=`
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2924
2023-11-05 11:19:03 -05:00
Raymond Hill 5f78d83fea
Fix various code paths in bodyFilterer
Related to extracting charset of response body.
2023-11-05 10:32:23 -05:00
Raymond Hill 1cf634cce0
Leave out hash part when extracting patch file name
Related commit:
aeff955667
2023-11-05 08:03:43 -05:00
Raymond Hill c38267bb2b
Fix `charset` sniffing in reworked response body filtering code
Related feedback:
https://github.com/uBlockOrigin/uBlock-discussions/discussions/830

Related commit:
https://github.com/uBlockOrigin/uBlock-issues/discussions/2537#discussioncomment-5277574
2023-11-04 09:50:10 -04:00
Raymond Hill 4de6bd6f07
Fix potential exceptions in new response body filtering code
Related feedback:
https://github.com/uBlockOrigin/uBlock-issues/discussions/2929

Related commit:
7c3e060c01
2023-11-04 09:43:38 -04:00
Raymond Hill 4774a39b17
[mv3] Fix converter to DNR rules 2023-11-03 21:32:44 -04:00
Raymond Hill 4c48148eb2
Support auto-complete for `from=` same as `domain=` 2023-11-03 20:19:19 -04:00
Raymond Hill d12d014b10
Import translation work from https://crowdin.com/project/ublock 2023-11-03 19:13:07 -04:00
Raymond Hill 7c3e060c01
Implement network filter option `replace=`
Reference documentation:
https://adguard.com/kb/general/ad-filtering/create-own-filters/#replace-modifier

This is a network filter option which can only be loaded from a
trusted source.

Since this filter is about modifying the response body, it currently
only works in Firefox.

As discussed with filter list maintainers.
2023-11-03 19:01:01 -04:00
Raymond Hill aeff955667
Extract list name from hash portion of `! Diff-Path:` 2023-11-03 18:39:14 -04:00
Raymond Hill 89b272775a
Prevent evaluating the SNFE until fully loaded
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2925
2023-11-03 10:01:40 -04:00
Raymond Hill 605ad238b4
Report modifier stats in built-in benchmark of SNFE 2023-11-02 19:23:06 -04:00
Raymond Hill 4032ac9149
Minor re-wording in log output 2023-11-02 11:49:15 -04:00
Raymond Hill 69fce3aa6e
Fine-tune details about when differential update should kick in
Manual update of one or more lists will cause the most recent version
of these lists to be fetched from the "origin" server, and since the
lists from "origin" servers cannot be updated through differential
update, the lists will be subsequently updated according to their
`Expires` directive.

When the lists are auto-updated, the "CDN" servers will be used,
and as a result the lists will start to be updated trhough
differential updates every 6-hour (currently).

Thus it is recommended and optimal to let the lists auto-update,
since you will benefit from a much shorter delay to get up-to-date
lists (i.e. every 6-hour instead of every 6-day).

You can force the auto-updater to fetch all the lists by clicking
"Purge all caches", then restart uBO without clicking "Update".
This will cause uBO to perform an emergency auto-update at restart
time, after which you will have all the lists which are candidates
for differential update.

The "Update now" button in the "Support" pane will also cause lists
to be fetched from their "origin" server.
2023-11-01 10:45:31 -04:00
Raymond Hill 9058fce037
Fire notification when no update but `writeTime` change 2023-10-31 16:33:03 -04:00
Raymond Hill abd8577096
Distinguish diff patch not-available from not-yet-available
A not-yet-available won't cause a list to become candidate for
full update.

A not-available will cause the list to be candidate for full
update.
2023-10-31 10:52:03 -04:00
Raymond Hill 153783cfaa
Forgot to lower update period limit to 6-hour 2023-10-30 20:42:38 -04:00
Raymond Hill 3dca115eb6
Add more dev console tracing to asset updater 2023-10-30 19:49:17 -04:00
Raymond Hill 19d6d7ff5c
Minor code review: remove pointless argument 2023-10-30 14:02:47 -04:00
Raymond Hill 2344cbdeca
Harden diff-updater against unexpected errors
The diff-updater worker will terminate upon unexpected
error, in order to avoid a stalled updater.
2023-10-30 13:47:24 -04:00
Raymond Hill d05ff8ffeb
Add support for diff-patching filter lists
Related discussion:
https://github.com/ameshkov/diffupdates

The benefits of diff-patching filter lists is much shorter update
schedule and significantly less bandwidth consumed.

At the moment, only default filter lists are subject to be
diff-patched.

External filter lists can make their lists diff-patchable by
following the specification link above.

Only filter lists fetched by the auto-updater are candidate for
diff-patching.

Forcing a manual update of the filter lists will prevent the
diff-patcher from kicking in until one or more lists are
auto-updated.

Some back-of-the-envelop calculations regarding the load on free
CDN solutions used by uBO to distribute its own filter lists:

Currently, for each CDN (with lists updating after days):
~560 M req/month, ~78 TB/month

With diff-patching lists on a 6-hour schedule:
~390 M req/month, 1 TB/month

Those estimates were done according to statistics shown by
jsDelivr, which is one of 4 CDNs picked randomly when a list
updates:
https://www.jsdelivr.com/package/gh/uBlockOrigin/uAssetsCDN?tab=stats
2023-10-30 12:30:46 -04:00
Raymond Hill 032f170dba
Make update-through-link equivalent to auto-update cycle 2023-10-30 12:09:16 -04:00
Raymond Hill 4fa92f930e
Support soft-updating through update links
Where "soft-updating" means updating from CDNs, and "hard-updating"
means updating from `ublockorigin.github.io/uAssets`.
2023-10-30 10:47:38 -04:00
Raymond Hill b061db229e
Prevent `contentScripts.register` from throwing
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2915
2023-10-30 10:44:04 -04:00
Raymond Hill 0b59ebf1c6
Import translation work from https://crowdin.com/project/ublock 2023-10-27 21:27:28 -04:00
Raymond Hill 07ac27e07b
Give precedence to `Last modified:` field when evaluating "age" if lists
Available network information is not very reliable and should be
used only when `Last modified:` is not available.
2023-10-27 21:09:49 -04:00
Raymond Hill 51dfde8f41
Import translation work from https://crowdin.com/project/ublock 2023-10-25 20:04:58 -04:00
Dave Vandyke 0fa30a73c5
Further improve google-ima shim script (#3900)
I worked through some of the websites listed in the google-ima shim
script issue[1], to see what was going wrong. It turned out the
addEventListener method supports an optional context Object, which is
bound to the listener if provided. Some websites make use of that,
and then break when `this` is not bound correctly when events are
dispatched.

See also https://github.com/duckduckgo/tracker-surrogates/pull/24

1 - https://github.com/uBlockOrigin/uBlock-issues/issues/2265
2023-10-25 12:00:21 -04:00
Raymond Hill c2217a170e
Update user interface srings as suggested 2023-10-25 07:31:23 -04:00
Raymond Hill e2b9e090ce
Update user interface srings as suggested 2023-10-25 07:28:41 -04:00
Raymond Hill c1657c2f9c
Import translation work from https://crowdin.com/project/ublock 2023-10-24 11:22:21 -04:00
Dave Vandyke b445f4f4cd
Improve google-ima shim script (#3899)
We made use of the google-ima shim script from uBlock in the
DuckDuckGo Privacy Essentials extension recently, but found a few
issues with website breakage. After debugging and fixing those[1], I
thought it would be worth sharing the improvements back again. Issues
we ran into:

- The AdDisplayContainer[2] constructor usually creates some DOM
  elements inside the given container. They are all nested within a
  DIV and some websites break when at least that top DIV element isn't
  created.
- While not documented (that I could see), the event listeners can be
  added using an array of event types. Some websites make use of that
  and then hang forever when the expected events aren't fired. This
  caused videos not to play on some websites.
- The Ad.prototype.getUniversalAdIds()[3] method should return an
  array of UniversalAdIdInfo Objects, instead of an Array of
  strings. When an array of strings are returned, some websites then
  break when the UniversalAdIdInfo's methods are missing.
- The UniversalAdIdInfo.prototype.getAdIdValue()[4] method had a typo
  in the name ("getAsIdValue") and was effectively missing. That
  caused some websites to break when the missing method was called.
- The google.ima.OmidVerificationVendor[5] Object was missing, which
  broke some websites that attempted to access it.

1 - 52f2999523
2 - https://developers.google.com/interactive-media-ads/docs/sdks/html5/client-side/reference/js/google.ima.AdDisplayContainer
3 - https://developers.google.com/interactive-media-ads/docs/sdks/html5/client-side/reference/js/google.ima.Ad#getUniversalAdIds
4 - https://developers.google.com/interactive-media-ads/docs/sdks/html5/client-side/reference/js/google.ima.UniversalAdIdInfo#getAdIdValue
5 - https://developers.google.com/interactive-media-ads/docs/sdks/html5/client-side/reference/js/google.ima#.OmidVerificationVendor
2023-10-24 11:16:14 -04:00
Raymond Hill e351551dc4
Move warning sentence to the start
Related discussion:
https://github.com/uBlockOrigin/uBlock-issues/discussions/2895#discussioncomment-7353454
2023-10-23 10:01:33 -04:00
Raymond Hill 6c672c99c2
Import translation work from https://crowdin.com/project/ublock 2023-10-23 09:20:59 -04:00
Raymond Hill 589f281f21
Fine tune CSS
Related discussion:
https://github.com/uBlockOrigin/uBlock-issues/discussions/2895#discussioncomment-7353182
2023-10-22 18:33:28 -04:00
Raymond Hill 1a1876e593
Import translation work from https://crowdin.com/project/ublock 2023-10-22 18:22:27 -04:00
Raymond Hill 1e745c2f4d
Emphasize warning in "My filters"
Related feedback:
f1ce3b2191 (commitcomment-130633291)
2023-10-22 17:52:38 -04:00
Raymond Hill 9bd483bec3
Enable "not permissions" filtering expression in logger
Related feedback:
https://github.com/uBlockOrigin/uBlock-issues/discussions/2714#discussioncomment-6959861
2023-10-22 17:50:45 -04:00
Raymond Hill 9e2c919c24
Import translation work from https://crowdin.com/project/ublock 2023-10-22 15:40:16 -04:00