Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/3206
The main purpose is to bypass URLs designed to track whether a user
visited a specific URL, typically used in click-tracking links.
The `urlskip=` option ...
- ... is valid only when used in a trusted filter list
- ... is enforced only on top documents
- ... is enforced on both blocked and non-blocked documents
- ... is a modifier, i.e. it cannot be used along with other
modifier options in a single filter
The syntax is `urlskip=[steps]`, where steps is a space-separated
list of extraction directives detailing what action to perform on
the current URL.
The only supported directive in this first commit is `?name`,
which purpose is to extract the value of a named URL parameter
and use the result as the new URL. Example:
||example.com/path/to/tracker$urlskip=?url
The above filter will cause navigation to
https://example.com/path/to/tracker?url=https://example.org/
to automatically bypass navigation to `example.com` and navigate
directly to
https://example.org/
It is possible to recursively extract URL parameters by using
more than one directive, example:
||example.com/path/to/tracker$urlskip=?url ?to
More extraction capabilities may be added in the future.
DNR conversion module is a dev tool, no point loading it by
default. This was done this way because in a distant past uBO
had to be compatible with browsers not supporting dynamically
loaded modules.
Currently all supported browser versions support dynamically
loaded modules.
In Firefox-specific contentScripts API used to register
scriptlets. This could potentially occurs when there are
registrations pending during a reload of filter lists.
This commit makes the DNS resolution code better suited for both
filtering on cname and ip address. The change allows early availability
of ip address so that `ipaddress=` option can be matched at
onBeforeRequest time.
As a result, it is now possible to block root document using
`ipaddress=` option -- so long as an ip address can be extracted
before first onBeforeRequest() call.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2792
Caveat
------
the ip address used is the first one among the list of ip
addresses returned by dns.resolve() method. There is no way for uBO
to know which exact ip address will be used by the browser when
sending the request, so this is at most a best guess. The exact IP
address used by the browser is available at onHeadersReceived time,
and uBO will also filter according to this value, but by then the
network request has already been sent to the remote server.
Possibly a future improvement would make available the whole list
of ip addresses to the filtering engine, but even then it's impossible
to know with certainty which ip address will ultimately be used by the
browser -- it is entirely possible that the ip address used by the
browser might not be in the list received through dns.resolve().
If an IP address can be extracted from the hostname portion of
a URL, the IP address matching will be performed at onBeforeRequest()
time.
Regardless, IP address matching will subsequently always be performed
at onHeadersReceived() time as the request details at that point
contain a reliable IP address value on supported platforms (Firefox-
only as of now).
The `cap_ipaddress` now evaluates to `true` in Chromium-based
browsers. Even though these browsers are unable to provide reliable
IP address value at onHeadersReceived() time, they can still
perform IP address matching for IP address extracted from hostname
portion of a URL.
Properly report unsupported `header=` and `ipaddress=` option in
log file.
`header=` support may become possible soon with Chromium 128
introducing blocking according to response headers content.
The purpose is to block according to the ip address of a network
request. In the current implementation, the filter option can only
be enforced at onHeadersReceived time.
The new filter option cannot be enforced in Chromium-based browsers
since the ip address of network requests is available only at
onResponseStarted time, which is not blocking.
The value assigned to `ipaddress` can either be a plain string which
must match exactly a given ip address, or a regex which will be
matched against the ip address.
The `ipaddress` option can only be enforced when the extension
framework does provide a valid ip address in a onHeadersReceived
listener. For instance, cached resources do not have a valid ip
address and thus can't be a match to `ipaddress` option.
Example:
*$script,ipaddress=93.184.215.14
For the sake of convenience for filter list maintainers, this commit
add ability to quote static network option values, so as to avoid the
need to escape commas when parser ambiguity arises.
The quotes can be `"`, `'`, or backticks.
Example, the following filter requires escaping commas:
example.com$xhr,replace=/"loremIpsum.*?([A-Z]"\}|"\}{2\,4})\}\]\,//,1p
Can be now rewritten with no need to escape when using quotes:
example.com$xhr,replace='/"loremIpsum.*?([A-Z]"\}|"\}{2,4})\}\],//',1p
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/3362
There used to be a way to test URL against the network filtering engine,
but this was removed in a distant past during refactoring.
The ability has been brought back through uBO's own developer tools,
accessible through the _More_ button in the _Support_ pane in the
dashboard.
To query the static network filtering engine, enter the following
in the text editor:
snfe?url-to-test [type] [url-of-context]
`snfe?` is a prompt indicating the intent to query the static network
filtering engine.
At a minimum there must be a URL to test.
Optionally the type of the resource to match, default to `xhr` if
none specified. Also optionally, the context from within which the
request is made. Example:
Enter:
snfe?https://www.google-analytics.com/analytics.js
Result:
url: https://www.google-analytics.com/analytics.js
blocked: ||google-analytics.com^
Enter:
snfe?https://www.google-analytics.com/analytics.js script
Result:
url: https://www.google-analytics.com/analytics.js
type: script
blocked: ||google-analytics.com^
modified: ||google-analytics.com/analytics.js$script,redirect-rule=google-analytics_analytics.js:5
Enter:
snfe?https://example.com/
Result:
url: https://example.com/
not blocked
Enter:
snfe?https://example.com/ ping
Result:
url: https://example.com/
type: ping
blocked: *$ping,3p
A new icon has been added to the popup panel, to open a popup
window with a terse list of DNR events for the current tab, in
reverse chronological order (most recent DNR event appears at
the top).
The new ability is available only when the extension is sideloaded,
as per `declarativeNetRequestFeedback` documentation. Ref:
https://developer.chrome.com/docs/extensions/reference/api/declarativeNetRequest#event-onRuleMatchedDebug
Purposefully minimal, so as to have something rather than nothing
when having to diagnose filtering issue with the DNR API. Example:
https://github.com/uBlockOrigin/uBOL-home/issues/156
The content of the popup window does not dynamically update, force
a refresh (F5) to get the most recent DNR events. This might be
improved in the future.
The DNR event buffer is not persisted, so the buffer is empty when
service worker is restarted. This might be improved in the future
by using session storage API.
There is no output filtering ability in this first draft. This
might be improved in the future.
DNR rules are reported. The filter from which a DNR rule
originates is not reported. Given that the rulesets are optimized
after conversion from original filter lists to reduce the DNR rule
count, this is unlikely to ever be possible.
`subject:matches-prop(arg)`
Description: Allows to select an element by a property name (or chain of
properties), and optionally the property value.
Chainable: Yes.
`subject`: Can be a plain CSS selector, or a procedural cosmetic filter.
`arg`: A declaration in the form `chain=value`, where `chain` is a dot-
separated string for the target property, and `value` is the optional
property value to match. `value` can be literal text or literal regular
expression. When no `value` is declared, the operator only tests for
the presence of the target property
Example:
example.org##div:matches-prop(imanad)
example.org##img:matches-prop(naturalWidth=160)
Corrpution would occur when reading back serialized data which
contained multiple references to same instance of an object.
The issue could manifest when reading cache storage-related
data from the browser storage API, since the serializer is not
used when reading from indexedDB. Private/incognito mode
fall back on using browser storage API as cache storage.
Off the top of my head, I think the following conditions all
together could result in high likelihood of malfunction caused
by improperly deserializing data at launch time:
- Load from a selfie
- Selfie created after uBO ran for a while
- Selfie loaded from browser storage API (not the case by
default)
Possibly related to reports of uBO malfunctioning:
https://github.com/uBlockOrigin/uBlock-issues/issues/3217#event-12686416838
Raymond Hill