Commit Graph

827 Commits

Author SHA1 Message Date
Raymond Hill 22022f636f
Modularize codebase with export/import
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664

The changes are enough to fulfill the related issue.

A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:

    ./tools/make-nodejs

This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.

From the root of the package, you can try:

    node test

This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.

The test.js file contains code which is typical example
of usage of the package.

Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.

This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.

This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-27 17:26:04 -04:00
Raymond Hill e3f8a612f4
Minor code review 2021-07-16 14:06:59 -04:00
Raymond Hill bad1506f20
Move chromium-specific code path to chromium directory 2021-07-16 10:29:39 -04:00
Raymond Hill a4aced69dc
Reorganize platform directory to avoid Chromium-centric appearance
Though Firefox shares a lot of WebExtensions code with Chromium,
these platforms have their own specific code paths, for various
reasons.

The reorganization here makes it clear that Chromium platform is
just one flavor of WebExtensions, and as such all Chromium-specific
code paths should no longer be automatically pulled by other
platforms where these code paths are not needed.

Given that the filepath of many files changed, here is the
parent commit to quickly browse back to the previous directory
layout:

ec7db30b2f
2021-07-16 10:06:58 -04:00
Raymond Hill d17e22a505
Fix spurious error messages when updating contextual menu
There were spurious error messages in the dev console of
uBO in Firefox, because Firefox does not silently ignore
duplicate contextual menu entries, which could occur
transiently when the contextual menu entries were updated.

The fix simplifies contextual menu code, and actually
fulfill the original goal of avoiding to call extensions
framework API as much as possible.
2021-07-15 13:14:37 -04:00
Raymond Hill bbdb68a2b6
Synthesize missing expected onCreatedNavigationTarget events
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1659

This commit introduces a workaround for missing
onCreatedNavigationTarget() in Chromium.
2021-07-15 11:34:37 -04:00
Raymond Hill a1241e0806
Don't use `self` to access `vAPI` in content script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1571

Testing through `self` breaks the injection of
`contentscript-extra.js` in Firefox 66.0 and
earlier.
2021-04-29 13:04:59 -04:00
Raymond Hill 6eb1246508
Add `userSettings` entry to managed storage
The managed `userSettings` entry is an array of entries,
where each entry is a name/value pair encoded into an array
of strings.

The first item in the entry array is the name of a setting,
and the second item is the stringified value for the
setting.

This is a more convenient way for administrators to set
specific user settings. The settings set through
`userSettings` policy will always be set at uBO launch
time.
2021-01-16 10:35:56 -05:00
Raymond Hill 649b3480e0
Add "toOverwrite.filters" entry as managed storage property
The new entry is an array of strings, each representing a
distinct line, and all entries are used to populate the
"My filters" pane.

This offers an more straightforward way for administrators
to specify a list of custom filters to use for all
installations.
2021-01-10 12:31:31 -05:00
Raymond Hill ecabc6d12c
Raise minimal version numbers 2021-01-10 12:14:07 -05:00
Raymond Hill 0e3071dd50
Add `filterLists` property to managed storage
The entry `toOverwrite.filterLists` is an array of
string, where each string is a token identifying a
stock filter list, or a URL for an external filter
list.

This new entry is to make it easier for an
administrator to centrally configure uBO with a
custom set of filter lists.
2021-01-08 09:18:26 -05:00
Raymond Hill cc9c45f1e4
Adding to and further reviewing admin-managed settings 2021-01-06 11:39:24 -05:00
Raymond Hill 5f323f270e
Remove stray comma 2021-01-05 15:58:55 -05:00
Raymond Hill c1130ec843
Add support for admin-managed hidden settings
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1437#issuecomment-754127066
2021-01-05 12:16:50 -05:00
Raymond Hill b28acfccbc
Add "extraTrustedSiteDirectives" as new admin policy
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1433

The new "extraTrustedSiteDirectives" policy is an array
of strings, each of which is parsed as a trusted-site
directive to append to a user's own set of trusted-site
directives at launch time.

The added trusted-site directives will be considered as
part of the default set of directives by uBO.
2021-01-04 07:54:24 -05:00
Raymond Hill db7f54dbf6
Add support to launch element picker in embedded frames
Related issue:
- https://github.com/gorhill/uBlock/issues/1744

A new context menu entry, "Block element in frame...", will
be present when right-clicking on a frame element. When
this entry is clicked, uBO's element picker will be
launched from within the embedded frame and function the
same way as when launched from within the page.
2020-12-05 15:26:29 -05:00
Raymond Hill f8b15ed6cc
Fix calls to tab.removeCSS()
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1375
2020-12-02 10:46:59 -05:00
Raymond Hill 4d68d7f586
Fix handling of no-longer-existing port condition
The condition has been spotted occurring when bringing
up the DOM inspector for a page on which cosmetic filters
are being applied.

Not clear why this happens, but uBO must be ready to
graciously handle such condition.
2020-12-02 10:07:14 -05:00
Raymond Hill 525d7b1b3b
Fine tune port connection code
Related commit:
- a223031b98
2020-11-13 08:32:51 -05:00
Raymond Hill a223031b98
Work around Firefox's `data:` favIconUrl leak
Related issue:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1652925
2020-11-12 12:14:59 -05:00
Raymond Hill 8985376b00
Fix timing issue with cached redirection to web accessible resources
Reported internally by @gwarser.

In rare occasion, a timing issue could cause uBO to redirect
to a web accessible resource meant to be used for another
network request. This is a regression introduced with the
following commit:

- 2e5d32e967

Additionally, I identified another issue which would cause
cached redirection to fail when a cache entry with redirection
to a web accessible resource was being reused, an issue which
could especially affect pages which are generated dynamically
(i.e. without full page reload).
2020-11-10 10:43:26 -05:00
Raymond Hill 1e2eb037e5
Add new filter option `queryprune=`
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/760

The purpose of this new network filter option is to remove
query parameters form the URL of network requests.

The name `queryprune` has been picked over `querystrip`
since the purpose of the option is to remove some
parameters from the URL rather than all parameters.

`queryprune` is a modifier option (like `csp`) in that it
does not cause a network request to be blocked but rather
modified before being emitted.

`queryprune` must be assigned a value, which value will
determine which parameters from a query string will be
removed. The syntax for the value is that of regular
expression *except* for the following rules:

- do not wrap the regex directive between `/`
- do not use regex special values `^` and `$`
- do not use literal comma character in the value,
  though you can use hex-encoded version, `\x2c`
- to match the start of a query parameter, prepend `|`
- to match the end of a query parameter, append `|`

`queryprune` regex-like values will be tested against each
key-value parameter pair as `[key]=[value]` string. This
way you can prune according to either the key, the value,
or both.

This commit introduces the concept of modifier filter
options, which as of now are:

- `csp=`
- `queryprune=`

They both work in similar way when used with `important`
option or when used in exception filters. Modifier
options can apply to any network requests, hence the
logger reports the type of the network requests, and no
longer use the modifier as the type, i.e. `csp` filters
are no longer reported as requests of type `csp`.

Though modifier options can apply to any network requests,
for the time being the `csp=` modifier option still apply
only to top or embedded (frame) documents, just as before.
In some future we may want to apply `csp=` directives to
network requests of type script, to control the behavior
of service workers for example.

A new built-in filter expression has been added to the
logger: "modified", which allow to see all the network
requests which were modified before being emitted. The
translation work for this new option will be available
in a future commit.
2020-10-31 10:42:53 -04:00
Raymond Hill 03596439ca
Remove obsolete code to import content of localStorage
Months ago, usage of synchronous localStorage was replaced
with asynchronous extension storage. There was code for the
conversion to be seamless by importing the content of now
obsolete localStorage.

This code is no longer needed as majority of users are
assumed to use versions of uBO above 1.25.0.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899
2020-10-10 14:18:19 -04:00
Raymond Hill db0d200fcb
Allow call to removeCSS in Chromium
Related discussion:
- https://github.com/gorhill/uBlock/issues/3588#issuecomment-702218677
2020-10-02 12:16:47 -04:00
Raymond Hill 06ddc945cd
Add `fanboy.co.nz` as valid location for subscription links 2020-09-14 15:46:24 -04:00
Raymond Hill bd949ca295
Add `forums.lanik.us` as a valid location for subscription links
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1246
2020-09-14 15:37:53 -04:00
Raymond Hill fe9797cacc
Add easylist.to as a valid target for subscriber content script 2020-09-14 08:43:51 -04:00
Raymond Hill e60042595c
Revisit the behavior of the click-to-subscribe content script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/763

Changes:

From now on, uBO will allow click-to-subscribe on only
a few select domains, currently:
- https://filterlists.com/
- https://github.com/
- https://github.io/

More domains can be added if and only the demonstration
is made that more than a marginal number of filter lists
can be subscribed from those domains.

The browser alert box is no longer used to confirm
subscription to a filter list. Instead, the asset
viewer has been expanded to serve that purpose. This
way, users can peruse at the content of a filter list
before subscribing to it.
2020-09-13 08:01:53 -04:00
Raymond Hill 35aefed926
Add support to chain `:style()` to procedural operators
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/382

Additionally, remnant code for pseudo-user stylesheets
has been removed. Related commit:
- 5c68867b92
2020-09-07 08:28:01 -04:00
Raymond Hill d23f9c6a8b
Isolate element picker's svg layers from page content
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1226

Related commit:
- 9eb455ab5e

In the previous commit, the element picker dialog was
isolated from the page content. This commit is to also
isolate the svg layers from the page content.

With this commit, there is no longer a need for an anonymous
iframe and the isolated world iframe is now directly
embedded in the page.

As a result, pages are now unable to interfere with any
of the element picker user interface. Pages can now only
see an iframe, but are unable to see the content of that
iframe. The styles applied to the iframe are from a user
stylesheet, so as to ensure pages can't override the
iframe's style properties set by uBO.
2020-09-03 10:27:35 -04:00
Raymond Hill 9eb455ab5e
Isolate element picker dialog from page content world
Related issues:
- https://github.com/gorhill/uBlock/issues/3497
- https://github.com/uBlockOrigin/uBlock-issues/issues/1215

To solve above issues, the element picker's dialog is now
isolated from the page content in which it is embedded.

The highly interactive, mouse-driven part of the element
picker is still visible by the page content.
2020-09-01 12:32:12 -04:00
Raymond Hill 9ab631b4c4
Ensure port instance exists when waiting for connection requests 2020-09-01 09:53:35 -04:00
Raymond Hill 6284eca351
Fix ability to overwrite overquota'ed cloud storage
This commit fixes deleting all entries when cloud
storage usage is beyond allowed limit.

The issue would prevent pushing new data well within
quota limit because the old data beyond limit was
never removed in the first place.
2020-08-24 11:47:57 -04:00
Raymond Hill f6d1c6402f
Add support for removal of cloud storage entries 2020-08-21 09:18:33 -04:00
Raymond Hill d8b6b31eca
Add support for cloud storage compression
Cloud storage is a limited resource, and thus it
makes sense to support data compression before
sending the data to cloud storage.

A new hidden setting allows to toggle on
cloud storage compression:

name: cloudStorageCompression
default: false

By default, this hidden setting is `false`, and a
user must set it to `true` to enable compression
of cloud storage items.

This hidden setting will eventually be toggled
to `true` by default, when there is good confidence
a majority of users are using a version of uBO
which can properly handle compressed cloud storage
items.

A cursory assessment shows that compressed items
are roughly 40-50% smaller in size.
2020-08-16 11:59:27 -04:00
Raymond Hill 2afcc13ca6
Add widget to convey the amount of sync storage in use 2020-08-14 15:29:25 -04:00
Raymond Hill b01cc6ca4f
Fix cloud storage errors not reported in user interface
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/i6e7lr/
2020-08-10 08:30:52 -04:00
Raymond Hill 39190ff469
Add Chromium detection for Chromium-based MS Edge
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1174
2020-07-24 12:32:47 -04:00
Raymond Hill 5c68867b92
Deprecate pseudo user styles code
The pseudo user styles code served only browsers based
on Chromium 65 and earlier -- Chromium 66 supports
native user styles and was first released more than two
years ago.

In Chromium-based browsers, the pseudo user styles code
is being unconditionally injected in every page/frame
just in case the browser is version 65 or earlier.

Removing pseudo user styles reduce uBO's main content
script in Chromium-based browsers by more than 20K.

Related thread:
- https://github.com/NanoAdblocker/NanoCore/issues/348#issuecomment-653646507
2020-07-22 10:21:16 -04:00
Raymond Hill 221983646e
Injects user styles and content scripts in `about:blank` frames
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/688
- https://github.com/uBlockOrigin/uBlock-issues/issues/1164

`match_about_blank` is now used for content scripts and
user styles.
2020-07-19 09:06:19 -04:00
Raymond Hill feabfe3793
Do let grow subframe dictionary grow unbound
Related discussion:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1652925

It's not clear the code here will fix the reported
issue, but I did identify that the subframe
dictionary of a very long-lived web page can
theoretically grow unbound.
2020-07-18 07:44:26 -04:00
Raymond Hill a5ba668c38
Fix infinitely appending style property
Related issue:
- https://github.com/NanoAdblocker/NanoCore/issues/348
2020-07-03 15:07:52 -04:00
Raymond Hill 64cfc5779d
Use `default_public_interface_only` in Chromium-based browser
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/hbea3z/

It appears the implementation of the `disable_non_proxied_udp`
setting changed in Chromium, leading to WebRTC becoming
wholly unfunctional. Fall back to use `default_public_interface_only`
in Chromium-based browsers.
2020-06-18 11:30:05 -04:00
Raymond Hill eeae4b2cab
Add ability to open the dashboard with a keyboard shortcut
Related feedback:
- https://github.com/gorhill/uBlock/commit/9a6feb34d546#commitcomment-38852663
2020-04-30 10:31:23 -04:00
Raymond Hill b295d4a0d0
Make the new "fenix" popup panel the default one
The old "classic" popup panel will still be used
when at least one of the following is true:

- advanced setting `uiFlavor` is set to `classic`; or
- the browser is Chromium 65 or older; or
- the browser is Firefox 67 or older

The default configuration of the new popup panel
at installation time is to show the power button,
statistics and the basic tool icons, i.e. access
to dashboard, logger, pickers.

For existing installations, the new popup panel
will be configured by respecting the existing
configuration of the classic one.

The new popup panel is currently already in use
on Firefox for Android, and the visual redesign
was made according to suggestions and feedback
from <https://github.com/brampitoyo> to be
optimal for Firefox for Android.

The new popup panel will allow closing the following
pending issues:

- https://github.com/uBlockOrigin/uBlock-issues/issues/255
- https://github.com/uBlockOrigin/uBlock-issues/issues/178
2020-04-30 06:54:51 -04:00
Raymond Hill bdf9030a98
Fix hostname extraction when URL has port
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1001

The issue affected only the uncloaking of canonical
names, when the authority part of a URL contained
port information.
2020-04-27 18:30:49 -04:00
Raymond Hill 7f999b759f
Fix broken DOM inspector [regression]
Regression from:
- 0bcf04d3dd

Related feedback:
- https://github.com/gorhill/uBlock/commit/0bcf04d3dda2#commitcomment-38594585

The DOM inspector client code is injected as
content script, thus it must be categorized as
non-privileged code.
2020-04-19 08:33:09 -04:00
Raymond Hill 0bcf04d3dd
Centralize access to browser.storage.local('localStorage')
Related commit:
- 2ac288397c

Instead of having the `localStorage` data being accessed
from different locations, all accesses are now funnelled
to the main process.

Doing so simplifies the code in auxiliary processes and
also remove the need for browser.storage.local.onChanged()
listeners.

No longer using an onChanged() listener also happens to
remove spurious warnings from the Firefox console.
2020-04-15 15:55:29 -04:00
Raymond Hill 72bb700568
Add procedural cosmetic operators remove() and upward()
***

New procedural cosmetic operator: `:remove()`

Related issue:
- https://github.com/gorhill/uBlock/issues/2252

The purpose is to outright remove elements from the
DOM tree. Since `:remove()` is an "action" operator,
it must only be used as a trailing operator (just
like the `:style()` operator).

AdGuard's cosmetic filter syntax `{ remove: true; }`
will be converted to uBO's `:remove()` operator
internally.

***

New procedural cosmetic operator: `:upward(...)`

The purpose is to lookup an ancestor element.

When used with an integer argument, it is synonym of
`:nth-ancestor()`, which will be deprecated and which
will no longer be supported once no longer used in
mainstream filter lists.

Filter lists maintainers must only use `:upward(int)`
instead of `:nth-ancestor(int)` once the new operator
become available in all stable releases of uBO.

`:upward()` can also accept a CSS selector as argument,
in which case the nearest ancestor which matches the
CSS selector will be selected.
2020-03-07 14:25:06 -05:00
Raymond Hill 08d370d32e
Catch possible exceptions when accessing window.localStorage
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899

This is a code review of previous commits related to
the above issue.
2020-03-03 11:31:14 -05:00