Commit Graph

1756 Commits

Author SHA1 Message Date
Xavier Vello c8d1834018 Add letsblock.it as valid location for subscription links 2022-02-16 16:28:13 +01:00
Raymond Hill 2933016d4b
Rework behavior of "Suspend network activity until ..."
The setting will default to the natural capability of the browser:

- Checked for Firefox
- Unchecked for Chromium-based browsers

For Chromium-based browser, if checked, network requests will be
redirected to an empty resources instead of blocking the
connection.

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1973
- https://www.reddit.com/r/uBlockOrigin/comments/squo8n/latest_update_blocks_network_connections_at/
2022-02-13 09:24:57 -05:00
Raymond Hill a2d83afe66
Further narrow when unconditionally calling webRTCIPHandlingPolicy.clear()
The enabled cached state probably exists in the browser only
when the setting was originally enabled -- so we need to clear
it only in such case.
2022-02-06 09:00:25 -05:00
Raymond Hill 00b257caa7
Unconditionally call webRTCIPHandlingPolicy.clear() at launch
On Chromium-based browsers only.

Related issue/feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1928
- https://www.reddit.com/r/uBlockOrigin/comments/sl7p74/
2022-02-05 12:04:08 -05:00
Raymond Hill b4911e2d7c
Raise minimum versions of browsers
For the following reasons:

- Support for CSS `gap`, so as to remove the need to
  fall back onto classic popup panel, and thus making
  it possible to fully remove long-ago deprecated
  classic panel (announced in 1.27.0)[1]:
  https://developer.mozilla.org/en-US/docs/Web/CSS/gap

- Support for dynamic import:
  https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import

Firefox's minimum version is raised to 68 because it's
the oldest ESR version supporting the features above.
From now on, minimum version of Firefox will always be
an ESR one.

---

[1] https://github.com/gorhill/uBlock/releases/tag/1.27.0
2022-01-28 12:02:56 -05:00
Raymond Hill 250cf96aae
Fix regression causing regex-based filters to be case sensitive
Related feedback:
- https://github.com/AdguardTeam/AdguardFilters/issues/88067#issuecomment-1019518277

Regression commit:
- 725e6931f5
2022-01-23 12:32:11 -05:00
Raymond Hill 0bc0af9d8d
Add test to detect case of improper deserialization
Related commit:
- 8f461072f5
2022-01-12 09:03:38 -05:00
Raymond Hill eddb9ec160
Raised guessed Firefox version to 91 (latest ESR) 2022-01-09 12:41:29 -05:00
Raymond Hill d38c19cabf
Improve test for presence of `browser` as extensions API
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1914

The issue affected only Chromium-based browsers.
2022-01-01 11:07:17 -05:00
Raymond Hill 925c8d5d0c
Add setting to control suspension on network activity at launch
Related discussion:
- a0a9497b4a (commitcomment-62560291)

The new setting, when disabled (enabled by default), allows a user
to prevent uBO from waiting for all filter lists to be loaded
before allowing network activity at launch. The setting is enabled
by default, meaning uBO waits for all filter lists to be loaded in
memory before unsuspending network activity. Some users may find
this behavior undesirable, hence the new setting.

This gives the option to potentially speed up page load at launch,
at the cost of potentially not properly filtering network requests
as per filter lists/rules.

For platforms not supporting the suspension of network activity,
the setting will merely prevent whatever mechanism exists on the
platform to mitigate improper filtering of network requests at
launch. For example, in Chromium-based browsers, unchecking the
new setting will prevent the browser from re-loading tabs for
which there was network activity while in "suspended" state at
launch.
2021-12-30 09:24:38 -05:00
Raymond Hill eab2e03f2b
New revision for dev build 2021-12-27 19:12:29 -05:00
Raymond Hill 0992a5b821
Add test for https://github.com/gorhill/uBlock/commit/d66cd1116c0e 2021-12-25 09:10:48 -05:00
Raymond Hill e2be049774
New revision for release candidate
Aiming for a pre-holiday release.
2021-12-18 13:47:38 -05:00
Raymond Hill 26e62eb29b
New revision for dev build 2021-12-16 09:39:44 -05:00
Raymond Hill 89e67887ee
Create synthetic event for new windows created from external application
Related discussion:
- https://github.com/uBlockOrigin/uAssets/issues/10323#issuecomment-993592288

Synthesize a onCreatedNavigationTarget event for new browser windows
created as a result of an external application triggering a navigation
event.
2021-12-14 11:23:46 -05:00
Raymond Hill 7de2359ccb
New revision for dev build 2021-12-13 12:56:56 -05:00
Raymond Hill 685a9779e5
New revision for dev build 2021-12-12 11:33:15 -05:00
Raymond Hill 25fc118337
New revision for dev build 2021-12-11 09:54:43 -05:00
Raymond Hill ca1ec1461b
Narrow usage of `webRTCIPHandlingPolicy` to only `firefox`
In addition of `mobile`, just in case -- some people are
using mobile version of chromium out there.
2021-12-11 07:37:18 -05:00
Raymond Hill 1cfcb1623f
New revision for dev build 2021-12-10 08:29:33 -05:00
Raymond Hill 62231b73e5
New revision for dev build 2021-12-06 12:08:58 -05:00
Raymond Hill 3004c8832a
Fix `dig-snfe modifiers` to properly detect changes in results 2021-12-06 08:01:05 -05:00
Raymond Hill 190bbf627b
New revision for dev build 2021-12-06 07:07:22 -05:00
Raymond Hill d3fe0ccfe0
Fix regression with `csp=`, deprecate `queryprune`, etc
Fixed serious regression in previous dev build in applying
`csp=` filters. Reported internally by uBO team.

Promote usage of `removeparam` in code instead of `queryprune`,
which is to be deprecated.

Removed test against previously tested hostname in
FilterHostnameDict since as per various benchmark, the
test does not really help.

Remove serialization API in Node.js code as the API is now
present in SNFE itself.
2021-12-06 07:01:39 -05:00
Raymond Hill ae124065af
New revision for dev build 2021-12-05 09:12:55 -05:00
Raymond Hill 4b68f28e08
New revision for dev build 2021-12-04 17:10:51 -05:00
Raymond Hill b98836ab8e
Fix NPM package documentation 2021-12-04 12:40:43 -05:00
Raymond Hill 82f31e7863
Bump npm package version number 2021-12-04 12:34:36 -05:00
Raymond Hill b1a338681b
Shield against possible case of invalid attribute name
Not sure this can really happen, but if ever Math.random() would
return `0.9999999999999999`, the attribute name would start with
`{`, i.e. an invalid attribute name.
2021-10-26 08:55:05 -04:00
Raymond Hill 318469b005
Remove unused function 2021-10-22 08:34:04 -04:00
Raymond Hill 18ae79cdf2
Remove unused code path as per current minimum browser version 2021-10-17 13:50:43 -04:00
Raymond Hill e4d75fa025
Ignore unused tabs.onUpdated() events 2021-10-17 12:55:31 -04:00
Raymond Hill b0038eac28
Remove unused code path
Versions of Firefox 54 and lower are no longer supported.
2021-10-13 11:17:21 -04:00
Raymond Hill c4b7ee80ea
Further work on JS modules
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
2021-10-07 14:41:29 -04:00
Raymond Hill 6464002088
Give precedence to negated types in case of ambiguity
Related feedback:
- https://github.com/uBlockOrigin/uAssets/issues/7639#issuecomment-933525018
2021-10-06 09:44:15 -04:00
Raymond Hill ebe173d273
Add a test for negated type with `all` option
Related feedback:
- https://github.com/uBlockOrigin/uAssets/issues/7639#issuecomment-933525018
2021-10-06 08:48:49 -04:00
Raymond Hill a76935b232
Add more npm tests
Also, disable wasm tests, currently erroring with:

    WebAssembly.Memory(): could not allocate memory
2021-10-04 12:47:12 -04:00
Raymond Hill 6d21bd4af9
Fix usage documentation
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1734
2021-09-25 07:47:40 -04:00
Raymond Hill 1130746a9b
Fix regression with `important` filter option
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1732

The regression affect filter with the `important` option when
the following conditions were fulfilled:

- The filter pattern is pure hostname
- The filter has not one of the following options:
  - domain
  - denyallow
  - header
  - strict1p, strict3p
  - csp
  - removeparam
- There is a matching exception filter

Related commit:
- a2a8ef7e85

A related mocha test has been added in order to detect this
specific regression in the future through `make test`.
2021-09-24 11:09:32 -04:00
Raymond Hill 4100761863
Keep ability to prevent local IP address leaks for mobile version
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1723

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1723#issuecomment-919913361
2021-09-15 08:28:10 -04:00
Raymond Hill 831cf478da
Revert "Remove now obsolete webrtc-related setting"
This reverts commit af41aadab7.
2021-09-15 07:40:32 -04:00
Raymond Hill af41aadab7
Remove now obsolete webrtc-related setting
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1723

The leaking of local IP addresses is no longer an issue
in modern browsers.
2021-09-14 10:04:35 -04:00
Manish Jethani b83e2d4e2b
Fix ESLint warning in useLists() (#3855) 2021-09-11 09:33:32 -04:00
Manish Jethani f316a326cf
Use native Punycode conversion (#3854) 2021-09-04 08:44:26 -04:00
Raymond Hill 6703ede9e8
Better diff output 2021-09-01 18:26:36 -04:00
Raymond Hill d41f28fcd8
Add median cost output to dig-snfe 2021-08-29 08:58:20 -04:00
Manish Jethani 9761b02c79
Convert publicsuffixlist.js into an ES module (#3846) 2021-08-23 09:42:27 -04:00
Manish Jethani 9ddbb293c0
Convert punycode.js into an ES module (#3845) 2021-08-22 12:03:59 -04:00
Manish Jethani d6339ada62
Use WebAssembly object directly in index.js (#3843) 2021-08-21 16:02:32 -04:00
Raymond Hill 02a17af810
Let the caller do the awaiting
Local useLists() no longer returns a reference to
internal snfe instance.
2021-08-19 07:34:02 -04:00
Manish Jethani 4c1c6309b3
Add tests for SNFE filter loading multiple calls (#3836) 2021-08-18 07:52:54 -04:00
Manish Jethani 2a130704e3
Add check-leaks make target (#3837) 2021-08-18 07:28:23 -04:00
Raymond Hill 7a5c4e9547
Throw when useLists() called concurrently
Related feedback:
- https://github.com/gorhill/uBlock/pull/3836/files#r690687656
2021-08-17 16:57:39 -04:00
Raymond Hill 60e254608a
Expose hasQuery() and fix coarse test for query parameters 2021-08-17 16:49:43 -04:00
Raymond Hill 8959cea3cc
Expose matchAndFetchModifiers() in npm package
Also, add instrumentation for the method in dig-snfe.
2021-08-17 12:48:39 -04:00
Manish Jethani a9aca818f9
Add tests for enableWASM() (#3835) 2021-08-17 09:53:28 -04:00
Manish Jethani 9a5a13a506
Add --full-battery option for tests (#3834) 2021-08-17 08:55:31 -04:00
Raymond Hill b44d9219c3
New revision for dev build 2021-08-17 08:25:31 -04:00
Manish Jethani 0bf19cc683
Run SNFE tests with Wasm off and on (#3833) 2021-08-17 08:24:07 -04:00
Manish Jethani f020e5334c
Use createWorld() in request data tests (#3832) 2021-08-17 07:20:13 -04:00
Raymond Hill f0cb4091f0
Bump up npm package version 2021-08-16 12:56:39 -04:00
Raymond Hill be7418264b
Exclude more resources from the published npm package 2021-08-16 12:40:46 -04:00
Raymond Hill 10ca7438d7
Increase the logging of first best/worst requests to 1000 2021-08-16 10:54:27 -04:00
Manish Jethani ffb4fe9b8f
Run request data tests with Wasm enabled too (#3829) 2021-08-16 09:22:35 -04:00
Manish Jethani 36377320ae
Run tests on request data (#3828) 2021-08-16 07:39:09 -04:00
Manish Jethani 514143cd80
Add c8 (#3826) 2021-08-15 17:33:56 -04:00
Manish Jethani e009d69f86
Add tests for SNFE deserialization (#3827) 2021-08-15 17:28:16 -04:00
Raymond Hill 8bb44242bd
Add platform "dig", to experiment/validate code changes
This is a replacement for the dubious approach when the
extension itself was used to run benchmarks to detect
performance and filtering behavior regressions.
2021-08-15 15:47:40 -04:00
Manish Jethani 29cea49197
Add tests for SNFE serialization (#3825) 2021-08-15 11:49:23 -04:00
Raymond Hill 0ec1204d37
Merge branch 'master' of https://github.com/gorhill/uBlock 2021-08-15 11:19:35 -04:00
Raymond Hill e9ae8e1a0f
Add static release() to StaticNetFilteringEngine class
The method ensures the actual SNFE is properly reset.
2021-08-15 11:18:01 -04:00
Manish Jethani 48d4f890d7
Add tests for SNFE initialization (#3823) 2021-08-15 11:16:57 -04:00
Manish Jethani f8f45cab70
Lint Mocha tests too (#3824) 2021-08-15 11:13:13 -04:00
Raymond Hill 087da6407d
Add support for nodejs flavors
The main nodejs flavor is "npm", which is to be used to
lint/test and the publication of an official npm
package -- and by design it has dependencies on mocha,
eslint, etc.

A new flavor "dig" has been created with minimal
dependencies and which purpose is to easily allow to
write specialized code to investigate local code changes
in uBO -- and it's not meant for publication.

Consequently, "make nodejs" has been replaced with
"make npm", and a new "dig" target has been added to the
makefile, to be used for instrumenting local code changes
for investigation purpose.
2021-08-15 10:43:36 -04:00
Manish Jethani 0a11999459
Test in new world (#3822) 2021-08-15 08:41:30 -04:00
Manish Jethani 6c2856cb7e
Add package-lock.json (#3819) 2021-08-14 11:59:14 -04:00
Manish Jethani fd9888f85e
Spawn mocha from test.js (#3818) 2021-08-14 10:33:49 -04:00
Manish Jethani bb5bfed779
Add tests for promise-based filter loading (#3817) 2021-08-14 07:56:03 -04:00
Manish Jethani ef0075acc1
Add tests for filter loading (#3816) 2021-08-13 16:21:36 -04:00
Manish Jethani 806fe5dbe1
Add Mocha tests (#3815) 2021-08-13 13:08:13 -04:00
Manish Jethani a56f43f007
Add .npmrc to stop generating package lock file (#3814) 2021-08-13 11:49:52 -04:00
Manish Jethani daff6b3a34
Add eslint dependency within Node.js package (#3813) 2021-08-13 06:43:20 -04:00
Manish Jethani 993835b333
Use test assertions (#3811) 2021-08-10 14:08:05 -04:00
Raymond Hill c6fb70b1f0
Refactor hntrie to avoid the need for boundary cells
Whereas before the string segment was encoded as:

LL OOOOOOOOOOOO

where L are the upper 8 bits and used to encode the length
of the segment, and O are the lower 24 bits and used to
encode the offset of the string data in the character
buffer, the new code encode as follow:

OOOOOOOOOOOO LL

And furthermore the most significant bit of the length
LL is now used to mark whether the current string segment
is a label boundary.

This means a cell can't reference a segment longer then
127 characters. To work around this limitation for when a
segment is longer than 127 characters (a rare occurrence),
the algorithm will simply split the segment into multiple
adjacent cells.

As a result, there is no longer a need to encode
"boundariness" into special cells, which simplifies
both the storing and matching algorithms.

Additionally, added minimal documentation for the NPM
package on how to import and use HNTrieContainer as a
standalone API.
2021-08-10 09:27:59 -04:00
Manish Jethani ab9f5ae100
Run build script via tools/make-nodejs.sh (#3808) 2021-08-09 20:02:58 -04:00
Raymond Hill 76abd04898
New revision for npm package 2021-08-09 19:34:35 -04:00
Manish Jethani 6b993f2f06
Fix pslInit() for install script (#3807) 2021-08-09 19:12:04 -04:00
Manish Jethani d9adf5a6fb
Add serialization API to StaticNetFilteringEngine (#3806) 2021-08-09 16:52:41 -04:00
Raymond Hill 19f59df22d
Increase minimum version to Firefox 60.0
JavaScript modules becames officially available in
Firefox 60.0, which was published more than 3 years
ago.
2021-08-08 10:47:04 -04:00
Raymond Hill 5879610597
Bump up nodejs package version 2021-08-08 09:21:03 -04:00
Raymond Hill 7cd583a301
Revisit the nodejs API 2021-08-08 09:17:14 -04:00
Manish Jethani 65f0909ba0
Implement class StaticNetFilteringEngine (#3805) 2021-08-08 07:22:02 -04:00
Raymond Hill 500c895f6b
update npm README 2021-08-07 11:38:22 -04:00
Raymond Hill af54415959
README has unsaved pending edits... 2021-08-06 08:29:13 -04:00
Raymond Hill 4495c37ca3
Provide basic documentation for the NPM package 2021-08-06 08:25:36 -04:00
Raymond Hill 5eb99a4dd6
... 2021-08-05 14:36:13 -04:00
Raymond Hill 05f0e04c07
Just trying to get this done.. 2021-08-05 14:34:21 -04:00
Raymond Hill fe7ce8f576
Need to bump version to publish README 2021-08-05 14:32:07 -04:00
Raymond Hill 70d5d28a7c
Add minimal README to nodejs package 2021-08-05 14:28:17 -04:00
Raymond Hill 8c48375caf
Fix test.js following changes in API 2021-08-05 14:19:04 -04:00
Raymond Hill 63e55a1696
Expose ability to compile raw list into their intermeditate form
For clients who may wish to persist the intermediate compiled form
in order to be able to skip costly parsing operation when the
list is fed to the static network filtering engine.
2021-08-05 13:37:41 -04:00
Raymond Hill 4818405cf6
Remove need to pass parser at every compile() call
The compiler instance is already initialized with a
reference to the parser, no need to keep passing the
reference at each call to compile().
2021-08-05 13:30:20 -04:00
Manish Jethani 543e1a3aea
Add ESLint rules for possible errors (#3804) 2021-08-04 15:27:32 -04:00
Raymond Hill 85c68116bd
Group all compiling-related code into FilterCompiler() class
In the static network filtering engine (snfe), the
compiling-related code was spread across two classes.
This commit makes it so that all the compiling-related
code is in FilterCompiler class, which clear purpose is
to compile raw filters into a form which can be persisted
and later fed to the snfe with no parsing overhead.

To compile raw static network filter, the new approach is:

    snfe.createCompiler(parser);

Then for each single raw filter to compile:

    compiler.compile(parser, writer);

The caller is responsible to keep a reference to the
compiler instance for as long as it is needed. This removes
the need for the clunky code used to keep an instance of
compiler alive in the snfe.

Additionally, snfe.tokenHistograms() has been moved to
benchmarks.js, as it has no dependency on the snfe, it's
just a utility function.
2021-08-04 15:14:48 -04:00
Raymond Hill cbec7397fc
Fix jshint warnings
Those warnings surfaced as a result of removing the braces
used to create a local namespace.
2021-08-03 12:08:59 -04:00
Manish Jethani 4094f48547
Fix style in index.js (#3802) 2021-08-03 12:03:11 -04:00
Manish Jethani 6ef74fc21b
Rewrite logical expressions for ESLint (#3801) 2021-08-03 10:59:01 -04:00
Manish Jethani 3ca5e6817d
Fix ESLint indentation warnings (#3800)
* Fix ESLint indentation warnings

* Undo code reformatting within parentheses

* Add exception for logical expressions

* Update array expression rule

* Disable rule for reProceduralOperator assignment

* Fix indentation in static-filtering-parser.js
2021-08-03 10:14:40 -04:00
Manish Jethani 6b85b7f2b1
Create build folder in npm install script (#3799) 2021-08-03 07:02:48 -04:00
Manish Jethani ad69c760fb
Run ESLint during Node.js package generation (#3798) 2021-08-02 16:55:03 -04:00
Manish Jethani 502e5b0ec8
Use fs API to load serialized PSL (#3797) 2021-08-02 12:32:47 -04:00
Manish Jethani 3879835324
Remove bundle generation from Node.js package (#3796) 2021-08-02 09:17:56 -04:00
Raymond Hill f1106acf4b
Rename nodejs package
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664#issuecomment-890834827
2021-08-02 08:08:43 -04:00
Manish Jethani 297bcea49b
Generate bundle.min.js in Node.js install script (#3794) 2021-08-01 18:33:13 -04:00
Manish Jethani 0b764934e0
Serialize PSL in Node.js install script (#3793) 2021-08-01 16:42:05 -04:00
Raymond Hill c6d275674d
Revert "Make the nodejs package load a serialized version of the PSL"
This reverts commit 46c6ff8708.
2021-08-01 15:42:46 -04:00
Raymond Hill 46c6ff8708
Make the nodejs package load a serialized version of the PSL
Related discussion:
- https://github.com/cliqz-oss/adblocker/pull/2091#issuecomment-890545926
2021-08-01 12:58:21 -04:00
Manish Jethani 8a33bda653
Use lower case for Node.js package name (#3792) 2021-08-01 05:22:57 -04:00
Raymond Hill cb72211795
Move orphanizeString() into text-utils module
Another small step toward the goal of reducing dependency
on `µb`.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664

text-iterators module has been renamed text-utils to better
reflect its content.
2021-07-31 08:38:33 -04:00
Raymond Hill 2eb02c5adc
Fix spurious error message in dev console
Ports can be disconnected under certain conditions,
and in such case runtime.lastError must be accessed
to avoid the extensions framework from emitting
error messages in the console.
2021-07-31 07:20:38 -04:00
Raymond Hill 23cabfd440
Fix erroneous removal of code from pslInit() 2021-07-31 07:09:01 -04:00
Raymond Hill d5da88ea87
Simplify loading of WASM modules in nodejs 2021-07-31 07:06:11 -04:00
Raymond Hill 5be4d5d877
Make pslInit() returns the publicSuffixList instance used internally
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664#issuecomment-889647595
2021-07-30 07:00:45 -04:00
Raymond Hill 076a088371
Remove stray `async` 2021-07-29 17:04:32 -04:00
Raymond Hill 98fc66bb1b
Add support for enabling WASM code paths in NodeJS package
See `test.js` for reference on how to enable WASM code
paths (which are disabled by default).
2021-07-29 16:54:51 -04:00
Raymond Hill 62b6826dd5
Further modularize uBO's codebase
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664

Modularization is a necessary step toward possibly publishing
a more complete nodejs package to allow using uBO's filtering
capabilities outside of the uBO extension.

Additionally, as per feedback, remove undue usage of console
output as per feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664#issuecomment-888451032
2021-07-28 19:48:38 -04:00
Raymond Hill e1222d1643
Use `require` instead of `fs`
Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664#issuecomment-888332409
2021-07-28 10:38:24 -04:00
Raymond Hill 22022f636f
Modularize codebase with export/import
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664

The changes are enough to fulfill the related issue.

A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:

    ./tools/make-nodejs

This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.

From the root of the package, you can try:

    node test

This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.

The test.js file contains code which is typical example
of usage of the package.

Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.

This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.

This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-27 17:26:04 -04:00
Raymond Hill c9710ac9d1
Better deal with no-longer-existing ports
Disconnected ports could still happen Even when the port
was still seen as valid internally. Using a try-catch
block makes invalid port detection more reliable. This
is an occurrence I often encountered when stepping into
content script code, causing suprious error messages to
be thrown into uBO's background dev console.
2021-07-19 11:25:52 -04:00
Raymond Hill e3f8a612f4
Minor code review 2021-07-16 14:06:59 -04:00
Raymond Hill bad1506f20
Move chromium-specific code path to chromium directory 2021-07-16 10:29:39 -04:00
Raymond Hill a4aced69dc
Reorganize platform directory to avoid Chromium-centric appearance
Though Firefox shares a lot of WebExtensions code with Chromium,
these platforms have their own specific code paths, for various
reasons.

The reorganization here makes it clear that Chromium platform is
just one flavor of WebExtensions, and as such all Chromium-specific
code paths should no longer be automatically pulled by other
platforms where these code paths are not needed.

Given that the filepath of many files changed, here is the
parent commit to quickly browse back to the previous directory
layout:

ec7db30b2f
2021-07-16 10:06:58 -04:00
Raymond Hill d17e22a505
Fix spurious error messages when updating contextual menu
There were spurious error messages in the dev console of
uBO in Firefox, because Firefox does not silently ignore
duplicate contextual menu entries, which could occur
transiently when the contextual menu entries were updated.

The fix simplifies contextual menu code, and actually
fulfill the original goal of avoiding to call extensions
framework API as much as possible.
2021-07-15 13:14:37 -04:00
Raymond Hill bbdb68a2b6
Synthesize missing expected onCreatedNavigationTarget events
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1659

This commit introduces a workaround for missing
onCreatedNavigationTarget() in Chromium.
2021-07-15 11:34:37 -04:00
Raymond Hill a1241e0806
Don't use `self` to access `vAPI` in content script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1571

Testing through `self` breaks the injection of
`contentscript-extra.js` in Firefox 66.0 and
earlier.
2021-04-29 13:04:59 -04:00
Raymond Hill 1c3b45f75d
Expose ability to toggle on/off cname-uncloaking to all users
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1513

Prior to this commit, the ability to enable/disable the
uncloaking of canonical names was only available to advanced
users. This commit make it so that the setting can be
toggled from the _Settings_ pane.

The setting is enabled by default. The documentation should
be clear that the setting should not be disabled unless it
actually solves serious network issues, for example:

https://bugzilla.mozilla.org/show_bug.cgi?id=1694404

Also, as a result, the advanced setting `cnameUncloak` is no
longer available from within the advanced settings editor.
2021-03-02 13:00:56 -05:00
Raymond Hill 6eb1246508
Add `userSettings` entry to managed storage
The managed `userSettings` entry is an array of entries,
where each entry is a name/value pair encoded into an array
of strings.

The first item in the entry array is the name of a setting,
and the second item is the stringified value for the
setting.

This is a more convenient way for administrators to set
specific user settings. The settings set through
`userSettings` policy will always be set at uBO launch
time.
2021-01-16 10:35:56 -05:00
Raymond Hill 649b3480e0
Add "toOverwrite.filters" entry as managed storage property
The new entry is an array of strings, each representing a
distinct line, and all entries are used to populate the
"My filters" pane.

This offers an more straightforward way for administrators
to specify a list of custom filters to use for all
installations.
2021-01-10 12:31:31 -05:00
Raymond Hill ecabc6d12c
Raise minimal version numbers 2021-01-10 12:14:07 -05:00
Raymond Hill 0e3071dd50
Add `filterLists` property to managed storage
The entry `toOverwrite.filterLists` is an array of
string, where each string is a token identifying a
stock filter list, or a URL for an external filter
list.

This new entry is to make it easier for an
administrator to centrally configure uBO with a
custom set of filter lists.
2021-01-08 09:18:26 -05:00
Raymond Hill cc9c45f1e4
Adding to and further reviewing admin-managed settings 2021-01-06 11:39:24 -05:00
Raymond Hill 5f323f270e
Remove stray comma 2021-01-05 15:58:55 -05:00
Raymond Hill c1130ec843
Add support for admin-managed hidden settings
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1437#issuecomment-754127066
2021-01-05 12:16:50 -05:00
Raymond Hill b28acfccbc
Add "extraTrustedSiteDirectives" as new admin policy
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1433

The new "extraTrustedSiteDirectives" policy is an array
of strings, each of which is parsed as a trusted-site
directive to append to a user's own set of trusted-site
directives at launch time.

The added trusted-site directives will be considered as
part of the default set of directives by uBO.
2021-01-04 07:54:24 -05:00
Raymond Hill db7f54dbf6
Add support to launch element picker in embedded frames
Related issue:
- https://github.com/gorhill/uBlock/issues/1744

A new context menu entry, "Block element in frame...", will
be present when right-clicking on a frame element. When
this entry is clicked, uBO's element picker will be
launched from within the embedded frame and function the
same way as when launched from within the page.
2020-12-05 15:26:29 -05:00
Raymond Hill f8b15ed6cc
Fix calls to tab.removeCSS()
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1375
2020-12-02 10:46:59 -05:00
Raymond Hill 4d68d7f586
Fix handling of no-longer-existing port condition
The condition has been spotted occurring when bringing
up the DOM inspector for a page on which cosmetic filters
are being applied.

Not clear why this happens, but uBO must be ready to
graciously handle such condition.
2020-12-02 10:07:14 -05:00
Raymond Hill 525d7b1b3b
Fine tune port connection code
Related commit:
- a223031b98
2020-11-13 08:32:51 -05:00
Raymond Hill a223031b98
Work around Firefox's `data:` favIconUrl leak
Related issue:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1652925
2020-11-12 12:14:59 -05:00
Raymond Hill 8985376b00
Fix timing issue with cached redirection to web accessible resources
Reported internally by @gwarser.

In rare occasion, a timing issue could cause uBO to redirect
to a web accessible resource meant to be used for another
network request. This is a regression introduced with the
following commit:

- 2e5d32e967

Additionally, I identified another issue which would cause
cached redirection to fail when a cache entry with redirection
to a web accessible resource was being reused, an issue which
could especially affect pages which are generated dynamically
(i.e. without full page reload).
2020-11-10 10:43:26 -05:00
Raymond Hill 1e2eb037e5
Add new filter option `queryprune=`
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/760

The purpose of this new network filter option is to remove
query parameters form the URL of network requests.

The name `queryprune` has been picked over `querystrip`
since the purpose of the option is to remove some
parameters from the URL rather than all parameters.

`queryprune` is a modifier option (like `csp`) in that it
does not cause a network request to be blocked but rather
modified before being emitted.

`queryprune` must be assigned a value, which value will
determine which parameters from a query string will be
removed. The syntax for the value is that of regular
expression *except* for the following rules:

- do not wrap the regex directive between `/`
- do not use regex special values `^` and `$`
- do not use literal comma character in the value,
  though you can use hex-encoded version, `\x2c`
- to match the start of a query parameter, prepend `|`
- to match the end of a query parameter, append `|`

`queryprune` regex-like values will be tested against each
key-value parameter pair as `[key]=[value]` string. This
way you can prune according to either the key, the value,
or both.

This commit introduces the concept of modifier filter
options, which as of now are:

- `csp=`
- `queryprune=`

They both work in similar way when used with `important`
option or when used in exception filters. Modifier
options can apply to any network requests, hence the
logger reports the type of the network requests, and no
longer use the modifier as the type, i.e. `csp` filters
are no longer reported as requests of type `csp`.

Though modifier options can apply to any network requests,
for the time being the `csp=` modifier option still apply
only to top or embedded (frame) documents, just as before.
In some future we may want to apply `csp=` directives to
network requests of type script, to control the behavior
of service workers for example.

A new built-in filter expression has been added to the
logger: "modified", which allow to see all the network
requests which were modified before being emitted. The
translation work for this new option will be available
in a future commit.
2020-10-31 10:42:53 -04:00
Raymond Hill 3059db77c4
Use proxy detection code path only on Firefox 79 and less
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/911

This was fixed in Firefox 80:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1618271
2020-10-14 13:37:09 -04:00
Raymond Hill 03596439ca
Remove obsolete code to import content of localStorage
Months ago, usage of synchronous localStorage was replaced
with asynchronous extension storage. There was code for the
conversion to be seamless by importing the content of now
obsolete localStorage.

This code is no longer needed as majority of users are
assumed to use versions of uBO above 1.25.0.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899
2020-10-10 14:18:19 -04:00
Raymond Hill db0d200fcb
Allow call to removeCSS in Chromium
Related discussion:
- https://github.com/gorhill/uBlock/issues/3588#issuecomment-702218677
2020-10-02 12:16:47 -04:00
Raymond Hill 06ddc945cd
Add `fanboy.co.nz` as valid location for subscription links 2020-09-14 15:46:24 -04:00
Raymond Hill bd949ca295
Add `forums.lanik.us` as a valid location for subscription links
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1246
2020-09-14 15:37:53 -04:00
Raymond Hill fe9797cacc
Add easylist.to as a valid target for subscriber content script 2020-09-14 08:43:51 -04:00
Raymond Hill e60042595c
Revisit the behavior of the click-to-subscribe content script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/763

Changes:

From now on, uBO will allow click-to-subscribe on only
a few select domains, currently:
- https://filterlists.com/
- https://github.com/
- https://github.io/

More domains can be added if and only the demonstration
is made that more than a marginal number of filter lists
can be subscribed from those domains.

The browser alert box is no longer used to confirm
subscription to a filter list. Instead, the asset
viewer has been expanded to serve that purpose. This
way, users can peruse at the content of a filter list
before subscribing to it.
2020-09-13 08:01:53 -04:00
Raymond Hill 35aefed926
Add support to chain `:style()` to procedural operators
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/382

Additionally, remnant code for pseudo-user stylesheets
has been removed. Related commit:
- 5c68867b92
2020-09-07 08:28:01 -04:00
Raymond Hill d23f9c6a8b
Isolate element picker's svg layers from page content
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1226

Related commit:
- 9eb455ab5e

In the previous commit, the element picker dialog was
isolated from the page content. This commit is to also
isolate the svg layers from the page content.

With this commit, there is no longer a need for an anonymous
iframe and the isolated world iframe is now directly
embedded in the page.

As a result, pages are now unable to interfere with any
of the element picker user interface. Pages can now only
see an iframe, but are unable to see the content of that
iframe. The styles applied to the iframe are from a user
stylesheet, so as to ensure pages can't override the
iframe's style properties set by uBO.
2020-09-03 10:27:35 -04:00
Raymond Hill 9eb455ab5e
Isolate element picker dialog from page content world
Related issues:
- https://github.com/gorhill/uBlock/issues/3497
- https://github.com/uBlockOrigin/uBlock-issues/issues/1215

To solve above issues, the element picker's dialog is now
isolated from the page content in which it is embedded.

The highly interactive, mouse-driven part of the element
picker is still visible by the page content.
2020-09-01 12:32:12 -04:00
Raymond Hill 9ab631b4c4
Ensure port instance exists when waiting for connection requests 2020-09-01 09:53:35 -04:00
Raymond Hill 6284eca351
Fix ability to overwrite overquota'ed cloud storage
This commit fixes deleting all entries when cloud
storage usage is beyond allowed limit.

The issue would prevent pushing new data well within
quota limit because the old data beyond limit was
never removed in the first place.
2020-08-24 11:47:57 -04:00
Raymond Hill f6d1c6402f
Add support for removal of cloud storage entries 2020-08-21 09:18:33 -04:00
Raymond Hill d8b6b31eca
Add support for cloud storage compression
Cloud storage is a limited resource, and thus it
makes sense to support data compression before
sending the data to cloud storage.

A new hidden setting allows to toggle on
cloud storage compression:

name: cloudStorageCompression
default: false

By default, this hidden setting is `false`, and a
user must set it to `true` to enable compression
of cloud storage items.

This hidden setting will eventually be toggled
to `true` by default, when there is good confidence
a majority of users are using a version of uBO
which can properly handle compressed cloud storage
items.

A cursory assessment shows that compressed items
are roughly 40-50% smaller in size.
2020-08-16 11:59:27 -04:00
Raymond Hill 2afcc13ca6
Add widget to convey the amount of sync storage in use 2020-08-14 15:29:25 -04:00
Raymond Hill b01cc6ca4f
Fix cloud storage errors not reported in user interface
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/i6e7lr/
2020-08-10 08:30:52 -04:00
Raymond Hill 39190ff469
Add Chromium detection for Chromium-based MS Edge
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1174
2020-07-24 12:32:47 -04:00
Raymond Hill 5c68867b92
Deprecate pseudo user styles code
The pseudo user styles code served only browsers based
on Chromium 65 and earlier -- Chromium 66 supports
native user styles and was first released more than two
years ago.

In Chromium-based browsers, the pseudo user styles code
is being unconditionally injected in every page/frame
just in case the browser is version 65 or earlier.

Removing pseudo user styles reduce uBO's main content
script in Chromium-based browsers by more than 20K.

Related thread:
- https://github.com/NanoAdblocker/NanoCore/issues/348#issuecomment-653646507
2020-07-22 10:21:16 -04:00
Raymond Hill 221983646e
Injects user styles and content scripts in `about:blank` frames
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/688
- https://github.com/uBlockOrigin/uBlock-issues/issues/1164

`match_about_blank` is now used for content scripts and
user styles.
2020-07-19 09:06:19 -04:00
Raymond Hill feabfe3793
Do let grow subframe dictionary grow unbound
Related discussion:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1652925

It's not clear the code here will fix the reported
issue, but I did identify that the subframe
dictionary of a very long-lived web page can
theoretically grow unbound.
2020-07-18 07:44:26 -04:00
Raymond Hill a5ba668c38
Fix infinitely appending style property
Related issue:
- https://github.com/NanoAdblocker/NanoCore/issues/348
2020-07-03 15:07:52 -04:00
Raymond Hill 64cfc5779d
Use `default_public_interface_only` in Chromium-based browser
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/hbea3z/

It appears the implementation of the `disable_non_proxied_udp`
setting changed in Chromium, leading to WebRTC becoming
wholly unfunctional. Fall back to use `default_public_interface_only`
in Chromium-based browsers.
2020-06-18 11:30:05 -04:00
Raymond Hill eeae4b2cab
Add ability to open the dashboard with a keyboard shortcut
Related feedback:
- https://github.com/gorhill/uBlock/commit/9a6feb34d546#commitcomment-38852663
2020-04-30 10:31:23 -04:00
Raymond Hill b295d4a0d0
Make the new "fenix" popup panel the default one
The old "classic" popup panel will still be used
when at least one of the following is true:

- advanced setting `uiFlavor` is set to `classic`; or
- the browser is Chromium 65 or older; or
- the browser is Firefox 67 or older

The default configuration of the new popup panel
at installation time is to show the power button,
statistics and the basic tool icons, i.e. access
to dashboard, logger, pickers.

For existing installations, the new popup panel
will be configured by respecting the existing
configuration of the classic one.

The new popup panel is currently already in use
on Firefox for Android, and the visual redesign
was made according to suggestions and feedback
from <https://github.com/brampitoyo> to be
optimal for Firefox for Android.

The new popup panel will allow closing the following
pending issues:

- https://github.com/uBlockOrigin/uBlock-issues/issues/255
- https://github.com/uBlockOrigin/uBlock-issues/issues/178
2020-04-30 06:54:51 -04:00
Raymond Hill bdf9030a98
Fix hostname extraction when URL has port
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1001

The issue affected only the uncloaking of canonical
names, when the authority part of a URL contained
port information.
2020-04-27 18:30:49 -04:00
Raymond Hill 7f999b759f
Fix broken DOM inspector [regression]
Regression from:
- 0bcf04d3dd

Related feedback:
- https://github.com/gorhill/uBlock/commit/0bcf04d3dda2#commitcomment-38594585

The DOM inspector client code is injected as
content script, thus it must be categorized as
non-privileged code.
2020-04-19 08:33:09 -04:00
Raymond Hill 0bcf04d3dd
Centralize access to browser.storage.local('localStorage')
Related commit:
- 2ac288397c

Instead of having the `localStorage` data being accessed
from different locations, all accesses are now funnelled
to the main process.

Doing so simplifies the code in auxiliary processes and
also remove the need for browser.storage.local.onChanged()
listeners.

No longer using an onChanged() listener also happens to
remove spurious warnings from the Firefox console.
2020-04-15 15:55:29 -04:00
Raymond Hill 11d24abea0
Move proxy-detection code to Firefox-specific code
Related commit:
- https://github.com/uBlockOrigin/uBlock-issues/issues/911

The motivation is to avoid executing code which is
unnecessary on platforms not supporting the browser.dns
API.
2020-03-23 13:31:43 -04:00
Raymond Hill 3f7ece9469
Do not cname-uncloak when a proxy is in use
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/911

Since cname-uncloaking is available only on Firefox
at the moment, the fix is relevant only to Firefox.

By default uBO will no longer cname-uncloak when it
detects that network requests are being being proxied.

This default behavior can be overriden by setting the
new advanced setting `cnameUncloakProxied` to `true`.
The new setting default to `false`, i.e. cname-uncloaking
is disabled when uBO detects that a proxy is in use.

This new advanced setting may disappear once the
following Firefox issue is fixed:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1618271
2020-03-22 14:52:58 -04:00
Raymond Hill 1400b146ec
Add 32- & 64-pixel icon entries in manifest
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/939
2020-03-14 13:05:46 -04:00
Raymond Hill 72bb700568
Add procedural cosmetic operators remove() and upward()
***

New procedural cosmetic operator: `:remove()`

Related issue:
- https://github.com/gorhill/uBlock/issues/2252

The purpose is to outright remove elements from the
DOM tree. Since `:remove()` is an "action" operator,
it must only be used as a trailing operator (just
like the `:style()` operator).

AdGuard's cosmetic filter syntax `{ remove: true; }`
will be converted to uBO's `:remove()` operator
internally.

***

New procedural cosmetic operator: `:upward(...)`

The purpose is to lookup an ancestor element.

When used with an integer argument, it is synonym of
`:nth-ancestor()`, which will be deprecated and which
will no longer be supported once no longer used in
mainstream filter lists.

Filter lists maintainers must only use `:upward(int)`
instead of `:nth-ancestor(int)` once the new operator
become available in all stable releases of uBO.

`:upward()` can also accept a CSS selector as argument,
in which case the nearest ancestor which matches the
CSS selector will be selected.
2020-03-07 14:25:06 -05:00
Raymond Hill 08d370d32e
Catch possible exceptions when accessing window.localStorage
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899

This is a code review of previous commits related to
the above issue.
2020-03-03 11:31:14 -05:00
Raymond Hill a1dabf3c1a
Use `disable_non_proxied_udp` for `webRTCIPHandlingPolicy`
The stricter mode `disable_non_proxied_udp` is preferable
to `default_public_interface_only` to prevent local IP
address leakage through WebRTC.

This mode is properly supported since Firefox 70, so the
less strict `default_public_interface_only` will now be
used only for Firefox 69 and older.
2020-02-28 14:30:40 -05:00
Raymond Hill 953e0c3197
Use getItemAsync() instead of getItem
Related commit:
- da0ef9454a

vAPI.localStorage.getItemAsync() was added after
the above commit.
2020-02-25 07:37:12 -05:00
Raymond Hill c17a9c8a93
Fix spurious warning when force-reloading the dashboard
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/905

Additionally, minor changes to avoid spurious errors in
browser console.
2020-02-24 09:59:35 -05:00
Raymond Hill 2ac288397c
Remove usage of synchronous localStorage API
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899

browser.storage.local is now used to store non-critical
local settings.

These settings are all collated under the key
`localStorage`, and vAPI.localStorage is an API to
handle access to these values stored under this key.

vAPI.localStorage.getItem() is still synchronous but
its purpose is to return internally cached values --
this minimizes code changes throughout uBO.
2020-02-21 15:34:54 -05:00
Raymond Hill 4f75a07b9c
Work toward removing use of localStorage
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899
2020-02-21 07:48:45 -05:00
Raymond Hill da0ef9454a
Neutralize `localStorage` access on mobile platform
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899

window.localStorage is synchronous and thus should be
avoided at launch time. Currently the avoidance is only
for mobile platforms.
2020-02-20 16:40:29 -05:00
Raymond Hill 784a78ad00
Remove useless manifest entry for Opera build
Opera's dashboard warns that the `file:///` permission
is not needed.
2020-02-14 12:08:00 -05:00
Raymond Hill bc6e585b5d
Rephrase `author` entry in manifest 2020-01-30 12:46:54 -05:00
Raymond Hill 9637ab56a9
Insert icon badge text in icon title only if needed
This is to fix the rendering of the menu entry in
Firefox Fenix, which displays both the icon title
and badge text as a menu entry:

Before the fix:
    uBlock Origin (76)         76

After the fix:
    uBlock Origin              76

The badge text will still be part of the icon title
when the icon badge is disabled by the user or when
the platform does not supprt browserAction.setIcon()
(as is the case on current Firefox for Android).
2020-01-23 09:16:09 -05:00
Raymond Hill d0738c0835
Visually distinguish canonical names in popup panel
Further fine-tuning support for canonical names.

Aliased canonical names will be rendered blue
in the dynamic filtering pane of the popup panel.
2019-12-31 16:36:51 -05:00
Raymond Hill 832ebd360e
Mind that Firefox 59 and less does not support `browser.dns` 2019-12-08 11:08:04 -05:00
Raymond Hill d71f69bd90
Minor code review 2019-12-05 09:21:02 -05:00
Raymond Hill 91e702cebb
Enable CNAME uncloaking by default
Advanced setting `cnameAliasList` has been removed.

New advanced settings:

cnameUncloak:
  Boolean
Default value:
  true
Description:
  Whether to CNAME-uncloak hostnames.

cnameIgnoreExceptions:
  Boolean
Default value:
  true
Description:
  Whether to bypass the uncloaking of network requests
  which were excepted by filters/rules. This is
  necessary so as to avoid undue breakage by having
  exception filters being rendered useless as a result
  of CNAME-uncloaking.
  For example, `google-analytics.com` uncloaks to
  `www-google-analytics.l.google.com` and both hostnames
  appear in Peter Lowe's list, which means exception
  filters for `google-analytics.com` (to fix site
  breakage) would be rendered useless as the uncloaking
  would cause the network request to be ultimately
  blocked.
2019-12-01 12:05:49 -05:00
Raymond Hill a16e4161de
Fine tune hostname uncloaking through CNAME-lookup
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/780

Related commit:
- https://github.com/gorhill/uBlock/commit/3a564c199260

This adds two new advanced settings:

- cnameIgnoreRootDocument
  - Default to `true`
  - Tells uBO to skip CNAME-lookup for root document.

- cnameReplayFullURL
  - Default to `false`
  - Tells uBO whether to replay the whole URL or just
    the origin part of it.
    Replaying only the origin part is meant to lower
    undue breakage and improve performance by avoiding
    repeating the pattern-matching of the whole URL --
    which pattern-matching was most likely already
    accomplished with the original request.

This commit is meant to explore enabling CNAME-lookup
by default for the next stable release while:

- Eliminating a development burden by removing the
  need to create a new filtering syntax to deal with
  undesirable CNAME-cloaked hostnames

- Eliminating a filter list maintainer burden by
  removing the need to find/deal with all base
  domains which engage in undesirable CNAME-cloaked
  hostnames

The hope is that the approach implemented in this
commit should require at most a few unbreak rules
with no further need for special filtering syntax
or filter list maintance efforts.
2019-11-23 13:07:23 -05:00
Raymond Hill 3463a60e6b
Fix incorrect resolved value in promise
This fixes the ability to block when a hostname
had to be cname-resolved the first time it was
encountered. The result being cached allowed
the subsequent requests to be correctly blockable.
2019-11-21 12:04:19 -05:00
Raymond Hill 563aed0d93
Code review for dns lookup code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/780

- Handle DNS lookup failure

- Skip DNS lookup for non network-based URLs

- Benchmark code to be able to provide an estimate
  based on objective measurements regarding added
  overhead when DNS lookup is enabled
  (quick answer: a complete non-issue)
2019-11-20 10:45:17 -05:00
Raymond Hill 02075ab97a
Test for existence of browser.dns
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/780

browser.dns is not available before Firefox 60,
test for presence.
2019-11-19 16:48:53 -05:00
Raymond Hill 3a564c1992
Add ability to uncloak CNAME records
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/780

New webext permission added: `dns`, which purpose is
to allow an extension to fetch the DNS record of
specific hostnames, reference documentation:

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/dns

The webext API `dns` is available in Firefox 60+ only.

The new API will enable uBO to "uncloak" the actual
hostname used in network requests. The ability is
currently disabled by default for now -- this is only
a first commit related to the above issue to allow
advanced users to immediately use the new ability.

Four advanced settings have been created to control the
uncloaking of actual hostnames:

cnameAliasList: a space-separated list of hostnames.
Default value: unset => empty list.
Special value: * => all hostnames.
A space-separated list of hostnames => this tells uBO
to "uncloak" the  hostnames in the list will.

cnameIgnoreList: a space-separated list of hostnames.
Default value: unset => empty list.
Special value: * => all hostnames.
A space-separated list of hostnames => this tells uBO
to NOT re-run the network request through uBO's
filtering engine with the CNAME hostname. This is
useful to exclude commonly used actual hostnames
from being re-run through uBO's filtering engine, so
as to avoid pointless overhead.

cnameIgnore1stParty: boolean.
Default value: true.
Whether uBO should ignore to re-run a network request
through the filtering engine when the CNAME hostname
is 1st-party to the alias hostname.

cnameMaxTTL: number of minutes.
Default value: 120.
This tells uBO to clear its CNAME cache after the
specified time. For efficiency purpose, uBO will
cache alias=>CNAME associations for reuse so as
to reduce calls to `browser.dns.resolve`. All the
associations will be cleared after the specified time
to ensure the map does not grow too large and too
ensure uBO uses up to date CNAME information.
2019-11-19 12:05:33 -05:00
Raymond Hill e2fdc1b94b
Support keyboard shortcut to open popup panel
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/dt47s0/
2019-11-09 09:53:31 -05:00
Raymond Hill a69b301d81
Fine-tune new bidi-trie code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761
2019-10-29 10:26:34 -04:00
Raymond Hill b79445320d
Force-unsuspend when starting webRequest listener
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/759

If for some reasons the launch code throws, uBO could
find itself in permanent suspend mode, thus preventing
web pages from loading in Firefox.

Though uBO should not have exceptions thrown during
it's initialization code, this commit will ensure
uBO complete its initialization process should it
ever happen for whatever reason.
2019-10-27 11:41:08 -04:00
Raymond Hill 35cb0eb377
Do not bypass network listener in suspended mode
Tabless network requests were bypassing uBO's
onBeforeRequest's listener when in suspended
mode. Suspend mode occurs during the time the
filter lists are all reloaded.

Regression from:
- 1dfdc40e09 (diff-d04c15ee6bff6a6269c6aee25a7c7522R1122)
2019-10-07 08:13:37 -04:00
Raymond Hill 41ab06a47b
Fix regression in cosmetic filtering
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/740

Related commit:
- 3f074d7908
2019-09-30 10:40:56 -04:00
Raymond Hill 3f074d7908
Use `self` when testing for `vAPI` presence 2019-09-30 07:50:35 -04:00
Raymond Hill efa4ff3bcf
Code review re. dynamically loaded vapi-client-extra.js
Related commit:
- 87d0e456f1

Ensure that the code which depends on extending
`vapi-client.js` is ready to deal with
`vapi-client-extra.js` failing to load.
2019-09-26 15:57:55 -04:00
Raymond Hill 733b2330de
Prevent spurious error messages to browser console
These spurious error messages can occur when a
extension framework API method return a
rejected Promise. In uBO the results of
browserAction methods is not used so it can be
safely discarded.
2019-09-23 09:29:17 -04:00
Raymond Hill f204d24bf4
Match static popup filter against local context
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/d6zbqv/

For static filter `popup` filter purpose, the URL of the
embedded frame from which the popup was launched will
be used in the matching algorithm.
2019-09-23 08:25:23 -04:00
Raymond Hill d15163d3bb
Clarify that the code base was really old and obsolete 2019-09-22 09:57:38 -04:00
Raymond Hill 010635acd6
Add support for `ping` static filter option
Related issue:
- https://github.com/gorhill/uBlock/issues/1493

Documentation:
- https://help.eyeo.com/adblockplus/how-to-write-filters#type-options

Test page:
- https://testpages.adblockplus.org/en/filters/ping

Additionally, network requests of type `beacon` will
be mapped to `ping` by the static filtering engine.
2019-09-22 09:11:55 -04:00
Raymond Hill 350e436c08
Remove remnants of `chrome` references...
... from platform-independent code.

This should complete the de-chromiumification of
uBO. Next step will be to swap the content of the
`platform/chromium` and `platform/firefox`
folders so as to reflect that Firefox is the
natural platform for uBO.
2019-09-21 12:06:57 -04:00
Raymond Hill 0c6407c082
Use `webext.menus`, not `browser.contextMenus` (oops) 2019-09-20 07:59:21 -04:00
Raymond Hill 4f89a6f89d
Fix missing "Block element" in Firefox's menu
Regression from:
- 58620fb051
2019-09-20 07:51:47 -04:00
Raymond Hill 1d93d6f64e
Fix typo
Related feedback:
- 149b5cf59c (r35158554)
2019-09-19 17:19:29 -04:00
Raymond Hill 58620fb051
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await.

Related commits:
- 022951547c
- 3224d9b5cc
- 26235d80d0
- 0051f3b5c7
- eec53c0154
- 915687fddb
- 55cc0c6997
- e27328f931
2019-09-19 16:41:44 -04:00
Raymond Hill 5f60c31f77
Fix copyright dates as per commit history
vapi.js
-https://github.com/gorhill/uBlock/commits/master/platform/chromium/vapi.js

vapi-common-js
- https://github.com/gorhill/uBlock/commits/master/platform/chromium/vapi-common.js
- 266f62914f/meta/crx/vapi-common.js
- 8444923c0c/src/js/vapi-common.js

vapi-background.js
- https://github.com/gorhill/uBlock/commits/master/platform/chromium/vapi-background.js
- 266f62914f/meta/crx/vapi-background.js
- 8444923c0c/src/js/vapi-background.js

vapi-client.js
- https://github.com/gorhill/uBlock/commits/master/platform/chromium/vapi-client.js
- 266f62914f/meta/crx/vapi-client.js
- 8444923c0c/src/js/vapi-client.js
2019-09-19 11:04:12 -04:00
Raymond Hill 60aa80b571
Fix erroneous copyright date
As per commit history:
266f62914f/meta/crx/vapi-client.js
2019-09-19 09:12:51 -04:00
Raymond Hill a45a962517
Fix property name inconsistency
Addtionally, fixed erroneous copyright date, as per
commit history:
https://github.com/gorhill/uBlock/commits/master/platform/chromium/vapi-client.js
2019-09-19 09:07:55 -04:00
Raymond Hill 87d0e456f1
Simplify client messaging code
Little-used code from vapi-client.js has been moved
to vapi-client-extra.js. Given that vapi-client.js
is injected in all web pages, this means less dead
code being injected in all pages.

Swathes of code in vapi-client.js was used only in
a few very specific cases, such as when the logger's
DOM inspector is opened or when the "Filter lists"
pane in the dashboard is opened -- and thus to avoid
that little used code to be loaded in every web page
unconditionally, it has been moved to its own
separate file, vapi-client.extra.js.

vapi-client-extra.js is loaded declaratively or
programmatically only where needed.
2019-09-19 08:31:38 -04:00
Raymond Hill 149b5cf59c
Reomving now obsolete Safari code base
See README.md file:
https://github.com/gorhill/uBlock/tree/master/platform/safari/README.md
2019-09-19 08:19:39 -04:00
Raymond Hill 022951547c
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await.

Related commits:
- 3224d9b5cc
- 26235d80d0
- 0051f3b5c7
- eec53c0154
- 915687fddb
- 55cc0c6997
- e27328f931
2019-09-18 08:34:55 -04:00
Raymond Hill ed99d52bc4
Simplify slightly messaging code
- No need to hold onto channel name and message data while
  waiting for response to resolve
- Use more representative `msgId` instead of `auxProcessId`
2019-09-17 17:43:52 -04:00
Raymond Hill 3224d9b5cc
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await.

Related commits:
- 26235d80d0
- 0051f3b5c7
- eec53c0154
- 915687fddb
- 55cc0c6997
- e27328f931
2019-09-17 15:15:01 -04:00
Raymond Hill 0051f3b5c7
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await. More left to do.

Related commits:
- eec53c0154
- 915687fddb
- 55cc0c6997
- e27328f931
2019-09-16 16:17:48 -04:00
Raymond Hill eec53c0154
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await. More left to do.

Related commits:
- 915687fddb
- 55cc0c6997
- e27328f931
2019-09-16 09:45:17 -04:00
Raymond Hill e27328f931
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await. More left to do.

In the process, a regression affecting the fix to
<https://github.com/uBlockOrigin/uBlock-issues/issues/682>
has been fixed.
2019-09-15 07:58:28 -04:00
Raymond Hill e393a52442
Fix icon title always showing `(0)` when badge is disabled
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/d33d37/
2019-09-12 12:26:09 -04:00
Raymond Hill 93f438f55e
Add advanced setting for extension reload on update
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/717

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/717#issuecomment-530275655

New advanced setting: `extensionUpdateForceReload`

Default value: `false`

If set to `true`, the extension will unconditionally reload
when an update is available; otherwise the extension will
reload only when being explicitly disabled then enabled, or
when the browser is restarted.
2019-09-11 08:00:55 -04:00
Raymond Hill c8c2e11d09
Prevent reload if updated version is higher than current
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/717

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/717#issuecomment-528921730
2019-09-06 13:03:06 -04:00
Raymond Hill bcf5ac1fee
Add advanced setting to control logger popup type
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/663

The advanced setting `loggerPopupType` has been added, to
control the type of window to be used when the logger is
launched as a separate window.

The default value is `popup`, it can be changed to any of
the values documented at:

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/windows/CreateType
2019-09-06 11:41:07 -04:00
Raymond Hill 5688888ec1
Remove pointless local scoping 2019-09-06 10:38:17 -04:00
Raymond Hill 59bdf2b4cc
Prevent uBO from being reloaded mid-session
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/717

Just registering a onUpdateAvailable() listener prevents
the browser from reloading the extension mid-session.

Ref:
- https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/onUpdateAvailable
2019-09-04 10:24:24 -04:00
Raymond Hill ca9b29c7ec
Distinguish between priviledge and unprivileged messages
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/710

Messages from unprivileged ports (i.e. from content scripts)
are no longer relayed to message handlers which are to be
strictly used to execute privileged code.

The last remaining case of unprivileged messages which
should be converted into a privileged ones will be taken
care of when the following issue is fixed:
- https://github.com/gorhill/uBlock/issues/3497
2019-09-01 12:43:12 -04:00
Raymond Hill 7c7be6c898
Add "Relax blocking mode" command to Opera manifest 2019-08-18 08:39:45 -04:00
Raymond Hill 7ff750eaf6
Reflect blocking mode in badge color of toolbar icon
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/cmh910/

Additionally, the `3p` rule has been made distinct from
`3p-script`/`3p-frame` for the purpose of
"Relax blocking mode" command.

The badge color will hint at the current blocking mode.
There are four colors for the four following blocking
modes:
- JavaScript wholly disabled
- All 3rd parties blocked
- 3rd-party scripts and frames blocked
- None of the above

The default badge color will be used when JavaScript is not
wholly disabled and when there are no rules for `3p`,
`3p-script` or `3p-frame`.

A new advanced setting has been added to let the user choose
the badge colors for the various blocking modes,
`blockingProfileColors`. The value *must* be a sequence of
4 valid CSS color values that match 6 hexadecimal digits
prefixed with`#` -- anything else will be ignored.
2019-08-10 10:57:24 -04:00
Raymond Hill aaee898d95
Ensure icon & context menu are up to date on window focus change
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/151

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/680#issuecomment-515215220
2019-07-26 08:53:20 -04:00
Raymond Hill 066440534d
Prevent popup panel to close when forcing a tab reload
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/672

Additionally, this commit add the ability to press F5 to
force a reload while the popup panel is opened.
2019-07-21 11:50:15 -04:00
Raymond Hill e1dd7f7043
Ensure toolbar icon reflect updated whitelist directives
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/680

Opportunistically, vAPI.tabs has been refactored toward
ES6 syntax.
2019-07-21 10:48:44 -04:00
Raymond Hill 7e1868b1c3
Add virtuous signal that remote code execution is not possible
Related Chromium issue:
- https://bugs.chromium.org/p/chromium/issues/detail?id=985759

By expressly restricting JavaScript execution to only code
from the extension package, this explicitly tells code reviewer
that uBO can't execute remote code.

I also had to add `object-src 'self'`, otherwise Chromium
refused to load the extension with the following error message:

> 'content_security_policy': CSP directive 'object-src' must be specified

`object-src 'self'` is the default value.
2019-07-21 10:37:58 -04:00
Raymond Hill 730a83377e
Minor code review re. context menu code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/151

I have been unsuccessful fixing the above issue, but I will
keep the changes made in the process of trying to fix it.
2019-07-02 09:43:26 -04:00
Raymond Hill 1dfdc40e09
Add ability to suspend network request handler at will
This works only for platforms supporting the return of
Promise by network listeners, i.e. only Firefox at this
point.

When filter lists are reloaded[1], there is a small
time window in which some network requests which should
have normally been blocked are not being blocked
because the static network filtering engine may not
have yet loaded all the filters in memory

This is now addressed by suspending the network request
handler when filter lists are reloaded -- again, this
works only on supported platforms.

[1] Examples: when a filter list update session
    completes; when user filters change, when
    adding/removing filter lists.
2019-06-30 10:09:27 -04:00
Raymond Hill 6c34b3c3c9
Use "relax" instead of "toggle"
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/371
2019-06-27 08:16:18 -04:00
Raymond Hill a9659d039d
Add missing command in manifest.json 2019-06-26 17:17:41 -04:00
Raymond Hill 693687fd74
Add keyboard support for toggling down blocking profile
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/371

By default, no specific keyboard shortcut is predefined,
this will have to be assigned by the user. The command
name in English is "Toggle blocking profile".

The default behavior is to toggle down according to one
of the following scenarios.

a) If script execution is disabled through the no-scripting
switch, the no-scripting switch will be locally toggled
so as to allow script execution. The page will be
automatically reloaded.

b) If script execution is not blocked but the 3rd-party
script and/or frame cells are blocked, local no-op rules
will be set so as to no longer block 3rd-party scripts
and/or frames. The page will be automatically reloaded.

Given this, it may take more than one toggle down command
to reach the lowest blocking profile, which is one where
JavaScript execution is not blocked and 3rd-party scripts
and frames resources block rules, if any, are bypassed
with local no-op rules.

TODO: At this point, I haven't yet decided whether
toggling from the lowest profile should restore the
original highest blocking profile.
2019-06-26 07:47:14 -04:00
Raymond Hill 793aca7ddb
Fix internal cosmetic filter being reported in the logger
Regression from:
- 62387fb87a

Repro steps were:
- Open the logger
- Navigate to `https://news.ycombinator.com/`
- Select an element using the element picker
- Click "Preview"

An attribute selector used internally by uBO to
hide targeted nodes was being reported in the
logger.
2019-06-20 07:14:34 -04:00
Raymond Hill 184e4f7a42
Avoid reloading the dashboard tab if it is already present
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/592
2019-05-24 20:15:40 -04:00
Raymond Hill eed13194fb
Handle possible exceptions in vAPI.messaging.broadcast()
I had exceptions thrown by that code once in a while,
about the `port` object not being connected; possibly
occurs only when using dev tools with breakpoints
in uBO's code. Even if this can be reproduced randomly
only when debugging uBO, it costs nothing to add code
in there to handle exceptions.
2019-05-21 14:07:38 -04:00
Raymond Hill 1386429382
Fix regression in applying procedural cosmetic filters
Related commit:
- 3573b6b32c
2019-05-16 17:22:20 -04:00