Commit Graph

7332 Commits

Author SHA1 Message Date
Raymond Hill b79445320d
Force-unsuspend when starting webRequest listener
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/759

If for some reasons the launch code throws, uBO could
find itself in permanent suspend mode, thus preventing
web pages from loading in Firefox.

Though uBO should not have exceptions thrown during
it's initialization code, this commit will ensure
uBO complete its initialization process should it
ever happen for whatever reason.
2019-10-27 11:41:08 -04:00
Raymond Hill 36fb671460
Make Firefox dev build auto-update 2019-10-27 08:43:52 -04:00
Raymond Hill 2fafe8081c
New revision for dev build 2019-10-27 08:40:49 -04:00
Raymond Hill 40de7d6489
Fix WASM memory allocation in bidi-trie
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/764
2019-10-27 08:36:17 -04:00
Raymond Hill 92235adeec
Make Firefox dev build auto-update 2019-10-26 17:42:58 -04:00
Raymond Hill cdb6697e4c
new revision for dev build 2019-10-26 17:40:27 -04:00
Raymond Hill c71624d1da
Fix access to detached buffer when using WASM in bidi-trie
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/764

WebAssembly.Memory.grow() preserves the buffer content when
we grow it, no need to manually copy it. Doing so was
causing an access to a no longer valid ArrayBuffer.
2019-10-26 17:37:47 -04:00
Raymond Hill 0373410635
Fix comments in WASM code 2019-10-26 15:34:40 -04:00
Raymond Hill fb457a3dff
Make Firefox dev build auto-update 2019-10-26 15:31:34 -04:00
Raymond Hill 1f1933981d
New revision for dev build 2019-10-26 15:28:48 -04:00
Raymond Hill 6c3296958c
Fix last commit due to bad last second change
Related feedback:
- b0cbc47d9a (commitcomment-35677572)

It seems I completely forgot to test the last
"trivial" change to the WASM code.
2019-10-26 15:25:47 -04:00
Raymond Hill da8f9fa37c
Make Firefox dev build auto-update 2019-10-26 13:26:03 -04:00
Raymond Hill 1df432c2e5
Import translation work from https://crowdin.com/project/ublock 2019-10-26 13:23:31 -04:00
Raymond Hill b0cbc47d9a
Add WASM versions for some bidi-trie methods
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761

Changes related to above issue made it possible to
create WASM versions of methods used in the bidi-trie.
In this commit, WASM versions for startsWith(), indexOf()
and lastIndexOf() have been implemented.
2019-10-26 13:13:53 -04:00
Raymond Hill dd2a9faa4c
Use request as its own context when none available
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/dn9zso/

When a network request is tabless and contextless, i.e.
`tabId === -1` and `frameId === -1`, use the URL of
the network request as the context.
2019-10-26 10:18:32 -04:00
Raymond Hill 30393fdcf1
Exclude data type (i.e. `csp=`) from bidi-trie
We need a `matchAll()` method on the bidi-trie before
we can store filters of type `data` in it.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761

Related commit:
- 7971b22385
2019-10-22 18:14:49 -04:00
Raymond Hill 2681209828
New revision for dev build 2019-10-21 11:59:51 -04:00
Raymond Hill c137630833
Merge branch 'snfe-refactor' 2019-10-21 11:59:02 -04:00
Raymond Hill a5601b849e
Import translation work from https://crowdin.com/project/ublock 2019-10-21 10:42:52 -04:00
Raymond Hill 29502a89f9
New revision for stable release 2019-10-21 09:12:20 -04:00
Raymond Hill 7971b22385
Expand bidi-trie usage in static network filtering engine
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761
- https://github.com/uBlockOrigin/uBlock-issues/issues/528

The previous bidi-trie code could only hold filters which
are plain pattern, i.e. no wildcard characters, and which
had no origin option (`domain=`), right and/or left anchor,
and no `csp=` option.

Example of filters that could be moved into a bidi-trie
data structure:

    &ad_box_
    /w/d/capu.php?z=$script,third-party
    ||liveonlinetv247.com/images/muvixx-150x50-watch-now-in-hd-play-btn.gif

Examples of filters that could NOT be moved to a bidi-trie:

    -adap.$domain=~l-adap.org
    /tsc.php?*&ses=
    ||ibsrv.net/*forumsponsor$domain=[...]
    @@||imgspice.com/jquery.cookie.js|$script
    ||view.atdmt.com^*/iview/$third-party
    ||postimg.cc/image/$csp=[...]

Ideally the filters above should be able to be moved to a
bidi-trie since they are basically plain patterns, or at
least partially moved to a bidi-trie when there is only a
single wildcard (i.e. made of two plain patterns).

Also, there were two distinct bidi-tries in which
plain-pattern filters can be moved to: one for patterns
without hostname anchoring and another one for patterns
with hostname-anchoring. This was required because the
hostname-anchored patterns have an extra condition which
is outside the bidi-trie knowledge.

This commit expands the number of filters which can be
stored in the bidi-trie, and also remove the need to
use two distinct bidi-tries.

- Added ability to associate a pattern with an integer
  in the bidi-trie [1].
    - The bidi-trie match code passes this externally
      provided integer when calling an externally
      provided method used for testing extra conditions
      that may be present for a plain pattern found to
      be matching in the bidi-trie.

- Decomposed existing filters into smaller logical units:
    - FilterPlainLeftAnchored =>
        FilterPatternPlain +
        FilterAnchorLeft
    - FilterPlainRightAnchored =>
        FilterPatternPlain +
        FilterAnchorRight
    - FilterExactMatch =>
        FilterPatternPlain +
        FilterAnchorLeft +
        FilterAnchorRight
    - FilterPlainHnAnchored =>
        FilterPatternPlain +
        FilterAnchorHn
    - FilterWildcard1 =>
        FilterPatternPlain + [
          FilterPatternLeft or
          FilterPatternRight
        ]
    - FilterWildcard1HnAnchored =>
        FilterPatternPlain + [
          FilterPatternLeft or
          FilterPatternRight
        ] +
        FilterAnchorHn
    - FilterGenericHnAnchored =>
        FilterPatternGeneric +
        FilterAnchorHn
    - FilterGenericHnAndRightAnchored =>
        FilterPatternGeneric +
        FilterAnchorRight +
        FilterAnchorHn
    - FilterOriginMixedSet =>
        FilterOriginMissSet +
        FilterOriginHitSet
    - Instances of FilterOrigin[...], FilterDataHolder
      can also be added to a composite filter to
      represent `domain=` and `csp=` options.

- Added a new filter class, FilterComposite, for
  filters which are a combination of two or more
  logical units. A FilterComposite instance is a
  match when *all* filters composing it are a
  match.

Since filters are now encoded into combination of
smaller units, it becomes possible to extract the
FilterPatternPlain component and store it in the
bidi-trie, and use the integer as a handle for the
remaining extra conditions, if any.

Since a single pattern in the bidi-trie may be a
component for different filters, the associated
integer points to a sequence of extra conditions,
and a match occurs as soon as one of the extra
conditions (which may itself be a sequence of
conditions) is fulfilled.

Decomposing filters which are currently single
instance into sequences of smaller logical filters
means increasing the storage and CPU overhead when
evaluating such filters. The CPU overhead is
compensated by the fact that more filters can now
moved into the bidi-trie, where the first match is
efficiently evaluated. The extra conditions have to
be evaluated if and only if there is a match in the
bidi-trie.

The storage overhead is compensated by the
bidi-trie's intrinsic nature of merging similar
patterns.

Furthermore, the storage overhead is reduced by no
longer using JavaScript array to store collection
of filters (which is what FilterComposite is):
the same technique used in [2] is imported to store
sequences of filters.

A sequence of filters is a sequence of integer pairs
where the first integer is an index to an actual
filter instance stored in a global array of filters
(`filterUnits`), while the second integer is an index
to the next pair in the sequence -- which means all
sequences of filters are encoded in one single array
of integers (`filterSequences` => Uint32Array). As
a result, a sequence of filters can be represented by
one single integer -- an index to the first pair --
regardless of the number of filters in the sequence.

This representation is further leveraged to replace
the use of JavaScript array in FilterBucket [3],
which used a JavaScript array to store collection
of filters. Doing so means there is no more need for
FilterPair [4], which purpose was to be a lightweight
representation when there was only two filters in a
collection.

As a result of the above changes, the map of `token`
(integer)  => filter instance (object) used to
associate tokens to filters or collections of filters
is replaced with a more efficient map of `token`
(integer) to filter unit index (integer) to lookup a
filter object from the global `filterUnits` array.

Another consequence of using one single global
array to store all filter instances means we can reuse
existing instances when a logical filter instance is
parameter-less, which is the case for FilterAnchorLeft,
FilterAnchorRight, FilterAnchorHn, the index to these
single instances is reused where needed.

`urlTokenizer` now stores the character codes of the
scanned URL into a bidi-trie buffer, for reuse when
string matching methods are called.

New method: `tokenHistogram()`, used to generate
histograms of occurrences of token extracted from URLs
in built-in benchmark. The top results of the "miss"
histogram are used as "bad tokens", i.e. tokens to
avoid if possible when compiling filter lists.

All plain pattern strings are now stored in the
bidi-trie memory buffer, regardless of whether they
will be used in the trie proper or not.

Three methods have been added to the bidi-trie to test
stored string against the URL which is also stored in
then bidi-trie.

FilterParser is now instanciated on demand and
released when no longer used.

***

[1] 135a45a878/src/js/strie.js (L120)
[2] e94024d350
[3] 135a45a878/src/js/static-net-filtering.js (L1630)
[4] 135a45a878/src/js/static-net-filtering.js (L1566)
2019-10-21 08:15:58 -04:00
Raymond Hill 9f069850e5
Make Firefox dev build auto-update 2019-10-17 17:30:02 -04:00
Raymond Hill 1352c30b1c
New revision for release candidate 2019-10-17 17:26:41 -04:00
Raymond Hill f2340bef3c
Fix bad returned value in case of empty URL
Though I do no expect the empty URL case
to ever occur, having the tokenizer return
the wrong value if it ever occur could cause
uBO to malfunction.
2019-10-17 17:23:05 -04:00
Raymond Hill e417c9237e
Make Firefox dev build auto-update 2019-10-15 11:57:35 -04:00
Raymond Hill 6975461f3c
Import translation work from https://crowdin.com/project/ublock 2019-10-15 11:53:44 -04:00
Raymond Hill 5686d09940
New revision for release candidate 2019-10-15 11:50:21 -04:00
Raymond Hill 0f19dfde38
Avoid or defer writing back to cache storage at launch
The readTime property is unused for compiled or selfie
assets, and thus writing back to storage to persist
this property is useless for those assets, and an undue
overhead when reading such assets, especially at launch
time.

Assets are always loaded from their compiled or selfie
counterparts at launch.

This commit will prevent those unnecessary storage
write operations.
2019-10-15 11:42:43 -04:00
Raymond Hill 5a85ff9a93
Make Firefox dev build auto-update 2019-10-14 09:08:14 -04:00
Raymond Hill a521eed4c7
New revision for release candidate 2019-10-14 09:05:15 -04:00
Raymond Hill f117c280d0
Fix minor bugs spotted during code review 2019-10-14 09:03:51 -04:00
Raymond Hill 389d8458e0
Make Firefox dev build auto-update 2019-10-11 18:10:50 -04:00
Raymond Hill 8245ba2ac2
New revision for release candidate 2019-10-11 18:07:14 -04:00
Raymond Hill 7459a3f369
Fix thunderbird build script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/753
2019-10-11 18:01:05 -04:00
Raymond Hill f4c25f0e74
Merge branch 'master' of github.com:gorhill/uBlock 2019-10-11 10:54:22 -04:00
Raymond Hill 8c6a08722f
Remove "RUS: AdGuard Russian" from stock filter list
As per feedback from maintainers, AdGuard Russian
and RU AdList are incompatible and web site breakage
can occur when both are used together.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/749
2019-10-11 10:50:25 -04:00
Dylan Sharhon 288ce6c601 Add heads up for Safari 13+ users (#3758)
See https://github.com/uBlockOrigin/uBlock-issues/issues/747
2019-10-08 14:43:09 -04:00
Raymond Hill 8a8edeab1f
Make Firefox dev build auto-update 2019-10-07 09:55:13 -04:00
Raymond Hill 5915c7044c
New revision for release candidate 2019-10-07 08:30:28 -04:00
Raymond Hill 79b3b1921e
Import translation work from https://crowdin.com/project/ublock 2019-10-07 08:28:56 -04:00
Raymond Hill 35cb0eb377
Do not bypass network listener in suspended mode
Tabless network requests were bypassing uBO's
onBeforeRequest's listener when in suspended
mode. Suspend mode occurs during the time the
filter lists are all reloaded.

Regression from:
- 1dfdc40e09 (diff-d04c15ee6bff6a6269c6aee25a7c7522R1122)
2019-10-07 08:13:37 -04:00
Raymond Hill eb1ccec242
Make Firefox dev build auto-update 2019-10-03 12:49:06 -04:00
Raymond Hill 0733f6c476
New revision for release candidate 2019-10-03 12:45:29 -04:00
Raymond Hill 01a4060d2c
Import translation work from https://crowdin.com/project/ublock 2019-10-03 12:44:45 -04:00
Raymond Hill 5a5523c0b5
Remove stats button from logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/741

The feature will be released in the future when
it works as I intend for it to work rather than
being a featureless bare-bone version.
2019-10-03 12:32:27 -04:00
Raymond Hill fcfa83dfc6
Make Firefox dev build auto-update 2019-10-01 09:14:41 -04:00
Raymond Hill 806364f2da
New revision for release candidate 2019-10-01 09:03:44 -04:00
Raymond Hill 520762f26a
New revision for dev build 2019-10-01 09:02:02 -04:00
Raymond Hill bf697f344a
Log procedural cosmetic exception filters
Related issue:
- https://github.com/gorhill/uBlock/issues/127

Procedural cosmetic exception filters were the
last class of cosmetic exception filters not
being reported in the logger; this commit fixes
this.

Additionally, ensure that a single DOM listener
can't prevent other listeners from being
processed by throwing an exception. Such approach
would have prevented regression leading to
emergency release 1.22.4:
- https://github.com/gorhill/uBlock/releases/tag/1.22.4
2019-09-30 18:21:24 -04:00
Raymond Hill 95469032a4
Make Firefox dev build auto-update 2019-09-30 11:46:38 -04:00