Commit Graph

299 Commits

Author SHA1 Message Date
Raymond Hill e112f6e3bd
Publish tgz package instead of zip
Related feedback:
- https://github.com/gorhill/uBlock/pull/3812#issuecomment-897057648
2021-08-12 08:33:43 -04:00
Manish Jethani 089c28861c
Use npm pack (#3812) 2021-08-12 07:57:56 -04:00
Manish Jethani ab9f5ae100
Run build script via tools/make-nodejs.sh (#3808) 2021-08-09 20:02:58 -04:00
Raymond Hill 5eb99a4dd6
... 2021-08-05 14:36:13 -04:00
Raymond Hill fe7ce8f576
Need to bump version to publish README 2021-08-05 14:32:07 -04:00
Manish Jethani 4ea0d134ad
Move linting out of tools/make-nodejs.sh (#3803) 2021-08-04 14:40:20 -04:00
Raymond Hill 314d6082f2
Fix broken GitHub actions
Related discussion:
- ad69c760fb (commitcomment-54363709)
2021-08-03 12:47:26 -04:00
Raymond Hill 89c5653bc6
Export the rule-based filtering engines to the nodejs package
The code exported to nodejs package was revised to use modern
JavaScript syntax. A few issues were fixed at the same time.

The exported classes are:
- DynamicHostRuleFiltering
- DynamicURLRuleFiltering
- DynamicSwitchRuleFiltering

These related to the content the of "My rules" pane in the
uBlock Origin extension.
2021-08-03 12:19:25 -04:00
Manish Jethani 6b85b7f2b1
Create build folder in npm install script (#3799) 2021-08-03 07:02:48 -04:00
Manish Jethani ad69c760fb
Run ESLint during Node.js package generation (#3798) 2021-08-02 16:55:03 -04:00
Raymond Hill baff4a2feb
Fix script aborting when rm fails 2021-08-02 10:24:24 -04:00
Manish Jethani 0b764934e0
Serialize PSL in Node.js install script (#3793) 2021-08-01 16:42:05 -04:00
Raymond Hill c6d275674d
Revert "Make the nodejs package load a serialized version of the PSL"
This reverts commit 46c6ff8708.
2021-08-01 15:42:46 -04:00
Raymond Hill 46c6ff8708
Make the nodejs package load a serialized version of the PSL
Related discussion:
- https://github.com/cliqz-oss/adblocker/pull/2091#issuecomment-890545926
2021-08-01 12:58:21 -04:00
Manish Jethani be68a6b5b4
Use set -e in Bash scripts (#3791) 2021-07-31 17:34:25 -04:00
Raymond Hill 6fa8ae619d
Revert "Pass -e flag to bash (#3790)"
This reverts commit bb20159495.
2021-07-31 14:49:32 -04:00
Manish Jethani bb20159495
Pass -e flag to bash (#3790) 2021-07-31 12:15:32 -04:00
Raymond Hill de1feb0948
Fix make-nodejs
Broken due to renaming text-iterators to text-utils.
2021-07-31 08:59:55 -04:00
Raymond Hill d5da88ea87
Simplify loading of WASM modules in nodejs 2021-07-31 07:06:11 -04:00
Raymond Hill 98fc66bb1b
Add support for enabling WASM code paths in NodeJS package
See `test.js` for reference on how to enable WASM code
paths (which are disabled by default).
2021-07-29 16:54:51 -04:00
Raymond Hill b02970f5ee
Add command to update submodules 2021-07-29 11:46:35 -04:00
Manish Jethani 63591ef2aa Make uAssets a submodule 2021-07-29 19:36:49 +05:30
Raymond Hill 09db8803c5
Remove requirement of uAssets presence to build a package
Related discussion:
- https://github.com/cliqz-oss/adblocker/pull/2075#discussion_r678654293
2021-07-28 19:40:11 -04:00
Raymond Hill e1222d1643
Use `require` instead of `fs`
Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664#issuecomment-888332409
2021-07-28 10:38:24 -04:00
Raymond Hill 22022f636f
Modularize codebase with export/import
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664

The changes are enough to fulfill the related issue.

A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:

    ./tools/make-nodejs

This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.

From the root of the package, you can try:

    node test

This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.

The test.js file contains code which is typical example
of usage of the package.

Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.

This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.

This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-27 17:26:04 -04:00
Raymond Hill a4aced69dc
Reorganize platform directory to avoid Chromium-centric appearance
Though Firefox shares a lot of WebExtensions code with Chromium,
these platforms have their own specific code paths, for various
reasons.

The reorganization here makes it clear that Chromium platform is
just one flavor of WebExtensions, and as such all Chromium-specific
code paths should no longer be automatically pulled by other
platforms where these code paths are not needed.

Given that the filepath of many files changed, here is the
parent commit to quickly browse back to the previous directory
layout:

ec7db30b2f
2021-07-16 10:06:58 -04:00
Raymond Hill aa011e040c
Fix thunderbird build script 2020-12-10 10:00:27 -05:00
Raymond Hill bafbb7d62c
Remove reference to resources.txt 2020-10-02 10:42:38 -04:00
Raymond Hill 68ac096b39
Import translation work from https://crowdin.com/project/ublock 2020-09-30 10:23:08 -04:00
Raymond Hill ef2159ba34
Use new download name for Crowdin package 2020-08-22 09:08:14 -04:00
Raymond Hill 5c68867b92
Deprecate pseudo user styles code
The pseudo user styles code served only browsers based
on Chromium 65 and earlier -- Chromium 66 supports
native user styles and was first released more than two
years ago.

In Chromium-based browsers, the pseudo user styles code
is being unconditionally injected in every page/frame
just in case the browser is version 65 or earlier.

Removing pseudo user styles reduce uBO's main content
script in Chromium-based browsers by more than 20K.

Related thread:
- https://github.com/NanoAdblocker/NanoCore/issues/348#issuecomment-653646507
2020-07-22 10:21:16 -04:00
Raymond Hill a13ac92089
Replace seemingly abandoned malware lists with urlhaus list
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1116
- https://github.com/uBlockOrigin/uBlock-issues/issues/984

It has been found that the two malware lists uBO uses by
default appear to be essentially no longer maintained.

urlhaus list[1] has been identified by the community as
being well maintained and as being actually useful as a
default malware-related list in uBO, since it's being updated
every day from a database of top domains and specific URLs
identified as serving malicious content. Additionally, the
maintainer of urlhaus list has taken steps to increase
compatibility with uBO[2].

The decision has been to replace the current two malware-
related lists with urlhaus list, which will be enabled by
default in uBO -- and this means that list will be part of
uBO's package from now on.

For those who have the two removed malware lists enabled,
these will be moved to the custom lists section -- they
will still be enabled. It is suggested users remove them from
their selection of lists as their usefulness at this point
is questionable.

[1] https://gitlab.com/curben/urlhaus-filter
[2] 859dfd03c6
2020-07-06 09:03:20 -04:00
Raymond Hill cef034f650
Remove Armenian translation from Opera build
Opera requires that there is a valid translated
description for the specific locale to be accepted.
2020-06-04 08:38:44 -04:00
Raymond Hill bb873cc776
Import translation work from https://crowdin.com/project/ublock 2020-05-15 12:13:54 -04:00
Raymond Hill 151ff5326a
Import translation work from https://crowdin.com/project/ublock 2020-03-24 12:18:33 -04:00
Raymond Hill b784b7d569
Support loading of benchmark dataset in published versions
New advanced setting: `benchmarkDatasetURL`
Default value: `unset`

To specify a URL from where the benchmark dataset will be
fetched. This allows to launch benchmark operations from
within published versions of uBO, rather than from just
a locally built version.
2020-02-21 08:06:52 -05:00
Raymond Hill 004e4a3285
Support Bosnian in Opera store 2019-11-09 07:09:04 -05:00
Raymond Hill 7459a3f369
Fix thunderbird build script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/753
2019-10-11 18:01:05 -04:00
Raymond Hill 79b3b1921e
Import translation work from https://crowdin.com/project/ublock 2019-10-07 08:28:56 -04:00
Raymond Hill eec53c0154
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await. More left to do.

Related commits:
- 915687fddb
- 55cc0c6997
- e27328f931
2019-09-16 09:45:17 -04:00
Raymond Hill a94df7f3b2
Remove Bosnian locale from Opera package
Opera Store requires that shipped locales have
a proper accompanying description. The Bosnian
locale so far has no description and is only
partial.
2019-07-26 12:50:08 -04:00
Raymond Hill 6f5aa947fb
Finalize converting resources.txt into immutable resources
With hindsight, I revised decisions made earlier during
this development cycle:

Un-redirectable scriptlets have been removed from
/web_accessible_resources and instead put in the new
/assets/resources/scriptlets.js, which contains all
scriptlets used for web page injection purpose.

uBO will no longer fetch a remote version of built-in
resources.

Advanced setting `userResourcesLocation` will still be
honoured by uBO, and if set, will be fetched every
time at least one asset is updated.
2019-07-06 12:36:28 -04:00
Raymond Hill caa8e7d35b
Add shell script to copy common files
This shell script will copy files commoon to
all platform into the package.
2019-07-03 09:52:03 -04:00
Raymond Hill 152cea2dfe
Refactor management of injectable resources
This is a first step, the ultimate goal is to remove
the need for resources.txt, or at least to reduce to
only hotfixes or for trivial resources targeting very
specific websites.

Most resources will become immutable, i.e. they will
be part of uBO's code base. Advantages include easier
code maintenance (jshint, syntax highlight), and to
make scriptlets more easy to code review by external
parties (for example extension store reviewers).

TODO:

- More scriptlets need to be imported before next
  release.
- Need to make legacy versions of uBO use a legacy
  version of resources.txt, as all the now obsolete
  scriptlets will have to be removed once uBO's
  next release become widespread.
- Possibly need to add code to load binary
  resources so that they can be injected as
  data: URI. So far it's unclear whether this is
  really needed. For example, this would be needed
  if a xmlhttprequest is redirected to an image
  resource.
2019-07-03 09:47:56 -04:00
Raymond Hill 1dfdc40e09
Add ability to suspend network request handler at will
This works only for platforms supporting the return of
Promise by network listeners, i.e. only Firefox at this
point.

When filter lists are reloaded[1], there is a small
time window in which some network requests which should
have normally been blocked are not being blocked
because the static network filtering engine may not
have yet loaded all the filters in memory

This is now addressed by suspending the network request
handler when filter lists are reloaded -- again, this
works only on supported platforms.

[1] Examples: when a filter list update session
    completes; when user filters change, when
    adding/removing filter lists.
2019-06-30 10:09:27 -04:00
Raymond Hill b0b43ad6b2
Add support to import en_GB description 2019-06-23 08:25:16 -04:00
Raymond Hill 81324b765d
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/636 2019-06-17 07:35:14 -04:00
Raymond Hill 7e38391a71
Import new Bosnian translations from https://crowdin.com/project/ublock 2019-06-15 09:38:06 -04:00
Raymond Hill bd961364cc
Import new `en-GB` from https://crowdin.com/project/ublock 2019-05-31 09:30:32 -04:00
Raymond Hill 4d0784f8f3
Fix Opera package build script: removal of WASM-related files 2019-03-13 10:12:09 -04:00
Raymond Hill 07d4a6b60c
Import translation work from https://crowdin.com/project/ublock 2019-03-05 10:52:56 -05:00
Raymond Hill 5733439f62
Leverage whotracks.me's huge dataset of URLs for benchmark purpose
As seen at:
  https://whotracks.me/blog/adblockers_performance_study.html

The requests.json.gz file can be downloaded from:
  https://cdn.cliqz.com/adblocking/requests_top500.json.gz

Copy the file into ./tmp/requests.json.gz

If the file is present when you build uBO using `make-[target].sh` from
the shell, the resulting package will contain `./assets/requests.json`,
which will be looked-up by the method below to launch a benchmark
session.

From uBO's dev console, launch the benchmark:
  µBlock.staticNetFilteringEngine.benchmark();

The usual browser dev tools can be used to obtain useful profiling
data, i.e. start the profiler, call the benchmark method from the
console, then stop the profiler when it completes.

Keep in mind that the measurements at the blog post above where obtained
with ONLY EasyList. The CPU reportedly used was:
  https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-6600U+%40+2.60GHz&id=2608

Rename ./tmp/requests.json.gz to something else if you no longer want
./assets/requests.json in the build.
2019-02-15 16:18:03 -05:00
Raymond Hill 69c87c5117
Fix Promise chain of WASM module load operations
The Promise chain was not properly designed for WASM module
loading. This became apparent when removing WASM modules
from Opera build[1].

The problem was that errors thrown by fetch() -- used to
load WASM modules -- were not properly handled.

[1] Opera refuses updating uBO if there are unrecognized file
types in the package, and `.wasm`/`.wat` files are not
recognized by Opera uploader.
2019-02-01 08:20:43 -05:00
Raymond Hill a6ebcc85be
Reuse both flavors of webRequest wrapper in webext package
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/407

Both flavors will be stitched together into a single
`vapi-qebrequest.js` file.

The decision of which flavor to use will be made at runtime,
according to the browser environment.
2019-01-28 16:12:26 -05:00
Raymond Hill 305cd4db1c
Experimental Thunderbird 65+ support: experimental
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/194

This is a first iteration, which purpose is to merely allow
uBO to load properly. Many things are known to not work,
quite probably due to the early Thunderbird support of the
WebExtensions framework.

Permission which had to be removed:
- contextMenus

Manifest entries which had to be removed:
- commands
- sidebar_action

uBO's webRequest listeners are not being called when loading a
feed item in the preview pane, *except* for resources fetched
from embedded iframes.

uBO appears to function properly when a feed item is opened in
its own tab.
2019-01-20 12:22:40 -05:00
Raymond Hill a038c8658f
Attempt to fix build script for Firefox 2018-12-23 18:19:16 -05:00
Raymond Hill d528f3811a
Use version number in the filename of packages 2018-12-23 10:16:08 -05:00
Raymond Hill 015e31b577
fix https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=909626 2018-09-26 07:20:54 -04:00
Raymond Hill 6dd6dc5f1d
fix https://www.reddit.com/r/uBlockOrigin/comments/9gwrna/when_you_right_click_and_go_to_options_it_keeps/ 2018-09-18 20:14:09 -04:00
Raymond Hill a8c4e0214f
fix https://github.com/uBlockOrigin/uBlock-issues/issues/217, https://github.com/uBlockOrigin/uBlock-issues/issues/218 2018-09-11 06:44:52 -04:00
Raymond Hill cc4bf3581f
add ampproject.org/v0.js as a web-accessible-resources 2018-09-09 18:11:44 -04:00
Raymond Hill 2663c19a81
remove translations for which no description is available (Opera) 2018-08-24 09:38:00 -04:00
Raymond Hill 2d2339a26b
add import of Azerbaijani, Thai 2018-07-22 15:49:44 -04:00
Raymond Hill 2da91b8aef
use plain version string: uBO will render itself to b/rc if needed 2018-06-01 08:12:44 -04:00
Raymond Hill 4136a51ba3
fix webext build for chromium (https://github.com/gorhill/uBlock/issues/3716#issuecomment-393207176) 2018-05-30 11:52:52 -04:00
Raymond Hill b6b1edb288
fix opera make script 2018-05-20 11:08:53 -04:00
Raymond Hill 91b0ac5b3f
fix dd92337a4a (commitcomment-29051501) 2018-05-20 07:45:39 -04:00
Raymond Hill dd92337a4a
"developement build" instead of "dev build" 2018-05-18 10:29:09 -04:00
Raymond Hill 686171c315
import translation work from https://crowdin.com/project/ublock 2018-05-14 07:51:49 -04:00
Raymond Hill 4d8ccfdc05
fix https://bugzilla.mozilla.org/show_bug.cgi?id=1459007 2018-05-10 16:11:29 -04:00
Raymond Hill 89fd76ac39
use image data instead of paths for browser icons
When using paths, platform implementations of setIcon typically
will fetch the resource then convert to image data internally.
It is preferable for uBO to do this conversion itself as it can
be done only once at launch time.

With chromium-based browsers, using image data eliminate the
incessant network traffic to fetch browser icons as reported
in the extension's dev tool, meaning a good chunk of overhead
is eliminated.

Also, use optimal icon sizes, as of now both chromium and firefox
prefers 16px instead of 19px, and 32px instead of 38px.
2018-05-07 19:03:50 -04:00
Raymond Hill d9e52b4052
rename "debian" target to more accurate "webext" 2018-04-27 12:42:49 -04:00
Raymond Hill 2b460da748
rename "debian" target to more accurate "webext" 2018-04-27 12:39:43 -04:00
Raymond Hill 0bf2f27d87
rename "webext" target to more accurate "firefox" 2018-04-27 12:38:09 -04:00
Raymond Hill c9b14e201a
rename "webext" target to more accurate "firefox" 2018-04-27 12:37:33 -04:00
Raymond Hill 08495c42e9
complete "debian" as a target platform 2018-04-27 11:43:35 -04:00
Raymond Hill 443010efdc
remove firefox/legacy platform, it now has its own branch: `firefox-legacy` 2018-04-27 08:40:53 -04:00
Raymond Hill 9baf17fa2a
patch firefox/legacy for changes in e5f2e9701d 2018-04-26 12:01:31 -04:00
Raymond Hill e5f2e9701d
better manage platform-specific code for user-stylesheet support 2018-04-26 11:36:22 -04:00
Raymond Hill 4585287187
fix build script for Opera 2018-04-24 17:15:18 -04:00
Raymond Hill 665166e9d0
remove obsolete shell scripts 2018-04-15 06:55:54 -04:00
Raymond Hill 06ea2fb858
fix Firefox/legacy build 2018-04-14 09:36:19 -04:00
Raymond Hill efcab14597
Squashed commit of the following:
commit ab2b328cf1360a751fa1c58b8521f907eeb1ec50
Author: Raymond Hill <rhill@raymondhill.net>
Date:   Thu Apr 12 12:08:30 2018 -0400

    fix #3588

commit c4ae7638dfa5a5c7ddec2f9dd2d2988450082542
Author: Raymond Hill <rhill@raymondhill.net>
Date:   Thu Apr 12 09:08:56 2018 -0400

    detect user stylesheets support from content scripts (#3588)
2018-04-12 12:17:38 -04:00
Raymond Hill e377b3e915
changes needed for releasing uBO/opera 1.15.24 2018-04-09 10:04:32 -04:00
Raymond Hill d9161aeffa
fix #3636 2018-03-23 15:03:29 -04:00
Raymond Hill c84308da6f
minor code review of Python scripts 2018-02-28 12:27:41 -05:00
Raymond Hill d674e992e5
centralize the managing version string for all platform 2018-02-23 09:56:45 -05:00
Raymond Hill b2ff50f039
abandon separate, distinct webext version for dev build 2018-02-16 09:37:14 -05:00
Raymond Hill 287f04b47e
code review for #3474, #2823: simplify management of "web accessible resources" 2018-02-16 09:07:20 -05:00
Raymond Hill 17930cc778
fix #3474, #2823 2018-02-15 17:25:38 -05:00
Raymond Hill 7f9435f8b2
import Kazakh translation work 2018-01-31 12:49:23 -05:00
Raymond Hill 9fc8b546ee
replace "no" locale by "nb" 2018-01-13 09:50:26 -05:00
Raymond Hill abcea432a4
shallow clone uAssets in travis build 2018-01-09 17:57:40 -05:00
Raymond Hill dfcfa5ab9e
fix shell script for Opera 2017-12-22 09:57:28 -05:00
Raymond Hill 7afc78e866
do not include optional annoyances.txt in extension package 2017-12-17 09:42:24 -05:00
gorhill 324a75c45b
list changed files after importing from https://crowdin.com/project/ublock 2017-11-27 14:52:25 -05:00
gorhill 00559d42d3
import Malay, some translated strings are now available 2017-11-11 07:56:40 -05:00
gorhill cfef3aeb2b
minor review of shell scripts 2017-11-10 11:36:55 -05:00
gorhill fd5f63d850
fix #3196 2017-11-10 11:36:14 -05:00
gorhill 23487a643a
prepend vapi-usercss.js to contentscript.js in extension package 2017-10-24 12:40:58 -04:00