Mirrors
/
uBlock
mirror of https://github.com/gorhill/uBlock.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
uBlock/tools/make-nodejs.sh

43 lines
1.5 KiB
Bash
Raw Normal View History

Revert "Pass -e flag to bash (#3790)" This reverts commit bb2015949516346edb137384882022f839a187d2.
2021-07-31 12:49:32 -06:00
#!/usr/bin/env bash
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
#
# This script assumes a linux environment
Use set -e in Bash scripts (#3791)
2021-07-31 15:34:25 -06:00
set -e
Add support for nodejs flavors The main nodejs flavor is "npm", which is to be used to lint/test and the publication of an official npm package -- and by design it has dependencies on mocha, eslint, etc. A new flavor "dig" has been created with minimal dependencies and which purpose is to easily allow to write specialized code to investigate local code changes in uBO -- and it's not meant for publication. Consequently, "make nodejs" has been replaced with "make npm", and a new "dig" target has been added to the makefile, to be used for instrumenting local code changes for investigation purpose.
2021-08-15 08:43:36 -06:00
DES=$1
Create build folder in npm install script (#3799)
2021-08-03 05:02:48 -06:00
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
mkdir -p $DES/js
cp src/js/base64-custom.js $DES/js
cp src/js/biditrie.js $DES/js
Export the rule-based filtering engines to the nodejs package The code exported to nodejs package was revised to use modern JavaScript syntax. A few issues were fixed at the same time. The exported classes are: - DynamicHostRuleFiltering - DynamicURLRuleFiltering - DynamicSwitchRuleFiltering These related to the content the of "My rules" pane in the uBlock Origin extension.
2021-08-03 10:19:25 -06:00
cp src/js/dynamic-net-filtering.js $DES/js
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
cp src/js/filtering-context.js $DES/js
Export the rule-based filtering engines to the nodejs package The code exported to nodejs package was revised to use modern JavaScript syntax. A few issues were fixed at the same time. The exported classes are: - DynamicHostRuleFiltering - DynamicURLRuleFiltering - DynamicSwitchRuleFiltering These related to the content the of "My rules" pane in the uBlock Origin extension.
2021-08-03 10:19:25 -06:00
cp src/js/hnswitches.js $DES/js
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
cp src/js/hntrie.js $DES/js
Add experimental mv3 version This create a separate Chromium extension, named "uBO Minus (MV3)". This experimental mv3 version supports only the blocking of network requests through the declarativeNetRequest API, so as to abide by the stated MV3 philosophy of not requiring broad "read/modify data" permission. Accordingly, the extension should not trigger the warning at installation time: Read and change all your data on all websites The consequences of being permission-less are the following: - No cosmetic filtering (##) - No scriptlet injection (##+js) - No redirect= filters - No csp= filters - No removeparam= filters At this point there is no popup panel or options pages. The default filterset correspond to the default filterset of uBO proper: Listset for 'default': https://ublockorigin.github.io/uAssets/filters/badware.txt https://ublockorigin.github.io/uAssets/filters/filters.txt https://ublockorigin.github.io/uAssets/filters/filters-2020.txt https://ublockorigin.github.io/uAssets/filters/filters-2021.txt https://ublockorigin.github.io/uAssets/filters/filters-2022.txt https://ublockorigin.github.io/uAssets/filters/privacy.txt https://ublockorigin.github.io/uAssets/filters/quick-fixes.txt https://ublockorigin.github.io/uAssets/filters/resource-abuse.txt https://ublockorigin.github.io/uAssets/filters/unbreak.txt https://easylist.to/easylist/easylist.txt https://easylist.to/easylist/easyprivacy.txt https://malware-filter.gitlab.io/malware-filter/urlhaus-filter-online.txt https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext The result of the conversion of the filters in all these filter lists is as follow: Ruleset size for 'default': 22245 Good: 21408 Maybe good (regexes): 127 redirect-rule= (discarded): 458 csp= (discarded): 85 removeparams= (discarded): 22 Unsupported: 145 The fact that the number of DNR rules are far lower than the number of network filters reported in uBO comes from the fact that lists-to-rulesets converter does its best to coallesce filters into minimal set of rules. Notably, the DNR's requestDomains condition property allows to create a single DNR rule out of all pure hostname-based filters. Regex-based rules are dynamically added at launch time since they must be validated as valid DNR regexes through isRegexSupported() API call. At this point I consider being permission-less the limiting factor: if broad "read/modify data" permission is to be used, than there is not much point for an MV3 version over MV2, just use the MV2 version if you want to benefit all the features which can't be implemented without broad "read/modify data" permission. To locally build the MV3 extension: make mv3 Then load the resulting extension directory in the browser using the "Load unpacked" button. From now on there will be a uBlock0.mv3.zip package available in each release.
2022-09-06 11:47:52 -06:00
cp src/js/static-dnr-filtering.js $DES/js
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
cp src/js/static-filtering-parser.js $DES/js
cp src/js/static-net-filtering.js $DES/js
cp src/js/static-filtering-io.js $DES/js
Add tasks.js module (#3839)
2021-08-19 05:19:20 -06:00
cp src/js/tasks.js $DES/js
Fix make-nodejs Broken due to renaming text-iterators to text-utils.
2021-07-31 06:59:55 -06:00
cp src/js/text-utils.js $DES/js
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
cp src/js/uri-utils.js $DES/js
Export the rule-based filtering engines to the nodejs package The code exported to nodejs package was revised to use modern JavaScript syntax. A few issues were fixed at the same time. The exported classes are: - DynamicHostRuleFiltering - DynamicURLRuleFiltering - DynamicSwitchRuleFiltering These related to the content the of "My rules" pane in the uBlock Origin extension.
2021-08-03 10:19:25 -06:00
cp src/js/url-net-filtering.js $DES/js
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
mkdir -p $DES/lib
cp -R src/lib/punycode.js $DES/lib/
cp -R src/lib/regexanalyzer $DES/lib/
Simplify loading of WASM modules in nodejs
2021-07-31 05:06:11 -06:00
cp -R src/lib/publicsuffixlist $DES/lib/
# Convert wasm modules into json arrays
mkdir -p $DES/js/wasm
Add experimental mv3 version This create a separate Chromium extension, named "uBO Minus (MV3)". This experimental mv3 version supports only the blocking of network requests through the declarativeNetRequest API, so as to abide by the stated MV3 philosophy of not requiring broad "read/modify data" permission. Accordingly, the extension should not trigger the warning at installation time: Read and change all your data on all websites The consequences of being permission-less are the following: - No cosmetic filtering (##) - No scriptlet injection (##+js) - No redirect= filters - No csp= filters - No removeparam= filters At this point there is no popup panel or options pages. The default filterset correspond to the default filterset of uBO proper: Listset for 'default': https://ublockorigin.github.io/uAssets/filters/badware.txt https://ublockorigin.github.io/uAssets/filters/filters.txt https://ublockorigin.github.io/uAssets/filters/filters-2020.txt https://ublockorigin.github.io/uAssets/filters/filters-2021.txt https://ublockorigin.github.io/uAssets/filters/filters-2022.txt https://ublockorigin.github.io/uAssets/filters/privacy.txt https://ublockorigin.github.io/uAssets/filters/quick-fixes.txt https://ublockorigin.github.io/uAssets/filters/resource-abuse.txt https://ublockorigin.github.io/uAssets/filters/unbreak.txt https://easylist.to/easylist/easylist.txt https://easylist.to/easylist/easyprivacy.txt https://malware-filter.gitlab.io/malware-filter/urlhaus-filter-online.txt https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext The result of the conversion of the filters in all these filter lists is as follow: Ruleset size for 'default': 22245 Good: 21408 Maybe good (regexes): 127 redirect-rule= (discarded): 458 csp= (discarded): 85 removeparams= (discarded): 22 Unsupported: 145 The fact that the number of DNR rules are far lower than the number of network filters reported in uBO comes from the fact that lists-to-rulesets converter does its best to coallesce filters into minimal set of rules. Notably, the DNR's requestDomains condition property allows to create a single DNR rule out of all pure hostname-based filters. Regex-based rules are dynamically added at launch time since they must be validated as valid DNR regexes through isRegexSupported() API call. At this point I consider being permission-less the limiting factor: if broad "read/modify data" permission is to be used, than there is not much point for an MV3 version over MV2, just use the MV2 version if you want to benefit all the features which can't be implemented without broad "read/modify data" permission. To locally build the MV3 extension: make mv3 Then load the resulting extension directory in the browser using the "Load unpacked" button. From now on there will be a uBlock0.mv3.zip package available in each release.
2022-09-06 11:47:52 -06:00
cp src/js/wasm/* $DES/js/wasm/
Simplify loading of WASM modules in nodejs
2021-07-31 05:06:11 -06:00
node -pe "JSON.stringify(Array.from(fs.readFileSync('src/js/wasm/hntrie.wasm')))" \
> $DES/js/wasm/hntrie.wasm.json
node -pe "JSON.stringify(Array.from(fs.readFileSync('src/js/wasm/biditrie.wasm')))" \
> $DES/js/wasm/biditrie.wasm.json
node -pe "JSON.stringify(Array.from(fs.readFileSync('src/lib/publicsuffixlist/wasm/publicsuffixlist.wasm')))" \
> $DES/lib/publicsuffixlist/wasm/publicsuffixlist.wasm.json
Modularize codebase with export/import Related issue: - https://github.com/uBlockOrigin/uBlock-issues/issues/1664 The changes are enough to fulfill the related issue. A new platform has been added in order to allow for building a NodeJS package. From the root of the project: ./tools/make-nodejs This will create new uBlock0.nodejs directory in the ./dist/build directory, which is a valid NodeJS package. From the root of the package, you can try: node test This will instantiate a static network filtering engine, populated by easylist and easyprivacy, which can be used to match network requests by filling the appropriate filtering context object. The test.js file contains code which is typical example of usage of the package. Limitations: the NodeJS package can't execute the WASM versions of the code since the WASM module requires the use of fetch(), which is not available in NodeJS. This is a first pass at modularizing the codebase, and while at it a number of opportunistic small rewrites have also been made. This commit requires the minimum supported version for Chromium and Firefox be raised to 61 and 60 respectively.
2021-07-25 08:55:35 -06:00
...
2021-08-05 12:36:13 -06:00
cp platform/nodejs/*.js $DES/
cp platform/nodejs/README.md $DES/
cp LICENSE.txt $DES/