Commit Graph

23 Commits

Author SHA1 Message Date
Raymond Hill ed4b31931a
Prepare to ship minified lists in package 2023-10-26 13:38:24 -04:00
Raymond Hill 223815342e
Make dev build use its own assets.json: assets.dev.json
This will make it easier to safely test new versions of
assets.json.
2023-05-04 19:20:18 -04:00
Raymond Hill e7f6b43659
Update as per filter list changes 2023-04-28 17:02:51 -04:00
Raymond Hill bc35110180
Pick only the packaged lists 2023-04-27 13:22:15 -04:00
Raymond Hill 17590c5a0c
Use `git clone` instead of submodule to pull uAssets dependencies
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/2358

We need two different branches of uAssets to properly add the
default filter lists into the packages. I couldn't figure out
how to do this with submodules, using simple `git clone` as a
dependency just work.
2022-11-14 09:50:53 -05:00
Manish Jethani be68a6b5b4
Use set -e in Bash scripts (#3791) 2021-07-31 17:34:25 -04:00
Raymond Hill 6fa8ae619d
Revert "Pass -e flag to bash (#3790)"
This reverts commit bb20159495.
2021-07-31 14:49:32 -04:00
Manish Jethani bb20159495
Pass -e flag to bash (#3790) 2021-07-31 12:15:32 -04:00
Manish Jethani 63591ef2aa Make uAssets a submodule 2021-07-29 19:36:49 +05:30
Raymond Hill 09db8803c5
Remove requirement of uAssets presence to build a package
Related discussion:
- https://github.com/cliqz-oss/adblocker/pull/2075#discussion_r678654293
2021-07-28 19:40:11 -04:00
Raymond Hill bafbb7d62c
Remove reference to resources.txt 2020-10-02 10:42:38 -04:00
Raymond Hill a13ac92089
Replace seemingly abandoned malware lists with urlhaus list
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1116
- https://github.com/uBlockOrigin/uBlock-issues/issues/984

It has been found that the two malware lists uBO uses by
default appear to be essentially no longer maintained.

urlhaus list[1] has been identified by the community as
being well maintained and as being actually useful as a
default malware-related list in uBO, since it's being updated
every day from a database of top domains and specific URLs
identified as serving malicious content. Additionally, the
maintainer of urlhaus list has taken steps to increase
compatibility with uBO[2].

The decision has been to replace the current two malware-
related lists with urlhaus list, which will be enabled by
default in uBO -- and this means that list will be part of
uBO's package from now on.

For those who have the two removed malware lists enabled,
these will be moved to the custom lists section -- they
will still be enabled. It is suggested users remove them from
their selection of lists as their usefulness at this point
is questionable.

[1] https://gitlab.com/curben/urlhaus-filter
[2] 859dfd03c6
2020-07-06 09:03:20 -04:00
Raymond Hill b784b7d569
Support loading of benchmark dataset in published versions
New advanced setting: `benchmarkDatasetURL`
Default value: `unset`

To specify a URL from where the benchmark dataset will be
fetched. This allows to launch benchmark operations from
within published versions of uBO, rather than from just
a locally built version.
2020-02-21 08:06:52 -05:00
Raymond Hill 6f5aa947fb
Finalize converting resources.txt into immutable resources
With hindsight, I revised decisions made earlier during
this development cycle:

Un-redirectable scriptlets have been removed from
/web_accessible_resources and instead put in the new
/assets/resources/scriptlets.js, which contains all
scriptlets used for web page injection purpose.

uBO will no longer fetch a remote version of built-in
resources.

Advanced setting `userResourcesLocation` will still be
honoured by uBO, and if set, will be fetched every
time at least one asset is updated.
2019-07-06 12:36:28 -04:00
Raymond Hill 5733439f62
Leverage whotracks.me's huge dataset of URLs for benchmark purpose
As seen at:
  https://whotracks.me/blog/adblockers_performance_study.html

The requests.json.gz file can be downloaded from:
  https://cdn.cliqz.com/adblocking/requests_top500.json.gz

Copy the file into ./tmp/requests.json.gz

If the file is present when you build uBO using `make-[target].sh` from
the shell, the resulting package will contain `./assets/requests.json`,
which will be looked-up by the method below to launch a benchmark
session.

From uBO's dev console, launch the benchmark:
  µBlock.staticNetFilteringEngine.benchmark();

The usual browser dev tools can be used to obtain useful profiling
data, i.e. start the profiler, call the benchmark method from the
console, then stop the profiler when it completes.

Keep in mind that the measurements at the blog post above where obtained
with ONLY EasyList. The CPU reportedly used was:
  https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-6600U+%40+2.60GHz&id=2608

Rename ./tmp/requests.json.gz to something else if you no longer want
./assets/requests.json in the build.
2019-02-15 16:18:03 -05:00
Raymond Hill abcea432a4
shallow clone uAssets in travis build 2018-01-09 17:57:40 -05:00
Raymond Hill 7afc78e866
do not include optional annoyances.txt in extension package 2017-12-17 09:42:24 -05:00
gorhill 00b19515cf minor change 2017-01-18 13:36:24 -05:00
Raymond Hill 3b9fd49c50 Assets management refactored (#2314)
* refactoring assets management code

* finalizing refactoring of assets management

* various code review of new assets management code

* fix #2281

* fix #1961

* fix #1293

* fix #1275

* fix update scheduler timing logic

* forward compatibility (to be removed once 1.11+ is widespread)

* more codereview; give admins ability to specify own assets.json

* "assetKey" is more accurate than "path"

* fix group count update when building dom incrementally

* reorganize content (order, added URLs, etc.)

* ability to customize updater through advanced settings

* better spinner icon
2017-01-18 13:17:47 -05:00
gorhill 909ef5303d prevent pushd/popd from polluting output 2016-04-03 18:06:19 -04:00
gorhill a9d36027e1 this fixes #1531 2016-04-03 17:45:02 -04:00
gorhill 5fda221031 this fixes travis? 2016-04-03 14:25:56 -04:00
gorhill cad3c5f5cb update packaging scripts for https://github.com/uBlockOrigin/uAssets repo 2016-04-03 13:07:46 -04:00