GitHub - ryanbr/fop-rs: FOP Filter Orderer and Preener re-written in Rust

A Rust port of the EasyList FOP tool for sorting and cleaning ad-blocking filter lists.

Features

• Filter sorting: Alphabetically sorts blocking rules and element hiding rules
• Domain combining: Merges rules with identical selectors/patterns but different domains
• Option normalization: Converts uBO-specific options to standard ABP format (can be disabled)
• Wildcard cleanup: Removes unnecessary wildcards from filters
• Validation: Removes invalid/overly-broad rules (TLD-only, too short, etc.)
• Git integration: Commit changes directly to repositories (can be disabled)
• easylist_adservers.txt validation: Ensures rules start with | or /
• :has-text() merging: Combines rules with same base selector into single regex
• Parallel processing: Uses all CPU cores for faster processing via Rayon

Extended Syntax Support

FOP preserves extended filter syntax from various adblockers:

uBlock Origin

• Scriptlet injection: ##+js(...), ##^script:has-text(...)
• Procedural cosmetics: :has-text(), :has(), :upward(), :remove(), :style(), :matches-css(), :xpath(), :remove-attr(), :remove-class(), :watch-attr(), :min-text-length(), :others()
• Network options: redirect=, redirect-rule=, removeparam=, denyallow=, replace=, header=, permissions=, to=, from=, method=
• Regex domain rules: /regex/##+js(...)

AdGuard

• Scriptlet injection: #%#//scriptlet(...)
• CSS injection: #$#body { ... }
• Exceptions: #@%#, #@$#

Adblock Plus

• Extended selectors: :-abp-has(), :-abp-contains(), :-abp-properties()
• Action syntax: {remove: true;}, {height:...}, {display:...}
• Snippets: #$#hide-if-contains, #$#simulate-mouse-event

Speed Comparison (FOP.py vs FOP Rust)

Installation

Building from source (optional)

Requires Rust 1.80+ (https://rustup.rs)

cd fop-rs
cargo build --release
cp target/release/fop /usr/local/bin/  # or add to PATH

Usage

# Sort filters in current directory (with commit prompt)
fop

# Sort filters in a specific directory
fop /path/to/easylist

# Sort without commit prompt (like FOP-nocommit.py)
fop --no-commit /path/to/easylist
fop -n .

# Sort without converting uBO options to ABP format
fop --no-ubo-convert /path/to/ublock-filters

# Sort multiple directories
fop -n ~/easylist ~/easyprivacy ~/fanboy-addon

Command Line Options

Configuration File

Create .fopconfig in your working directory or home directory:

# Skip commit prompt
no-commit = false

# Skip uBO to ABP option conversion
no-ubo-convert = false

# Skip commit message format validation
no-msg-check = false

# Skip sorting (only tidy and combine rules)
no-sort = false

# Alternative sorting method
alt-sort = false

# Sort hosts file entries
localhost = false

# Disable colored output
no-color = false

# Disable large change warning prompt
no-large-warning = false

# Create .backup files before modifying
backup = false

# Keep empty lines in output
keep-empty-lines = false

# Don't skip rules without dot in domain
ignore-dot-domains = false

# Comment line prefixes
comments = !

# Output warnings to file
warning-output =

# Additional files to ignore
ignorefiles = .json,.backup,.bak,.swp,.gz

# Additional directories to ignore
ignoredirs =

# File extensions to process
file-extensions = txt

# Create PR branch instead of committing
create-pr =

# Include rule changes in PR body
pr-show-changes = false

# Base branch for PR (default: auto-detect)
git-pr-branch =

# Convert trusted scriptlets to non-trusted when value is safe
convert-trusted = false

# Fix cosmetic typos during sort
fix-typos = false

# Check typos in git additions
fix-typos-on-add = false

# Auto-fix without prompting
auto-fix = false

# Only sort git-changed files (skip unchanged)
only-sort-changed = false

# Path to banned domain list file
check-banned-list =

# Auto-remove banned domains and commit
auto-banned-remove = false

# CI mode - exit with error code on failures
ci = false

# Auto rebase and retry if push fails
rebase-on-fail = false

# Mask URLs in commit messages: 1=[.], 2=(.), 3=space, 4=preserve subdomain dot, 5=Unicode lookalike
# github.com, gitlab.com, and codeberg.org (and subdomains) are always exempt so PR/issue links stay clickable.
commit-mask =

# If non-empty, --commit-mask only applies when git user.name matches one of these (case-insensitive).
commit-mask-users =

# Also mask bare hostnames (no http/https). Risks FP on filenames like config.toml.
commit-mask-bare = false

# Additional apex hosts (and subdomains) exempt from masking. Adds to the
# built-in github.com / gitlab.com / codeberg.org list. Use for self-hosted
# Gitea, Forgejo, or private GitLab instances.
commit-mask-exempt-hosts =

# Override the 'Commit successful:' URL template (placeholders: {base}, {sha}).
# Default is auto-detected per host: bitbucket.org gets {base}/commits/{sha},
# everything else gets {base}/commit/{sha}. Override here if your platform
# uses something different (e.g. cgit, trac, custom redirector).
commit-url-template =

# Suppress most output (for CI)
quiet = false

# Users allowed to push directly when create-pr is enabled (comma-separated, case-insensitive)
direct-push-users =

# Predefined commit messages for arrow key selection (comma-separated)
# Use up/down arrows at commit prompt to cycle through these
history = A: ,P: ,M: Update,M: Cleanup,M: Sort,M: Adjust

Command line arguments override config file settings.

Per-File Overrides

Add [filename] sections to override settings for specific files:

# Global settings apply to all files
fix-typos = true

# Override settings for specific files
[adguard_base.txt]
parse-adguard = true

[hosts.txt]
localhost = true

[easylist.txt]
add-checksum = true
add-timestamp = true

Supported per-file options: no-sort, alt-sort, parse-adguard, localhost, add-checksum, add-timestamp, no-ubo-convert, abp-convert, convert-trusted, keep-empty-lines, ignore-dot-domains, fix-typos.

Commit Message URL Masking

--commit-mask=N defangs URLs in commit messages so they're stored in git history in a non-clickable form. Useful for filter-list maintainers who don't want third-party scrapers, search engines, or auto-linkers to follow URLs cited in commit messages. Masking happens after commit-message validation (so A:/P: prefixes still validate against the real URL) but before git commit -m, so the masked form is what lands in git.

Levels

Using https://www.example.com/foo/bar.html and https://www.example.co.nz/foo as inputs:

Notes:

• Path/query/fragment dots are never masked (only host dots), so file extensions like bar.html and query values like ?t=1.2 stay readable.
• Level 4 masks only the registrable domain plus eTLD ("eTLD+1"). Compound TLDs (co.uk, co.nz, com.au, co.jp, com.ng, com.hk, co.il, com.vn, etc. — ~25 country codes) are recognised so example.co.uk is treated as a 2-label TLD. Apex-only inputs (example.com, no subdomain) still defang the only dot.
• Level 5 is the strongest defang: U+2024 is visually nearly identical to . but is not in IDNA UTS #46 normalization tables, so URLs containing it won't resolve in browsers, curl, or most parsers — even after copy/paste. Reviewers won't realise the link is dead until they try to click it.
• Unknown levels (0, 6, 99…) silently fall through to level 1.
• Exempt hosts: github.com, gitlab.com, codeberg.org, and any of their subdomains (gist.github.com, docs.gitlab.com, etc.) are never masked, so PR/issue links remain clickable on the hosting platform. Lookalike hosts like notgitlab.com or evil.gitlab.com.attacker.com still get masked. For self-hosted Gitea/Forgejo/GitLab, add your host with --commit-mask-exempt-hosts=git.company.internal,....

Choosing a level

Restricting masking by user

--commit-mask-users=name1,name2,... (or commit-mask-users = ... in config) gates masking on the current git config user.name. If the list is empty, masking applies to whoever runs fop. If the list is non-empty, only matching users (case-insensitive) get masking; everyone else commits with the original URL.

commit-mask = 4
commit-mask-users = fanboynz, ryanbr

Bare hostnames (--commit-mask-bare)

Off by default. When enabled, also matches hostnames without an http(s):// prefix (e.g. forums.lanik.us or www.example.com.ng). The matcher requires at least one dot and a 2+ letter final label, so version numbers like 1.2.3 are not matched. Filenames like config.toml will match — that's the documented false-positive cost of opting in. Scheme URLs still take priority where both forms appear, and the host-exempt list (github.com, gitlab.com) applies to bare matches too.

Display label

After a successful commit, fop prints Commit message: followed by what landed in git. If masking actually changed the message, the label becomes Commit message (masked): instead, so it's obvious the displayed text is the masked form.

Commit message (masked):  A: https://www.example[.]com/foo
Commit successful:        https://github.com/easylist/easylist/commit/abc1234

If commit-mask is unset, or every URL was github/gitlab (a no-op mask), the plain Commit message: label is shown.

Platform Support

Pre-built Binaries

Which binary should I use?

Linux/Windows x86_64:

• Use -v3 for CPUs from ~2015+ (Haswell, Ryzen) - ~10-20% faster due to AVX2
• Use baseline if unsure or on older CPUs

Linux ARM64 (Raspberry Pi / Orange Pi):

• Use linux-arm64 for Pi 3, Pi 4, Orange Pi 3/4, or if unsure
• Use linux-arm64-n1 for Pi 5, Orange Pi 5, AWS Graviton2+ (~10-20% faster)

npm install downloads baseline binaries for maximum compatibility. Optimized versions are available from GitHub Releases.

Build Details

Unsupported platform? FOP.rs builds from source on any platform with Rust 1.80+:

Migrating from Python FOP

This Rust version is a drop-in replacement for both FOP.py and FOP-nocommit.py:

Performance

The Rust version is significantly faster than Python FOP due to:

• Compiled native code
• Parallel file processing with Rayon
• Optimized regex handling
• Efficient memory management

Windows PowerShell Note

If you see UnauthorizedAccess or PSSecurityException when running fop in PowerShell, run this once to fix permanently:

Set-ExecutionPolicy RemoteSigned -Scope CurrentUser

Alternatively, use Git Bash, MINGW64, or WSL.

Security Warnings

Pre-built binaries aren't code-signed, so your OS may show a warning on first run.

macOS

If blocked by Gatekeeper ("cannot be opened because the developer cannot be verified"):

xattr -d com.apple.quarantine ./fop-mac

Windows

If SmartScreen blocks the file:

1. Click "More info"
2. Click "Run anyway"

Alternative: Install via npm (npm install -g fop-cli) to avoid these warnings entirely.

License

GPL-3.0 (same as original Python FOP)

Credits

• Original Python FOP by Michael (EasyList project)
• Rust port maintains feature parity with Python version 3.9