- Rust 81.7%
- Shell 13.3%
- Roff 5%
Eduard Tolosa
a9b2339c21
Some checks failed
Github Actions / build (push) Has been cancelled
Correctness: - HTTPS and HTTP were raced with select!, so a refused port 80 cancelled a working TLS handshake and the host was reported dead - retries defaults to 0, and the loop bounded by it never issued a request - the 1 MB body cap counted characters and was applied after downloading - content_length fell back to a character count instead of bytes - status code filters matched as a substring, so 40 matched 404 - concurrency of 0 hung on buffer_unordered Performance: - a new HTTP client was built per host in the filters path - the user agent list was rebuilt on every call - the response body is no longer read when the caller does not want it - the HeaderMap was cloned per host to read two headers - the body buffer is sized from Content-Length instead of grown by doubling - selectors are parsed once, not per response - stdout is buffered for the whole run Dependencies: - reqwest 0.13, scraper 0.27, rand 0.10, edition 2021 - dropped async-recursion (unused) and the rustls backend (native-tls is used) - tokio trimmed from "full" to the features actually used API: - collect_body gates reading the body; print_results gates printing |
||
|---|---|---|
| .github | ||
| .vscode | ||
| examples | ||
| files | ||
| src | ||
| .gitignore | ||
| .rustfmt.toml | ||
| builder.sh | ||
| Cargo.lock | ||
| Cargo.toml | ||
| fhc.1 | ||
| LICENSE | ||
| README.md | ||
FHC
FHC stands for Fast HTTP Checker, it's written in Rust. Works on Linux, Windows, macOS, Android, Aarch64, ARM and possibly in your oven.
Goal
Offer the community an efficient HTTP checker tool.
Methodology
FHC try to resolve https first, if this fails then fallback to http, in that way you don't miss any active HTTP host.
Performance & speed
FHC is very resource friendly, you can use up to 1000 threads in an single core machine and this will work without any problem, the bottleneck for this tool is your network speed. By default, FHC is able to perform HTTP check for ~913 hosts per second in good network conditions (tested in an Google Cloud machine). Depending on how much host have only http (not https) and/or are alive the number of host resolved in average can be low/higher as the tool have or not to perform a double check. In our demo we used an real-world scenario performing resolution for google.com subdomains.
Demo
The hosts file used in the demo is here.
Installation
Using precompiled binaries.
Download the asset from the releases page according to your platform.
Using the source code.
- You need to have the latest stable Rust version installed in your system.
- Clone the repo or download the source code, then run
cargo build --release. - Execute the tool from
./target/release/fhcor add it to your system PATH to use from anywhere.
Using the AUR packages. (Arch Linux)
fhc can be installed from available AUR packages using an AUR helper. For example,
yay -S fhc
If you prefer, you can clone the AUR packages and then compile them with makepkg. For example,
git clone https://aur.archlinux.org/fhc.git && cd fhc && makepkg -si
Usage
- Show all HTTP urls depite their response codes:
cat hosts.txt | fhc
- If you want to see only the hosts answering with specific status codes:
cat hosts.txt | fhc -f 200,204
- Or to leave some out:
cat hosts.txt | fhc -e 404,500
- To also show the status code and the final URL:
cat hosts.txt | fhc -s
Each host is tried over HTTPS first and only then over plain HTTP. You can tune
the --timeout, -t/--threads, -r/--retries and other options according to
your needs. See fhc --help