diff --git a/README.md b/README.md index 6059669..bd21ac3 100644 --- a/README.md +++ b/README.md @@ -31,38 +31,37 @@ High level project goals: - [x] Download and cache pages - [x] Don't require a network connection for anything besides updating the cache -- [x] Command line interface similar or equivalent to the [NodeJS client][tldr-node-client] +- [x] Command line interface similar or equivalent to the [NodeJS client][node-gh] +- [x] Comply with the [tldr client specification][client-spec] +- [x] Advanced highlighting and configuration - [x] Be fast -A tool like `tldr` should be as frictionless as possible to use. It should be -easy to invoke (just `tldr tar`, not using another subcommand like `tldr find -tar`) and it should show the output as fast as possible. +A tool like `tldr` should be as frictionless as possible to use and show the +output as fast as possible. -tealdeer reaches these goals. During a (highly non-scientific) test (see -[#38](https://github.com/dbrgn/tealdeer/issues/38) for details), I tested the -invocation speed of `tldr ` for a few of the existing clients: +We think that `tealdeer` reaches these goals. We put together a (more or less) +reproducible benchmark that compiles a handful of clients from source and +measures the execution times on a cold disk cache. The benchmarking is run in a +Docker container using sharkdp's [`hyperfine`][hyperfine-gh] +([Dockerfile][benchmark-dockerfile]). -| Client | Times (ms) | Avg of 5 (ms) | -| --- | --- | --- | -| [Tealdeer](https://github.com/dbrgn/tealdeer/) | `15/11/5/5/11` | `9.4` (100%) | -| [C client](https://github.com/tldr-pages/tldr-cpp-client) | `11/5/12/11/15` | `10.8` (115%) | -| [Bash client](https://github.com/pepa65/tldr-bash-client) | `15/19/22/25/24` | `21.0` (223%) | -| [Go client by k3mist](https://github.com/k3mist/tldr/) | `98/96/100/95/101` | `98.8` (1'051%) | -| [Python client](https://github.com/lord63/tldr.py) | `152/148/151/158/140` | `149.8` (1'594%) | -| [NodeJS client](https://github.com/tldr-pages/tldr-node-client) | `169/171/170/170/170` | `170.0` (1'809%) | - -tealdeer was the winner here, although the C client and the Bash client are in -the same speed class. Interpreted languages are clearly much slower to invoke, -a delay of 170 milliseconds is definitely noticeable and increases friction for -the user. - -These are the clients I tried but failed to compile or run: -[Haskell client](https://github.com/psibi/tldr-hs), -[Ruby client](https://github.com/YellowApple/tldrb), -[Perl client](https://github.com/skaji/perl-tldr), -[Go client by anoopengineer](https://github.com/anoopengineer/tldr/), -[PHP client](https://github.com/BrainMaestro/tldr-php). +| Client (50 runs, 29.07.2021) | Programming Language | Mean in ms | Deviation in ms | Comments | +| :---: | :---: | :---: | :---: | :---: | +| `tealdeer` | Rust | 16.2 | 1.2 | | +| [`tldr-c`][c-gh] | C | 35.7 | 3.2 | | +| [`tldr-hs`][hs-gh] | Haskell | 23.6 | 1.0 | no example highlighting | +| [`fast-tldr`][fast-tldr-gh] | Haskell | 15.2 | 0.7 | no example highlighting | +| [`tldr-bash`][bash-gh] | Bash | 28.3 | 1.1 | | +| [`tldr-python-client`][python-gh] | Python | 97.9 | 3.5 | | +| [`outfieldr`][outfieldr-gh] | Zig | 4.7 | 0.5 | no user configuration | +| [`tldr-node-client`][node-gh] | Javascript / Node | 533.1 | 12.5 | | +As you can see, `tealdeer` is not the fastest of the tested clients anymore. We +strive for useful features and code quality over raw performance, even if that +means that we don't come out on top in this friendly competition. That said, we +are still optimizing the code, for example when the `outfieldr` developers +[suggested to switch][outfieldr-comment-tls] to a native TLS implementation +instead of the native libraries. ## Development @@ -113,7 +112,18 @@ be dual licensed as above, without any additional terms or conditions. Thanks to @severen for coming up with the name "tealdeer"! -[tldr-node-client]: https://github.com/tldr-pages/tldr-node-client +[node-gh]: https://github.com/tldr-pages/tldr-node-client +[c-gh]: https://github.com/tldr-pages/tldr-c-client +[hs-gh]: https://github.com/psibi/tldr-hs +[fast-tldr-gh]: https://github.com/gutjuri/fast-tldr +[bash-gh]: https://4e4.win/tldr +[outfieldr-gh]: https://gitlab.com/ve-nt/outfieldr +[python-gh]: https://github.com/tldr-pages/tldr-python-client + +[benchmark-dockerfile]: https://github.com/dbrgn/tealdeer/blob/master/benchmarks/Dockerfile +[client-spec]: https://github.com/tldr-pages/tldr/blob/main/CLIENT-SPECIFICATION.md +[hyperfine-gh]: https://github.com/sharkdp/hyperfine +[outfieldr-comment-tls]: https://github.com/dbrgn/tealdeer/issues/129#issuecomment-833596765 [github-actions]: https://github.com/dbrgn/tealdeer/actions?query=branch%3Amaster diff --git a/benchmarks/Dockerfile b/benchmarks/Dockerfile new file mode 100644 index 0000000..b70c58c --- /dev/null +++ b/benchmarks/Dockerfile @@ -0,0 +1,139 @@ +# Benchmark Dockerfile for tealdeer +# +# To run the benchmarks, execute +# +# docker build -t tldr-benchmark . +# docker run --privileged --rm -it tldr-benchmark +# +# as root in the directory of this Dockerfile. This will build the compared +# clients and benchmark them with `hyperfine` at the end. +# +# The `--privileged` flag is needed to drop the disk caches before every run. If +# you want to test with hot caches or don't want to use this flag, you will have +# to remove the `--prepare` line from the `hyperfine` command at the end of this +# file and rebuild the image. + +################################################################################ + +FROM rust AS tealdeer-builder + +WORKDIR /build +RUN git clone https://github.com/dbrgn/tealdeer.git \ + && cd tealdeer \ + && cargo build --release \ + && mkdir /build-outputs \ + && cp target/release/tldr /build-outputs/tealdeer + +################################################################################ + +FROM ubuntu:latest AS tldr-c-builder + +WORKDIR /build +RUN apt-get update && apt-get install -y build-essential git && rm -rf /var/lib/apt/lists/* +RUN git clone https://github.com/tldr-pages/tldr-c-client.git \ + && cd tldr-c-client \ + && DEBIAN_FRONTEND=noninteractive ./deps.sh \ + && make \ + && mkdir /build-outputs /deps \ + && cp tldr /build-outputs/tldr-c \ + && cp deps.sh /deps/tldr-c-deps.sh + +################################################################################ + +FROM haskell AS haskell-builder + +WORKDIR /build + +RUN git clone https://github.com/psibi/tldr-hs.git \ + && cd tldr-hs \ + && stack build --install-ghc + +RUN git clone https://github.com/gutjuri/fast-tldr \ + && cd fast-tldr \ + && stack build --install-ghc + +RUN mkdir /build-outputs \ + && find tldr-hs/.stack-work/dist -type f -iname tldr -exec mv '{}' /build-outputs/tldr-hs \; \ + && find fast-tldr/.stack-work/dist -type f -iname tldr -exec mv '{}' /build-outputs/fast-tldr \; + +################################################################################ + +FROM node:slim AS node-builder + +WORKDIR /build-outputs +RUN npm install tldr \ + && cp $(which node) . \ + && echo './node -- ./node_modules/.bin/tldr "$@"' > tldr-node \ + && chmod +x tldr-node + +################################################################################ + +FROM euantorano/zig:0.8.0 AS zig-builder + +WORKDIR /build +RUN apk add git \ + && git clone https://gitlab.com/ve-nt/outfieldr.git \ + && cd outfieldr \ + && git submodule init \ + && git submodule update \ + && zig build -Drelease-safe \ + && mkdir /build-outputs \ + && cp bin/tldr /build-outputs/outfieldr + +################################################################################ + +FROM ubuntu:latest AS benchmark + +ENV LANG="en_US.UTF-8" + +WORKDIR /deps +RUN apt-get update && apt-get install -y wget unzip python3 python3-venv && rm -rf /var/lib/apt/lists/* +COPY --from=tldr-c-builder /deps/* ./ +RUN for file in *; do DEBIAN_FRONTEND=noninteractive sh $file; done + +WORKDIR /clients +COPY --from=tealdeer-builder /build-outputs/* ./ +COPY --from=tldr-c-builder /build-outputs/* ./ +COPY --from=haskell-builder /build-outputs/* ./ +RUN wget -qO tldr-bash https://4e4.win/tldr && chmod +x tldr-bash +COPY --from=node-builder /build-outputs/node /build-outputs/tldr-node ./ +COPY --from=node-builder /build-outputs/node_modules/ ./node_modules/ +COPY --from=zig-builder /build-outputs/* ./ + +# python is really hard to isolate in a package, using pyinstaller didn't really work either, so for now we just use it like this +RUN python3 -m venv tldr-python \ + && cd tldr-python \ + && bash -c 'source bin/activate; pip install wheel; pip install tldr; deactivate' \ + && cd .. \ + && echo '#!/bin/bash' > tldr-python.bash \ + && echo 'source tldr-python/bin/activate; tldr $@' >> tldr-python.bash \ + && chmod +x tldr-python.bash + +# Update all the individual caches +RUN bash -c 'mkdir -p /caches/{tealdeer,tldr-c,tldr-hs,fast-tldr,tldr-bash,tldr-node,tldr-python,outfieldr/.local/share}' \ + && TEALDEER_CACHE_DIR=/caches/tealdeer ./tealdeer -u \ + && TLDR_CACHE_DIR=/caches/tldr-c ./tldr-c -u \ + && XDG_DATA_HOME=/caches/tldr-hs ./tldr-hs -u \ + && XDG_DATA_HOME=/caches/fast-tldr ./fast-tldr -u \ + && XDG_DATA_HOME=/caches/tldr-bash ./tldr-bash -u \ + && HOME=/caches/tldr-node ./tldr-node -u \ + && HOME=/caches/tldr-python ./tldr-python.bash -u \ + && HOME=/caches/outfieldr ./outfieldr -u + +WORKDIR /tools +RUN wget -q https://github.com/sharkdp/hyperfine/releases/download/v1.11.0/hyperfine_1.11.0_amd64.deb && dpkg -i hyperfine_1.11.0_amd64.deb + +ENV PAGE="tar" +WORKDIR /clients +CMD hyperfine \ + --warmup 10 \ + --runs 50 \ + --prepare 'sync; echo 3 | tee /proc/sys/vm/drop_caches' \ + "TEALDEER_CACHE_DIR=/caches/tealdeer ./tealdeer $PAGE" \ + "TLDR_CACHE_DIR=/caches/tldr-c ./tldr-c $PAGE" \ + "XDG_DATA_HOME=/caches/tldr-hs ./tldr-hs $PAGE" \ + "XDG_DATA_HOME=/caches/fast-tldr ./fast-tldr $PAGE" \ + "XDG_DATA_HOME=/caches/tldr-bash TLDR_LESS=0 ./tldr-bash $PAGE" \ + "HOME=/caches/tldr-python ./tldr-python.bash $PAGE" \ + "HOME=/caches/outfieldr ./outfieldr $PAGE" \ + "HOME=/caches/tldr-node ./tldr-node $PAGE"