Released · 2.0.0

Still a UNIX pipe. Now a log store.

rtail has always put a stream of lines in your browser a second after they were printed. With --data it now keeps them: every line lands in Parquet, every field becomes a column, and one query language runs over the live tail and the history alike.

The wire format

Frozen, and always will be. The client you already have installed works against a v2 server unchanged.

The binaries

The server is one Go binary with the webapp and DuckDB embedded. The client is another: 2.6 MB, and no runtime of any kind.

Without --data

It is what it always was: an in-memory ring, nothing on disk, nothing to clean up.

on the host you are tailing
$ ./my-service 2>&1 | rtail --id api.example.com

The explorer

One filter drives the tail and the history.

Type a query and the live tail narrows — the server compiles the same rQL to a predicate over the ingest stream. Switch to history and it runs against Parquet. Same semantics either way, because there is one parser behind both.

rtail api 6 History 1 file · 19ms
1
2
level>=WARN service=api
rQLSQL level>=ERROR service=api latency_ms>500 "connection reset" Last hour › Run query ⌘⏎
⌄ Timeline 1,105 events
Aug 14 07:47 60s buckets Aug 14 08:47

Log fields

Filter fields
  • region 18,172
  • service 18,172
  • host envelope
  • level envelope
  • stream envelope
  • latency_ms 18,172
  • req 18,172
  • user_id 18,172
Sev Timestamp service latency_ms region Summary
! 08:46:24.212api50.6eu-west-1 retrying request
! 08:46:13.902api61.9us-west-2 cache miss storm
! 08:46:11.507api246.1us-east-1 slow query
! 08:45:45.463api1588.2us-east-1 upstream timeout
! 08:45:33.064api207.7us-west-2 slow query
! 08:45:22.842api3755.1eu-west-1 connection reset by peer
! 08:45:10.310api3481.7us-east-1 payment declined
! 08:44:57.059api3507.3us-west-2 connection reset by peer
! 08:44:31.008api67.8us-east-1 slow query
! 08:43:54.217api2824.2us-east-1 connection reset by peer
! 08:43:42.818api241.5eu-west-1 retrying request
! 08:43:28.330api3475.7us-east-1 upstream timeout

A drawing of the explorer, dimensioned by the app's own tokens: level>=WARN on the api stream, with service, latency_ms and region promoted to columns.

The histogram is the navigation

Counts over time, stacked by severity. Drag across it to zoom the range — the spike is the thing you came for, so it is also the control.

Most queries are built by clicking

Expand an event and every value carries filter to this and exclude this. The command bar updates to match, so what you built by pointing is something you could have typed.

Every view is a link

Stream, filter, time range, promoted columns and whether the timeline is open all live in the URL. A screenshot of a problem is a URL somebody can send you.

Completion is real

It comes from the catalog's key inventory — every field that has actually appeared, with its type and how often — not from a guess at what a log line might contain.

The sidebar answers “what is in here”

Top values of every field within the current results, before you know what to search for. The ⊞ beside one pulls it out into a column of its own.

Keyboard, not mouse

/ or ⌘K focuses the editor, ⌘⏎ runs it — plain ⏎ is a newline, because it is an editor. j and k move between events.

rQL

Terse enough to type, exact enough to trust.

k=v is equality, k:v a case-insensitive substring, k:~re a regular expression, k>n a comparison, k=* “the key exists”. Terms are ANDed unless you write OR; parentheses group. And level compares by severity rather than alphabetically, so level>=ERROR matches FATAL too — which is what you meant.

38 events

Severity compares by rank, not alphabetically, so this matches FATAL too. The bare phrase is full text across the message and the original line.

! 08:45:22.842connection reset by peerlatency_ms=3755.1
! 08:44:57.059connection reset by peerlatency_ms=3507.3
! 08:43:54.217connection reset by peerlatency_ms=2824.2

Full rQL reference →

Getting logs in

Four doors, one envelope.

A record is indistinguishable once stored, whichever way it came in.

UNIX pipes

The original, frozen forever. UDP, at-most-once, one datagram per line, and your process never blocks on it.

$ ./my-service 2>&1 | rtail --id api.example.com
$ tail -F /var/log/nginx/access.log | rtail --id nginx

HTTP

For anything that would mind losing a line. With --data, a 200 means every line has been fsynced to the write-ahead log; without it the server keeps only an in-memory ring, and a 200 means received, not stored. Anything else may follow a partial write, so every response says how many lines it accepted: resume after those rather than replay the batch.

$ curl -XPOST \
    'localhost:8888/v1/ingest?stream=api' \
    --data-binary @events.ndjson

OpenTelemetry

Protobuf or JSON at /v1/logs. service.name becomes the stream, and trace_id and span_id are promoted so correlation actually works.

exporters:
  otlphttp:
    logs_endpoint: http://rtail:8888/v1/logs

Syslog

RFC5424 and RFC3164, UDP and TCP on one port. The priority becomes a real severity, so level>=ERROR works without anyone writing a parser. It listens on loopback unless told otherwise, and carries no token — --auth-token covers HTTP only — so off loopback you are trusting your network.

$ rtail-server --data ./logs \
    --syslog-host 0.0.0.0 --syslog-port 5140

Anything that can write NDJSON to an HTTP endpoint works without a plugin — Vector and Fluent Bit are three lines of config each.

Storage

Your logs are just files.

Point --data at a directory and every line is written to Parquet. There is no schema to define: the envelope is always there, each root-level JSON key becomes its own typed column, and two files written a week apart with completely different keys reconcile by name at read time.

The layout
logs/
  catalog.sqlite
  wal/
  streams/api.example.com/2026/08/11/L0-….parquet
With rtail uninstalled
$ duckdb -c "
    SELECT a_service, count(*)
    FROM read_parquet('logs/streams/**/*.parquet',
                      union_by_name := true)
    WHERE level = 'ERROR' AND a_latency_ms > 500
    GROUP BY 1"

That is the point, and it is covered by a test rather than merely claimed.

Durability

Lines land in a write-ahead log first, fsynced every 100ms, so an unclean shutdown costs at most that and a clean one costs nothing. Recovery is idempotent: crashing midway through a flush produces the same files, not duplicates.

Compaction

Sort order is the point, not the file count. Time-sorted files make range pruning exact at row-group granularity, and a low-cardinality leading column turns level>=ERROR over a day into a skip instead of a scan.

Retention

Whole files are deleted once past the horizon; --retention-raw drops the original line but keeps every promoted column, which roughly halves the footprint.

The compaction levels
Level Holds Sorted by
L0 one flush seq — the order it arrived in
L1 an hour ts
L2 a day (level, ts), or --cluster-by

SQL

And raw SQL for everything else.

logs is a view over the pruned files. The query runs read-only in a DuckDB sandbox that can see nothing but your data directory.

The time range is the partition pruner, not decoration: it is always applied — a query without one gets the last 24 hours injected — and every response reports the window it used along with how many files and rows the query implied reading. A slow search can explain itself.

POST /v1/sql
{
  "sql": "SELECT a_service, count(*) n, quantile_cont(a_latency_ms, 0.95) p95 FROM logs WHERE level = 'ERROR' GROUP BY 1 ORDER BY n DESC",
  "from": "-6h"
}

Install

A server, and a pipe.

The server ships as a container image, which is the recommended way to run it; tagged releases also carry a binary for linux_amd64, linux_arm64, darwin_amd64 and darwin_arm64, each started and asked for /healthz before it is published. There is no Windows server: DuckDB is cgo. The client has no such problem — it cross-compiles anywhere, Windows included.

01 · the server
$ docker run -d --name rtail \
    -p 8888:8888 -p 9999:9999/udp \
    ghcr.io/kilianc/rtail

Then open localhost:8888 and start piping. Every flag is also an RTAIL_* environment variable.

02 · keep the lines
$ rtail-server --data ./logs \
    --retention 30d \
    --retention-raw 7d

Without --data nothing touches the disk. With it, the directory is yours to read with any tool.

03 · the client pipe
$ curl -fsSL \
    https://raw.githubusercontent.com/kilianc/rtail/main/install.sh \
    | sh

One 2.6 MB binary on the host you are tailing, with no runtime at all. Also go install, a tarball with checksums, or npm i -g rtail if you are already standing in a project.

Where 2.0 is

Everything on this page is built and tested: storage, search, compaction, the explorer, the four ingest protocols, authentication, TLS, and a release pipeline that publishes four tarballs, six npm packages and a container image per tag. 2.0.0 is the current release, and the plain install line, npm i -g rtail and the image's latest all reach it.

Before you put it on the internet

One shared token, or nothing: --auth-token puts every route except /healthz behind it. Programs send a bearer header, browsers sign in once and get a cookie. Without a token the server is open, and the image binds 0.0.0.0 — so published ports are reachable from your whole network.