Access Log Parser

Combined and common log formats encode client IP, identity fields, timestamp, request line, status code, bytes sent, referer, and user agent on a single line. An access log parser turns those lines into structured JSON so you can filter with jq, spreadsheets, or a notebook during traffic spikes and incident response.

Nginx and Apache both ship popular presets (combined, common) and allow custom log_format / LogFormat strings. If parsing fails, compare a sample line against the active format on the server. A single missing field or alternate timestamp layout breaks naive splitters.

Behind load balancers and CDNs, the leftmost client IP is often wrong. Prefer $http_x_forwarded_for or equivalent, and document which hop is authoritative. IPv6 addresses and quoted strings change field boundaries; parsers must not assume dotted-quad IPv4 only.

Status code histograms answer first questions: flood of 499/502 during a deploy, 404 crawl noise, or 401 spikes after a token rotation. Request methods and paths surface misconfigured health checks and bots hammering a single endpoint.

User agents and referers help separate browsers, mobile apps, and scrapers, but they are spoofable. Use them as hints alongside rate, path, and status patterns rather than as a sole security control.

Common mistakes include treating bytes as always present, ignoring timezone offsets in timestamps, and pasting multi-line error logs into a combined-format parser. Keep error logs separate from access logs when diagnosing application stack traces.

When should you parse access logs online? During on-call when you need a quick JSON view of a pasted sample, when teaching juniors how combined format works, and when validating a new log_format before rolling it fleet-wide. Prefer redacted samples.

Privacy note: access logs often include session tokens in query strings and identifiable paths. Redact before sharing. DevPipe parses in the browser so samples stay on your machine.