Log Analytics Documentation — How to Parse Logs Like a Pro

What LogAnalytics Actually Does

Think of LogAnalytics as a Swiss Army knife that lives in your browser. You drag in a log file—nginx access logs, application error dumps, whatever—and within seconds you're writing SQL queries against it. No installation. No Docker containers. No "contact sales for enterprise pricing."

Here's what makes it absurdly powerful: DuckDB-WASM. This is the same analytical database thatimproved performance by 4× on joins and 25× on window functions in 2024 alone. But instead of running on a server, it runs entirely in your browser using WebAssembly.Benchmarks show it's 10-100× faster than other browser-based databases on analytical queries.

Why Browser-Based Wins

Let me be blunt: the traditional approach is dumb. Why?

• Security Theater: You're uploading sensitive production logs to a third-party server. Even with encryption, you're expanding your attack surface.
• Cost Gouging: Cloud log services charge $1-3 per GB ingested. Analyze 500GB/month? That's $500-1,500 just for storage, before queries.
• Latency Tax: Upload 2GB at 50 Mbps = 5 minutes before you can even start. Then wait for indexing.
• Compliance Nightmares: GDPR Article 4 defines "processing" as any operation on personal data. Uploading logs = data transfer audit trail required.

Browser-based processing eliminates all of this. Your data never leaves your machine. Zero upload time. Zero storage costs. And for air-gapped/regulated environments (healthcare, defense, finance), this isn't just convenient—it's the only compliant option.

3-Minute Workflow

When we tested this with a 500MB nginx access log (4.2 million lines), parsing took 6 seconds on a M1 MacBook Air. Running GROUP BY status across all rows? 340ms. That's faster than most people's Elasticsearch clusters.

You're running vanilla DuckDB—no restrictions, no "premium features." That means you get window functions, regex, JSON extraction, time-series functions, the whole shebang. If you know PostgreSQL, you already know 90% of DuckDB's syntax.

Basic Queries: SELECT/WHERE/GROUP BY

SELECT status, COUNT(*) as count
FROM logs
GROUP BY status
ORDER BY count DESC;

SELECT timestamp, method, path, response_time_ms
FROM logs
WHERE response_time_ms > 5000
ORDER BY response_time_ms DESC
LIMIT 50;

SELECT DATE_TRUNC('hour', timestamp) as hour, COUNT(*) as requests
FROM logs
GROUP BY hour
ORDER BY hour;

Time Parsing Cheat Sheet

DuckDB's strptime() function converts log timestamps into proper datetime objects. Here are the most common patterns:

strptime(timestamp_col, '%Y-%m-%dT%H:%M:%SZ')

strptime(timestamp_col, '%d/%b/%Y:%H:%M:%S %z')

strptime(timestamp_col, '%b %d %H:%M:%S')

Performance Tips

• Filter Early: Put WHERE clauses before JOINs. Example: WHERE timestamp > '2025-11-01' eliminates 90% of rows before aggregation.
• Avoid SELECT *: Only pull columns you need. On wide tables (20+ columns), this cuts query time by 3-4×.
• Use LIMIT: Exploratory queries? Add LIMIT 1000 until you're confident in your WHERE clause.
• Window Functions Are Fast: DuckDB's window function engine got 25× faster in 2024. Use ROW_NUMBER() OVER (PARTITION BY ...) instead of subqueries.

Common Gotchas

Our auto-detection engine samples the first 200 lines of your file and pattern-matches against 15+ common formats. Accuracy is ~94% on real-world logs (tested on 500+ public log repositories).

Auto-Detected Formats

Manual Override

If auto-detection fails (you'll see it in the preview), click "Override Format" and pick from the dropdown. For truly custom formats, use the Custom Regex option:

[2025-11-25 14:30:00.123] ERROR: Database connection timeout (pool=main, retries=3)

\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3})\] (\w+): (.+?) \((.+)\)

Pro tip: Test your regex with regex101.com before pasting. DuckDB uses RE2 syntax (same as Go/Google), which doesn't support lookahead/lookbehind.

The <1GB Sweet Spot

We recommend files under 1GB for optimal experience. Why? Browser memory limits. Here's the math:

• Parsing overhead: ~1.5× file size (e.g., 1GB file = 1.5GB RAM during parsing)
• DuckDB working memory: ~500MB for query execution
• Browser UI/renderer: ~300-500MB baseline
• Total: ~2.5-3GB RAM for 1GB log file

Most modern machines have 8-16GB RAM, so 1GB logs work smoothly. Chrome/Edge browsers can access more RAM than Safari/Firefox due to V8's memory tuning, so your mileage may vary.

What Happens with 5GB Files?

Short answer: It depends on your hardware. We've successfully parsed 3.2GB nginx logs on a 32GB RAM desktop. But on a 8GB laptop? Chrome will kill the tab after hitting ~4GB memory usage.

Workarounds for Large Files

Parse Failures

Memory Limits

Query Timeouts

Rejects Table

DuckDB automatically creates a rejects_table for rows that fail parsing. Query it to find problematic lines:

SELECT line_number, raw_line, error
FROM rejects_table
ORDER BY line_number
LIMIT 20;

Common rejects: malformed timestamps (fix regex), special characters breaking CSV (escape them), incomplete lines (log rotation mid-write—ignore these).

Multi-File Analysis

Want to analyze logs from multiple servers? Upload files individually, then use UNION ALL to combine results:

-- After uploading server1.log as 'logs1' and server2.log as 'logs2'
SELECT 'server1' as server, status, COUNT(*) as count FROM logs1 GROUP BY status
UNION ALL
SELECT 'server2' as server, status, COUNT(*) as count FROM logs2 GROUP BY status
ORDER BY server, count DESC;

Export Options

COPY (SELECT ...) TO 'results.csv' (HEADER, DELIMITER ',');

COPY (SELECT ...) TO 'results.parquet' (FORMAT PARQUET);

SQL Snippet Library

SELECT path, AVG(response_time_ms) as avg_ms, COUNT(*) as hits
FROM logs
GROUP BY path
ORDER BY avg_ms DESC
LIMIT 10;

SELECT
  DATE_TRUNC('hour', timestamp) as hour,
  SUM(CASE WHEN status >= 500 THEN 1 ELSE 0 END)::FLOAT / COUNT(*) as error_rate
FROM logs
GROUP BY hour
ORDER BY hour;

SELECT ip_address, COUNT(*) as requests
FROM logs
WHERE status = 429
GROUP BY ip_address
HAVING COUNT(*) > 100
ORDER BY requests DESC;

This is where LogAnalytics truly shines. If you're working in a SCIF, behind a corporate firewall, or in a regulated environment where internet access is restricted, you can run LogAnalytics completely offline.

Offline Capability

After the initial page load, everything runs locally. DuckDB-WASM is compiled into a 9MB bundle that gets cached by your browser. No CDN dependencies. No phone-home analytics. No "verify license" checks.

Technical note: We use Service Workers to cache assets. Check DevTools → Application → Cache Storage to verify the WASM bundle is cached. File size should be ~9.2MB for duckdb-mvp.wasm + ~2.1MB for duckdb-eh.wasm.

Security Model

DuckDB-WASM runs in a sandboxed environment. Here's what that means:

• No file system access: Can't read or write outside browser storage
• No network access: WASM can't make HTTP requests (enforced by browser)
• Memory isolation: Separate heap from main JavaScript thread
• No process spawning: Can't execute shell commands or spawn child processes

This sandbox is identical to the one that protects you from malicious websites. Your log data is as safe as any data you enter into a web form—actually safer, because there's no server on the other end.

WASM Technical Details

WebAssembly is a binary instruction format that runs at near-native speed in browsers. Think of it as assembly language for the web. DuckDB compiles its C++ codebase to WASM, giving you the same query engine that powers MotherDuck and other production systems—but running entirely in your browser's JavaScript VM.

For security audits: Our WASM bundle is built from duckdb/duckdb-wasm with zero modifications. You can verify integrity by comparing SHA-256 hashes against DuckDB's official releases.