Log analysis

In short A log is a journal: the system records what happened (who logged in, what request arrived, what error occurred). To a defender, logs are like CCTV footage — the trail of an attack shows up in them. Log analysis is the skill of finding, in a stream of thousands of lines, the few that matter.

Single log lines add up to the timeline of an attack.

Why defenders need logs

An attacker leaves traces at almost every step (defensive security): a password brute-force is hundreds of failed-login records; directory brute-forcing is hundreds of 404s; an intruder's entry is a successful authentication at a strange hour. Logs are where those traces live. Without analysing them, defence is blind.

Kinds of logs and where to look

Category	Examples	Where (Linux / Windows)
System	logins, services, sudo	`/var/log/auth.log`, `/var/log/syslog` / Windows Event Log
Web server	requests, errors	`/var/log/apache2/access.log`, `error.log`
Security	firewall, IDS, antivirus	depends on the product
Applications	databases, services	their own files and journals

A record is usually: time — source — what happened. A web-log line: IP, date, request method and path (how HTTP works), response code, User-Agent.

Reading logs on the command line

Logs are text, so the ordinary Linux tools work (Linux commands basics):

tail -f /var/log/auth.log            # follow in real time
grep "Failed password" auth.log      # only failed logins
grep "Failed password" auth.log | grep -oE "([0-9]{1,3}\.){3}[0-9]{1,3}" \
  | sort | uniq -c | sort -nr        # top IPs by failures → brute force
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head   # top IPs on the web

grep — filter by a marker (IP, user, response code).
cut / awk — pull out a field (just the IP, or just the URL).
sort | uniq -c — count repeats. This is the key: an anomaly is almost always "suddenly a lot of something."

What to look for — signs of an attack

A spike of failed logins from one IP → password brute-force (Hydra).
Many 404s and odd paths from one address → directory brute-forcing (directory brute-forcing).
A successful login at an unusual time or from an unusual country, especially after a run of failures → the brute-force may have worked.
Suspicious commands, surges in traffic volume, access to things normally never touched.

The timeline — an incident in order

Individual lines are arranged into a sequence in time: recon → login attempts → successful login → actions inside. That reconstructs the whole attack — the heart of an investigation (DFIR).

Where it scales — SIEM

Reading one machine's logs by hand is doable. Hundreds of machines' logs are not. That's what a SIEM is for: it collects logs from everywhere into one place, correlates them and raises alerts by rules on its own. Reading logs by hand is the foundation that understanding a SIEM stands on.