378 lines
15 KiB
Markdown
378 lines
15 KiB
Markdown
# 🔍 Log Monitor — User Guide
|
|
|
|
## Overview
|
|
|
|
The Log Monitor scans log files on a schedule, matches regex patterns, and sends alerts via Telegram, Webhooks, Microsoft Teams, or Email. It supports time-based filtering, scheduled scans (daily, hourly, continuous), and automatic attachment of full results when matches exceed a threshold.
|
|
|
|
---
|
|
|
|
## Starting the Monitor
|
|
|
|
### Option 1: Environment File
|
|
```bash
|
|
cp .env.monitor .env
|
|
node dist-server/server.js
|
|
```
|
|
|
|
### Option 2: Inline
|
|
```bash
|
|
LOG_MODE=monitor PORT=8082 MONITOR_SCAN_INTERVAL=30000 node dist-server/server.js
|
|
```
|
|
|
|
### Option 3: Docker
|
|
```bash
|
|
docker run -d -p 8082:8080 \
|
|
-e LOG_MODE=monitor \
|
|
-e MONITOR_SCAN_INTERVAL=30000 \
|
|
-v /var/log:/logs:ro \
|
|
logviewer
|
|
```
|
|
|
|
### Option 4: Config File
|
|
```json
|
|
{
|
|
"mode": "monitor",
|
|
"port": 8082,
|
|
"monitor": {
|
|
"enabled": true,
|
|
"scanIntervalMs": 30000,
|
|
"rules": []
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## How It Works
|
|
|
|
```
|
|
┌─────────────┐ ┌──────────────┐ ┌─────────────────┐
|
|
│ Log Files │────▶│ Scan Engine │────▶│ Pattern Match │
|
|
│ (sources) │ │ (interval) │ │ (regex) │
|
|
└─────────────┘ └──────────────┘ └────────┬────────┘
|
|
│
|
|
matches found?
|
|
│
|
|
┌──────────────┴──────────────┐
|
|
│ │
|
|
≤ maxLines > maxLines
|
|
│ │
|
|
┌─────▼─────┐ ┌───────▼───────┐
|
|
│ Inline │ │ Inline (N) │
|
|
│ in alert │ │ + .gz attach │
|
|
└─────┬─────┘ └───────┬───────┘
|
|
│ │
|
|
└──────────────┬──────────────┘
|
|
│
|
|
┌──────────────▼──────────────┐
|
|
│ Send Notifications │
|
|
│ Telegram │ Webhook │ Teams │
|
|
└─────────────────────────────┘
|
|
```
|
|
|
|
1. **Scan** — Every N seconds (configurable), the monitor reads all source files
|
|
2. **Filter** — Only files matching the time range are scanned (today, this week, etc.)
|
|
3. **Match** — Each line is tested against the configured regex patterns
|
|
4. **Alert** — If matches are found AND cooldown has expired, notifications are sent
|
|
5. **Attach** — If matches exceed `maxLinesInMessage`, full results are gzipped and attached
|
|
|
|
---
|
|
|
|
## GUI Guide
|
|
|
|
### Layout
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ 🔍 Log Monitor 2 active rules · 3 alerts│
|
|
├──────────────┬──────────────────────────────────────────────┤
|
|
│ │ │
|
|
│ 📋 Rules │ Rule Editor / Alert Detail │
|
|
│ ┌──────────┐│ │
|
|
│ │●Root DB ││ [General] │
|
|
│ │ 2 src ││ Name: Root DB Access Detection │
|
|
│ │ 6 match ││ Enabled: Yes │
|
|
│ ├──────────┤│ │
|
|
│ │●Failed ││ [Sources] │
|
|
│ │ 1 src ││ /var/log/db-audit.log │
|
|
│ │ 4 match ││ /var/log/db-audit-*.log.gz │
|
|
│ └──────────┘│ │
|
|
│ │ [Patterns] │
|
|
│ 📊 Alerts │ root.*login │
|
|
│ ┌──────────┐│ GRANT.*PRIVILEGES │
|
|
│ │🚨 Root DB││ DROP TABLE │
|
|
│ │ 6 matches││ │
|
|
│ │ 10:30:01 ││ [Schedule] │
|
|
│ ├──────────┤│ Mode: Daily at 07:00 │
|
|
│ │🚨 Failed ││ │
|
|
│ │ 4 matches││ [Notifications] │
|
|
│ │ 10:30:01 ││ [TELEGRAM] Bot: 811... Chat: 859... │
|
|
│ └──────────┘│ [WEBHOOK] https://hooks.slack.com/... │
|
|
│ │ │
|
|
│ │ [💾 Save] [🧪 Test Now] [🗑 Delete] │
|
|
└──────────────┴──────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Creating a Rule
|
|
|
|
1. Click **"+ New"** in the sidebar
|
|
2. Fill in the fields:
|
|
- **Name** — descriptive name for the rule
|
|
- **Sources** — one file path per line (supports globs like `/var/log/*.log`)
|
|
- **Time Range** — which files to scan (today, week, month, last N hours, all)
|
|
- **Patterns** — one regex per line
|
|
- **Match Logic** — "any" (OR) or "all" (AND)
|
|
- **Schedule** — continuous, daily at time, or every N hours
|
|
- **Max Lines** — how many lines to include inline in notifications
|
|
- **Cooldown** — minimum time between alerts (prevents spam)
|
|
3. Add notification channels
|
|
4. Click **"💾 Save Rule"**
|
|
|
|
### Testing a Rule
|
|
|
|
1. Select a rule from the sidebar
|
|
2. Click **"🧪 Test Now"**
|
|
3. The rule runs immediately and shows results below the form
|
|
4. This does NOT respect cooldown — always runs
|
|
|
|
### Testing Notifications
|
|
|
|
1. In the Notifications section, configure a channel
|
|
2. Click **"🧪 Test Notification"**
|
|
3. A test message is sent to verify the channel works
|
|
|
|
### Viewing Alerts
|
|
|
|
- Alerts appear in the **"📊 Recent Alerts"** section of the sidebar
|
|
- Click an alert to see the full detail view with all matched lines
|
|
- Lines are color-coded: red for ERROR/FATAL, orange for WARN
|
|
|
|
---
|
|
|
|
## Configuration Reference
|
|
|
|
### Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `LOG_MODE` | `both` | Set to `monitor` |
|
|
| `PORT` | `8080` | HTTP port |
|
|
| `MONITOR_ENABLED` | `true` | Enable/disable scanning |
|
|
| `MONITOR_SCAN_INTERVAL` | `60000` | Scan interval in ms |
|
|
| `MONITOR_STORAGE_DIR` | `./monitor-data` | Where to store state |
|
|
| `MONITOR_RULES` | `[]` | JSON array of rules (see below) |
|
|
|
|
### Rule Structure
|
|
|
|
```json
|
|
{
|
|
"id": "unique-id",
|
|
"name": "Human-readable name",
|
|
"enabled": true,
|
|
"sources": ["/path/to/logs/*.log"],
|
|
"patterns": ["regex1", "regex2"],
|
|
"patternLogic": "any",
|
|
"timeRange": { "type": "week", "hours": 24 },
|
|
"schedule": { "mode": "daily", "value": "07:00", "timezone": "Europe/Berlin" },
|
|
"notifications": [
|
|
{ "type": "telegram", "botToken": "123:ABC", "chatId": "-100123" }
|
|
],
|
|
"maxLinesInMessage": 20,
|
|
"cooldownMs": 300000,
|
|
"matchCount": 0,
|
|
"lastTriggered": null
|
|
}
|
|
```
|
|
|
|
### Time Range Options
|
|
|
|
| Type | Description | Use Case |
|
|
|------|-------------|----------|
|
|
| `today` | Only files modified today | Daily reports |
|
|
| `week` | Files from last 7 days | Weekly audits |
|
|
| `month` | Files from last 30 days | Monthly reviews |
|
|
| `hours` | Files from last N hours | Recent activity |
|
|
| `all` | All files regardless of age | Full history scan |
|
|
|
|
### Schedule Options
|
|
|
|
| Mode | Value | Description |
|
|
|------|-------|-------------|
|
|
| `interval` | — | Runs every scan interval (continuous monitoring) |
|
|
| `daily` | `07:00` | Runs once per day at the specified time |
|
|
| `hourly` | `2` | Runs every N hours |
|
|
|
|
### Notification Channels
|
|
|
|
**Telegram:**
|
|
```json
|
|
{ "type": "telegram", "botToken": "123456:ABC-DEF...", "chatId": "8599028500" }
|
|
```
|
|
|
|
**Webhook (Slack, Discord, custom):**
|
|
```json
|
|
{ "type": "webhook", "url": "https://hooks.slack.com/services/T.../B.../..." }
|
|
```
|
|
|
|
**Microsoft Teams:**
|
|
```json
|
|
{ "type": "teams", "url": "https://outlook.office.com/webhook/..." }
|
|
```
|
|
|
|
**Email (SMTP):**
|
|
```json
|
|
{ "type": "email", "to": "admin@company.com", "smtpHost": "smtp.gmail.com", "smtpPort": 587, "smtpUser": "alerts@company.com", "smtpPass": "app-password" }
|
|
```
|
|
|
|
---
|
|
|
|
## Example Configurations
|
|
|
|
### 1. Real-time Root Login Detection
|
|
|
|
Scans every 10 seconds, alerts immediately via Telegram:
|
|
```env
|
|
LOG_MODE=monitor
|
|
PORT=8082
|
|
MONITOR_SCAN_INTERVAL=10000
|
|
MONITOR_RULES=[{"id":"root-login","name":"Root Login Alert","enabled":true,"sources":["/var/log/auth.log"],"patterns":["session opened for user root","su.*root","sudo.*root"],"patternLogic":"any","timeRange":{"type":"today"},"schedule":{"mode":"interval"},"notifications":[{"type":"telegram","botToken":"YOUR_BOT_TOKEN","chatId":"YOUR_CHAT_ID"}],"maxLinesInMessage":10,"cooldownMs":60000,"matchCount":0,"lastTriggered":null}]
|
|
```
|
|
|
|
### 2. Daily DB Audit Report at 7:00 AM
|
|
|
|
Scans once per day, sends full report:
|
|
```env
|
|
LOG_MODE=monitor
|
|
PORT=8082
|
|
MONITOR_SCAN_INTERVAL=60000
|
|
MONITOR_RULES=[{"id":"db-audit","name":"Daily DB Audit","enabled":true,"sources":["/var/log/postgresql/audit.log","/var/log/mysql/audit*.log"],"patterns":["admin.*LOGIN","GRANT","DROP","ALTER.*TABLE","CREATE USER"],"patternLogic":"any","timeRange":{"type":"today"},"schedule":{"mode":"daily","value":"07:00","timezone":"Europe/Berlin"},"notifications":[{"type":"telegram","botToken":"TOKEN","chatId":"CHAT_ID"},{"type":"webhook","url":"https://hooks.slack.com/services/..."}],"maxLinesInMessage":20,"cooldownMs":86400000,"matchCount":0,"lastTriggered":null}]
|
|
```
|
|
|
|
### 3. Error Spike Detection (every 5 minutes)
|
|
|
|
```env
|
|
LOG_MODE=monitor
|
|
PORT=8082
|
|
MONITOR_SCAN_INTERVAL=30000
|
|
MONITOR_RULES=[{"id":"error-spike","name":"Error Spike","enabled":true,"sources":["/app/logs/production.log"],"patterns":["\\bERROR\\b","\\bFATAL\\b","Exception","panic:"],"patternLogic":"any","timeRange":{"type":"hours","hours":1},"schedule":{"mode":"hourly","value":"0.083"},"notifications":[{"type":"teams","url":"https://outlook.office.com/webhook/..."}],"maxLinesInMessage":30,"cooldownMs":300000,"matchCount":0,"lastTriggered":null}]
|
|
```
|
|
|
|
### 4. Security: Unauthorized Access Attempts
|
|
|
|
```env
|
|
MONITOR_RULES=[{"id":"unauth","name":"Unauthorized Access","enabled":true,"sources":["/var/log/nginx/access.log","/var/log/nginx/error.log"],"patterns":["403 Forbidden","401 Unauthorized","\\.\\./\\.\\./","etc/passwd","<script>"],"patternLogic":"any","timeRange":{"type":"today"},"schedule":{"mode":"interval"},"notifications":[{"type":"telegram","botToken":"TOKEN","chatId":"CHAT"},{"type":"webhook","url":"https://siem.internal/alerts"}],"maxLinesInMessage":15,"cooldownMs":120000,"matchCount":0,"lastTriggered":null}]
|
|
```
|
|
|
|
### 5. Multiple Rules via Config File
|
|
|
|
```json
|
|
{
|
|
"mode": "monitor",
|
|
"port": 8082,
|
|
"monitor": {
|
|
"enabled": true,
|
|
"scanIntervalMs": 30000,
|
|
"rules": [
|
|
{
|
|
"id": "root-access",
|
|
"name": "Root DB Access",
|
|
"enabled": true,
|
|
"sources": ["/var/log/db-audit.log"],
|
|
"patterns": ["root.*login", "GRANT", "DROP TABLE"],
|
|
"patternLogic": "any",
|
|
"timeRange": { "type": "today" },
|
|
"schedule": { "mode": "interval" },
|
|
"notifications": [
|
|
{ "type": "telegram", "botToken": "TOKEN", "chatId": "CHAT" }
|
|
],
|
|
"maxLinesInMessage": 20,
|
|
"cooldownMs": 300000
|
|
},
|
|
{
|
|
"id": "disk-space",
|
|
"name": "Disk Space Warning",
|
|
"enabled": true,
|
|
"sources": ["/var/log/syslog"],
|
|
"patterns": ["No space left", "disk.*9[5-9]%", "filesystem full"],
|
|
"patternLogic": "any",
|
|
"timeRange": { "type": "hours", "hours": 2 },
|
|
"schedule": { "mode": "hourly", "value": "1" },
|
|
"notifications": [
|
|
{ "type": "webhook", "url": "https://hooks.slack.com/..." }
|
|
],
|
|
"maxLinesInMessage": 5,
|
|
"cooldownMs": 3600000
|
|
}
|
|
]
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## API Reference
|
|
|
|
| Method | Path | Description |
|
|
|--------|------|-------------|
|
|
| GET | `/api/monitor/rules` | List all rules |
|
|
| POST | `/api/monitor/rules` | Create a new rule |
|
|
| PUT | `/api/monitor/rule/:id` | Update a rule |
|
|
| DELETE | `/api/monitor/rule/:id` | Delete a rule |
|
|
| POST | `/api/monitor/trigger/:id` | Manually trigger a rule (ignores cooldown) |
|
|
| GET | `/api/monitor/results` | Get recent alert results |
|
|
| GET | `/api/monitor/config` | Get monitor configuration |
|
|
| POST | `/api/monitor/test-notification` | Send a test notification |
|
|
|
|
### Automating Rule Creation
|
|
|
|
You can fully automate monitor setup via the API:
|
|
```bash
|
|
# Create a rule
|
|
curl -X POST http://localhost:8082/api/monitor/rules \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"name": "My Alert",
|
|
"enabled": true,
|
|
"sources": ["/var/log/app.log"],
|
|
"patterns": ["ERROR", "FATAL"],
|
|
"patternLogic": "any",
|
|
"timeRange": {"type": "today"},
|
|
"notifications": [{"type": "telegram","botToken":"...","chatId":"..."}],
|
|
"maxLinesInMessage": 20,
|
|
"cooldownMs": 300000
|
|
}'
|
|
|
|
# Trigger it manually
|
|
curl -X POST http://localhost:8082/api/monitor/trigger/my-alert
|
|
|
|
# Check results
|
|
curl http://localhost:8082/api/monitor/results
|
|
```
|
|
|
|
---
|
|
|
|
## Alert Message Format
|
|
|
|
When a rule triggers, the notification message looks like:
|
|
|
|
```
|
|
🚨 Log Monitor Alert: Root DB Access Detection
|
|
|
|
Matches: 6 lines
|
|
Sources: /var/log/db-audit.log
|
|
Time: 2026-04-30T07:00:01.234Z
|
|
Patterns: root.*login | GRANT.*PRIVILEGES | DROP TABLE
|
|
|
|
Sample (first 5 of 6):
|
|
2026-04-30T06:30:01Z WARN [db] root login detected from 10.0.0.1
|
|
2026-04-30T06:30:05Z WARN [db] root executed: GRANT ALL PRIVILEGES
|
|
2026-04-30T06:30:06Z ERROR [db] CRITICAL: root executed: DROP TABLE audit_log
|
|
2026-04-30T07:05:35Z WARN [db] root login detected from 192.168.1.51
|
|
2026-04-30T07:05:40Z INFO [db] root executed: ALTER TABLE users
|
|
|
|
📎 Full results (6 lines) attached as .gz file.
|
|
```
|
|
|
|
If total matches ≤ `maxLinesInMessage`: all lines are included inline.
|
|
If total matches > `maxLinesInMessage`: first N lines inline + full results as gzipped attachment.
|