Keviin/LogMaster

Fork 0

Files

Kevin 44cd9ab001

LogMaster CI/CD / build-and-test (push) Has been cancelled

Details

LogMaster CI/CD / docker (push) Has been cancelled

Details

Initial commit: LogMaster with Viewer Collector Monitor modes

2026-05-04 13:50:15 +02:00

15 KiB

Raw Blame History

🔍 Log Monitor — User Guide

Overview

The Log Monitor scans log files on a schedule, matches regex patterns, and sends alerts via Telegram, Webhooks, Microsoft Teams, or Email. It supports time-based filtering, scheduled scans (daily, hourly, continuous), and automatic attachment of full results when matches exceed a threshold.

Starting the Monitor

Option 1: Environment File

cp .env.monitor .env
node dist-server/server.js

Option 2: Inline

LOG_MODE=monitor PORT=8082 MONITOR_SCAN_INTERVAL=30000 node dist-server/server.js

Option 3: Docker

docker run -d -p 8082:8080 \
  -e LOG_MODE=monitor \
  -e MONITOR_SCAN_INTERVAL=30000 \
  -v /var/log:/logs:ro \
  logviewer

Option 4: Config File

{
  "mode": "monitor",
  "port": 8082,
  "monitor": {
    "enabled": true,
    "scanIntervalMs": 30000,
    "rules": []
  }
}

How It Works

┌─────────────┐     ┌──────────────┐     ┌─────────────────┐
│  Log Files  │────▶│  Scan Engine │────▶│  Pattern Match  │
│  (sources)  │     │  (interval)  │     │  (regex)        │
└─────────────┘     └──────────────┘     └────────┬────────┘
                                                   │
                                          matches found?
                                                   │
                                    ┌──────────────┴──────────────┐
                                    │                             │
                              ≤ maxLines                    > maxLines
                                    │                             │
                              ┌─────▼─────┐              ┌───────▼───────┐
                              │  Inline   │              │  Inline (N)   │
                              │  in alert │              │  + .gz attach │
                              └─────┬─────┘              └───────┬───────┘
                                    │                             │
                                    └──────────────┬──────────────┘
                                                   │
                                    ┌──────────────▼──────────────┐
                                    │      Send Notifications     │
                                    │  Telegram │ Webhook │ Teams │
                                    └─────────────────────────────┘

Scan — Every N seconds (configurable), the monitor reads all source files
Filter — Only files matching the time range are scanned (today, this week, etc.)
Match — Each line is tested against the configured regex patterns
Alert — If matches are found AND cooldown has expired, notifications are sent
Attach — If matches exceed maxLinesInMessage, full results are gzipped and attached

GUI Guide

Layout

┌─────────────────────────────────────────────────────────────┐
│  🔍 Log Monitor                    2 active rules · 3 alerts│
├──────────────┬──────────────────────────────────────────────┤
│              │                                              │
│  📋 Rules    │         Rule Editor / Alert Detail           │
│  ┌──────────┐│                                              │
│  │●Root DB  ││  [General]                                   │
│  │ 2 src    ││  Name: Root DB Access Detection              │
│  │ 6 match  ││  Enabled: Yes                                │
│  ├──────────┤│                                              │
│  │●Failed   ││  [Sources]                                   │
│  │ 1 src    ││  /var/log/db-audit.log                       │
│  │ 4 match  ││  /var/log/db-audit-*.log.gz                  │
│  └──────────┘│                                              │
│              │  [Patterns]                                   │
│  📊 Alerts   │  root.*login                                  │
│  ┌──────────┐│  GRANT.*PRIVILEGES                           │
│  │🚨 Root DB││  DROP TABLE                                   │
│  │ 6 matches││                                              │
│  │ 10:30:01 ││  [Schedule]                                   │
│  ├──────────┤│  Mode: Daily at 07:00                        │
│  │🚨 Failed ││                                              │
│  │ 4 matches││  [Notifications]                              │
│  │ 10:30:01 ││  [TELEGRAM] Bot: 811...  Chat: 859...        │
│  └──────────┘│  [WEBHOOK] https://hooks.slack.com/...       │
│              │                                              │
│              │  [💾 Save] [🧪 Test Now] [🗑 Delete]          │
└──────────────┴──────────────────────────────────────────────┘

Creating a Rule

Click "+ New" in the sidebar
Fill in the fields:
- Name — descriptive name for the rule
- Sources — one file path per line (supports globs like /var/log/*.log)
- Time Range — which files to scan (today, week, month, last N hours, all)
- Patterns — one regex per line
- Match Logic — "any" (OR) or "all" (AND)
- Schedule — continuous, daily at time, or every N hours
- Max Lines — how many lines to include inline in notifications
- Cooldown — minimum time between alerts (prevents spam)
Add notification channels
Click "💾 Save Rule"

Testing a Rule

Select a rule from the sidebar
Click "🧪 Test Now"
The rule runs immediately and shows results below the form
This does NOT respect cooldown — always runs

Testing Notifications

In the Notifications section, configure a channel
Click "🧪 Test Notification"
A test message is sent to verify the channel works

Viewing Alerts

Alerts appear in the "📊 Recent Alerts" section of the sidebar
Click an alert to see the full detail view with all matched lines
Lines are color-coded: red for ERROR/FATAL, orange for WARN

Configuration Reference

Environment Variables

Variable	Default	Description
`LOG_MODE`	`both`	Set to `monitor`
`PORT`	`8080`	HTTP port
`MONITOR_ENABLED`	`true`	Enable/disable scanning
`MONITOR_SCAN_INTERVAL`	`60000`	Scan interval in ms
`MONITOR_STORAGE_DIR`	`./monitor-data`	Where to store state
`MONITOR_RULES`	`[]`	JSON array of rules (see below)

Rule Structure

{
  "id": "unique-id",
  "name": "Human-readable name",
  "enabled": true,
  "sources": ["/path/to/logs/*.log"],
  "patterns": ["regex1", "regex2"],
  "patternLogic": "any",
  "timeRange": { "type": "week", "hours": 24 },
  "schedule": { "mode": "daily", "value": "07:00", "timezone": "Europe/Berlin" },
  "notifications": [
    { "type": "telegram", "botToken": "123:ABC", "chatId": "-100123" }
  ],
  "maxLinesInMessage": 20,
  "cooldownMs": 300000,
  "matchCount": 0,
  "lastTriggered": null
}

Time Range Options

Type	Description	Use Case
`today`	Only files modified today	Daily reports
`week`	Files from last 7 days	Weekly audits
`month`	Files from last 30 days	Monthly reviews
`hours`	Files from last N hours	Recent activity
`all`	All files regardless of age	Full history scan

Schedule Options

Mode	Value	Description
`interval`	—	Runs every scan interval (continuous monitoring)
`daily`	`07:00`	Runs once per day at the specified time
`hourly`	`2`	Runs every N hours

Notification Channels

Telegram:

{ "type": "telegram", "botToken": "123456:ABC-DEF...", "chatId": "8599028500" }

Webhook (Slack, Discord, custom):

{ "type": "webhook", "url": "https://hooks.slack.com/services/T.../B.../..." }

Microsoft Teams:

{ "type": "teams", "url": "https://outlook.office.com/webhook/..." }

Email (SMTP):

{ "type": "email", "to": "admin@company.com", "smtpHost": "smtp.gmail.com", "smtpPort": 587, "smtpUser": "alerts@company.com", "smtpPass": "app-password" }

Example Configurations

Scans every 10 seconds, alerts immediately via Telegram:

LOG_MODE=monitor
PORT=8082
MONITOR_SCAN_INTERVAL=10000
MONITOR_RULES=[{"id":"root-login","name":"Root Login Alert","enabled":true,"sources":["/var/log/auth.log"],"patterns":["session opened for user root","su.*root","sudo.*root"],"patternLogic":"any","timeRange":{"type":"today"},"schedule":{"mode":"interval"},"notifications":[{"type":"telegram","botToken":"YOUR_BOT_TOKEN","chatId":"YOUR_CHAT_ID"}],"maxLinesInMessage":10,"cooldownMs":60000,"matchCount":0,"lastTriggered":null}]

2. Daily DB Audit Report at 7:00 AM

Scans once per day, sends full report:

LOG_MODE=monitor
PORT=8082
MONITOR_SCAN_INTERVAL=60000
MONITOR_RULES=[{"id":"db-audit","name":"Daily DB Audit","enabled":true,"sources":["/var/log/postgresql/audit.log","/var/log/mysql/audit*.log"],"patterns":["admin.*LOGIN","GRANT","DROP","ALTER.*TABLE","CREATE USER"],"patternLogic":"any","timeRange":{"type":"today"},"schedule":{"mode":"daily","value":"07:00","timezone":"Europe/Berlin"},"notifications":[{"type":"telegram","botToken":"TOKEN","chatId":"CHAT_ID"},{"type":"webhook","url":"https://hooks.slack.com/services/..."}],"maxLinesInMessage":20,"cooldownMs":86400000,"matchCount":0,"lastTriggered":null}]

3. Error Spike Detection (every 5 minutes)

LOG_MODE=monitor
PORT=8082
MONITOR_SCAN_INTERVAL=30000
MONITOR_RULES=[{"id":"error-spike","name":"Error Spike","enabled":true,"sources":["/app/logs/production.log"],"patterns":["\\bERROR\\b","\\bFATAL\\b","Exception","panic:"],"patternLogic":"any","timeRange":{"type":"hours","hours":1},"schedule":{"mode":"hourly","value":"0.083"},"notifications":[{"type":"teams","url":"https://outlook.office.com/webhook/..."}],"maxLinesInMessage":30,"cooldownMs":300000,"matchCount":0,"lastTriggered":null}]

4. Security: Unauthorized Access Attempts

MONITOR_RULES=[{"id":"unauth","name":"Unauthorized Access","enabled":true,"sources":["/var/log/nginx/access.log","/var/log/nginx/error.log"],"patterns":["403 Forbidden","401 Unauthorized","\\.\\./\\.\\./","etc/passwd","<script>"],"patternLogic":"any","timeRange":{"type":"today"},"schedule":{"mode":"interval"},"notifications":[{"type":"telegram","botToken":"TOKEN","chatId":"CHAT"},{"type":"webhook","url":"https://siem.internal/alerts"}],"maxLinesInMessage":15,"cooldownMs":120000,"matchCount":0,"lastTriggered":null}]

5. Multiple Rules via Config File

{
  "mode": "monitor",
  "port": 8082,
  "monitor": {
    "enabled": true,
    "scanIntervalMs": 30000,
    "rules": [
      {
        "id": "root-access",
        "name": "Root DB Access",
        "enabled": true,
        "sources": ["/var/log/db-audit.log"],
        "patterns": ["root.*login", "GRANT", "DROP TABLE"],
        "patternLogic": "any",
        "timeRange": { "type": "today" },
        "schedule": { "mode": "interval" },
        "notifications": [
          { "type": "telegram", "botToken": "TOKEN", "chatId": "CHAT" }
        ],
        "maxLinesInMessage": 20,
        "cooldownMs": 300000
      },
      {
        "id": "disk-space",
        "name": "Disk Space Warning",
        "enabled": true,
        "sources": ["/var/log/syslog"],
        "patterns": ["No space left", "disk.*9[5-9]%", "filesystem full"],
        "patternLogic": "any",
        "timeRange": { "type": "hours", "hours": 2 },
        "schedule": { "mode": "hourly", "value": "1" },
        "notifications": [
          { "type": "webhook", "url": "https://hooks.slack.com/..." }
        ],
        "maxLinesInMessage": 5,
        "cooldownMs": 3600000
      }
    ]
  }
}

API Reference

Method	Path	Description
GET	`/api/monitor/rules`	List all rules
POST	`/api/monitor/rules`	Create a new rule
PUT	`/api/monitor/rule/:id`	Update a rule
DELETE	`/api/monitor/rule/:id`	Delete a rule
POST	`/api/monitor/trigger/:id`	Manually trigger a rule (ignores cooldown)
GET	`/api/monitor/results`	Get recent alert results
GET	`/api/monitor/config`	Get monitor configuration
POST	`/api/monitor/test-notification`	Send a test notification

Automating Rule Creation

You can fully automate monitor setup via the API:

# Create a rule
curl -X POST http://localhost:8082/api/monitor/rules \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Alert",
    "enabled": true,
    "sources": ["/var/log/app.log"],
    "patterns": ["ERROR", "FATAL"],
    "patternLogic": "any",
    "timeRange": {"type": "today"},
    "notifications": [{"type": "telegram","botToken":"...","chatId":"..."}],
    "maxLinesInMessage": 20,
    "cooldownMs": 300000
  }'

# Trigger it manually
curl -X POST http://localhost:8082/api/monitor/trigger/my-alert

# Check results
curl http://localhost:8082/api/monitor/results

Alert Message Format

When a rule triggers, the notification message looks like:

🚨 Log Monitor Alert: Root DB Access Detection

Matches: 6 lines
Sources: /var/log/db-audit.log
Time: 2026-04-30T07:00:01.234Z
Patterns: root.*login | GRANT.*PRIVILEGES | DROP TABLE

Sample (first 5 of 6):
  2026-04-30T06:30:01Z WARN [db] root login detected from 10.0.0.1
  2026-04-30T06:30:05Z WARN [db] root executed: GRANT ALL PRIVILEGES
  2026-04-30T06:30:06Z ERROR [db] CRITICAL: root executed: DROP TABLE audit_log
  2026-04-30T07:05:35Z WARN [db] root login detected from 192.168.1.51
  2026-04-30T07:05:40Z INFO [db] root executed: ALTER TABLE users

📎 Full results (6 lines) attached as .gz file.

If total matches ≤ maxLinesInMessage: all lines are included inline. If total matches > maxLinesInMessage: first N lines inline + full results as gzipped attachment.

15 KiB Raw Blame History