131 lines
5.4 KiB
Markdown
131 lines
5.4 KiB
Markdown
# Daily AI Log Summary
|
|
|
|
A lightweight, automated tool that analyzes system logs—specifically optimized for **Postfix** and **Dovecot** mail servers—extracts key security and operational facts, and generates a concise daily summary using a local AI model (Ollama).
|
|
|
|
## Features
|
|
|
|
- **Structured Log Analysis**: Deterministically extracts facts from `journalctl` logs (Postfix/postscreen rejections, Dovecot logins, relay abuse, SPF/DKIM anomalies, etc.).
|
|
- **AI Summarization**: Uses Ollama (e.g., `llama3.2:1b`) to synthesize raw facts into a readable daily report.
|
|
- **Repeat Offender Tracking**: Persistently tracks suspicious IPs across multiple days to identify long-term probing.
|
|
- **Automated Emailing**: Sends the summary and raw facts via SMTP.
|
|
- **Dockerized**: Easy deployment with a minimal footprint.
|
|
- **Systemd Integration**: Includes an example unit for robust "always-on" operation.
|
|
|
|
## How It Works
|
|
|
|
1. **Fetch**: Retrieves yesterday's logs for a specific systemd unit using `journalctl`.
|
|
2. **Analyze**: Python regex-based extraction identifies successful logins, authentication failures, blocked relay attempts, and more.
|
|
3. **Track**: Suspicious IPs are stored in a persistent state file. If an IP appears on multiple days, it's flagged as a "repeat offender."
|
|
4. **Summarize**: Raw facts are fed to a local Ollama instance with a specific system prompt to generate a concise narrative.
|
|
5. **Report**: An email is sent to the administrator with the AI summary and the full list of extracted facts.
|
|
|
|
## Prerequisites
|
|
|
|
- **Docker**: For running the analyzer.
|
|
- **Ollama**: A running Ollama instance (locally or accessible via network).
|
|
- **Systemd Journals**: The host must use systemd journals (the container mounts these as read-only).
|
|
|
|
## Ollama Setup
|
|
|
|
This tool requires an Ollama server to handle the AI summarization.
|
|
|
|
### 1. Install Ollama
|
|
If you don't have Ollama installed, you can install it on Linux with:
|
|
```bash
|
|
curl -fsSL https://ollama.com/install.sh | sh
|
|
```
|
|
Alternatively, you can run Ollama as a Docker container:
|
|
```bash
|
|
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
|
|
```
|
|
|
|
### 2. Select and Pull a Model
|
|
The script defaults to `llama3.2:1b`, which is lightweight and fast for this task. You must pull the model before the script can use it:
|
|
```bash
|
|
ollama pull llama3.2:1b
|
|
```
|
|
You can choose other models (like `llama3`, `mistral`, `gemma`) by pulling them and updating the `OLLAMA_MODEL` variable in your `.env` file.
|
|
|
|
### 3. Run the Server
|
|
If installed natively, Ollama usually starts automatically as a systemd service. You can check its status:
|
|
```bash
|
|
systemctl status ollama
|
|
```
|
|
If you need to run it manually:
|
|
```bash
|
|
ollama serve
|
|
```
|
|
|
|
## Setup
|
|
|
|
1. **Clone the repository**:
|
|
```bash
|
|
git clone https://github.com/youruser/daily-ai-summary.git
|
|
cd daily-ai-summary
|
|
```
|
|
|
|
2. **Configure Environment**:
|
|
Copy `.env.example` to `.env` (or whatever path you'll use in your systemd unit) and fill in your details:
|
|
```bash
|
|
cp .env.example .env
|
|
nano .env
|
|
```
|
|
*Key variables to set:* `REPORT_TO_EMAIL`, `REPORT_FROM_EMAIL`, `SMTP_SERVER`, `OLLAMA_API_URL`.
|
|
|
|
3. **Build the Docker Image**:
|
|
```bash
|
|
docker build -t daily-ai-summary .
|
|
```
|
|
|
|
## Running
|
|
|
|
### Manual Test Run
|
|
You can run the container manually to verify your configuration:
|
|
```bash
|
|
docker run --rm \
|
|
--network=host \
|
|
--env-file=.env \
|
|
-v /var/log/journal:/var/log/journal:ro \
|
|
-v /run/log/journal:/run/log/journal:ro \
|
|
-v /etc/machine-id:/etc/machine-id:ro \
|
|
-v $(pwd)/data:/data \
|
|
daily-ai-summary
|
|
```
|
|
*Note: Ensure `RUN_NOW=true` is set in your `.env` for an immediate run.*
|
|
|
|
### Production Deployment (Systemd)
|
|
1. Adjust `systemd-daily-ai-summary.service.example` with your actual paths (e.g., where your `.env` and `data` folder live).
|
|
2. Copy the service file:
|
|
```bash
|
|
sudo cp systemd-daily-ai-summary.service.example /etc/systemd/system/daily-ai-summary.service
|
|
```
|
|
3. Enable and start:
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable daily-ai-summary
|
|
sudo systemctl start daily-ai-summary
|
|
```
|
|
|
|
## Configuration
|
|
|
|
| Variable | Default | Description |
|
|
| :--- | :--- | :--- |
|
|
| `OLLAMA_API_URL` | `http://localhost:11434/api/generate` | URL to your Ollama API. |
|
|
| `OLLAMA_MODEL` | `llama3.2:1b` | The model to use for summarization. |
|
|
| `REPORT_TO_EMAIL` | - | Recipient of the daily report. |
|
|
| `JOURNAL_UNIT` | `mailserver` | The systemd unit name to analyze. |
|
|
| `TRUSTED_LOGIN_NETWORKS` | `10.0.0.0/24` | CIDR ranges that won't be flagged as "untrusted" logins. |
|
|
| `REPORT_TIME` | `08:00` | When to run the daily report (24h format). |
|
|
| `RUN_NOW` | `false` | If `true`, runs the analysis immediately on startup. |
|
|
| `DEBUG` | `false` | Enable verbose logging. |
|
|
|
|
## Troubleshooting
|
|
|
|
- **Journal Access**: Ensure the user running Docker has permissions to read `/var/log/journal`. On many systems, this means being in the `systemd-journal` group.
|
|
- **Ollama Connectivity**: If Ollama is running on the host, the container needs `--network=host` to reach `localhost:11434`.
|
|
- **State Persistence**: If "repeat offenders" aren't being tracked across restarts, verify the `/data` mount is writable and correctly mapped in the systemd unit.
|
|
- **No Logs Found**: Check that `JOURNAL_UNIT` matches the exact name of the systemd service you want to monitor (e.g., `postfix.service` or `dovecot.service`).
|
|
|
|
## License
|
|
MIT
|