diff --git a/README.md b/README.md index e69de29..7f01e3e 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,130 @@ +# Daily AI Log Summary + +A lightweight, automated tool that analyzes system logs—specifically optimized for **Postfix** and **Dovecot** mail servers—extracts key security and operational facts, and generates a concise daily summary using a local AI model (Ollama). + +## Features + +- **Structured Log Analysis**: Deterministically extracts facts from `journalctl` logs (Postfix/postscreen rejections, Dovecot logins, relay abuse, SPF/DKIM anomalies, etc.). +- **AI Summarization**: Uses Ollama (e.g., `llama3.2:1b`) to synthesize raw facts into a readable daily report. +- **Repeat Offender Tracking**: Persistently tracks suspicious IPs across multiple days to identify long-term probing. +- **Automated Emailing**: Sends the summary and raw facts via SMTP. +- **Dockerized**: Easy deployment with a minimal footprint. +- **Systemd Integration**: Includes an example unit for robust "always-on" operation. + +## How It Works + +1. **Fetch**: Retrieves yesterday's logs for a specific systemd unit using `journalctl`. +2. **Analyze**: Python regex-based extraction identifies successful logins, authentication failures, blocked relay attempts, and more. +3. **Track**: Suspicious IPs are stored in a persistent state file. If an IP appears on multiple days, it's flagged as a "repeat offender." +4. **Summarize**: Raw facts are fed to a local Ollama instance with a specific system prompt to generate a concise narrative. +5. **Report**: An email is sent to the administrator with the AI summary and the full list of extracted facts. + +## Prerequisites + +- **Docker**: For running the analyzer. +- **Ollama**: A running Ollama instance (locally or accessible via network). +- **Systemd Journals**: The host must use systemd journals (the container mounts these as read-only). + +## Ollama Setup + +This tool requires an Ollama server to handle the AI summarization. + +### 1. Install Ollama +If you don't have Ollama installed, you can install it on Linux with: +```bash +curl -fsSL https://ollama.com/install.sh | sh +``` +Alternatively, you can run Ollama as a Docker container: +```bash +docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama +``` + +### 2. Select and Pull a Model +The script defaults to `llama3.2:1b`, which is lightweight and fast for this task. You must pull the model before the script can use it: +```bash +ollama pull llama3.2:1b +``` +You can choose other models (like `llama3`, `mistral`, `gemma`) by pulling them and updating the `OLLAMA_MODEL` variable in your `.env` file. + +### 3. Run the Server +If installed natively, Ollama usually starts automatically as a systemd service. You can check its status: +```bash +systemctl status ollama +``` +If you need to run it manually: +```bash +ollama serve +``` + +## Setup + +1. **Clone the repository**: + ```bash + git clone https://github.com/youruser/daily-ai-summary.git + cd daily-ai-summary + ``` + +2. **Configure Environment**: + Copy `.env.example` to `.env` (or whatever path you'll use in your systemd unit) and fill in your details: + ```bash + cp .env.example .env + nano .env + ``` + *Key variables to set:* `REPORT_TO_EMAIL`, `REPORT_FROM_EMAIL`, `SMTP_SERVER`, `OLLAMA_API_URL`. + +3. **Build the Docker Image**: + ```bash + docker build -t daily-ai-summary . + ``` + +## Running + +### Manual Test Run +You can run the container manually to verify your configuration: +```bash +docker run --rm \ + --network=host \ + --env-file=.env \ + -v /var/log/journal:/var/log/journal:ro \ + -v /run/log/journal:/run/log/journal:ro \ + -v /etc/machine-id:/etc/machine-id:ro \ + -v $(pwd)/data:/data \ + daily-ai-summary +``` +*Note: Ensure `RUN_NOW=true` is set in your `.env` for an immediate run.* + +### Production Deployment (Systemd) +1. Adjust `systemd-daily-ai-summary.service.example` with your actual paths (e.g., where your `.env` and `data` folder live). +2. Copy the service file: + ```bash + sudo cp systemd-daily-ai-summary.service.example /etc/systemd/system/daily-ai-summary.service + ``` +3. Enable and start: + ```bash + sudo systemctl daemon-reload + sudo systemctl enable daily-ai-summary + sudo systemctl start daily-ai-summary + ``` + +## Configuration + +| Variable | Default | Description | +| :--- | :--- | :--- | +| `OLLAMA_API_URL` | `http://localhost:11434/api/generate` | URL to your Ollama API. | +| `OLLAMA_MODEL` | `llama3.2:1b` | The model to use for summarization. | +| `REPORT_TO_EMAIL` | - | Recipient of the daily report. | +| `JOURNAL_UNIT` | `mailserver` | The systemd unit name to analyze. | +| `TRUSTED_LOGIN_NETWORKS` | `10.0.0.0/24` | CIDR ranges that won't be flagged as "untrusted" logins. | +| `REPORT_TIME` | `08:00` | When to run the daily report (24h format). | +| `RUN_NOW` | `false` | If `true`, runs the analysis immediately on startup. | +| `DEBUG` | `false` | Enable verbose logging. | + +## Troubleshooting + +- **Journal Access**: Ensure the user running Docker has permissions to read `/var/log/journal`. On many systems, this means being in the `systemd-journal` group. +- **Ollama Connectivity**: If Ollama is running on the host, the container needs `--network=host` to reach `localhost:11434`. +- **State Persistence**: If "repeat offenders" aren't being tracked across restarts, verify the `/data` mount is writable and correctly mapped in the systemd unit. +- **No Logs Found**: Check that `JOURNAL_UNIT` matches the exact name of the systemd service you want to monitor (e.g., `postfix.service` or `dovecot.service`). + +## License +MIT diff --git a/systemd-ollama.service.example b/systemd-ollama.service.example new file mode 100644 index 0000000..789c129 --- /dev/null +++ b/systemd-ollama.service.example @@ -0,0 +1,33 @@ +[Unit] +Description=ollama (virt-ollama) +Requires=docker.service +After=docker.service +DefaultDependencies=no + +[Service] +Type=simple +Environment="HOME=/root" +ExecStartPre=-/usr/bin/env sh -c '/usr/bin/env docker kill virt-ollama 2>/dev/null || true' +ExecStartPre=-/usr/bin/env sh -c '/usr/bin/env docker rm virt-ollama 2>/dev/null || true' + +ExecStart=/usr/bin/env docker run \ + --rm \ + --name=virt-ollama \ + -p 10.0.0.1:11434:11434 \ + --log-driver=none \ + --network=traefik \ + --cpus="2.0" \ + --memory="2g" \ + --mount type=bind,src=/etc/localtime,dst=/etc/localtime,ro \ + --mount type=bind,src=/virt/ollama/data,dst=/root/.ollama \ + ollama/ollama:latest + +ExecStop=-/usr/bin/env sh -c '/usr/bin/env docker kill virt-ollama 2>/dev/null || true' +ExecStop=-/usr/bin/env sh -c '/usr/bin/env docker rm virt-ollama 2>/dev/null || true' + +Restart=always +RestartSec=30 +SyslogIdentifier=virt-ollama + +[Install] +WantedBy=multi-user.target