Vigilant - Complete Documentation

Table of Contents

  1. Overview
  2. Architecture
  3. Installation & Building
  4. Usage
  5. Service Configuration
  6. Deployment
  7. Dashboard
  8. Advanced Features
  9. Troubleshooting
  10. Example Configurations

Overview

Vigilant is a lightweight, dynamic proxy server and service manager written in C++17. It combines four core functions:

  1. Reverse Proxy - Routes HTTP/WebSocket traffic to backend services
  2. Service Orchestrator - Manages service lifecycle and auto-sleep
  3. Deployment CLI - Deploy services from Git repos or .vig files
  4. Operations Dashboard - Live monitoring of traffic and system logs

Key Features

Design Philosophy

Vigilant is designed strictly with performance, simplicity, and usefulness in mind. It targets businesses, SaaS platforms, and advanced homelab users who need a dynamic, lightweight, and completely transparent proxy and service manager.

Architecture

Request Lifecycle

When an incoming request arrives at Vigilant:

  1. Domain Extraction - Extract the host from request headers (X-Forwarded-Host or Host)
  2. Service Lookup - Find the service mapped to that domain
  3. Rate Limiting - Apply optional per-service rate limits
  4. Service Waking - If sleeping, start the service asynchronously
  5. Health Checking - Verify service is ready via health endpoint
  6. Proxying - Forward the request and relay responses
  7. Monitoring - Record request stats, latency, and bytes transferred

Component Model

┌─────────────────────────────────────────────────────────┐
│                  Vigilant Daemon                         │
├────────────────────┬──────────────────┬─────────────────┤
│  Proxy Server      │ Dashboard Server │  Config Watcher │
│  (Port 9000)       │ (Port 9001)      │                 │
├────────────────────┼──────────────────┼─────────────────┤
│            Service Manager                              │
│  ├─ Service State Tracking                             │
│  ├─ Process Lifecycle (fork, exec, signal)             │
│  ├─ Docker Lifecycle (run, start, stop)                │
│  ├─ Health Check Loop                                  │
│  └─ Reaper Thread (idle timeout logic)                 │
├────────────────────┬──────────────────┬─────────────────┤
│  Logging System    │ Stats Manager    │ CLI Commands    │
│  (File + Console)  │ (Metrics)        │ (Deploy, List)  │
└────────────────────┴──────────────────┴─────────────────┘

         ↓ (routes traffic to) ↓

┌─────────────────────────────────────────────────────────┐
│              Backend Services                           │
│  ├─ Service 1 (Process) - api.example.com:3001         │
│  ├─ Service 2 (Docker) - web.example.com:8080          │
│  └─ Service 3 (Docker) - worker.example.com:5000       │
└─────────────────────────────────────────────────────────┘

Service States

Each service has three possible states:

A background reaper thread checks idle time every 30 seconds and puts idle services to sleep after the configured timeout period.

Installation & Building

Quick Install (Linux/macOS)

curl -sL vigi.sh/install | sudo bash

This downloads the latest Vigilant binary and installs it to your system path.

Quick Install (Windows)

powershell -ExecutionPolicy Bypass -Command "irm https://vigi.sh/install-ps | iex"

This downloads and installs the latest Vigilant binary on Windows.

Download from GitHub

Pre-built binaries are available on the Vigilant releases page. Download the appropriate binary for your platform and place it in your path.

Building from Source

Prerequisites

Linux/macOS

git clone https://github.com/Zia-ullah-khan/Vigilant.git
cd Vigilant
mkdir build
cd build
cmake ..
cmake --build .

# Optionally install globally
sudo make install

Windows

git clone https://github.com/Zia-ullah-khan/Vigilant.git
cd Vigilant
mkdir build
cd build
cmake ..
cmake --build . --config Debug

# Executable will be at: build\Debug\vigilant.exe

Verify Installation

./vigilant --help
./vigilant_tests

Usage

Running the Daemon

The primary use is to run Vigilant as a daemon:

./vigilant [command] [options]

Server Command (Default)

Run the proxy server and service manager:

./vigilant server [options]
# Or simply: ./vigilant [options]

Options:

Example:

./vigilant server -p 8080 -dash 8081 -t 15 -d ./my_services_config

CLI Commands

Common subcommands: server (daemon), deploy, ls, logs, rm, cert / certbot (TLS via Certbot webroot), start / restart (systemd, Linux).

Deploy a Service

Deploy a local .vig file:

./vigilant deploy ./myservice.vig

Deploy a Git repository:

./vigilant deploy https://github.com/user/app.git \
  --branch main \
  --domain myapp.example.com \
  --port 8080

Deploy Options:

List Deployed Services

./vigilant ls

Output format: name <tab> domain <tab> -> localhost:port

Remove a Service

./vigilant rm myservice

The daemon will automatically spin it down.

View Service Logs

./vigilant logs myservice

Shows last 100 log lines. For Docker containers, shows Docker logs. For processes, shows stdout/stderr captured to temp logs.

Request TLS certificates: cert and certbot

Vigilant can run Certbot in HTTP-01 webroot mode for you. The default webroot is /var/www/html, which matches where the proxy serves /.well-known/acme-challenge/ (same path you need for manual webroot issuance). The subcommands cert and certbot are aliases—the same code path.

Requirements (Linux): Certbot must be installed on the system. Windows: validation of options may run, but invoking Certbot is not supported (you will get a clear error—use Linux or WSL for issuance).

Typical usage (Vigilant already listening on port 80 via your router, DNS pointing at your public IP):

# Interactive (Certbot may prompt for email / ToS on first run)
sudo vigilant cert api.example.com

# Staging CA (test without burning production rate limits)
sudo vigilant cert api.example.com --staging

# Dry run
sudo vigilant cert api.example.com --dry-run

# Non-interactive with email
sudo vigilant cert api.example.com --email you@example.com

# Unattended without email (only if you accept Certbot’s tradeoffs)
sudo vigilant cert api.example.com --unsafe-register

# Several hostnames on one certificate
sudo vigilant cert api.example.com www.example.com

# Custom webroot (if Vigilant’s ACME directory is configured elsewhere)
sudo vigilant cert api.example.com --webroot /var/www/html

After a successful run, the CLI prints the cert and key paths to paste into your .vig file (for example under /etc/letsencrypt/live/<primary-domain>/). Hot-reload picks up .vig changes without restarting the daemon.

Options summary

FlagPurpose
(positional)One or more domain names to include on the certificate
--stagingUse Let’s Encrypt staging (untrusted in browsers; safe for testing)
--dry-runSimulate issuance without storing a cert
--email <addr>Account email for Let’s Encrypt (non-interactive registration)
--unsafe-registerRegister without email (unattended; understand Certbot implications)
--webroot <path>Webroot directory (default: /var/www/html)

Implementation creates /.well-known/acme-challenge under the webroot as needed and invokes Certbot with properly escaped arguments.

The equivalent manual command is:

sudo certbot certonly --webroot -w /var/www/html -d api.example.com

Manage Systemd Service (Linux only)

./vigilant start    # Start the systemd service
./vigilant restart  # Restart the systemd service

Service Configuration

.vig File Format

Services are defined in simple key-value .vig files. Place them in the config directory (default: /etc/vigilant/services).

name = myservice
domain = api.example.com
port = 3001
type = process
command = cd /path/to/app && node server.js
pidfile = /var/run/myservice.pid
health = /health
timeout = 30
ratelimit = 0
cert = /etc/letsencrypt/live/api.example.com/fullchain.pem
key = /etc/letsencrypt/live/api.example.com/privkey.pem

Configuration Keys

Required Fields

KeyTypeDescription
namestringLogical name of the service (alphanumeric, no spaces)
domainstringDomain name to route to this service; used for SNI matching
portintegerInternal port where backend service listens
typestringService type: process or docker

Process Type Fields (Required when type=process)

KeyTypeDescription
commandstringCommand to execute to start the service
pidfilestringPath to store process ID file (default: /var/run/{name}.pid)

Docker Type Fields (Required when type=docker)

KeyTypeDescription
imagestringDocker image name:tag
containerstringDocker container name

Optional Fields

KeyTypeDefaultDescription
healthstring/Health check endpoint path
timeoutinteger30Service startup timeout in seconds
ratelimitinteger0Rate limit in requests/minute (0 = unlimited)
certstring-Path to SSL certificate for this domain
keystring-Path to SSL private key for this domain

Git Deployment Metadata Fields

When deployed via Git, these are automatically added:

KeyTypeDescription
sourcerepostringGit repository URL
sourcebranchstringGit branch (if deployed with --branch)
sourcetagstringGit tag (if deployed with --tag)
sourcecommitstringResolved commit SHA
buildcontextstringDocker build context path
dockerfilestringDockerfile path
envstringRuntime environment variables (repeatable)

Configuration Examples

Process Service (Node.js)

name = api
domain = api.example.com
port = 3001
type = process
command = cd /home/app && npm start
pidfile = /tmp/api.pid
health = /health
timeout = 30

Docker Service

name = web
domain = web.example.com
port = 8080
type = docker
image = myregistry/web:latest
container = web-prod
health = /healthz
timeout = 45
ratelimit = 500

Service with HTTPS

name = secure
domain = secure.example.com
port = 3443
type = process
command = /opt/app/run.sh
pidfile = /var/run/secure.pid
cert = /etc/letsencrypt/live/secure.example.com/fullchain.pem
key = /etc/letsencrypt/live/secure.example.com/privkey.pem

Deployment

Local Service Deployment

  1. Create a .vig file describing your service:
name = myapp
domain = myapp.local
port = 8000
type = process
command = cd /path/to/myapp && python app.py
  1. Deploy it:
./vigilant deploy ./myapp.vig -d /etc/vigilant/services
  1. Vigilant will automatically pick it up and make it available at the configured domain.

Git-based Deployment

Deploy a GitHub repository directly:

./vigilant deploy https://github.com/user/my-app.git \
  --branch main \
  --domain myapp.example.com \
  --port 8080

The deployment workflow:

  1. Clone the repository and checkout the specified branch/tag/commit
  2. Detect Runtime - Looks for package.json (Node), requirements.txt (Python), or Dockerfile
  3. Build Docker Image (if Docker available and Dockerfile exists)
  4. Fallback to Process Mode (if Docker not available or build fails)
  5. Generate .vig Config with deployment metadata
  6. Atomically Write config to services directory
  7. Auto-Spin Up on first request

Node.js Auto-Build

If no Dockerfile is found but package.json exists:

FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN if [ -f package-lock.json ]; then npm ci; else npm install; fi
COPY . .
RUN npm run build --if-present
EXPOSE 8080
CMD ["npm", "start"]

Python Auto-Build

If no Dockerfile is found but requirements.txt exists:

FROM python:3.10-slim
WORKDIR /app
COPY requirements.txt* ./
RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
COPY . .
EXPOSE 8080
CMD ["python", "app.py"]

Reverse Proxy Setup with Nginx

Vigilant can run behind Nginx for additional features like rate limiting, caching, or TLS termination.

server {
    listen 80;
    server_name *.example.com;

    location / {
        proxy_pass http://127.0.0.1:9000;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 60s;
        proxy_connect_timeout 60s;
    }
}

For HTTPS:

server {
    listen 443 ssl http2;
    server_name *.example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:9000;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 300s;
        proxy_connect_timeout 60s;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Dashboard

Accessing the Dashboard

The dashboard is available at http://localhost:9001 (default) or the port specified with -dash.

Dashboard Interface

The dashboard displays:

All data updates every 1 second via the /api/stats endpoint.

API Endpoint

GET /api/stats

Returns JSON:

{
  "totalRequests": 12345,
  "blockedRequests": 23,
  "bytesTransferred": 5242880,
  "recentRequests": [
    {
      "time": "14:23:45",
      "method": "POST",
      "domain": "api.example.com",
      "path": "/users",
      "status": 201,
      "latencyMs": 145
    }
  ],
  "recentLogs": [
    {
      "time": "14:23:44",
      "level": "INFO",
      "message": "Service awake: myservice"
    }
  ]
}

Advanced Features

HTTPS and TLS

Global Certificate

Serve HTTPS with a single certificate for all domains:

./vigilant server \
  --cert /path/to/cert.pem \
  --key /path/to/key.pem \
  -p 9000

Per-Domain Certificates (SNI)

Each service can have its own certificate:

name = api
domain = api.example.com
port = 3001
type = process
command = npm start
cert = /etc/letsencrypt/live/api.example.com/fullchain.pem
key = /etc/letsencrypt/live/api.example.com/privkey.pem

Vigilant uses the TLS Server Name Indication (SNI) to route to the correct certificate.

Let's Encrypt with Certbot

This section describes how to obtain TLS certificates when Vigilant is the public entrypoint (for example, your router forwards external 80 and 443 to the machine and port where Vigilant listens).

Prerequisites (all methods)

  1. DNS — An A (or AAAA) record so your hostname resolves to your public IP.
  2. Port forwarding — Forward TCP 80 (and 443 for HTTPS clients) to the host and port where Vigilant listens (e.g. 192.168.1.x:9000). Use an external port check if validation fails.
  3. Webroot — Vigilant serves /.well-known/acme-challenge/ from /var/www/html by default (unless you use a custom --webroot with vigilant cert). Ensure Certbot can write there and Vigilant can read; see permissions below if you see [ACME] Failed to find validation file.

Preferred: built-in vigilant cert / vigilant certbot

Use the CLI described in Request TLS certificates: cert and certbot. It runs Certbot in webroot mode with the correct default path, creates .well-known/acme-challenge under the webroot, and prints cert / key lines for your .vig after success.

sudo vigilant cert api.example.com
# or
sudo vigilant certbot api.example.com

What goes wrong with certbot --standalone behind Vigilant

For HTTP-01 validation, Let's Encrypt always sends the challenge to your hostname on port 80 from the internet. It does not follow Certbot's --http-01-port setting—that flag only changes which local port Certbot's temporary server binds to. If your router sends port 80 to Vigilant (e.g. 192.168.x.x:9000), the CA never reaches Certbot on 8888 or another alternate port, so validation fails with 404 or unauthorized. Use webroot (via vigilant cert or manual certbot certonly --webroot) instead.

Manual webroot (equivalent to the built-in command)

If you prefer to run Certbot yourself:

  1. Create the webroot tree (once), if needed:

    sudo mkdir -p /var/www/html/.well-known/acme-challenge

  2. Permissions — If you see validation failures, fix modes so both Certbot and Vigilant can access the challenge files:

    sudo chmod -R 755 /var/www/html/.well-known

  3. Request the certificate (production):

    sudo certbot certonly --webroot -w /var/www/html -d api.example.com

    Certificates are written under /etc/letsencrypt/live/api.example.com/ (fullchain.pem, privkey.pem).

  4. Point your service at the new cert — Add to the .vig for that domain:

    cert = /etc/letsencrypt/live/api.example.com/fullchain.pem
key = /etc/letsencrypt/live/api.example.com/privkey.pem

    Save the file; Vigilant picks up changes on hot reload (no full restart required in normal setups).

  5. Health checks — If your app does not serve /, set health to a real endpoint (for example /api/health) so startup checks succeed. Slow-start apps need a large enough timeout (seconds) so Vigilant does not mark the service failed before the backend is ready.

Test with staging (avoids production rate limits)

Let's Encrypt limits failed authorizations per hostname (roughly five per hour). While debugging, use staging:

sudo vigilant cert api.example.com --staging
# or
sudo certbot certonly --webroot -w /var/www/html -d api.example.com --staging

Staging certs are not trusted by browsers (you will see a certificate warning). When validation succeeds on staging, run production issuance without --staging after any cooldown, or use Certbot’s --force-renewal only when you intend to replace an existing cert.

Renewal

certbot renew typically reuses the same webroot. If your renewal config uses -w /var/www/html, Vigilant can keep serving challenges without downtime. Review /etc/letsencrypt/renewal/ and test with:

sudo certbot renew --dry-run

If you cannot use port 80 (ISP block, etc.)

Use DNS-01 validation instead (TXT record at _acme-challenge.<hostname>). For example:

sudo certbot certonly --manual --preferred-challenges dns -d api.example.com

Follow Certbot's prompts to add the TXT record at your DNS provider. No inbound HTTP is required.

Standalone on port 80 (alternative)

If you use certbot certonly --standalone, Certbot must bind port 80 on the machine. You must stop Vigilant (or anything else on that path) for the duration of the run, then start it again afterward. Prefer webroot when Vigilant already handles port 80 traffic.

WebSocket Support

Vigilant fully supports WebSocket proxying with bidirectional streaming. Simply configure a domain and Vigilant will upgrade HTTP connections to WebSocket automatically.

Example test:

# Terminal 1: Start a WebSocket server on localhost:8000
node websocket-server.js

# Terminal 2: Create a service
# (in /etc/vigilant/services/ws.vig):
# name = ws
# domain = ws.example.com
# port = 8000
# type = process
# command = node websocket-server.js

# Terminal 3: Connect via Vigilant
wscat -c ws://ws.example.com

Rate Limiting

Limit requests per minute per service:

name = api
domain = api.example.com
port = 3001
type = process
command = npm start
ratelimit = 1000

Rate limiting is per IP+domain combination. When exceeded, client receives HTTP 429 (Too Many Requests).

Hot Configuration Reload

Configuration changes are detected automatically every 5 seconds:

Critical fields that trigger restart: port, command, image, container, type

Service Idle Timeout

Set the global inactivity timeout:

./vigilant server -t 15  # 15 minutes

Services automatically sleep after this period of inactivity. Default is 10 minutes.

Logging

Vigilant logs to file and console. Logs include all system events, service lifecycle changes, and request errors.

Log locations (in order of preference):

  1. Specified path: -l /custom/path/vigilant.log
  2. System log dir: /var/log/vigilant.log
  3. User local: $HOME/.local/state/vigilant/vigilant.log (Linux) or %LOCALAPPDATA%\Vigilant\logs\vigilant.log (Windows)
  4. Current directory: ./vigilant.log

Troubleshooting

Service Won't Start

Check the logs:

./vigilant logs myservice

Common causes:

Health Check Timeout

Increase the timeout value in .vig:

timeout = 60  # Increase to 60 seconds

Ensure the health endpoint is responding correctly.

ACME / Let's Encrypt validation fails (404 or unauthorized)

WebSocket Connection Fails

Ensure:

Rate Limiting Too Aggressive

Adjust the rate limit:

ratelimit = 10000  # Higher = more permissive

Monitor via dashboard to tune appropriately.

Port Already in Use

Change the proxy port:

./vigilant server -p 9002

Or kill the existing process:

# Find process using port 9000
lsof -i :9000
kill -9 <PID>

Dashboard Not Accessible

Verify the dashboard port:

./vigilant server -dash 9002  # Different port

Check that the port isn't blocked by firewall.

Config Not Reloading

Ensure the config directory is writable and Vigilant has permissions. Config changes take effect within 5 seconds.

Test by modifying a .vig file and checking the logs.

Example Configurations

Single Node.js API

File: /etc/vigilant/services/api.vig

name = api
domain = api.example.com
port = 3001
type = process
command = cd /opt/api && npm start
pidfile = /var/run/api.pid
health = /health
timeout = 30
cert = /etc/letsencrypt/live/api.example.com/fullchain.pem
key = /etc/letsencrypt/live/api.example.com/privkey.pem

Start Vigilant:

./vigilant server -p 9000 -dash 9001 -d /etc/vigilant/services

Multi-Service Deployment

API Service

name = api
domain = api.mycompany.com
port = 3001
type = process
command = cd /opt/api && npm start
health = /health
timeout = 30

Web UI

name = web
domain = mycompany.com
port = 3000
type = docker
image = mycompany/web:latest
container = web-prod
health = /
timeout = 45

Database Migration Service

name = db-migrations
domain = internal.mycompany.com
port = 5000
type = process
command = ./db-migrate.sh
timeout = 120

Start:

./vigilant server \
  -p 9000 \
  -dash 9001 \
  -t 20 \
  -d /etc/vigilant/services

Docker Compose Integration

While Vigilant isn't a replacement for Docker Compose, you can use it alongside:

File: docker-compose.yml

version: '3'
services:
  api:
    image: myapp/api:latest
    ports:
      - "3001:3001"
    environment:
      NODE_ENV: production

File: /etc/vigilant/services/api.vig

name = api
domain = api.example.com
port = 3001
type = docker
image = myapp/api:latest
container = api-prod
health = /health

Real-World Example Services

24Control Backend

# File: 24controlapi.vig
name = 24control
domain = 24controlapi.rfas.software
port = 3001
type = process
command = cd /home/pi/Desktop/Projects/24Control/backend && node server.js
pidfile = /tmp/24control.pid
health = /
timeout = 30
cert = /etc/letsencrypt/live/24controlapi.rfas.software/fullchain.pem
key = /etc/letsencrypt/live/24controlapi.rfas.software/privkey.pem

RoboTrader (Docker)

# File: robotrader.vig
name = robotrader
domain = robotrader.rfas.software
port = 8083
type = docker
image = robotrader:latest
container = robotrader-container
health = /ping
timeout = 30

StudySync (Docker)

# File: studysync.vig
name = studysync
domain = studysync.rfas.software
port = 8081
type = docker
image = studysync:latest
container = studysync-container
health = /health
timeout = 30

Run all three:

./vigilant server \
  -d /etc/vigilant/services \
  -p 9000 \
  -dash 9001 \
  -t 10

Vision & Roadmap

Vigilant is designed strictly with performance, simplicity, and usefulness in mind.

Current Capabilities

✅ Dynamic reverse proxy with domain-based routing
✅ Process and Docker service support
✅ Automatic service waking and sleeping
✅ HTTPS/TLS with SNI support
✅ WebSocket proxying
✅ Hot configuration reload
✅ Rate limiting
✅ Built-in dashboard with stats
✅ Git-based deployments
✅ CLI service management

Future Roadmap

License

Vigilant is currently developed as a proprietary, closed-source solution intended for commercial licensing.

For commercial use or licensing inquiries, please contact the development team.