Getting Started with Self-Hosted XAI Router

The self-hosted version is functionally identical to the XAI Control cloud service, but all data and configurations are kept locally. It is ideal for enterprises or teams with strict requirements for data compliance and network isolation.

1. Environment and Domain Requirements

1.1 Server Configuration

Instance Recommendation: We recommend a Linux server with root access, preferably located overseas. An AWS t3.small instance (2-core CPU, 2GB Memory, 10GB+ Disk) or an equivalent machine is a good choice.

1.2 Domain Requirements

Before deployment, you need to prepare three domain names that are resolved to your server's IP address, for example:

• api.your-domain.com (for the API service)
• manage.your-domain.com (for the user management dashboard)
• admin.your-domain.com (for the system administration dashboard)

2. Deployment Steps

This document will guide you through the process of deploying XAI Router on your own server. The deployment relies on Docker and Docker Compose, so please ensure your server has the necessary environment set up.

Step 1: Install Docker

We recommend using the official Docker script for installation, which supports most major Linux distributions.

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Step 2: Install Docker Compose

This command downloads the latest stable release of Docker Compose from GitHub.

curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

Step 3: Grant Execute Permissions

Add execute permissions to the Docker Compose binary you just downloaded.

chmod +x /usr/local/bin/docker-compose

Step 4: Generate docker-compose.yml Configuration File

This is the most critical step. You need to generate a custom docker-compose.yml file tailored for you through our deployment service.

Execute the following command, but be sure to replace the values of xai_mail and xai_api with your own information:

curl -X POST https://deploy.xaixapi.com/xai?raw=true \
  -H "Content-Type: application/json" \
  -d '{
    "xai_mail": "[email protected]",
    "xai_api": "api.your-domain.com"
  }' > docker-compose.yml

Parameter Descriptions:

• xai_mail (Required): Your administrator's email address. It will be used for receiving important notifications and initializing the admin account.
• xai_api (Required): The API domain you have prepared for the service. Make sure this domain is correctly resolved to your server's IP address. After the service starts, all API requests will be sent to this address.

Step 5: Start the Services

Use the generated configuration file to start all services in the background.

docker-compose -f docker-compose.yml up -d

This command will automatically pull the required images and start the complete XAI Router service stack according to the definitions in the docker-compose.yml file. After the services are started, you can check the status of all containers using the docker-compose ps command.

The core service stack mainly includes the following three components listening on internal server ports:

• Core API Service: Listens on port 3443. This is the main backend service of the system, responsible for handling all API routing and logic.
• Configuration Management Service: Listens on port 3080. It serves the web interface for the system administration dashboard (admin.your-domain.com).
• Account Management Service: Listens on port 3081. It serves the web interface for the user management dashboard (manage.your-domain.com).

Please note: These ports are only exposed internally on the server. You do not need to (and should not) access them directly from the public internet. In the next step, we will configure Nginx as a reverse proxy to securely and efficiently forward external requests from your domains to these internal service ports.

Step 6: Configure Nginx Reverse Proxy

To make your services accessible via domain names and to serve the management dashboards, you need to configure Nginx as a reverse proxy.

6.1 Install Nginx

# Ubuntu/Debian
sudo apt update && sudo apt install nginx -y

# CentOS/RHEL
sudo yum install nginx -y
# or (RHEL 8+)
sudo dnf install nginx -y

6.2 Create Nginx Configuration File

It is recommended to back up and replace the entire nginx.conf file for optimal compatibility.

# Back up the existing configuration
sudo mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

# Create a new configuration file
sudo vim /etc/nginx/nginx.conf

Copy the entire content below into the /etc/nginx/nginx.conf file. Be sure to replace api.your-domain.com, manage.your-domain.com, and admin.your-domain.com with your actual domain names.

user root;
worker_processes auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections 65535;
    multi_accept on;
}

http {
    include mime.types;
    default_type application/octet-stream;
    charset utf-8;
    http2 on;

    # Basic Settings
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 600;
    keepalive_requests 10000;
    reset_timedout_connection on;

    # Compression
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_min_length 1024;
    gzip_buffers 16 8k;
    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/x-javascript application/x-font-ttf application/vnd.ms-fontobject font/opentype image/svg+xml image/x-icon text/event-stream;

    # Client Request
    client_max_body_size 256m;
    client_body_buffer_size 1m;
    client_header_buffer_size 2k;
    large_client_header_buffers 4 8k;

    # Proxy Settings
    proxy_read_timeout 900s;
    proxy_send_timeout 900s;
    proxy_connect_timeout 60s;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Connection "";
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    # Streaming Response
    proxy_cache off;
    proxy_buffering off;
    proxy_request_buffering off;
    chunked_transfer_encoding on;

    # Logging
    log_format main "$status  $request_time  $request_method  $http_host$request_uri  [$http_user_agent]  [$time_local]  [$http_x_forwarded_for $remote_addr]  $http_referer";
    access_log /var/log/nginx/access.log main buffer=32k flush=30s;
    error_log /var/log/nginx/error.log;

    # Upstream Service (points to the Docker container)
    upstream xai_backend {
        server 127.0.0.1:3443;
        keepalive 32;
        keepalive_timeout 60s;
    }

    # API Service
    server {
        listen 80;
        server_name api.your-domain.com;

        # WebSocket Support
        location /v1/realtime {
            proxy_pass http://xai_backend;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }

        # All other API requests
        location / {
            if ($request_method = 'OPTIONS') {
               add_header 'Access-Control-Allow-Origin' '*';
               add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
               add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type';
               add_header 'Access-Control-Max-Age' 1728000;
               add_header 'Content-Type' 'text/plain; charset=utf-8';
               add_header 'Content-Length' 0;
               return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
            add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type' always;

            proxy_pass http://xai_backend;
        }
    }

    # Admin System Configuration (only accessible with the admin root key)
    server {
        listen 80;
        server_name admin.your-domain.com;
        location / {
            proxy_pass http://127.0.0.1:3080;
        }
    }

    # Manage Account Management
    server {
        listen 80;
        server_name manage.your-domain.com;
        location / {
            proxy_pass http://127.0.0.1:3081;
        }
    }
}

6.3 Restart Nginx

# Test the configuration file syntax
sudo nginx -t

# Restart Nginx and enable it to start on boot
sudo systemctl restart nginx
sudo systemctl enable nginx

Step 7: Configure HTTPS (Recommended)

To ensure data security, it is highly recommended to configure HTTPS for all your services.

Step 8: Verify Deployment

1. Check Docker service status:

   docker-compose ps
   

   Ensure all services are in the Up or running state.

2. Test service access:

   • API Service: curl http://api.your-domain.com
   • User Management Dashboard: Open http://manage.your-domain.com in your browser
   • System Administration Dashboard: Open http://admin.your-domain.com in your browser

3. Service Access Guide

Congratulations, the deployment is complete! You now have three core services accessible via different domains:

• http://api.your-domain.com

  • Purpose: API Gateway. This is the core of the system. All requests to large language models are routed through this domain.
  • Access Method: Call it via code, SDKs, or API testing tools like Postman.

• http://manage.your-domain.com

  • Purpose: User Management Dashboard. For use by your end-users or customers.
  • Access Method: Open in a web browser.
  • Key Features: Users can register, obtain API Keys, view their usage credits, query request logs, and create/view/top-up/update/delete sub-accounts on this platform.

• http://admin.your-domain.com

  • Purpose: System Administration Dashboard. Used by the system administrator. Login is restricted to the XAI API ROOT Key.
  • Access Method: Open in a web browser.
  • Key Features: Perform system-level configurations, such as managing model providers.