π—π€πˆ π—π€ππˆ

Self Hosted

This document will guide you through the process of deploying XAI Gateway on your own server. The deployment process relies on Docker and Docker Compose, so please ensure your server has the appropriate environment set up.

Prerequisites

  • A Linux server with root access, preferably located overseas. We recommend an instance like AWS t3.small (2-core CPU, 2GB RAM, 10GB disk) in a US region.
  • Three domain names that have been resolved to the server's IP address, for example:
    • api.your-domain.com (for the API service)
    • manage.your-domain.com (for the user management dashboard)
    • admin.your-domain.com (for the system management dashboard)

Deployment Steps

Step 1: Install Docker

We recommend using the official Docker script for installation, which supports most major Linux distributions.

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Step 2: Install Docker Compose

This command will download the latest stable release of Docker Compose from GitHub.

curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

Step 3: Grant Executable Permissions

Add executable permissions to the Docker Compose binary you just downloaded.

chmod +x /usr/local/bin/docker-compose

Step 4: Generate the docker-compose.yml Configuration File

This is the most critical step. You need to generate a custom docker-compose.yml file tailored for you through our deployment service.

Execute the following command, but be sure to replace the values of xai_mail and xai_api with your own information:

curl -X POST https://deploy.xaixapi.com/xai \
  -H "Content-Type: application/json" \
  -d '{
    "xai_mail": "[email protected]",
    "xai_api": "api.your-domain.com"
  }' | jq -r '.content' > docker-compose.yml

Parameter Descriptions:

  • xai_mail (Required): Your administrator email address. It will be used for receiving important notifications and initializing the administrator account.
  • xai_api (Required): The API domain you have prepared for the service. Ensure this domain is correctly resolved to your server's IP address. After the service starts, all API requests will be sent to this address.

Step 5: Start the Services

Using the generated configuration file, start all services in the background.

docker-compose -f docker-compose.yml up -d

Step 6: Configure Nginx Reverse Proxy

To make your services accessible via domain names and to serve the management dashboards, you need to configure Nginx as a reverse proxy.

6.1 Download Static Assets for Management Dashboards

Before configuring Nginx, first download the static asset package required for the management dashboards and extract it to the Nginx html directory.

# Assuming the Nginx root directory is /usr/share/nginx/html or /var/www/html

# Download static assets
wget https://filelist.cn/disk/0/pages.tgz
tar xzf pages.tgz

After execution, your directory structure should look like this:

${NGINX_HTML_PATH}/
└── pages/
    β”œβ”€β”€ admin/
    β”‚   └── ... (admin dashboard files)
    └── manage/
        └── ... (manage dashboard files)
6.2 Install Nginx
# Ubuntu/Debian
sudo apt update && sudo apt install nginx -y

# CentOS/RHEL
sudo yum install nginx -y
# Or (RHEL 8+)
sudo dnf install nginx -y
6.3 Create Nginx Configuration File

For best compatibility, it is recommended to back up and replace the entire nginx.conf file.

# Back up the existing configuration
sudo mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

# Create a new configuration file
sudo vim /etc/nginx/nginx.conf

Copy the following content completely into the /etc/nginx/nginx.conf file. Be sure to replace api.your-domain.com, manage.your-domain.com, and admin.your-domain.com with your actual domain names.

user root;
worker_processes auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections 65535;
    multi_accept on;
}

http {
    include mime.types;
    default_type application/octet-stream;
    charset utf-8;
    http2 on;

    # Basic Settings
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 600;
    keepalive_requests 10000;
    reset_timedout_connection on;

    # Compression
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_min_length 1024;
    gzip_buffers 16 8k;
    gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/x-javascript application/x-font-ttf application/vnd.ms-fontobject font/opentype image/svg+xml image/x-icon text/event-stream;

    # Client Request
    client_max_body_size 256m;
    client_body_buffer_size 1m;
    client_header_buffer_size 2k;
    large_client_header_buffers 4 8k;

    # Proxy
    proxy_read_timeout 900s;
    proxy_send_timeout 900s;
    proxy_connect_timeout 60s;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Connection "";
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    # Streaming Response
    proxy_cache off;
    proxy_buffering off;
    proxy_request_buffering off;
    chunked_transfer_encoding on;

    # Logging
    log_format main "$status  $request_time  $request_method  $http_host$request_uri  [$http_user_agent]  [$time_local]  [$http_x_forwarded_for $remote_addr]  $http_referer";
    access_log /var/log/nginx/access.log main buffer=32k flush=30s;
    error_log /var/log/nginx/error.log;

    # Upstream Service (points to the Docker container)
    upstream xai_backend {
        server 127.0.0.1:3443;
        keepalive 32;
        keepalive_timeout 60s;
    }

    # API Service
    server {
        listen 80;
        server_name api.your-domain.com;

        # WebSocket Support
        location /v1/realtime {
            proxy_pass http://xai_backend;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }

        # All other API requests
        location / {
            if ($request_method = 'OPTIONS') {
               add_header 'Access-Control-Allow-Origin' '*';
               add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
               add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type';
               add_header 'Access-Control-Max-Age' 1728000;
               add_header 'Content-Type' 'text/plain; charset=utf-8';
               add_header 'Content-Length' 0;
               return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
            add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type' always;

            proxy_pass http://xai_backend;
        }
    }

    # Manage - User Account Management (Static Site)
    server {
        listen 80;
        server_name manage.your-domain.com;
        # Nginx static asset directory, adjust the path according to Step 6.1
        root /usr/share/nginx/html/pages/manage;
        index index.html;
        location / {
            try_files $uri $uri/ /index.html;
        }
    }

    # Admin - System Configuration (Static Site)
    server {
        listen 80;
        server_name admin.your-domain.com;
        # Nginx static asset directory, adjust the path according to Step 6.1
        root /usr/share/nginx/html/pages/admin;
        index index.html;
        location / {
            try_files $uri $uri/ /index.html;
        }
    }
}
6.4 Restart Nginx
# Test configuration file syntax
sudo nginx -t

# Restart the Nginx service and enable it to start on boot
sudo systemctl restart nginx
sudo systemctl enable nginx

It is recommended to configure HTTPS for all your services.

Step 8: Verify the Deployment

  1. Check Docker Service Status:

    docker-compose ps

    Ensure all services are in the Up or running state.

  2. Test Service Access:

    • API Service: curl http://api.your-domain.com
    • User Management Dashboard: Open http://manage.your-domain.com in your browser
    • System Management Dashboard: Open http://admin.your-domain.com in your browser

Deployment Complete: Service Access Guide

Congratulations, the deployment is complete! You now have three core services, accessible via different domain names:

  • http://api.your-domain.com

    • Purpose: API Gateway. This is the core of the system; all requests to large models are routed through this domain.
    • Access Method: Call via code, SDK, or API debugging tools (like Postman).

  • http://manage.your-domain.com

    • Purpose: User Management Dashboard. For use by your end-users or customers.
    • Access Method: Open in a web browser.
    • Key Features: Users can register, obtain API Keys, check their usage quotas, query request logs, and create, view, top-up, update, and delete sub-accounts on this platform.

  • http://admin.your-domain.com

    • Purpose: System Management Dashboard. Used by the system administrator. Login is restricted to the XAI API ROOT Key.
    • Access Method: Open in a web browser.
    • Key Features: Perform system-level configurations, such as managing model providers.
User Hierarchy and Inheritance →