DBConvert Streams

Appearance

Docker Cloud Deployment ​

This guide covers deploying DBConvert Streams to cloud hosting providers such as AWS, Google Cloud, Azure, or DigitalOcean using Docker.

🔥 Quick Start ​

bash
curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

This automatically:

  • ✅ Installs Docker Compose (if missing)
  • ✅ Sets up networking & volumes
  • ✅ Detects public/local IP
  • ✅ Configures NGINX
  • ✅ Generates SSL Certificates.

DigitalOcean Marketplace ​

For the easiest cloud deployment option, DBConvert Streams is available as a 1-Click App on the DigitalOcean Marketplace: DBConvert Streams on DigitalOcean Marketplace

📖 Table of Contents ​

  1. Prerequisites
  2. Required Ports
  3. Service Architecture
  4. Installation
  5. IP Address Configuration
  6. Cloud Provider-Specific Instructions
  7. Managing Services
  8. Security Considerations
  9. Scaling Considerations
  10. Troubleshooting

Prerequisites ​

  • Docker Engine 20.10.0 or newer
  • Docker Compose v2.0.0 or newer
  • Either curl OR wget for downloading files
  • At least 2GB of available RAM
  • 2GB of free disk space
  • 2 CPU cores minimum (3+ cores recommended)
  • A cloud server instance with a public IP address

Required Ports ​

When deploying to a cloud hosting provider, you need to ensure the following ports are open in your hosting provider's security group or firewall:

PortServiceDescription
22SSHFor SSH access to the server
80HTTPFor Streams UI and API access
443HTTPSFor secure UI and API access (if using HTTPS)

These ports must be opened on your hosting provider to allow incoming connections from the internet. All other services run internally and don't need exposed ports.

API Access

The Streams API is now accessible through the Nginx proxy at:

  • HTTP: http://SERVER/api/
  • HTTPS: https://SERVER/api/ (if HTTPS is enabled)

You no longer need to expose port 8020 directly.

🛠 Service Architecture ​

DBConvert Streams consists of 3 main parts:

1️⃣ Core Services ​

  • stream-api: REST API (via Nginx)
  • stream-reader: Reads source databases
  • stream-writer (multiple instances): Writes to target DBs
  • stream-ui: Web interface (via Nginx)

2️⃣ Infrastructure Services ​

  • nats: Messaging system (internal only)
  • consul: Service discovery (internal only)
  • vault: Secrets management (internal only)
  • nginx: Reverse proxy + SSL (ports 80, 443)

3️⃣ Network & Security ​

  • All services run in stream-net (Docker network).
  • Nginx handles HTTPS & API routing.
  • Internal services are not exposed.
  • Stream logs are delivered via Server-Sent Events (SSE) through the Nginx proxy.

📥 Installation ​

1️⃣ Basic Install ​

Using curl: ​

bash
curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Using wget: ​

bash
wget -qO- https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

2️⃣ Advanced Install Options ​

FeatureCommand
Install Specific Versionsh -s -- --version v1.0.0
Use Custom IPsh -s -- --ip YOUR_PUBLIC_IP
Local Developmentsh -s -- --local

Note about HTTPS

The --secure option is not available during installation but can be used when starting services:

bash
# Start services with HTTPS enabled
./start.sh --secure

IP Address Configuration ​

DBConvert Streams needs to know your server's public IP address to properly generate SSL certificates and configure network settings when deployed to cloud hosting providers. The system handles this in the following way:

  1. During installation, the install.sh script detects your public IP address and saves it to .env.public_ip
  2. The start.sh script uses this saved IP address for configuration
  3. You can override the IP at any time by using the --ip flag with any script
  4. If you need to change the IP after installation, simply run any script with the --ip flag

If you're experiencing connection issues, you can manually specify the IP address:

bash
./start.sh --ip YOUR_PUBLIC_IP

Cloud Provider-Specific Instructions ​

Option 1: 1-Click App (Easiest Method)

  1. Visit the DBConvert Streams on DigitalOcean Marketplace
  2. Click "Create DBConvert Streams Droplet"
  3. Select your preferred Droplet size (2GB RAM / 2 vCPUs minimum recommended)
  4. Select a datacenter region
  5. Add your SSH keys
  6. Click "Create Droplet"
  7. Wait for deployment to complete (~2 minutes)
  8. Access your instance at the Droplet's IP address

Option 2: Manual Installation on a Droplet

  1. Create a Droplet:
    • Recommended: Standard Droplet with 2GB RAM / 2 vCPUs or larger
    • OS: Ubuntu 20.04 or newer
  2. Configure firewall:
    • Create a firewall and allow ports 22, 80, and 443 (if using HTTPS)
  3. Install DBConvert Streams:
    bash
    curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Other Cloud Providers ​

The installation process is similar for AWS, Google Cloud, and Azure:

  1. Create a VM/instance:

    • Recommended specs: 2GB+ RAM, 2+ vCPUs
    • Supported OS: Ubuntu 20.04+, Debian 10+, CentOS 7+, Amazon Linux 2
    • Storage: At least 20GB

  2. Configure security/firewall rules:

    • Allow inbound traffic on ports 22, 80, and 443 (if using HTTPS)

  3. Install DBConvert Streams:

    bash
    curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Provider-Specific Notes ​

AWS EC2:

  • Recommended instance type: t3.medium or larger

Google Cloud:

  • Recommended machine type: e2-standard-2 or larger
  • Make sure to allow HTTP/HTTPS traffic when creating the VM

Azure:

  • Recommended VM size: Standard_B2s or larger
  • Set up appropriate network security group rules

📌 Managing Services ​

🏁 Start Services ​

Before starting the services, obtain your API key from https://streams.dbconvert.com/account.

bash
# Start with standard HTTP
./start.sh

If you need to manually specify the IP address:

bash
./start.sh --ip YOUR_PUBLIC_IP

To enable HTTPS (requires generated certificates):

bash
./start.sh --secure

To explicitly use HTTP only:

bash
./start.sh --no-secure

❌ Stop Services ​

To temporarily stop all services while preserving data:

bash
./stop.sh

🔄 Update Services ​

To update the DBConvert Streams services to a newer version:

bash
# Update to latest version
./update.sh

# Update to specific version
./update.sh --version v1.0.0

Security Considerations ​

Firewall Configuration ​

It's recommended to use a firewall to restrict access to only the necessary ports:

bash
# For Ubuntu/Debian with UFW
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp  # If using HTTPS
sudo ufw enable

SSL/TLS for Production ​

The installation automatically sets up Nginx as a reverse proxy with self-signed SSL certificates. While these work for testing, they cause browser warnings in production. For a production environment:

  1. Register a domain and point it to your server's IP address
  2. Install Certbot:
    bash
    sudo apt update
sudo apt install -y certbot python3-certbot-nginx
  3. Obtain certificates:
    bash
    sudo certbot --nginx -d yourdomain.com
  4. Start with HTTPS enabled:
    bash
    ./start.sh --secure

Certbot automatically configures Nginx to use the new certificates, so no additional configuration is needed.

Option 2: Use Your Own Certificates ​

If you already have SSL certificates from another provider:

  1. Place your certificates in the config/certs directory:
    • config/certs/certificate.crt (your certificate file)
    • config/certs/private.key (your private key file)
  2. Start with HTTPS enabled:
    bash
    ./start.sh --secure

Scaling Considerations ​

Vertical Scaling ​

To improve performance, you can increase resources on your cloud instance:

  • Increase CPU and RAM
  • Use SSD storage for better I/O performance

🛠 Troubleshooting ​

Common Issues & Fixes ​

IssueFix
Docker not running?sudo systemctl start docker
Docker Compose not found?docker compose version
Wrong IP detected?./start.sh --ip YOUR_IP
HTTPS not working?./generate_certs.sh → ./start.sh
Firewall blocking ports?sudo ./configure_firewall.sh

Check Service Status ​

bash
docker compose ps

View logs:

bash
docker compose logs -f [service_name]

Reset Vault Credentials ​

bash
docker compose logs vault-init | grep "Root Token:"

IP Address Issues ​

If you're experiencing connection issues related to the IP address:

  1. Check what IP address is being used:

    bash
    cat .env.public_ip

  2. If the IP is incorrect, specify the correct IP manually:

    bash
    ./start.sh --ip YOUR_PUBLIC_IP

Cloud Provider-Specific Issues ​

  1. AWS:

    • Check that your security group is correctly configured for ports 22, 80, and 443
    • Verify that your instance type has sufficient resources

  2. Google Cloud:

    • Ensure firewall rules are correctly applied for ports 22, 80, and 443
    • Check that you've allowed HTTP traffic during VM creation

  3. Azure:

    • Verify network security group rules for ports 22, 80, and 443
    • Check that endpoints are properly configured

  4. DigitalOcean:

    • Confirm firewall settings allow traffic on required ports (22, 80, and 443)
    • Check that your Droplet has sufficient resources

🗑️ Uninstallation ​

To completely remove DBConvert Streams:

bash
./uninstall.sh

This will:

  • ✅ Stop & remove all containers
  • ✅ Delete the Docker network
  • ✅ Remove logs & configs
  • ✅ Prompt before deleting data

DBConvert Streams - event driven replication for databases

Copyright © 2025 Slotix s.r.o.