Docker Cloud Deployment

This guide covers deploying DBConvert Streams to cloud hosting providers such as AWS, Google Cloud, Azure, DigitalOcean, Hetzner.

🔥 Quick Start

Easiest Installation (Recommended)

bash

curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

This automatically:

✅ Installs Docker Compose (if missing)
✅ Sets up networking & volumes
✅ Detects public/local IP
✅ Configures NGINX
✅ Generates SSL Certificates.

DigitalOcean Marketplace

For the easiest cloud deployment option, DBConvert Streams is available as a 1-Click App on the DigitalOcean Marketplace: DBConvert Streams on DigitalOcean Marketplace

Prerequisites

Docker Engine 20.10.0 or newer
Docker Compose v2.0.0 or newer
Either curl OR wget for downloading files
At least 2GB of available RAM
2GB of free disk space
2 CPU cores minimum (3+ cores recommended)
A cloud server instance with a public IP address

Required Ports

When deploying to a cloud hosting provider, you need to ensure the following ports are open in your hosting provider's security group or firewall:

Port	Service	Description
22	SSH	For SSH access to the server
80	HTTP	For Streams UI and API access
443	HTTPS	For secure UI and API access (if using HTTPS)

These ports must be opened on your hosting provider to allow incoming connections from the internet. All other services run internally and don't need exposed ports.

API Access

The Streams API is now accessible through the Nginx proxy at:

HTTP: http://HOST_ADDR/api/
HTTPS: https://HOST_ADDR/api/ (if HTTPS is enabled)

You no longer need to expose port 8020 directly.

📥 Installation

1️⃣ Basic Install

Using `curl`:

bash

curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Using `wget`:

bash

wget -qO- https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

2️⃣ Advanced Install Options

Feature	Command
Install Specific Version	`./install.sh -v VERSION`
Custom Install Directory	`./install.sh -d /path/to/dir`
Show Help	`./install.sh --help`

Example:

bash

# Install specific version to custom directory
./install.sh -v 1.0.0 -d /opt/dbconvert-streams

IP Address Configuration

DBConvert Streams automatically detects and configures your server's public IP address for proper SSL certificate generation and network settings.

If you're experiencing connection issues, you can manually specify the IP address:

bash

./start.sh --ip YOUR_PUBLIC_IP

Cloud Provider-Specific Instructions

DigitalOcean (Recommended)

Option 1: 1-Click App (Easiest Method)

Visit the DBConvert Streams on DigitalOcean Marketplace
Click "Create DBConvert Streams Droplet"
Select your preferred Droplet size (2GB RAM / 2 vCPUs minimum recommended)
Select a datacenter region
Add your SSH keys
Click "Create Droplet"
Wait for deployment to complete (~2 minutes)
Access your instance at the Droplet's IP address

Option 2: Manual Installation on a Droplet

Create a Droplet:
- Recommended: Standard Droplet with 2GB RAM / 2 vCPUs or larger
- OS: Ubuntu 20.04 or newer

Install DBConvert Streams:

bash

curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Other Cloud Providers

The installation process is similar for AWS, Google Cloud, and Azure:

Create a VM/instance:
- Recommended specs: 2GB+ RAM, 2+ vCPUs
- Supported OS: Ubuntu 20.04+, Debian 10+, CentOS 7+, Amazon Linux 2
- Storage: At least 20GB
Configure security/firewall rules:
- Allow inbound traffic on ports 22, 80, and 443 (if using HTTPS)

Install DBConvert Streams:

bash

curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/docker-install.sh | sh

Provider-Specific Notes

AWS EC2:

Recommended instance type: t3.medium or larger

Google Cloud:

Recommended machine type: e2-standard-2 or larger
Make sure to allow HTTP/HTTPS traffic when creating the VM

Azure:

Recommended VM size: Standard_B2s or larger
Set up appropriate network security group rules

📌 Managing Services

🏁 Start Services

Before starting the services, obtain your API key from https://streams.dbconvert.com/account.

bash

# Start with standard HTTP
./start.sh

To enable HTTPS (requires generated certificates):

bash

./start.sh --secure

To explicitly use HTTP only:

bash

./start.sh --no-secure

If you need to manually specify the IP address:

bash

./start.sh --ip YOUR_PUBLIC_IP

❌ Stop Services

To temporarily stop all services while preserving data:

bash

./stop.sh

🔄 Update Services

To update the DBConvert Streams services to a newer version:

bash

# Update to latest version
./update.sh

# Update to specific version
./update.sh --version v1.0.0

🛠 Service Architecture

DBConvert Streams consists of 3 main parts:

1️⃣ Core Services

stream-api: REST API (via Nginx)
stream-reader: Reads source databases
stream-writer (multiple instances): Writes to target DBs
stream-ui: Web interface (via Nginx)

2️⃣ Infrastructure Services

nats: Messaging system (internal only)
consul: Service discovery (internal only)
vault: Secrets management (internal only)
nginx: Reverse proxy + SSL (ports 80, 443)

3️⃣ Network & Security

All services run in stream-net (Docker network).
Nginx handles HTTPS & API routing.
Internal services are not exposed.
Stream logs are delivered via Server-Sent Events (SSE) through the Nginx proxy.

Security Considerations

Firewall Configuration

Each cloud provider has its own firewall/security group system. You'll need to allow these ports:

Port 22 (SSH)
Port 80 (HTTP)
Port 443 (HTTPS) - if using HTTPS

SSL/TLS for Production

The installation automatically sets up Nginx as a reverse proxy with self-signed SSL certificates. While these work for testing, they cause browser warnings in production. For a production environment:

Option 1: Use Let's Encrypt (Recommended)

Register a domain and point it to your server's IP address

Install Certbot:

bash

sudo apt update
sudo apt install -y certbot python3-certbot-nginx

Obtain certificates:
bash
```
sudo certbot --nginx -d yourdomain.com
```
Start with HTTPS enabled:
bash
```
./start.sh --secure
```

Certbot automatically configures Nginx to use the new certificates, so no additional configuration is needed.

Option 2: Use Your Own Certificates

If you already have SSL certificates from another provider:

Place your certificates in the config/certs directory:
- config/certs/certificate.crt (your certificate file)
- config/certs/private.key (your private key file)
Start with HTTPS enabled:
bash
```
./start.sh --secure
```

🛠 Troubleshooting

Common Issues & Fixes

Issue	Fix
Docker not running?	`sudo systemctl start docker`
Docker Compose not found?	`docker compose version`
Wrong IP detected?	`./start.sh --ip YOUR_IP`
HTTPS not working?	`./generate_certs.sh` → `./start.sh`

Check Service Status

bash

docker compose ps

View logs:

bash

docker compose logs -f [service_name]

Available services for log viewing:

stream-ui - Web interface
stream-api - REST API service
stream-reader - Database reader service
stream-writer-1, stream-writer-2 - Database writer services
nginx - Reverse proxy and SSL termination
nats - Message broker
vault - Secrets management
consul - Service discovery

Example:

bash

# View API service logs
docker compose logs -f stream-api

# View writer logs
docker compose logs -f stream-writer-1

Find Vault Token

If you need to find your Vault token, check the installation log:

bash

grep VAULT_TOKEN install.log

The output will show your token in the format: VAULT_TOKEN=hvs.xxxxxxxxxxxxxxxx

IP Address Issues

If you're experiencing connection issues related to the IP address:

Check what IP address is being used:

bash

cat config/host_info.conf

You'll see information like:

PUBLIC_IP="34.172.226.34"
ENVIRONMENT="Cloud (gcp)"
DETECTED_AT="2025-04-07 16:30:34 UTC"

If the IP is incorrect, specify the correct IP manually:
bash
```
./start.sh --ip YOUR_PUBLIC_IP
```

🗑️ Uninstallation

To completely remove DBConvert Streams:

bash

./uninstall.sh

This will:

✅ Stop & remove all containers
✅ Delete the Docker network
✅ Remove logs & configs
✅ Prompt before deleting data

Docker Cloud Deployment ​

🔥 Quick Start ​

Easiest Installation (Recommended) ​

DigitalOcean Marketplace ​

📖 Table of Contents ​

Prerequisites ​

Required Ports ​

📥 Installation ​

1️⃣ Basic Install ​

Using curl: ​

Using wget: ​

2️⃣ Advanced Install Options ​

IP Address Configuration ​

Cloud Provider-Specific Instructions ​

DigitalOcean (Recommended) ​

Other Cloud Providers ​

Provider-Specific Notes ​

📌 Managing Services ​

🏁 Start Services ​

❌ Stop Services ​

🔄 Update Services ​

🛠 Service Architecture ​

1️⃣ Core Services ​

2️⃣ Infrastructure Services ​

3️⃣ Network & Security ​

Security Considerations ​

Firewall Configuration ​

SSL/TLS for Production ​

Option 1: Use Let's Encrypt (Recommended) ​

Option 2: Use Your Own Certificates ​

🛠 Troubleshooting ​

Common Issues & Fixes ​

Check Service Status ​

Find Vault Token ​

IP Address Issues ​

🗑️ Uninstallation ​

Docker Cloud Deployment

🔥 Quick Start

Easiest Installation (Recommended)

DigitalOcean Marketplace

📖 Table of Contents

Prerequisites

Required Ports

📥 Installation

1️⃣ Basic Install

Using `curl`:

Using `wget`:

2️⃣ Advanced Install Options

IP Address Configuration

Cloud Provider-Specific Instructions

DigitalOcean (Recommended)

Other Cloud Providers

Provider-Specific Notes

📌 Managing Services

🏁 Start Services

❌ Stop Services

🔄 Update Services

🛠 Service Architecture

1️⃣ Core Services

2️⃣ Infrastructure Services

3️⃣ Network & Security

Security Considerations

Firewall Configuration

SSL/TLS for Production

Option 1: Use Let's Encrypt (Recommended)

Option 2: Use Your Own Certificates

🛠 Troubleshooting

Common Issues & Fixes

Check Service Status

Find Vault Token

IP Address Issues

🗑️ Uninstallation