Binary Deployment Guide

Prerequisites

• Unix-based operating system (Linux or macOS)
• Either curl OR wget for downloading files
• unzip for extracting files
• sudo privileges for installing and configuring services
• At least 1GB of available RAM
• 1GB of free disk space
• 2 CPU cores minimum (3+ cores recommended)

Quick Start

One-Line Installation

Using curl:

# Install latest version
curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/binary-install.sh | sh

# Install specific version
curl -fsSL https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/binary-install.sh | sh -s -- --version v1.0.0

Using wget:

# Install latest version
wget -qO- https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/binary-install.sh | sh

# Install specific version
wget -qO- https://dbconvert.nyc3.digitaloceanspaces.com/downloads/streams/latest/binary-install.sh | sh -s -- --version v1.0.0

After installation completes:

cd ~/dbconvert-streams-binary && ./start.sh

System Components

Core Services

1. stream-api (port 8020)

   • REST API for managing database connections and stream configurations
   • Handles authentication and authorization
   • Integrates with Consul for configuration storage
   • Now proxied through nginx on port 80/443 at /api/ path

2. stream-reader

   • Reads from source databases
   • Publishes to NATS
   • Reports metrics and status

3. stream-writer

   • Consumes data from NATS
   • Writes to target databases
   • Handles schema creation

Infrastructure Services

• NATS: Message streaming
• Hashicorp Vault: Secrets management for securely storing:
  • Database passwords and credentials
  • SSL/TLS certificates
  • Client certificates
  • Other sensitive connection information
• Consul: Service discovery, health monitoring, and persistent storage for stream and connection configurations

Service Management

Starting Services

cd ~/dbconvert-streams-binary && ./start.sh

Stopping Services

cd ~/dbconvert-streams-binary && ./stop.sh

Manual Service Control

sudo systemctl start/stop nats
sudo systemctl start/stop consul
sudo systemctl start/stop vault
sudo systemctl start/stop stream-api
sudo systemctl start/stop stream-reader
sudo systemctl start/stop stream-writer
sudo systemctl start/stop nginx

Configuration Files

Core Configuration Files

• nats.conf: NATS data directory settings, connection parameters, and JetStream configuration
• vault.config.hcl: Vault storage backend settings, listener configuration, and security parameters
• consul.config.hcl: Consul configuration including data directory and server settings

Systemd Service Files

• nats.service: NATS service configuration
• vault.service: Vault service configuration
• consul.service: Consul service configuration
• stream-api.service: API service configuration
• stream-reader.service: Reader service configuration
• stream-writer.service: Writer service configuration

Logging

All logs are stored in ~/dbconvert-streams-binary/logs/ by default:

• api.log: API service logs
• reader.log: Source reader logs
• writer.log: Target writer logs
• nats.log: NATS server logs
• vault.log: Vault server logs
• consul.log: Consul server logs

Logging Configuration

DBConvert Streams services support the following logging flags for customized logging:

--log-file

Specifies a custom location and filename for the service's log file:

./stream-api --log-file=../logs/api.log
./stream-reader --log-file=../logs/source.log 
./stream-writer --log-file=../logs/target.log

--log-level

Sets the logging verbosity level. Default is info (includes info, warning, and error messages). Use debug for more verbose output:

./stream-api --log-level=debug
./stream-reader --log-level=debug
./stream-writer --log-level=debug

Updating

1. Using Update Scripts

The recommended way to update is using our update scripts:

cd ~/dbconvert-streams-binary

# Update everything (both binaries and UI)
./update.sh

# Update to specific version
./update.sh --version 1.2.3

# Update with custom installation directory
./update.sh --dir /custom/path

2. Component-Specific Updates

You can update specific components individually:

Update Binaries Only

./update_dbconvert.sh [--version VERSION] [--dir DIR]

This will:

• Download new binary versions
• Back up existing binaries
• Verify binary installations
• Preserve all configurations

Update UI Only

./update_dbconvert_ui.sh [--dir DIR]

This will:

• Clone the latest UI code
• Build using Node.js 22
• Back up existing UI files
• Preserve environment configuration
• Install new UI files

Update Process Details

During the update:

1. A backup is automatically created in ~/dbconvert-streams-binary/backup_TIMESTAMP/
   • Binaries are backed up to backup_TIMESTAMP/bin/
   • UI files are backed up to backup_TIMESTAMP/ui/
2. New versions are downloaded/built
3. Installation is verified
4. All configurations and environment settings are preserved

After Updating

After any update, you should:

1. Stop all services:

./stop.sh

2. Start services with new versions:

./start.sh

Rolling Back

If you need to rollback to a previous version:

1. Stop the services:

./stop.sh

2. Restore from backup:

cd ~/dbconvert-streams-binary
cp -r backup_TIMESTAMP/bin/* bin/
cp -r backup_TIMESTAMP/ui/* ui/

3. Restart the services:

./start.sh

Troubleshooting Updates

If you encounter issues during update:

1. Check the logs:

ls -l ~/dbconvert-streams-binary/logs/

2. Common issues and solutions:

   • UI build fails: Ensure Node.js 22+ is installed
   • Backup fails: Check available disk space
   • Services won't start: Check logs for specific errors

3. If problems persist:

   • Restore from backup using the rollback procedure above
   • Contact support with logs from ~/dbconvert-streams-binary/logs/

Uninstallation

To remove DBConvert Streams and dependencies:

cd ~/dbconvert-streams-binary && ./uninstall.sh

The uninstall script will:

• Stop all services
• Remove systemd service files
• Delete configuration directories
• Remove binary files
• Clean up data directories

Troubleshooting

1. Service Won't Start

   • Check logs in ~/dbconvert-streams-binary/logs/
   • Verify service configuration
   • Ensure required ports are available

2. Permission Issues

   • Verify sudo access
   • Check file permissions
   • Review service user settings

3. Communication Problems

   • Confirm NATS is running
   • Check network connectivity
   • Verify service discovery

For additional support, contact us at https://streams.dbconvert.com/contact