DBConvert Streams Docker images.

DBConvert Streams docker images.

DBConvert Streams services are available as Docker images on Docker Hub repository. DBS docker images are extremely lightweight under 10 Mb each.

DBConvert Streams consists of the following internal services:

DBS platform depends on the following third-party services:

NATS server provides connectivity between internal DBS Services.
Prometheus is optionally used to collect internal metrics of DBS services.

Creating Docker Compose configuration.

Since DBConvert Streams depends on multiple services, the best way to start docker containers using Docker Compose.

Here is a simple example of docker-compose configuration for staring DBConvert Streams.

yaml

version: "3.9"
services:
  dbs-api:
    container_name: api
    image: slotix/dbs-api
    entrypoint:
      - ./dbs-api
      - --nats=nats:4222
      - --source=source-reader:8021
      - --target=target-writer:8022
    ports:
      - 8020:8020
    depends_on:
      - nats

  dbs-source-reader:
    container_name: source-reader
    image: slotix/dbs-source-reader
    entrypoint:
      - ./dbs-source-reader
      - --nats=nats:4222
    ports:
      - 8021:8021
    depends_on:
      - dbs-api

  dbs-target-writer:
    container_name: target-writer
    image: slotix/dbs-target-writer
    entrypoint:
      - ./dbs-target-writer
      - --host=dbs-target-writer:8022  
      - --nats=nats:4222
      - --prometheus=http://prometheus:9090
    ports:
      - 8022:8022
    depends_on:
      - dbs-api


  # dbs-target-writer2:
  #   container_name: target-writer2
  #   image: slotix/dbs-target-writer
  #   entrypoint:
  #     - ./dbs-target-writer
  #     - --host=dbs-target-writer2:8023  
  #     - --nats=nats:4222
  #     - --prometheus=http://prometheus:9090
  #   ports:
  #     - 8023:8023
  #   depends_on:
  #     - dbs-api


  # dbs-target-writer3:
  #   container_name: target-writer3
  #   image: slotix/dbs-target-writer
  #   entrypoint:
  #     - ./dbs-target-writer
  #     - --host=dbs-target-writer3:8024  
  #     - --nats=nats:4222
  #     - --prometheus=http://prometheus:9090
  #   ports:
  #     - 8024:8024
  #   depends_on:
  #     - dbs-api

  nats:
    container_name: nats
    image: nats
    entrypoint: /nats-server
    command: "--jetstream -m 8222 --store_dir /data/nats-server"
    ports:
      - 4222:4222
      - 8222:8222
    # volumes:
    #   - ./_storage/nats:/data/nats-server/jetstream

  prometheus:
    image: slotix/dbs-prometheus:latest
    container_name: prom
    user: root
    ports:
      - 9090:9090

To improve overall performance, you can run multiple instances of the 'Target Writer' by uncommenting the 'dbs-target-writer-2' and 'dbs-target-writer-3' services. This will add more Target Writer instances and distribute the processing of multiple 'INSERT' statements among several Target Writers, greatly increasing the speed of the entire process.

Please copy content of the docker-compose config above and save it as docker-compose.yml file.

Check out our github repository for a few config examples.

Run.

Start all services with the following docker-compose command.

bash

docker-compose up -d

Check if all services are started properly.

bash

docker ps

bash

CONTAINER ID   IMAGE                          COMMAND                  CREATED          STATUS          PORTS                                                                                            NAMES
c7079e6488b8   slotix/dbs-api                 "./dbs-api --nats=na…"   48 seconds ago   Up 46 seconds   0.0.0.0:8020->8020/tcp, :::8020->8020/tcp                                                        api
f538a75dd135   slotix/dbs-target-writer       "./dbs-target-writer…"   48 seconds ago   Up 47 seconds   0.0.0.0:8022->8022/tcp, :::8022->8022/tcp                                                        target-writer
c8188a721f5e   slotix/dbs-source-reader       "./dbs-source-reader…"   48 seconds ago   Up 46 seconds   0.0.0.0:8021->8021/tcp, :::8021->8021/tcp                                                        source-reader
72ec84efad6a   slotix/dbs-prometheus:latest   "/bin/prometheus --c…"   48 seconds ago   Up 47 seconds   0.0.0.0:9090->9090/tcp, :::9090->9090/tcp                                                        prom
3d091c386b0b   nats                           "/nats-server --jets…"   48 seconds ago   Up 47 seconds   0.0.0.0:4222->4222/tcp, :::4222->4222/tcp, 0.0.0.0:8222->8222/tcp, :::8222->8222/tcp, 6222/tcp   nats

If you now check prometheus target endpoint in a browser, you should see that all source and target endpoints have changed their status to UP.

prometheus running services

Examples

For actual examples and configurations, refer to the public DBConvert Streams Github repository

Follow the steps in Run binaries section if you prefer to install binaries instead of running Docker containers.

DBConvert Streams Docker images. #