Skip to content
On this page

Source configuration.

DBS readers collect data from external sources such as MySQL and PostgreSQL and write the collected events to the target system of your choice.

This section provides general information about source configuration options. Some specific configuration options for different source types are described in the relevant sections.

Here is an example of a JSON source config object.

json
"source": {
    "mode": "reader-mode",
    "type": "source-type",
    "connection": "connection-string",
    "dataBundleSize": 100,
    "reportingInterval": 5,
    "settings": {
      //settings are unique for each source type
    },
    "filter": {
      "tables": [
        { "name": "table1",
          "operations": ["insert", "update", "delete"],
          "query": "SELECT * FROM table1 LIMIT 10000 OFFSET 0"
        },
        { "name": "table2",
          "operations": ["insert", "update", "delete"]
        }
      ]
    }
  }
propertytypedescription
modesourcerepresent mode to read the source. It may be either cdc or convert
typestringSource type. It can be either mysql or postgresql.
connectionstringConnection settings. See the relevant sections for specific data sources.
dataBundleSizenumberhelps prevent NATS errors related to slow consumers and dropped messages by optimizing the size of data bundles during transmission.
reportingIntervalnumber(in seconds). It allows users to define the frequency at which progress reports are generated to keep users informed about the status of data transfer. If reportingInterval is set to zero or omitted, no statistics will be returned while the stream is running, providing flexibility for silent operation.
settings-The settings are unique for each source type. Find more details about the settings in the relevant sections of the documentation.
filter/ tablesobjectTables for which change data is returned. Tables must have primary keys (required for logical replication).
namestreamName of table to be converted or replicated.
operationsenumA set of Change Data Capture types (insert, update, delete) of events to track.
querystringA custom SQL query to be executed against the source table. You can customize queries for each table, including conditions, ordering, limiting.

DBConvert Streams - event driven replication for databases