Database connectors

What the server needs to connect to PostgreSQL, MySQL or Apache Doris, and how to prepare it on a server with no internet access.

Summary

To reach a remote database, the Linkr server relies on DuckDB extensions, and for Apache Doris on an optional Python driver. The Docker image ships everything. In a manual installation, extensions download on the first connection: on a server with no internet access, install them beforehand with one command.

Client Backend

What each connector needs

DuckDB is Linkr’s query engine. It connects to other databases through extensions — small modules it downloads the first time it needs them, then keeps in the data folder.

Engine	What the server needs	Docker image	Ports to open towards the database
DuckDB, Parquet folder	nothing	✓	—
SQLite	`sqlite` extension	✓	—
PostgreSQL	`postgres` extension	✓	5432 by default
MySQL	`mysql` extension	✓	3306 by default
Apache Doris	`mysql` extension, plus the `doris` Python extra for large results	✓	9030 (FE query port); 8070 (Arrow Flight SQL) and the BEs for large results

Excel files (.xlsx) imported in server mode also go through an extension, excel.

The “postgres” extra is not for connecting to PostgreSQL

The manual installation offers apps/api[dev,postgres]. That extra stores Linkr’s own application database in PostgreSQL (see Configuration). To query a PostgreSQL warehouse, only the DuckDB postgres extension is needed.

With Docker

Nothing to do: the image ships the extensions and the Doris driver, precisely so that it works without internet access. What remains is opening, from the API container, the ports of the database to reach.

In a manual installation

DuckDB extensions without internet access

On a connected machine, the first connection to a database downloads the extension from extensions.duckdb.org and nobody notices. On a server cut off from the internet — a project space of a health data warehouse, for instance — that first connection fails with a download error.

So install them once, from the repository root, after installing the backend:

npm run duckdb:extensions

They go into the data folder (LINKR_DATA_DIR, under _duckdb_ext), where Linkr looks for them. If the server cannot reach extensions.duckdb.org at all, two options:

A mirror: if your institution runs a Nexus- or Artifactory-style repository, it can proxy http://extensions.duckdb.org. Give its address:
```
npm run duckdb:extensions -- --repository https://nexus.example.org/repository/duckdb-extensions
```
A copy: run the command on a connected machine with the same operating system and the same DuckDB version as the server, then copy its _duckdb_ext folder into the server’s data folder.

The Arrow Flight SQL driver for Apache Doris

Linkr connects to Doris over the MySQL protocol, which is enough for counts and statistics. Large results — an extraction of several million rows — are much faster over Arrow Flight SQL, a protocol built for bulk data transfer. It needs a Python driver, the doris extra:

pip install -e "apps/api[dev,doris]"

Without it everything works, but slowly for large results, and the database page says so: “Linkr’s server cannot use Arrow Flight SQL (its “doris” extra is not installed)…”. Arrow Flight SQL also needs the FE’s arrow_flight_sql_port (8070 by default) and every BE to be reachable from the Linkr server.

Extras combine: apps/api[dev,postgres,doris].

Going further

Databases — adding a database and configuring its connection.
Manual installation — installing the backend before the connectors.
Configuration — the other settings of a server on a closed network, including package repositories.