Skip to main content
Version: edge

The gbq Connector

The GBQ connector integrates Google BigQuery.

Configuration

define connector gbq from gbq
with
config = {
"table_id": "projects/tremor/datasets/test/tables/streaming_test",
"connect_timeout": 1000000,
"request_timeout: 1000000
}

The timeouts are in nanoseconds.

optiondescription
table_idThe identifier of the table in the format: projects/{project-name}/datasets/{dataset-name}/tables/{table-name}
connect_timeoutThe timeout in nanoseconds for connecting to the Google API
request_timeoutThe timeout in nanoseconds for each request to the Google API

Metadata

There is no metadata needed for this connector.

Payload structure

Currently the connector is only a sink - it expects events sent to it to be objects, where the keys match the table field names, and values are the values.

Example

For a table defined like:

CREATE TABLE test (
id INT64,
payload STRUCT<a INT64, b INT64>,
name STRING
)

An example event payload would be:

{
"id": 1234,
"payload": {"a": 1, "b": 2},
"name": "Tremor"
}