Skip to main content

Source configuration

sh.oso.servicenow.source.ServiceNowSourceConnector

Property-by-property reference generated from the connector's ConfigDef, so it always matches the release you are running. Usage guide: Source Connector.

Framework properties (name, tasks.max, converters, errors.*) follow the standard Kafka Connect connector configuration. Each alias listed in snow.tables names its table and topic with the per-table properties at the end of this page; there is no topics property on the source.

High importance​

snow.tables​

Comma-separated aliases, one per table to poll. Each alias is configured with snow.table.<alias>.* keys; aliases are lower-case letters, digits and underscores. Tables are assigned whole to tasks by rendezvous hashing.

  • Type: list
  • Default: none (required)
  • Importance: high

snow.source.schema.mode​

Record value shape: schemaless (a map, for the JSON converter without schemas), strings (a Struct of optional strings whose schema only grows) or typed (a Struct from snow.source.typed.fields; experimental).

  • Type: string
  • Default: schemaless
  • Valid values: [schemaless, strings, typed]
  • Importance: high

snow.url​

ServiceNow instance URL, for example https://acme.service-now.com. Only the scheme, host and port are used; the lower-cased host identifies the instance in source partitions.

  • Type: string
  • Default: none (required)
  • Valid values: non-empty string
  • Importance: high

snow.auth.type​

Authentication flow: basic (username and password on every request) or oauth2 (bearer tokens from the instance token endpoint, see snow.oauth.grant.type).

  • Type: string
  • Default: basic
  • Valid values: [basic, oauth2]
  • Importance: high

snow.auth.username​

Integration user. Required for basic and for the OAuth password grant. The user must have its time zone set to UTC.

  • Type: string
  • Default: null
  • Importance: high

snow.auth.password​

Password for snow.auth.username. Required for basic and for the OAuth password grant.

  • Type: password
  • Default: null
  • Importance: high

Medium importance​

snow.source.poll.interval.ms​

Milliseconds a table waits after a sweep that ended on a short page before polling again. A full page continues immediately. Overridable per alias.

  • Type: int
  • Default: 5000
  • Valid values: [1,...]
  • Importance: medium

snow.source.batch.size​

Rows requested per Table API call (sysparm_limit). Overridable per alias.

  • Type: int
  • Default: 5000
  • Valid values: [1,...,50000]
  • Importance: medium

snow.source.typed.fields​

field:type entries for typed mode, for example sys_id:string,priority:int32,opened_at:timestamp,active:boolean. Types: string, int8, int16, int32, int64, float32, float64, boolean, bytes, timestamp, date, time, decimal, decimal(scale).

  • Type: list
  • Default: "" (empty)
  • Importance: medium

snow.oauth.grant.type​

OAuth 2.0 grant when snow.auth.type=oauth2: client_credentials (needs the instance property glide.oauth.inbound.client.credential.grant_type.enabled and an application user) or password (resource owner password, reusing snow.auth.username and snow.auth.password).

  • Type: string
  • Default: client_credentials
  • Valid values: [client_credentials, password]
  • Importance: medium

snow.oauth.client.id​

OAuth application registry client id. Required when snow.auth.type=oauth2.

  • Type: string
  • Default: null
  • Importance: medium

snow.oauth.client.secret​

OAuth application registry client secret. Required when snow.auth.type=oauth2.

  • Type: password
  • Default: null
  • Importance: medium

snow.http.max.concurrent.requests​

Maximum requests in flight to this instance from this worker JVM, shared by every connector that targets the same instance.

  • Type: int
  • Default: 8
  • Valid values: [1,...]
  • Importance: medium

snow.retry.max.attempts​

Total attempts per request for retryable failures (429, 408, 425, 5xx, transient I/O) before the failure surfaces to the task as retriable.

  • Type: int
  • Default: 5
  • Valid values: [1,...]
  • Importance: medium

snow.retry.max.elapsed.ms​

Retrying stops once the next wait would take the request past this many milliseconds in total.

  • Type: long
  • Default: 300000
  • Valid values: [0,...]
  • Importance: medium

Low importance​

snow.source.overlap.seconds​

Seconds re-read before the committed cursor on restart and at the start of every streaming sweep, so rows committed late with an earlier timestamp are still seen. Re-read rows are suppressed by the dedup cache; after a restart they are delivered again (at-least-once).

  • Type: int
  • Default: 2
  • Valid values: [0,...]
  • Importance: low

snow.source.safety.lag.seconds​

Seconds subtracted from the instance clock to form a sweep's high-water mark, so the second still being written is never closed early.

  • Type: int
  • Default: 1
  • Valid values: [0,...]
  • Importance: low

snow.source.dedup.window.records​

Per-table bound on the in-memory cache of emitted (timestamp, sys_id, sys_mod_count) versions used to suppress overlap re-reads.

  • Type: int
  • Default: 50000
  • Valid values: [1,...]
  • Importance: low

snow.source.bad.row.behavior​

A row whose sys_id or timestamp field is missing or unparseable either fails the task naming the row (fail) or is logged, counted and skipped (skip).

  • Type: string
  • Default: fail
  • Valid values: [fail, skip]
  • Importance: low

snow.source.startup.probe​

On task start, read one row per table to verify access and the cursor fields, and warn when the integration user's time zone is not UTC.

  • Type: boolean
  • Default: true
  • Importance: low

snow.source.schema.evolution​

In typed mode, what happens with a field that is not in the mapping or does not coerce: fail rejects the record, backward adds unknown fields as optional strings but still rejects bad values, permissive adds unknown fields and nulls bad values.

  • Type: string
  • Default: fail
  • Valid values: [fail, backward, permissive]
  • Importance: low

snow.source.emit.envelope​

Wrap each value as {before: null, after, source: {table, instance}, op: "u", ts_ms}. before is always null because the Table API has no previous image.

  • Type: boolean
  • Default: false
  • Importance: low

snow.oauth.token.url​

Token endpoint override. Defaults to {snow.url}/oauth_token.do.

  • Type: string
  • Default: null
  • Importance: low

snow.oauth.scope​

Optional OAuth scope sent with the token request.

  • Type: string
  • Default: null
  • Importance: low

snow.http.connect.timeout.ms​

TCP connect timeout in milliseconds.

  • Type: int
  • Default: 10000
  • Valid values: [1,...]
  • Importance: low

snow.http.request.timeout.ms​

Per-request timeout in milliseconds (time to response headers). Also bounds the wait for a concurrency permit.

  • Type: int
  • Default: 60000
  • Valid values: [1,...]
  • Importance: low

snow.http.proxy.url​

HTTP proxy, for example http://proxy.internal:3128.

  • Type: string
  • Default: null
  • Importance: low

snow.http.proxy.username​

Proxy username, when the proxy requires authentication.

  • Type: string
  • Default: null
  • Importance: low

snow.http.proxy.password​

Proxy password.

  • Type: password
  • Default: null
  • Importance: low

snow.http.max.response.bytes​

Responses larger than this many bytes (after decompression) fail the request.

  • Type: long
  • Default: 67108864
  • Valid values: [1024,...]
  • Importance: low

snow.http.user.agent​

User-Agent header sent with every request.

  • Type: string
  • Default: kafka-connect-servicenow/<version>
  • Importance: low

snow.http.adaptive.throttling​

Halve the in-flight limit when the instance answers 429 and recover one permit per 50 successful requests, never above snow.http.max.concurrent.requests.

  • Type: boolean
  • Default: true
  • Importance: low

snow.tls.truststore.path​

Truststore file for the instance certificate chain, when the JDK defaults are not enough.

  • Type: string
  • Default: null
  • Importance: low

snow.tls.truststore.password​

Truststore password.

  • Type: password
  • Default: null
  • Importance: low

snow.tls.truststore.type​

Truststore type (JKS or PKCS12).

  • Type: string
  • Default: JKS
  • Importance: low

snow.tls.keystore.path​

Keystore holding the client certificate for mutual TLS.

  • Type: string
  • Default: null
  • Importance: low

snow.tls.keystore.password​

Keystore password.

  • Type: password
  • Default: null
  • Importance: low

snow.tls.keystore.type​

Keystore type (JKS or PKCS12).

  • Type: string
  • Default: JKS
  • Importance: low

snow.retry.initial.backoff.ms​

Ceiling of the first full-jitter backoff in milliseconds; doubles per attempt.

  • Type: long
  • Default: 500
  • Valid values: [0,...]
  • Importance: low

snow.retry.max.backoff.ms​

Largest backoff ceiling in milliseconds. A Retry-After header overrides the backoff.

  • Type: long
  • Default: 30000
  • Valid values: [0,...]
  • Importance: low

Per-table properties (snow.table.<alias>.*)​

Every alias listed in snow.tables is configured with these keys, with <alias> replaced by the alias (for example snow.table.incident.name). They are defined per alias in the configuration group Table <alias> and are validated and reported by PUT /connector-plugins/{class}/config/validate like any other key.

snow.table.<alias>.name​

ServiceNow table or database view to poll, for example incident. Lower-case letters, digits and underscores only. The partition is keyed on this name, not on the alias.

  • Type: string
  • Default: none (required)
  • Valid values: non-empty string
  • Importance: high

snow.table.<alias>.topic​

Kafka topic that receives this table's records.

  • Type: string
  • Default: none (required)
  • Valid values: non-empty string
  • Importance: high

snow.table.<alias>.start.timestamp​

Where the backfill starts when no offset is stored for this table: rows whose timestamp field is at or after this UTC yyyy-MM-dd HH:mm:ss value are emitted. Ignored once an offset exists.

  • Type: string
  • Default: 1970-01-01 00:00:00
  • Importance: medium

snow.table.<alias>.timestamp.field​

Cursor timestamp field. sys_updated_on captures creates and updates; sys_created_on captures creates only. For a database view use the prefixed column, for example inc_sys_updated_on.

  • Type: string
  • Default: sys_updated_on
  • Valid values: non-empty string
  • Importance: medium

snow.table.<alias>.query​

Base encoded query ANDed with the cursor predicates, for example active=true^priority<=2. Must not contain ORDERBY; the connector orders by the cursor fields. Changing it changes the partition (the table starts again from its start timestamp).

  • Type: string
  • Default: "" (empty)
  • Importance: medium

snow.table.<alias>.fields​

Projection (sysparm_fields). Empty means every field. When set it must include the sys_id field and the timestamp field; sys_mod_count is added automatically.

  • Type: list
  • Default: "" (empty)
  • Importance: medium

snow.table.<alias>.sys.id.field​

Cursor identity field and record key. Only database views need to change it, to the prefixed column such as inc_sys_id.

  • Type: string
  • Default: sys_id
  • Valid values: non-empty string
  • Importance: low

snow.table.<alias>.display.value​

sysparm_display_value: false emits raw values, all emits {value, display_value, link} per field, and true emits display values for every field except the cursor fields, which keep their raw values so the cursor stays parseable (fetched as all on the wire).

  • Type: string
  • Default: false
  • Valid values: [false, true, all]
  • Importance: low

sysparm_exclude_reference_link: drop the link of reference fields.

  • Type: boolean
  • Default: true
  • Importance: low

snow.table.<alias>.query.domain​

true restricts rows to the integration user's domains (ServiceNow default). false sends sysparm_query_no_domain=true, which needs the query_no_domain_table_api role. Part of the partition fingerprint.

  • Type: boolean
  • Default: true
  • Importance: low

snow.table.<alias>.query.category​

sysparm_query_category for instance-side query routing and auditing.

  • Type: string
  • Default: null
  • Importance: low

snow.table.<alias>.batch.size​

Rows per request for this table; overrides snow.source.batch.size.

  • Type: int
  • Default: null
  • Importance: low

snow.table.<alias>.poll.interval.ms​

Milliseconds to wait after a sweep that found no full page; overrides snow.source.poll.interval.ms.

  • Type: int
  • Default: null
  • Importance: low