Skip to main content
Version: Next

HTTP

HTTP Source Connector

Configuration

Before using HTTP source connector, you need to configure the server.

  • Please configure sourceEnable to true in /resource/server-config.yml to enable source functionality.
  • Please configure the source connector in /resource/source-config.yml, only the configuration under connectorConfig is described here:
    • connectorName, name of the connector.
    • (required) path, path of the API.
    • (required) port, port of the API.
    • idleTimeout, idle TCP connection timeout in seconds. A connection will timeout and be closed if no data is received nor sent within the idleTimeout seconds. The default is 0, which means don't timeout.

Startup

  1. start EventMesh Runtime
  2. start eventmesh-connector-http

When finished, the HTTP source connector will act as an HTTP server.

Sending messages

You can send messages to the source connector via HTTP.

connectorConfig:
connectorName: httpSource
path: /test
port: 3755
idleTimeout: 5

The above example configures a URL http://localhost:3755/test in source-config.yml.

You can send messages in binary mode or structured mode as specified in cloudevent-spec.

Here are two examples:

  1. Sending a message in binary mode.
curl --location --request POST 'http://localhost:3755/test' \
--header 'ce-id: 1' \
--header 'ce-specversion: 1.0' \
--header 'ce-type: com.example.someevent' \
--header 'ce-source: /mycontext' \
--header 'ce-subject: test_topic' \
--header 'Content-Type: text/plain' \
--data-raw 'testdata'
  1. Sending a message in structured mode.
curl --location --request POST 'http://localhost:3755/test' \
--header 'Content-Type: application/cloudevents+json' \
--data-raw '{
"id": "1",
"specversion": "1.0",
"type": "com.example.someevent",
"source": "/mycontext",
"subject":"test_topic",
"datacontenttype":"text/plain",
"data": "testdata"
}'

HTTP Sink Connector

The HTTP sink connector has two modes: common and webhook.

First of all, either mode has the basic function of sinkConnector, which is to send a message to the target HTTP server. When the response is received from the HTTP server, the common mode only cares about the status code of the response to determine whether it was sent successfully, while the webook mode not only cares about whether it was sent successfully or not, but also stores the data of the response (called the callback data) and provides callback data exposure services to the outside world.

The specific differences between the two are as follows:

  • Default Idle TCP Connection Timeout

    The default value is 5000ms for common mode and 15000ms for webhook mode.

  • Processing of response results

    The common mode only cares about the status code of the response, while the webhook mode will also store the callback data and provide callback data exposure services to the public.

Configuration

Before using the HTTP sink connector, you need to configure the sink.

  • Please configure sinkEnable to true in /resource/server-config.yml to enable sink function.

  • Please configure the sink connector in /resource/sink-config.yml, only the configuration under connectorConfig is described here:

    The simplest configuration of the common mode:

    connectorConfig:
    connectorName: httpSink
    urls:
    - http://127.0.0.1:8987/test

    The simplest configuration for webhook mode:

    connectorConfig:
    connectorName: httpSink
    urls:
    - http://127.0.0.1:8987/test
    webhookConfig:
    activate: true
    port: 8988

All configurations for the HTTP sink connector are listed below:

FieldTypeRequired or notClarificationRemark
connectorNameStringyesConnector Name
urlsListyesList of URLs to receive messagesThe default port is 80 for HTTP protocol and 443 for HTTPS protocol
The legal URL format are:
http://127.0.0.1:8987/test
http://127.0.0.1/test
https://example.com:4943/test
https://example.com/test
keepAliveBooleannoWhether to use HTTP persistent connectionDefault: true
keepAliveTimeoutIntegernoHTTP persistent connection timeoutUnit: ms,default: 60000
connectionTimeoutIntegernoTCP connection timeoutUnit: ms,default: 5000
idleTimeoutIntegernoTCP idle timeoutUnit: ms,default: 5000(common), 15000(webhook)
maxConnectionPoolSizeIntegernoMaximum number of HTTP connections for the clientDefault: 5
retryConfigObjectConfiguration of the retry mechanism
retryConfig.maxRetriesIntegernoMaximum number of retriesDoes not include first attempt, default: 2
retryConfig.intervalIntegernoRetry intervalUnit: ms, default: 2000
retryConfig.retryOnNonSuccessBooleannoWhether to retry requests that receive a non-2xx responseDefault: false, retries only network level error requests
webhookConfigObjectConfiguration of Webhook Mode
webhookConfig.activateBooleannoWhether to enable Webhook modeDefault: false
webhookConfig.exportPathStringnoCallback data exposure pathDefault: /export
The request method is fixed to GET
Request Parameters:
type:peek (default value, get data)/poll (get and delete data)
pageNum: must be greater than 0 when type is peek, invalid when type is poll
pageSize: must be greater than 0
webhookConfig.portIntegerYes (when enabled)Callback data exposure port
webhookConfig.serverIdleTimeoutIntegernoTCP idle timeout length for servers exposing callback dataUnit: ms, default: 5000
webhookConfig.maxStorageSizeIntegernoStore the maximum value of the callback data queueDefault: 5000

Startup

  1. start EventMesh Runtime
  2. start eventmesh-connector-http

Once that's done, send a message to the EventMesh, and then your message will happen to the target HTTP server via an HTTP request. If webhook mode is enabled, an HTTP server will also be opened at the same time to provide callback data exposure to the outside world.

Data Format

The data format of the message sent by the HTTP sink connector is as follows:

  • type: consists of connectorName, protocol, and mode.
  • time: the time the request was sent
  • uuid: request unique identifier, one-to-one with the uuid of the callback data structure in webhook mode
  • eventId: event ID, composed of type and offset
  • data: actual data to be sent
{
"type": "httpSink.http.webhook",
"time": "2024-05-09T22:27:02.028",
"uuid": "951d88ee-2503-4df3-a1dd-6e4a6a1c3d3b",
"eventId": "httpSink.http.webhook-0",
"data": ${ConnectRecord}
}

When webhook mode is enabled, an HTTP server is also enabled to provide callback data exposure to the outside world. The format of the callback data is as follows:

  • pageNum: current page number
  • pageSize: size per page
  • pageItems: the callback data array obtained
  • The meaning of the fields of pageItem is as follows:
    • data: callback data returned from the callback server
    • metadata: metadata, the fields have the following meanings:
      • url: the URL to get the callback data
      • code: the status code of the response to the request callback data, or -1 if a network level error occurred ( response could not be received).
      • message: response information from the request callback data, or exception information if a network level error occurred (response could not be received).
      • uuid: a unique request identifier for the callback data, corresponding to the uuid carried in the message sent above.
      • receivedTime: response time for requesting callback data
      • retryNum: number of retries
      • retriedBy: record the uuid of the request being retried if a retry is sent, otherwise null
{
"pageNum": 1,
"pageSize": 10,
"pageItems": [
{
"data": "callbackData",
"metadata": {
"url": "http://127.0.0.1:8987/test",
"code": 200,
"message": "OK",
"receivedTime": "2024-05-09 22:53:21.556",
"uuid": "fba29061-1a1f-4482-9c83-43ba4e0bcf3f",
"retriedBy": null,
"retryNum": 0
}
}
]
}