Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1112,7 +1112,7 @@ from KAFKA_SOURCE;
### Flink CDC 同步 DDL 语句
一般同步 MySQL 等上游数据源的时候,上游增加或删除字段的时候需要同步在 Doris 中进行 Schema Change 操作。

对于这种场景,通常需要编写 DataStream API 的程序,同时使用 DorisSink 提供的 JsonDebeziumSchemaSerializer 序列化,便可以自动做到 SchemaChange,具体可参考[CDCSchemaChangeExample.java](https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/src/test/java/org/apache/doris/flink/example/CDCSchemaChangeExample.java)
对于这种场景,通常需要编写 DataStream API 的程序,同时使用 DorisSink 提供的 JsonDebeziumSchemaSerializer 序列化,便可以自动做到 SchemaChange,具体可参考[CDCSchemaChangeExample.java](https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/flink-doris-connector-flink1/src/test/java/org/apache/doris/flink/example/CDCSchemaChangeExample.java)

在 Connector 提供的整库同步工具中,无需额外配置,自动会同步上游 DDL,并在 Doris 进行 SchemaChange 操作。

Expand Down
4 changes: 2 additions & 2 deletions versioned_docs/version-2.1/ecosystem/flink-doris-connector.md
Original file line number Diff line number Diff line change
Expand Up @@ -1112,7 +1112,7 @@ from KAFKA_SOURCE;
### Flink CDC Synchronize DDL Statements
Generally, when synchronizing upstream data sources such as MySQL, when adding or deleting fields in the upstream, you need to synchronize the Schema Change operation in Doris.

For this scenario, you usually need to write a program for the DataStream API and use the JsonDebeziumSchemaSerializer serializer provided by DorisSink to automatically perform SchemaChange. For details, please refer to [CDCSchemaChangeExample.java](https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/src/test/java/org/apache/doris/flink/example/CDCSchemaChangeExample.java)
For this scenario, you usually need to write a program for the DataStream API and use the JsonDebeziumSchemaSerializer serializer provided by DorisSink to automatically perform SchemaChange. For details, please refer to [CDCSchemaChangeExample.java](https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/flink-doris-connector-flink1/src/test/java/org/apache/doris/flink/example/CDCSchemaChangeExample.java)

In the whole database synchronization tool provided by the Connector, no additional configuration is required, and the upstream DDL will be automatically synchronized and the SchemaChange operation will be performed in Doris.

Expand Down Expand Up @@ -1147,4 +1147,4 @@ In the whole database synchronization tool provided by the Connector, no additio

7. **stream load error: HTTP/1.1 307 Temporary Redirect**

Flink will first request FE, and after receiving 307, it will request BE after redirection. When FE is in FullGC/high pressure/network delay, HttpClient will send data without waiting for a response within a certain period of time (3 seconds) by default. Since the request body is InputStream by default, when a 307 response is received, the data cannot be replayed and an error will be reported directly. There are three ways to solve this problem: 1. Upgrade to Connector25.1.0 or above to increase the default time; 2. Modify auto-redirect=false to directly initiate a request to BE (not applicable to some cloud scenarios); 3. The unique key model can enable batch mode.
Flink will first request FE, and after receiving 307, it will request BE after redirection. When FE is in FullGC/high pressure/network delay, HttpClient will send data without waiting for a response within a certain period of time (3 seconds) by default. Since the request body is InputStream by default, when a 307 response is received, the data cannot be replayed and an error will be reported directly. There are three ways to solve this problem: 1. Upgrade to Connector25.1.0 or above to increase the default time; 2. Modify auto-redirect=false to directly initiate a request to BE (not applicable to some cloud scenarios); 3. The unique key model can enable batch mode.