cbcopy issue

[adnanhamdussalam]

Hi,

I am facing SQLSTATE 22P04 bad_copy_file_format as few tables. PFB the error snapshot

20260622:15:30:01 cbcopy:gpadmin:CB-Coord:166687-[ERROR]:-[Worker 0] Failed to copy table "staging.tmp_base_deduped": ERROR: missing data for column "data_tec_asset" (seg2
slice1 192.168.14.127:50002 pid=455087) (SQLSTATE 22P04)

cbcopy version is 1.1.4

Any solution for above

[talmacschen-arch]

Hi @adnanhamdussalam, thanks for the report. A few things to help narrow this down.

SQLSTATE 22P04 / missing data for column "data_tec_asset" is raised on the destination side (COPY ... FROM): a CSV row arriving at the target had fewer fields than the table has, so parsing ran out of input before reaching data_tec_asset. The seg2 slice1 in the message means this table is being transferred on the segments (large table / copy-on-segment), not on the coordinator. There are two main causes:

1. The data transfer was interrupted mid-stream. This is the most common source of this exact message — see the troubleshooting section in the cbcopy README. If the destination cbcopy_helper exits abruptly or the connection breaks, the QE reads a partial row from the pipe and reports missing data for column. Typical triggers:

• Segment-to-segment network/firewall blocking. A tell-tale sign is small tables succeed but large tables fail, because large tables go segment-to-segment.
• A manually cancelled query, or a helper that crashed/was killed.
• This case is usually accompanied by a could not write to copy program: Broken pipe error on the source side.

2. A column-count / column-order mismatch between the source and destination tables. cbcopy transfers via positional CSV with no explicit column list, so the source and destination must have the same number of columns in the same order. If the destination staging.tmp_base_deduped was pre-created (e.g. by an ETL job) with an extra column or a different order, every row fails at the same column. This case is deterministic — it fails at the same column on every run, regardless of table size.

To tell these apart, could you share:

1. The full cbcopy command line (copy mode, --truncate, push/pull, etc.).
2. Whether it fails on the same column every time, and whether only large tables fail while small ones succeed.
3. Whether the source log shows could not write to copy program: Broken pipe around the same time(/home/gpadmin/gpAdminLogs/ and pg_log)
4. Whether staging.tmp_base_deduped was created by cbcopy or pre-existed at the destination.
5. \d staging.tmp_base_deduped on both source and destination (column count and the position of data_tec_asset), plus:

   SELECT attname, attnum, attisdropped FROM pg_attribute
   WHERE attrelid = 'staging.tmp_base_deduped'::regclass ORDER BY attnum;

Also, a friendly heads-up: cbcopy has its own issue tracker at https://github.com/cloudberry-contrib/cbcopy/issues — filing cbcopy-specific issues there will get them in front of the cbcopy maintainers faster.

[adnanhamdussalam]

Hi @talmacschen-arch ,

Thanks for the update.

I have facing on different tables. Now I have try again on table the table size is 115 GB on source GP and on dest CB when it reaches at 334 MB the copy command on source GP get terminates.

Below are details (cbcopy command , source and dest table info and cbcopy_helper logs ):

CBcbopy command:
cbcopy --source-host 10.195.157.29 --source-port 55432 --source-user gpadmin --dest-host 10.195.157.19 --dest-user gpadmin --dest-port 55432
--include-table gpafiniti.applied_ai_staging.tim_expansion_crm_btntiid --data-only --copy-jobs 2 --compression --truncate --verbose

Cbopy error log :
20260624:13:56:19 cbcopy:gpadmin:TIM-CB-Coord:195204-[ERROR]:-[Worker 1] Failed to copy table
"applied_ai_staging.tim_expansion_crm_btntiid": ERROR: missing data for column "offer_name_servizio_btn"
(seg6 slice1 192.168.14.8:50002 pid=593483) (SQLSTATE 22P04)

Source GP : gpafiniti=# \dt+ applied_ai_staging.tim_expansion_crm_btntiid
List of relations
Schema | Name | Type | Owner | Storage | Size | Description
--------------------+---------------------------+-------+-----------------+-------------+--------+-------------
applied_ai_staging | tim_expansion_crm_btntiid | table | tim_aai_da_user | append only | 115 GB |
(1 row)

SELECT attname, attnum, attisdropped FROM pg_attribute
WHERE attrelid = 'applied_ai_staging.tim_expansion_crm_btntiid'::regclass ORDER BY attnum;

Source & Dest have same output below :
attname | attnum | attisdropped
------------------------------------------------+--------+--------------
gp_foreign_server | -8 | f
gp_segment_id | -7 | f
tableoid | -6 | f
cmax | -5 | f
xmax | -4 | f
cmin | -3 | f
xmin | -2 | f
ctid | -1 | f
numero_telefono_main_btn | 1 | f
ti_id_intestatario_new_btn | 2 | f
num_sport_btn | 3 | f
num_active_sport_btn | 4 | f
num_cancelled_sport_btn | 5 | f
num_suspended_sport_btn | 6 | f
is_sport_btn | 7 | f
is_modem_btn | 8 | f
num_smartphone_btn | 9 | f
num_active_smartphone_btn | 10 | f
num_cancelled_smartphone_btn | 11 | f
num_suspended_smartphone_btn | 12 | f
is_smart_phone_btn | 13 | f
is_decoder_btn | 14 | f
num_tv_btn | 15 | f
num_active_tv_btn | 16 | f
num_cancelled_tv_btn | 17 | f
num_suspended_tv_btn | 18 | f
is_tv_btn | 19 | f
num_tablet_btn | 20 | f
num_active_tablet_btn | 21 | f
num_cancelled_tablet_btn | 22 | f
num_suspended_tablet_btn | 23 | f
is_tablet_btn | 24 | f
num_cordless_btn | 25 | f
num_active_cordless_btn | 26 | f
num_cancelled_cordless_btn | 27 | f
num_suspended_cordless_btn | 28 | f
is_cordless_btn | 29 | f
num_corded_btn | 30 | f
num_active_corded_btn | 31 | f
num_cancelled_corded_btn | 32 | f
num_suspended_corded_btn | 33 | f
is_corded_btn | 34 | f
num_tfwa_btn | 35 | f
num_active_tfwa_btn | 36 | f
num_cancelled_tfwa_btn | 37 | f
num_suspended_tfwa_btn | 38 | f
is_tfwa_btn | 39 | f
num_t_fttcab_ftth_btn | 40 | f
num_active_t_fttcab_ftth_btn | 41 | f
num_cancelled_t_fttcab_ftth_btn | 42 | f
num_suspended_t_fttcab_ftth_btn | 43 | f
is_t_fttcab_ftth_btn | 44 | f
num_t_fibra_btn | 45 | f
num_active_t_fibra_btn | 46 | f
num_cancelled_t_fibra_btn | 47 | f
num_suspended_t_fibra_btn | 48 | f
is_t_fibra_btn | 49 | f
is_disney_btn | 50 | f
is_netflix_btn | 51 | f
is_o_timunica_btn | 52 | f
is_carte_servizi_valore_btn | 53 | f
num_mobile_btn | 54 | f
num_active_mobile_btn | 55 | f
num_cancelled_mobile_btn | 56 | f
num_suspended_mobile_btn | 57 | f
is_mobile_btn | 58 | f
num_main_mobile_btn | 59 | f
num_active_main_mobile_btn | 60 | f
num_cancelled_main_mobile_btn | 61 | f
num_suspended_main_mobile_btn | 62 | f
is_main_mobile_btn | 63 | f
num_fixed_line_btn | 64 | f
num_active_fixed_line_btn | 65 | f
num_cancelled_fixed_line_btn | 66 | f
num_suspended_fixed_line_btn | 67 | f
is_fixed_line_btn | 68 | f
num_main_fixed_btn | 69 | f
num_active_main_fixed_line_btn | 70 | f
num_cancelled_main_fixed_line_btn | 71 | f
num_suspended_main_fixed_line_btn | 72 | f
is_main_fixed_line_btn | 73 | f
is_broadband_btn | 74 | f
is_ultrabroadband_btn | 75 | f
is_ultrabroadbandh_btn | 76 | f
is_active_broadband_btn | 77 | f
is_active_ultrabroadband_btn | 78 | f
is_active_ultrabroadbandh_btn | 79 | f
churn_percentile_btn | 80 | f
churn_probability_btn | 81 | f
customer_tenure_btn | 82 | f
customer_tenure_bin_btn | 83 | f
num_products_btn | 84 | f
prodotto_subtype_btn | 85 | f
segmento_btn | 86 | f
metodo_pagamento_btn | 87 | f
tipo_pagamento_btn | 88 | f
codice_fiscale_btn | 89 | f
sesso_btn | 90 | f
sottosegmento_btn | 91 | f
self_payer_btn | 92 | f
self_user_btn | 93 | f
count_user_btn | 94 | f
cap_btn | 95 | f
indirizzo_btn | 96 | f
distinct_relaz_btn | 97 | f
is_opzione_voce_btn | 98 | f
is_tim_vision_btn | 99 | f
is_infinity_btn | 100 | f
is_hbo_btn | 101 | f
domiciliazioni_btn | 102 | f
is_adsl_btn | 103 | f
min_data_prima_attivazione_btn | 104 | f
ambito_btn | 105 | f
num_calcio_btn | 106 | f
num_active_calcio_btn | 107 | f
num_cancelled_calcio_btn | 108 | f
num_suspended_calcio_btn | 109 | f
is_calcio_btn | 110 | f
offer_name_no_filter_btn | 111 | f
offer_name_servizio_btn | 112 | f
offer_name_elemento_btn | 113 | f
offer_name_servizio_attivo_btn | 114 | f
offer_name_elemento_attivo_btn | 115 | f
offer_name_servizio_cancelled_btn | 116 | f
offer_name_elemento_cancelled_btn | 117 | f
offer_name_servizio_suspended_btn | 118 | f
offer_name_elemento_suspended_btn | 119 | f
prodotto_subtype_no_filter_btn | 120 | f
prodotto_subtype_servizio_btn | 121 | f
prodotto_subtype_elemento_btn | 122 | f
prodotto_subtype_servizio_attivo_btn | 123 | f
prodotto_subtype_elemento_attivo_btn | 124 | f
prodotto_subtype_servizio_cancelled_btn | 125 | f
prodotto_subtype_elemento_cancelled_btn | 126 | f
prodotto_subtype_servizio_suspended_btn | 127 | f
prodotto_subtype_elemento_suspended_btn | 128 | f
prodotto_class_no_filter_btn | 129 | f
prodotto_subtype_fibra_no_filter_btn | 130 | f
prodotto_subtype_fibra_servizio_btn | 131 | f
prodotto_subtype_fibra_elemento_btn | 132 | f
prodotto_subtype_fibra_servizio_attivo_btn | 133 | f
prodotto_subtype_fibra_elemento_attivo_btn | 134 | f
prodotto_subtype_fibra_servizio_cancelled_btn | 135 | f
prodotto_subtype_fibra_elemento_cancelled_btn | 136 | f
prodotto_subtype_fibra_servizio_suspended_btn | 137 | f
prodotto_subtype_fibra_elemento_suspended_btn | 138 | f
is_high_revenue_product_btn | 139 | f
is_high_revenue_product_all_btn | 140 | f
is_moderate_revenue_product_btn | 141 | f
is_moderate_revenue_product_all_btn | 142 | f
is_low_revenue_product_btn | 143 | f
is_low_revenue_product_all_btn | 144 | f
is_high_revenue_services_btn | 145 | f
is_high_revenue_services_all_btn | 146 | f
is_moderate_revenue_services_btn | 147 | f
is_moderate_revenue_services_all_btn | 148 | f
is_low_revenue_services_btn | 149 | f
is_low_revenue_services_all_btn | 150 | f
is_old_customer_btn | 151 | f
ti_id_intestatario_new_tiid | 152 | f
numero_telefono_main_tiid | 153 | f
num_sport_tiid | 154 | f
num_active_sport_tiid | 155 | f
num_cancelled_sport_tiid | 156 | f
num_suspended_sport_tiid | 157 | f
is_sport_tiid | 158 | f
is_modem_tiid | 159 | f
num_smartphone_tiid | 160 | f
num_active_smartphone_tiid | 161 | f
num_cancelled_smartphone_tiid | 162 | f
num_suspended_smartphone_tiid | 163 | f
is_smart_phone_tiid | 164 | f
is_decoder_tiid | 165 | f
num_tv_tiid | 166 | f
num_active_tv_tiid | 167 | f
num_cancelled_tv_tiid | 168 | f
num_suspended_tv_tiid | 169 | f
is_tv_tiid | 170 | f
num_tablet_tiid | 171 | f
num_active_tablet_tiid | 172 | f
num_cancelled_tablet_tiid | 173 | f
num_suspended_tablet_tiid | 174 | f
is_tablet_tiid | 175 | f
num_cordless_tiid | 176 | f
num_active_cordless_tiid | 177 | f
num_cancelled_cordless_tiid | 178 | f
num_suspended_cordless_tiid | 179 | f
is_cordless_tiid | 180 | f
num_corded_tiid | 181 | f
num_active_corded_tiid | 182 | f
num_cancelled_corded_tiid | 183 | f
num_suspended_corded_tiid | 184 | f
is_corded_tiid | 185 | f
num_tfwa_tiid | 186 | f
num_active_tfwa_tiid | 187 | f
num_cancelled_tfwa_tiid | 188 | f
num_suspended_tfwa_tiid | 189 | f
is_tfwa_tiid | 190 | f
num_t_fttcab_ftth_tiid | 191 | f
num_active_t_fttcab_ftth_tiid | 192 | f
num_cancelled_t_fttcab_ftth_tiid | 193 | f
num_suspended_t_fttcab_ftth_tiid | 194 | f
is_t_fttcab_ftth_tiid | 195 | f
num_t_fibra_tiid | 196 | f
num_active_t_fibra_tiid | 197 | f
num_cancelled_t_fibra_tiid | 198 | f
num_suspended_t_fibra_tiid | 199 | f
is_t_fibra_tiid | 200 | f
is_disney_tiid | 201 | f
is_netflix_tiid | 202 | f
is_o_timunica_tiid | 203 | f
is_carte_servizi_valore_tiid | 204 | f
num_mobile_tiid | 205 | f
num_active_mobile_tiid | 206 | f
num_cancelled_mobile_tiid | 207 | f
num_suspended_mobile_tiid | 208 | f
is_mobile_tiid | 209 | f
num_main_mobile_tiid | 210 | f
num_active_main_mobile_tiid | 211 | f
num_cancelled_main_mobile_tiid | 212 | f
num_suspended_main_mobile_tiid | 213 | f
is_main_mobile_tiid | 214 | f
num_fixed_line_tiid | 215 | f
num_active_fixed_line_tiid | 216 | f
num_cancelled_fixed_line_tiid | 217 | f
num_suspended_fixed_line_tiid | 218 | f
is_fixed_line_tiid | 219 | f
num_main_fixed_tiid | 220 | f
num_active_main_fixed_line_tiid | 221 | f
num_cancelled_main_fixed_line_tiid | 222 | f
num_suspended_main_fixed_line_tiid | 223 | f
is_main_fixed_line_tiid | 224 | f
is_broadband_tiid | 225 | f
is_ultrabroadband_tiid | 226 | f
is_ultrabroadbandh_tiid | 227 | f
is_active_broadband_tiid | 228 | f
is_active_ultrabroadband_tiid | 229 | f
is_active_ultrabroadbandh_tiid | 230 | f
churn_percentile_tiid | 231 | f
churn_probability_tiid | 232 | f
customer_tenure_tiid | 233 | f
customer_tenure_bin_tiid | 234 | f
num_products_tiid | 235 | f
prodotto_subtype_tiid | 236 | f
segmento_tiid | 237 | f
metodo_pagamento_tiid | 238 | f
tipo_pagamento_tiid | 239 | f
codice_fiscale_tiid | 240 | f
sesso_tiid | 241 | f
sottosegmento_tiid | 242 | f
self_payer_tiid | 243 | f
self_user_tiid | 244 | f
count_user_tiid | 245 | f
cap_tiid | 246 | f
indirizzo_tiid | 247 | f
distinct_relaz_tiid | 248 | f
is_opzione_voce_tiid | 249 | f
is_tim_vision_tiid | 250 | f
is_infinity_tiid | 251 | f
is_hbo_tiid | 252 | f
domiciliazioni_tiid | 253 | f
is_adsl_tiid | 254 | f
min_data_prima_attivazione_tiid | 255 | f
ambito_tiid | 256 | f
num_calcio_tiid | 257 | f
num_active_calcio_tiid | 258 | f
num_cancelled_calcio_tiid | 259 | f
num_suspended_calcio_tiid | 260 | f
is_calcio_tiid | 261 | f
offer_name_no_filter_tiid | 262 | f
offer_name_servizio_tiid | 263 | f
offer_name_elemento_tiid | 264 | f
offer_name_servizio_attivo_tiid | 265 | f
offer_name_elemento_attivo_tiid | 266 | f
offer_name_servizio_cancelled_tiid | 267 | f
offer_name_elemento_cancelled_tiid | 268 | f
offer_name_servizio_suspended_tiid | 269 | f
offer_name_elemento_suspended_tiid | 270 | f
prodotto_subtype_no_filter_tiid | 271 | f
prodotto_subtype_servizio_tiid | 272 | f
prodotto_subtype_elemento_tiid | 273 | f
prodotto_subtype_servizio_attivo_tiid | 274 | f
prodotto_subtype_elemento_attivo_tiid | 275 | f
prodotto_subtype_servizio_cancelled_tiid | 276 | f
prodotto_subtype_elemento_cancelled_tiid | 277 | f
prodotto_subtype_servizio_suspended_tiid | 278 | f
prodotto_subtype_elemento_suspended_tiid | 279 | f
prodotto_class_no_filter_tiid | 280 | f
prodotto_subtype_fibra_no_filter_tiid | 281 | f
prodotto_subtype_fibra_servizio_tiid | 282 | f
prodotto_subtype_fibra_elemento_tiid | 283 | f
prodotto_subtype_fibra_servizio_attivo_tiid | 284 | f
prodotto_subtype_fibra_elemento_attivo_tiid | 285 | f
prodotto_subtype_fibra_servizio_cancelled_tiid | 286 | f
prodotto_subtype_fibra_elemento_cancelled_tiid | 287 | f
prodotto_subtype_fibra_servizio_suspended_tiid | 288 | f
prodotto_subtype_fibra_elemento_suspended_tiid | 289 | f
is_high_revenue_product_tiid | 290 | f
is_high_revenue_product_all_tiid | 291 | f
is_moderate_revenue_product_tiid | 292 | f
is_moderate_revenue_product_all_tiid | 293 | f
is_low_revenue_product_tiid | 294 | f
is_low_revenue_product_all_tiid | 295 | f
is_high_revenue_services_tiid | 296 | f
is_high_revenue_services_all_tiid | 297 | f
is_moderate_revenue_services_tiid | 298 | f
is_moderate_revenue_services_all_tiid | 299 | f
is_low_revenue_services_tiid | 300 | f
is_low_revenue_services_all_tiid | 301 | f
is_old_customer_tiid | 302 | f
start_date_tiid | 303 | f
end_date_tiid | 304 | f
start_date_btn | 305 | f
end_date_btn | 306 | f
start_date_final | 307 | f
end_date_final | 308 | f
rno | 309 | f
(317 rows)

source gp cbcopy helper log :

20260624:13:46:21 cbcopy_helper:gpadmin:afitim-gp:145443-[DEBUG]:-Successfully established connection to 192.168.14.8:1027
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145438-[DEBUG]:-Connection to 192.168.14.8:1025 is closed.
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145438-[DEBUG]:-Helper stopping
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145438-[ERROR]:-cbcopy_helper exited with error: failed to redirect stream: write tcp 192.168.14.29:55168->192.168.14.8:1025: write: connection reset by peer
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145420-[DEBUG]:-Connection to 192.168.14.8:1025 is closed.
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145420-[DEBUG]:-Helper stopping
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145420-[ERROR]:-cbcopy_helper exited with error: failed to redirect stream: write tcp 192.168.14.29:55162->192.168.14.8:1025: write: connection reset by peer
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145446-[DEBUG]:-Connection to 192.168.14.7:1026 is closed.
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145446-[DEBUG]:-Helper stopping
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145446-[ERROR]:-cbcopy_helper exited with error: failed to redirect stream: write tcp 192.168.14.29:56950->192.168.14.7:1026: write: connection reset by peer
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145426-[DEBUG]:-Connection to 192.168.14.7:1026 is closed.
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145426-[DEBUG]:-Helper stopping
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145426-[ERROR]:-cbcopy_helper exited with error: failed to redirect stream: write tcp 192.168.14.29:56924->192.168.14.7:1026: write: connection reset by peer
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145432-[DEBUG]:-Connection to 192.168.14.7:1026 is closed.
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145432-[DEBUG]:-Helper stopping
20260624:13:56:18 cbcopy_helper:gpadmin:afitim-gp:145432-[ERROR]:-cbcopy_helper exited with error: failed to redirect stream: write tcp 192.168.14.29:56962->192.168.14.7:1026: write: connection reset by peer

CB COORD cbcbopy helper logs:

20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195217-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 0 --resolve TIM-CB-Seg1]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195217-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195225-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 1 --resolve TIM-CB-Seg1]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195225-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195233-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 2 --resolve TIM-CB-Seg1]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195233-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195240-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 3 --resolve TIM-CB-Seg1]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195240-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195248-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 4 --resolve TIM-CB-Seg2]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195248-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195255-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 5 --resolve TIM-CB-Seg2]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195255-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195263-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 6 --resolve TIM-CB-Seg2]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195263-[DEBUG]:-Helper stopping
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195270-[DEBUG]:-Helper started with args [cbcopy_helper --seg-id 7 --resolve TIM-CB-Seg2]
20260624:13:46:16 cbcopy_helper:gpadmin:TIM-CB-Coord:195270-[DEBUG]:-Helper stopping

Segment 1 cbcbopy log :

20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-Helper started with args [cbcopy_helper --compress-type zstd --listen --seg-id 1 --cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8 --c
lient-numbers 4,4,4,4,3,3,3,3 --data-port-range 1024-65535 --direction receive]
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-Creating ConcurrentServer...
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[DEBUG]:-Listening on addr [::]:1024, seg-id 2 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-Listening on addr [::]:1025, seg-id 1 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547988-[DEBUG]:-Listening on addr [::]:1026, seg-id 0 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-Helper started with args [cbcopy_helper --compress-type zstd --listen --seg-id 3 --cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8 --c
lient-numbers 4,4,4,4,3,3,3,3 --data-port-range 1024-65535 --direction receive]
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-Creating ConcurrentServer...
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-Listening on addr [::]:1027, seg-id 3 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:44310
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:42602
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547988-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:56924
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547988-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:56938
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:42614
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:42624
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547988-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:56950
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:24898
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:24900
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:44326
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547988-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:56962
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:24902
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:24906
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:44340
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547990-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:42636
20260624:13:46:19 cbcopy_helper:gpadmin:TIM-CB-Seg1:547987-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:44342
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg1:547989-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout

Segment 2 cbcbopy helper log:

20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[DEBUG]:-Helper started with args [cbcopy_helper --compress-type zstd --listen --seg-id 4 --cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8 --c
lient-numbers 4,4,4,4,3,3,3,3 --data-port-range 1024-65535 --direction receive]
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[DEBUG]:-Creating ConcurrentServer...
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[DEBUG]:-Listening on addr [::]:1024, seg-id 4 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593493-[DEBUG]:-Helper started with args [cbcopy_helper --compress-type zstd --listen --seg-id 6 --cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8 --c
lient-numbers 4,4,4,4,3,3,3,3 --data-port-range 1024-65535 --direction receive]
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593493-[DEBUG]:-Creating ConcurrentServer...
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593493-[DEBUG]:-Listening on addr [::]:1025, seg-id 6 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593494-[DEBUG]:-Helper started with args [cbcopy_helper --compress-type zstd --listen --seg-id 7 --cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8 --c
lient-numbers 4,4,4,4,3,3,3,3 --data-port-range 1024-65535 --direction receive]
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593494-[DEBUG]:-Creating ConcurrentServer...
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593494-[DEBUG]:-Listening on addr [::]:1026, seg-id 7 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593492-[DEBUG]:-Helper started with args [cbcopy_helper --compress-type zstd --listen --seg-id 5 --cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8 --c
lient-numbers 4,4,4,4,3,3,3,3 --data-port-range 1024-65535 --direction receive]
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593492-[DEBUG]:-Creating ConcurrentServer...
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593492-[DEBUG]:-Listening on addr [::]:1027, seg-id 5 cmd-id 30099330-a9f8-4e9f-b6ad-f01a3a35f2a8
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:57838
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:57848
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593493-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:55156
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593492-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:36688
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593494-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:59864
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593494-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:59872
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593494-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:59888
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:57850
20260624:13:46:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593493-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:55162
20260624:13:46:20 cbcopy_helper:gpadmin:TIM-CB-Seg2:593493-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:55168
20260624:13:46:20 cbcopy_helper:gpadmin:TIM-CB-Seg2:593492-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:36694
20260624:13:46:21 cbcopy_helper:gpadmin:TIM-CB-Seg2:593492-[DEBUG]:-ConcurrentServer accept a connection from 192.168.14.29:36698
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout
20260624:13:56:18 cbcopy_helper:gpadmin:TIM-CB-Seg2:593491-[ERROR]:-Listener.Accept() failed: accept tcp [::]:1024: i/o timeout

[talmacschen-arch]

Thanks — the logs make this clear, and your pg_attribute dumps rule out a schema mismatch (source and dest columns are identical), so this is not a column-layout problem. It's the interrupted-transfer case: segment-to-segment network connectivity.

Reading the three logs together:

• Dest CB segment helper: Listener.Accept() failed: accept tcp [::]:1024: i/o timeout. cbcopy's helper sets a 600-second listener deadline (helper/server.go). Your timestamps are 13:46:18 → 13:56:18 — exactly 10 minutes. So a destination segment listener waited the full 10 minutes for an inbound connection from a source segment that never arrived, then timed out.
• Source GP helper: failed to redirect stream: write tcp 192.168.14.29:...->192.168.14.8:1025: write: connection reset by peer — a source segment was streaming data to a dest segment when the connection was torn down.
• Dest QE: missing data for column ... (SQLSTATE 22P04) — it read a truncated row from the broken pipe.

In push mode each destination segment listener expects a fixed number of inbound connections from the source segments (your client-numbers 4,4,4,4,3,3,3,3). At 13:46 a subset connected and data flowed (hence the ~334 MB). But at least one source→dest segment connection never established (the listener stuck on port 1024). After 10 minutes that listener timed out, the helper process group tore down, the connections that were streaming got reset, and the COPY aborted with 22P04. A single missing segment-to-segment link brings down the whole table — which is why large tables (transferred segment-to-segment) fail while small ones (coordinator-to-coordinator) succeed.

This is an environment/firewall issue, not a cbcopy bug. To fix:

1. Verify full-mesh reachability: from every source segment host to every destination segment host, on the data ports. Note these are the internal segment addresses (192.168.14.7 / 192.168.14.8, from gp_segment_configuration), not just the --dest-host 10.195.157.19. Test each pair, e.g. nc -vz <dest-seg-host> <port>.
2. The data port range defaults to 1024–65535. Narrow it with --data-port-range (e.g. --data-port-range 22000-22100) and open exactly that range in the firewall on all segment hosts.
3. If the destination segments cannot accept inbound connections (NAT / restricted network), use --connection-mode=pull.
4. To pin down the culprit, cross-reference the IP/port pairs in the two helper logs to find which source segment failed to reach which dest segment on port 1024 — that usually points straight at an iptables/firewalld/security-group rule on one host.

--copy-jobs 2 and --compression are unrelated to this failure.

BTW : Are the segment nodes on the 192.168.14.X subnet? Defferent with the master nodes's ?
cbcopy --source-host 10.195.157.29 --source-port 55432 --source-user gpadmin --dest-host 10.195.157.19 --dest-user gpadmin --dest-port 55432