Skip to content

Commit 8a9e3af

Browse files
branch-4.0: [fix](docker): Fix Hive3 docker OSS configuration to use Jindo fs #61822 (#61926)
Cherry-picked from #61822 Co-authored-by: zgxme <zhenggaoxiong@selectdb.com>
1 parent d428716 commit 8a9e3af

4 files changed

Lines changed: 11 additions & 6 deletions

File tree

docker/thirdparties/docker-compose/hive/hadoop-hive-3x.env.tpl

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,9 @@ HIVE_SITE_CONF_fs_cosn_bucket_region=${COSRegion}
3636
HIVE_SITE_CONF_fs_cosn_impl=org.apache.hadoop.fs.CosFileSystem
3737
HIVE_SITE_CONF_fs_cos_impl=org.apache.hadoop.fs.CosFileSystem
3838
HIVE_SITE_CONF_fs_AbstractFileSystem_cosn_impl=org.apache.hadoop.fs.CosN
39-
HIVE_SITE_CONF_fs_oss_impl=org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem
39+
40+
HIVE_SITE_CONF_fs_oss_impl=com.aliyun.jindodata.oss.JindoOssFileSystem
41+
HIVE_SITE_CONF_fs_AbstractFileSystem_oss_impl=com.aliyun.jindodata.oss.JindoOSS
4042
HIVE_SITE_CONF_fs_oss_accessKeyId=${OSSAk}
4143
HIVE_SITE_CONF_fs_oss_accessKeySecret=${OSSSk}
4244
HIVE_SITE_CONF_fs_oss_endpoint=${OSSEndpoint}

docker/thirdparties/docker-compose/hive/scripts/prepare-hive-data.sh

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -130,8 +130,9 @@ jars=(
130130
aliyun-java-sdk-ecs-4.2.0.jar
131131
aliyun-java-sdk-ram-3.0.0.jar
132132
aliyun-java-sdk-sts-3.0.0.jar
133-
aliyun-sdk-oss-3.4.1.jar
134-
hadoop-aliyun-3.2.1.jar
133+
jindo-core-6.3.4.jar
134+
jindo-core-linux-el7-aarch64-6.3.4.jar
135+
jindo-sdk-6.3.4.jar
135136
aws-java-sdk-bundle-1.11.375.jar
136137
hadoop-huaweicloud-3.1.1-hw-54.5.jar
137138
hadoop-cos-3.1.0-8.3.22.jar

docker/thirdparties/docker-compose/kerberos/entrypoint-hive-master.sh

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,9 @@ curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/aliyun-java-sdk-
4646
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/aliyun-java-sdk-ecs-4.2.0.jar
4747
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/aliyun-java-sdk-ram-3.0.0.jar
4848
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/aliyun-java-sdk-sts-3.0.0.jar
49-
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/aliyun-sdk-oss-3.4.1.jar
50-
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/hadoop-aliyun-3.2.1.jar
49+
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/jindo-core-6.3.4.jar
50+
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/jindo-core-linux-el7-aarch64-6.3.4.jar
51+
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/jindo-sdk-6.3.4.jar
5152
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/aws-java-sdk-bundle-1.11.375.jar
5253
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/hadoop-huaweicloud-3.1.1-hw-54.5.jar
5354
curl -O https://s3BucketName.s3Endpoint/regression/docker/hive3/hadoop-cos-3.1.0-8.3.22.jar

docker/thirdparties/docker-compose/kerberos/hadoop-hive.env.tpl

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,8 @@ HIVE_SITE_CONF_fs_cosn_bucket_region=${COSRegion}
8080
HIVE_SITE_CONF_fs_cosn_impl=org.apache.hadoop.fs.CosFileSystem
8181
HIVE_SITE_CONF_fs_cos_impl=org.apache.hadoop.fs.CosFileSystem
8282
HIVE_SITE_CONF_fs_AbstractFileSystem_cosn_impl=org.apache.hadoop.fs.CosN
83-
HIVE_SITE_CONF_fs_oss_impl=org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem
83+
HIVE_SITE_CONF_fs_oss_impl=com.aliyun.jindodata.oss.JindoOssFileSystem
84+
HIVE_SITE_CONF_fs_AbstractFileSystem_oss_impl=com.aliyun.jindodata.oss.JindoOSS
8485
HIVE_SITE_CONF_fs_oss_accessKeyId=${OSSAk}
8586
HIVE_SITE_CONF_fs_oss_accessKeySecret=${OSSSk}
8687
HIVE_SITE_CONF_fs_oss_endpoint=${OSSEndpoint}

0 commit comments

Comments
 (0)