Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,7 @@ App 只负责提供 `deviceID`、本地配置文件路径、上传持久化目
| `credentialBase64` | 上传凭证 | 来自配置文件或安全下发渠道 |
| `authRefreshBefore` | 认证缓存提前刷新时间 | `60s` |
| `requestTimeout` | 单次请求超时 | `10s` |
| `persistRootURL` | 上传快照和 staging 根目录 | App 私有 `Application Support` 子目录 |
| `persistRootURL` | 上传快照根目录 | App 私有 `Application Support` 子目录 |
| `endpointsURL` | 运行期公共端点文件 | App 私有 `Application Support/Archebase/archebase-endpoints.json` |
| `retryPolicy` | 请求重试策略 | `.recommended` |
| `execution` | 上传执行策略 | `.recommended` |
Expand All @@ -442,7 +442,7 @@ App 只负责提供 `deviceID`、本地配置文件路径、上传持久化目
| `reconcileRemotePartsOnResume` | `true` | 恢复时校验已上传分片状态 |
| `cleanupOnTerminalFailure` | `true` | 终态失败时允许 SDK 清理不可恢复状态 |
| `credentialRefreshSkew` | `30s` | 上传凭证过期前提前刷新 |
| `persistence` | `.recommended` | 本地快照和 staging 策略 |
| `persistence` | `.recommended` | 本地快照和直读文件策略 |

默认本地持久化策略:

Expand All @@ -452,7 +452,7 @@ App 只负责提供 `deviceID`、本地配置文件路径、上传持久化目
| `keepCompletedSnapshot` | `false` | 完成后不保留快照 |
| `completedSnapshotTTL` | `0s` | 完成快照保留时间 |
| `terminalSnapshotTTL` | `3600s` | 失败终态快照保留时间 |
| `copyExternalFileIntoManagedStaging` | `true` | 上传前复制外部文件到 SDK staging 目录,提升恢复稳定性 |
| `copyExternalFileIntoManagedStaging` | `false` | 兼容字段;当前版本不再生成 staging 数据副本,上传直接读取源文件 |

## 10. 发起上传

Expand Down Expand Up @@ -609,7 +609,7 @@ let result = try await client.resumeUpload(logicalUploadID: pending.logicalUploa
print(result.objectKey)
```

恢复时 SDK 会校验本地 staging 文件是否仍然存在、大小是否一致、文件指纹是否匹配。如果文件丢失或被修改,会抛出 `DataGatewayClientError.resumeNotPossible`。
恢复时 SDK 会校验原始文件是否仍然存在、大小是否一致、文件指纹是否匹配。如果文件丢失、移动或被修改,会抛出 `DataGatewayClientError.resumeNotPossible`。

恢复逻辑对 App 的承诺:

Expand All @@ -624,7 +624,7 @@ print(result.objectKey)
|---|---|
| `logicalUploadID` | 恢复和取消使用的稳定上传标识。 |
| `uploadID` | 最近一次上传会话标识。 |
| `fileURL` | SDK 管理的本地文件 URL,可能是 staging 文件。 |
| `fileURL` | 本地源文件 URL。 |
| `fileSize` | 文件大小,单位 bytes。 |
| `phase` | 本地记录的上传阶段。 |
| `restartCount` | 已自动重建上传会话的次数。 |
Expand Down Expand Up @@ -842,11 +842,11 @@ func makeClientOrRequireInitialization() async throws -> DataGatewayClient? {

### 17.3 文件选择与安全作用域

如果文件来自 `UIDocumentPickerViewController` 或其他安全作用域 URL,SDK 会尽量访问安全作用域资源并把文件复制到 SDK staging 目录。推荐保持 `copyExternalFileIntoManagedStaging = true`,这样用户移动或撤销原始文件访问权限后,已进入 staging 的上传仍更容易恢复
如果文件来自 `UIDocumentPickerViewController` 或其他安全作用域 URL,SDK 会尽量访问安全作用域资源并记录 bookmark,但不会把文件复制到 SDK staging 目录。上传和恢复都直接依赖原始文件 URL;因此 App 需要确保上传期间和恢复前源文件仍然存在且可访问,不要在任务完成前移动、删除或改写该文件

### 17.4 文件大小与内存

当前版本会在本地读取文件并按分片上传。请根据目标设备内存和网络条件控制单个文件大小。对于非常大的文件、后台长传或高并发上传需求,建议在上线前与 Archebase 支持团队确认版本能力和压测结果。
当前版本直接读取源文件上传:单对象上传使用文件 body,multipart 上传按分片构造区间流,不再把完整文件 `readAll` 到内存。SDK 仍会读取首 1 MiB 计算恢复指纹,并按分片小块读取计算分片 MD5。对于非常大的文件、后台长传或高并发上传需求,建议在上线前与 Archebase 支持团队确认版本能力和压测结果。

## 18. 完整示例

Expand Down Expand Up @@ -1213,7 +1213,7 @@ public enum DataGatewayClientError: Error, Sendable, Equatable {
| 上传立即失败 `zeroByteFile` | 用户选择的文件是否为空。 |
| 上传立即失败 `invalidLocalFile` | 文件是否仍存在,App 是否有访问权限。 |
| 上传失败 `rawTagConflict` | `UploadRequest.rawTags` 是否覆盖了设备配置 tags 的同名 key。 |
| 恢复失败 `resumeNotPossible` | staging 文件是否被删除,原始文件是否被修改,用户是否清理过 App 数据。 |
| 恢复失败 `resumeNotPossible` | 原始文件是否被删除、移动或修改,用户是否撤销文件访问权限或清理过 App 数据。 |
| 频繁 `authenticationFailed` | 设备是否需要重新初始化,凭证是否已被管理员轮换或撤销。 |
| 频繁 `ossFailed` 或 `retryExhausted` | 网络质量、代理、防火墙、系统时间和对象存储可用性。 |
| `uploadRestartExceeded` | 让用户重新上传,并保留 `logicalUploadID` 与 SDK 日志联系支持。 |
107 changes: 93 additions & 14 deletions Sources/DGWOss/OssMultipartClient.swift
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,81 @@ package struct OssPutObjectOutput: Sendable, Equatable {
}
}

package struct OssUploadBody: @unchecked Sendable {
package enum Kind: Sendable, Equatable {
case data
case file
case stream
}

fileprivate enum Storage {
case data(Data)
case file(URL, sizeBytes: Int64)
case stream(@Sendable () throws -> InputStream, sizeBytes: Int64)
}

fileprivate let storage: Storage
fileprivate let contentMD5Base64: String?

package var sizeBytes: Int64 {
switch self.storage {
case .data(let data):
return Int64(data.count)
case .file(_, let sizeBytes), .stream(_, let sizeBytes):
return sizeBytes
}
}

package var kind: Kind {
switch self.storage {
case .data:
return .data
case .file:
return .file
case .stream:
return .stream
}
}

package static func data(_ data: Data) -> OssUploadBody {
OssUploadBody(storage: .data(data), contentMD5Base64: nil)
}

package static func file(
_ fileURL: URL,
sizeBytes: Int64,
contentMD5Base64: String? = nil
) -> OssUploadBody {
OssUploadBody(storage: .file(fileURL, sizeBytes: sizeBytes), contentMD5Base64: contentMD5Base64)
}

package static func stream(
sizeBytes: Int64,
contentMD5Base64: String? = nil,
makeStream: @escaping @Sendable () throws -> InputStream
) -> OssUploadBody {
OssUploadBody(storage: .stream(makeStream, sizeBytes: sizeBytes), contentMD5Base64: contentMD5Base64)
}

package func byteStream() throws -> ByteStream {
switch self.storage {
case .data(let data):
return .data(data)
case .file(let fileURL, _):
return .file(fileURL)
case .stream(let makeStream, _):
return try .stream(makeStream())
}
}

fileprivate func addIntegrityHeaders(to request: inout some RequestModel) {
guard let contentMD5Base64 else {
return
}
request.addHeader("Content-MD5", contentMD5Base64)
}
}

package struct OssCompleteMultipartUploadOutput: Sendable, Equatable {
package let etag: String?

Expand Down Expand Up @@ -724,12 +799,12 @@ package protocol OssMultipartClientProtocol: Sendable {
objectKey: String,
multipartUploadID: String,
partNumber: Int,
body: Data
body: OssUploadBody
) async throws -> UploadedPartDescriptor

func putObject(
objectKey: String,
body: Data
body: OssUploadBody
) async throws -> UploadedPartDescriptor

func completeMultipartUpload(
Expand Down Expand Up @@ -784,24 +859,26 @@ package struct OssMultipartClient: OssMultipartClientProtocol {
objectKey: String,
multipartUploadID: String,
partNumber: Int,
body: Data
body: OssUploadBody
) async throws -> UploadedPartDescriptor {
do {
let request = UploadPartRequest(
var request = UploadPartRequest(
bucket: self.configuration.bucket,
key: objectKey,
partNumber: partNumber,
uploadId: multipartUploadID,
body: .data(body)
body: try body.byteStream()
)
request.addHeader("Content-Length", body.sizeBytes.description)
body.addIntegrityHeaders(to: &request)
let result = try await self.sdkClient.uploadPart(request)
guard let etag = result.etag?.nilIfBlank else {
throw OssOperationError.invalidResponse("UploadPart response missing ETag")
}
return UploadedPartDescriptor(
partNumber: partNumber,
etag: etag,
size: Int64(body.count),
size: body.sizeBytes,
lastModified: nil,
hashCRC64: nil
)
Expand All @@ -812,22 +889,24 @@ package struct OssMultipartClient: OssMultipartClientProtocol {

package func putObject(
objectKey: String,
body: Data
body: OssUploadBody
) async throws -> UploadedPartDescriptor {
do {
let request = PutObjectRequest(
var request = PutObjectRequest(
bucket: self.configuration.bucket,
key: objectKey,
body: .data(body)
body: try body.byteStream()
)
request.addHeader("Content-Length", body.sizeBytes.description)
body.addIntegrityHeaders(to: &request)
let result = try await self.sdkClient.putObject(request)
guard let etag = result.etag?.nilIfBlank else {
throw OssOperationError.invalidResponse("PutObject response missing ETag")
}
return UploadedPartDescriptor(
partNumber: 1,
etag: etag,
size: Int64(body.count),
size: body.sizeBytes,
lastModified: nil,
hashCRC64: nil
)
Expand Down Expand Up @@ -1066,7 +1145,7 @@ package actor OssUploadSession {
package func uploadPart(
multipartUploadID: String,
partNumber: Int,
body: Data
body: OssUploadBody
) async throws -> UploadedPartDescriptor {
try await self.executeDataPlaneOperation {
try await self.performUploadPart(
Expand All @@ -1077,7 +1156,7 @@ package actor OssUploadSession {
}
}

package func putObject(body: Data) async throws -> UploadedPartDescriptor {
package func putObject(body: OssUploadBody) async throws -> UploadedPartDescriptor {
try await self.executeDataPlaneOperation {
try await self.performPutObject(body: body)
}
Expand Down Expand Up @@ -1150,7 +1229,7 @@ package actor OssUploadSession {
private func performUploadPart(
multipartUploadID: String,
partNumber: Int,
body: Data
body: OssUploadBody
) async throws -> UploadedPartDescriptor {
try await self.client.uploadPart(
objectKey: self.context.objectKey,
Expand All @@ -1160,7 +1239,7 @@ package actor OssUploadSession {
)
}

private func performPutObject(body: Data) async throws -> UploadedPartDescriptor {
private func performPutObject(body: OssUploadBody) async throws -> UploadedPartDescriptor {
try await self.client.putObject(
objectKey: self.context.objectKey,
body: body
Expand Down
Loading