当文件夹中的文件数为1000时,Dropbox Files.Download不会启动
我在Dropbox论坛上交叉发布了我最初的问题。我认为在这里为快速下载框的用户提供这一功能也是一件好事。
我无法通过wiftyDropbox将整个文件夹下载到本地设备。 我正在执行ListFold和ListFolderContinue(我观察到每个响应将其分块为大约500个文件),并将其附加到本地数组。
之后,我将此数组传递给文件。下载。但是,我发现如果我的文件夹是>;1000个文件(txt文件大小约为0.5-1KB),下载过程将不会开始。
static func downloadMissingFiles(client: DropboxClient, callingProcess: String) {
let fileManager = FileManager.default
let localBaseURL = fileManager.urls(for: .cachesDirectory, in: .userDomainMask)[0].appendingPathComponent("Cloud/Dropbox", isDirectory: true)
// Data will be in the form of
// key : "/workouts/workout list 1/peye.mrc"
// value : "//workouts/workout list 1/peye.mrc=_-_=015ca880b135d01000000020cb26de0"
for dbFiles in Array(dbFileNameRevDict) {
let dbFilePathLower = dbFiles.key
let dbFileNameRev = dbFiles.value
let fullURL = localBaseURL.appendingPathComponent(dbFileNameRev)
if fileManager.fileExists(atPath: fullURL.path) {
print(" -> FILE EXISTS dbFileNameRev:(dbFileNameRev)")
localFileList.append(dbFileNameRev)
} else {
let destination : (URL, HTTPURLResponse) -> URL = { temporaryURL, response in
return fullURL
}
client.files.download(path:dbFilePathLower, overwrite: true, destination: destination)
.response { response, error in
if let (_, url) = response {
print("====> DOWNLOADED:(url.lastPathComponent)")
} else if let error = error {
print(error)
}
/// This gives a progress of every single file on it's own. Hence, useless
// .progress { progressData in
// print(progressData)
// }
}
}
}
}
我已经尝试了各种方法来下载这些文件,我也尝试了一个串行队列来逐个迭代文件数组,但都不起作用。
这就是我在查看hasmore属性时处理ListFold和ListFolderContinue的方式。
// https://stackoverflow.com/a/52870045/14414215
if result.hasMore == true {
processDBMore(client: client, cursor: result.cursor)
} else {
// When there is no more files (as indicated by hasMore == false)
// start downloading the files
downloadMissingFiles(client: client, callingProcess: "processDBMore-Finish")
print("PrcessDBMore - dropboxGroup.leave")
dropboxGroup.leave()
}
解决方案
根据Greg(SwiftyDropbox)
每个‘client.files.download’调用通过创建一个文件来下载一个文件 向Dropbox API服务器发出的HTTPS请求。此外,这些呼叫 异步运行,并且在调用完成之前不会阻塞。那 是,调用‘client.files.download’将启动HTTPS请求,但是 将在完成之前返回,并且响应已完全完成 收到了。(一旦请求完成,它只运行提供的块。) 在这种情况下,在您这里展示的代码中,您实际上是 几乎同时启动1000个连接,所以它是 可能会耗尽您的网络连接。您应该更新您的代码 一次只提交其中的一个(或几个)。你提到过你 已尝试使用串行队列,但可能遇到相同的问题, 因为实际请求是异步运行的。
所以我在寻找其他解决方案时,看到了这篇https://stackoverflow.com/a/66227963/14414215,它对我理解信号量是如何工作的,以及实现信号量(除了使用DispatchGroups之外)如何能够正确地控制文件。下载调用有很大帮助。
static func downloadMissingFiles(client: DropboxClient, callingProcess: String) {
let fileManager = FileManager.default
let localBaseURL = fileManager.urls(for: .cachesDirectory, in: .userDomainMask)[0].appendingPathComponent("Cloud/Dropbox", isDirectory: true)
let semaphore = DispatchSemaphore(value: 1) // insert desired concurrent downloads value here.
// Data will be in the form of
// key : "/workouts/workout list 1/peye.mrc"
// value : "//workouts/workout list 1/peye.mrc=_-_=015ca880b135d01000000020cb26de0"
DispatchQueue.global().async { // Wrap the call within an async block
for dbFiles in Array(dbFileNameRevDict) {
semaphore.wait() // Decrement the semaphore counter
let dbFilePathLower = dbFiles.key
let dbFileNameRev = dbFiles.value
let fullURL = localBaseURL.appendingPathComponent(dbFileNameRev)
if fileManager.fileExists(atPath: fullURL.path) {
print(" -> FILE EXISTS dbFileNameRev:(dbFileNameRev)")
localFileList.append(dbFileNameRev)
semaphore.signal() // Increment semaphore counter
} else {
let destination : (URL, HTTPURLResponse) -> URL = { temporaryURL, response in
return fullURL
}
client.files.download(path:dbFilePathLower, overwrite: true, destination: destination)
.response { response, error in
if let (_, url) = response {
print("====> DOWNLOADED:(url.lastPathComponent)")
// we've reached here means we've successfully download the file
// So we can (release)increment semaphore counter
semaphore.signal()
} else if let error = error {
print(error)
}
/// This gives a progress of every single file on it's own. Hence, useless
// .progress { progressData in
// print(progressData)
// }
}
}
}
}
}
相关文章