使用 Node JS 递归获取 DynamoDB 查询中的所有项目

2022-01-15 00:00:00 amazon-web-services 递归 node.js amazon-dynamodb javascript

这可能更像是一个 JS/Async 问题而不是 DynamoDB 特定问题 -

This is probably more of an JS/Async question than a DynamoDB specific question -

我想在 Amazon 的 DynamoDB 中使用哈希键获取表中的所有项目.该表中还包含 Range 键.

I want to fetch all the items in a table with a hash key in Amazon's DynamoDB. The table also has Range key in it.

我正在使用一个 NodeJS 库，它是 AWS DynamoDB REST API 的包装器.-节点-DynamoDB

I am using a NodeJS library which is a wrapper around AWS DynamoDB REST API. - Node-DynamoDB

DynamoDB 每次查询仅返回 1 MB 的结果.为了获取结果提醒，它包括 lastEvaluatedKey .我们可以将其包含在另一个查询中，以获取另外 1 MB 的结果等等...

DynamoDB only returns 1 MB worth of results with each query. To fetch reminder of results, it includes lastEvaluatedKey . We can include this in another query to fetch another 1 MB worth of results and so on...

我在编写递归异步函数时遇到了困难，该函数应该按顺序访问服务，直到我可以取回所有结果.(对于我的用例，表永远不会超过 10 MB，不会出现失控查询)

I am facing difficulty in writing a recursive async function which should hit the service sequentially till i can get all the results back. (table will never have more than 10 MB for my use case, no chance of a runaway query)

一些伪代码用于说明:

ddb.query('products', primarykey, {}, function(err,result){
    //check err
    if(result && result.lastEvaluatedKey){
        //run the query again
        var tempSet = result.items;
        //temporarily store result.items so we can continue and fetch remaining items.
    }
    else{
        var finalSet = result.items;
        //figure out how to merge with items that were fetched before.
    }
});

推荐答案

var getAll = function(primarykey, cb) {
    var finalSet = [],
        nextBatch = function(lek) {
            ddb.query('products', primarykey, {
                exclusiveStartKey: lek
            }, function(err, result) {
                if (err) return cb(err);

                if (result.items.length)
                    finalSet.push.apply(finalSet, result.items);

                if (result.lastEvaluatedKey)
                    nextBatch(result.lastEvaluatedKey);
                else
                    cb(null, finalSet);
            });
        };

    nextBatch();
};


getAll(primarykey, function(err, all) {
    console.log(err, all);
});

相关文章