如何使用 AWS JavaScript SDK (dynamoDB) 处理 UnprocessedItems?

2022-01-15 00:00:00 amazon-web-services lambda node.js amazon-dynamodb javascript

我正在尝试使用 AWS Lambda 函数来处理来自 SendGrid 的事件.据我了解，该事件将是一个包含可变数量 JSON 对象的数组，每个对象代表一个给定事件.我想使用 batchWriteItem 将这些事件写入 DynamoDB 并循环该过程，直到我没有返回任何 UnprocessedItems.但是，我陷入了无限循环.这是我现在的代码:

I am trying to use an AWS Lambda function to handle events from SendGrid. As I understand it, the event will be an array with a variable number of JSON objects, each representing a given event. I want to write these events to DynamoDB using batchWriteItem and loop the process until I'm not returned any UnprocessedItems. However, I'm getting stuck in an infinite loop. Here is my code now:

console.log('Loading function');

var aws = require('aws-sdk');
var dynamo = new aws.DynamoDB();
params = {};

exports.handler = function(sg_event, context) {

    var items = [];
    for(var i = 0; i < sg_event.length; i++) {
        var obj = sg_event[i];
        var request = {
            PutRequest: {
                Item: {
                    email: { S: obj.email },
                    timestamp: { S: obj.timestamp.toString() },
                    sg_message_id: { S: obj.sg_message_id },
                    event: { S: obj.event }
                }
            }
        };
        items.push(request);
    }

    params = {
        RequestItems: {
            sendgrid_response: items
        }
    }

    do {
        dynamo.batchWriteItem( params, function(err, data) {
            if(err)
                context.fail(err);
            else
                params.RequestItems = data.UnprocessedItems;
        });
    } while(!isEmpty(params.RequestItems));
};

function isEmpty(obj) {
    return (Object.keys(obj).length === 0);
}

我认为问题在于尝试在回调函数中设置参数，但我不知道我应该怎么做......我知道我可以在回调中使用 UnprocessedItems 调用另一个 batchWriteItem原始的，但我仍然需要能够根据需要多次运行该函数，以确保写入所有未处理项.如何正确循环 batchWriteItem?

I think the problem is trying to set the params in the callback function, but I don't know how else I'm supposed to do it...I know I could just call another batchWriteItem using UnprocessedItems within the callback of the original one, but I still need to be able to run the function as many times as needed to ensure all UnprocessedItems are written. How can I loop the batchWriteItem correctly?

推荐答案

@Daniela Miao，感谢分享解决方案.

@Daniela Miao, Thanks for sharing the solution.

我们可以在您发布的代码中添加一个代码块，以避免 DynamoDB 异常.这将在再次请求 DynamoDB 进行批量写入之前检查 params.RequestItems 是否有未处理的数据.

We can add one code block in your posted code which will avoid exception from DynamoDB. This will check if params.RequestItems has Unprocessed data before requesting the DynamoDB for batch write again.

//db is AWS.DynamoDB Client
var processItemsCallback = function(err, data) {
  if (err) { 
     //fail
  } else {
    var params = {};
    params.RequestItems = data.UnprocessedItems;
    /*
    * Added Code block 
    */
    if(Object.keys(params.RequestItems).length != 0) {
      db.batchWriteItem(params, processItemsCallback);
    }
  }
};

db.batchWriteItem(/*initial params*/, processItemsCallback);

相关文章