Dynamodb 过滤器表达式不返回所有结果

2022-01-13 00:00:00 node.js nosql amazon-dynamodb javascript

我想扫描过去 7 天的所有项目，所以我要做的是生成 7 天前的时间戳并过滤大于该值的时间戳.但此扫描返回了一些结果.

I want to scan all items for last 7 days, so what I do is I generate timestamp for 7 days back and filter for timestamp greater than that value. But this scan is returning a few results.

请参阅以下 Javascript:

See the following Javascript:

const daysBack = (days) => {
  let date = new Date();
  date.setDate(date.getDate() - days);
  return date.getTime() ; 
}


const params = {
  TableName: process.env.DYNAMODB_TABLE,
  FilterExpression: "#ts > :z",
  ExpressionAttributeNames:{
      "#ts": "timestamp"
  },
  ExpressionAttributeValues: {
      ":z": daysBack(7)
  },
};

dynamoDb.scan(params, (error, result) => {
  // ... 
}

推荐答案

这是因为在 SCAN 操作上 dynamoDb 只会发送 最多 1mb 的数据.如果您想要的记录大小超过 1mb，则会自动分页.

It is because on a SCAN operation dynamoDb will only send data upto 1mb only. If the records you want are of size more than 1mb automatically pagination happens.

如果你记录你的结果，那么你会发现一个名为 LastEvaluatedKey 的属性如果此属性存在，那么您将不得不再次调用以获取剩余数据.此调用必须递归实现，并且您必须在 LastEvaluatedKey 属性不存在时停止它.

If you log your Result then you will find an attribute called LastEvaluatedKey if this attribute is present then you will have to make another call to get the remaining data. This call has to be implemented recursively and you have to stop it when LastEvaluatedKey attribute is not present.

让我们看看这个例子，项目数据被递归获取，整个数据被附加到数组中，然后发送.

Lets see this example where project data is been fetched recursively and the whole data is appended in the array and then send.

let getFromDb = function (params, callback) {
    params.ConsistentRead = true;
    let projectCollection = [];
    dynamodbclient.scan(params, onQuery);

    function onQuery(err, data) {
        const methodName = 'onQuery';
        if (err) {
            callback(err);
            log.error(err, {
                class: className,
                func: methodName
            });
        } else {
            for (let i = constant.LENGTH_ZERO; i < data.Items.length; i++) {
                projectCollection.push(data.Items[i]);
            }
            if (typeof data.LastEvaluatedKey !== 'undefined') {
                params.ExclusiveStartKey = data.LastEvaluatedKey;
                dynamodbclient.scan(params, onQuery);
            } else {
                callback(err, projectCollection); //recursive call
            }
        }
    }
};

相关文章