ELK-基础系列(五)-Logstash 正则实践
Logstash 正则参考地址
参考地址：
- https://www.elastic.co/guide/en/logstash/current/filter-plugins.html
- https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns
- http://doc.yonyoucloud.com/doc/logstash-best-practice-cn/filter/date.html
应用日志文件内容
Sep 2 16:00:01 cc-prd-3-tk-13-pcs1 CRON[10590]: (root) CMD ( /usr/sbin/ntpdate -u ntp1.aliyun.com ntp2.aliyun.com >/dev/null 2>&1)
Sep 2 16:00:18 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:18 2019: PCS: recv request 339773 from ip: 27.102.113.76:55534, public_ip: 223.104.145.136
Sep 2 16:00:18 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:18 2019: PCS: 223.104.145.136:24533 223.104.145.136 00003b9f21835d46c2d36def87b223502dddf60ecb182032d20f5d3bc32124c2fac8859c873db6 e587ec4269d2ff63a25b0dc3716e63f9 5c611e85bfd0d6c50c906dfded565a0c 1.0.6.020 0 1567433717 1567440017 1 1 26 1058673192
Sep 2 16:00:18 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:18 2019: PCS: recv request, app_id: app_1w5B6O4R2o1k881k12177, ip: 223.104.145.136, os: 0, device_id: 02c3864050502d43dc514905133bcc9c
Sep 2 16:00:18 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:18 2019: PCS: allocate AN, app_id: app_1w5B6O4R2o1k881k12177, ip: 223.104.145.136, AN: an_1z566D44161870620, region: cn-east
Sep 2 16:00:18 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:18 2019: PCS: end client request 339773
Sep 2 16:00:18 cc-prd-3-tk-13-pcs1 systemd-journald: Forwarding to syslog missed 12 messages.
Sep 2 16:00:27 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:27 2019: PCS: recv request 339781 from ip: 34.92.148.220:33430, public_ip: 119.251.248.84
Sep 2 16:00:27 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:27 2019: PCS: 119.251.248.84:1420 119.251.248.84 00005435801f5d3788f16b0f2dc1f8dd794d5e06410ba8a5e7177559ca45d933dcefedd0ff2f4e e587ec4269d2ff63a25b0dc3716e63f9 5c611e85bfd0d6c50c906dfded565a0c 1.0.6.020 0 1567440027 1567440026 0 1 0 1345592700
Sep 2 16:00:27 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:27 2019: PCS: recv request, app_id: app_1w5B6O4R2o1k881k12177, ip: 119.251.248.84, os: 0, device_id: a658a1ea82ca183d10856399bba7d3ca
Sep 2 16:00:27 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:27 2019: HBase got exception THRIFT_EAGAIN (timed out)
Sep 2 16:00:27 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:27 2019: PCS: allocate AN, app_id: app_1w5B6O4R2o1k881k12177, ip: 119.251.248.84, AN: an_1b5m6i42161870620, region: cn-north
Sep 2 16:00:27 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:27 2019: PCS: end client request 339781
Sep 2 16:00:44 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:00:44 2019: mq(PC-cc-prd-3-tk-13-pcs1) buffer delivered: CMD_PUSH_CONN_STATUS#015#012{#012 "access-node" : [#012 {#012 "node-name" : "an_1P5n6R6k973774022",#012 "nr_cur_clients" : 0,#012 "nr_cur_conns" : 0#012 },#012 {#012 "node-name" : "an_1H5X6e6F726082009",#012 "nr_cur_clients" : 6,#012 "nr_cur_conns" : 7#012 },#012 {#012 "node-name" : "an_1z566D44161870620",#012 "nr_cur_clients" : 4,#012 "nr_cur_conns" : 6#012 },#012 {#012 "node-name" : "an_1b5m6i42161870620",#012 "nr_cur_clients" : 5,#012 "nr_cur_conns" : 6#012 }#012 ],#012 "app-list" : [#012 {#012 "app_id" : "app_1w5B6O4R2o1k881k12177",#012 "reject_new_conn" : 0#012 }#012 ]#012}
Sep 2 16:01:30 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:01:30 2019: PCS: recv request 339841 from ip: 35.194.199.109:48494, public_ip: 27.18.224.156
Sep 2 16:01:30 cc-prd-3-tk-13-pcs1 systemd-journald: Forwarding to syslog missed 8 messages.
Sep 2 16:01:30 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:01:30 2019: PCS: 27.18.224.156:26775 27.18.224.156 00004713c0715d6f54686d5739952af9d6a86337c27810a2e71f95d49efade528447794e53018a e587ec4269d2ff63a25b0dc3716e63f9 5c611e85bfd0d6c50c906dfded565a0c 1.0.6.020 0 1567438620 1567440090 1 1 26 1133366485
Sep 2 16:01:30 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:01:30 2019: PCS: recv request, app_id: app_1w5B6O4R2o1k881k12177, ip: 27.18.224.156, os: 0, device_id: 88743977340299423bdb63baad439b3e
Sep 2 16:01:30 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:01:30 2019: PCS: allocate AN, app_id: app_1w5B6O4R2o1k881k12177, ip: 27.18.224.156, AN: an_1z566D44161870620, region: cn-east
Sep 2 16:01:30 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:01:30 2019: PCS: end client request 339841
Sep 2 16:01:51 cc-prd-3-tk-13-pcs1 pcs[16118]: Mon Sep 2 16:01:51 2019: PCS: recv request 339862 from ip: 27.102.113.76:55544, public_ip: 117.136.11.24
"pcs1-20190903.log" 50466L, 9818998C相关文章