Skip to main content
 首页 » 编程设计

python之构建连接日志系统

2025年01月19日8小虾米

我正在构建一个“智能”日志系统,我可以在其中监控客户连接,例如启动和停止与服务器的连接建立时间。

原始日志:

Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: terminating... - peer is not responding 
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info,account customer1 logged out, 4486 1009521 23444247 12573 18159 
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: disconnected 
Dec 19 00:00:07 172.16.20.24 pppoe,info PPPoE connection established from 60:E3:27:A2:60:09 
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info,account customer2 logged in, 10.171.3.185 
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: authenticated 
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: connected 
Dec 19 00:00:13 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41 
Dec 19 00:00:14 172.16.20.24 pppoe,ppp,error <ccfa>: user customer3 authentication failed 
Dec 19 00:00:32 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41 
Dec 19 00:00:36 172.16.20.24 pppoe,ppp,error <ccfb>: user customer3 authentication failed 
Dec 19 00:01:06 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41 

对我来说重要的是:捕获带有连接 和断开 字符串的行。

我明白了:

import os 
import re 
import sys 
 
f = open('log.log','r') 
log = [] 
for line in f: 
 if re.search(r': connected|: disconnected',line): 
  ob = dict() 
  ob['USER'] = re.search(r'<pppoe(.*?)>',line).group(0).replace("<pppoe-","").replace(">","") 
  ob['DATA'] = re.search(r'^\w{3} \d{2} \d{2}:\d{2}:\d{2}',line).group(0) 
  ob['CONNECTION'] = re.search(r': .*',line).group(0).replace(": ", "") 
  log.append(ob) 

我还在学习,所以这不是最出色的正则表达式,但没关系! 需要现在优化这个日志列表,想要得到这个样本:

{"connection" : [{ 
"start" : "Dec 19 10:12:58",  
"username" : "customer2"} 
 
{"connection" : [{ 
"start" : "Dec 20 10:12:58",  
"username" : "customer1"} 
 
{"connection" : [{ 
"start" : "Dec 19 10:12:58",  
"stop" : Dec 22 10:04:35", 
"username" : "customer4"} 
 
{"connection" : [{ 
"start" : "Dec 19 10:12:58", 
"stop" : "Dec 24 10:04:35"  
"username" : "customer3"} 

我的障碍,

  • 不断生成 RAW 日志,我需要确定是否有一些 用户已存在。 如果是:更新连接(客户 2 断开他的连接,需要注册它!)但是如果他经常断开连接会发生什么?

例如:

Dec 19 10:20:58 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: disconnected     
Dec 19 01:00:36 172.16.20.24 pppoe,ppp,error <ccfb>: user customer3 authentication failed 
Dec 19 01:01:06 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41 
Dec 19 10:21:38 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: authenticated 
Dec 19 10:21:48 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: connected 
Dec 19 10:22:38 172.16.20.24 pppoe,ppp,info <pppoe-customer3>: authenticated 
Dec 19 10:22:58 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: disconnected   

先断开连接,添加起来简单。

{"connection" : [{ 
"start" : "Dec 19 10:12:58" 
"stop" : "Dec 19 10:20:58",  
"username" : "customer2"} 

在下一次身份验证中,我需要搜索这个特定的用户,插入新的“开始”连接时间并删除“停止”。等等。

{"connection" : [{ 
"start" : "Dec 19 10:21:48"  
"username" : "customer2"} 
  • 我的下一个挑战者,它创建了这个新的优化列表。

试过了,但是不行!

conn = [] 
for l in log: 
 obcon = dict() 
 if not obcon: 
    obcon['USER'] = l['USER'] 
    if l['DATA'] == 'connected': 
        obcon['START'] = l['DATA']       
        obcon['STOP'] = "" 
    else: 
        obcon['STOP'] = l['DATA'] 
 conn.append(obcon) 

在构建新列表之前,我需要检查是否存在某个用户,如果不存在,让我们构建它!我用来识别开始/停止连接的 ['CONNECTION']:

Disconnected -> STOP 
Connected -> START 

我不知道我是否需要更具体一些。 需要想法。请!

请您参考如下方法:

在我看来,var log应该是 dict 类型因为它将帮助您更轻松地找到现有的用户数据。
接下来,您使用了 re(...).group(0)到处都是entire matching string .比如提取用户名的时候,你写了'<pppoe(.*?)>' , 但它位于 group(1) (在正则表达式中,括号用于匹配提取)。

我的建议是(注意 - 我删除了 sys 和 os 的导入,因为它们没有被使用):

import re 
 
f = open('log.log', 'r') 
log = dict() 
for line in f: 
    reg = re.search(r': ((?:dis)?connected)', line) # finds connected or disconnected 
    if reg is not None: 
        user = re.search(r'<pppoe-(.*?)>', line).group(1) 
        # if the user in the log, get it, else create it with empty dict 
        ob = log.setdefault(user, dict({'USER': user}))  
        ob['CONNECTION'] = reg.group(1) 
        time = re.search(r'^\w{3} \d{2} \d{2}:\d{2}:\d{2}', line).group(0) 
        if ob['CONNECTION'].startswith('dis'): 
            ob['END'] = time 
        else: 
            ob['START'] = time 
            if 'END' in ob: 
                ob.pop('END') 

如果日志文件是:

Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: terminating... - peer is not responding 
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info,account customer1 logged out, 4486 1009521 23444247 12573 18159 
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: disconnected 
Dec 19 00:00:07 172.16.20.24 pppoe,info PPPoE connection established from 00:00:00:00:00:00 
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info,account customer2 logged in, 127.0.0.1 
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: authenticated 
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: connected 
Dec 19 00:00:13 172.16.20.24 pppoe,info PPPoE connection established from 00:00:00:00:00:00 
Dec 19 00:00:14 172.16.20.24 pppoe,ppp,error <ccfa>: user customer3 authentication failed 
Dec 19 00:02:03 172.16.20.24 pppoe,ppp,info,account customer2 logged out, 4486 1009521 23444247 12573 18159 
Dec 19 00:02:03 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: disconnected 
Dec 19 00:02:08 172.16.20.24 pppoe,ppp,info,account customer3 logged in, 127.0.0.1 
Dec 19 00:02:08 172.16.20.24 pppoe,ppp,info <pppoe-customer3>: authenticated 
Dec 19 00:02:08 172.16.20.24 pppoe,ppp,info <pppoe-customer3>: connected 

log 的值将是:

{ 
    'customer1': { 
        'CONNECTION': 'disconnected', 
        'END': 'Dec 19 00:00:03', 
        'USER': 'customer1' 
    },  
    'customer3': { 
        'START': 'Dec 19 00:02:08', 
        'CONNECTION': 'connected', 
        'USER': 'customer3' 
    },  
    'customer2': { 
        'START': 'Dec 19 00:00:08', 
        'CONNECTION': 'disconnected', 
        'END': 'Dec 19 00:02:03',  
        'USER': 'customer2' 
    } 
}