Skip to main content
 首页 » 编程设计

python之使用 python 进行高级解析(多行)

2025年02月15日16mayingbao

我被 python 2.7 的解析问题困在这里,让我解释一下:

我正在解析来自 incapsula API 的事件。目标是使它们在 excel 表中可读,以制作统计数据和图表。

在签名栏上,您可以阅读事件/攻击的类型和数字。 该数字包括攻击次数,因此我决定在“signature=”字段后将每一行乘以相应的攻击次数总和。

喜欢这个捕获:

 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3} 

到目前为止,一切都按预期进行,我得到了正确的攻击计数。

但是

在一些罕见的事件中,它们是签名字段上的多个值,就像这个捕获:

 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
 visit_id=324001290181618591, src_country=Ukraine, event_timestamp=1484493309742, src_ip=91.223.133.30, dest_name=www.xxx.com, dest_id=1551642, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
 visit_id=86001060468746692, src_country=Netherlands, event_timestamp=1483867285054, src_ip=178.22.232.53, dest_name=www.yyy.com, dest_id=1551642, signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
 visit_id=86001060468746692, src_country=Netherlands, event_timestamp=1483867285054, src_ip=178.22.232.53, dest_name=www.yyy.com, dest_id=1551642, signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
 visit_id=86001060468746692, src_country=Netherlands, event_timestamp=1483867285054, src_ip=178.22.232.53, dest_name=www.yyy.com, dest_id=1551642, signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
 visit_id=86001060468746692, src_country=Netherlands, event_timestamp=1483867285054, src_ip=178.22.232.53, dest_name=www.yyy.com, dest_id=1551642, signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 

我仍然对那些稀有行进行了正确的攻击计数,但我想从这里安排签名字段:

signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 

对此:

signature={api.threats.sql_injection} 
signature={api.threats.sql_injection} 
signature={api.threats.sql_injection} 
signature={api.threats.bot_access_control} 
signature={api.threats.illegal_resource_access} 
signature={api.threats.cross_site_scripting} 
signature={api.threats.bot_access_control} 
signature={api.threats.illegal_resource_access} 
signature={api.threats.illegal_resource_access} 
signature={api.threats.illegal_resource_access} 

(前六行是第一个事件重复6次(3+1+1+1 =6),后4行是第二个事件重复4次(1+3=4)

我当前的源代码:

#count the number of attack per line 
f = open('monthlyLogShort.txt','r') 
g = open("count.txt", 'w') 
kensu = f.readlines() 
f.close() 
for line in kensu: 
        st = line.find('signature=') 
        end = line.find('}') 
        unprecise = line[st:end+1] 
        #count = int(re.search(r'\d+', unprecise).group()) 
        count = sum(map(int,re.findall(r'[0-9]+', unprecise))) 
        print >> g, count 
 
g.close() 
 
#replicate lines according to the number of attack             
h = open('flog.txt','w') 
 
with open("monthlyLogShort.txt") as textfile1, open("count.txt") as textfile2: 
    for x, y in izip(textfile1, textfile2): 
        x = x.strip() 
        y = y.strip() 
        print >> h, x * int(y) 
h.close() 

请您参考如下方法:

如果我正确阅读了您的要求,您将尝试为每次威胁事件发出一行,同时保留其余记录。该解决方案不直接输出计数,而是转换数据,使其统一为每行一个威胁。

代码:

sig_str = 'signature={' 
for line in kensu: 
    record, signature = line.split(sig_str) 
    threats = signature.split('}')[0] 
    for counts in threats.split(','): 
        if '=' in counts: 
            threat, count = tuple(counts.split('=')) 
            for i in range(int(count)): 
                print '%s%s%s}' % (record, sig_str, threat.strip()) 

示例数据:

kensu = [x.strip() for x in """ 
    record=0, signature={api.threats.sql_injection=1} 
    record=1, signature={api.threats.sql_injection=3, api.threats.bot_access_control=1, api.threats.illegal_resource_access=1, api.threats.cross_site_scripting=1,} 
    record=2, signature={api.threats.bot_access_control=1, api.threats.illegal_resource_access=3,} 
""".split('\n')[1:-1]] 

输出:

record=0, signature={api.threats.sql_injection} 
record=1, signature={api.threats.sql_injection} 
record=1, signature={api.threats.sql_injection} 
record=1, signature={api.threats.sql_injection} 
record=1, signature={api.threats.bot_access_control} 
record=1, signature={api.threats.illegal_resource_access} 
record=1, signature={api.threats.cross_site_scripting} 
record=2, signature={api.threats.bot_access_control} 
record=2, signature={api.threats.illegal_resource_access} 
record=2, signature={api.threats.illegal_resource_access} 
record=2, signature={api.threats.illegal_resource_access}