Skip to main content
 首页 » 编程设计

python之用 python pandas 解析日志文件

2025年05月04日18lidabo

问题1

我正在尝试从模拟中读取和解析日志文件。理想情况下,我想用 Pandas 来做这件事,但我遇到了问题。有问题的文件称为 log,下面包含一个示例。现在我试着做

     import pandas as pd 
     import numpy as np 
     import csv 
     
     # This method works: 
     results = [] 
     with open('log') as inputfile: 
             for row in csv.reader(inputfile): 
                 results.append(row) 
     
    
      # This method doesn't work: 
      pd.read_csv('log')  

错误是

pandas.parser.CParserError: Error tokenizing data. C error: Expected 1 fields in line 77, saw 4 

为什么会这样?

问题2

解析日志文件的最有效方法是什么?请记住,该文件实际上会非常大。基本上我想将每个时间步长的残差(每个时间步长由 Time= x 指定)解析为一个 numpy 数组用于绘图。例如

smoothSolver:  Solving for gamma, Initial residual = 0.813857557031, Final residual = 6.74558898691e-06, No Iterations 2 

我想将 0.813857557031 附加到数组 res_gamma_init

最有效的方法是什么?

/*---------------------------------------------------------------------------*\ 
| =========                 |                                                 | 
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           | 
|  \\    /   O peration     | Version:  2.3.x                                 | 
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      | 
|    \\/     M anipulation  |                                                 | 
\*---------------------------------------------------------------------------*/ 
Build  : 2.3.x-590d57f32fed 
Exec   : simpleFoam 
Date   : Oct 30 2015 
Time   : 17:24:08 
Host   : "jack" 
PID    : 4034 
Case   : /home/jack/OpenFOAM/jack-2.3.x/run/sim 
nProcs : 1 
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE). 
fileModificationChecking : Monitoring run-time modified files using timeStampMaster 
allowSystemOperations : Allowing user-supplied system call operations 
 
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // 
Create time 
 
Create mesh for time = 0 
 
Reading field p 
 
Reading field U 
 
Reading/calculating face flux field phi 
 
Selecting incompressible transport model Newtonian 
Selecting RAS turbulence model kOmegaSST 
Using gammaReThetat-correlations by Langtry and Menter (2009) 
gammaReThetatSSTCoeffs 
{ 
    ca1             2; 
    ce1             1; 
    ca2             0.06; 
    ce2             50; 
    cThetat         0.03; 
    sigmaf          1; 
    sigmaThetat     2; 
    s1              2; 
    dUds            false; 
    alphaK1         0.85034; 
    alphaK2         1; 
    alphaOmega1     0.5; 
    alphaOmega2     0.85616; 
    gamma1          0.5532; 
    gamma2          0.4403; 
    beta1           0.075; 
    beta2           0.0828; 
    betaStar        0.09; 
    a1              0.31; 
    c1              10; 
    kInf            0; 
    omegaInf        0; 
} 
 
No finite volume options present 
 
 
SIMPLE: convergence criteria 
    field p  tolerance 1e-06 
    field U  tolerance 1e-06 
    field "(k|omega|ReThetaTilda|nut|gamma)"     tolerance 1e-06 
 
 
Starting time loop 
 
 
forceCoeffs forceCoeffs: 
    Not including porosity effects 
 
Time = 1 
 
smoothSolver:  Solving for Ux, Initial residual = 0.999999999388, Final residual = 0.00692443749034, No Iterations 2 
smoothSolver:  Solving for Uy, Initial residual = 0.999999994742, Final residual = 0.0027182321684, No Iterations 2 
DICPCG:  Solving for p, Initial residual = 1, Final residual = 0.00976261323556, No Iterations 326 
time step continuity errors : sum local = 8.88353600919, global = -0.154492041821, cumulative = -0.154492041821 
smoothSolver:  Solving for omega, Initial residual = 1, Final residual = 0.0110458877993, No Iterations 2 
smoothSolver:  Solving for k, Initial residual = 0.999999999804, Final residual = 0.022014322402, No Iterations 2 
bounding k, min: 0 max: 0.415754458684 average: 0.410526729282 
smoothSolver:  Solving for ReThetatTilda, Initial residual = 1, Final residual = 0.0259070897862, No Iterations 2 
smoothSolver:  Solving for gamma, Initial residual = 1, Final residual = 1.51565684699e-05, No Iterations 2 
ExecutionTime = 1.41 s  ClockTime = 1 s 
 
forceCoeffs forceCoeffs output: 
    Cm    = 0 
    Cd    = -19811.1501384 
    Cl    = 4.01861806726e-15 
    Cl(f) = 2.00930903363e-15 
    Cl(r) = 2.00930903363e-15 
 
Time = 2 
 
smoothSolver:  Solving for Ux, Initial residual = 0.510606164486, Final residual = 0.0151751146151, No Iterations 2 
smoothSolver:  Solving for Uy, Initial residual = 0.570639175137, Final residual = 0.0164524779792, No Iterations 2 
DICPCG:  Solving for p, Initial residual = 0.160667758287, Final residual = 0.00156013775625, No Iterations 143 
time step continuity errors : sum local = 88.8245306208, global = 11.6476210117, cumulative = 11.4931289699 
smoothSolver:  Solving for omega, Initial residual = 0.531389189498, Final residual = 0.00899532326598, No Iterations 2 
smoothSolver:  Solving for k, Initial residual = 0.562103650426, Final residual = 0.0138022250836, No Iterations 2 
bounding k, min: 0 max: 0.425403200237 average: 0.411310758596 
smoothSolver:  Solving for ReThetatTilda, Initial residual = 0.301017401945, Final residual = 0.00693946297375, No Iterations 2 
smoothSolver:  Solving for gamma, Initial residual = 0.813857557031, Final residual = 6.74558898691e-06, No Iterations 2 
ExecutionTime = 2.13 s  ClockTime = 2 s 
 
forceCoeffs forceCoeffs output: 
    Cm    = 0 
    Cd    = -20283.0256988 
    Cl    = 3.99364280461e-15 
    Cl(f) = 1.9968214023e-15 
    Cl(r) = 1.9968214023e-15 
 
Time = 3 

请您参考如下方法:

回答你的第二个问题。这可能会有所帮助,尽管正则表达式可能会更好。

line1 = 'smoothSolver:  Solving for Ux, Initial residual = 0.999999999388, Final residual = 0.00692443749034, No Iterations 2' 
line2 = 'smoothSolver:  Solving for Uy, Initial residual = 0.999999994742, Final residual = 0.0027182321684, No Iterations 2' 
test = [line1, line2] 
import re 
for line in test: 
    if line.startswith('smoothSolver'): 
        print re.findall(r'[residual = ][\d][.][\d]+', line) 

控制台的结果是:

[' 0.999999999388', ' 0.00692443749034'] 
[' 0.999999994742', ' 0.0027182321684']