Skip to main content
 首页 » 编程设计

python之Beautifulsoup 提取 li> 和 ul> 标签并将结果写入 CSV

2025年01月19日8lyj

我正在尝试从以下内容中提取所有行 ('li') 内的文本:

<ul id="tco_detail_data"> 
        <li> 
            <ul class="list-title"> 
                <li class="first"> </li> 
                <li>Year 1</li> 
                <li>Year 2</li> 
                <li>Year 3</li> 
                <li>Year 4</li> 
                <li>Year 5</li> 
                <li class="last">5 Yr Total</li> 
            </ul> 
        </li> 
        <hr class="loose-dotted" /> 
        <li class="first"> 
            <ul class="first"> 
                <li class="first">Depreciation</li> 
                <li>$5,390</li> 
                <li>$1,658</li> 
                <li>$1,459</li> 
                <li>$1,293</li> 
                <li>$1,161</li> 
                <li class="last">$10,961</li> 
            </ul> 
        </li> 
        <hr class="loose-dotted" /> 
        <li> 
            <ul> 
                <li class="first">Taxes &amp; Fees</li> 
                <li>$1,424</li> 
                <li>$61</li> 
                <li>$61</li> 
                <li>$61</li> 
                <li>$61</li> 
                <li class="last">$1,668</li> 
            </ul> 
        </li> 
        <hr class="loose-dotted" /> 
        <li> 
            <ul> 
                <li class="first">Financing</li> 
                <li>$1,022</li> 
                <li>$817</li> 
                <li>$603</li> 
                <li>$375</li> 
                <li>$135</li> 
                <li class="last">$2,952</li> 
            </ul> 

为了达到这一点,我使用了以下内容:

import requests 
from bs4 
import BeautifulSoup 
import csv 
page = requests.get('https://www.edmunds.com/ford/escape/2017/cost-to-own/') 
soup = BeautifulSoup(page.content, 'html.parser') 
data = soup.find_all("ul", {"id": "tco_detail_data"}) 

现在,为了提取 class="first"下的所有行,我使用了:

details = soup.find_all("li", {"class":"first"}) 

但是,它只获取第一个父 li 标签和它下面的子 li 标签。如何重复选择每个 li 类“第一”部分并将结果写入 CSV 的过程? 我将不胜感激任何指导。

请您参考如下方法:

这是与上一个答案类似的方法,它将以嵌套列表形式为您提供网页中的表格(即 [[table row], [table row], ...' :

data = soup.find_all("ul", {"id": "tco_detail_data"}) 
 
# get all list elements 
lis = data[0].find_all('li') 
 
# add a helper lambda, just for readability 
find_ul = lambda x: x.find_all('ul') 
uls = [find_ul(elem) for elem in lis if find_ul(elem) != []] 
 
# use a nested list comprehension to iterate over the <ul> tags 
# and extract text from each <li> into sublists 
text = [[li.text.encode('utf-8') for li in ul[0].find_all('li')] for ul in uls] 
 
# [ 
#   ['\xc2\xa0', 'Year 1', 'Year 2', 'Year 3', 'Year 4', 'Year 5', '5 Yr Total'], 
#   ['Depreciation', '$4,853', '$1,658', '$1,459', '$1,293', '$1,161', '$10,424'], 
#   ['Taxes & Fees', '$2,057', '$21', '$66', '$21', '$66', '$2,231'], 
#   ['Financing', '$1,026', '$821', '$605', '$376', '$136', '$2,964'], 
#   ['Fuel', '$1,606', '$1,654', '$1,704', '$1,755', '$1,808', '$8,527'], 
#   ['Insurance', '$764', '$791', '$818', '$847', '$877', '$4,097'], 
#   ['Maintenance', '$230', '$601', '$385', '$1,653', '$1,504', '$4,373'], 
#   ['Repairs', '$0', '$0', '$109', '$257', '$374', '$740'], 
#   ['Tax Credit', '$0', '', '', '', '', '$0'], 
#   ['True Cost to Own \xc2\xae', '$10,536', '$5,546', '$5,146', '$6,202', '$5,926', '$33,356'] 
# ] 
 
# write "text" list to csv 
with open('ford_escape_2017.csv', 'w') as f: 
    writer = csv.writer(f) 
    writer.writerows(text)