Skip to main content
 首页 » 编程设计

python之使用lxml xpath解析xml文件

2024年12月31日7findumars

我正在使用 lxml XPath 来解析以下 xml 文件

<urlset 
    xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" 
    xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" 
    xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"> 
    <url> 
        <loc> 
    https://www.reuters.com/article/us-campbellsoup-thirdpoint/campbell-soup-nears-deal-with-third-point-to-end-board-challenge-sources-idUSKCN1NU11I 
    </loc> 
        <image:image> 
            <image:loc> 
    https://www.reuters.com/resources/r/?m=02&d=20181126&t=2&i=1328589868&w=&fh=&fw=&ll=460&pl=300&r=LYNXNPEEAO0WM 
    </image:loc> 
        </image:image> 
        <news:news> 
            <news:publication> 
                <news:name>Reuters</news:name> 
                <news:language>eng</news:language> 
            </news:publication> 
            <news:publication_date>2018-11-26T02:55:00+00:00</news:publication_date> 
            <news:title> 
    Campbell Soup nears deal with Third Point to end board challenge: sources 
    </news:title> 
            <news:keywords>Headlines,Business, Industry</news:keywords> 
            <news:stock_tickers>NYSE:CPB</news:stock_tickers> 
        </news:news> 
    </url> 
</urlset> 

Python 代码示例

import lxml.etree 
import lxml.html 
import requests 
 
def main(): 
    r = requests.get("https://www.reuters.com/sitemap_news_index1.xml") 
 
    namespace = "http://www.google.com/schemas/sitemap-news/0.9" 
    root = lxml.etree.fromstring(r.content) 
 
 
    records = root.xpath('//news:title', namespaces = {"news": "http://www.google.com/schemas/sitemap-news/0.9"}) 
    for record in records: 
        print(record.text) 
 
 
    records = root.xpath('//sitemap:loc', namespaces = {"sitemap": "http://www.sitemaps.org/schemas/sitemap/0.9"}) 
    for record in records: 
        print(record.text) 
 
 
if __name__ == "__main__": 
    main() 

目前,我使用 XPath 获取所有 URL 和 title,但这不是我想要的,因为我不知道哪个 URL 属于哪个标题。我的问题是如何获得每个 <url> ,然后循环每个 <url>作为项目得到相应的<loc>和 <news:keywords>等谢谢!

编辑: 期待输出

foreach <url> 
      get <loc> 
      get <news:publication_date> 
      get <news:title> 

请您参考如下方法:

使用相对 XPath 从每个标题获取其关联的 URL:

ns = { 
    "news": "http://www.google.com/schemas/sitemap-news/0.9", 
    "sitemap": "http://www.sitemaps.org/schemas/sitemap/0.9", 
    "image": "http://www.google.com/schemas/sitemap-image/1.1" 
} 
 
r = requests.get("https://www.reuters.com/sitemap_news_index1.xml") 
root = lxml.etree.fromstring(r.content) 
 
for title in root.xpath('//news:title', namespaces=ns): 
    print(title.text) 
 
    loc = title.xpath('ancestor::sitemap:url/sitemap:loc', namespaces=ns) 
    print(loc[0].text) 

练习:重写此代码以从 URL 获取关联的标题。

注意:标题(可能还有 URL)似乎是 HTML 转义的。使用unescape()函数

from html import unescape 

取消转义。