Skip to main content
 首页 » 编程设计

python之从 HTML 中提取标签之间的特定文本部分

2024年11月24日9powertoolsteam

我想从 HTML 文件(“项目 1A”部分)的特定部分中提取文本。我想从内容部分而不是目录中的“项目 1A”开始获取文本,并在“项目 1B”处停止。但是“Item 1A”和“Item 1B”有几个相同的文本。我如何确定要开始和停止的特定文本。

import requests 
from bs4 import BeautifulSoup 
import re 
 
url = "https://www.sec.gov/Archives/edgar/data/1606163/000114420416089184/v434424_10k.htm" 
res = requests.get(url) 
soup = BeautifulSoup(res.text, "html.parser") 
text = soup.get_text() 
 
item1a = re.search(r"(item\s1A\.?)(.+)(item\s1B\.?)", text, re.DOTALL | re.IGNORECASE) 
 
item1a.group(2) 
 

输出捕获目录中第一个“Item 1A”的文本,而不是该部分的标题。

因此我想知道:

  1. 如何从内容部分的“项目 1A”而不是目录中的“项目 1A”捕获文本。

  2. 为什么它捕获了最后一个“Item 1B”而不是停止在目录中的“Item 1B”。

请您参考如下方法:

既然您有一个可以帮助您处理 HTML 结构的 soup,为什么不利用它呢?

一种表述方式是“在具有特定属性的两个标签之间查找文本”。 (代表 1A 和 1B header 的标签。)为此,您可以将可调用(函数)传递给 soup.find():

import requests 
from bs4 import BeautifulSoup 
from bs4.element import Tag 
import re 
 
url = "https://www.sec.gov/Archives/edgar/data/1606163/000114420416089184/v434424_10k.htm" 
res = requests.get(url) 
soup = BeautifulSoup(res.text, "html.parser") 
 
def is_pstyle(tag: tag) -> bool: 
    return tag.name == "p" and tag.has_attr("style") 
 
def is_i1a(tag: Tag) -> bool: 
    return is_pstyle(tag) and re.match(r"Item 1A\..*", tag.text) 
 
def is_i1b(tag: Tag) -> bool: 
    return is_pstyle(tag) and re.match(r"Item 1B\..*", tag.text) 
 
def grab_1a_thru_1b(soup: BeautifulSoup) -> str: 
    start = soup.find(is_i1a) 
    def gen_t(): 
        for tag in start.next_siblings: 
            if is_i1b(tag): 
                break 
            else: 
                if hasattr(tag, "get_text"): 
                    yield tag.get_text()  # get_text("\n") 
                else: 
                    yield str(tag) 
    return "".join(gen_t()) 
 
if __name__ == "__main__": 
    print(grab_1a_thru_1b(soup)) 

第一部分输出:

The risks and uncertainties described below 
are those specific to the Company which we currently believe have the potential to be material, but they may not be the only ones 
we face. If any of the following risks, or any other risks and uncertainties that we have not yet identified or that we currently 
consider not to be material, actually occur or become material risks, our business, prospects, financial condition, results of 
operations and cash flows could be materially and adversely affected. Investors are advised to consider these factors along with 
the other information included in this Annual Report and to review any additional risks discussed in our filings with the SEC. 
  
Risks Associated with Our Business 
  
We are a newly formed company with no operating history and, accordingly, you have no basis on which to evaluate our ability to achieve our business 
objective. 

您可以将迷你函数 is_pstyle、is_i1a 和 is_i1b 视为“过滤器”——只是通过不同的方式来精确查找开始和结束标签。然后迭代这些标签之间的兄弟标签。 (.get_text() 将在每个同级标签内递归工作。)