Skip to main content
 首页 » 编程设计

python之无法从某些元素中抓取特定项目

2025年02月15日10dflying

当没有容器或组可供选择以循环解析所需项目(每个组中共有的项目)时怎么办?我愿意从粘贴的元素中解析文本、日期和作者。我所追求的三个结果不属于任何特定的组或容器,因此我找不到让它们创建循环的正确方法。

元素如下:

html = ''' 
<div class="view-content">             
  <p class="text-large experts-more-h">    
  <a href="/publications/commentary/we-have-no-idea-universal-preschool-actually-helps-kids">We Have No Idea if Universal Preschool Actually Helps Kids</a> 
  </p> 
  <p class="text-sans">     
  By David J. Armor. Washington Post. <span class="date-display-single" property="dc:date" datatype="xsd:dateTime" content="2014-10-21T09:34:00-04:00">October 21, 2014</span>. 
  </p>         
  <p class="text-large experts-more-h">    
  <a href="/publications/commentary/last-parent-resistance-collective-standardized-tests">At Last, Parent Resistance to Collective Standardized Tests</a> 
  </p> 
  <p class="text-sans">     
  By Nat Hentoff. Cato.org. <span class="date-display-single" property="dc:date" datatype="xsd:dateTime" content="2014-01-15T09:57:00-05:00">January 15, 2014</span>. 
  </p>   
  <p class="text-sans">     
  By Darcy Ann Olsen and Eric Olsen. Cato.org. <span class="date-display-single" property="dc:date" datatype="xsd:dateTime" content="1999-04-15T00:00:00-04:00">April 15, 1999</span>. 
  </p>        
  <p class="text-large experts-more-h">    
  <a href="/publications/commentary/day-care-parents-versus-professional-advocates-0">Day Care: Parents versus Professional Advocates</a> 
  </p> 
  <p class="text-sans">    
  By Darcy Ann Olsen. Cato.org. <span class="date-display-single" property="dc:date" datatype="xsd:dateTime" content="1998-06-01T00:00:00-04:00">June 1, 1998</span>. 
  </p>   
</div> 
''' 

如果你运行我的脚本,你可以看到抓取的结果只有第一个:

from lxml.html import fromstring 
 
tree = fromstring(html) 
post= tree.cssselect(".text-large a")[0].text 
date = tree.cssselect(".date-display-single")[0].text 
author = tree.cssselect(".text-sans")[0].text.strip() 
print(post+'\n', date+'\n', author) 

结果:

We Have No Idea if Universal Preschool Actually Helps Kids 
 October 21, 2014 
 By David J. Armor. Washington Post. 

如果你运行这个脚本,你会看到这个脚本能够解析我想要的所有结果:

from lxml.html import fromstring 
 
tree = fromstring(html) 
count = tree.cssselect(".text-large a") 
 
for item in range(len(count)): 
    post= tree.cssselect(".text-large a")[item].text 
    date = tree.cssselect(".date-display-single")[item].text 
    author = tree.cssselect(".text-sans")[item].text.strip() 
    print(post+'\n', date+'\n', author) 

结果:

We Have No Idea if Universal Preschool Actually Helps Kids 
 October 21, 2014 
 By David J. Armor. Washington Post. 
At Last, Parent Resistance to Collective Standardized Tests 
 January 15, 2014 
 By Nat Hentoff. Cato.org. 
Day Care: Parents versus Professional Advocates 
 April 15, 1999 
 By Darcy Ann Olsen and Eric Olsen. Cato.org. 

但是,我对第二个脚本所做的完全不是 pythonic,如果缺少任何数据,它将给出错误的结果。那么,如何选择一个组或容器,循环遍历并解析它们呢?提前致谢。

请您参考如下方法:

如果其中一个文本节点(post、date、author)缺失,tree.cssselect(selector)[index ].text 应该返回一个 NoneType 对象,您不能将其作为字符串处理。为避免这种情况,您可以实现

post= tree.cssselect(".text-large a")[item].text or " " 

您也可以尝试以下 XPath 解决方案:

container = tree.cssselect(".text-large") 
 
for item in container: 
    post = item.xpath('./a')[0].text or " " 
    date = item.xpath('./following-sibling::p/span[@class="date-display-single"]')[0].text or " " 
    author = item.xpath('./following-sibling::p[@class="text-sans"]')[0].text.strip() or " " 
    print(post+'\n', date+'\n', author)