Skip to main content
 首页 » 编程设计

python之用python在网站文章中搜索关键词

2024年11月24日11zhenyulu

刚加入你们,开始学习python已经1个月了。我想用 Python ( http://aeconf.com/may2013.htm ) 从这个站点搜索关键字。

通常我手动单击 View 摘要并在“关键词:”之后搜索词。我如何使用 python 自动执行此操作?

请您参考如下方法:

你应该看看 Selenium

pip install selenium 

我提供了它可以做什么的示例代码,您应该对其进行测试。

示例代码:

from selenium import webdriver 
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities 
from selenium.webdriver.common.by import By 
from selenium.common.exceptions import NoSuchElementException 
 
 
caps = DesiredCapabilities().CHROME 
caps["pageLoadStrategy"] = "normal"   
driver = webdriver.Chrome(desired_capabilities=caps, executable_path=r'C:\Users\My-PC-Name\AppData\Local\Programs\Python\Python37-32\Scripts\chromedriver.exe') 
 
 
url = "http://aeconf.com/may2000.htm" #Link 
driver.get(url) 
links = [x.get_attribute('href') for x in driver.find_elements_by_link_text('View Abstract')] 
htmls = [] 
 
for link in links: 
   driver.get(link) 
   Keyword = [y.text for y in driver.find_elements_by_xpath("//font[2]/span[@style = 'mso-bidi-font-size: 1.0pt']")] 
   if not Keyword: #If link is a dead link 
     continue 
   print(Keyword[0]) 
   htmls.append(driver.page_source) 

在此示例中,我将 url 更改为 http://aeconf.com/may2000.htm我提供的代码主要获取您需要的“关键词”,但在某些情况下,“关键词”的索引位置会根据上述 Url 中的链接而变化。

“已更改”链接的输出:

Fiscal decentralization; Corruption; Tax evasion. 
Incentive mechanism design; Walrasian allocations; Implementation. 
Debt and equity flows; Asymmetric information; Bankruptcy cost; Market failures;  
Corrective taxation. 
Transitory volatility; Price formation; Exogenous liquidity demand. 
Investment horizon; Beta; Size; Book-to-market equity; CAPM. 
G11, G13.                         #At This part you can see that the 'Key Words' printed are not correct 
Portfolio constraints; Stochastic income; Relaxation-projection methods. 
Foreign aid; Foreign borrowing; Capital accumulation. 
Entrepreneurial ability; Asymmetric information; Liquidity constraints. 
Contract; Human capital; Labor. 
Endogenous structure of the division of labor; Dual economy; Endogenous trade policy regime. 

如果我们将示例代码中的“url”变量更改为您原来的 link即使第一个链接是死链接,索引位置也会更改的情况更多。作为挑战,我会让你自己弄明白:-) 有更多的模块可以像 Selenium 一样做同样的事情。希望这会让您对浏览器自动化、网络抓取等(网络爬虫等)产生更多兴趣。

只是提示(可能不是提示)您只需更改“关键字”变量的索引位置即可获得所需的“关键字”。