我正在尝试收集 google 搜索页面的标题和链接,我正在使用 selenium。 我使用 xpath 填充字段并单击按钮。
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import pandas as pd
browser = webdriver.Chrome(executable_path ='c:\\chromedriver.exe')
browser.get('https://www.google.com')
search_bar_xpath=browser.find_element_by_xpath('//* [@id="tsf"]/div[2]/div/div[1]/div/div[1]/input')
search_bar_xpath.send_keys('mybirthday')
search_button = browser.find_element_by_xpath('//*[@id="tsf"]/div[2]/div/div[3]/center/input[1]')
search_button.click()
search_results= browser.find_elements_by_xpath('//*[@id="rso"]/div[2]/div/div[2]/div/div/div[1]')
scrap_data=[]
for search_result in search_results:
title = search_result.text.encode('utf8')
link = search_result['href']
scrap_data.append(title,link)
我收到此消息错误:
TypeError Traceback (most recent call last)
<ipython-input-15-3341663b53f6> in <module>()
2 for search_result in search_results:
3 title = search_result.text.encode('utf8')
----> 4 link = search_result['href']
5 scrap_data.append(title,link)
TypeError: 'WebElement' object has no attribute '__getitem__'
请您参考如下方法:
你错了:
link = search_result['href']
可能你需要这样的.get_attribute:
link = search_result.get_attribute('href')
