Skip to main content
 首页 » 编程设计

python之BeautifulSoup:找不到表

2025年02月15日13jillzhang

我正试图从 this page 中获取价格.

我需要这张表:

table class = "table table-condensed table-info" 

但是,当我打印内容并搜索表格时,却找不到:

from BeautifulSoup import BeautifulSoup 
import urllib2 
from bs4 import BeautifulSoup 
 
url = "https://www.predictit.org/Contract/4393/Will-Obama-pardon-Hillary-Clinton#openoffers"   
 
page = urllib2.urlopen(url).read() 
soup = BeautifulSoup(page) 
print soup 

任何帮助将不胜感激!

请您参考如下方法:

实际问题 - 价格通过一个单独的异步请求加载到不同的端点。你需要在你的代码中模拟它:

from bs4 import BeautifulSoup 
import requests 
 
url = "https://www.predictit.org/Contract/4393/Will-Obama-pardon-Hillary-Clinton#openoffers" 
price_url = "https://www.predictit.org/PrivateData/GetPriceListAjax?contractId=4393" 
 
with requests.Session() as session: 
    session.headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36'} 
    session.get(url)  # visit main page 
 
    # request prices 
    response = session.get(price_url) 
    soup = BeautifulSoup(response.content, "html.parser") 
    tables = soup.select("table.table-info") 
    for row in tables[0].select("tr")[2:]: 
        values = [td.find(text=True, recursive=False) for td in row('td') if td.text] 
        print(values) 

打印第一个"is"表的内容(用于演示目的):

[u'13', u'1555', u'12', u'240'] 
[u'14', u'707', u'11', u'2419'] 
[u'15', u'2109', u'10', u'3911'] 
[u'16', u'1079', u'9', u'2634'] 
[u'17', u'760', u'8', u'2596'] 
[u'18', u'510', u'7', u'970'] 
[u'19', u'973', u'6', u'1543'] 
[u'20', u'483', u'5', u'2151'] 
[u'21', u'884', u'4', u'1195'] 
[u'22', u'701', u'3', u'950'] 

请注意,我们在此处通过 requests.Session() 维护网络抓取 session 。

另请注意,price_url 包含 contractId GET 参数 - 如果您要请求包含价格的不同页面,请确保使用适当的 contractId.