我想下载这个ONS webpage上的Excel文件在 Python 中使用 MechanicalSoup 包。我读过 MechanicalSoup documentation .我在 StackOverflow 和其他地方广泛搜索了一个示例,但没有成功。
我的尝试是:
# Install dependencies
# pip install requests
# pip install BeautifulSoup4
# pip install MechanicalSoup
# Import libraries
import mechanicalsoup
import urllib.request
import requests
from bs4 import BeautifulSoup
# Create a browser object that can collect cookies
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://www.ons.gov.uk/economy/grossdomesticproductgdp/timeseries/l2kq/qna")
browser.download_link("https://www.ons.gov.uk/generator?format=xls&uri=/economy/grossdomesticproductgdp/timeseries/l2kq/qna")
在最后一行,我也尝试过:
browser.download_link(link="https://www.ons.gov.uk/generator?format=xls&uri=/economy/grossdomesticproductgdp/timeseries/l2kq/qna",file="c:/test/filename.xls")
2019 年 1 月 25 日更新:感谢 AKX 在下方的评论,我已尝试过
browser.download_link(re.escape("https://www.ons.gov.uk/generator?format=xls&uri=/economy/grossdomesticproductgdp/timeseries/l2kq/qna"))
在每种情况下,我都会收到错误:
mechanicalsoup.utils.LinkNotFoundError
但链接确实存在。尝试将其粘贴到您的地址栏以确认:
https://www.ons.gov.uk/generator?format=xls&uri=/economy/grossdomesticproductgdp/timeseries/l2kq/qna
我做错了什么?
2019 年 1 月 25 日更新 2:感谢 AKX 在下面的回答,这是回答我问题的完整 MWE(发布给以后遇到同样困难的人) :
# Install dependencies
# pip install requests
# pip install BeautifulSoup4
# pip install MechanicalSoup
# Import libraries
import mechanicalsoup
import urllib.request
import requests
from bs4 import BeautifulSoup
import re
# Create a browser object that can collect cookies
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://www.ons.gov.uk/economy/grossdomesticproductgdp/timeseries/l2kq/qna")
browser.download_link(link_text=".xls",file="c:/py/ONS_Data.xls" )
请您参考如下方法:
我没用过 Mechanical Soup,但是看了文档,
This function behaves similarly to follow_link()
和follow_link说(强调我的)
- If link is a bs4.element.Tag (i.e. from a previous call to links() or find_link()), then follow the link.
- If link doesn’t have a href-attribute or is None, treat link as a url_regex and look it up with find_link(). Any additional arguments specified are forwarded to this function.
问号(除其他外)是正则表达式 (regex) 元字符,因此如果您想将它们用于 follow_link/download_link,则需要对它们进行转义:
import re
# ...
browser.download_link(re.escape("https://www.ons.gov.uk/generator?format=xls&uri=/economy/grossdomesticproductgdp/timeseries/l2kq/qna"))
但是,如果您访问的第一个页面不包含该直接链接,我不确定它是否有帮助。 (尽管先尝试。)
您可以使用浏览器的底层 requests session 来直接下载文件:
resp = browser.session.get("https://www.ons.gov.uk/generator?format=xls&uri=/economy/grossdomesticproductgdp/timeseries/l2kq/qna")
resp.raise_for_status() # raise an exception for 404, etc.
with open('filename.xls', 'wb') as outf:
outf.write(resp.content)
