Skip to main content
 首页 » 编程设计

python之如何使用 BeautifulSoup 从所有脚本中提取正确的脚本

2024年11月24日11itcoder

我目前正在使用 BS4 从 Kickstarter 网页提取一些信息:https://www.kickstarter.com/projects/louisalberry/louis-alberry-debut-album-uk-european-tour

项目信息位于其中一个脚本标签内:(伪代码)

... 
<script>...</script> 
<script> 
window.current_ip = ... 
... 
window.current_project = "<I want this part>" 
</script> 
... 

我当前的代码:

from bs4 import BeautifulSoup 
from urllib.request import urlopen 
import html 
 
html_ = urlopen("https://www.kickstarter.com/projects/louisalberry/louis-alberry-debut-album-uk-european-tour").read() 
soup = BeautifulSoup(html_, 'html.parser') 
# why does this not work? 
# soup.find('script', re.compile("window.current_project")) 
 
# currently, I'm doing this: 
all_string = html.unescape(soup.find_all('script')[4].get_text()) 
# then some regex here on all_string to extract the current_project information 

目前,我可以使用索引 [4] 获取我想要的部分,但由于我不确定这在一般情况下是否正确,我如何从正确的脚本标签中提取文本?

谢谢!

请您参考如下方法:

您可以收集所有脚本元素并循环。用请求访问响应对象内容

from bs4 import BeautifulSoup 
import requests 
res = requests.get("https://www.kickstarter.com/projects/louisalberry/louis-alberry-debut-album-uk-european-tour") 
soup = BeautifulSoup(res.content, 'lxml') 
scripts = soup.select('script') 
scripts = [script for script in scripts] 
for script in scripts: 
    if 'window.current_project' in script.text: 
        print(script)