我有一个包含许多列的 Pandas 数据框,下面是其中的一个子集:
df.info()
SQLDATE datetime64[ns]
SOURCEURL object
df['SQLDATE', 'SOURCEURL'].sample()
SQLDATE SOURCEURL
2017-01-08 http://www.huffingtonpost.co.uk/a/abc
2018-09-25 http://www.taiwannews.com.tw/a/news/123
2016-03-19 https://www.theguardian.com/a/2016/a/1/ab-bc
2015-12-12 https://nz.news.yahoo.com/world/a/3/a/
2017-04-07 https://www.thelocal.fr/2122/jkl
2019-02-21 http://today.az/news/a/123.html
2018-05-13 The BBC World News Report
我想创建一个可以提取域名的列,以便获得如下所示的新列:
df.sample()
SQLDATE SOURCEURL DOMAINNAME
2017-01-08 http://www.huffingtonpost.co.uk/a/abc www.huffingtonpost.co.uk
2018-09-25 http://www.taiwannews.com.tw/a/news/123 www.taiwannews.com.tw
2016-03-19 https://www.theguardian.com/a... www.theguardian.com
2015-12-12 https://nz.news.yahoo.com/world/a/3/a/ nz.news.yahoo.com
2017-04-07 https://www.thelocal.fr/2122/jkl www.thelocal.fr
2019-02-21 http://today.az/news/a/123.html today.az
2018-05-13 The BBC World News Report The BBC World News Report
数据框确实看起来很乱,其中一些 SOURCEURL 字段只包含文本,没有 URL。我只想将这些值复制到 DOMAINNAME 列中。我不太熟悉正则表达式,但这可能适用于这种情况。
感谢审阅!
请您参考如下方法:
我们可以使用积极的回顾 ?<=和正面前瞻?=使用正则表达式,获取 http:// 之间的所有内容或者 https://第一个/ :
m = df['SOURCEURL'].str.extract('(?<=http://)(.*?)(?=/)|(?<=https://)(.*?)(?=/)')
m = m[0].fillna(m[1]).fillna(df['SOURCEURL'])
df['DOMAINNAME'] = m
SQLDATE SOURCEURL DOMAINNAME
0 2017-01-08 http://www.huffingtonpost.co.uk/a/abc www.huffingtonpost.co.uk
1 2018-09-25 http://www.taiwannews.com.tw/a/news/123 www.taiwannews.com.tw
2 2016-03-19 https://www.theguardian.com/a/2016/a/1/ab-bc www.theguardian.com
3 2015-12-12 https://nz.news.yahoo.com/world/a/3/a/ nz.news.yahoo.com
4 2017-04-07 https://www.thelocal.fr/2122/jkl www.thelocal.fr
5 2019-02-21 http://today.az/news/a/123.html today.az
6 2018-05-13 The BBC World News Report The BBC World News Report
编辑:添加 mrzasa's answer , 我们也可以使用 apply与 urllib.parse.urlparse :
from urllib.parse import urlparse
df["DOMAIN"] = df["SOURCEURL"].apply(lambda row: urlparse(row).netloc)
SQLDATE SOURCEURL \
0 2017-01-08 http://www.huffingtonpost.co.uk/a/abc
1 2018-09-25 http://www.taiwannews.com.tw/a/news/123
2 2016-03-19 https://www.theguardian.com/a/2016/a/1/ab-bc
3 2015-12-12 https://nz.news.yahoo.com/world/a/3/a/
4 2017-04-07 https://www.thelocal.fr/2122/jkl
5 2019-02-21 http://today.az/news/a/123.html
6 2018-05-13 The BBC World News Report
DOMAIN
0 www.huffingtonpost.co.uk
1 www.taiwannews.com.tw
2 www.theguardian.com
3 nz.news.yahoo.com
4 www.thelocal.fr
5 today.az
6
