我有以下格式的数据:
Bxxxx, Mxxxx F Birmingham AL (123) 555-2281 NCC Clinical Mental Health, Counselor Education, Sexual Abuse Recovery, Depression/Grief/Chronically or Terminally Ill, Mental Health/Agency Counseling English 99.52029 -99.8115
Axxxx, Axxxx Brown Birmingham AL (123) 555-2281 NCC Clinical Mental Health, Depression/Grief/Chronically or Terminally Ill, Mental Health/Agency Counseling English 99.52029 -99.8115
Axxxx, Bxxxx Mobile AL (123) 555-8011 NCC Childhood & Adolescence, Clinical Mental Health, Sexual Abuse Recovery, Disaster Counseling English 99.68639 -99.053238
Axxxx, Rxxxx Lunsford Athens AL (123) 555-8119 NCC, NCCC, NCSC Career Development, Childhood & Adolescence, School, Disaster Counseling, Supervision English 99.804501 -99.971283
Axxxx, Mxxxx Mobile AL (123) 555-5963 NCC Clinical Mental Health, Counselor Education, Depression/Grief/Chronically or Terminally Ill, Mental Health/Agency Counseling, Supervision English 99.68639 -99.053238
Axxxx, Txxxx Mountain Brook AL (123) 555-3099 NCC Addictions and Dependency, Career Development, Childhood & Adolescence, Corrections/Offenders, Sexual Abuse Recovery English 99.50214 -99.75557
Axxxx, Lxxxx Birmingham AL (123) 555-4550 NCC Addictions and Dependency, Eating Disorders English 99.52029 -99.8115
Axxxx, Wxxxx Birmingham AL (123) 555-2328 NCC English 99.52029 -99.8115
Axxxx, Rxxxx Mobile AL (123) 555-9411 NCC Addictions and Dependency, Childhood & Adolescence, Couples & Family, Sexual Abuse Recovery, Depression/Grief/Chronically or Terminally Ill English 99.68639 -99.053238
并且只需要提取人名。理想情况下,我将能够使用 humanName 获取一堆具有字段 name.first、name.middle、name.last 的名称对象>, name.title...
我已经尝试遍历,直到我击中前两个连续的大写字母代表状态,然后将前面的内容存储到列表中,然后调用 humanName 但这是一场灾难。我不想继续尝试这种方法。
有没有办法感知单词的开头和结尾?这可能会有帮助...
建议?
请您参考如下方法:
最好的办法是找到不同的数据源。严重地。这个是 farked。
如果你做不到,那么我会做一些这样的工作:
- 用单空格替换所有双空格。
- 用空格分隔行
- 取列表中的最后两项。那些是纬度和经度
- 在列表中向后循环,在潜在语言列表中查找每个项目。如果查找失败,您就完成了语言。
- 用空格连接剩余的列表项
- 在该行中,找到第一个左括号。阅读大约 13 或 14 个字符,将所有标点符号替换为空字符串,并将其重新格式化为普通电话号码。
- 在电话号码后用逗号分隔该行的剩余部分。
- 使用该拆分循环遍历列表中的每个项目。如果文本以超过 1 个大写字母开头,请将其添加到认证中。否则,将其添加到实践领域。
- 回到您在第 6 步中找到的索引,直到那时才排队。将其拆分为空格,然后取最后一项。那就是状态。剩下的就是名字和城市!
- 取空格分隔线中的前 2 项。到目前为止,这是您对名字的最佳猜测。
- 看第 3 项。如果是单个字母,则将其添加到名称中并从列表中删除。
- 从这里下载 US.zip:http://download.geonames.org/export/zip/US.zip
- 在美国数据文件中,将所有文件拆分到选项卡上。取索引2和4处的数据,分别是城市名和州名缩写。循环遍历所有数据并将每一行插入新列表,将其连接为缩写 +“:”+ 城市名称(即 AK:Sand Point)。
- 以与第 13 步中相同的格式,将行中剩余项目的所有可能连接组合起来。所以你最终会在第二行找到 AL:Brown Birmingham 和 AL:Birmingham。
- 遍历每个组合并在您在第 13 步中创建的列表中搜索它。如果找到它,请将其从拆分列表中删除。
- 将字符串拆分列表中的所有剩余项目添加到此人的姓名中。
- 如果需要,用逗号分隔名称。 index[0] 是姓氏 index[1] 是所有剩余的名字。不要对中间名做出任何假设。
为了好玩,我实现了这个。享受吧。
import itertools
# this list of languages could be longer and should read from a file
languages = ["English", "Spanish", "Italian", "Japanese", "French",
"Standard Chinese", "Chinese", "Hindi", "Standard Arabic", "Russian"]
languages = [language.lower() for language in languages]
# Loop through US.txt and format it. Download from geonames.org.
cities = []
with open('US.txt', 'r') as us_data:
for line in us_data:
line_split = line.split("\t")
cities.append("{}:{}".format(line_split[4], line_split[2]))
# This is the dataset
with open('state-teachers.txt', 'r') as teachers:
next(teachers) # skip header
for line in teachers:
# Replace all double spaces with single spaces
while line.find(" ") != -1:
line = line.replace(" ", " ")
line_split = line.split(" ")
# Lat/Lon are the last 2 items
longitude = line_split.pop().strip()
latitude = line_split.pop().strip()
# Search for potential languages and trim off the line as we find them
teacher_languages = []
while True:
language_check = line_split[-1]
if language_check.lower().replace(",", "").strip() in languages:
teacher_languages.append(language_check)
del line_split[-1]
else:
break
# Rejoin everything and then use phone number as the special key to split on
line = " ".join(line_split)
phone_start = line.find("(")
phone = line[phone_start:phone_start+14].strip()
after_phone = line[phone_start+15:]
# Certifications can be recognized as acronyms
# Anything else is assumed to be an area of practice
certifications = []
areas_of_practice = []
specialties = after_phone.split(",")
for specialty in specialties:
specialty = specialty.strip()
if specialty[0:2].upper() == specialty[0:2]:
certifications.append(specialty)
else:
areas_of_practice.append(specialty)
before_phone = line[0:phone_start-1]
line_split = before_phone.split(" ")
# State is the last column before phone
state = line_split.pop()
# Name should be the first 2 columns, at least. This is a basic guess.
name = line_split[0] + " " + line_split[1]
line_split = line_split[2:]
# Add initials
if len(line_split[0].strip()) == 1:
name += " " + line_split[0].strip()
line_split = line_split[1:]
# Combo of all potential word combinations to see if we're dealing with a city or a name
combos = [" ".join(combo) for combo in set(itertools.permutations(line_split))] + line_split
line = " ".join(line_split)
city = ""
# See if the state:city combo is valid. If so, set it and let everything else be the name
for combo in combos:
if "{}:{}".format(state, combo) in cities:
city = combo
line = line.replace(combo, "")
break
# Remaining data must be a name
if line.strip() != "":
name += " " + line
# Clean up names
last_name, first_name = [piece.strip() for piece in name.split(",")]
print first_name, last_name
