python之Unicode解码错误: 'utf8' codec can't decode byte 0xba in position 1266: invalid start byte

我正在尝试使用 scikit 训练一些文本数据。相同的代码在其他 PC 上使用时没有任何错误，但在我的系统上却出现错误:

File "/root/Desktop/karim/svn/questo-anso/v5/trials/classify/domain_detection_final/test_classifier_temp.py", line 130, in trainClassifier 
    X_train = self.vectorizer.fit_transform(self.data_train.data) 
  File "/root/Desktop/karim/software/scikit-learn-0.15.1/sklearn/feature_extraction/text.py", line 1270, in fit_transform 
    X = super(TfidfVectorizer, self).fit_transform(raw_documents) 
  File "/root/Desktop/karim/software/scikit-learn-0.15.1/sklearn/feature_extraction/text.py", line 808, in fit_transform 
    vocabulary, X = self._count_vocab(raw_documents, self.fixed_vocabulary) 
  File "/root/Desktop/karim/software/scikit-learn-0.15.1/sklearn/feature_extraction/text.py", line 741, in _count_vocab 
    for feature in analyze(doc): 
  File "/root/Desktop/karim/software/scikit-learn-0.15.1/sklearn/feature_extraction/text.py", line 233, in <lambda> 
    tokenize(preprocess(self.decode(doc))), stop_words) 
  File "/root/Desktop/karim/software/scikit-learn-0.15.1/sklearn/feature_extraction/text.py", line 111, in decode 
    doc = doc.decode(self.encoding, self.decode_error) 
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode 
    return codecs.utf_8_decode(input, errors, True) 
UnicodeDecodeError: 'utf8' codec can't decode byte 0xba in position 1266: invalid start byte 

我已经检查过类似的线程但没有帮助。

更新:

self.data_train = self.fetch_data(cache, subset='train') 
if not os.path.exists(self.root_dir+"/autocreated/vectorizer.txt"): 
                self.vectorizer = TfidfVectorizer(sublinear_tf=True, max_df=0.5, 
                                                 stop_words='english') 
                start_time = time() 
                print("Transforming the dataset") 
                X_train = self.vectorizer.fit_transform(self.data_train.data)  // Error is here 
                joblib.dump(self.vectorizer, self.root_dir+"/autocreated/vectorizer.txt") 

请您参考如下方法:

您的文件实际上是在 ISO-8869-1 中编码的，而不是 UTF-8。您需要对其进行正确解码，然后才能再次对其进行编码。

0xBA 是 numero sign (º) 在 ISO-8869-1 中。