Skip to main content
 首页 » 编程设计

python之PySpark:[Errno 8] 节点名或服务名已提供,或未知

2025年05月04日37lautakyan007

我正在使用预构建的“spark-2.0.1-bin-hadoop2.7”,当我尝试启动 pyspark 时,我收到以下消息。 任何想法可能是错的?我尝试使用 python3,将 SPARK_LOCAL_IP 设置为 127.0.0.1 但同样的错误。

~ -> cd /Applications/spark-2.0.1-bin-hadoop2.7/bin/ 
/Applications/spark-2.0.1-bin-hadoop2.7/bin -> pyspark 
Python 2.7.12 (default, Oct 11 2016, 05:24:00) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin 
Type "help", "copyright", "credits" or "license" for more information. 
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 
Setting default log level to "WARN". 
To adjust logging level use sc.setLogLevel(newLevel). 
16/12/19 14:50:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
16/12/19 14:50:47 WARN Utils: Your hostname, XXXXXX.com resolves to a loopback address: 127.0.0.1; using XX.XX.XX.XXX instead (on interface en0) 
16/12/19 14:50:47 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address 
Traceback (most recent call last): 
  File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/shell.py", line 43, in <module> 
    spark = SparkSession.builder\ 
  File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/sql/session.py", line 169, in getOrCreate 
    sc = SparkContext.getOrCreate(sparkConf) 
  File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/context.py", line 294, in getOrCreate 
    SparkContext(conf=conf or SparkConf()) 
  File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/context.py", line 115, in __init__ 
    conf, jsc, profiler_cls) 
  File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/context.py", line 174, in _do_init 
    self._accumulatorServer = accumulators._start_update_server() 
  File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/accumulators.py", line 259, in _start_update_server 
    server = AccumulatorServer(("localhost", 0), _UpdateRequestHandler) 
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 417, in __init__ 
    self.server_bind() 
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 431, in server_bind 
    self.socket.bind(self.server_address) 
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth 
    return getattr(self._sock,name)(*args) 
socket.gaierror: [Errno 8] nodename nor servname provided, or not known 

谢谢

请您参考如下方法:

找到了。一些我的主机映射是如何搞砸的。将其更改为指向本地主机有效。:

/etc/主机

#127.0.0.1 XXXXXX.com

127.0.0.1 本地主机