Skip to main content
 首页 » 编程设计

python之Twisted webapp 在生成 HTTP 响应时卡住

2024年10月25日30cmt

我使用 Twisted 和 SQLAlchemy 创建了一个 Web 应用程序。由于 SQLAlchemy 与 Twisted 基于回调的设计 ( Twisted + SQLAlchemy and the best way to do it ) 不能很好地协同工作,因此我在根资源中使用 deferToThread() 以便在其自己的线程中运行每个请求。虽然这通常有效,但大约 10% 的请求会“卡住”。这意味着当我在浏览器中单击一个链接时,请求由 Twisted 处理,相应资源的代码运行并生成 HTML 输出。但无论出于何种原因,该输出永远不会发送回浏览器。相反,Twisted 发送 HTTP header (连同正确的 Content-Length),但从不发送正文。连接会无限期地保持打开状态,浏览器会显示微调器图标。 Twisted 在日志文件中没有生成任何错误。

下面是一个最小的例子。如果您想运行它,请将其保存为 .tac 扩展名,然后运行 ​​twistd -noy example.tac。在我的服务器上,这个问题似乎很少出现在这段特定的代码中。使用类似 while true;做 wget -O- 'http://server.example.com:8080' >/dev/null;完成来测试它。

from twisted.web.server import Site 
from twisted.application import service, internet 
from twisted.web.resource import Resource 
from twisted.internet import threads 
from twisted.web.server import NOT_DONE_YET 
from sqlalchemy.ext.declarative import declarative_base 
from sqlalchemy.orm import sessionmaker 
from sqlalchemy import create_engine, Column, Integer, String 
 
Base = declarative_base() 
class User(Base): 
    '''A user account.''' 
    __tablename__ = 'user' 
    id = Column(Integer, primary_key=True) 
    login = Column(String(64)) 
 
 
class WebInterface(Resource): 
 
    def __init__(self): 
        Resource.__init__(self) 
        db_url = "mysql://user:password@mysql-server.example.com/myapp?charset=utf8" 
        db_engine = create_engine(db_url, echo=False, pool_recycle=300) #discard connections after 300 seconds 
        self.DBSession = sessionmaker(bind=db_engine) 
 
    def on_request_done(self, _, request): 
        '''All actions that need to be done after a request has been successfully handled.''' 
        request.db_session.close() 
        print('Session closed') #does get printed, so session should get closed properly 
 
 
    def on_request_failed(self, err, call): 
        '''What happens if the request failed on a network level, for example because the user aborted the request''' 
        call.cancel() 
 
 
    def on_error(self, err, request): 
        '''What happens if an exception occurs during processing of the request''' 
        request.setResponseCode(500) 
        self.on_request_done(None, request) 
        request.finish() 
        return err  
 
 
    def getChild(self, name, request): 
        '''We dispatch all requests to ourselves in order to be able to do the processing in separate threads''' 
        return self 
 
 
    def render(self, request): 
        '''Dispatch the real work to a thread''' 
        d = threads.deferToThread(self.do_work, request) 
        d.addCallbacks(self.on_request_done, errback=self.on_error, callbackArgs=[request], errbackArgs=[request]) 
        #If the client aborts the request, we need to cancel it to avoid error messages from twisted 
        request.notifyFinish().addErrback(self.on_request_failed, d) 
        return NOT_DONE_YET 
 
 
    def do_work(self, request): 
        '''This method runs in thread context.''' 
        db_session = self.DBSession() 
        request.db_session = db_session 
        user = db_session.query(User).first() 
 
        body = 'Hello, {} '.format(user.login) * 1024 #generate some output data 
        request.write(body) 
        request.finish() 
 
 
application = service.Application("My Testapp") 
s = internet.TCPServer(8080, Site(WebInterface()), interface='0.0.0.0') 
s.setServiceParent(application) 

请您参考如下方法:

您可能没有使用 SQLAlchemy 关闭数据库连接或数据库中的某些死锁情况?我之前因未关闭连接/未结束交易而被 flask 锁定。