python之Google App Engine之使用 URLFetch 的基本 Cookie 处理

专家，

似乎使用 Google App Engine 检索即使是最基本的网站也是相当具有挑战性的!

在我的例子中，我想通过这个 url 检索网站:

http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Index/9809

我想接受所有的 cookie，然后对这个 url 发布一个响应(这是一个简单的表单发布):

http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Search

我想发布的字符串是:

'POSTDATA': 'Q585=1&Q2926=1&Q586=200000&Q587=240000&Q588=&Q9166=07071&Q591=1&Q592=1&Q594=3&searchButton=Search' 

我遇到的问题是该网页在我的网络浏览器上显示“未启用 Cookie”。

正如您从下面的代码中看到的，我尝试手动添加 cookie，但这并不成功。

求助! -托德

import cgi 
import webapp2 
import gzip 
import StringIO 
 
from google.appengine.api import users 
from google.appengine.api import urlfetch 
from BeautifulSoup import BeautifulSoup 
 
class MainPage(webapp2.RequestHandler): 
    def get(self): 
#        self.response.headers['Content-Type'] = 'text/html' 
        url = "http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Index/9809" 
        result = urlfetch.fetch(url, 
                                headers={'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3', 
                                                 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 
                                                 'Accept-Language': 'en-us', 
                                                 'Accept-Encoding': 'gzip', 
                                                 'Connection': 'keep-alive'}) 
 
        cookie = result.headers.get('set-cookie') 
        input_text = {'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3', 
                      'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 
                      'Accept-Language': 'en-us', 
                      'Accept-Encoding': 'gzip', 
                      'Connection': 'keep-alive', 
                      'Content-Type': 'application/x-www-form-urlencoded', 
                      'Content-Length': '97', 
                      'POSTDATA': 'Q585=1&Q2926=1&Q586=200000&Q587=240000&Q588=&Q9166=07071&Q591=1&Q592=1&Q594=3&searchButton=Search', 
                      'SiteProfile':'ProfileId=9809', 
                      's_sess':'c_m=undefinedfeedity.comfeedity.com; s_sq=; s_cc=true;;', 
                      'bhCookieSaveSess':'1', 
                      'bhPrevResults':'bhjs=1&bhrf=http://www.google.com/'} 
 
        input_text['set-cookie'] = cookie 
##        self.response.out.write(input_text) 
 
        url2 = "http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Search" 
        result2 = urlfetch.fetch(url2, method='POST',headers=input_text) 
 
#        self.response.out.write(result.headers) 
        f = StringIO.StringIO(result2.content) 
        c = gzip.GzipFile(fileobj=f) 
        content = c.read() 
        self.response.out.write(content) 
#        self.response.out.write(result.content) 
 
 
app = webapp2.WSGIApplication([('/', MainPage)], 
                              debug=True) 

yaml文件:

application: fimrates 
version: 1 
runtime: python27 
api_version: 1 
threadsafe: true 
 
handlers: 
- url: /.* 
  script: fimrates.app 

请您参考如下方法:

您实际上并没有用您的代码设置 cookie。这是您的代码的修改版本。

#import this module to be able to create a cookie 
import Cookie 
 
class BankHandler(webapp2.RequestHandler): 
 
    #put this function to create the cookie header 
    def createCookieHeader(self, cookie): 
        cookieHeader = "" 
        for value in cookie.values(): 
            cookieHeader += "%s=%s; " % (value.key, value.value) 
        return cookieHeader 
 
    def get(self): 
        ... 
        self.cookie = Cookie.SimpleCookie() #create the cookie 
 
        result = urlfetch.fetch({..., 
                       #inject the cookie header                                                  
                      'Cookie': self.createCookieHeader(self.cookie)}) 
 
        cookie = result.headers.get('set-cookie', '') 
        input_text = {...} 
 
        #the header is 'Cookie'  
        input_text['Cookie'] = cookie 
 
        url2 = "http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Search" 
        result2 = urlfetch.fetch(url2, method='POST',headers=input_text) 
        ...