python之在 Django 应用程序中线程化

我正在尝试创建我称之为 scrobbler 的东西。任务是从队列中读取一个 Delicious 用户，获取他们所有的书签并将它们放入书签队列中。然后应该通过该队列，进行一些解析，然后将数据存储在数据库中。

这显然需要线程，因为大部分时间都花在等待 Delicious 响应，然后等待已添加书签的网站响应并通过某些 API 传递，如果一切都等待它是愚蠢的。

但是，我在处理线程时遇到了问题，并不断收到奇怪的错误，例如未定义数据库表。任何帮助表示赞赏:)

相关代码如下:

# relevant model # 
class Bookmark(models.Model): 
 account = models.ForeignKey( Delicious ) 
 url = models.CharField( max_length=4096 ) 
 tags = models.TextField() 
 hash = models.CharField( max_length=32 ) 
 meta = models.CharField( max_length=32 ) 
 
# bookmark queue reading # 
def scrobble_bookmark(account): 
 try: 
  bookmark = Bookmark.objects.all()[0] 
 except Bookmark.DoesNotExist: 
  return False 
 
 bookmark.delete() 
 
 tags = bookmark.tags.split(' ') 
 user = bookmark.account.user 
 
 for concept in Concepts.extract( bookmark.url ): 
  for tag in tags: 
   Concepts.relate( user, concept['name'], tag ) 
 
 return True 
 
def scrobble_bookmarks(account): 
 semaphore = Semaphore(10) 
 for i in xrange(Bookmark.objects.count()): 
  thread = Bookmark_scrobble(account, semaphore) 
  thread.start() 
 
class Bookmark_scrobble(Thread): 
 def __init__(self, account, semaphore): 
  Thread.__init__(self) 
  self.account = account 
  self.semaphore = semaphore 
 
 def run(self): 
  self.semaphore.acquire() 
  try: 
   scrobble_bookmark(self.account) 
  finally: 
   self.semaphore.release() 

这是我得到的错误:

Exception in thread Thread-65: 
Traceback (most recent call last): 
  File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner 
    self.run() 
  File     "/home/swizec/Documents/trees/bookmarklet_server/../bookmarklet_server/Scrobbler/Scrobbler.py", line 60, in run 
    scrobble_bookmark(self.account) 
  File     "/home/swizec/Documents/trees/bookmarklet_server/../bookmarklet_server/Scrobbler/Scrobbler.py", line 28, in scrobble_bookmark 
    bookmark = Bookmark.objects.all()[0] 
  File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 152, in __getitem__ 
    return list(qs)[0] 
  File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 76, in __len__ 
self._result_cache.extend(list(self._iter)) 
  File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 231, in iterator 
    for row in self.query.results_iter(): 
  File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 281, in results_iter 
    for rows in self.execute_sql(MULTI): 
  File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 2373, in execute_sql 
cursor.execute(sql, params) 
  File "/usr/local/lib/python2.6/dist-packages/django/db/backends/sqlite3/base.py", line 193, in execute 
    return Database.Cursor.execute(self, query, params) 
OperationalError: no such table: Scrobbler_bookmark 

PS:依赖于同一张表的所有其他测试都以优异的成绩通过。

请您参考如下方法:

您不能在 Django 的内存数据库(本例中为 sqlite3)中使用线程，请参阅 bug .这可能适用于 PostgreSQL 或 MySQL。

我建议使用 celeryd 之类的东西而不是线程，消息队列比线程更容易使用。