Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Jun 9, 2011

• Simplified Request/Response __repr__ to be the same as __str__. This improves legibility and shouldn't affect any functionality, since we never use __repr__ for reconstructing a response AFAIK. Also …

  Show description for 88e33ad
  pablohoffman

  committed
  88e33ad

  Copy full SHA for 88e33ad

Commits on Jun 8, 2011

• scrapyd.webservice: use twisted.web multipart data parsing, to simplify code. closes scrapy#324

  pablohoffman

  committed
  07df0ed

  Copy full SHA for 07df0ed

Commits on Jun 6, 2011

• fixed bug handling truncated gzipped responses. closes scrapy#319

  pablohoffman

  committed
  7643f14

  Copy full SHA for 7643f14

• fixed some tests accidentally broken in previous commit

  pablohoffman

  committed
  48509b0

  Copy full SHA for 48509b0

• make --headers output of fetch command resemble curl format, and also show request headers

  pablohoffman

  committed
  f793515

  Copy full SHA for f793515

• Scheduler refactoring which introduces the following changes:

  Show description for 0375174
  pablohoffman

  committed
  0375174

  Copy full SHA for 0375174

• simplified MemoryDebugger extension to use stats for dumping memory debugging info

  pablohoffman

  committed
  474cba5

  Copy full SHA for 474cba5

• call stats collector engine_stopped() after the engine is closed (to make sure all data from extensions has been collected), and added that method to documented api

  pablohoffman

  committed
  5fbc32c

  Copy full SHA for 5fbc32c

• removed deprecated stat 'envinfo/request_depth_limit'. we should instead support dumping settings, for these cases

  pablohoffman

  committed
  35b52fc

  Copy full SHA for 35b52fc

• added 'scrapy edit' command

  pablohoffman

  committed
  9d9c887

  Copy full SHA for 9d9c887

Commits on Jun 5, 2011

• simplified DownloaderStats middleware

  pablohoffman

  committed
  ffbc929

  Copy full SHA for ffbc929

• simplified CoreStats extension

  pablohoffman

  committed
  3d823d6

  Copy full SHA for 3d823d6

Commits on Jun 3, 2011

• removed crawlspider v2 tests

  pablohoffman

  committed
  61cc95d

  Copy full SHA for 61cc95d

• removed experimental crawlspider v2

  pablohoffman

  committed
  03ae481

  Copy full SHA for 03ae481

• Changed default representation of items to pretty-printed dicts. This improves

  Show description for 5bf733b
  pablohoffman

  committed
  5bf733b

  Copy full SHA for 5bf733b

• Merged item passed and item scraped concepts, as they have often proved

  Show description for 1bc2339
  pablohoffman

  committed
  1bc2339

  Copy full SHA for 1bc2339

Commits on May 30, 2011

• fixed doc typo

  pablohoffman

  committed
  e6091df

  Copy full SHA for e6091df

• added spider_error signal

  pablohoffman

  committed
  1d98fc8

  Copy full SHA for 1d98fc8

Commits on May 27, 2011

• removed undocumented (and untested) extension: SpiderCloseDelay

  pablohoffman

  committed
  13d8066

  Copy full SHA for 13d8066

• removed support for spider.dont_throttle attribute

  pablohoffman

  committed
  6c369c5

  Copy full SHA for 6c369c5

• added COOKIES_ENABLED setting to support disabling the cookies middleware

  pablohoffman

  committed
  2fa0f75

  Copy full SHA for 2fa0f75

• register AutoThrottle extension by default, and made AUTOTHROTTLE_ENABLED disabled by default

  pablohoffman

  committed
  756bf0c

  Copy full SHA for 756bf0c

Commits on May 22, 2011

• added setting: AUTOTHROTTLE_ENABLED

  pablohoffman

  committed
  dcc28b7

  Copy full SHA for dcc28b7

• added Spider.dont_throttle attribute to disable AutoThrottle extension per spider

  pablohoffman

  committed
  110cd05

  Copy full SHA for 110cd05

Commits on May 20, 2011

• fix error messages due to fetching pages during shutdown process

  Show description for 88dbe2a
  shaneaevans

  committed
  88dbe2a

  Copy full SHA for 88dbe2a

• fixed stupid bug in scheduler introduced in previous change

  pablohoffman

  committed
  3897e33

  Copy full SHA for 3897e33

• removed unused imports

  pablohoffman

  committed
  70b0e42

  Copy full SHA for 70b0e42

• stack trace dump extension: also dump engine status, and support triggering it with SIGQUIT, besides SIGUSR2

  pablohoffman

  committed
  d72d3f4

  Copy full SHA for d72d3f4

• Fixed 100% cpu loop that ocurred in some cases where Scrapy was shutting donw

  pablohoffman

  committed
  6069b0e

  Copy full SHA for 6069b0e

• Removed support for default values in Scrapy items, which have proven confusing in the past

  pablohoffman

  committed
  951ba50

  Copy full SHA for 951ba50

Commits on May 18, 2011

• removed remaining references to scheduler middleware from doc, as it will be removed on next release

  pablohoffman

  committed
  503f302

  Copy full SHA for 503f302

• fixed outdated documentation

  pablohoffman

  committed
  3fd1743

  Copy full SHA for 3fd1743

• added role to link to scrapy source code (not yet used)

  pablohoffman

  committed
  9016e7e

  Copy full SHA for 9016e7e

• minor fix to spider closed count stat

  pablohoffman

  committed
  a98e9e0

  Copy full SHA for a98e9e0

• Some Link extractor improvements:

  Show description for cd85c12
  pablohoffman

  committed
  cd85c12

  Copy full SHA for cd85c12

Pagination

Previous

Next