twscrape not working
twscrape is a pool of X accounts behind one API. When it fails, the cause is almost always an account in that pool, and the log line names it.
Three checks first
1. Update. X changes its web API often, and twscrape follows release by release.
python3 -m pip install -U twscrape twscrape version
2. Look at the pool. Most failures show up there before they show up in your code. accounts lists every account with whether it is active and the last error; stats shows the locks per queue.
twscrape accounts twscrape stats
3. Turn on the warnings. twscrape reports most problems as log lines, not exceptions, and a search that quietly returns nothing usually has a warning behind it.
These pages are written against twscrape 0.21.0 and quote its source.
Find your message
| Log line or exception | What happened | Fix |
|---|---|---|
No account available for queue ... | Every account is locked or inactive | No account |
No active accounts. Stopping... | The pool has no usable account | No account |
(88) Rate limit exceeded, or a lock until a reset time | The account reached X's limit for that operation | Rate limit |
Ban detected: ... (326) Authorization: Denied by access control | X locked or limited the account | Access denied |
Ban detected: ... (64) Your account is suspended | The account is suspended | Access denied |
Session expired or banned: ... (32) Could not authenticate you | The cookies stopped working | Could not authenticate |
Blocked by Cloudflare or Blocked by HTML | An HTML error page instead of JSON | Blocked |
Failed to login 'USERNAME': ... | The login flow failed | Login failed |
Failed to get XClIdGen. See: .../issues/248 | Three 404s in a row on the transaction ID | Update; see issue #248 |
[DEV] Update required: ... needs GQL_FEATURES | X asked for new feature flags | twscrape patched it for now; update |
What stops a run
twscrape sorts what goes wrong into three kinds, and the kind decides what your code sees.
- Handled. A rate limit, a ban or an expired session locks or deactivates that one account, and the same request goes out again with another. Your code sees nothing but a slower result.
- Aborted. An HTML block page,
Failed to get XClIdGenand outdated feature flags raiseAbortReqError(orGqlFeaturesOutdatedError), which stops the request and everything waiting behind it. - Unknown. Anything else gets a few retries, then the account is locked for 15 minutes with
Unknown error. Account timeouted for 15 minutes.and the advice to unlock withtwscrape reset_locksif it was a mistake.
The handled kind is why a broken pool can look like an empty result instead of an error. An X error that arrives with no data raises ApiError rather than returning nothing, so a failure is not mistaken for a search with no results.
The X reads that need no account pool
twscrape exists to run X's own web API through accounts you supply. SocialScrape reads X with an API key and no pool, and its X endpoints work differently from twscrape's, so check the limits in each row:
| You use twscrape for | SocialScrape endpoint | Difference |
|---|---|---|
api.user_by_login() | GET /v1/twitter/profile | Same fields, no likes count |
api.tweet_details() | GET /v1/twitter/tweet | Views can be missing |
api.user_tweets() | GET /v1/twitter/profile/tweets | About 20 posts, no further pages |
api.search() | GET /v1/twitter/search/posts | Found through a web search engine, not X's search |
| Followers, likes, retweeters, lists | Not offered |
A profile, a post or a page is 1 credit, and a failed request is free. New accounts start with 100 credits.
Questions, answered.
Common questions
Why does twscrape return an empty list instead of an error?
Usually because no account could serve the request. With raise_when_no_account=True it raises NoAccountError instead; the no account page has the details.
Are twscrape accounts banned often?
The maintainer publishes no figure. In the issue thread on account survival, users reported losses ranging from a few accounts a day to most of a large pool.
Does twscrape need X's paid API?
No. It uses the same web API the X website uses, signed in with your accounts.
Which HTTP client does twscrape use?
curl, through curl-cffi, by default. TWS_HTTP_BACKEND=httpx switches to httpx.
Start pulling social data.
100 free credits. No card needed.