Start with 100 free credits. No card needed. Get an API key

twscrape not working

twscrape is a pool of X accounts behind one API. When it fails, the cause is almost always an account in that pool, and the log line names it.

Three checks first

1. Update. X changes its web API often, and twscrape follows release by release.

Shell
python3 -m pip install -U twscrape
twscrape version

2. Look at the pool. Most failures show up there before they show up in your code. accounts lists every account with whether it is active and the last error; stats shows the locks per queue.

Shell
twscrape accounts
twscrape stats

3. Turn on the warnings. twscrape reports most problems as log lines, not exceptions, and a search that quietly returns nothing usually has a warning behind it.

These pages are written against twscrape 0.21.0 and quote its source.

Find your message

Log line or exceptionWhat happenedFix
No account available for queue ...Every account is locked or inactiveNo account
No active accounts. Stopping...The pool has no usable accountNo account
(88) Rate limit exceeded, or a lock until a reset timeThe account reached X's limit for that operationRate limit
Ban detected: ... (326) Authorization: Denied by access controlX locked or limited the accountAccess denied
Ban detected: ... (64) Your account is suspendedThe account is suspendedAccess denied
Session expired or banned: ... (32) Could not authenticate youThe cookies stopped workingCould not authenticate
Blocked by Cloudflare or Blocked by HTMLAn HTML error page instead of JSONBlocked
Failed to login 'USERNAME': ...The login flow failedLogin failed
Failed to get XClIdGen. See: .../issues/248Three 404s in a row on the transaction IDUpdate; see issue #248
[DEV] Update required: ... needs GQL_FEATURESX asked for new feature flagstwscrape patched it for now; update

What stops a run

twscrape sorts what goes wrong into three kinds, and the kind decides what your code sees.

  • Handled. A rate limit, a ban or an expired session locks or deactivates that one account, and the same request goes out again with another. Your code sees nothing but a slower result.
  • Aborted. An HTML block page, Failed to get XClIdGen and outdated feature flags raise AbortReqError (or GqlFeaturesOutdatedError), which stops the request and everything waiting behind it.
  • Unknown. Anything else gets a few retries, then the account is locked for 15 minutes with Unknown error. Account timeouted for 15 minutes. and the advice to unlock with twscrape reset_locks if it was a mistake.

The handled kind is why a broken pool can look like an empty result instead of an error. An X error that arrives with no data raises ApiError rather than returning nothing, so a failure is not mistaken for a search with no results.

The X reads that need no account pool

twscrape exists to run X's own web API through accounts you supply. SocialScrape reads X with an API key and no pool, and its X endpoints work differently from twscrape's, so check the limits in each row:

You use twscrape forSocialScrape endpointDifference
api.user_by_login()GET /v1/twitter/profileSame fields, no likes count
api.tweet_details()GET /v1/twitter/tweetViews can be missing
api.user_tweets()GET /v1/twitter/profile/tweetsAbout 20 posts, no further pages
api.search()GET /v1/twitter/search/postsFound through a web search engine, not X's search
Followers, likes, retweeters, listsNot offered

A profile, a post or a page is 1 credit, and a failed request is free. New accounts start with 100 credits.

Questions, answered.

Common questions

Why does twscrape return an empty list instead of an error?

Usually because no account could serve the request. With raise_when_no_account=True it raises NoAccountError instead; the no account page has the details.

Are twscrape accounts banned often?

The maintainer publishes no figure. In the issue thread on account survival, users reported losses ranging from a few accounts a day to most of a large pool.

Does twscrape need X's paid API?

No. It uses the same web API the X website uses, signed in with your accounts.

Which HTTP client does twscrape use?

curl, through curl-cffi, by default. TWS_HTTP_BACKEND=httpx switches to httpx.

Get started

Start pulling social data.

100 free credits. No card needed.