New pagination bug in GitHub GraphQL: hasNextPage cannot be trusted

[simonw]

I just started seeing a weird bug in pagination in the GitHub GraphQL API. Consider the following query:

{
  viewer {
    repositories(first: 10, privacy: PUBLIC) {
      totalCount
      pageInfo {
        hasNextPage
        endCursor
      }
      nodes {
        name
      }
    }
  }
}

Running this against https://docs.github.com/en/graphql/overview/explorer for me returns:

{
  "data": {
    "viewer": {
      "repositories": {
        "totalCount": 568,
        "pageInfo": {
          "hasNextPage": false,
          "endCursor": "Y3Vyc29yOnYyOpHOAALtCQ=="
        },
        "nodes": [
          {
            "name": "django-debug-toolbar"
          },
          {
            "name": "simonw.github.com"
          },
          {
            "name": "ratelimitcache"
          },
          {
            "name": "api-explorer"
          },
          {
            "name": "python-guardianapi"
          },
          {
            "name": "tweetersation"
          },
          {
            "name": "qypeexplorer"
          },
          {
            "name": "getlatlon.com"
          },
          {
            "name": "django-shorturls"
          },
          {
            "name": "soupselect"
          }
        ]
      }
    }
  }
}

Note that it shows a totalCount of 568 - which is correct - but despite only listing the first 10 records it returns "hasNextPage": false.

This should be "hasNextPage": true.

This was working fine a few hours ago - I spotted the bug because my Actions workflow that updates my personal profile on https://github.com/simonw suddenly deleted a bunch of my project listings: simonw/simonw@1ed4132

More notes here:

• Many of my projects are now missing  simonw/simonw#6

[simonw]

I found a workaround: don't trust the hasNextPage boolean and instead check to see if endCursor is null in order to terminate pagination.

That seems to have fixed the bug I was seeing.

[clayne]

Yep - @Roohn is right. The search query only returns a maximum of 1,000 results, even if there are more matches. I'm going to update the docs to clarify that.

Where is RepositorySearch even documented? I was under the impression it was REST only as I couldn't find a similar query in the GQL docs at the time.

[timrogers]

We have a search query which allows you to search repos, amongst other things: https://docs.github.com/en/graphql/reference/queries#search

[clayne]

We have a search query which allows you to search repos, amongst other things: https://docs.github.com/en/graphql/reference/queries#search

Ah yes, sorry my bad - I see now the actual search() query. However, this is still at a disadvantage to the REST based repo search as one cannot specify file or language types, etc. right?

[timrogers]

I think should be able to put that kind of thing in the query - but depends exactly what you’re trying to do, I guess!

[rodgomes]

Good catch @Roohn and thanks for clarifying @timrogers, I was not aware of the search result limitations.

[timrogers]

@simonw Thanks for taking the time to report this bug! It was fixed about a week ago, but I forgot to jump back to this thread and let you know 😓