Rate Limits and File Uploads in GitHub Models

[solitude-alive]

Select Topic Area

Question

Body

Hi,

I'm currently using GitHub Models, but I have some confusion regarding rate limits. Taking 4o-mini as an example, according to GitHub's rate limits documentation, it states that the tokens per request are 8000 in and 4000 out. However, 4o-mini clearly has a larger context length, as seen on the GitHub Marketplace page for 4o-mini, where the context is listed as 131k input and 4k output.

I would like to understand why there is such a discrepancy, and whether this means I cannot request a query longer than 8k tokens.

Additionally, I would like to ask how to include a file in an API request.

Thank you for your help!

[arham-kk]

GitHub imposes token limits per request (e.g., 8000 tokens) to manage server load, even though models like GPT-4o mini can handle larger contexts (131k tokens). To include a file in an API request, encode it in Base64 and include it in the request body. Ensure the file size complies with GitHub's limits.

[solitude-alive]

Thank you for your rely.
What a pity. GitHub Models is a good platform offering a variety of model APIs. Do you think there's a possibility that this restriction will be lifted in the future, allowing us to make better use of the models?