Skip to content

feat: add zstd request body compression - #1218

Merged
vdusek merged 46 commits into
masterfrom
sync/0e1932b28308
Nov 16, 2026
Merged

vdusek merged 46 commits into
masterfrom
sync/0e1932b28308

Conversation

@apify-factory

@apify-factory apify-factory commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Ports apify/apify-client-js#1200.

  • compression='zstd' on ApifyClient, ApifyClientAsync, and ActorServerClient.request selects the new ZstdHttpCompressor (apify_client.http_compressors).
  • quality goes from 1 to 19 and defaults to 3, as in JS. Levels 20 to 22 are rejected because their window is larger than the 8 MB that RFC 9659 requires HTTP decoders to support. An invalid quality raises ValueError, like the other Python compressors.
  • Python 3.14 and newer use the standard library compression.zstd. Older versions need the new zstd extra, which installs backports.zstd.
  • Without zstd, import apify_client still works. Only ZstdHttpCompressor and compression='zstd' raise ImportError. This covers a missing extra, a Python 3.14+ build without zstd, and an installed backport that can't load.
  • The same applies to brotli: if the brotli package is installed but its C extension can't load, import apify_client works and only BrotliHttpCompressor and compression='brotli' raise ImportError.
  • try_import treats a missing parent package (backports) as a missing dependency. A load error re-raised from another import error gets no install hint, because installing the extra doesn't fix it.
  • ty checks against Python 3.11, so backports.zstd is listed in allowed-unresolved-imports for runs on 3.14+, where it isn't installed.
  • The HTTP compression concepts page, the introduction, and the README document the option and the extra. Unit tests cover both import paths. Integration tests send gzip, brotli, and zstd bodies to the live API with one sync Impit client.

✍️ Drafted by Claude Code

@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 10, 2026
@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.92857% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.67%. Comparing base (e8c6ba6) to head (59bc8b6).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
src/apify_client/http_compressors/_zstd.py 75.86% 7 Missing ⚠️
.../apify_client/http_compressors/_zstd_dependency.py 80.00% 1 Missing ⚠️
src/apify_client/types.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1218      +/-   ##
==========================================
- Coverage   95.76%   95.67%   -0.09%     
==========================================
  Files          63       65       +2     
  Lines        6703     6753      +50     
==========================================
+ Hits         6419     6461      +42     
- Misses        284      292       +8     
Flag Coverage Δ
integration 90.68% <55.35%> (-0.25%) ⬇️
unit 91.33% <83.92%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pijukatel
Pijukatel requested a review from vdusek November 10, 2026 10:17

@vdusek vdusek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a self-review / staff review to the porting process?

Comment thread pyproject.toml Resolved
Comment thread docs/01_introduction/index.mdx Resolved
Comment thread README.md Resolved
@Pijukatel
Pijukatel requested a review from vdusek November 10, 2026 14:57

@vdusek vdusek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: compressor_case in tests/unit/test_http_clients.py (outside the diff) still covers only gzip and brotli. Consider adding a zstd case: pytest.param((ZstdHttpCompressor(), 'zstd', zstd.decompress), id='zstd').

✍️ Drafted by Claude Code

Comment thread tests/unit/test_http_compressors.py Resolved
Comment thread tests/unit/test_key_value_store.py Resolved
Comment thread README.md Resolved
Comment thread tests/unit/test_http_compressors.py Resolved
Cover a key-value store record and a dataset push through a client
that compresses request bodies with zstd, for both transports and
both the sync and async client.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ocuAK2fAevKdoyPCB2HvC
@Pijukatel
Pijukatel requested a review from vdusek November 12, 2026 09:06
# Conflicts:
#	tests/_utils.py
#	tests/integration/conftest.py
# Conflicts:
#	tests/unit/test_http_clients.py

@vdusek vdusek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vdusek
vdusek merged commit 95a05a2 into master Nov 16, 2026
35 checks passed
@vdusek
vdusek deleted the sync/0e1932b28308 branch November 16, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api-client-sync t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants