Fix Any recursion depth bypass in Python json_format.ParseDict

[aviralgarg05]

This fixes a security vulnerability where nested google.protobuf.Any messages could bypass the max_recursion_depth limit, potentially leading to denial of service via stack overflow.

The root cause was that _ConvertAnyMessage() was calling itself recursively via methodcaller() for nested well-known types, bypassing the recursion depth tracking in ConvertMessage().

The fix routes well-known type parsing through ConvertMessage() to ensure proper recursion depth accounting for all message types including nested Any.

Fixes #25070

[ankitsinha-cmyk]

Consider:

1. Add a comment clarifying how you count depth and whether max_recursion_depth is inclusive/exclusive.

2. Adding a test that explicitly asserts that depth==max_recursion_depth-1 succeeds and depth==max_recursion_depth fails, so the boundary behavior is documented.

[benglewis]

I understand this this is failing CI/CD and all... but we are stuck with our packages failing on vulnerabilities with this fix. What is the ETA for getting this merged?

[ankitsinha-cmyk]

@aviralgarg05 thanks for incorporating the changes. Looks fine to me now.

[crmpicco]

Thanks @paulhodson , that is a useful tip for future

[obs-gh-nikhildua]

@aviralgarg05 @ankitsinha-cmyk @anandolee Is there any plans to bring this to version 4.25.x? We run a cloud function which depends on this version and updating will cause backwards incompatibility with other library versions as well for our customers. Thanks!

[sfc-gh-ndua]

@aviralgarg05 @ankitsinha-cmyk @anandolee Is there any plans to bring this to version 4.25.x? We run a cloud function which depends on this version and updating will cause backwards incompatibility with other library versions as well for our customers. Thanks!

Any update on above? Thanks!

[esrauchg]

We don't plan to backport to 4.25 as Py 4.x has has been end of lifed for a year (after a year of high prio fixes only). See the chart here:
https://protobuf.dev/support/version-support/#:~:text=TBD-,Release%20support%20chart,-Protobuf%20Python

The 25.x line is still specially maintenance support just for Java (so the same issue happening in Java would get a backport by contrast).

Note that we're even in the final quarter of official high prio backports only for Py 5.x, so 4.x is getting quite behind the version treadmill here.

Note that the breaking changes across this version boundaries are intentionally minor/obscure including that old gencode is still compatible, as described here:
https://protobuf.dev/support/cross-version-runtime-guarantee/#:~:text=minor%2C%20or%20micro).-,Python%2Dspecific%20Guarantees,-Since%20the%203.20.0

So we expect it should be rare that anyone should not be able to upgrade past these major version numbers without changing their application code at all.

[sfc-gh-ndua]

We don't plan to backport to 4.25 as Py 4.x has has been end of lifed for a year (after a year of high prio fixes only). See the chart here: https://protobuf.dev/support/version-support/#:~:text=TBD-,Release%20support%20chart,-Protobuf%20Python

The 25.x line is still specially maintenance support just for Java (so the same issue happening in Java would get a backport by contrast).

Note that we're even in the final quarter of official high prio backports only for Py 5.x, so 4.x is getting quite behind the version treadmill here.

Note that the breaking changes across this version boundaries are intentionally minor/obscure including that old gencode is still compatible, as described here: https://protobuf.dev/support/cross-version-runtime-guarantee/#:~:text=minor%2C%20or%20micro).-,Python%2Dspecific%20Guarantees,-Since%20the%203.20.0

So we expect it should be rare that anyone should not be able to upgrade past these major version numbers without changing their application code at all.

Thanks! We have other dependencies that will break such as google-cloud-sdk libs (https://github.com/observeinc/google-cloud-functions/blob/main/requirements.txt#L48)

venv) ➜ google-cloud-functions git:(main) ✗ python -m pip install -U "protobuf==6.33.5" Collecting protobuf==6.33.5 Using cached protobuf-6.33.5-cp39-abi3-macosx_10_9_universal2.whl.metadata (593 bytes) Downloading protobuf-6.33.5-cp39-abi3-macosx_10_9_universal2.whl (427 kB) Installing collected packages: protobuf Attempting uninstall: protobuf Found existing installation: protobuf 4.25.8 Uninstalling protobuf-4.25.8: Successfully uninstalled protobuf-4.25.8 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. google-cloud-org-policy 1.8.3 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-access-context-manager 0.1.16 requires protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-os-config 1.15.3 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-tasks 2.14.2 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-appengine-logging 1.4.3 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. grpc-google-iam-v1 0.12.6 requires protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-pubsub 2.18.4 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-asset 3.25.0 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. googleapis-common-protos 1.61.0 requires protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0.dev0,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. google-cloud-compute 1.14.1 requires protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5, but you have protobuf 6.33.5 which is incompatible. proto-plus 1.22.3 requires protobuf<5.0.0dev

[psibre]

@sfc-gh-ndua I don't want to tell you how to manage your dependencies, but all of the Google Cloud packages you have are apparently pinned to versions from around 2023. But they're all actively maintained and have had many releases since then, which are presumably compatible with protobuf v5+. So you could just upgrade them...?

[esrauchg]

Speaking authoritatively for Protobuf project: it does looking to me that cloud-sdk libs are stuck behind here (at least for their default set version, unclear about true-compatibility) and they is one of our biggest Google library usages. We'll follow up on that internally to see why its still stuck.

Broadly we also know being 3 years behind on some deps is also relatively normal for lots of projects; we are just trying to thread the needle on being able to make changes that keep Protobuf healthy while still avoiding breaking people; there's unfortunately a direct tension where total stability (which we did for a while) makes things stagnant on all axes which ends up slowly building up over time to harm.

We do try very hard to keep the breaking changes obscure to avoid people's difficulties upgrading. We had made some missteps with our last Java breaking change, which is partially why it has an extended support window, so we also know that even smaller seeming breaks can cause big problems for some customers.

We are likely to do a 25 release for an unrelated Java issue; depending on the state of Cloud SDK and difficulty in releasing the backport we may consider backporting this thing to Py 4.x as well even though its officially end of life (but no promises here).

Thanks!