You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
All crawlee persistence names should be valid names for Apify platform #1354
In FileSystemRequestQueueClient there is persistance to kvs with name persist_state_kvs_name=f'__RQ_STATE_{self._metadata.id}',
When used on Apify platform, it will throw an error:
apify_client._errors.ApifyApiError: Invalid value provided in store: name can only contain letters 'a' through 'z', the digits '0' through '9', and the hyphen ('-') but only in the middle of the string (e.g. 'my-value-1')
which is caused by the underscores in the name.
Fix the name to be compatible with the Apify constraints and check all similar names of RecoverableStates in code base
To reproduce, run this on Apify platform, which create file system base request queue, that persists to Apify kvs:
This is more of a problem that RecoverableState relies on a service locator to give it suitable storage, and this can lead to scenarios where RecoverableState for XYZ storage is being saved to ABC storage, which makes no sense. A similar problem is waiting to be triggered in KeyValueStore.get_auto_saved_value, which also uses RecoverableState that relies on the service locator.
Reasonable default should be that RecoverableState is using the same storage for which it is recovering...
changed the title [-]All crawlee persistance names should be valid names for Apify platform[/-][+]All crawlee persistence names should be valid names for Apify platform[/+]on Oct 7, 2025
Activity
Pijukatel commentedon Aug 21, 2025
This is more of a problem that
RecoverableStaterelies on a service locator to give it suitable storage, and this can lead to scenarios whereRecoverableStatefor XYZ storage is being saved to ABC storage, which makes no sense. A similar problem is waiting to be triggered inKeyValueStore.get_auto_saved_value, which also usesRecoverableStatethat relies on the service locator.Reasonable default should be that
RecoverableStateis using the same storage for which it is recovering...refactor!: Refactor storage creation and caching, configuration and s…
Add docs versioning section [apify#1354]
[-]All crawlee persistance names should be valid names for Apify platform[/-][+]All crawlee persistence names should be valid names for Apify platform[/+]fix: Add storages name validation (#1457)
fix: Fix `KeyValueStore.auto_saved_value` failing in some scenarios (#…