# RequestManagerTandem<!-- -->

Implements a tandem behaviour for a pair of `RequestLoader` and `RequestManager`.

In this scenario, the contents of the "loader" get transferred into the "manager", allowing processing the requests from both sources and also enqueueing new requests (not possible with plain `RequestManager`).

### Hierarchy

* [RequestManager](https://crawlee.dev/python/api/class/RequestManager)
  * *RequestManagerTandem*

## Index[](#Index)

### Methods

* [\_\_init\_\_](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#__init__)
* [add_request](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#add_request)
* [add_requests](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#add_requests)
* [drop](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#drop)
* [fetch_next_request](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#fetch_next_request)
* [get_handled_count](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#get_handled_count)
* [get_total_count](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#get_total_count)
* [is_empty](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#is_empty)
* [is_finished](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#is_finished)
* [mark_request_as_handled](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#mark_request_as_handled)
* [purge](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#purge)
* [reclaim_request](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#reclaim_request)
* [to_tandem](https://docs.apify.com/sdk/python/reference/class/RequestManagerTandem.md#to_tandem)

## Methods<!-- -->[](#Methods)

### [](#__init__)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L31)\_\_init\_\_

* **\_\_init\_\_**(request_loader, request_manager): None

- #### Parameters

  * ##### request_loader: [RequestLoader](https://crawlee.dev/python/api/class/RequestLoader)
  * ##### request_manager: [RequestManager](https://crawlee.dev/python/api/class/RequestManager)

  #### Returns None

### [](#add_request)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L52)add_request

* async **add_request**(request, \*, forefront): [ProcessedRequest](https://crawlee.dev/python/api/class/ProcessedRequest) | None

- Overrides [RequestManager.add_request](https://crawlee.dev/python/api/class/RequestManager#add_request)

  Add a single request to the manager and store it in underlying resource client.

  ***

  #### Parameters

  * ##### request: str | [Request](https://crawlee.dev/python/api/class/Request)

    The request object (or its string representation) to be added to the manager.

  * ##### optionalkeyword-onlyforefront: bool = <!-- -->False

    Determines whether the request should be added to the beginning (if True) or the end (if False) of the manager.

  #### Returns [ProcessedRequest](https://crawlee.dev/python/api/class/ProcessedRequest) | None

  Information about the request addition to the manager or None if the request was not added.

### [](#add_requests)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L56)add_requests

* async **add_requests**(requests, \*, forefront, batch_size, wait_time_between_batches, wait_for_all_requests_to_be_added, wait_for_all_requests_to_be_added_timeout): None

- Overrides [RequestManager.add_requests](https://crawlee.dev/python/api/class/RequestManager#add_requests)

  Add requests to the manager in batches.

  ***

  #### Parameters

  * ##### requests: Sequence\[str | [Request](https://crawlee.dev/python/api/class/Request)]

    Requests to enqueue.

  * ##### optionalkeyword-onlyforefront: bool = <!-- -->False

    If True, add requests to the beginning of the queue.

  * ##### optionalkeyword-onlybatch_size: int = <!-- -->1000

    The number of requests to add in one batch.

  * ##### optionalkeyword-onlywait_time_between_batches: timedelta = <!-- -->timedelta(seconds=1)

    Time to wait between adding batches.

  * ##### optionalkeyword-onlywait_for_all_requests_to_be_added: bool = <!-- -->False

    If True, wait for all requests to be added before returning.

  * ##### optionalkeyword-onlywait_for_all_requests_to_be_added_timeout: timedelta | None = <!-- -->None

    Timeout for waiting for all requests to be added.

  #### Returns None

### [](#drop)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L110)drop

* async **drop**(): None

- Overrides [RequestManager.drop](https://crawlee.dev/python/api/class/RequestManager#drop)

  Remove persistent state either from the Apify Cloud storage or from the local database.

  ***

  #### Returns None

### [](#fetch_next_request)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L76)fetch_next_request

* async **fetch_next_request**(): Request | None

- Overrides [RequestManager.fetch_next_request](https://crawlee.dev/python/api/class/RequestManager#fetch_next_request)

  Return the next request to be processed, or `None` if there are no more pending requests.

  The method should return `None` if and only if `is_finished` would return `True`. In other cases, the method should wait until a request appears.

  ***

  #### Returns Request | None

### [](#get_handled_count)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L36)get_handled_count

* async **get_handled_count**(): int

- Overrides [RequestManager.get_handled_count](https://crawlee.dev/python/api/class/RequestManager#get_handled_count)

  Get the number of requests in the loader that have been handled.

  ***

  #### Returns int

### [](#get_total_count)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L40)get_total_count

* async **get_total_count**(): int

- Overrides [RequestManager.get_total_count](https://crawlee.dev/python/api/class/RequestManager#get_total_count)

  Get an offline approximation of the total number of requests in the loader (i.e. pending + handled).

  ***

  #### Returns int

### [](#is_empty)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L44)is_empty

* async **is_empty**(): bool

- Overrides [RequestManager.is_empty](https://crawlee.dev/python/api/class/RequestManager#is_empty)

  Return True if there are no more requests in the loader (there might still be unfinished requests).

  ***

  #### Returns bool

### [](#is_finished)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L48)is_finished

* async **is_finished**(): bool

- Overrides [RequestManager.is_finished](https://crawlee.dev/python/api/class/RequestManager#is_finished)

  Return True if all requests have been handled.

  ***

  #### Returns bool

### [](#mark_request_as_handled)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L106)mark_request_as_handled

* async **mark_request_as_handled**(request): ProcessedRequest | None

- Overrides [RequestManager.mark_request_as_handled](https://crawlee.dev/python/api/class/RequestManager#mark_request_as_handled)

  Mark a request as handled after a successful processing (or after giving up retrying).

  ***

  #### Parameters

  * ##### request: [Request](https://crawlee.dev/python/api/class/Request)

  #### Returns ProcessedRequest | None

### [](#purge)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager.py#L25)purge

* async **purge**(): None

- Inherited from [RequestManager.purge](https://crawlee.dev/python/api/class/RequestManager#purge)

  Remove all queued requests from this manager while keeping it usable.

  Default implementation is a no-op. Subclasses with persistent state should override to clear it.

  ***

  #### Returns None

### [](#reclaim_request)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_manager_tandem.py#L102)reclaim_request

* async **reclaim_request**(request, \*, forefront): [ProcessedRequest](https://crawlee.dev/python/api/class/ProcessedRequest) | None

- Overrides [RequestManager.reclaim_request](https://crawlee.dev/python/api/class/RequestManager#reclaim_request)

  Reclaims a failed request back to the source, so that it can be returned for processing later again.

  It is possible to modify the request data by supplying an updated request as a parameter.

  ***

  #### Parameters

  * ##### request: [Request](https://crawlee.dev/python/api/class/Request)
  * ##### optionalkeyword-onlyforefront: bool = <!-- -->False

  #### Returns [ProcessedRequest](https://crawlee.dev/python/api/class/ProcessedRequest) | None

### [](#to_tandem)[](https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/request_loaders/_request_loader.py#L56)to_tandem

* async **to_tandem**(request_manager): [RequestManagerTandem](https://crawlee.dev/python/api/class/RequestManagerTandem)

- Inherited from [RequestLoader.to_tandem](https://crawlee.dev/python/api/class/RequestLoader#to_tandem)

  Combine the loader with a request manager to support adding and reclaiming requests.

  ***

  #### Parameters

  * ##### optionalrequest_manager: RequestManager | None = <!-- -->None

    Request manager to combine the loader with. If None is given, the default request queue is used.

  #### Returns [RequestManagerTandem](https://crawlee.dev/python/api/class/RequestManagerTandem)
