Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
1d02154
added initial implementation of mutate_rows
daniel-sanche Apr 24, 2023
ab63cba
implemented mutation models
daniel-sanche Apr 24, 2023
cf9daa5
added retries to mutate_row
daniel-sanche Apr 24, 2023
1247da4
return exception group if possible
daniel-sanche Apr 24, 2023
3b3ed8c
check for idempotence
daniel-sanche Apr 24, 2023
5d20037
initial implementation for bulk_mutations
daniel-sanche Apr 24, 2023
3d322a1
include successes in bulk mutation error message
daniel-sanche Apr 24, 2023
a31232b
fixed style checks
daniel-sanche Apr 24, 2023
8da2d65
added basic system tests
daniel-sanche Apr 24, 2023
2b89d9c
added unit tests for mutate_row
daniel-sanche Apr 25, 2023
47c5985
ran blacken
daniel-sanche Apr 25, 2023
38fdcd7
improved exceptions
daniel-sanche Apr 25, 2023
504d2d8
added bulk_mutate_rows unit tests
daniel-sanche Apr 25, 2023
b16067f
ran blacken
daniel-sanche Apr 25, 2023
3ab1405
support __new___ for exceptions for python3.11+
daniel-sanche Apr 25, 2023
0a6c0c6
added exception unit tests
daniel-sanche Apr 25, 2023
ec043cf
makde exceptions tuple
daniel-sanche Apr 26, 2023
518530e
got exceptions to print consistently across versions
daniel-sanche Apr 26, 2023
9624729
added test for 311 rich traceback
daniel-sanche Apr 27, 2023
3087081
moved retryable row mutations to new file
daniel-sanche Apr 27, 2023
9df588f
use index map
daniel-sanche Apr 27, 2023
7ed8be3
added docstring
daniel-sanche Apr 27, 2023
2536cc4
added predicate check to failed mutations
daniel-sanche Apr 27, 2023
1f6875c
added _mutate_rows tests
daniel-sanche Apr 27, 2023
1ea24e6
improved client tests
daniel-sanche Apr 27, 2023
25ca2d2
refactored to loop by raising exception
daniel-sanche Apr 28, 2023
c0787db
refactored retry deadline logic into shared wrapper
daniel-sanche Apr 28, 2023
3ed5c3d
ran black
daniel-sanche Apr 28, 2023
a91fbcb
pulled in table default timeouts
daniel-sanche Apr 28, 2023
df8a058
added tests for shared deadline parsing function
daniel-sanche Apr 28, 2023
b866b57
added tests for mutation models
daniel-sanche Apr 28, 2023
54a4d43
fixed linter errors
daniel-sanche Apr 28, 2023
bd51dc4
added tests for BulkMutationsEntry
daniel-sanche Apr 28, 2023
921b05a
improved mutations documentation
daniel-sanche Apr 28, 2023
82ea61f
refactored mutate_rows logic into helper function
daniel-sanche May 2, 2023
fa42b86
implemented callbacks for mutate_rows
daniel-sanche May 2, 2023
01a16f3
made exceptions into a tuple
daniel-sanche May 5, 2023
6140acb
remove aborted from retryable errors
daniel-sanche May 22, 2023
36ba2b6
improved SetCell mutation
daniel-sanche May 22, 2023
b3c9017
fixed mutations tests
daniel-sanche May 22, 2023
cac9e2d
SetCell timestamps use millisecond precision
daniel-sanche May 22, 2023
34b051f
renamed BulkMutationsEntry to RowMutationEntry
daniel-sanche May 22, 2023
63ac35c
Merge branch 'v3' into mutate_rows
daniel-sanche May 24, 2023
a51201c
added metadata to mutate rows and bulk mutate rows
daniel-sanche May 25, 2023
a21bebf
moved _convert_retry_deadline wrapper from exceptions into _helpers
daniel-sanche May 25, 2023
4ca89d9
fixed system tests
daniel-sanche May 25, 2023
b240ee1
only handle precision adjustment when creating timestamp
daniel-sanche May 26, 2023
cb0e951
added _from_dict for mutation models
daniel-sanche May 26, 2023
a9cf385
rpc timeouts adjust when approaching operation_timeout
daniel-sanche May 26, 2023
eddc1c9
pass table instead of request dict
daniel-sanche May 26, 2023
f8b26aa
refactoring mutate rows
daniel-sanche May 27, 2023
5b80dc5
made on_terminal_state into coroutine
daniel-sanche May 27, 2023
9e5b80a
fixed style issues
daniel-sanche May 27, 2023
f7539f6
moved callback rewriting into retryable attempt
daniel-sanche May 27, 2023
e77a4fa
fixed tests
daniel-sanche May 27, 2023
4e19ed0
pop successful mutations from error dict
daniel-sanche May 30, 2023
920e4b7
removed unneeded check
daniel-sanche May 30, 2023
725f5ff
refactoring
daniel-sanche May 30, 2023
1054bc4
pass list of exceptions in callback
daniel-sanche May 30, 2023
f39a891
raise error in unexpected state
daniel-sanche May 30, 2023
1d97135
removed callback
daniel-sanche May 31, 2023
88e2bf5
refactoring mutation attempt into class
daniel-sanche May 31, 2023
a3c0166
use partial function
daniel-sanche May 31, 2023
70c35ef
renamed class
daniel-sanche May 31, 2023
e00f592
added comments
daniel-sanche Jun 1, 2023
18af78a
added tests
daniel-sanche Jun 1, 2023
23e84f5
improved helpers
daniel-sanche Jun 1, 2023
56fdf7c
refactored operation into class only
daniel-sanche Jun 5, 2023
aca31f0
restructured how remaining indices are tracked
daniel-sanche Jun 5, 2023
5a5d541
fixed tests
daniel-sanche Jun 5, 2023
afed731
added docstrings
daniel-sanche Jun 5, 2023
2396ec8
moved index deletion to end of block
daniel-sanche Jun 6, 2023
3d441a2
added comment to exception types
daniel-sanche Jun 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
added docstring
  • Loading branch information
daniel-sanche committed Apr 27, 2023
commit 7ed8be3fccc64e35db9648e987ff6d5f74a4bdad
19 changes: 19 additions & 0 deletions google/cloud/bigtable/_mutate_rows.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,25 @@ async def _mutate_rows_retryable_attempt(
mutation_dict: dict[int, "BulkMutationsEntry"|None],
error_dict: dict[int, list[Exception]],
):
"""
Helper function for managing the mutate_rows lifecycle.

Function will continuously retry failed mutations, until all mutations succeed or encounter
an unrecoverable error.

This function is intended to be wrapped in an api_core.retry.AsyncRetry object, which will handle
timeouts and retrying raised exceptions.

Args:
- gapic_client: the client to use for the mutate_rows call
- request: the request to send to the server, populated with table name and app profile id
- per_request_timeout: the timeout to use for each mutate_rows attempt
- mutation_dict: a dictionary tracking which entries are outstanding
(stored as BulkMutationsEntry), and which have reached a terminal state (stored as None).
At the start of the request, all entries are outstanding.
- error_dict: a dictionary tracking errors associated with each entry index.
Each retry will append a new error. Successful mutations will clear the error list.
"""
new_request = request.copy()
while any(mutation is not None for mutation in mutation_dict.values()):
await asyncio.sleep(0)
Expand Down