Skip to content

Firestore: Fix sorting 'delete_changes' in 'Watch._compute_snapshot'.#8809

Merged
tseaver merged 4 commits intogoogleapis:masterfrom
liamuk:patch-1
Aug 1, 2019
Merged

Firestore: Fix sorting 'delete_changes' in 'Watch._compute_snapshot'.#8809
tseaver merged 4 commits intogoogleapis:masterfrom
liamuk:patch-1

Conversation

@liamuk
Copy link
Contributor

@liamuk liamuk commented Jul 30, 2019

delete_changes on the changed line is a list of string, not a list of DocumentSnapshots, so it should not be sorted with the query comparator function (see https://github.com/googleapis/google-cloud-python/blob/master/firestore/google/cloud/firestore_v1/watch.py#L575)

For context, I'm getting this error in one of my snapshot listeners in google-cloud-firestore==1.3.0

Thread-ConsumeBidirectionalStream caught unexpected exception 'str' object has no attribute 'reference' and will exit.
Traceback (most recent call last):
  File ".../venv/lib/python3.5/site-packages/google/api_core/bidi.py", line 635, in _thread_main
    self._on_response(response)
  File ".../venv/lib/python3.5/site-packages/google/cloud/firestore_v1/watch.py", line 443, in on
_snapshot
    meth(proto)
  File ".../venv/lib/python3.5/site-packages/google/cloud/firestore_v1/watch.py", line 378, in _o
n_snapshot_target_change_no_change
    self.push(change.read_time, change.resume_token)
  File ".../venv/lib/python3.5/site-packages/google/cloud/firestore_v1/watch.py", line 540, in pu
sh
    self.doc_tree, self.doc_map, deletes, adds, updates
  File ".../venv/lib/python3.5/site-packages/google/cloud/firestore_v1/watch.py", line 669, in _c
ompute_snapshot
    delete_changes = sorted(delete_changes, key=key)
  File ".../venv/lib/python3.5/site-packages/google/cloud/firestore_v1/query.py", line 819, in _c
omparator
    comp = Order()._compare_to(doc1.reference._path, doc2.reference._path)
AttributeError: 'str' object has no attribute 'reference'

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it!) and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added the cla: no This human has *not* signed the Contributor License Agreement. label Jul 30, 2019
@liamuk
Copy link
Contributor Author

liamuk commented Jul 30, 2019

@googlebot I signed it!

@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Jul 30, 2019
@tseaver tseaver changed the title Fix type-checking error in firestore watch.py Firestore: Fix type-checking error. Jul 30, 2019
@tseaver tseaver changed the title Firestore: Fix type-checking error. Firestore: Fix sorting 'delete_changes' in 'Watch._compute_snapshot'. Jul 30, 2019
@tseaver tseaver added api: firestore Issues related to the Firestore API. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Jul 30, 2019
Copy link
Contributor

@tseaver tseaver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liamuk Thanks for the patch! Our tests are passing here because they do not exercise Watch._compute_snapshot on an instance with a non-dummy comparator. Can you please add a test to the TestWatch class tests/unit/v1/test_watch.py which exercises that case? E.g.:

    def test__compute_snapshot_deletes_w_real_comparator(self):
        from google.cloud.firestore_v1.watch import WatchDocTree

        doc_tree = WatchDocTree()

        class DummyDoc(object):
            update_time = mock.sentinel

        def _comparator(doc1, doc2):
            if doc1.field > doc2.field:
                return 1
            if doc1.field < doc2.field:
                return -1
            return 0

        deleted_doc_1 = DummyDoc()
        deleted_doc_2 = DummyDoc()
        doc_tree = doc_tree.insert(deleted_doc_1, None)
        doc_tree = doc_tree.insert(deleted_doc_2, None)
        doc_map = {
            "/deleted_1": deleted_doc_1,
            "/deleted_2": deleted_doc_2,
        }
        delete_changes = ["/deleted_1", "/deleted_2"]
        add_changes = []
        update_changes = []
        inst = self._makeOne(comparator=_comparator)
        updated_tree, updated_map, applied_changes = inst._compute_snapshot(
            doc_tree, doc_map, delete_changes, add_changes, update_changes
        )
        self.assertEqual(updated_map, {})

@tseaver tseaver added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 30, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 30, 2019
@liamuk
Copy link
Contributor Author

liamuk commented Jul 31, 2019

Added that test verbatim

@tseaver tseaver added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 31, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 31, 2019
Copy link
Contributor

@tseaver tseaver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to have to ask for changes in the code I suggested: I checked that the test ran and failed without your patch, but not that it appeased coverage / lint checkers.

@liamuk
Copy link
Contributor Author

liamuk commented Jul 31, 2019

Changed those two things, and also added you as a collaborator to the patch's fork in case anything else comes up.

You writing code for me to copy into the patch branch isn't the most efficient process haha.

@tseaver tseaver added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 31, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 31, 2019
@tseaver
Copy link
Contributor

tseaver commented Jul 31, 2019

The Video Intelligence failure is due to a Kokoro glitch: "Cannot check out non-mergeable pull request as if merged."

@tseaver tseaver added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 1, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 1, 2019
@tseaver tseaver merged commit c9b9c9d into googleapis:master Aug 1, 2019
@tseaver
Copy link
Contributor

tseaver commented Aug 1, 2019

@liamuk Thanks again for the patch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: firestore Issues related to the Firestore API. cla: yes This human has signed the Contributor License Agreement. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants