PubSub: Inconsistent documentation on received message types

[enugentdt]

Issue Description

At various points in receiving messages through PubSub, they will come in as different types, which has no documentation on what types occur when. The majority of messages are:

<class 'google.cloud.pubsub_v1.subscriber.message.Message'>

However, I see some:

<class 'google.cloud.pubsub_v1.types.ReceivedMessage'>

Those aren't much of a problem, I just have to do message = message.message. Not great, because I have to detect for those types, but nothing bad really.

Lastly, I see a couple with the name PubsubMessage. They appear to be of the first type above, but they don't have any ack(), nack(), etc. methods. They seem to be messages that the client would create, but not receive? There doesn't appear to be any documentation on when these different message types would occur.

Environment details

Using:
PubSub on OS X Mojave
Python 3.6
google-cloud-pubsub==0.39.1

Steps to reproduce

Loosely, the issue occurs on receiving messages. I can't track it down to anything in particular, because I can't debug the messages themselves.

Stack trace

Traceback (most recent call last):
  File "logger.py", line 56, in <module>
    pubsub.run(process_message)
  File "<projectdir>/pubsub.py", line 68, in run
    pull()
  File "<projectdir>/pubsub.py", line 80, in pull
    pubsub_callback(message)
  File "<projectdir>/pubsub.py", line 57, in pubsub_callback
    messageobj.ack()
AttributeError: 'PubsubMessage' object has no attribute 'ack'

Thanks in advance for the help!

[rberlier]

I didn't check the types. But I think its somehow related to already acknowledged messages in batch with the SubscriberClient.acknowledge function. Even though I'm acknowledging received messages I still receive the messages with already received IDS. So I went in and added a call to the message function : ack() so I got the same error ! And now I'm here ^^

[sduskis]

@anguillanneuf or @plamut, do you have any answers for this?

[plamut]

• google.cloud.pubsub_v1.types.ReceivedMessage:
  This is the ReceivedMessage type that the server actually returns as a response for the Pull method, or a response for the StreamingPull method. It contains the message itself (i.e. message.message of type PubsubMessage) and its acknowledge ID.

• google.cloud.pubsub_v1.subscriber.message.Message:
  This is a custom type defined in the subscriber sub-package. It wraps the received PubsubMessage and itst ACK ID when asynchronously receiving messages from the server using the subscriber.subscribe() method, and this is what async callbacks get called with.

  Unlike the "plain" PubsubMessage the wrapper provides the methods such as ack(), nack(), etc. that can be used to asynchronously (n)ack() a message, for instance.

TL; DR - messages received with an asynchronous streaming pull process are additionally wrapped into the subscriber.message.Message class that provides methods to ack/nack them asynchronously - as opposed to messages pulled synchronously with the subscriber.pull() method.

@enugentdt Does your app pull the messages from the server in both sync and async fashion, resulting in mixed types? That seems the most plausible explanation based on the given info.

If this is not explained in the current docs well, then I agree the latter should be improved.

---

@relbekri Receiving the already acknowledged messages again could be a sign of acknowledgements not arriving soon enough, and the server thinking that the messages got lost, thus re-sending them. One way of mitigating this is to modify the messages' ACK deadline upon receipt, if processing them is expected to take "long".

An AttributeError when calling ack() is most likely the result of an attempt to call it on a non-wrapped message type that does not implement that method.

[sduskis]

@plamut and @anguillanneuf is the documentation technically correct? If so, is there anything we can do to make the docs better?

[anguillanneuf]

The information in the documentation is appropriate and up to date:

• PubsubMessage
• ReceivedMessage
• subscriber.message.Message

But I propose we link received_messages in PullResponse and StreamingPullResponse documentation to the appropriate definition, i.e. ReceivedMessage.

I want to do some cleanup of the documentation so it's easier for users to find out which message type is returned by various publisher and subscriber client methods.

[sduskis]

That sounds like a good plan, @anguillanneuf. Thanks!