Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Conditional for a specific Message Value to control Message Routing in Azure Service Bus #6

Closed
jkears opened this issue Jan 26, 2023 · 4 comments
Labels
documentation Improvements or additions to documentation question Further information is requested

Comments

@jkears
Copy link

jkears commented Jan 26, 2023

Love this library, working very well so far.

How can I publish a custom Property that I can use to filter my message.

For example this is a sample test message.. and I'd like only messages with the field data.Authentication.Name set equal to "Jimmy" to be forwarded to the subscriber ....

{
                    ""eventID"": ""74c5a2db-2a9a-4ce3-93c5-c77fb566b8d7"",
                    ""cloudEventsVersion"": ""0.1"",
                    ""eventType"": ""AuthenticationDefinitionCreatedDomainEvent"",
                    ""eventTypeVersion"": ""1.0"",
                    ""source"": ""ede603ab-4e15-400c-8d4a-c903166b590d-created-event#AuthenticationDefinitionCreatedDomainEvent"",
                    ""eventTime"": ""01/26/2023 23:00:50 +00:00"",
                    ""data"": {
                        ""AuthenticationDefinition"": {
                            ""AccessTokenExpiration"": 0,
                            ""AccessTokenUrl"": ""string"",
                            ""Audience"": ""string"",
                            ""AuthenticationType"": 0,
                            ""AuthorizationUrl"": ""string"",
                            ""CheckAuthVerification"": true,
                            ""ClientId"": ""string"",
                            ""ClientSecret"": ""string"",
                            ""DisableIntrospection"": true,
                            ""Name"": ""Jimmy"",
                            ""OAuthCallbackUrl"": ""string"",
                            ""Scopes"": ""string"",
                            ""UseClientCredentialFlow"": true,
                            ""modelVersion"": ""0.0.0"",
                            ""id"": ""8ee21697-b51c-493b-b0e5-fb26aa91210f"",
                            ""deleted"": false,
                            ""createdAt"": ""2023-01-26T18:00:50.8332556-05:00"",
                            ""modifiedAt"": ""2023-01-26T18:00:50.8332557-05:00"",
                            ""deletedAt"": null,
                            ""modifiedBy"": ""d9ecc60b-4867-4cb8-9a8d-5f5971f4558d""
                        },
                        ""Payload"": null,
                        ""AuthenticationDefinitionId"": ""8ee21697-b51c-493b-b0e5-fb26aa91210f"",
                        ""AggregateId"": ""8ee21697-b51c-493b-b0e5-fb26aa91210f"",
                        ""Action"": ""AuthenticationDefinitionCreatedDomainEvent"",
                        ""ModelKey"": ""ede603ab-4e15-400c-8d4a-c903166b590d-created-event"",
                        ""MessageHeaders"": {
                            ""domain-command"": ""Create"",
                            ""nameidentifier"": ""d9ecc60b-4867-4cb8-9a8d-5f5971f4558d"",
                            ""oid"": ""d9ecc60b-4867-4cb8-9a8d-5f5971f4558d"",
                            ""role"": ""access:metamodel-api"",
                            ""tenant"": ""e01a7bdc-ab45-48b0-ac38-0fac30cc1d90"",
                            ""iss"": ""https://login.microsoftonline.com/e01a7bdc-ab45-48b0-ac38-0fac30cc1d90/v2.0"",
                            ""exp"": ""1674773871"",
                            ""aud"": ""e86b11e0-1394-48f2-b298-0fb58f330c4e"",
                            ""azp"": ""875879f8-dbcd-4f06-81ab-c0079df6ecd6""
                        },
                        ""GeneratedFromModelVersion"": """",
                        ""GeneratedSequence"": """",
                        ""id"": ""74c5a2db-2a9a-4ce3-93c5-c77fb566b8d7"",
                        ""deleted"": false,
                        ""createdAt"": ""2023-01-26T18:00:50.833257-05:00"",
                        ""modifiedAt"": null,
                        ""deletedAt"": null,
                        ""modifiedBy"": null
                    }
                }

I see in the BaseAzureServiceBusPublisher class it appears to be adding a number of ApplicationProperties for ProcessorType, ProcessSender etc...

It also seems to be pulling from a headers section and adding Application Properties per each.

Is there a way of extracting the value for data.Authentication.Name property from the message and adding it to "NameCheck" such that I can add a rule in Azure Service bus such as: NameCheck = 'Jimmy'.

@cajuncoding
Copy link
Owner

cajuncoding commented Jan 27, 2023

@jkears That's awesome that you are getting value out of this and enjoying it... 🙌

So you are on the right track! The original design was that the message sender could/should control various aspects dynamically. And since the SqlTransactionalOutbox is intended to be agnostic of the messaging broker being used (though Azure Service Bus is all I've implemented so far 😉 )... I used a few terms that convey the meaning but are not coupled specifically to Azure Service Bus, or any messaging system.

So if your payload has any Json properties (objects) in the root of the payload with the name headers, or appProperties or userProperties then all properties of that Json node are dynamically mapped into the Azure Service Bus message.ApplicationProperties collection and can be used for subscription routing per Microsoft Docs 🚀.... this was an intentional feature. NOTE: the Json names noted above are checked in that order of priority and the first one found is used so it's expected that the message is consistent and chooses only one. Personally I liked the generic term headers as it maps well to Http headers that are passed along with requests and can be used for routing, etc.

You can see in the null coalescing fallback logic for these three dynamic Json object nodes in the BaseAzureServiceBusPublisher:

                //Populate HeadersLookup/User Properties if defined dynamically...
                var headers = GetJsonValueSafely<JObject>(json, JsonMessageFields.Headers)
                                        ?? GetJsonValueSafely<JObject>(json, JsonMessageFields.AppProperties)
                                        ?? GetJsonValueSafely<JObject>(json, JsonMessageFields.UserProperties);

At this time there is not a configurable way to intercept and mutate the Json payload to move/copy the data.Authentication.Name value up into a new headers.NameCheck property. It's assumed that whatever code is creating the message to be stored in the Outbox would do this since it has access to or creates the json payload before saving to the outbox.

So whatever code creates and stores your messages can simply preprocess it with something like this (not tested):

//Parse the payload if it's a string...
var json = JObject.Parse(messagePayload);
//Dynamically extract and copy the value into a new `headers` node of the payload...
var nameCheck = json.SelectToken("$.data.AuthenticationDefinition.Name")?.ToString();
if (nameCheck != null)
{
     headersJson = new JObject();
     headersJson["NameCheck"] = nameCheck;
     json["headers"] = headersJson; 
}
//Now re-serialize to json string...
messagePayload = json.ToString();

I'm sure you already know this but here's the MS docs discussing how to filter based on Message.ApplicaitonProperties which is what the latest Az Service Bus libraries offer:
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-filter-examples#filter-on-message-properties

@cajuncoding cajuncoding added documentation Improvements or additions to documentation question Further information is requested labels Jan 27, 2023
@jkears
Copy link
Author

jkears commented Jan 27, 2023

@cajuncoding, thank you for your prompt support.

As a quick test I added the following header and Correlation rule, and it worked perfectly....

image

It may not be so obvious but I am using CloudEvents which promotes a specific schema, and so ideally, it would be great if somehow we could push the headers section into the data section of the cloud event. This way we would adhere to the intended schema of the CloudEvent, and place the header section within the data section.

Specifically, this is what is defined...

public const string Headers = "headers";

It would be idea for this to be a configuration driven value such that I could place headers in the Data Payload, such as "data.headers", and actually in my case I already have a messageheaders section so "data.messageheaders".

Is that possible with the existing code base and without modifying any code?

@cajuncoding
Copy link
Owner

cajuncoding commented Jan 27, 2023

@jkears Ok, I sort of mis-spoke in my response above... so I edited to clarify. There is no configurable way to intercept the json and mutate it (e.g. re-locating a value into the headers node). But there is definitely an elegant solution 😁!

However, as an alternative, and by design, the version intentionally allows you to create a customized version of the BaseAzureServiceBusPublisher<Guid> as all methods are virtual by design . . . so that will easily allow you to mutate the actual Azure Service Bus Message that is created. So you def. can extract the data.Authentication.Name value and manually add it into the message.ApplicationProperties .

So rather than using the DefaultAzureServiceBusOutboxPublisher, you'd just new up your own MyCustomAzureServiceBusOutboxPublisher and use that exactly the same.

Here's an sample (most of the below are code comments you can remove):

    public class MyCustomAzureServiceBusOutboxPublisher : BaseAzureServiceBusPublisher<Guid>
    {
        public MyCustomAzureServiceBusOutboxPublisher(string azureServiceBusConnectionString, AzureServiceBusPublishingOptions options = null) 
            : base(azureServiceBusConnectionString, options)
        {
            //All logic is currently in the Base Constructor
        }

        protected override ServiceBusMessage CreateEventBusMessage(ISqlTransactionalOutboxItem<Guid> outboxItem)
        {
            //First parse our input because it's easier to access the Payload before
            //  it's converted to Binary within the Message...
            var payloadJson = base.ParsePayloadAsJsonSafely(outboxItem);
            
            //Now create let the Default Outbox handler create the message...
            var eventBusMessage = base.CreateEventBusMessage(outboxItem);

            //Now we can update it with custom Applicatoin properties without affecting
            //  the Payload structure (e.g. no Headers node/section)...
            var nameCheck = payloadJson.SelectToken("$.data.AuthenticationDefinition.Name")?.ToString();
            eventBusMessage.ApplicationProperties.Add(MessageHeaders.ToHeader("data_AuthenticationDefinition_Name"), nameCheck);

            //Finally return the customized message...
            return eventBusMessage;
        }
    }

@jkears
Copy link
Author

jkears commented Jan 31, 2023

Thank you so very much, this works like a charm!

@jkears jkears closed this as completed Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants