Skip to content

[Citrix ADC] Fix Grok parsing failure for LOGOUT and HTTPREQUEST of #14496#14781

Merged
yahyaghani merged 4 commits intoelastic:mainfrom
yahyaghani:14496-citrix-adc-grok-processing
Aug 19, 2025
Merged

[Citrix ADC] Fix Grok parsing failure for LOGOUT and HTTPREQUEST of #14496#14781
yahyaghani merged 4 commits intoelastic:mainfrom
yahyaghani:14496-citrix-adc-grok-processing

Conversation

@yahyaghani
Copy link
Copy Markdown
Contributor

@yahyaghani yahyaghani commented Aug 2, 2025

📦 Citrix ADC Grok Parsing Fix: LOGOUT & HTTPREQUEST (#14496)

📝 Description

Fix parsing of LOGOUT and HTTPREQUEST messages that were failing due to misconfigured grok patterns.
Added third grok pattern for grok_sslvpn_logout and grok_sslvpn_httprequest to handle log format variations seen in bug [Citrix ADC]: Grok Parsing Failure for LOGOUT & HTTPREQUEST #14496.

💬 Proposed Commit Message

Fix Grok parsing failure for LOGOUT and HTTPREQUEST messages of #14496  
Add additional grok patterns to handle AAATM log format more robustly

🕵️ Root Cause: Citrix ADC Grok Failures (LOGOUT & HTTPREQUEST)

🔴 1. AAATM LOGOUT Event Failure

❗️ Failed Log

<134> 07/09/2025:19:04:20 GMT test-ns-internal-b 0-PPE-0 : default AAATM LOGOUT 316696334 0 : Context flsmsp0154@hs.mydomain.com@175.16.199.1 - SessionId: 40982902 - User flsmsp0154@hs.mydomain.com - Client_ip 175.16.199.1 - Nat_ip \"Mapped Ip\" - Vserver 0.0.0.0:0 - Start_time \"07/09/2025:18:28:20 GMT\" - End_time \"07/09/2025:19:04:20 GMT\" - Duration 00:36:00  - Http_resources_accessed 0 - Total_TCP_connections 0 - Total_policies_allowed 2 - Total_policies_denied 0 - Total_bytes_send 0 - Total_bytes_recv 0 - Total_compressedbytes_send 0 - Total_compressedbytes_recv 0 - Compression_ratio_send 0.00% - Compression_ratio_recv 0.00% - LogoutMethod \"TimedOut\" - Group(s) \"N/A\"

❌ Original Parsing Issues

  • Pattern mismatch due to missing optional fields NonHttp_services_accessed & Total_UDP_flows
  • Complex username structure not matched
  • Quote escaping inconsistencies

✅ Resolution

Added consolidated pattern supporting:

  • Resolved Domain extraction from complex usernames
  • Optional fields for the missing fields [SSLVPN vs AAATM]
  • Escaped/unescaped quotes
  • Improved Spacing pattern , that were causing errors due to certain missed values, by using {SPACE} instead of , this allows for duplicate spaces , which have appeared in the wild
  • Improved Duration parsing, partly due to spacing management

🔴 2. AAATM HTTPREQUEST Event Failure

❗️ Failed Log

<135> 07/09/2025:19:48:23 GMT test-ns-internal-b 0-PPE-0 : default AAATM HTTPREQUEST 318039900 0 : Context 48x0529@hs.mydomain.com@175.16.199.1 - SessionId: 39526570 - test.customdomain.com User 48x0529@hs.mydomain.com : Group(s) N/A : Vserver 67.43.156.1:8085 - 07/09/2025:19:48:23 GMT : SSO is OFF : CONNECT test.customdomain.com:443 - -

❌ Original Parsing Issues

  • Unexpected hostname in log structure
  • Complex user patterns not handled
  • Misconfigured timezone, method, path fields

✅ Resolution

New flexible pattern supports:

  • Hostname capture
  • Optional timezone & SSO fields [this ensures other pipelines like waf-native don't break]
  • Robust username and domain parsing
  • Improved Path parsing

📊 Summary of Pattern Improvements

Field AAATM LOGOUT AAATM HTTPREQUEST ✅ Resolution
Context-based start ✅ Yes ✅ Yes ✅ Handled
Domain extraction ❌ Missing ❌ Missing ✅ Added
Optional SSLVPN fields ❌ Expected ❌ N/A ✅ Optional
Hostname token before User ❌ Ignored ❌ Ignored ✅ Captured
Timezone extraction N/A ❌ Missing ✅ Optional
Quote escaping ❌ Rigid ❌ Rigid ✅ Flexible

🎯 Benefits

  • Richer structured fields (domain, hostname, timezone)
  • Better resilience against log format drift
  • Clean backward compatibility with existing patterns
  • Improves older sample logs to parse Duration & Timezone more cleanly due to previously neglected spacing

✅ Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices.

🧑‍💻 Author's Checklist

  • Added grok patterns for LOGOUT message variations
  • Added grok patterns for HTTPREQUEST message variations
  • Updated changelog with bugfix entry
  • Tested patterns against failing log samples

🧪 Testing

  1. Added test log samples that previously failed
  2. Run integration test
  3. Confirm logs now parse without Grok failures
78F76546-4F61-4051-AC0B-A0BEA2B736C2

🔗 Related Issues

Closes #14496 — [Citrix ADC]: Grok Parsing Failure for LOGOUT & HTTPREQUEST


🔗 Resources

https://developer-docs.netscaler.com/en-us/netscaler-syslog-message-reference/current-release

@yahyaghani yahyaghani requested a review from a team as a code owner August 2, 2025 12:00
yahyaghani added a commit to yahyaghani/integrations that referenced this pull request Aug 2, 2025
@yahyaghani yahyaghani requested a review from a team as a code owner August 2, 2025 12:04
@andrewkroh andrewkroh added Integration:citrix_adc Citrix ADC Team:Security-Deployment and Devices DEPRECATED Deployment and Devices Security team [elastic/sec-deployment-and-devices] labels Aug 4, 2025
@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

@taylor-swanson taylor-swanson added the Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience] label Aug 4, 2025
Copy link
Copy Markdown
Contributor

@ilyannn ilyannn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid start, but we need to figure out if any pattern changes, such as changes in optionality of the fields, are intended or not and whether they are what we want.

@andrewkroh andrewkroh added Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] bugfix Pull request that fixes a bug issue labels Aug 5, 2025
@yahyaghani yahyaghani requested a review from a team as a code owner August 7, 2025 23:51
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@yahyaghani yahyaghani force-pushed the 14496-citrix-adc-grok-processing branch from 6a2f1a3 to f6a8bed Compare August 9, 2025 01:59
Copy link
Copy Markdown
Contributor

@ilyannn ilyannn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Please do add the missing end of line.
Would be also good to ask the opinion of people who worked on this integration.

"session_id": "1756710",
"timestamp": "2024-07-12T06:54:39.000Z",
"timezone": "GMT :",
"timezone": "GMT",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a clear bug! We should have caught this earlier.

@ilyannn
Copy link
Copy Markdown
Contributor

ilyannn commented Aug 12, 2025

Please add to the changelog.yml as well.

patterns:
- '^User %{DATA:citrix_adc.log.user} - Client_ip (%{IP:citrix_adc.log.client_ip})? - Nat_ip (%{IP:citrix_adc.log.nat.ip}|"%{DATA}") - Vserver %{IP:citrix_adc.log.vserver.ip}:%{INT:citrix_adc.log.vserver.port} - Start_time "%{DATA:_tmp.start_time}" - End_time "%{DATA:_tmp.end_time}" - Duration %{DATA:citrix_adc.log.duration} - Http_resources_accessed %{INT:citrix_adc.log.http_resources_accessed} - (NonHttp_services_accessed %{INT:citrix_adc.log.non_http_services_accessed} - )?Total_TCP_connections %{INT:citrix_adc.log.total_tcp_connections} - (Total_UDP_flows %{INT:citrix_adc.log.total_udp_flows} - )?Total_policies_allowed %{INT:citrix_adc.log.total_policies_allowed} - Total_policies_denied %{INT:citrix_adc.log.total_policies_denied} - Total_bytes_send %{INT:citrix_adc.log.total_bytes_send} - Total_bytes_recv %{INT:citrix_adc.log.total_bytes_received} - Total_compressedbytes_send %{INT:citrix_adc.log.total_compressed_bytes_send} - Total_compressedbytes_recv %{INT:citrix_adc.log.total_compressed_bytes_recieved} - Compression_ratio_send %{NUMBER:citrix_adc.log.compression_ratio_send}% - Compression_ratio_recv %{NUMBER:citrix_adc.log.compression_ratio_recieved}% - LogoutMethod "%{DATA:citrix_adc.log.logout_method}" - Group\(s\) "%{DATA:citrix_adc.log.groups}" ?$'
- '^(Logout handler : )?Context %{DATA:citrix_adc.log.username}@%{IP} - SessionId: %{NUMBER:citrix_adc.log.session_id} - User %{DATA:citrix_adc.log.user} - Client_ip %{IP:citrix_adc.log.client_ip} - Nat_ip (%{IP:citrix_adc.log.nat.ip}|"%{DATA}") - Vserver %{IP:citrix_adc.log.vserver.ip}:%{INT:citrix_adc.log.vserver.port} - Start_time "%{DATA:_tmp.start_time}" - End_time "%{DATA:_tmp.end_time}" - Duration %{DATA:citrix_adc.log.duration} - Http_resources_accessed %{INT:citrix_adc.log.http_resources_accessed} - NonHttp_services_accessed %{INT:citrix_adc.log.non_http_services_accessed} - Total_TCP_connections %{INT:citrix_adc.log.total_tcp_connections} - Total_UDP_flows %{INT:citrix_adc.log.total_udp_flows} - Total_policies_allowed %{INT:citrix_adc.log.total_policies_allowed} - Total_policies_denied %{INT:citrix_adc.log.total_policies_denied} - Total_bytes_send %{INT:citrix_adc.log.total_bytes_send} - Total_bytes_recv %{INT:citrix_adc.log.total_bytes_received} - Total_compressedbytes_send %{INT:citrix_adc.log.total_compressed_bytes_send} - Total_compressedbytes_recv %{INT:citrix_adc.log.total_compressed_bytes_recieved} - Compression_ratio_send %{NUMBER:citrix_adc.log.compression_ratio_send}% - Compression_ratio_recv %{NUMBER:citrix_adc.log.compression_ratio_recieved}% - LogoutMethod "%{DATA:citrix_adc.log.logout_method}" - Group\(s\) "%{DATA:citrix_adc.log.groups}" ?$'
- '^(Logout handler : )?Context %{DATA:citrix_adc.log.username}@%{IP}%{SPACE}-%{SPACE}SessionId: %{NUMBER:citrix_adc.log.session_id}%{SPACE}-%{SPACE}User %{DATA:citrix_adc.log.user}%{SPACE}-%{SPACE}Client_ip %{IP:citrix_adc.log.client_ip}%{SPACE}-%{SPACE}Nat_ip (%{IP:citrix_adc.log.nat.ip}|\\?"%{DATA}\\?")%{SPACE}-%{SPACE}Vserver %{IP:citrix_adc.log.vserver.ip}:%{INT:citrix_adc.log.vserver.port}%{SPACE}-%{SPACE}Start_time \\?"%{DATA:_tmp.start_time}\\?"%{SPACE}-%{SPACE}End_time \\?"%{DATA:_tmp.end_time}\\?"%{SPACE}-%{SPACE}Duration %{NOTSPACE:citrix_adc.log.duration}%{SPACE}-%{SPACE}Http_resources_accessed %{INT:citrix_adc.log.http_resources_accessed}%{SPACE}-%{SPACE}(?:NonHttp_services_accessed %{INT:citrix_adc.log.non_http_services_accessed}%{SPACE}-%{SPACE})?Total_TCP_connections %{INT:citrix_adc.log.total_tcp_connections}%{SPACE}-%{SPACE}(?:Total_UDP_flows %{INT:citrix_adc.log.total_udp_flows}%{SPACE}-%{SPACE})?Total_policies_allowed %{INT:citrix_adc.log.total_policies_allowed}%{SPACE}-%{SPACE}Total_policies_denied %{INT:citrix_adc.log.total_policies_denied}%{SPACE}-%{SPACE}Total_bytes_send %{INT:citrix_adc.log.total_bytes_send}%{SPACE}-%{SPACE}Total_bytes_recv %{INT:citrix_adc.log.total_bytes_received}%{SPACE}-%{SPACE}Total_compressedbytes_send %{INT:citrix_adc.log.total_compressed_bytes_send}%{SPACE}-%{SPACE}Total_compressedbytes_recv %{INT:citrix_adc.log.total_compressed_bytes_recieved}%{SPACE}-%{SPACE}Compression_ratio_send %{NUMBER:citrix_adc.log.compression_ratio_send}%%{SPACE}-%{SPACE}Compression_ratio_recv %{NUMBER:citrix_adc.log.compression_ratio_recieved}%%{SPACE}-%{SPACE}LogoutMethod \\?"%{DATA:citrix_adc.log.logout_method}\\?"%{SPACE}-%{SPACE}Group\(s\) \\?"%{DATA:citrix_adc.log.groups}\\?" ?$'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good, but using pattern_definitions to break up the pattern into something more manageable could be better.

I think it would reduce the very long line lengths, reduce repetition with the first pattern, and make the first pattern more robust by also giving it the %{SPACE} improvements, so it's probably worth trying to use pattern_definitions

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok sure , I will try to break it up with pattern_definitions

@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

@elastic-sonarqube
Copy link
Copy Markdown

@yahyaghani yahyaghani merged commit a190bc2 into elastic:main Aug 19, 2025
9 checks passed
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

Package citrix_adc - 1.17.1 containing this change is available at https://epr.elastic.co/package/citrix_adc/1.17.1/

tehbooom pushed a commit to tehbooom/integrations that referenced this pull request Nov 19, 2025
…lastic#14496 (elastic#14781)

* [Citrix ADC]Fix Grok parsing failure for LOGOUT and HTTPREQUEST of [elastic#14496]

* Add pattern_definitions to make the long patterns of LOGOUT & HTTPREQUEST more manageable

* Update changelog and manifest
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:citrix_adc Citrix ADC Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience] Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Deployment and Devices DEPRECATED Deployment and Devices Security team [elastic/sec-deployment-and-devices]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Citrix ADC]: Grok Parsing Failure for LOGOUT & HTTPREQUEST

6 participants