You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Sitemap.from_xml_string cannot opt into enqueue host filtering #2118
Found while triaging GHSA-c533-9w4q-667c (which I'm closing as informative - a raw XML string has no origin, so there's no host policy to bypass). It does point at a real API gap though.
Sitemap.from_xml_string takes only content and hardcodes SitemapSource(type='raw', content=content) - no url, no ParseSitemapOptions
so if you have raw sitemap XML and you know where it came from, there's no way to say so - filtering is silently skipped and the same-hostname default never applies
explicitly not worth doing: rejecting <loc> values when the origin is unknown - that makes the default behaviour return an empty list
while in there: _get_origin_url stamps raw sources with raw://<sha256>, so SitemapUrl.origin_sitemap_url is a pseudo-URL that would fail scheme validation if anyone fed it back into filter_url - harmless today, footgun later
While tracing the advisory I noticed most of the plumbing for this already exists, in case it helps whoever picks it up.
SitemapSource already declares url as NotRequired, and the filter in _process_sitemap_item keys on source.get('url') rather than on the source type, so a raw source that carries a url would be filtered today with no change to the filtering code. What blocks it is only that from_xml_string hardcodes the source:
Two things worth deciding before anyone writes it, since they change behaviour beyond filtering:
_get_origin_url() returns raw://<sha256> for any raw source. If a caller declares an origin, should SitemapUrl.origin_sitemap_url become that URL instead, or keep the content hash so raw sources stay distinguishable in output?
That same value is the dedup key for nested sitemaps. Switching it for url-bearing raw sources means a raw sitemap and the fetched copy of the same sitemap would dedup against each other, which may be what you want or may not.
Happy to send the PR if you tell me which way you want those two, otherwise feel free to use any of this.
Activity
arpitjain099 commentedon Aug 6, 2026
While tracing the advisory I noticed most of the plumbing for this already exists, in case it helps whoever picks it up.
SitemapSourcealready declaresurlasNotRequired, and the filter in_process_sitemap_itemkeys onsource.get('url')rather than on the source type, so a raw source that carries aurlwould be filtered today with no change to the filtering code. What blocks it is only thatfrom_xml_stringhardcodes the source:so the shape could be as small as an optional origin plus the options passthrough that is currently missing:
Two things worth deciding before anyone writes it, since they change behaviour beyond filtering:
_get_origin_url()returnsraw://<sha256>for any raw source. If a caller declares an origin, shouldSitemapUrl.origin_sitemap_urlbecome that URL instead, or keep the content hash so raw sources stay distinguishable in output?Happy to send the PR if you tell me which way you want those two, otherwise feel free to use any of this.
fix(sitemaps): allow Sitemap.from_xml_string to opt into enqueue host…