Skip to content

A helper function for simple HTML tag removal #659

Closed

Description

@janbuchar

See https://github.com/apify/crawlee/blob/master/packages/utils/src/internals/cheerio.ts#L42-L85 for inspiration.

The Python version should probably use BeautifulSoup and tell the user to install crawlee[beautifulsoup] if they try to use it without dependencies.

Activity

  1. added
    enhancementNew feature or request.
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Nov 6, 2024
  2. Pijukatel commented on Dec 8, 2024

    @Pijukatel
    Collaborator

    I think that BS should already handle it using: soup.get_text("\n", strip=True)
    https://www.crummy.com/software/BeautifulSoup/bs4/doc/#get-text . So maybe I should just wrap that in our utility function.

  3. added a commit that references this issue on Dec 18, 2024
    2b9d970
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions