Skip to content

Template/Docker Build Failing After Recent Fix #975

Closed

Description

@MMf-mmf

Template Build Failing After Recent Fix

Related Issue

This is a follow-up to issue #972 which addressed similar template creation issues.

Description

While the previous fixes have been implemented, the BeautifulSoup template is still experiencing build failures during the deployment process. The error appears to be related to Playwright dependency management during the Docker build process.

Current Version

  • crawlee-0.5.5

Steps to Reproduce

  1. Run crawlee create my_crawler
  2. cd my_crawler
  3. Run apify init
  4. Run apify push

Error Output

2025-02-11T16:05:56.771Z /bin/bash: line 1: playwright: command not found

ERROR: Invalid requirement: 'Installing dependencies: cat requirements.txt sed s/^playwright==.*/playwright==/': Expected end or semicolon (after name and no valid version specifier)
Installing dependencies: cat requirements.txt sed s/^playwright==.*/playwright==/
^ (from line 1 of /dev/stdin)
Hint: It looks like a path. File 'Installing dependencies: cat requirements.txt sed s/^playwright==.*/playwright==/' does not exist.

---> Removed intermediate container 8fbe002991e1
ACTOR: ERROR: Docker image build failed: The command '/bin/bash -o pipefail -c echo "Python version:" && python --version && echo "Installing dependencies:" cat requirements.txt sed "s/^playwright==.*/playwright==$(playwright --version | cut -d ' ' -f 2)/" | pip install -r /dev/stdin --no-dependencies && echo "All installed Python packages:" && pip freeze' returned a non-zero code: 1

Expected Behavior

The BeautifulSoup template should build and deploy successfully following the standard documentation steps.

Actual Behavior

The build process fails during dependency installation, specifically when trying to handle Playwright-related dependencies. This occurs even though BeautifulSoup template was selected (not Playwright).

Additional Notes

  • This issue persists after the fixes implemented in fix: Fix playwright template and dockerfile #972
  • The error suggests there might be an issue with the below (although it should work regardless of the template selected?):
    1. Playwright dependencies being included in BeautifulSoup template
    2. The dependency installation script's handling of requirements.txt
    3. The Docker build process attempting to use Playwright commands regardless of template choice
    4. when the apify init command is run there is a Warning message: "Warning: The current directory does not look like a Node.js or Python project." is this the expected output?

Questions for Maintainers

  1. Is there a configuration step missing from the documentation?

Activity

  1. added
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Feb 11, 2025
  2. janbuchar commented on Feb 11, 2025

    @janbuchar
    Collaborator

    Interesting, could you share the exact options that you picked in the project creation dialog? I recall pushing the BeautifulSoupCrawler bootstrapped project just after today's fix.

  3. MMf-mmf commented on Feb 12, 2025

    @MMf-mmf
    Author

    $ crawlee create my_crawler

    [?] Please select the Crawler type:
    [✓] Beautifulsoup
    [ ] Parsel
    [ ] Playwright
    [ ] Playwright-camoufox

    [?] Please select the HTTP client:
    [✓] Httpx
    [ ] Curl-impersonate

    [?] Please select the package manager:
    [ ] Poetry
    [✓] Pip
    [ ] Manual

    [?] Please specify the start URL: https://crawlee.dev

    [?] Should Apify integration be set up for you? (y/N): y

  4. self-assigned this
    on Feb 12, 2025
  5. added a commit that references this issue on Feb 12, 2025
    1e3b8eb
  6. MMf-mmf commented on Feb 18, 2025

    @MMf-mmf
    Author

    @janbuchar unfortunately, after pulling the latest releases, the tool (crawlee create my_crawler) is still not working,

    would you be so kind as to provide me with the exact set up you have and use to push new crawlers to apify

  7. MMf-mmf commented on Feb 18, 2025

    @MMf-mmf
    Author

    Note: after trying to use the builder to build the other options such as the Playwright option as well as when selecting the poetry option as the package manager there where template building errors that did not allow the tool to even build a starter project

  8. janbuchar commented on Feb 18, 2025

    @janbuchar
    Collaborator

    @MMf-mmf you need to pull the latest beta release, the stable one doesn't have the fix yet.

  9. MMf-mmf commented on Feb 18, 2025

    @MMf-mmf
    Author

    @janbuchar correct it was running on version 0.6.0

  10. janbuchar commented on Feb 18, 2025

    @janbuchar
    Collaborator

    which beta version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions