Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Prefect #1589

Merged
merged 1 commit into from
Sep 22, 2020
Merged

Add Prefect #1589

merged 1 commit into from
Sep 22, 2020

Conversation

karmijo
Copy link
Contributor

@karmijo karmijo commented Aug 11, 2020

What is this Python project?

Prefect is a workflow orchestration framework that takes care of a lot of the "negative engineering" aspects of building and scheduling workflows:

  • intuitive scheduling interface
  • first-class caching
  • state management of tasks
  • parametrization of workflows
  • ability to scale on most platforms

What's the difference between this Python project and similar ones?

  • Pythonic interface
  • can actually handle scaling (first-class Dask integrations too!)
  • dynamic pipelines (parameters, mapping, looping, etc.)
  • lightweight, easy-to-use and flexible

--

Anyone who agrees with this pull request could submit an Approve review to it.

@@ -799,6 +799,7 @@ Inspired by [awesome-php](https://github.com/ziadoz/awesome-php).
* [gunnery](https://github.com/gunnery/gunnery) - Multipurpose task execution tool for distributed systems with web-based interface.
* [Joblib](https://joblib.readthedocs.io/) - A set of tools to provide lightweight pipelining in Python.
* [Plan](https://github.com/fengsp/plan) - Writing crontab file in Python like a charm.
* [Prefect](https://github.com/PrefectHQ/prefect) - A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve

Copy link

@mpancia mpancia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great package to include.

Copy link

@abtrout abtrout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect addition.

Copy link

@akontaxis akontaxis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great package to add!

Copy link

@sbreidbach-edr sbreidbach-edr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great project

@gaby
Copy link

gaby commented Sep 22, 2020

+1 great!

@vinta vinta merged commit 9be5dee into vinta:master Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.