https://github.com/PrefectHQ/prefect
https://docs.prefect.io/
Prefect
We've rebuilt data engineering for the data science era.
Prefect is a new workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine. Users organize
Tasks into Flows, and Prefect takes care of the rest.
Demo code
Prefect's Pythonic API should feel familiar for newcomers. Mark functions as tasks and call them on each other to build up a flow.
from prefect import task, Flow, Parameter
@task(log_stdout=True)
def say_hello(name):
print("Hello, {}!".format(name))
with Flow("My First Flow") as flow:
name = Parameter('name')
say_hello(name)
flow.run(name='world') # "Hello, world!"
flow.run(name='Marvin') # "Hello, Marvin!"
安装
https://docs.prefect.io/core/getting_started/installation.html
https://docs.docker.com/compose/install/
发布
https://docs.prefect.io/orchestration/concepts/flows.html#registration
https://docs.prefect.io/api/latest/cli/create.html
prefect create project "my project"
prefect register --project "my project" --path test.py
本文版权归作者所有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接。
