Starts Airflow task execution directly from the triggerer

Transcript

1. Starts Air fl ow task execution directly from the triggerer
2. $ cat weilee.py __name__ = 李唯 / Wei Lee __what_i_am_doing__ = [ Software Engineer @ Astronomer, Committer @ Apache Airflow, Volunteers @ PyCon Taiwan, Maintainer of commitizen-tools, ] __github__ = G Lee-W __linkedin__ = l clleew __site__ = p http://wei-lee.me
3. File "weilee.py", line 1 __name__ = 李唯 / Wei Lee ^^^ SyntaxError: invalid syntax $ python weilee.py
5. How did deferrable operators work before this change?
6. What has been changed?
7. What has been changed?
8. What it could be in the future
9. But why?
10. "execute" methods are not always needed This is the only logic when deferrable is set to true and does not need to be executed in the worker.
11. How does it affect DAG authors? More ef fi cient operators to come
12. How does it affect operator authors? A new way to implement operators in an async manner
13. The Original Way
14. The New Way
15. How does it work under the hook?
16. How does it work under the hook?
17. How does it work under the hook?
18. Limitations • It is necessary to go back to workers to run the next_method (e.g., "execute_complete") • This feature does not yet fully support MappedOperator.