We Are Going To Discuss About What is context variable in Airflow operators. So lets Start this Python Article.
What is context variable in Airflow operators
- How to solve What is context variable in Airflow operators
When Airflow runs a task, it collects several variables and passes these to the
context
argument on theexecute()
method. These variables hold information about the current task, you can find the list here: https://airflow.apache.org/docs/apache-airflow/stable/macros-ref.html#default-variables.
Information from the context can be used in your task, for example to reference a folderyyyymmdd
, where the date is fetched from the variableds_nodash
, a variable in thecontext
:def do_stuff(**context): data_path = f"/path/to/data/{context['ds_nodash']}" # write file to data_path... PythonOperator(task_id="do_stuff", python_callable=do_stuff)
*context
and**context
are different Python notations for accepting arguments in a function. Google for “args vs kwargs” to find more on this topic. Basically*context
accepts non-keyword arguments, while**context
takes keyword arguments:def print_context(*context_nokeywords, **context_keywords): print(f"Non keywords args: {context_nokeywords}") print(f"Keywords args: {context_keywords}") print_context("a", "b", "c", a="1", b="2", c="3") # Non keywords args: ('a', 'b', 'c') # Keywords args: {'a': '1', 'b': '2', 'c': '3'}
- What is context variable in Airflow operators
When Airflow runs a task, it collects several variables and passes these to the
context
argument on theexecute()
method. These variables hold information about the current task, you can find the list here: https://airflow.apache.org/docs/apache-airflow/stable/macros-ref.html#default-variables.
Information from the context can be used in your task, for example to reference a folderyyyymmdd
, where the date is fetched from the variableds_nodash
, a variable in thecontext
:def do_stuff(**context): data_path = f"/path/to/data/{context['ds_nodash']}" # write file to data_path... PythonOperator(task_id="do_stuff", python_callable=do_stuff)
*context
and**context
are different Python notations for accepting arguments in a function. Google for “args vs kwargs” to find more on this topic. Basically*context
accepts non-keyword arguments, while**context
takes keyword arguments:def print_context(*context_nokeywords, **context_keywords): print(f"Non keywords args: {context_nokeywords}") print(f"Keywords args: {context_keywords}") print_context("a", "b", "c", a="1", b="2", c="3") # Non keywords args: ('a', 'b', 'c') # Keywords args: {'a': '1', 'b': '2', 'c': '3'}
Solution 1
When Airflow runs a task, it collects several variables and passes these to the context
argument on the execute()
method. These variables hold information about the current task, you can find the list here: https://airflow.apache.org/docs/apache-airflow/stable/macros-ref.html#default-variables.
Information from the context can be used in your task, for example to reference a folder yyyymmdd
, where the date is fetched from the variable ds_nodash
, a variable in the context
:
def do_stuff(**context):
data_path = f"/path/to/data/{context['ds_nodash']}"
# write file to data_path...
PythonOperator(task_id="do_stuff", python_callable=do_stuff)
*context
and **context
are different Python notations for accepting arguments in a function. Google for “args vs kwargs” to find more on this topic. Basically *context
accepts non-keyword arguments, while **context
takes keyword arguments:
def print_context(*context_nokeywords, **context_keywords):
print(f"Non keywords args: {context_nokeywords}")
print(f"Keywords args: {context_keywords}")
print_context("a", "b", "c", a="1", b="2", c="3")
# Non keywords args: ('a', 'b', 'c')
# Keywords args: {'a': '1', 'b': '2', 'c': '3'}
Original Author Bas Harenslak Of This Content
Conclusion
So This is all About This Tutorial. Hope This Tutorial Helped You. Thank You.