Skip to content

Use Task SDK BaseHook in Amazon executors#62090

Open
dylanschw wants to merge 2 commits intoapache:mainfrom
dylanschw:fix-58205-ecs-command-as-list
Open

Use Task SDK BaseHook in Amazon executors#62090
dylanschw wants to merge 2 commits intoapache:mainfrom
dylanschw:fix-58205-ecs-command-as-list

Conversation

@dylanschw
Copy link

Closes #58205

What

Update Amazon Batch and ECS executors to use the Task SDK BaseHook import path. (Update Amazon Batch and ECS executors to use the Task SDK BaseHook import path instead of the deprecated airflow.hooks.base.BaseHook.)

Why

Avoid deprecated BaseHook import usage and align these executors with the Airflow Task SDK hook location. The previous import path is deprecated. This change aligns the Amazon executors with the current Task SDK structure and removes usage of deprecated core hook imports.

Tests

Verified by running Amazon provider executor unit tests:

AIRFLOW__SECRETS__BACKEND="airflow.secrets.environment_variables.EnvironmentVariablesBackend"
AIRFLOW_CONN_SSH_DEFAULT='ssh://airflowtest:airflowtest@127.0.0.1:22?extra={"look_for_keys":false,"allow_agent":false,"allow_host_key_change":true}'
PYTHONPATH="$PWD/airflow-core/src:$PWD/task-sdk/src:$PWD/devel-common/src:$PWD/dev:$PWD:$PROVIDERS_SRC"
.venv-breeze/bin/pytest -q providers/amazon/tests -k "batch_executor or ecs_executor"

All relevant Batch and ECS executor tests pass.

@dylanschw dylanschw requested a review from o-nikolas as a code owner February 17, 2026 19:17
@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels Feb 17, 2026
@boring-cyborg
Copy link

boring-cyborg bot commented Feb 17, 2026

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Copy link
Contributor

@SameerMesiah97 SameerMesiah97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs some work. I’m not sure this change is fully backwards compatible, given as this introduces Task SDK APIs that only exist in Airflow 3.x.

The tests also seem incomplete. At a minimum, the construction of the command (both for Airflow ≥3.0 and Airflow <3.0) should be covered by new tests. I can also see existing tests, which are currently skipped for ECS and Batch executors (test_try_adopt_task_instances) that might be worth revisiting, especially since the regression occurred in that path.

"airflow.sdk.execution_time.execute_workload",
"--json-string",
workload_json,
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this compatible with airflow 2.x? It seems like this will not work for versions before Airflow 3.0 as airflow.sdk was introduced then. I think you should have a look at PR #58207. There are issues with that but I think the implementation in that PR was more backwards compatible.

class AwsEcsExecutor(BaseExecutor):
"""Executor that runs Airflow tasks on AWS ECS."""

def _ti_to_execute_workload_cmd(self, ti):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this functionality can be included in the executor/utils folder to avoid duplication.. I do not think this is a massive issue as it is only duplicated once but I think it should be considered. I would wait for the codeowner to weigh in here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ECS Executor in apache-airflow-providers-amazon 9.16.0 incompatible with Airflow 3.x - calls removed method command_as_list()

2 participants