Skip to content

Commit

Permalink
Migrate Google example life_sciences to new design AIP-47 (#25264)
Browse files Browse the repository at this point in the history
related: #22447, #22430
  • Loading branch information
chenglongyan authored Jul 26, 2022
1 parent 1eb91b9 commit 28db8c1
Show file tree
Hide file tree
Showing 5 changed files with 30 additions and 48 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,15 @@ Pipeline Configuration
In order to run the pipeline, it is necessary to configure the request body.
Here is an example of the pipeline configuration with a single action.

.. exampleinclude:: /../../airflow/providers/google/cloud/example_dags/example_life_sciences.py
.. exampleinclude:: /../../tests/system/providers/google/cloud/life_sciences/example_life_sciences.py
:language: python
:dedent: 0
:start-after: [START howto_configure_simple_action_pipeline]
:end-before: [END howto_configure_simple_action_pipeline]

The pipeline can also be configured with multiple action.

.. exampleinclude:: /../../airflow/providers/google/cloud/example_dags/example_life_sciences.py
.. exampleinclude:: /../../tests/system/providers/google/cloud/life_sciences/example_life_sciences.py
:language: python
:dedent: 0
:start-after: [START howto_configure_multiple_action_pipeline]
Expand All @@ -59,7 +59,7 @@ Use the
:class:`~airflow.providers.google.cloud.operators.life_sciences.LifeSciencesRunPipelineOperator`
to execute pipelines.

.. exampleinclude:: /../../airflow/providers/google/cloud/example_dags/example_life_sciences.py
.. exampleinclude:: /../../tests/system/providers/google/cloud/life_sciences/example_life_sciences.py
:language: python
:dedent: 0
:start-after: [START howto_run_pipeline]
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
# under the License.
import pytest

from airflow.providers.google.cloud.example_dags.example_life_sciences import BUCKET
from tests.providers.google.cloud.utils.gcp_authenticator import GCP_GCS_KEY
from tests.system.providers.google.cloud.life_sciences.example_life_sciences import BUCKET
from tests.test_utils.gcp_system_helpers import CLOUD_DAG_FOLDER, GoogleSystemTest, provide_gcp_context


Expand Down
16 changes: 16 additions & 0 deletions tests/system/providers/google/cloud/life_sciences/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
from airflow import models
from airflow.providers.google.cloud.operators.life_sciences import LifeSciencesRunPipelineOperator

ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
DAG_ID = "example_gcp_life_sciences"

PROJECT_ID = os.environ.get("GCP_PROJECT_ID", "example-project-id")
BUCKET = os.environ.get("GCP_GCS_LIFE_SCIENCES_BUCKET", "INVALID BUCKET NAME")
FILENAME = os.environ.get("GCP_GCS_LIFE_SCIENCES_FILENAME", 'input.in')
Expand Down Expand Up @@ -74,7 +77,7 @@
# [END howto_configure_multiple_action_pipeline]

with models.DAG(
"example_gcp_life_sciences",
DAG_ID,
schedule_interval='@once',
start_date=datetime(2021, 1, 1),
catchup=False,
Expand All @@ -95,3 +98,9 @@
)

simple_life_science_action_pipeline >> multiple_life_science_action_pipeline


from tests.system.utils import get_test_run # noqa: E402

# Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
test_run = get_test_run(dag)

0 comments on commit 28db8c1

Please sign in to comment.