Run an automation test using Azure DevOps

Learn how to run an automation test using Azure DevOps.

Before you start

You’ll need to complete the following:

Add the task to your YAML file

In your Azure DevOps project, select Pipelines, then Release.

Select Pipelines then Release on Azure DevOps

Create a new release pipeline, or edit an existing one.

Select your release pipeline

Under Pipeline, add or edit a stage.

Add or edit a stage in the release pipeline

If you are adding a new stage, select Empty job, then Apply.

Select empty job as a template

Switch to the Tasks pane of the new stage, then add a task to Agent job.

Switch to Tasks pane then add a task to Agent job

Search for Kobiton Automation Test Executor and select Add.

Search for Kobiton Automation Test Executor and select Add

The task details display as below:

The Kobiton Automation Test Executor details screen

Use the following table to configure the task:

Parameter Required Environment Variable(s) Description

Display name

Yes

N/A

The release pipeline’s display name.

Kobiton API credentials

Yes

KOBITON_USERNAME
KOBITON_API_KEY

Your Kobiton API credentials.

Kobiton executor server credentials

Yes

N/A

Your Kobiton executor server credentials.

Git repository URL

Yes

N/A

The URL for your project’s cloud repository.

Git repository branch

Yes

N/A

The release branch for your project’s cloud repository.

Git repository SSH key

Yes

N/A

The SSH key for your project’s cloud repository.

Upload New App Version

No

KOBITON_SESSION_APPLICATION

Upload and install a new version of an existing app during the test session. If selected, enter the following:

  • App file path: the path from your project root to your .apk or .ipa file.

  • App ID: your app’s Kobiton App ID. Assigned to the KOBITON_SESSION_APPLICATION environment variable.

App ID/URL

No

KOBITON_SESSION_APPLICATION

Your app’s Kobiton App ID. Requires Upload New App Version to be unchecked.

Use Custom Device

No

KOBITON_DEVICE_NAME
KOBITON_SESSION_PLATFORM_VERSION
KOBITON_DEVICE_PLATFORM_NAME

Use a specific device from your devices. If selected, enter the following:

  • Kobiton Cloud Device Name: the device’s name in Kobiton. Assigned to the KOBITON_DEVICE_NAME environment variable.

  • Device platform version: the device’s platform version. Assigned to the KOBITON_SESSION_PLATFORM_VERSION environment variable.

  • Device platform: the device’s platform version, either Android or iOS. Assigned to the KOBITON_DEVICE_PLATFORM_NAME environment variable.

Root directory

No

N/A

Description

Commands

Yes

N/A

The command(s) to install project dependencies and run your test script.

Wait for execution

No

N/A

Wait for your automation script(s) to finish before starting the release pipeline. If selected, choose the following:

  • Log type: choose Combined to display all logs, or Separated to display either output or error logs only.

  • Terminate release pipeline on failed job: terminate the pipeline if the automation script fails.

Review your task and select Save.

Review the filled-in details and select Save

Run your automation test

Select your release pipeline and create a new release.

Select your release and choose Create release

If you want to test out the release pipeline and do not want it to start automatically, select a stage to manually trigger it before selecting Create.

Open the new release, highlight the stage with the Kobiton Automation Test Executor task and select Deploy.

Make sure you have an agent available before deploying.

Hover the mouse on the stage and select Deploy

After the deployment starts, highlight the stage and select Logs.

Highlight the deployment in progress and select Logs

Wait for the deployment to complete and view the automation test result under Kobiton Automation Test Executor logs.

Wait for the deployment to complete then view the logs of Kobiton Automation Test Executor