Copied to clipboard

Run Automation Test

1. Release Task Inputs

After adding Kobiton Automation Test Executor task to your pipeline, you are required to fill in some inputs. We are going to split them into 3 parts from top to bottom:

Part 1: Required inputs

  • Display name: The Kobiton API Credentials that you created in the previous section. You can add more credentials if you want. For now let's pick one from the dropdown list.

  • Kobiton executor server credentials: The Kobiton Executor Server credentials in your service connnection store.

  • Git repository URL: This is the URL to your GIT repository that contains your test script. If the repository is private, please use the SSH URL.

  • Git repository branch: The branch of the repository to execute automation test script.

  • Git repository SSH key: The SSH Key to access the above Git repository if it is set as private. In this document, we will use Kobiton's public sample repository so we will leave this field empty.

    Image not found

Part 2: Desired Capabilities

  • Upload New App Version:

    • By letting this checked, you are required to upload a new version of your existing app on Kobiton App Repository.

      • App file path: Select an apk/ipa file in your release artifact.

      Image not found

      Image not found

      Image not found

      As an example in the above picture, once the uploading process is successfully completed, a value of “4058” will be parsed to your KOBITON_SESSION_APPLICATION environment variable.

    • If you leave this field unchecked, you have the option to input your existing App ID or a URL to your apk file in App ID input. This value will be directly passed to your KOBITON_SESSION_APPLICATION env variable, so make sure to type in the correct format for “app” field in desiredCaps like below:

      Image not found

  • Use Custom Device: This will check if you want to define the device related parameters in your desiredCaps via env variables.

    • If you let this unchecked, the extension will ignore the env variables.
    • If this is checked, you are required to type in 3 fields which are self-explanatory:

      • Kobiton Cloud Device Name: KOBITON_DEVICE_NAME

      • Device platform version: KOBITON_SESSION_PLATFORM_VERSION

      • Device platform: KOBITON_DEVICE_PLATFORM

      Image not found

Part 3: Execution inputs:

  • Root Directory: The root directory is the path to git your repository.

  • Commands: The command lines to execute your test script. These commands will be called at the root directory of your Git repository. The commands will depend on your test project. In this example, we will use the command “mvn test” to build the project and run automation test script.

  • Wait for execution:

    • If this is unchecked, the release will continue to the next release task. The execution job ID of this task will be printed, but not the console logs or report URL.
    • If this is checked, the task will wait for the execution to be either COMPLETED (commands ran successfully) or FAILED (cannot clone git repository, server error,...) then print the console log and report URL (if it existed).

      • Log Type: User can pick a type of log to be printed. Combined means the output logs and error logs will be printed in chronological order. The Separated option allows you to choose one of the 2 types of logs.
        • It is recommended to log your test case result with appropriate log type. For example, in Java you can use System.err.println to print test case errors, this will be consumed in the error logs while System.out.println will return texts in output logs.
      • Terminate release pipeline on failed job: If this is checked, once the execution job status is FAILED, an error will be returned and the whole release is terminated (All the following release tasks below this task will be skipped).

      Image not found

2. Execute the release

  • Once everything is set up, click on the Save button on the right.

    Image not found

  • Leave everything blank for this time, then click OK.

    Image not found

  • Click Create Release button on the right to create a new release.

    Image not found

  • A new window will appear, leave everything blank and click Create.

    Image not found

  • A notification will appear, click on Release-1 to view the release status.

    Image not found

  • When the release in completed, you can check the release log by clicking on the Logs button right below the release Stage.

    Image not found

  • Click on the Kobiton Automation Test Executor task to view its logs

    Image not found

    Image not found

    If you chose to Wait for execution, in happy cases (no server error, all test cases passed) you can see the test result, execution job ID and report URL

    Image not found

    Image not found

    The Job ID is formatted with prefix “rN” where N is the release ID of your Azure Devops project. Clicking on the Report URL will lead you to the report interface.

  • Please contact us to be provided username, password and domain.

  • After the session is created successfully, we can look up the session in the Kobiton Portal .

    Image not found

3. Overall

Here is a simple case we made to demonstrate how to trigger the automation test script on Kobiton devices.

  • Filling in the inputs

    Image not found

    Image not found

    • Our public git repoitory:

    • In Root directory input of the execution task. You have to change directory to test scripts before run the test.


    • In command input of the execution task to run the test.

      $ mvn test -Dtest=azureTestNG

    • Since this demo repository is public, you can leave the SSH Key field blank and use HTTPS URL in the Kobiton Azure Devops Release Task.

  • Preparing the automation script

    Due to not using custom device, the script will set default value for these following capabilities:

    Environment Description
    KOBITON_USERNAME The kobiton Username you set in Azure Service conneciton
    KOBITON_API_KEY The Kobiton API Key you set in Azure Service connection
    KOBITON_DEVICE_NAME Value of the device name in Kobiton Extension Task
    KOBITON_SESSION_APPLICATION Value of the app in Kobiton Extension Task

    Image not found




You can select a specific word, line of text, or one or more paragraphs to comment.


Please give us some feedback !
Thank you!
  • We are listening,
  • we will do our best to fix this soon