Use natural language locators in Appium AI

Before you begin

Ensure the following:

  • Your environment is configured to support natural language locators. If you are unsure, contact your lab administrator or Kobiton Support.

  • You are familiar with basic Appium commands such as findElement(…​)

  • You are using a private device deployment (natural language locators are not available for Public Cloud devices)

Use a natural language locator

To locate an element using natural language, pass a descriptive string into findElement(…​) using the "natural" locator strategy.

python
element = driver.find_element("natural", "The login button at the bottom")
element.click()
java
WebElement el = driver.findElement(ByNatural.natural("the Accessibility button"));
el.click();
The above Java example used the custom ByNatural class to extend the default custom locator strategy value. See this guide for instructions.

Write effective descriptions

Use clear, specific descriptions to help identify the correct element.

Good examples:

java
driver.findElement(ByNatural.natural("the Accessibility button"));
driver.findElement(ByNatural.natural("the Username field"));
driver.findElement(ByNatural.natural("the City dropdown list"));

Less effective examples:

driver.findElement(ByNatural.natural("Click button")
driver.findElement("Select item")

When multiple similar elements are present, include additional context such as position, label, or surrounding UI to improve accuracy.

Expected behavior

Appium AI evaluates the description and returns the most relevant matching element, using either metadata from the view hierarchy or vision-based analysis of a screenshot, depending on what the application exposes.

Results are not guaranteed to be deterministic. If multiple elements match the description, you may need to refine the description to improve accuracy.

Natural language locators work alongside traditional selector strategies and are best used in combination with them, rather than as a complete replacement.

Troubleshooting

If an element is not found or the wrong element is returned:

  • Use more specific language in the description

  • Include additional context such as position, labels, or surrounding elements

  • Verify that the UI element is visible and accessible in the view hierarchy

  • For metadata-based resolution, confirm that the application exposes sufficient UI metadata (view hierarchy or accessibility attributes) for element identification

  • For vision-based resolution, ensure the target element is clearly visible on the current screen

If natural language locators are not working as expected, contact your lab administrator to verify the following:

  • OpenAI or Azure OpenAI API credentials are configured correctly

  • Natural language locators are properly configured in your environment

  • The deployment uses private devices (natural language locators are not available on Public Cloud devices)

If issues persist, contact Kobiton Support for assistance.

Next steps

For an overview of how natural language locators work and their limitations, see the Appium AI guide.