Interaction with interface elements

Principles of work with interface elements in applications

Introduction

To interact with application elements, you have to find and identify them first.

The platform helps you to do this in several ways:

  • The 'Elements' tab allows to create, view or modify interface elements.

  • The 'Desktop Picker' allows you to pick elements from the application interface directly.

Interface

The figure below shows the interface of the "ElectroNeek Studio" program with the active "Elements" tab.

To create a new element:

  1. Click on the "Add" button

  2. Specify a name for the element

  3. Click on the "Change the element" button

After that, the 'Desktop picker' window will appear, where you can select elements in applications.

To select an element in the application, press the "Pick" button in the "Desktop picker" window and point the cursor at the necessary element in the application.

When pointing the cursor, the "Desktop picker" program illuminates this element with a red frame. To store the selection of the element you should press the Ctrl key. To check if the element was selected correctly, press the "Test" button - the element will be highlighted in a yellow frame.

To save all the settings you should press the "Save" button. The new element will be saved on the "Elements" tab.

To use this element select an activity from the "Interface elements" section and select the element from the list in the function settings.

Thus, when developing the workflow, you can select the elements in the applications and refer to them later by selecting the name from the list.

Advanced options

In the Desktop Picker you can open 'Advanced options' and select several attributes to identify the element. The basic attributes are 'Type', 'ClassName', 'Name', 'Value', etc.

The main goal is to select as many attributes as it needed to select a unique element. But avoid to select an attribute that could change in future.

Example

Let's read a Windows Calculator's result field:

If we select this elements and check the attributes we will see, that three of them are selected by default: Type, ClassName and Name

But as you can notice, the Name attribute contains the value of this field, it says 'Display is 0'. If we change the value of the Calculator results field to another value and press 'Test' button, the Picker will show that 0 elements are found. Because 'Name' value has changed and the attributes don't match anymore.

Let's uncheck the Name attribute and check the Index. Let's test again:

Now we have unique selection and we can use this element to read Calculator Result field in Get Element Property activity.

Selector Tree

At the bottom of the Desktop Picker you can see several tabs of selectors tree. In this example the selected button is a child of a group, which is a child of another group, which is a child of a window, etc.

We can specify the attributes of each parent of selected element by choosing corresponding tabs.

Dynamic search

Sometimes you might need to select an element that has attributes changing through workflow execution.

That is possible using various options listed in Calculation column.

Calculate
Wildcard
Contains
Calculate

Allows to use a variable in a attribute value. For example you can set an Index attribute value equal to a variable index_value.

Wildcard

Allows to use following symbols to set a value:

Symbols

Searching for

?

Any single symbol

*

Any symbols (or none)

#

Any digit (0-9)

[abcd]

Any symbol from abcd

[!abcd]

Any symbol outside of abcd

Contains

Allows to set a string the attribute should contain.