Mouse Events and Control

This section describes the Eggplant Functional commands and functions that execute mouse events or return information about the mouse cursor on the SUT. Many of these commands and functions, including Click, DoubleClick, Drag, Drop, DragandDrop, MouseButtonDown, and MouseLocation(), work against mobile SUTs as well as desktop SUTs, which is helpful when creating scripts that automate both mobile and desktop platforms. SenseTalk also includes a set of mobile commands and functions, a subset of which (Tap, DoubleTap, Press, etc.) are compatible with desktop SUTs.

Click Command

Note: This code definition explains the use of the Click command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: Click Command.

Behavior: Clicks the SUT mouse on the hot spot (Image) or center (OCR) of the first location found. If more than one image or text string is passed, Eggplant Functional searches for images in the order in which they are listed.

Tip: The Tap and Click commands perform the same event and can be used against both desktop and mobile SUTs.

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Example:

repeat with each item of ["Option1","Option2","Option3"] //Repeats through a list of images

click it//Clicks the hot spot of the current image in the list

end repeat

Example:

click{text:"Tweet",waitFor:10,searchRectangle:["AnchorLeft","AnchorRight"]} //Clicks the center of string "Tweet", waiting up to ~10 seconds for it to appear with the specified searchRectangle

Example:

Use code like this to select a section of items, such as files in File Explorer/Finder, and then copy the files

Click "FirstEntry" //Clicks the hot spot of image "FirstEntry"

KeyDown ShiftKey //Holds down the shift key

Click "LastEntry" //Clicks the hot spot of image "LastEntry"

KeyUp ShiftKey //Releases the shift key

RightClick //Right clicks at the same location as the previous mouse event

Click "CopyMenuItem"

Example:

Sometimes searching for an image twice, once before the click, is helpful when automating UIs that shift as their content loads

WaitFor 10, "NewTweet" //Waits for the image NewTweet to appear

Click{Image:"NewTweet",HotSpot:[80,0]} //Searches for NewTweet again and sends the click 80 pixels to the right of the image's upper left corner. Note the addition of the "image:" property when including the HotSpot image property

Example:

if ImageFound(text:"Cancel") then click FoundImageLocation() //Clicks the location of the center of the found string "Cancel"

Example:

Click the RemoteScreenSize*.5 //Clicks the center point of the screen

Example:

//Use code like this to perform a triple click event

on TripleClick imageToClick //Declares a custom command handler named "TripleClick" with parameter "imageToClick"

//The next 3 lines change the timing between Eggplant Functional commands so that 3 quick clicks can be sent in succession

put the RemoteWorkInterval into defaultRWI //Stores the current Remote Work Interval value so that it can be restored later

set the RemoteWorkInterval to .01 //Sets the RemoteWorkInterval to .01 seconds, allowing events to be sent to the SUT much more rapidly

//These 3 lines click the image once and then click twice more in the same location, at the interval specified by the RemoteWorkInterval

Click imagetoClick //Sends a click at the hot spot location of the image passed in as a parameter

//The subsequent 2 clicks occur at the same location as the previous click

Click

Click

set the RemoteWorkInterval to defaultRWI //Restores the RemoteWorkInterval to the original value

end TripleClick

DoubleClick Command

Note: This code definition explains the use of the DoubleClick command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: DoubleClick Command.

Behavior: Double-clicks the SUT mouse on the hot spot (Image) or center (OCR) of the first location found. If more than one image or text string is passed, Eggplant Functional searches for images in the order in which they are listed.

Tip: The DoubleTap and DoubleClick commands perform the same event and can be used against both desktop and mobile SUTs.

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Example:

//Use code like this to scrape values from the SUT using the clipboard

doubleclick "ConfirmationNumber" //DoubleClicks the hot spot of image ConfirmatioNumber in order to highlight text on the SUT

typetext controlKey, "c" //Uses key strokes to perform the copy event on the SUT in order to put the highlighted text into the SUT clipboard

put the remoteClipboard into ConfNum //Retrieves the content of the SUT's clipboard and stores it in a variable

Example:

//This example is similar to the TripleClick example above, but uses a slightly different approach with DoubleClick as well as alternate syntax

to handle TripleClick //Declares a custom command handler named "TripleClick"

put the MouseDoubleClickDelay into clickTime //Stores the MouseDoubleClickDelay in variable clickTime for later use

set the RemoteWorkInterval to clickTime //Sets the RemoteWorkInterval, the interval between events sent to the SUT, to the value stored in clickTime

DoubleClick param(1) //DoubleClicks the first parameter value (image name, coordinate pair, etc.) passed to the TripleClick handler

Click//Clicks the same location as the previous DoubleClick

set the RemoteWorkInterval to .7 //Sets the RemoteWorkInterval to .7 second

end TripleClick

RightClick Command

Note: This code definition explains the use of the RightClick command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: RightClick Command.

Behavior: Right-clicks the SUT mouse on the hot spot (Image) or center (OCR) of the first location found. If more than one image or text string is passed, Eggplant Functional searches for images in the order in which they are listed.

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Example:

RightClick {imageName: "Login", searchType: "smooth"} //Right-clicks the hot spot of image "Login", using the searchType:smooth image property for the image search. Note the addition of the "imageName:" property when including an image property.

Example:

rightClick text:"Firefox" //Right clicks at the center of string "Firefox"

moveTo text:"Send to" //Moves the mouse cursor to the center of string "Send to"

Example:

//Use code like this to select multiple items, such as files in File Explorer/Finder, and then copy the files

Click "FirstEntry" //Clicks the hot spot of image "FirstEntry"

KeyDown ControlKey //Holds down the control key

Click "LastEntry" //Clicks the hot spot of image "LastEntry"

KeyUp AllKeys //Releases any and all keys that might be down, including the control key

RightClick //Right clicks at the same location as the previous mouse event

Click "CopyMenuItem"

MouseButtonDown, MouseButtonUp Commands

Behavior: Presses (MouseButtonDown) or releases (MouseButtonUp) the mouse button indicated by the parameter.

Note: For standard 3-button mice, button 1 is the left button, button 2 is the middle button, and button 3 is the right button. Buttons 4 and 5 control scroll-wheel behavior.
Tip: The Swipe, Drag, and Drop commands are useful in many of the same situations as MouseButtonDown/Up 1 in many cases as they all leverage the left mouse button event.
Tip: When scrolling on desktop, the ScrollWheelDown/Up commands are generally easier to use than MouseButtonDown/Up 4/5.

Parameter: A single mouse button number, 1-8.

Example:

MouseButtonDown 3//Holds down the mouse right click on most mice

wait 1

MouseButtonUp 3

Example:

MoveTo "FirefoxIcon" //Moves the mouse cursor to the hot spot location of FirefoxIcon

MouseButtonDown 1 //Holds down the left mouse button

MoveTo the remoteScreenSize*(3/4) //Moves the mouse cursor to the lower right quadrant of the SUT, dragging the Firefox icon along with it

MouseButtonUp 1 //Releases the left mouse button

Example:

//Use code like this to paste content into a mobile SUT

setremoteClipboard "This is my test message." //Puts the specified string into the clipboard of the SUT

moveto "MemoEditor" //Moves the mouse cursor to the hot spot location of the AddressBar image

mousebuttondown 1 //Holds down the left mouse button

wait 2 //Waits a sufficient amount of time for the paste dialog to appear

mousebuttonup 1 //Releases the left mouse button

tap "PasteDialog"

Example:

set the remoteWorkInterval to .1 //Decreases the remoteWorkInterval so that the mousebuttondown and mousebuttonup events occur .1 seconds apart

repeat until imagefound(image:"NewAddress",waitFor:0) //Repeats until the image of interest shows on the SUT

//The next two commands use the mouse scroll wheel to scroll down the page one increment at a time

MouseButtonDown 5

MouseButtonUp 5

Wait 1 //Waits 1 second to allow the UI to settle before searching for "NewAddress"

end repeat

set the remoteWorkInterval to .7

ScrollWheelDown, ScrollWheelUp Commands

Behavior: Scrolls the mouse wheel up or down. The actual amount of scrolling per increment varies by mouse driver, platform, and application.

Tip: An easy way to experiment with the mouse wheel of a SUT is to run a ScrollWheelUp or ScrollWheelDown command with a parameter of 1 in the Ad Hoc Do Box. At times, the direction (up/down) that the scroll wheel needs to turn to scroll the desired direction is counter-intuitive, so if the direction you tried first does not work, try the other direction as well.
Tip: In most cases, a Moveto command is required before the ScrollWheelDown/Up commands, in order to position the cursor over the window or pane you want to scroll.

Parameter: A single integer representing mouse wheel increments (audible clicks on some mice.).

Example:

ScrollWheelDown 8

Example:

repeat until imagefound(image:"NewAddress",waitFor:0) //Repeats until the specified image shows on the SUT

Moveto remotescreensize()*(1/2) //Moves the mouse cursor to the center of the SUT screen

repeat until imagefound(image:"NewAddress",waitFor:0) //Repeats until the specified image shows on the SUT

ScrollWheelDown 4 //Scrolls 4 increments

wait 1 //Waits 1 second to allow the UI to settle before searching for "NewAddress"

end repeat

click "NewAddress"

Example:

Moveto {image:"CustomerPanelHeader",hotspot:[200,150]} //Moves the cursor to a location 200 pixels to the right and 150 pixels down from the upper left corner of the CustomerPanelHeader image.

ScrollWheelUp 10 //Scrolls 10 increments.

Example:

Moveto{image:"CustomerPanelHeader",hotspot:[200,150]} //Moves the cursor to a location 200 pixels to the right and 150 pixels down from the upper left corner of the CustomerPanelHeader image.

ScrollWheelUp 10 //Scrolls 10 increments.

MoveTo Command

Note: This code definition explains the use of the MoveTo command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: MoveTo Command.

Behavior: Moves the SUT’s mouse cursor to the hot spot of the first image found, or in the location indicated by a coordinate pair.

Tip: Sometimes when working with hover-over elements (menus or tooltips), changing the MouseMoveSpeed global property is necessary so that the hover-over event registers properly on the SUT. Using the default value of 0, the cursor jumps between locations instead of moving smoothly between them.
Tip: When transitioning between linked hover-over elements, such as in a series of fly-out menus, changing the MouseMoveMode to avoid the default diagonal movement of the cursor might be necessary.

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Example:

moveto "Link" //Moves the mouse cursor over the Link image

waitfor 5, "Tooltip" //Waits for the expected tooltip to appear

Example:

moveto the remotescreensize times .5 //Moves the mouse cursor to the center of the SUT screen

repeat until imagefound(image:"ContactUsLink",waitFor:0) //Repeats until the "ContactUsLink" image is visible on the SUT

if the repeatIndex > 3 then Throw "Image not found.","Contact Us link not found." //Throws an exception if the repeat loop repeats more than 3 times

scrollwheeldown 1 //Scrolls one increment

wait 1 //Waits 1 second to allow the UI to settle after the scroll

end repeat

Example:

Use code like this to work with a horizontal fly-out menu:

Click "ObjectsMenu"

set the MouseMoveMode to 1 //Changes the movement behavior of the mouse cursor so that it moves first in the horizontal direction and then in the vertical

set the MouseMoveSpeed to 5 //Changes the movement speed of the mouse cursor so that it moves smoothly between locations instead of jumping

Moveto "Circles" //Moves the mouse cursor to the hot spot location of image "Circles"

Moveto text:"Radius" //Moves the mouse cursor to the center of string "Radius"

Moveto "Value"

set the MouseMoveMode to 0 //Returns the mouse movement behavior back to the default

set the MouseMoveSpeed to 0 //Returns the mouse movement speed back to the default

MoveToEach Command

Behavior: Moves the SUT’s mouse to the locations indicated by the parameters, in the order they are listed.

Parameters: One or more Image References or Text Properties (for OCR).

Example:

MoveToEach "Edit", (780, 91), "Copy"

Tip: Sometimes when working with hover-over elements (menus or tooltips), changing the MouseMoveSpeed is necessary so that the hover-over event registers properly on the SUT. Using the default value of 0, the cursor jumps between locations instead of moving smoothly between them.
Tip: When transitioning between linked hover-over elements, such as in a series of fly-out menus, changing the MouseMoveMode to avoid the default diagonal movement of the cursor might be necessary.

Example:

Use code like this to work with a horizontal fly-out menu:

Click "ObjectsMenu"

setOptions {MouseMoveMode:2,MouseMoveSpeed:10} //Changes the movement behavior and movement speed of the mouse cursor.

MovetoEach "Circles", {text:"Radius"}, "Value" //Moves the mouse cursor to the hot spot or center of each image or text string, respectively, in the specified order.

setOptions {MouseMoveMode:0,MouseMoveSpeed:0} //Returns the MouseMoveMode and the MouseMoveSpeed to their default values of 0.

Drag Command

Note: This code definition explains the use of the Drag command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: Drag Command.

Behavior: Presses and holds the SUT’s left mouse button in the hot spot of the first image found, or in the location indicated by a coordinate pair.

Tip: Drag and Drop and DragandDrop work on mobile SUTs and provide the option to choose where to start and release the left mouse button event, while the Swipe commands allow you to choose where to start the left mouse button event only.

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Example:

set the MouseDragSpeed to 5 //Slows down the speed of the drag action

Drag "FirefoxIcon" //Begins a drag action on the "FirefoxIcon" image

Drop the remoteScreenSize*(3/4) //Ends the drag action in the lower right quadrant of the SUT screen

set the MouseDragSpeed to 10 //Returns the speed of dragging to the default value

Example:

put imageLocation("Node") into ElementLoc //Stores the location of the "Node" image in a variable.

Drag ElementLoc //Begins a drag action at the location stored in variable ElementLoc.

Drag ElementLoc - [400,300] //Ends the drag action 400 pixels to the left and 300 pixels up from the location in ElementLoc.

Example:

put the remoteScreenSize into ScreenSize //Stores the resolution of the SUT in a variable.

Drag ScreenSize*[.7,.5] //Begins the drag 70% across and 50% down the SUT screen.

Drop ScreenSize*[.7,.2] //Ends the drag 70% across and 20% down the SUT screen.

Drop Command

Note: This code definition explains the use of the Drop command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: Drop Command.

Behavior: Moves to the given location and releases the left mouse button.

Tip: Drag and Drop and DragandDrop work on mobile SUTs and provide the option to choose where to start and release the left mouse button event, while the Swipe commands allow you to choose where to start the left mouse button event only.

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Example:

Drag "Apple" //Begins a drag action on the "Apple" image

Moveto "Polish" //Continues the drag to the "Polish" image

Drop text:"Bucket" //Ends the drag at the string "Bucket"

Example:

set ScreenSize to remoteScreenSize() //Stores the resolution of the SUT in a variable.

Drag ScreenSize times[7/10,1/2] //Begins the drag 7/10 across and 1/2 down the SUT screen.

Drop ScreenSize times[7/10,1/5] //Ends the drag 7/10 across and 1/5 down the SUT screen.

DragAndDrop Command

Behavior: Presses and holds the SUT’s left mouse button in the first location, moves the mouse to subsequent locations, and releases the mouse button in the final location.

Tip: Drag and Drop and DragandDrop work on mobile SUTs and provide the option to choose where to start and release the left mouse button event, while the Swipe commands allow you to choose where to start the left mouse button event only.

Parameters: Two or more locations, identified by Image References, Text Properties (for OCR), or coordinate pairs.

Example:

DragandDrop "Apple", "Polish", text:"Bucket" //Begins the drag on the "Apple" image, continues the drag to the "Polish" image, and ends the drag on the string "Bucket"

Example:

set ScreenSize to remoteScreenSize() //Stores the resolution of the SUT in a variable.

DragandDrop ScreenSize*[.7,.5],ScreenSize*[.7,.2] //Begins the drag 70% across and 50% down the SUT screen and ends it 70% across and 20% down.

MouseLocation Function

Behavior: Returns the coordinates of the current mouse location. This function works with mobile SUTs as well, though the cursor is generally not visible.

Example:

log MouseLocation() //Logs the current location of the mouse cursor

Example:

Click "Submit"

put the mouseLocation into StartingPoint //Stores the current mouse cursor location, based on where the previous action left it, in a variable

Click "Update"

Click StartingPoint //Clicks the location stored in StartingPoint

 

This topic was last updated on August 19, 2021, at 03:30:51 PM.

Eggplant icon Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2022 Eggplant