Mouse Events and Control

This section describes the Eggplant Functional commands and functions that execute mouse events or return information about the mouse cursor on the SUT. Many of these commands and functions, including Click, DoubleClick, Drag, Drop, DragandDrop, MouseButtonDown, and MouseLocation(), work against mobile SUTs as well as desktop SUTs, which is helpful when creating scripts that automate both mobile and desktop platforms. SenseTalk also includes a set of mobile commands and functions, a subset of which (Tap, DoubleTap, Press, etc.) are compatible with desktop SUTs.

Click Command

Note: This code definition explains the use of the Click command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: Click Command.

Example:

repeat with each item of ("Option1","Option2","Option3") //Repeats through a list of images

click it//Clicks the hot spot of the current image in the list

end repeat

Example:

click(text:"Tweet",waitFor:10,searchRectangle:("AnchorLeft","AnchorRight")) //Clicks the center of string "Tweet", waiting up to ~10 seconds for it to appear with the specified searchRectangle

Example:

//Use code like this to select a section of items, such as files in File Explorer/Finder, and then copy the files

Click "FirstEntry" //Clicks the hot spot of image "FirstEntry"

KeyDown ShiftKey //Holds down the shift key

Click "LastEntry" //Clicks the hot spot of image "LastEntry"

KeyUp ShiftKey //Releases the shift key

RightClick //Right clicks at the same location as the previous mouse event

Click "CopyMenuItem"

Example:

//Sometimes searching for an image twice, once before the click, is helpful when automating UIs that shift as their content loads

WaitFor 10, "NewTweet" //Waits for the image NewTweet to appear

Click(Image:"NewTweet",HotSpot:(80,0)) //Searches for NewTweet again and sends the click 80 pixels to the right of the image's upper left corner. Note the addition of the "image:" property when including the HotSpot image property

Example:

if ImageFound(text:"Cancel") then click FoundImageLocation() //Clicks the location of the center of the found string "Cancel"

Example:

Click the RemoteScreenSize*.5 //Clicks the center point of the screen

Example:

//Use code like this to perform a triple click event

on TripleClick imageToClick //Declares a custom command handler named "TripleClick" with parameter "imageToClick"

//The next 3 lines change the timing between Eggplant Functional commands so that 3 quick clicks can be sent in succession

put the RemoteWorkInterval into defaultRWI //Stores the current Remote Work Interval value so that it can be restored later

set the RemoteWorkInterval to .01 //Sets the RemoteWorkInterval to .01 seconds, allowing events to be sent to the SUT much more rapidly

//These 3 lines click the image once and then click twice more in the same location, at the interval specified by the RemoteWorkInterval

Click imagetoClick //Sends a click at the hot spot location of the image passed in as a parameter

//The subsequent 2 clicks occur at the same location as the previous click

Click

Click

set the RemoteWorkInterval to defaultRWI //Restores the RemoteWorkInterval to the original value

end TripleClick

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Behavior: Clicks the SUT mouse on the hot spot (Image) or center (OCR) of the first location found. If more than one image or text string is passed, Eggplant Functional searches for images in the order in which they are listed.

Tip: The Tap and Click commands perform the same event and can be used against both desktop and mobile SUTs.

DoubleClick Command

Note: This code definition explains the use of the DoubleClick command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: DoubleClick Command.

Example:

//Use code like this to scrape values from the SUT using the clipboard

doubleclick "ConfirmationNumber" //DoubleClicks the hot spot of image ConfirmatioNumber in order to highlight text on the SUT

typetext controlKey, "c" //Uses key strokes to perform the copy event on the SUT in order to put the highlighted text into the SUT clipboard

put the remoteClipboard into ConfNum //Retrieves the content of the SUT's clipboard and stores it in a variable

Example:

//This example is similar to the TripleClick example above, but uses a slightly different approach with DoubleClick as well as alternate syntax

to handle TripleClick //Declares a custom command handler named "TripleClick"

put the MouseDoubleClickDelay into clickTime //Stores the MouseDoubleClickDelay in variable clickTime for later use

set the RemoteWorkInterval to clickTime //Sets the RemoteWorkInterval, the interval between events sent to the SUT, to the value stored in clickTime

DoubleClick param(1) //DoubleClicks the first parameter value (image name, coordinate pair, etc.) passed to the TripleClick handler

Click//Clicks the same location as the previous DoubleClick

set the RemoteWorkInterval to .7 //Sets the RemoteWorkInterval to .7 second

end TripleClick

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Behavior: Double-clicks the SUT mouse on the hot spot (Image) or center (OCR) of the first location found. If more than one image or text string is passed, Eggplant Functional searches for images in the order in which they are listed.

Tip: The DoubleTap and DoubleClick commands perform the same event and can be used against both desktop and mobile SUTs.

RightClick Command

Note: This code definition explains the use of the RightClick command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: RightClick Command.

Example:

RightClick(imageName: "Login", searchType: "smooth") //Right-clicks the hot spot of image "Login", using the searchType:smooth image property for the image search. Note the addition of the "imageName:" property when including an image property

Example:

rightClick text:"Firefox" //Right clicks at the center of string "Firefox"

moveTo text:"Send to" //Moves the mouse cursor to the center of string "Send to"

Example:

//Use code like this to select multiple items, such as files in File Explorer/Finder, and then copy the files

Click "FirstEntry" //Clicks the hot spot of image "FirstEntry"

KeyDown ControlKey //Holds down the control key

Click "LastEntry" //Clicks the hot spot of image "LastEntry"

KeyUp AllKeys //Releases any and all keys that might be down, including the control key

RightClick //Right clicks at the same location as the previous mouse event

Click "CopyMenuItem"

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Behavior: Right-clicks the SUT mouse on the hot spot (Image) or center (OCR) of the first location found. If more than one image or text string is passed, Eggplant Functional searches for images in the order in which they are listed.

MouseButtonDown, MouseButtonUp Commands

Example:

MouseButtonDown 3//Holds down the mouse right click on most mice

wait 1

MouseButtonUp 3

Example:

MoveTo "FirefoxIcon" //Moves the mouse cursor to the hot spot location of FirefoxIcon

MouseButtonDown 1 //Holds down the left mouse button

MoveTo the remoteScreenSize*(3/4) //Moves the mouse cursor to the lower right quadrant of the SUT, dragging the Firefox icon along with it

MouseButtonUp 1 //Releases the left mouse button

Example:

//Use code like this to paste content into a mobile SUT

setremoteClipboard "This is my test message." //Puts the specified string into the clipboard of the SUT

moveto "MemoEditor" //Moves the mouse cursor to the hot spot location of the AddressBar image

mousebuttondown 1 //Holds down the left mouse button

wait 2 //Waits a sufficient amount of time for the paste dialog to appear

mousebuttonup 1 //Releases the left mouse button

tap "PasteDialog"

Example:

set the remoteWorkInterval to .1 //Decreases the remoteWorkInterval so that the mousebuttondown and mousebuttonup events occur .1 seconds apart

repeat until imagefound(image:"NewAddress",waitFor:0) //Repeats until the image of interest shows on the SUT

//The next two commands use the mouse scroll wheel to scroll down the page one increment at a time

MouseButtonDown 5

MouseButtonUp 5

Wait 1 //Waits 1 second to allow the UI to settle before searching for "NewAddress"

end repeat

set the remoteWorkInterval to .7

Parameter: A single mouse button number, 1-8.

Behavior: Presses (MouseButtonDown) or releases (MouseButtonUp) the mouse button indicated by the parameter.

Note: For standard 3-button mice, button 1 is the left button, button 2 is the middle button, and button 3 is the right button. Buttons 4 and 5 control scroll-wheel behavior.
Tip: The Swipe, Drag, and Drop commands are useful in many of the same situations as MouseButtonDown/Up 1 in many cases as they all leverage the left mouse button event.
Tip: When scrolling on desktop, the ScrollWheelDown/Up commands are generally easier to use than MouseButtonDown/Up 4/5.

ScrollWheelDown, ScrollWheelUp Commands

Example:

ScrollWheelDown 8

Example:

repeat until imagefound(image:"NewAddress",waitFor:0) //Repeats until the specified image shows on the SUT

Moveto remotescreensize()*(1/2) //Moves the mouse cursor to the center of the SUT screen

repeat until imagefound(image:"NewAddress",waitFor:0) //Repeats until the specified image shows on the SUT

ScrollWheelDown 4 //Scrolls 4 increments

wait 1 //Waits 1 second to allow the UI to settle before searching for "NewAddress"

end repeat

click "NewAddress"

Parameter: A single integer representing mouse wheel increments (audible clicks on some mice.).

Behavior: Scrolls the mouse wheel up or down. The actual amount of scrolling per increment varies by mouse driver, platform, and application.

Tip: An easy way to experiment with the mouse wheel of a SUT is to run a ScrollWheelUp or ScrollWheelDown command with a parameter of 1 in the Ad Hoc Do Box. At times, the direction (up/down) that the scroll wheel needs to turn to scroll the desired direction is counter-intuitive, so if the direction you tried first does not work, try the other direction as well.
Tip: In most cases, a Moveto command is required before the ScrollWheelDown/Up commands, in order to position the cursor over the window or pane you want to scroll.

Example:

Moveto(image:"CustomerPanelHeader",hotspot:(200,150)) //Moves the cursor to a location 200 pixels to the right and 150 pixels down from the upper left corner of the CustomerPanelHeader image

ScrollWheelUp 10 //Scrolls 10 increments

Example:

Moveto(image:"CustomerPanelHeader",hotspot:(200,150)) //Moves the cursor to a location 200 pixels to the right and 150 pixels down from the upper left corner of the CustomerPanelHeader image

ScrollWheelUp 10 //Scrolls 10 increments

MoveTo Command

Note: This code definition explains the use of the MoveTo command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: MoveTo Command.

Example:

moveto "Link" //Moves the mouse cursor over the Link image

waitfor 5, "Tooltip" //Waits for the expected tooltip to appear

Example:

moveto the remotescreensize times .5 //Moves the mouse cursor to the center of the SUT screen

repeat until imagefound(image:"ContactUsLink",waitFor:0) //Repeats until the "ContactUsLink" image is visible on the SUT

if the repeatIndex > 3 then Throw "Image not found.","Contact Us link not found." //Throws an exception if the repeat loop repeats more than 3 times

scrollwheeldown 1 //Scrolls one increment

wait 1 //Waits 1 second to allow the UI to settle after the scroll

end repeat

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Behavior: Moves the SUT’s mouse cursor to the hot spot of the first image found, or in the location indicated by a coordinate pair.

Tip: Sometimes when working with hover-over elements (menus or tooltips), changing the MouseMoveSpeed is necessary so that the hover-over event registers properly on the SUT. Using the default value of 0, the cursor jumps between locations instead of moving smoothly between them.
Tip: When transitioning between linked hover-over elements, such as in a series of fly-out menus, changing the MouseMoveMode to avoid the default diagonal movement of the cursor might be necessary.

Example:

//Use code like this to work with a horizontal fly-out menu

Click "ObjectsMenu"

set the MouseMoveMode to 1 //Changes the movement behavior of the mouse cursor so that it moves first in the horizontal direction and then in the vertical

set the MouseMoveSpeed to 5 //Changes the movement speed of the mouse cursor so that it moves smoothly between locations instead of jumping

Moveto "Circles" //Moves the mouse cursor to the hot spot location of image "Circles"

Moveto text:"Radius" //Moves the mouse cursor to the center of string "Radius"

Moveto "Value"

set the MouseMoveMode to 0 //Returns the mouse movement behavior back to the default

set the MouseMoveSpeed to 0 //Returns the mouse movement speed back to the default

MoveToEach Command

Example:

MoveToEach "Edit", (780, 91), "Copy"

Parameters: One or more Image References or Text Properties (for OCR).

Behavior: Moves the SUT’s mouse to the locations indicated by the parameters, in the order they are listed.

Tip: Sometimes when working with hover-over elements (menus or tooltips), changing the MouseMoveSpeed is necessary so that the hover-over event registers properly on the SUT. Using the default value of 0, the cursor jumps between locations instead of moving smoothly between them.
Tip: When transitioning between linked hover-over elements, such as in a series of fly-out menus, changing the MouseMoveMode to avoid the default diagonal movement of the cursor might be necessary.

Example:

//Use code like this to work with a horizontal fly-out menu

Click "ObjectsMenu"

setOptions(MouseMoveMode:2,MouseMoveSpeed:10) //Changes the movement behavior and movement speed of the mouse cursor

MovetoEach"Circles", (text:"Radius"), "Value" //Moves the mouse cursor to the hot spot or center of each image or text string, respectively, in the specified order

setOptions(MouseMoveMode:0,MouseMoveSpeed:0) //Returns the MouseMoveMode and the MouseMoveSpeed to their default values of 0

Drag Command

Note: This code definition explains the use of the Drag command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: Drag Command.

Example:

set the MouseDragSpeed to 5 //Slows down the speed of the drag action

Drag "FirefoxIcon" //Begins a drag action on the "FirefoxIcon" image

Drop the remoteScreenSize*(3/4) //Ends the drag action in the lower right quadrant of the SUT screen

set the MouseDragSpeed to 10 //Returns the speed of dragging to the default value

Example:

put imageLocation("Node") into ElementLoc //Stores the location of the "Node" image in a variable

Drag ElementLoc //Begins a drag action at the location stored in variable "ElementLoc"

Drag ElementLoc - (400,300) //Ends the drag action 400 pixels to the left and 300 pixels up from the location in ElementLoc

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Behavior: Presses and holds the SUT’s left mouse button in the hot spot of the first image found, or in the location indicated by a coordinate pair.

Tip: Drag and Drop and DragandDrop work on mobile SUTs and provide the option to choose where to start and release the left mouse button event, while the Swipe commands allow you to choose where to start the left mouse button event only.

Example:

put the remoteScreenSize into ScreenSize //Stores the resolution of the SUT in a variable

Drag ScreenSize*(.7,.5) //Begins the drag 70% across and 50% down the SUT screen

Drop ScreenSize*(.7,.2) //Ends the drag 70% across and 20% down the SUT screen

Drop Command

Note: This code definition explains the use of the Drop command for image-based scripting. For information about using this command for WebDriver connections, see WebDriver Mouse and Keyboard Events: Drop Command.

Example:

Drag "Apple" //Begins a drag action on the "Apple" image

Moveto "Polish" //Continues the drag to the "Polish" image

Drop text:"Bucket" //Ends the drag at the string "Bucket"

Parameters: One or more Image References or Text Properties (for OCR), or a single coordinate location.

Behavior: Moves to the given location and releases the left mouse button.

Tip: Drag and Drop and DragandDrop work on mobile SUTs and provide the option to choose where to start and release the left mouse button event, while the Swipe commands allow you to choose where to start the left mouse button event only.

Example:

set ScreenSize to remoteScreenSize() //Stores the resolution of the SUT in a variable

Drag ScreenSize times(7/10,1/2) //Begins the drag 7/10 across and 1/2 down the SUT screen

Drop ScreenSize times(7/10,1/5) //Ends the drag 7/10 across and 1/5 down the SUT screen

DragAndDrop Command

Example:

DragandDrop "Apple", "Polish", text:"Bucket" //Begins the drag on the "Apple" image, continues the drag to the "Polish" image, and ends the drag on the string "Bucket"

Parameters: Two or more locations, identified by Image References, Text Properties (for OCR), or coordinate pairs.

Behavior: Presses and holds the SUT’s left mouse button in the first location, moves the mouse to subsequent locations, and releases the mouse button in the final location.

Tip: Drag and Drop and DragandDrop work on mobile SUTs and provide the option to choose where to start and release the left mouse button event, while the Swipe commands allow you to choose where to start the left mouse button event only.

Example:

set ScreenSize to remoteScreenSize() //Stores the resolution of the SUT in a variable

DragandDrop ScreenSize*(.7,.5),ScreenSize*(.7,.2) //Begins the drag 70% across and 50% down the SUT screen and ends it 70% across and 20% down

MouseLocation() Function

Example:

log MouseLocation() //Logs the current location of the mouse cursor

Example:

Click "Submit"

put the mouseLocation into StartingPoint //Stores the current mouse cursor location, based on where the previous action left it, in a variable

Click "Update"

Click StartingPoint //Clicks the location stored in StartingPoint

Returns: Coordinates of the current mouse location.

Behavior: Returns the coordinates of the current mouse location. This function works with mobile SUTs as well, though the cursor is generally not visible.

 

This topic was last updated on January 16, 2020, at 03:56:57 PM.

Eggplant icon Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2020 Eggplant