Typing on Soft Keyboards

The TypeText command can be used to enter text on any SUT. In most situations, this is the simplest and most straightforward approach. However, on mobile devices, it does not exactly imitate the user experience. In some situations it is desirable to type against the soft keyboard on a mobile device instead of injecting the TypeText events directly into the system. This is especially useful if your application has a custom soft keyboard that you want to test, or if you just want to imitate the user experience more closely.

Typing on the Soft Keyboard of a Mobile Device

This scenario is also a great example of where modularization of scripts is helpful. The code to type against a soft keyboard can be broken out into a handler and stored in a separate script or handler library. Once this has been created, that code and all of the associated images can be stored in a helper suite and reused over and over again. Every time you need to interact with a soft keyboard on a mobile device, simply call the related code, and pass the string you want to type to that handler.

Example of this type of handler:

to SoftType WordToType //The variable "WordToType" catches a string to type that is passed to it from a calling script.
put the remoteworkinterval into RWI //The remoteworkinterval can be adjusted for this handler depending on the soft keyboard being used and how it animates.
  set the remoteworkinterval to .5 //The setting used in this example works with both iOS 6 and Android 4.1.1.
  repeat with each character letter of WordToType //Iterates through the characters of the passed string and uses each letter as a reference to an image that is named the same as that letter. For example, an image of the letter "a" would be called "a".
    Click letter //Clicks each letter in the sequence passed to the variable "WordToType”.
  end repeat
  set the remoteworkinterval to RWI //This re-sets the remoteworkinterval to its original value.
end SoftType

Typing on a Soft Keyboard using Swipe Type

Typing on a soft keyboard can be done with the method shown above, where each letter key is tapped individually, imitating the way a user types on the keyboard. However, swipe type can also be imitated with Eggplant Functional.

While this is a less common method of typing on keyboards, it can be important to test swipe type capability either to more closely mimic the user experience, or to verify swipe type capability on a custom soft keyboard. Additionally, this method used to script swipe-typing can be modified and applied to other kinds of shape drawing, such as when using a pattern to unlock a device.

Example of this type of handler:

Note: This script was written to work against the native Android swipe type keyboard on a Samsung device. Timing settings in this script may need adjustment depending on the device and keyboard being tested.

to SwipeType WordToType //The variable "WordToType" catches a string to type that is passed to it from a calling script.
  put the remoteworkinterval into RWI//The remoteworkinterval can be adjusted for this handler depending on the soft keyboard being used and how it animates.
  set the remoteworkinterval to .01 //The setting used in this example works with Android 4.1.1.
  Repeat with each character letter of WordToType //Iterates through the characters of the passed string and uses each letter as a reference to an image that is named the same as that letter. For example, an image of the letter "a" would be called "a".
    insert imagelocation(letter) nested after CharLoc //Find the location of each letter in the sequence and store it, so that the key can be referenced by location during an extended drag and drop.
  end repeat
  Repeat with each item Letter of CharLoc //This now iterates through the CharLoc list, which is a list of the saved locations of each letter image.
    drag Letter //Clicks each letter in the sequence based on the location sequence in the CharLoc variable. This will drag through each location without dropping until the end of the sequence, creating the swipe type action for that word.
  end repeat
  drop
  set the remoteworkinterval to RWI //Restore the remoteworkinterval to its original value.
end swipeType

Multiple Word Swipe Type

When doing a swipe type with multiple words, it is important to make sure that the spaces between the words are being created properly. A standard swipe type keyboard is likely to insert the spaces automatically if no spaces are manually added, but the spaces can also be added in order to ensure correct typing of the desired text.

A line of code like the one below can be used to tap the space key after each word is swipe typed on the keyboard.

Tap "SpaceKey" //This taps an image of the space key and can be inserted at the end of the handler being called, or in the calling script after each word is passed to the handler.

The calling script will need to pass each word to the handler individually, though the handler could also be set up to iterate through a list, and a list could be passed instead.

Example of this type of handler:

Calling Script

put ("TestPlant","is","the","Best") into WordList //Create a list of words to be typed, that will be passed to a called handler.
Handlers.SwipeSentence WordList //Call the SwipeSentence handler, which is set up to iterate through a list of words to type on a swipe type enabled keyboard.

Handler to Type a Sentence with Swipe Type

to SwipeSentence WordList //The variable "WordList" catches the list of words to be typed passed to it from the calling script.
  put the remoteworkinterval into RWI //The remoteworkinterval can be adjusted for this handler depending on the soft keyboard being used and how it animates.
  Set the remoteworkinterval to .1 //The setting used in this example works with Android 4.1.1.
  Repeat with each item WordToType of WordList //Iterate through the list of words to be typed, as passed to the WordList variable.
    Repeat with each character letter of WordToType //Iterates through the characters of the passed string and uses each letter as a reference to an image that is named the same as that letter. For example, an image of the letter "a" would be called "a".
      insert imagelocation (letter) nested after CharLoc // Searches for each letter in the sequence and stores its location in a list.
      repeat with each item letter of CharLoc //Iterates through the list of image locations.
        drag letter //Drags to each location in the list to swipe type the word.
      end repeat
      drop
    Tap space //This may be unnecessary depending on the keyboard being tested. Some swipe type keyboards will automatically insert a space between words, and some will not.
    set CharLoc to empty //It is necessary to empty this variable for the swipe typing of the next word in the sequence.
    set wordtotype to empty //It is necessary to empty this variable for the next word in the sequence.
  end repeat
  set the remoteworkinterval to RWI //Reset the remoteworkinterval to its original value.
end SwipeSentence