Working with Optical Character Recognition

When you want to find text on your system under test (SUT), and capturing an image of the text is not practical, you can rely on the optical character recognition (OCR) capabilities in Eggplant Functional and SenseTalk. OCR is most useful when searching for dynamic text, although you might find many other practical uses. You should spend time to understand the OCR properties discussed here so that you can make appropriate adjustments and achieve better search results in your test environment.

The Eggplant Functional application is set by default to use the Generic OCR platform. The Generic OCR platform includes a Text Engine, and any number of predefined text styles. Generally, you should not need to change this setting. For information about other text preference settings, see Text Preferences.

Using OCR

You can use OCR almost any time you need to search for text on the SUT. このファンクションは特に以下の状況に適しています。

• 前もって検索するテキストを知らなくて、しかし検索が行われるまでにスクリプトがその情報にアクセスする場合。
• テキストのフォント、サイズまたは色が変化する場合。
• Your testing includes a variety of browsers, and different browsers can render fonts differently.
• You need to read a text value from the SUT and bring that information back into your script for further testing or to record in an external data file.

OCR text searches are similar to image searches, and you typically use the same commands or functions. Instead of specifying the image name, you specify the text string you want to find, in quotes, preceded by text:.

例：

If you want to find all of the locations of a text string, use the EveryImageLocation() function:

例：

OCRエンジンの作動方法

ジェネリックテキストプロパティリストでテキストを検索する場合、OCRエンジンはSUT上の各テキストのインスタンスを検出し、テキスト値を読んで検索している文字列を検出します。You do not have to describe the appearance of your text, because text formatting is not considered.

When the OCR engine does not find your text string, Eggplant Functional throws an exception, which can cause your script to fail. However, if you enable the OCR Update panel, the OCR Update functionality attempts to apply different properties to the search to see if it can make a match. You can set this functionality to Auto Update, so that it performs without user interaction, or Show Panel, which opens the OCR Update panel on a failed OCR search.

この機能の使用についての詳細情報は、The OCR Update Panel（OCRアップデートパネル）を参照してください。

The properties that you can adjust using the OCR Update panel are also available by using the OCR Tuner. The OCR Tuner lets you adjust these OCR properties while showing you the text being read in the Capture Area. You can use the OCR Tuner to improve your OCR searches.

For detailed information about using this feature, see Using the OCR Tuner.

Using OCR Properties in Searches

To improve the reliability of OCR searches, you can add these properties to your SenseTalk code:

• SearchRectangle（検索範囲）
• Contrast and ContrastColor（コントラストと背景色）
• ValidCharacters（有効な文字）
• IgnoreSpaces（スペースを無視）
• IgnoreUnderscores（アンダースコアを無視）
• Language（言語）

SearchRectangle（検索範囲）

It is almost always helpful to add a search rectangle, which limits what part of the screen the search takes place in. When OCR searches the entire screen, it is not only slower, but it can also be less accurate because it is likely to come up with extra possible matches. It might also result in no matches at all.

If your OCR search doesn't find text that you expect should be found, you might try expanding the size of your search rectangle to provide the OCR engine a little more text context, which can sometimes result in better matches.

You can specify a search rectangle with exact coordinates, but it is more common in Eggplant Functional to use images to define the rectangle. Using images works better in case the location of the text is not always the same. This situation might happen, for instance, if the window the text is doesn’t appear in the same location on the screen every time the test is run.

When using images to define the boundaries of the search rectangle, keep in mind that the points used to set the rectangle are based on the hot spot, which can be moved outside of the image.

Example: Reading Dynamic Text from a Website

You might have a test that navigates to the Google Finance page, search for a specific company, and then read the stock price. To make sure that OCR reads the value of the price reliably, and nothing else, you can define a search rectangle using images. 使用するコードはこのようにできるだけ簡単にします。

You need to capture two images for the above code to work. In this example, you used images TLImage and BRImage to define the top left and bottom right corners of the search rectangle. To capture these images, choose one or more elements of the screen that are stable in relation to the text OCR reads.

この例では画面のひとつの要素である「会社」ラベルを使用しています。

TLImageは「会社」ラベルの画像で、株価が表示されているエリアの左上の隅に動くホットスポットを伴います。

Without even moving the capture area, capture the BRImage with the hot spot moved to the bottom right corner of the area where the stock price displays.

After capturing the images and writing the SenseTalk code, the above example is able to successfully read the stock price for any company in Google Finance.

"TLImage" and "BRImage" are used to set the search rectangle for a successful ReadText() search using OCR .

画像を使って検索範囲の長方形の設定が出来ない場合は、画面の座標を用いて設定します。その場合、Cursor Location（カーソル位置）ツールバーアイコンが役に立ちます。なぜならSUT上での現在のマウスの位置が表示されるからです。See Contrast for instructions on adding Cursor Location to the toolbar.

Contrast（コントラスト）

If the background color of the sought text is known, use the Contrast property to set the contrast.

This property causes Eggplant Functional to see in black and white only. Whatever color is being used as the ContrastColor (the background color) turns white, and everything else turns black. For black text on a white background, the code looks like this:

In this situation, every pixel close to white (within the tolerance range of 45 on either side) is turned white, and everything else is read as black. This can help OCR to more clearly read the text.

実際に見えるのは、アンチエイリアシングの無いこの画像です。

コントラストオンでOCRに見えるもの

When looking for text on a gray background (or another color of medium value), it can get a little more complex. It is good to set the ContrastTolerance a little lower (down to 20 or so), which narrows the number of pixels that OCR might try to turn white. Notice in the image above, a pixel between the “h” and the “e” in “hello” turned black, joining the two letters together. In this instance, OCR was still able to read the letters, but in some other circumstances, this could make the letters even more difficult for OCR to process. これが、なぜ全スクリプトを実行する前にまず試してみてその状況をチェックするのが良いかという理由です。

背景色の決定

背景色/コントラスト色が未知の場合、どんなプラットフォーム上でも、特定の位置で背景色のRGB値を検出する方法が2つあります。

最初の方法はカラーピッカーを使う事です。This can be found by clicking the Find Text icon in the toolbar of the remote screen window, and selecting the Contrast checkbox. 次にカラーをイネーブルにして右側のカラーボックスをクリックし、特定の色を選択します。このウィンドウが表示されたら、拡大鏡をクリックして、テキストの隣の背景色のいくつかのサンプルを取ります（アンチエイリアシングをキャプチャしないようにしてください）。

Macで背景色を検出するのに拡大鏡を使用

2番目の方法はColorAtLocation()ファンクションを使います。To use this function, bring up the remote screen window in Live Mode.

If the Cursor Location toolbar icon is not already on the toolbar, it can be added by doing the following:

1. Right-click within the toolbar.
2. Select Customize Toolbar.
3. Drag and drop the Cursor Location icon onto the toolbar.
4. Click Done to close the customization dashboard.

Now do the following to use the ColorAtLocation() function.

1. マウスを必要なRGBカラー値の背景に移動させます。
2. Using the coordinates of this mouse location (shown in the field of the Cursor Location toolbar icon), run this line of code in the ad hoc do box (AHDB) found at the bottom of the run window, or in your script:

   put colorAtLocation(x,y) // where (x,y) refers to the coordinates found in the remote screen window

   Running this line of code returns the RGB value for the color at the location specified.

ValidCharacters（有効な文字）

ValidCharacters tells OCR which characters to take into account, and which characters to ignore. For example, this can be used to prevent OCR from misreading “O” as “0″ or vice versa.

この設定はスクリプトを最適に制御するために手動で行うか、またはvalidCharacters:"*"を使って自動的にvalidCharactersを検索するテキストに設定します。

上の例で「Brown」の検索を続行するためには、次のコードを使ってvalidCharactersを手動で設定します。

It is also possible to use ValidCharacters with a variable like in this example, where the desired characters are first stored into the MyText variable :

validCharacters（有効な文字）を検索するテキストに設定するには、以下のコードを使用します。

IgnoreSpaces（スペースを無視）

As a property in a text search, IgnoreSpaces can be used to do exactly that – ignore any spaces within the search. This approach can be helpful because spacing between characters is not consistent. Sometimes OCR sees spaces where there are not any, or ignores spaces where they exist. Setting the IgnoreSpaces property to ON causes OCR to match “flowerpot” with a search for “flower pot” and vice versa. When this property is used, OCR strips spaces from both the string for which it is searching and the string that it finds to come up with a match.

IgnoreUnderscores（アンダースコアを無視）

The ignoreUnderscores property causes OCR text searches to treat underscores as spaces during searches. For example, the "My_Computer" string would match My_Computer or My Computer. The ignoreUnderscores property is on by default because the OCR engine sometimes fails to recognize underscores.

Language（言語）

As a property in a text search, Language can be used to search for text of the specified language or languages. サポートする言語のリストについては、OCR Language Support（OCR言語サポート）を参照してください。

For example, you can specify more than one language when you have characters of different languages in the same sentence. In the case of a line that reads as follows:

料理4出6「です。")

Use the following SenseTalk code to read the multi-Language characters included on the line:

Tips for Working with OCR

1. When OCR doesn't find text that you think it should, try running the readText() function to see what it is seeing. Once you know what it is seeing, you are better-equipped to fix the problem. 例えば、必要なテキストがパスワードで実際に「hell0」を読む場合に、readText()ファンクションが「hello」を返したら、validCharacters:”hell0″を指定すると結果が改善されます。
2. The Ad Hoc Do Box (AHDB) can be used to verify your script and make sure it is doing what it is supposed to be doing. その他の有用な情報もAHDBを使って得られます。A good example of this is prompting it to return a list of the colors that Eggplant Functional knows, along with their RGB values. このリストを得るには、AHDBに移動してput the namedColorsと入力します。

   

   What OCR Sees with Contrast On

OCR検索の速度を上げる

OCRテキスト検索の柔軟性はトレードオフをもたらします。OCRテキスト検索は画像検索ほど速くはありません。できるだけスクリプトを効率的に実行するには、最良の事例に従います。

1. 検索範囲の長方形を使います。If you can narrow the location of the text you are looking for (e.g. in the taskbar, or within a particular window), you can set a search rectangle to limit your search to that location. For more information, see Text Preferences or Searching Part of the Screen.
2. Minimize clutter on the SUT. It never hurts to keep a neat SUT (i.e., close unnecessary windows, use a simple desktop background). When your script includes OCR text searches, a neat desktop can save you valuable seconds or even minutes of execution time.

 

This topic was last updated on 2月 01, 2019, at 11:13:23 午前.

Eggplant.io | Documentation Home | User Forums | Support | Copyright © 2019 Eggplant