Chunk Types

Chunk expressions let you work with all of these chunk types:

Type	Definition
characters	individual characters within text
words	words separated by any amount of white space (spaces, tabs, returns) within text
lines	paragraphs separated by any of several standard line endings (CR, LF, CRLF, etc.)
text items	portions of text separated by commas
list items	the individual items in a list
bytes	the bytes within binary data
occurrences	the text matches of a defined pattern
matches	the text matches and text range of a defined pattern and its capture groups

In addition, you can specify custom delimiters to be used in identifying text items, lines, and words, giving even greater functionality. These three text chunk types each have distinctive types of delimiters: text items are delimited by a single text string, lines are delimited by any of a list of text strings, and words are delimited by any number and combination of characters from a set of characters.

Characters

The simplest type of chunk is the character chunk. A character is simply one character of text, including both visible and invisible characters (invisible characters include control characters such as tab, carriage return, and linefeed characters). The word character may be abbreviated as char.

Words

A single word is defined as a sequence of characters not containing any whitespace characters, or a sequence of characters contained in quotation marks. A range of words includes all characters from the first word specified through the last word specified, including all intervening words and whitespace. Whitespace characters are spaces, tabs, and returns (newlines).

Note that quoted phrases are ordinarily treated as a single word, including the quotation marks:

Related Local and Global Properties

SenseTalk includes local and global properties you can use to govern aspects of working with words in chunks. The set of characters that are used to identify words can be changed to something other than Space, Tab, and Return by setting the wordDelimiter local property or the defaultWordDelimiter global property. The quote characters used to identify a quoted word (or whether word quoting should be disabled completely) can be specified with the wordQuotes local property or the defaultWordQuotes global property.

These local properties are defined on Local and Global Properties for Chunk Expressions:

• the wordDelimiter, the defaultWordDelimiter
• the wordQuotes, the defaultWordQuotes

Lines

A line chunk expression allows you to specify one or more lines or paragraphs of text within the subject text, where lines are initially defined as the characters between any of the standard line ending characters.

Related Local and Global Properties

SenseTalk includes two properties you can use to govern aspects of working with lines in chunks. The set of line endings (delimiter strings) that define what a line is can be changed to something other than the default by setting the lineDelimiter local property. Setting the lineDelimiter to empty causes it to return to the default list.

the defaultLineDelimiter global property defines the default set of line delimiters. This property is initially set to: CRLF, Return, CarriageReturn, LineSeparator, ParagraphSeparator.

These properties are defined on Local and Global Properties for Chunk Expressions:

• the lineDelimiter, the defaultLineDelimiter

Text Items

An item within text is usually defined as the portion of text between commas:

The separation (delimiter) character can be specified as something other than a comma by setting the itemDelimiter property. the itemDelimiter's default value is determined by the defaultItemDelimiter global property. These two properties are defined on Local and Global Properties for Chunk Expressions:

• the itemDelimiter, the defaultItemDelimiter

List Items

The word items can also refer to the elements in a list.

SenseTalk decides whether item refers to text items or list items depending on whether the value is a list or not. When referring to items within a value which is a list, SenseTalk will automatically assume the reference is to list items, not text items. However, if the itemDelimiter is set to “” (empty), items will refer to list items rather than text items. You may explicitly refer to list items or text items instead of the more generic items if you need to control the way items are treated. This is especially important if you are trying to create a list by putting values into individual items, like this:

The code above will generate a text string, with the middle character being the itemDelimiter (unless the itemDelimiter has been set to empty). To generate a list instead of text, specify list item:

See Lists and Property Lists for more information on working with lists.

Bytes

A byte chunk can be used to refer to a portion of binary data.

See Binary Data Manipulation, for more information on byte chunks.

Occurrences

The words occurrence and occurrences let you access a pattern match as chunks of a string and return the matched text. Use occurrence to access a specific single occurrence of a pattern within a string, and use occurrences to return a list of occurrences.

You can use instance as a synonym for occurrence, and instances in place of occurrences in all cases.

Example:

Example:

Requesting a range of occurrences returns a list of values rather than a substring of the source string.

Example:

Used with the Set commands in the previous example

Example:

For information about using patterns, see SenseTalk Pattern Language Basics.

Matches

The matches keyword lets you access a pattern as chunks of a string. The value returned is a match property list that contains the full text of the match as the text property and the range of the match as the text_range property.

Requesting a range by using matches returns a list of property lists, which includes one property list for each match of the pattern.

For information about using patterns, see SenseTalk Pattern Language Basics.

Custom Chunks

The standard word, line, and text item chunks are useful for many things just as they are. Sometimes you may have text in specific formats that you would like to divide in other ways, however. For example, many programs can produce data files containing several values separated by tab characters on each line of the file.

One way to work with such data would be to set the itemDelimiter to tab and then access the items of each line. But suppose that each tab-separated item contains several values separated by commas. To access these values individually would require switching the itemDelimiter back and forth between tab and comma.

SenseTalk offers an easier alternative for such cases, by specifying the delimiter to be used as part of each chunk, using the phrase delimited by:

The same syntax may be used with line chunks if you like:

The delimiters used to separate text items and lines are not restricted to a single character:

Custom delimiters are also allowed with word chunks, but the behavior is different than with items and lines. Words are normally separated by spaces, tabs, and line breaks. Any number of these “whitespace” characters may appear in sequence between two words. If you specify a custom delimiter for a word chunk, the “words” will be delimited by any number and combination of the characters contained in the delimiter string you supply:

The following example may help to illustrate the difference between the use of custom delimiters for line chunks (which treat each delimiter string found as a separate chunk) and for word chunks (which treat each sequence of delimiter characters as a single word break):

 

This topic was last updated on August 19, 2021, at 03:30:51 PM.

Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2022 Eggplant