:UNIQ_PROCESS, UNIQ_PROCESS

Use UNIQ_PROCESS to make adjacent lines in a text container unique. You can use this scripting element with process handles and select specific columns and work only within them.

Note: The terms Data Sequences and Text Containers are used interchangeably.

This scripting element can be used as both, a script statement and a script function:

When used as a script statement, it modifies the text container of the input handle and provides the result in that text container. This is useful for stacking commands to modify text containers, thus avoiding reallocating new process handles.
When used as a script function, the text container of the process handle is provided read-only, and the result is provided in a new text container as a return value of the function.

Syntax

When used as a script statement:

:UNIQ_PROCESS Handle[, Column[, Flags]]

When used as a script function:

UNIQ_PROCESS (Handle[, Column[, Flags]])

Parameters

Handle

Process handle or a text container

Format: script literal or script variable

Column

(Optional) The index indicates which column to search, beginning with 1, or the entire line if the index is 0.

If the text container (data sequence) loaded had columns defined, you can select a specific column and use this script element to work within the relevant column. To do so, make sure you know which column delimiter was used at the time of loading the text container (data sequence). Otherwise, you can define the delimiter using the ‘d’ flag.

Format: script literal, script variable or number without quotation marks, Column number or 0 for plain line.

Default value: 0

Note: When you specify a column to be used for unique filtering, the result still contains the full lines.

Flags

(Optional) Use flags to modify the operation. The flags available are:
- i: Case-insensitive
  
  By default, the search is case sensitive.
- u: Only unique lines
  
  Only unique lines are selected.
- 2: Duplicates
  
  Finds only duplicate lines.
- d: Delimiter that separates columns
  
  Allows you to define a specific delimiter, which can be any text of at least one character, to virtually break up text into columns. The delimiter is enclosed by the character immediately following the "d". This overwrites any default delimiter that the text container might have. For example:
  
  d' ' for a single blank space
  
  d'$$' for a sequence of two dollar signs
  
  d+,+ for a comma
  
  d!foo! for the word "foo"
Flags can be combined and provided in any order with the exception of flags "u" and "2". For example, if you combine flag i with flag u, you can do so using "iu" or "ui", or the script variable &FLAG#.

Examples

Sorting lines and words from a file and making them unique:

!* Load text container from a file

:SET &HND_TEXT# = PREP_PROCESS_FILE(AR_MAIN_JWX6_VFRKARWIN01_01, "C:\Temp\ContentFile.txt",,,'UC_LOGIN=AR.LOGIN.OK')

!* Get all words of the text

:SET &HND_WORDS# = PREP_PROCESS_REGEX(&HND_TEXT#, "[[:alpha:]]+",,,"g")

!* 1.) Sort all lines of the text

:SET &HND_TEXT_SORTED# = SORT_PROCESS(&HND_TEXT#)

!* Output sorted words

:SET &NUMBER_WORDS# = GET_PROCESS_INFO(&HND_WORDS#, ROWS)

:PRINT &NUMBER_WORDS# WORDS

:PROCESS &HND_WORDS#

: SET &WORD# = GET_PROCESS_LINE(&HND_WORDS#)

: PRINT &WORD#

:ENDPROCESS

!* 2.) Sort words lexographically

:SORT_PROCESS &HND_WORDS#

!* Output sorted words

:SET &NUMBER_WORDS# = GET_PROCESS_INFO(&HND_WORDS#, ROWS)

:PRINT &NUMBER_WORDS# WORDS

:PROCESS &HND_WORDS#

: SET &WORD# = GET_PROCESS_LINE(&HND_WORDS#)

: PRINT &WORD#

:ENDPROCESS

!* 3.) Reverse sort words

:SET &REVERSE_WORDS# = SORT_PROCESS(&HND_WORDS#,,"r")

!* Output reverse sorted words

:PROCESS &REVERSE_WORDS#

: SET &WORD# = GET_PROCESS_LINE(&REVERSE_WORDS#)

: PRINT &WORD#

:ENDPROCESS

!* 4.) Make words unique, i.e., get all different words of the text, one each

:SET &DIFFERENT_WORDS# = UNIQ_PROCESS(&HND_WORDS#, , "i")

!* Output sorted words

:SET &NUMBER_WORDS# = GET_PROCESS_INFO(&DIFFERENT_WORDS#, ROWS)

:PRINT &NUMBER_WORDS# DIFFERENT WORDS

:PROCESS &DIFFERENT_WORDS#

: SET &WORD# = GET_PROCESS_LINE(&DIFFERENT_WORDS#)

: PRINT &WORD#

:ENDPROCESS

!* 5.) get all unique words of the text

:SET &UNIQUE_WORDS# = UNIQ_PROCESS(&HND_WORDS#, , "ui")

!* Output sorted words

:SET &NUMBER_WORDS# = GET_PROCESS_INFO(&UNIQUE_WORDS#, ROWS)

:PRINT &NUMBER_WORDS# UNIQUE WORDS

:PROCESS &UNIQUE_WORDS#

: SET &WORD# = GET_PROCESS_LINE(&UNIQUE_WORDS#)

: PRINT &WORD#

:ENDPROCESS

!* 6.) get all words of the text which have duplicates, one each

:SET &DUPLICATE_WORDS# = UNIQ_PROCESS(&HND_WORDS#, , "2i")

!* Output sorted words

:SET &NUMBER_WORDS# = GET_PROCESS_INFO(&DUPLICATE_WORDS#, ROWS)

:PRINT &NUMBER_WORDS# DUPLICATE WORDS

:PROCESS &DUPLICATE_WORDS#

: SET &WORD# = GET_PROCESS_LINE(&DUPLICATE_WORDS#)

: PRINT &WORD#

:ENDPROCESS