/* $DOC$
   $NAME$
      AtToken()
   $CATEGORY$
      CT3 string functions
   $ONELINER$
      Position of a token in a string
   $SYNTAX$
      AtToken( <cString>, [<cTokenizer>],
               [<nTokenCount>], [<nSkipWidth>] ) -> nPosition
   $ARGUMENTS$
      <cString>          is the processed string
      [<cTokenizer>]     is a list of characters separating the tokens
                         in <cString>
                         Default: Chr( 0 ) + Chr( 9 ) + Chr( 10 ) + Chr( 13 ) +
                                  Chr( 26 ) + hb_BChar( 138 ) + hb_BChar( 141 ) +
                                  Chr( 32 ) + ",.;:!\?/\\<>()#&%+-*"
      [<nTokenCount>]    specifies the count of the token whose
                         position should be calculated
                         Default: last token
      [<nSkipWidth>]     specifies the maximum number of successive
                         tokenizing characters that are combined as
                         ONE token stop, e.g. specifying 1 can
                         yield to empty tokens
                         Default: 0, any number of successive tokenizing
                         characters are combined as ONE token stop
   $RETURNS$
      <nPosition>        The start position of the specified token or
                         0 if such a token does not exist in <cString>.
   $DESCRIPTION$
      The AtToken() function calculates the start position of tne
      <nTokenCount>th token in <cString>. By setting the new <nSkipWidth>
      parameter to a value different than 0, you can specify how many tokenizing
      characters are combined at most to one token stop. Be aware that
      this can result to empty tokens there the start position is not
      defined clearly. Then, AtToken() returns the position there the
      token WOULD start if its length is larger than 0. To check for
      empty tokens, simply look if the character at the returned position
      is within the tokenizer list.
   $EXAMPLES$
      AtToken( "Hello, World!" ) // --> 8 (empty strings after tokenizer are not a token !)
   $STATUS$
      Ready
   $COMPLIANCE$
      AtToken() is compatible with CT3's ATTOKEN, but has an additional
      4th parameter to let you specify a skip width equal to that in the
      Token() function.
   $PLATFORMS$
      All
   $FILES$
      Library is hbct.
   $SEEALSO$
      Token(), NumToken(), TokenLower(), TokenUpper(), TokenSep()
   $END$
 */

/* $DOC$
   $NAME$
      Token()
   $CATEGORY$
      CT3 string functions
   $ONELINER$
      Tokens of a string
   $SYNTAX$
      Token( <cString>, [<cTokenizer>],
             [<nTokenCount], [<nSkipWidth>],
             [<@cPreTokenSep>], [<@cPostTokenSep>] ) -> cToken
   $ARGUMENTS$
      <cString>          is the processed string
      [<cTokenizer>]     is a list of characters separating the tokens
                         in <cString>
                         Default: Chr( 0 ) + Chr( 9 ) + Chr( 10 ) + Chr( 13 ) +
                                  Chr( 26 ) + hb_BChar( 138 ) + hb_BChar( 141 ) +
                                  Chr( 32 ) + ",.;:!\?/\\<>()#&%+-*"
      [<nTokenCount>]    specifies the count of the token that
                         should be extracted
                         Default: last token
      [<nSkipWidth>]     specifies the maximum number of successive
                         tokenizing characters that are combined as
                         ONE token stop, e.g. specifying 1 can
                         yield to empty token
                         Default: 0, any number of successive tokenizing
                         characters are combined as ONE token stop
      [<@cPreTokenSep>]  If given by reference, the tokenizer before
                         the actual token will be stored
      [<@cPostTokenSep>] If given by reference, the tokenizer after
                         the actual token will be stored
   $RETURNS$
      <cToken>           the token specified by the parameters given above
   $DESCRIPTION$
      The Token() function extracts the <nTokenCount>th token from the
      string <cString>. In the course of this, the tokens in the
      string are separated by the character(s) specified in <cTokenizer>.
      The function may also extract empty tokens, if you specify a skip
      width other than zero.
      Be aware of the new 5th and 6th parameter there the Token() function
      stores the tokenizing character before and after the extracted token.
      Therefore, additional calls to the TokenSep() function are not
      necessary.
   $EXAMPLES$
      ? Token( "Hello, World!" )            -->  "World"
      ? Token( "Hello, World!",, 2, 1 )     --> ""
      ? Token( "Hello, World!", ",", 2, 1 ) --> " World!"
      ? Token( "Hello, World!", " ", 2, 1 ) --> "World!"
   $STATUS$
      Ready
   $COMPLIANCE$
      Token() is compatible with CT3's TOKEN, but two additional
      parameters have been added there the Token() function can store
      the tokenizers before and after the current token.
   $PLATFORMS$
      All
   $FILES$
      Library is hbct.
   $SEEALSO$
      NumToken(), AtToken(), TokenLower(), TokenUpper(), TokenSep()
   $END$
 */

/* $DOC$
   $NAME$
      NumToken()
   $CATEGORY$
      CT3 string functions
   $ONELINER$
      Retrieves the number of tokens in a string
   $SYNTAX$
      NumToken( <cString>, [<cTokenizer>], [<nSkipWidth>] ) -> nTokenCount
   $ARGUMENTS$

   $RETURNS$

   $DESCRIPTION$

   $EXAMPLES$

   $STATUS$
      Ready
   $COMPLIANCE$
      NumToken() is compatible with CT3's NumToken().
   $PLATFORMS$
      All
   $FILES$
      Library is hbct.
   $SEEALSO$
      Token(), AtToken(), TokenLower(), TokenUpper(), TokenSep()
   $END$
 */

/* $DOC$
   $NAME$
      TokenLower()
   $CATEGORY$
      CT3 string functions
   $ONELINER$
      Change the first letter of tokens to lower case
   $SYNTAX$
      TokenLower( <[@]cString>, [<cTokenizer>], [<nTokenCount>],
                  [<nSkipWidth>] ) -> cString
   $ARGUMENTS$
      <[@]cString>      is the processed string
      [<cTokenizer>]    is a list of characters separating the tokens
                        in <cString>
                         Default: Chr( 0 ) + Chr( 9 ) + Chr( 10 ) + Chr( 13 ) +
                                  Chr( 26 ) + hb_BChar( 138 ) + hb_BChar( 141 ) +
                                  Chr( 32 ) + ",.;:!\?/\\<>()#&%+-*"
      [<nTokenCount>]   specifies the number of tokens that
                        should be processed
                        Default: all tokens
      [<nSkipWidth>]    specifies the maximum number of successive
                        tokenizing characters that are combined as
                        ONE token stop, e.g. specifying 1 can
                        yield to empty token
                        Default: 0, any number of successive tokenizing
                        characters are combined as ONE token stop
   $RETURNS$
      <cString>         the string with the lowercased tokens
   $DESCRIPTION$
      The TokenLower() function changes the first letter of tokens in <cString>
      to lower case. To do this, it uses the same tokenizing mechanism
      as the Token() function. If TokenLower() extracts a token that starts
      with a letter, this letter will be changed to lower case.
      You can omit the return value of this function by setting the CSetRef()
      switch to .T., but you must then pass <cString> by reference to get
      the result.
   $EXAMPLES$
      ? TokenLower( "Hello, World, here I am!" )         // "hello, world, here i am!"
      ? TokenLower( "Hello, World, here I am!",, 3 )     // "hello, world, here I am!"
      ? TokenLower( "Hello, World, here I am!", ",", 3 ) // "hello, World, here I am!"
      ? TokenLower( "Hello, World, here I am!", " W" )   // "hello, World, here i am!"
   $STATUS$
      Ready
   $COMPLIANCE$
      TokenLower() is compatible with CT3's TokenLower(),
      but a new 4th parameter, <nSkipWidth> has been added for
      synchronization with the the other token functions.
   $PLATFORMS$
      All
   $FILES$
      Library is hbct.
   $SEEALSO$
      Token(), NumToken(), AtToken(), TokenUpper(), TokenSep(), CSetRef()
   $END$
 */

/* $DOC$
   $NAME$
      TokenUpper()
   $CATEGORY$
      CT3 string functions
   $ONELINER$
      Change the first letter of tokens to upper case
   $SYNTAX$
      TokenUpper( <[@]cString>, [<cTokenizer>], [<nTokenCount>],
                  [<nSkipWidth>] ) -> cString
   $ARGUMENTS$
      <[@]cString>      is the processed string
      [<cTokenizer>]    is a list of characters separating the tokens
                        in <cString>
                         Default: Chr( 0 ) + Chr( 9 ) + Chr( 10 ) + Chr( 13 ) +
                                  Chr( 26 ) + hb_BChar( 138 ) + hb_BChar( 141 ) +
                                  Chr( 32 ) + ",.;:!\?/\\<>()#&%+-*"
      [<nTokenCount>]   specifies the number of tokens that
                        should be processed
                        Default: all tokens
      [<nSkipWidth>]    specifies the maximum number of successive
                        tokenizing characters that are combined as
                        ONE token stop, e.g. specifying 1 can
                        yield to empty token
                        Default: 0, any number of successive tokenizing
                        characters are combined as ONE token stop
   $RETURNS$
      <cString>         the string with the uppercased tokens
   $DESCRIPTION$
      The TokenUpper() function changes the first letter of tokens in <cString>
      to upper case. To do this, it uses the same tokenizing mechanism
      as the Token() function. If TokenUpper() extracts a token that starts
      with a letter, this letter will be changed to upper case.
      You can omit the return value of this function by setting the CSetRef()
      switch to .T., but you must then pass <cString> by reference to get
      the result.
   $EXAMPLES$
      ? TokenUpper( "Hello, world, here I am!" )         // "Hello, World, Here I Am!"
      ? TokenUpper( "Hello, world, here I am!",, 3 )     // "Hello, World, Here I am!"
      ? TokenUpper( "Hello, world, here I am!", ",", 3 ) // "Hello, world, here I am!"
      ? TokenUpper( "Hello, world, here I am!", " w" )   // "Hello, wOrld, Here I Am!"
   $STATUS$
      Ready
   $COMPLIANCE$
      TokenUpper() is compatible with CT3's TokenUpper(),
      but a new 4th parameter, <nSkipWidth> has been added for
      synchronization with the the other token functions.
   $PLATFORMS$
      All
   $FILES$
      Library is hbct.
   $SEEALSO$
      Token(), NumToken(), AtToken(), TokenLower(), TokenSep(), CSetRef()
   $END$
 */

/* $DOC$
   $NAME$
      TokenSep()
   $CATEGORY$
      CT3 string functions
   $ONELINER$
      Retrieves the token separators of the last Token() call
   $SYNTAX$
      TokenSep( [<lMode>] ) -> cSeparator
   $ARGUMENTS$
      [<lMode>]   if set to .T., the token separator BEHIND the token
                  retrieved from the Token() call will be returned.
                  Default: .F., returns the separator BEFORE the token
   $RETURNS$
      Depending on the setting of <lMode>, the separating character of the
      the token retrieved from the last Token() call will be returned.
      These separating characters can now also be retrieved with the Token()
      function.
   $DESCRIPTION$
      When one does extract tokens from a string with the Token() function,
      one might be interested in the separator characters that have been
      used to extract a specific token. To get this information you can
      either use the TokenSep() function after each Token() call, or
      use the new 5th and 6th parameter of the Token() function.
   $EXAMPLES$
      see Token() function
   $STATUS$
      Ready
   $COMPLIANCE$
      TokenSep() is compatible with CT3's TokenSep().
   $PLATFORMS$
      All
   $FILES$
      Library is hbct.
   $SEEALSO$
      Token(), NumToken(), AtToken(), TokenLower(), TokenUpper()
   $END$
 */
