* harbour/include/hbdefs.h
* moved HB_WCHAR definition from hbapicdp.h to hbdefs.h
* harbour/include/hbapicdp.h
* harbour/include/hbcdpreg.h
* harbour/src/rtl/cdpapi.c
* harbour/src/rtl/cdpapihb.c
+ added support for custom sorting redirected from HVM
+ added support for custom character indexes in strings
+ added support for custom character flags (upper, lower, alpha, digit)
+ added support for custom upper/lower conversions
+ added support for CPs using unicode character values instead of
ASCII ones
+ added new CP functions which respects custom CP settings:
hb_cdpUpperWC(), hb_cdpTextLen(), hb_cdpTextPos(),
hb_cdpTextPosEx(), hb_cdpTextGetU16(), hb_cdpTextPutU16(),
hb_cdpCharEq(), hb_cdpCharCaseEq()
+ added new conversion functions:
hb_cdpGetUC(), hb_cdpGetWC(), hb_cdpGetU16Ctrl()
+ added macros to detect codepage parameters:
HB_CDP_ISBINSORT() - codepage uses simple binary sorting
HB_CDP_ISCUSTOM() - codepage uses custom string decoding
HB_CDP_ISCHARIDX() - codepage use character indexes instead
of bytes ones
HB_CDP_ISCHARUNI() - CHR(), ASC() and similar functions operates
on Unicode values instead of bytes
HB_CDP_ISUTF8() - codepage uses UTF-8 encoding
* harbour/include/inkey.ch
- removed HB_INKEY_EXTENDED - it was not used in Harbour
+ added new flag HB_INKEY_EXT - it allows to used different
event encoding system with automatic translation to
standard Clipper values. It will be used in the future
as base for low level GTs code. Now it's possible to
selectively switch to the new system. New codes are
unique and do not interacts with Clipper.
+ added HB_INKEY_ALL macro - it's similar to INKEY_ALL but
enables also GTEVENT keys.
* harbour/include/hbgtcore.h
* harbour/src/rtl/hbgtcore.c
* changed character values in internal screen buffer to unicode.
Warning: this modification interacts with SAVESCREE()/RESTSCREEN()
data. Now all GTs uses extended definition. If somene
has code which needs VGA like screen buffers returned
by SAVESCREE() then it can be forced by
hb_gtInfo( HB_GTI_COMPATBUFFER, .t. )
Anyhow it will for to use only one CP in output.
+ added new GT methods which operated on HB_WCHAR values
* modified existing methods using HB_WCHAR/HB_USHORT parameters
to operate on HB_WCHAR values
+ added codepage conversion code directly to core code so it can
be eliminated from low level GT drivers which do not need some
special operations on it.
+ added support for extended inkey codes, they allow to encoded
unicode values, mouse events with flags, keyboard events with
modifier flags (shift,alt,ctrl,...), etc.
Now in low level GT code only unicode key values are used but
I plan to updated all GTs and switch to new code - it should
resolve many small problems inheriting with Clipper inkey codes.
* harbour/include/hbapifs.h
* moved hb_fsNameConvU16() definition from hbapicdp.h to hbapifs.h
* harbour/include/hbxvm.h
- removed comment copied by mistake from GT header file
* harbour/include/hbwinuni.h
* changed HB_CHARDUP() and HB_CHARDUPN() macros - now they
operate on functions which can be used without active HVM.
In such case ANSI CP is used as source encoding.
+ added new macros: HB_OSSTRDUP() and HB_OSSTRDUP2().
They make conversions from OS encoding to HVM one.
They can be used without active HVM and in such case ANSI CP
is used as destination encoding.
* harbour/include/hbapi.h
* harbour/src/vm/cmdarg.c
+ added new function hb_cmdargProgName()
It returns application name with path or NULL if not set,
caller must free returned value with hb_xfree() if not NULL.
The string is in HVM encoding so it does not need any additional
encodings.
* modified hb_cmdargString() to return strings in HVM encoding.
% some small code optimizations
; NOTE: parameters passed to MS-WINDOWS GUI programs which use
WinMain() instead of main() as startup entry are translated
to ANSICP before they can be accessed by application.
This can be eliminated though not for console programs
using where C compiler makes such translation to main()
parameters.
in all systems parameters are passed to application
startup and init functions before programmer can set OS CP.
This can be resolved by adding:
INIT PROC CLIPINIT()
SET( _SET_CODEPAGE, <userCP> )
SET( _SET_OSCODEPAGE, <systemCP> )
RETURN
to linked code. INIT procedures called CLIPINIT() are
executed before any other ones.
* harbour/include/hbapi.h
* harbour/src/vm/set.c
+ added new functions for CP converisons which can be used with
and without active HVM:
hb_osStrEncode(), hb_osStrEncodeN(),
hb_osStrDecode(), hb_osStrDecode2(),
hb_osStrU16Encode(), hb_osStrU16EncodeN(),
hb_osStrU16Decode(), hb_osStrU16Decode2(),
* harbour/include/hbvm.h
* harbour/src/vm/hvm.c
+ added new function hb_vmIsReady() - it's similar to hb_vmActive()
but it also checks if current thread has active HVM stack.
* moved hb_cdpReleaseAll() to the end of HVM cleanup code so
CP conversions are longer active.
* harbour/include/hbcomp.h
* harbour/src/compiler/cmdcheck.c
* harbour/src/compiler/hbusage.c
* harbour/src/common/expropt2.c
+ added new compiler switch:
-ku - strings in user encoding
Now it informs compiler that strings use custom encoding so some
optimizations which are byte oriented cannot be used.
It's possible that in the future we will change above definition
to sth like: "strings in UTF8 encoding" but now I would like to
keep more general.
* harbour/src/vm/macro.c
* inform macrocompiler about custom CPs using own character indexes
to disable byte oriented optimizations.
* harbour/include/hbapigt.h
* harbour/src/rtl/gtapi.c
* harbour/src/rtl/inkeyapi.c
+ added HB_B_*_W macros with unicode box character definitions
+ added HB_MBUTTON_* macros
+ added new GT functions: hb_gtHostCP() and hb_gtBoxCP() which
allows to extract CPs used in translations by GTs.
+ added new function hb_inkeyKeyString() - it converts inkey value
to corresponding string
+ added new function hb_inkeyKeyStd() - it converts new extended
key value to standard Clipper one.
* harbour/include/hbapifs.h
* harbour/src/common/hbffind.c
* moved OS codepage translations fully to hb_fsFind*() functions.
It fixes few problems which existed before, i.e. double CP
conversions in MS-Windows builds and simplifies upper level code.
* harbour/src/pp/hbpp.c
* harbour/src/rtl/direct.c
* harbour/src/rtl/fssize.c
* harbour/src/rtl/file.c
* harbour/contrib/hbct/files.c
* eliminated not longer necessary CP conversions in code calling
hb_fsFind*() functions.
* harbour/src/common/hbgete.c
* moved OS codepage translations to hb_getenv(), hb_getenv_buffer()
and hb_setenv() functions. It fixes few problems which existed
before, i.e. double CP conversions in MS-Windows builds and
simplifies upper level code.
* harbour/src/rtl/net.c
* harbour/src/rtl/gete.c
* eliminated not longer necessary CP conversions in code calling
hb_getenv()/hb_setenv() functions.
; NOTE: additional parameters in HB_GETENV() and HB_SETENV() which
disabled CP conversions are not longer supported.
They were strictly platform dependent and ignored in chosen
cases (i.e. in MS-Windows UNICODE builds we always have to
convert strings transferred between HVM and OS. If someone
needs old functionality for other platform then he should
temporary disable _SET_OSCODEPAGE.
* harbour/src/common/hbver.c
* harbour/src/common/hbfsapi.c
* harbour/src/rtl/fstemp.c
* harbour/src/rtl/fslink.c
* eliminated HB_TCHAR_*() macros
* harbour/src/common/strwild.c
+ added supprot for custom CPs using own character indexes in:
hb_strMatchWild(),
hb_strMatchWildExact(),
hb_strMatchCaseWildExact()
* harbour/src/nortl/nortl.c
+ added new dummy function replacement for binaries which are not
linked with HVM.
* harbour/src/rtl/filesys.c
! fixed double CP conversions in MS-Windows builds of hb_fsCurDirBuff()
* use hb_vmIsReady() instead of hb_stackId() in file name conversions.
* use hb_cmdargProgName() in hb_fsBaseDirBuff()
* harbour/src/rtl/philes.c
* use hb_cmdargProgName() in HB_PROGNAME() function.
* harbour/src/rtl/gtcgi/gtcgi.c
* harbour/src/rtl/gtstd/gtstd.c
* harbour/src/rtl/gtpca/gtpca.c
* harbour/src/rtl/gtdos/gtdos.c
* harbour/src/rtl/gtos2/gtos2.c
* harbour/src/rtl/gtwin/gtwin.c
* harbour/src/rtl/gtwvt/gtwvt.h
* harbour/src/rtl/gtwvt/gtwvt.c
* harbour/src/rtl/gttrm/gttrm.c
* harbour/src/rtl/gtcrs/gtcrs.c
* harbour/src/rtl/gtsln/gtsln.c
* harbour/src/rtl/gtsln/kbsln.c
* harbour/src/rtl/gtsln/gtsln.h
* harbour/src/rtl/gtxwc/gtxwc.h
* harbour/src/rtl/gtxwc/gtxwc.c
* harbour/contrib/gtwvg/gtwvg.c
* harbour/contrib/gtwvg/gtwvg.h
* harbour/contrib/gtalleg/gtalleg.c
* harbour/contrib/hbqt/gtqtc/gtqtc.cpp
* harbour/contrib/hbqt/gtqtc/gtqtc.h
* updated to work with new unicode GT API
please make tests with different GTs - I'm not able to test
all of them, i.e. I do not have any OS2 machine.
* harbour/src/rtl/box.c
* harbour/src/rtl/oldbox.c
* harbour/src/rtl/scroll.c
* harbour/src/rtl/console.c
* updated to work with new unicode GT API and
CPs using custom character indexes
* harbour/src/rtl/at.c
* harbour/src/rtl/ati.c
* harbour/src/rtl/rat.c
* harbour/src/rtl/len.c
* harbour/src/rtl/transfrm.c
* harbour/src/rtl/left.c
* harbour/src/rtl/right.c
* harbour/src/rtl/substr.c
* harbour/src/rtl/stuff.c
* harbour/src/rtl/padc.c
* harbour/src/rtl/padl.c
* harbour/src/rtl/padr.c
* updated to work with CPs using custom character indexes
* harbour/src/rtl/chrasc.c
+ added support for HB_CDP_ISCHARUNI() CPs.
* harbour/src/rtl/mlcfunc.c
* rewritten from scratch to work with CPs using custom character
indexes
* harbour/src/rtl/accept.c
* updated to work with unicode inkey values and
CPs using custom character indexes
* harbour/src/rtl/strmatch.c
% small optimization
* harbour/src/rtl/Makefile
+ harbour/src/rtl/chruni.c
+ added new PRG functions which allows to make byte/binary and
unicode/character operations on strings:
HB_UCHAR( <nCode> ) -> <cText>
return string with U+nCode character in HVM CP encoding
HB_BCHAR( <nCode> ) -> <cText>
return 1 byte string with <nCode> value
HB_UCODE( <cText> ) -> <nCode>
return unicode value of 1-st character (not byte) in given string
HB_BCODE( <cText> ) -> <nCode>
return value of 1-st byte in given string
HB_ULEN( <cText> ) -> <nChars>
return string length in characters
HB_BLEN( <cText> ) -> <nBytes>
return string length in bytes
HB_UPEEK( <cText>, <n> ) -> <nCode>
return unicode value of <n>-th character in given string
HB_BPEEK( <cText>, <n> ) -> <nCode>
return value of <n>-th byte in given string
HB_UPOKE( [@]<cText>, <n>, <nVal> ) -> <cText>
change <n>-th character in given string to unicode <nVal> one and
return modified text
HB_BPOKE( [@]<cText>, <n>, <nVal> ) -> <cText>
change <n>-th byte in given string to <nVal> and return modified
text
* harbour/src/rtl/hbdoc.prg
* harbour/src/rtl/memvarhb.prg
* use HB_BCHAR() for binary string definitions
* harbour/src/rtl/hbi18n2.prg
* use hb_utf8CHR( 0xFEFF ) instead of hardcoded binary sting and
HB_BLEN() instead of LEN()
* harbour/src/rtl/inkey.c
+ added new functions:
HB_KEYCHAR( <nKey> ) -> <cChar>
HB_KEYSTD( <nExtKey> ) -> <nClipKey>
! use HB_INKEY_ALL instead of INKEY_ALL in LASTKEY()
* harbour/src/rtl/achoice.prg
* harbour/src/rtl/browse.prg
* harbour/src/rtl/menuto.prg
* harbour/src/rtl/tgetlist.prg
* harbour/src/rtl/teditor.prg
* harbour/src/rtl/tlabel.prg
* harbour/src/rtl/tpopup.prg
* harbour/src/rtl/radiobtn.prg
* harbour/src/rtl/radiogrp.prg
* harbour/src/rtl/wait.prg
* updated to work with unicode inkey() values
* harbour/src/rtl/listbox.prg
* use box.ch macros instead of explicit CHR(...) definitions
* harbour/src/rtl/ttopbar.prg
* updated to work with different type of CPs
* harbour/src/rtl/scrollbr.prg
* formatting
* harbour/src/rtl/mouse53.c
* use HB_MBUTTON_* macros instead of local ones
* harbour/src/codepage/cp_utf8.c
+ harbour/src/codepage/uc16def.c
+ harbour/src/codepage/utf8sort.c
* replaced UTF8ASC with new CP: UTF8EX
This CP uses character indexes instead of bytes one
and operates on unicode characters flags.
Tables for upper/lower conversions and upper/lower/alpha/digit
flags were generated automatically from
http://www.unicode.org/Public/UNIDATA/UnicodeData.txt
It also uses custom collation rules. It's very simple one
level sorting based on UTF8 C collation.
If someone needs some advanced sorting rules, then it's enough
to create copy of this cp with user custom version of UTF8_cmp()
and UTF8_cmpi() functions, i.e. they can be redirected to some
external library like ICU (icu-project.org).
* harbour/contrib/hbct/ctwin.c
* harbour/contrib/hbct/ctwin.h
* harbour/contrib/hbct/ctwfunc.c
* added support for new unicode GT API
* harbour/contrib/xhb/xhbfunc.c
* redirected HB_CMDARGARGV() to HB_PROGNAME()
* harbour/contrib/hbnf/origin.c
* redirected FT_ORIGIN() to HB_PROGNAME()
* harbour/contrib/hbnf/getenvrn.c
! windows version of FT_GETE() fully rewritten - it should
fix well known problems reported to the devel list. Please test.
% small optimization for other systems
* harbour/contrib/hbfship/exec.c
* use hb_cmdargProgName() in EXECNAME()
+ harbour/tests/uc16_gen.prg
+ added code which generates tables with unicode character flags
from http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
harbour/src/codepage/uc16def.c was generated by this code.
* harbour/tests/inkeytst.prg
* harbour/tests/wvtext.prg
* harbour/tests/gtkeys.prg
* use HB_INKEY_ALL
* harbour/include/harbour.hbx
* harbour/include/hbcpage.hbx
* harbour/include/hblang.hbx
* regenerated
; It was quite big peace of modifications and for sure not everything
is well tested so please make test and report problems you will find.
; This is basic version which introduce to HVM CPs with custom encodings.
Some contrib code has to be updated to work correctly with it.
I hope that developers interesting in will make necessary updates.
I haven't touched GTWVW code at all - sorry but it needs very serious
work to make it production ready and fix all existing problems.
; Special thanks to OTC - this firm sponsored adding basic UTF8 support
to HVM.
* harbour/include/hbapicdp.h
* harbour/src/rtl/cdpapi.c
+ added new C functions:
hb_cdpDupn(), hb_cdpStrDupU16(), hb_cdpnStrDupU16()
* harbour/include/hbwinuni.h
+ added new macros: HB_CHARDUP() and HB_CHARDUPN()
Warning: this macros uses HVM functions so cannot be used in
code which is executed without active HVM i.e. in
pure harbour compiler code (common library)
* harbour/src/rtl/filesys.c
* use hb_cdpStrDupU16()
* harbour/src/rtl/hbproces.c
! fixed double OS codepage conversion in hb_fsProcessRun()
* use HB_CHARDUP*() macros to respect _SET_CODEPAGE and _SET_OSCODEPAGE
in Windows builds
* harbour/include/hbwinuni.h
+ added HB_FSNAMECONV() macro - it's automatically redirected to
hb_fsNameConv() or hb_fsNameConvU16() functions depending on
UNICODE windows macro
* harbour/include/hbapifs.h
* harbour/src/rtl/filesys.c
+ added hb_fsNameConvU16() C function - it makes similar operatin to
hb_fsNameConv() but returns UTF16 string.
+ added hb_fsPipeUnblock() C function - currently it works only
in POSIX systems
* HB_FSNAMECONV() instead of HB_TCHAR_*() macros
* harbour/src/rtl/hbcom.c
* HB_FSNAMECONV() instead of HB_TCHAR_*() macros
% encapsulate port open and close operations inside
hb_vmUnlock()/hb_vmLock() - on some systems this can
be slow operations, i.e. close() is delayed until
byte in output buffer are not transmitted.
* harbour/src/rtl/hbproces.c
% prefer read then write in pipe operations
! in POSIX systems set unblocking mode for PIPE handles in
hb_fsProcessRun() function - it fixes potential deadlock
* harbour/src/common/strwild.c
* formatting
* harbour/src/rtl/dircmd.prg
% use space( n ) instead of repl( chr( 0 ), n ) for allocating
dirty buffer
* harbour/src/rtl/gttrm/gttrm.c
* harbour/src/rtl/gtsln/kbsln.c
* ignore broken UTF8 characters in input
* harbour/src/pp/ppcore.c
* pacified BCC warning
* harbour/src/compiler/harbour.yyc
* harbour/src/compiler/harbour.yyh
* regenerated with patched bison to pacify BCC warnings
* harbour/include/hbwinuni.h
+ added HB_ITEMCOPYSTR() macro
* harbour/src/common/hbwince.c
* redirected Beep() to MessageBeep() - it does not allow to set
time and duration but should generate some default system voice
signal - please test the results with real devices using WinCE/
Win Mobile
TODO: New MinGW support LockFileEx() and UnlockFileEx() but they are
part of coredll6 library not coredll. I guess it's designed to
use with Windows Mobile 6.x. Add compile and link time support
for it.
TODO: Add link time support for MinGWCE and non ARM processors.
New MinGWCE supports x86 compatible Intel CPUs for pocket devices.
* harbour/include/hbwinuni.h
+ added new macro HB_ITEMGETSTR()
* harbour/src/rtl/gtwin/gtwin.c
* modified to use new UNICODE API instead of HB_TCHAR_*()
* harbour/contrib/hbwin/win_misc.c
* rewritten WIN_GETCOMMANDLINEPARAM()
New version does not use old HB_TCHAR_*() API and respect multiple
quoting by "" (i.e. .\"myapp"li"cation".exe)
TOFIX: WIN_ANSITOWIDE() and WIN_WIDETOANSI() are wrong and have
to be fixed. Please remember that number of character
after conversion to/from UNICODE can be different then
original number of chars, i.e. multibyte characters
can be replaced by single Unicode character or one Unicode
character may use few bytes representation.
* src/vm/strapi.c
% Minor optimization (using cdp variable instead of repeated hb_vmCDP() calls.)
* include/hbwinuni.h
+ Added HB_ITEMPUTSTR[LEN]() macros.
+ Added HB_STRUNSHARE() macro.
! Fixed typos in HB_ARRAYSETSTRLEN() macro values.
* contrib/hbwin/win_prn2.c
+ Using new HB_ITEMPUTSTR() macro instead of HB_TCHAR*().
* contrib/hbwin/win_misc.c
! WIN_RUNDETACHED(): Fixed to guarantee writable buffer in second
parameter of CreateProcess(). I've choosen to always pass a writable
buffer, although MS requires this only for UNICODE mode.
* contrib/hbwin/wapi_commctrl.c
+ Using new UNICODE parameter passing macros instead of HB_TCHAR*() ones.
! Fixed return values of WAPI_IMAGELIST_GETIMAGEINFO() to be empty
string instead of unset (NIL) in case of error.
+ contrib/hbqt/filelist.mk
! Missed from previous commit.
* contrib/hbwin/win_regc.c
* Removed trailing spaces (realized I used a temp instance
of 64-bit Far Manager for the last couple of days)
(there will be more)
* harbour/src/vm/strapi.c
* harbour/include/hbapistr.h
+ added new functions:
hb_itemPutStr(), hb_itemPutStrUTF8(), hb_itemPutStrU16(),
hb_arrayGetStr(), hb_arrayGetStrUTF8(), hb_arrayGetStrU16(),
hb_arraySetStrLen(), hb_arraySetStrLenUTF8(), hb_arraySetStrLenU16(),
hb_arraySetStr(), hb_arraySetStrUTF8(), hb_arraySetStrU16(),
* harbour/include/hbwinuni.h
* removed unnecessary and danger casting in UNICODE version of
HB_RETSTR(), HB_RETSTRLEN(), HB_STORSTR() and HB_STORSTRLEN() macros
* modify HB_ARRAYSETSTR() and HB_ARRAYSETSTRLEN() macros to use new
STR API functions
* harbour/external/Makefile
* modified the order of compiled libraries for better performance in
parallel compilation (-j<N>)
* include/hbwinuni.h
+ Added HB_ARRAYSETSTR*(). Based on Przemek's post.
* contrib/hbwin/wce_simc.c
* contrib/hbwin/wce_smsc.c
* contrib/hbwin/win_prn1.c
* contrib/hbwin/win_prn2.c
* contrib/hbwin/win_regc.c
+ Using new UNICODE parameter passing macros instead of HB_TCHAR*() ones.
* Minor rework WIN_TEXTOUT() and WIN_GETTEXTSIZE() to not use hb_parclen()
and protect some more against wrong length parameter.
; TOFIX: HB_ARRAYSETSTR() usage crashes (tested in non-UNICODE with testprn3)
I didn't want to undo the whole thing, but can't find the err either.
; TOFIX: hb_parclen() usage.
; TOFIX: HB_ARRAYSETSTR() with NULL string parameter.
; TOFIX: HB_PARSTRDEF() to really default to empty string for non-string params.
; TODO: Remaining HB_TCHAR_*() usages.
; TODO: Peer-review what I did.
* contrib/hbwin/wapi_winbase.c
! Fixed for WinCE after recent addition of new wrappers.
This means exclusion of WAPI_WAITFORSINGLEOBJECTEX() and
WAPI_WAITFORMULTIPLEOBJECTSEX() for this platform.
! Minor fix in size of allocated buffer in WAPI_FORMATMESSAGE().
(buffer was too large in non-UNICODE).
* include/hbwinuni.h
+ Added HB_PARSTRDEF() macro. So far the same as HB_PARSTR(), but
it's supposed to mark places where hb_parcx() was used (where
WinAPI expects non-optional string parameter).
* contrib/hbwin/wapi_winbase.c
* contrib/hbwin/wapi_winuser.c
* contrib/hbwin/wapi_shellapi.c
* contrib/hbwin/legacyco.c
+ Using new UNICODE parameter passing macros instead of HB_TCHAR*() ones.
; NOTE: I'm unsure how to apply the same to wapi_commctrl.c, could anyone help?
Here the string is assigned to win structure and passed to winapi,
which may mean that string space should be kept there after
returning from function.
* contrib/hbxbp/xbpgeneric.prg
! Using HB_SYMBOL_UNUSED() to mark unused parameters, instead of
local solution.
% Minor optimizations.
* MSGBOX() changed to not display all text in bold.
+ Changed <BR> to <br /> in MSGBOX().
