* harbour/tests/cpinfo.prg
* small modification for non latin CPs having first lower letter before
upper
* keep signature for generated CP files as "$Id' + "$" to avoid
automatic replacing by SVN
* harbour/include/hbapicdp.h
* harbour/include/hbcdpreg.h
* harbour/src/rtl/cdpapi.c
* harbour/tests/cpinfo.prg
+ added support to define CPs using different letter case sorting
in human readable form. Now it's possible to use:
// uppers before lowers: ABCDE...abcde...
#define HB_CP_CSSORT HB_CDP_CSSORT_UPLO
// uppers and lowers are mixed: AaBbCcDdEe....
#define HB_CP_CSSORT HB_CDP_CSSORT_MIXED
// ignore case
#define HB_CP_CSSORT HB_CDP_CSSORT_IGNORE
* harbour/src/rtl/gtwin/gtwin.c
* harbour/src/rtl/gtwvt/gtwvt.h
* harbour/src/rtl/gtwvt/gtwvt.c
* modified to use in UNICODE builds current HVM CP if user
has not set display CP using HB_SETTERMCP() or HB_SETDISPCP()
* harbour/src/rtl/gtwvt/gtwvt.h
* harbour/src/rtl/gtwvt/gtwvt.c
* eliminated most of HB_TCHAR_*() translations
% optimized setting SelectCopy menu item name to avoid unnecessary
memory allocation and to use hb_str*() API
! allow to set window title before displaying the window
(HB_GTI_WINTITLE)
% use hb_str*() API for window title setting
! removed redundant window title setting just after creating new window
! pass pointer to HB_GTWVT structure as lpParam in CreateWindow()
function to allow WM_CREATE message processing and early window
handle registration
* harbour/tests/cpinfo.prg
* added HB_SETTERMCP() setting
* harbour/examples/gtwvw/tests/cbtest1.prg
* harbour/examples/gtwvw/tests/cbtest6.prg
* harbour/examples/gtwvw/tests/ebtest7.prg
* small cleanup to compile with Harbour
* harbour/src/rtl/cdpapi.c
* updated to work well with CPs where CHR( 0 ) is not sorted as 1-st
character
* harbour/tests/cpinfo.prg
* chose binary form for CPs where upper and lower letters are sorted
in different order
* harbour/src/codepage/cpnl850.c
* updated to binary form
* harbour/include/hbdefs.h
* added small hack to eliminate negative handles from 32 bit windows
builds. Please remember that this trick can work only in 32 bit
windows version. In 64 bit ones valid handle after casting to integer
value can give negative 64 bit number so checking for wrong handle
please compare them with FS_ERROR (-1) instead of using < 0 or sth
like that. Otherwise your code may not work properly with some new
windows versions or may even suddenly stop to work with existing
ones even if they worked for some time without problems.
* harbour/src/rdd/dbf1.c
* pacified warning after adding above hack
* harbour/tests/cpinfo.prg
! removed unnecessary EOL reported by Viktor
* harbour/include/hbcdpreg.h
* use macro instead of direct value
* harbour/tests/cpinfo.prg
+ generate .c files with CP definition in human readable form
when it's can be safely use or with optional binary table when
it should be verified if such form is acceptable.
* harbour/tests/cpinfo.prg
* harbour/include/hbapicdp.h
* harbour/include/hbcdpreg.h
* modified format used for CP definition using binary tables
+ added Latin character striping from non Latin based CPs
* changed warning message to be more clear. In most of cases when
this message is shown:
Warning: irregular CP which needs verification for human
readable definition in Harbour
it should be possible to create human non binary CP definition.
Usually it should be enough to copy HB_CP_UPPER and HB_CP_LOWER
strings and verify if final result are compatible by comparing
automatically generated .c file with binary tables. If you will
find some CPs where it's not compatible then please inform me
about it and I'll try to update cpinfo.prg
When this message is shown:
Warning: irregular CP which needs special definition in Harbour
using binary tables generated by this program
then binary tables have to be used to replicate exact CP conditions.
If no warning message is shown then human readable form of CP
definition can be used without (I hope ;)) additional verification.
* harbour/src/codepage/cpno865.c
* harbour/src/codepage/cpit850m.c
* harbour/src/codepage/cpis850.c
* harbour/src/codepage/cpfr850.c
* harbour/src/codepage/cpes850c.c
* harbour/src/codepage/cpsv850m.c
* harbour/src/codepage/cpde850m.c
* harbour/src/codepage/cppt860.c
* harbour/src/codepage/cpnl850m.c
* harbour/src/codepage/cpnl850.c
* harbour/src/codepage/cpde850.c
* harbour/src/codepage/cpfi850.c
* harbour/src/codepage/cpit850.c
* harbour/src/codepage/cpis861.c
* harbour/src/codepage/cpes850m.c
* harbour/src/codepage/cpfr850m.c
* harbour/src/codepage/cpsv437c.c
* harbour/src/codepage/cpdk865.c
* harbour/src/codepage/cpgr437.c
* updated for new format
* harbour/tests/cpinfo.prg
+ added additional information that binary tables generated by cpinfo.prg
have to be used instead of human readable form when such situation can
be easy detected
* harbour/src/codepage/cpsvclip.c
* replaced old definition with binary tables generated by cpinfo.prg
to fully replicate wrong Clipper NTXSWE.OBJ definition.
* harbour/tests/cpinfo.prg
+ try to generate HB_CP_UPPER and HB_CP_LOWER strings which are ready to
use in Harbour CP definition using human readable form.
Warning: please remember that not all CPs can be created in human
readable form and if you want to create Clipper compatible
definition then it should be always verified by comparing .c
files generated by cpinfo compiled using Clipper and Harbour.
* harbour/ChangeLog
* marked last TOFIX note as [DONE] - thanks to Mindaugas
* harbour/src/rtl/cdpapi.c
* use 1-st upper<->lower conversion pair instead of last one
if more then one conversion pair exist - it's more natural
form in most of cases
% skip translation code when two different CDP modules use
the same Unicode table
* harbour/tests/cpinfo.prg
+ detect CPs which have corresponding upper or lower characters
for non alpha characters and inform about it.
Such situations is in NTXSPA.OBJ so such CP can be fully
replicated in Harbour using alternative CP definition only
which uses static tables.
* harbour/src/codepage/cpes850c.c
* harbour/src/codepage/cpesisoc.c
* harbour/src/codepage/cpeswinc.c
! removed the hack with changed order for characters which do not
have corresponding upper latter.
Now there is only one difference between ES850C and Clipper NTXSPA.OBJ
In NTXSPA 'UPPER( Chr( 163 ) )' gives "U" though Chr( 163 ) is not
marked as letter and not sorted with other letters. For me it looks
like a bug anyhow if you want to fully replicate NTXSPA.OBJ in Harbour
then current cpes850c.c code should be replaced by code generated
using tests/cpinfo.prg compiler by clipper and linked with ntxspa.obj.
Such code should be generated using:
cpinfo.exe es850c "Spanish CP-850 (ntxspa.obj compatible)" 850
Now translations can be done between CPs using different number of
letters or even defined for different languages so I have question
to Spanish users: Do you need ESISOC and ESWINC for anything?
If not then I would like to remove them.
Similar situation is with HUISOS and HUWINS.
* harbour/include/hbapicdp.h
* harbour/src/rtl/cdpapi.c
* harbour/src/codepage/ucmacce.c
* harbour/src/codepage/uckoi8.c
* harbour/src/codepage/uc885910.c
* harbour/src/codepage/ucmacice.c
* harbour/src/codepage/uc885914.c
* harbour/src/codepage/uc874.c
* harbour/src/codepage/uc857.c
* harbour/src/codepage/uc1250.c
* harbour/src/codepage/uc1254.c
* harbour/src/codepage/uc1258.c
* harbour/src/codepage/uckoi8u.c
* harbour/src/codepage/uckam.c
* harbour/src/codepage/uc424.c
* harbour/src/codepage/ucmacrom.c
* harbour/src/codepage/uc862.c
* harbour/src/codepage/cpesisoc.c
* harbour/src/codepage/uc866.c
* harbour/src/codepage/uc8859_3.c
* harbour/src/codepage/cpes850c.c
* harbour/src/codepage/uc8859_7.c
* harbour/src/codepage/ucmacgrk.c
* harbour/src/codepage/uc037.c
* harbour/src/codepage/uc850.c
* harbour/src/codepage/uc885911.c
* harbour/src/codepage/uc885915.c
* harbour/src/codepage/uc875.c
* harbour/src/codepage/uc1251.c
* harbour/src/codepage/uc1255.c
* harbour/src/codepage/uc863.c
* harbour/src/codepage/ucmaz.c
* harbour/src/codepage/uc1026.c
* harbour/src/codepage/uc8859_4.c
* harbour/src/codepage/uc8859_8.c
* harbour/src/codepage/uc855.c
* harbour/src/codepage/cpeswinc.c
* harbour/src/codepage/uc885916.c
* harbour/src/codepage/ucascii.c
* harbour/src/codepage/uc1252.c
* harbour/src/codepage/uc1256.c
* harbour/src/codepage/ucatari.c
* harbour/src/codepage/uc860.c
* harbour/src/codepage/ucmaccyr.c
* harbour/src/codepage/ucmik.c
* harbour/src/codepage/uc864.c
* harbour/src/codepage/uc1006.c
* harbour/src/codepage/uc1125.c
* harbour/src/codepage/uc8859_1.c
* harbour/src/codepage/uc88591b.c
* harbour/src/codepage/uc8859_5.c
* harbour/src/codepage/uc8859_9.c
* harbour/src/codepage/uc852.c
* harbour/src/codepage/ucnext.c
* harbour/src/codepage/uc737.c
* harbour/src/codepage/uc885913.c
* harbour/src/codepage/ucmactrk.c
* harbour/src/codepage/uc856.c
* harbour/src/codepage/uc775.c
* harbour/src/codepage/uc1253.c
* harbour/src/codepage/uc1257.c
* harbour/src/codepage/uc500.c
* harbour/src/codepage/uc861.c
* harbour/src/codepage/uc865.c
* harbour/src/codepage/uc869.c
* harbour/src/codepage/uc8859_2.c
* harbour/src/codepage/uc8859_6.c
% declare Unicode tables with 'const' qualifier
* harbour/src/vm/set.c
! fixed small typos in returned values (FALSE instead of NULL and 0)
* harbour/src/vm/estack.c
* make calling hb_stackTotalItems() safe in MT mode when stack is not
initialized - now this function can be used to check if HVM stack
exists and is initialized in ST and MT mode
* harbour/src/rtl/errintlo.c
! fixed at least 4 GPF traps exploited when hb_errInternal() was called
and HVM stack was not initialized
* harbour/include/Makefile
+ harbour/include/hbcdpreg.h
* harbour/include/hbapicdp.h
* harbour/src/rtl/cdpapi.c
+ added new CDP API which fixes many errors and limitations in old
implementation.
Warning !!! some very serious bugs in national sorting have been
fixed. It may cause that old indexes using some national
sorting will not be compatible. Please rebuild existing
indexes.
* harbour/src/vm/itemapi.c
* harbour/src/rdd/hsx/hsx.c
* harbour/src/rdd/dbfntx/dbfntx1.c
* harbour/src/rdd/dbfnsx/dbfnsx1.c
* harbour/src/rdd/dbfcdx/dbfcdx1.c
* harbour/contrib/hbbmcdx/bmdbfcdx.c
* harbour/contrib/rddads/adsx.c
* harbour/contrib/rddsql/sqlmix.c
* updated to work with new CDP API
TOFIX: update adsx.c and sqlmix.c to use hb_cdpcmp() which resepcts
sorting with accented and mulitybyts characters.
Mindaugas, can you update above code?
* harbour/src/codepage/cpbg866.c
* harbour/src/codepage/cpbgiso.c
* harbour/src/codepage/cpbgmik.c
* harbour/src/codepage/cpbgwin.c
* harbour/src/codepage/cpcs852.c
* harbour/src/codepage/cpcsiso.c
* harbour/src/codepage/cpcskam.c
* harbour/src/codepage/cpcswin.c
* harbour/src/codepage/cpde850.c
* harbour/src/codepage/cpdeiso.c
* harbour/src/codepage/cpdewin.c
* harbour/src/codepage/cpel737.c
* harbour/src/codepage/cpeliso.c
* harbour/src/codepage/cpelwin.c
* harbour/src/codepage/cpes850.c
* harbour/src/codepage/cpes850c.c
* harbour/src/codepage/cpesiso.c
* harbour/src/codepage/cpesisoc.c
* harbour/src/codepage/cpeswin.c
* harbour/src/codepage/cpeswinc.c
* harbour/src/codepage/cpeswinm.c
* harbour/src/codepage/cpfr850.c
* harbour/src/codepage/cpfriso.c
* harbour/src/codepage/cpfrwin.c
* harbour/src/codepage/cphr437.c
* harbour/src/codepage/cphr852.c
* harbour/src/codepage/cphrwin.c
* harbour/src/codepage/cphu852.c
* harbour/src/codepage/cphu852s.c
* harbour/src/codepage/cphuiso.c
* harbour/src/codepage/cphuisos.c
* harbour/src/codepage/cphuwin.c
* harbour/src/codepage/cphuwins.c
* harbour/src/codepage/cpit437.c
* harbour/src/codepage/cpit850.c
* harbour/src/codepage/cpitisb.c
* harbour/src/codepage/cpitiso.c
* harbour/src/codepage/cpitwin.c
* harbour/src/codepage/cpltwin.c
* harbour/src/codepage/cppl852.c
* harbour/src/codepage/cppliso.c
* harbour/src/codepage/cpplmaz.c
* harbour/src/codepage/cpplwin.c
* harbour/src/codepage/cppt850.c
* harbour/src/codepage/cpptiso.c
* harbour/src/codepage/cpro852.c
* harbour/src/codepage/cproiso.c
* harbour/src/codepage/cprowin.c
* harbour/src/codepage/cpru866.c
* harbour/src/codepage/cpruiso.c
* harbour/src/codepage/cprukoi.c
* harbour/src/codepage/cpruwin.c
* harbour/src/codepage/cpsk852.c
* harbour/src/codepage/cpskiso.c
* harbour/src/codepage/cpskkam.c
* harbour/src/codepage/cpskwin.c
* harbour/src/codepage/cpsl437.c
* harbour/src/codepage/cpsl852.c
* harbour/src/codepage/cpsliso.c
* harbour/src/codepage/cpslwin.c
* harbour/src/codepage/cpsrwin.c
* harbour/src/codepage/cpsv850.c
* harbour/src/codepage/cpsvclip.c
* harbour/src/codepage/cpsviso.c
* harbour/src/codepage/cpsvwin.c
* harbour/src/codepage/cptr857.c
* harbour/src/codepage/cptriso.c
* harbour/src/codepage/cptrwin.c
* harbour/src/codepage/cpua1125.c
* harbour/src/codepage/cpua866.c
* harbour/src/codepage/cpuakoi.c
* harbour/src/codepage/cpuawin.c
* harbour/src/codepage/uc037.c
* harbour/src/codepage/uc1006.c
* harbour/src/codepage/uc1026.c
* harbour/src/codepage/uc1125.c
* harbour/src/codepage/uc1250.c
* harbour/src/codepage/uc1251.c
* harbour/src/codepage/uc1252.c
* harbour/src/codepage/uc1253.c
* harbour/src/codepage/uc1254.c
* harbour/src/codepage/uc1255.c
* harbour/src/codepage/uc1256.c
* harbour/src/codepage/uc1257.c
* harbour/src/codepage/uc1258.c
* harbour/src/codepage/uc424.c
* harbour/src/codepage/uc500.c
* harbour/src/codepage/uc737.c
* harbour/src/codepage/uc775.c
* harbour/src/codepage/uc850.c
* harbour/src/codepage/uc852.c
* harbour/src/codepage/uc855.c
* harbour/src/codepage/uc856.c
* harbour/src/codepage/uc857.c
* harbour/src/codepage/uc860.c
* harbour/src/codepage/uc861.c
* harbour/src/codepage/uc862.c
* harbour/src/codepage/uc863.c
* harbour/src/codepage/uc864.c
* harbour/src/codepage/uc865.c
* harbour/src/codepage/uc866.c
* harbour/src/codepage/uc869.c
* harbour/src/codepage/uc874.c
* harbour/src/codepage/uc875.c
* harbour/src/codepage/uc885910.c
* harbour/src/codepage/uc885911.c
* harbour/src/codepage/uc885913.c
* harbour/src/codepage/uc885914.c
* harbour/src/codepage/uc885915.c
* harbour/src/codepage/uc885916.c
* harbour/src/codepage/uc88591b.c
* harbour/src/codepage/uc8859_1.c
* harbour/src/codepage/uc8859_2.c
* harbour/src/codepage/uc8859_3.c
* harbour/src/codepage/uc8859_4.c
* harbour/src/codepage/uc8859_5.c
* harbour/src/codepage/uc8859_6.c
* harbour/src/codepage/uc8859_7.c
* harbour/src/codepage/uc8859_8.c
* harbour/src/codepage/uc8859_9.c
* harbour/src/codepage/ucascii.c
* harbour/src/codepage/ucatari.c
* harbour/src/codepage/uckam.c
* harbour/src/codepage/uckoi8.c
* harbour/src/codepage/uckoi8u.c
* harbour/src/codepage/ucmacce.c
* harbour/src/codepage/ucmaccyr.c
* harbour/src/codepage/ucmacgrk.c
* harbour/src/codepage/ucmacice.c
* harbour/src/codepage/ucmacrom.c
* harbour/src/codepage/ucmactrk.c
* harbour/src/codepage/ucmaz.c
* harbour/src/codepage/ucmik.c
* harbour/src/codepage/ucnext.c
* updated for new CP API
Warning: please test all national CPs. I'm not able to test unknown for
me national character conditions so it's possible that I made
something wrong. Additionally current CP API does not have
some limitations which existed in the old one so now some hacks
or workarounds can be cleaned. I.e. now all translations are
done using Unicode tables so it's not necessary to use the
same number of characters to make them working.
Updating CP files I've found the following potential problems
in definitions which I haven't fixed or fixed only partially:
cpes850c.c - update for new format removing hacks for old CP
implementation
cpesisoc.c > remove, it's not longer necessary
cpeswinc.c /
cpeswinm.c - it is the same as cpesiso.c so it should be removed
cphu852.c \
cphu852s.c \
cphuiso.c > update for new format removing hacks for old CP
cphuwin.c / implementation
cphuisos.c > remove, it's not longer necessary
cphuwins.c /
cphr437.c > does it really use CP-437? and ^]\[@~}|{` instead of
cpsl437.c / nationalletters or it uses different then 437 CP for
7-bit Croatian characters encoding. It's important
because without valid unicode table translations will
not work.
cpit437.c - update for new version if necessary (upper characters)
cpsv850.c \
cpsviso.c > they had ACCENTED_INTERLEAVED but without ACCENTED_EQUAL
cpsvwin.c / so this setting was ignored. Now it's not ignored.
cpsvclip.c - update for new format to be fully not only
"almost compatible with Clipper's NTXSWE.OBJ"
For sure these are not all problems inherited from old CP
implementation so please verify all existing CPs.
* harbour/src/codepage/cp_tpl.c
+ added new example for CP definition. Working on new CPs please read
this description:
If accented characters need special sorting ( HB_CDP_ACSORT_EQUAL or
HB_CDP_ACSORT_INTERLEAVE ) then you need to mark the accented
characters with the symbol '~' before each of them, for example:
a~_
If there is two-character sequence, which is considered as one, it
should be marked with '.' before and after it, for example:
... h.ch.i ...
if such multibyte character has its own Unicode representation then
this Unicode value can be set using '=' symbol, for example:
....h.ch=2A7C.i
and it will be used in translations.
The Upper case string and the Lower case string should use the
same letters. If some characters does not have corresponding upper
or lower letter then space ' ' can be used as dummy character, for
example in German CPs there is no upper case 'scharfes S' letter
so space is used as dummy character.
HB_CP_LATIN should be 1 if the national alphabet is based on Latin
otherwise 0
Please note that now you can use two versions of CP definition.
The 1-st one which uses human readable form similar to the old CP
definition and 2-nd which uses directly final CP tables so it
allows to replicate any conditions which existed in Clipper or
any other compatible language which uses static tables.
tests/cpinfo.prg can generate automatically .c files with new CP
definition in the 2-nd version. Now if user wants to create new
Harbour CP which is fully Clipper compatible he only has to
compile and link cpinfo.prg with given national sorting module
and then execute it with new Harbour CP name as 1-st parameter.
cp<cpname>.c file is generated which can be added to Harbour
repository. Optional 2-nd parameter is CP description and 3-rd
is Harbour Unicode table ID (if not given 437 is used and should
be updated by user in .c file). This method should definitely
resolve any problems with creating Clipper compatible CP definition.
It also allows to easy verify if CP definitions are compatible. It's
enough to compare generated .c files.
* harbour/src/codepage/uckam.c
! fixed Kamenicky Unicode table definition - it was Mazovia CP :)
* harbour/src/codepage/cpde850.c
* harbour/src/codepage/cpdeiso.c
* harbour/src/codepage/cpdewin.c
! fixed CP wrongly encoded CP due to limitations in old CP definition.
Now it's possible to define different number of upper an lower
characters (space " " can be used as dummy character) and I used
it to eliminate hack for 'scharfes S' which does not exists as upper
case.
Please clean other CPs which contain similar hacks.
* harbour/src/codepage/cphu852.c
* harbour/src/codepage/cphuiso.c
* harbour/src/codepage/cphuwin.c
* harbour/src/codepage/cpua866.c
* removed dummy letters introduced for old translation code
* harbour/src/rtl/natmsg.c
* harbour/src/rtl/cdpapi.c
* harbour/include/hbextern.ch
+ added HB_CDPINFO() function
* redirected _NATSORTVER() function to HB_CDPINFO()
To All,
Please check your national sorting in current version and make
any updates if necessary or inform me about found problems.
I've checked that Polish CPs are working fine and all existing
CPs passes my test program which verified some technical aspects
of CP implementation. But I cannot verify if this definitions
satisfy users in different countries.
+ harbour/tests/cpinfo.prg
+ added simple program to generate information for Harbour CP module
definition. Compile it with Clipper and link with given national
sorting module (usually ntx*.obj) and then execute to generate
letters strings for given national sorting module. Then use this
string to define Harbour CP module in source/codepage/ directory.
