64 Commits

Author SHA1 Message Date
e95afad4ee feat: Harbour RDD parity — NTX/CDX 100% compatible, FIELD-> works
Five RDD engine now matches Harbour DBFNTX and DBFCDX byte-for-byte
in ordering, seek, navigation, and field access. Verified against
Harbour 3.2.0dev with a 281-line comparison test covering:
  - Natural/NAME/CITY/AGE/SALARY/UPPER ordering
  - SEEK (exact/not-found), GoTop/GoBottom per order
  - DELETE/RECALL with SET DELETED
  - CDX compound index read with 5 tags (BYNAME, BYCITY, BYAGE, BYSAL, BYUNAME)
  - Reverse traversal

Fixes:

1. FIELD->NAME returned NIL
   GetAliasField returned interface{} but runtime expected hbrt.Value,
   so the type assertion in PushAliasField failed and pushed NIL.
   - workarea.go: change return type to hbrt.Value, handle FIELD/_FIELD
     as current-workarea alias, add SetAliasField
   - gengo.go: emit SetAliasField() for alias->field := value in both
     statement and expression contexts

2. OrdSetFocus(n) silently switched to natural order
   v.AsString() returns "" for a numeric Value, so OrderListFocus("")
   set current=-1.
   - indexrtl.go: convert numeric param via fmt.Sprintf("%d", ...)

3. CDX compound tag order mismatched Harbour
   Five decoded the structural B-tree which is alphabetical, but
   Harbour sorts tags by TagBlock (file offset = creation order).
   - cdx/cdx.go: sort tagEntries by offset ascending after decoding,
     matching hb_cdxIndexLoadAvailTags in dbfcdx1.c

4. OutStd()/OutErr() not registered — caused panic on call
   - hbrtl/console.go: add rtlOutStd/rtlOutErr implementations
   - hbrtl/register.go: register OUTSTD and OUTERR
   - analyzer.go: add OUTSTD/OUTERR to RTL known-functions

5. FIELD keyword triggered "undeclared variable" warnings
   - analyzer.go: add FIELD, _FIELD, M, MEMVAR as builtin constants

Tests:
  go test ./...        — ALL PASS (17 packages)
  FiveSql2 43/43       — 100%
  compat_harbour 51/51 — 100%
  Harbour diff         — 0 lines differ (281-line comparison)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 16:37:47 +09:00
486e466592 feat: FiveSql2 43/43, @byref, mutable closure, RTL 479, DateTime fix
Major changes since last commit:
- FiveSql2 SQL:1999 engine (10,458 LOC) — 43/43 ALL PASS
- 21 compiler/runtime bugs fixed (short-circuit AND/OR, FOR LOOP, etc.)
- @byref pass-by-reference via RefCell pattern
- Mutable closure capture (EnsureLocalRef + RefCell sharing)
- RTL: 400 → 479 functions (+79: file, string, datetime, hash, UTF-8)
- DateTime/Timestamp fully working (hb_DateTime, hb_Hour/Min/Sec, display)
- Reserved word guard (39 keywords blocked from function calls)
- AEval arg order fix (element before index)
- Closure capture redecl fix (unique _cap_ names per block)
- Hash/string indexing in ArrayPush/ArrayPop
- Harbour compat test suite: 51/51
- 4 docs: Porting Report, Implementation Plan, Optimization Plan, Commercialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:35:37 +09:00
d451b836a6 perf: inline Str/PadR/PadL/SubStr/Left/Right/At/IIF in gengo
13 more RTL functions inlined — no Frame/EndProc, no VM dispatch:
- Str(n,w,d) → fmt.Sprintf("%*.*f", w, d, n)
- PadR(s,n) → s + hbrtl.Spaces(n-len(s))
- PadL(s,n[,fill]) → Spaces(pad) + s or Repeat(fill, pad) + s
- SubStr(s,p,l) → s[p:p+l] with bounds check
- Left(s,n) → s[:n], Right(s,n) → s[len-n:]
- At(search,target) → strings.Index + 1
- IIF(cond,a,b) → if/else without function call

Also: Spaces() exported for generated code access.

50K SEEK random: 62ms (Harbour 67ms — Five FASTER!)
82/82 stress PASS. 14 packages ALL PASS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:16:38 +09:00
05ccef05e2 perf: EndProcFast — eliminate defer recover() from RTL hot paths
Problem: every RTL function calls defer t.EndProc() which does recover().
50K SEEK loop = 250K recover() calls = ~12ms wasted.

Solution: EndProcFast() skips recover (only needs endFrame restore).
Applied to ALL RTL functions in strings.go, rdd.go, missing.go, database.go.
EndProc() with recover kept for generated PRG code (needs BEGIN SEQUENCE).

Analysis (50K sequential SEEK breakdown):
  Go NTX Seek direct: 7ms (faster than Harbour 27ms!)
  PRG VM overhead:    38ms (Frame + RTL calls + key generation)
  Key generation:     25ms (Str+LTrim+PadL+PadR = 5 RTL Frame/EndProc per iter)

With EndProcFast: RTL overhead reduced ~30%.

CDX SCOPE: 2ms (Harbour 4ms — 2x FASTER!)
82/82 stress PASS. 14 packages ALL PASS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 21:43:39 +09:00
5b378318a0 perf: RTL optimization — cached WA, spaces pool, stack-alloc fmt_int64
rdd.go:
- getWA() cached type assertion (avoid repeated interface check)
- waCache stores last WA pointer → O(1) for repeated calls

strings.go:
- spacesCache[257]: pre-built space strings for pad sizes 0-256
- spaces(n) returns cached string (no Repeat allocation)
- PadR/PadL use spaces() for fill=" " (most common case)
- Str() uses spaces() for right-padding

missing.go:
- fmt_int64: stack-allocated [20]byte array (was heap make([]byte))
- Reverse iteration (no prepend overhead)
- PadC uses spaces() for left/right padding

Benchmark (ext4, home dir):
  10K APPEND: 28ms → 26ms (Harbour 27ms!)
  50K APPEND: 130ms → 113ms (13% improvement)
  50K SCAN: 24ms → 23ms
  50K DUPKEY: 42ms → 35ms (17% improvement)
  CDX SCOPE: 12ms → 10ms (17% improvement)

82/82 stress PASS. 14 packages ALL PASS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 18:16:22 +09:00
b7028791d6 fix: 5 seek/dbf bugs — 77/77 thorough Harbour compatibility
1. SOFTSEEK: use idx.CurRecNo() for positioning (was checking recNo > 0)
   - SEEK with SET SOFTSEEK ON now positions at next higher key
   - SEEK command reads SET SOFTSEEK at runtime (was compile-time only)
   - rtlDbSeek defaults to GetSetSoftSeek() when no explicit param

2. SET DELETED ON + INDEX: SkipIndexed skips deleted records
   - GoTopIndexed: skip deleted record at top position
   - SkipIndexed: inner loop continues past deleted records

3. Compound key (CITY+NAME): field name TrimSpace before lookup
   - evalKeyExprInner: TrimSpace on fieldName after FIELD-> strip
   - Fixed "CITY " != "CITY" mismatch from + operator splitting

4. SET INDEX TO filename: treated as string, not variable
   - gengo uses exprToString for SET INDEX TO (was emitExpr)
   - Prevents identifier being resolved as local variable

5. hasXBaseCommands: recursive scan into nested blocks
   - BEGIN SEQUENCE, IF, FOR, DO WHILE, SWITCH bodies now scanned
   - Fixes missing hbrdd import for DB commands inside blocks

Thorough test: 77 items (14 sections) covering exact/partial/soft seek,
SET DELETED, duplicate keys, numeric keys, compound keys, empty/single
table, state consistency, order switching, full traversal — all identical.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 14:08:51 +09:00
7e2a159b88 feat: CDX support + ORDSCOPE + cross-read Harbour compatibility
CDX Integration:
- IndexEngine interface: common for NTX Index and CDX Tag
- OrderListAdd: auto-detects .cdx/.ntx extension, opens CDX tags
- decodeCompoundLeaf: proper bit-packed tag directory decoding
  (was stub falling through to scanCompoundLeaves with wrong names)
- CDX Tag: added KeyLen(), KeyExpr(), ForExpr(), IsDescending(), Close()
- CDX compound recNo = direct byte offset (not page number)

ORDSCOPE:
- SetScope/ClearScope/SetScopeTop/SetScopeBottom on DBFArea
- GoTopIndexed: seeks to scopeTop, validates within scopeBottom
- GoBottomIndexed: seeks to scopeBottom boundary
- SkipIndexed: stops at scope boundaries (top and bottom)
- OrdScope RTL function registered (nScope: 0=TOP, 1=BOTTOM)
- scopeKeyFromValue: converts Value to padded key bytes

Index Order Management:
- OrderListFocus: handles numeric order ("2" → order 2)
- SET ORDER TO n: gengo emits hbrt.NtoS for int-to-string conversion
- IndexOrd/OrdCount/OrdName/OrdKey: real implementations (were stubs)
- OrderCount/CurrentOrder/OrderName/OrderKeyExpr accessors on DBFArea
- ClearScope on order switch (prevents stale scope)

Cross-read test: Harbour-created CDX → Five reads, 20/20 items match:
  NAME/CITY/ID seek, ORDSCOPE count, GoTop/GoBottom all identical

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 12:21:26 +09:00
6e78d12cc2 fix: 3 RDD compat bugs — FIELD->, AsNumInt Double, PACK/ZAP with index
Bug 1: FIELD->NAME in INDEX ON expression
- evalKeyExprInner: strip FIELD->/alias-> prefix before field lookup
- exprToString: handle AliasExpr (FIELD->NAME → "FIELD->NAME")

Bug 2: AsNumInt() on Double returned IEEE 754 raw bits
- Value.AsNumInt(): check tDouble and convert via Float64frombits
- Fixed array index crash when index is result of % modulo

Bug 3: PACK/ZAP crash with open indexes
- OrderListRebuild: fully implemented (was TODO stub)
  Saves index info, closes all, sets idxState=nil, recreates
- OrderCreate: set current=-1 during key evaluation (natural GoTo)
- PACK/ZAP: save/restore idxState, rebuild after operation
- Register __DBPACK, __DBZAP, DBRECALL symbol aliases

Harbour vs Five: 45/47 match (96%), 2 diffs are duplicate-key sort order

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 04:41:19 +09:00
21fd9dc65c feat: SET DELETED filtering, SEEK/LOCATE/CONTINUE, SET command codegen
- skipFilter: skip deleted records in GoTop/GoBottom/Skip when SET DELETED ON
- hbrdd.IsSetDeleted callback: avoids circular import hbrdd→hbrtl
- Parser: capture ON/OFF for boolean SET commands (DELETED, EXACT, SOFTSEEK, etc.)
- Parser: capture TO expr for SET DATE/DECIMALS/EPOCH
- Gengo: emit proper t.Do() calls for 11 SET toggles + 3 value SETs
- stmtSet: was stub (skipToEOL), now calls parseSet()
- RTL: register 11 SET toggle functions (SETDELETED, SETEXACT, etc.)
- RTL: DBLOCATE/DBCONTINUE for sequential search
- RTL: DBSETFILTER/DBCLEARFILTER/DBFILTER
- PadL/PadR: support 3rd param fill character
- Area interface: added SetFound, SetLocate, LocateBlock, filter methods
- MemRDD: implements new Area interface methods
- Comprehensive PRG test: test_search.prg (7 test suites all pass)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 22:33:59 +09:00
827adeeb99 feat: SET commands + ErrorBlock/Break error handling
SET commands (setcmd.go):
  - SetDateFunc: __SetDateFormat([cNew]) → cOld
  - SetDecimalsFunc: SET DECIMALS TO n
  - SetEpochFunc: SET EPOCH TO n
  - 11 toggle functions: SetExact, SetDeleted, SetSoftSeek, SetExclusive,
    SetFixed, SetCancel, SetBell, SetConfirm, SetInsert, SetEscape, SetWrap
  - SET constants: _SET_EXACT, _SET_DELETED, etc. for PRG code
  - GetSetDateFormat(), GetSetDecimals(), GetSetEpoch() helpers
  - Default: DATE="mm/dd/yy", EPOCH=1900, DECIMALS=2

Error handling (error.go):
  - Break(xValue): panics with BreakValue, caught by BEGIN SEQUENCE
  - BreakBlock(): returns {|e| Break(e)} code block
  - LaunchError(): dispatches error through ErrorBlock handler
  - RuntimeError(): creates + launches standard runtime error
  - IsBreak(): checks if recovered panic is a BreakValue
  - createErrorHash(): builds Harbour-compatible error hash

Registration: ErrorBlock, ErrorNew, DosError, FError, Break + all SET functions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 15:33:07 +09:00
f950cb0784 fix: Phase 2 — HIGH #6,9,10,11,12,19,23,32,46,47
Files modified (5):
  compiler/gengo/gengo.go — #6,#32: Deduplicate 3 Generate functions into 1
    doGenerate(file, debug, library) replaces 170 lines of copy-paste
    Dead GenerateDebug method removed
  cmd/five/main.go — #9,10,11,12: Fix 5 silently ignored errors
    filepath.Abs, tidy.Run, tidyCmd.CombinedOutput now checked
  hbrtl/strings.go — #19: Str() now reads caller's nWidth/nDec params
    Was ignoring explicit Str(123, 10, 2) arguments
  compiler/pp/pp.go — #46: Fix stale "NOT implemented" comment
    #47: Extract maxIncludeDepth constant
  compiler/lexer/lexer.go — #23: Remove unused LookupKeyword result

Issues resolved: 10 (HIGH: 7, MEDIUM: 3)
Total fixed: 26/53

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 11:47:26 +09:00
7c61db70c3 fix: Critical code review fixes — race conditions, panic recovery, LTRIM/RTRIM
CRITICAL fixes:
- fileio.go: Add sync.Mutex to file handle table (race condition #2)
  allocHandle/getHandle/removeHandle thread-safe helpers
- goroutine.go: Add defer/recover to GoLaunch/GoLaunchBlock
  Goroutine panic no longer crashes entire process (#5)

HIGH fixes:
- strings.go: Implement proper LTrim (TrimLeft) and RTrim (TrimRight)
  Previously both aliased to AllTrim — silent semantic bug (#18)
- register.go: TRIM = RTrim (Harbour compatible)

From 53-issue senior code review.
Remaining: 47 issues (HIGH: 10, MEDIUM: 18, LOW: 16, CRITICAL: 3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 10:17:30 +09:00
272576f6ce feat: Harbour-compatible VM shutdown sequence
Implements full cleanup on program exit (normal, Ctrl+C, crash):
- EXIT PROCEDURE auto-execution (reverse module order)
- AtExit callback registry (LIFO order)
- All WorkAreas auto-close (child before parent)
- Terminal restore (raw → normal) on signal/exit
- Static variables clear
- Signal handlers (SIGINT, SIGTERM) for clean shutdown
- shutdown.go: Harbour hb_vmQuit() 25-step sequence adapted for Five
- vm.go: Run() now calls Shutdown() via defer
- rawtty.go: terminal restore registered with shutdown system

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 10:07:42 +09:00
59568f3301 Five v0.9 — Harbour + Go fusion language
- Compiler: PP → Lexer → Parser → Analyzer → Gengo pipeline
- Parser: 232/236 (98%) Harbour compatibility, registry-based dispatch
- RTL: 351 Harbour-compatible functions
- RDD: DBF/NTX/CDX engines with Rushmore bitmap optimization
- Go Interop: IMPORT + pkg.Func() + obj:Method() with FastPath (15M calls/sec)
- HB_FUNC API: Full Harbour C API compatible Go bridge
- Concurrency: SPAWN/LAUNCH/GOROUTINE, <-, WATCH, PARALLEL FOR, ASYNC/AWAIT
- Extensions: Multi-return, DEFER, Slice, f-string, Nil-safe ?:, CONST
- Macro Compiler: Runtime AST parsing and evaluation
- Debugger: TUI debugger with source display, breakpoints, stepping
- FRB: Native + Pcode dual mode runtime binary
- Tests: 13 packages ALL PASS

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:41:50 +09:00