Create Five-1.0-Phase-C-TODO.md capturing the remaining 1.0 work:
three Harbour contrib libraries (hbct Clipper Tools, hbnf Numeric
Functions, hbtip TCP/IP/SMTP/POP3/HTTP). Each entry lists the
Harbour source path, a minimum first-pass scope, and an effort
estimate. Suggested order: hbct → hbtip → hbnf. Total ~6-10 days.
Update RTL-Go-Native-Migration.md "남은 병목" with the Phase A/B
completion list — six features shipped this session — plus a note
that the 11 HBTYPE functions the initial analysis flagged are
actually Harbour's internal scalar class factories, not user-facing
blockers (Five's SendBuiltin covers the same surface).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six new migration-log entries covering this session's 21 commits:
#27 VM in-place stack ops + symbol hoist (global 3-15%)
#28 gengo compile-time peepholes (9 commits, 1-7% bench)
#29 SELECT WA cache extension (single-table 2x+)
#30 JOIN temp-alias stabilisation (B6 1.67x)
#31 Stat-loop gates — view + CTE (CPU -40pp in rawsyscalln)
#32 Go-native SqlIsAggName + FetchRow (agg/window 1.3-1.7x)
Plus a cumulative bench table vs the 3caadb2 baseline and an
updated "남은 병목" section pointing at EvalExpr / JOINRECURSE /
HASHJOIN / Go runtime primitives as the remaining levers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The VM call path (PushSymbol → Function → Frame) is traversed by every
PRG function call. Three changes together cut per-call overhead across
the entire bench suite.
Changes
- hbrt/call.go Function(): replace pop-push dance with a single slice
shift (N+2 pops + N pushes → 1 copy of N slots + sp adjust). Kills
the per-call `make([]Value, nArgs)` heap alloc. Resolved function
pointer is cached back into sym.Func so subsequent calls on the
same Symbol skip the VM lookup entirely.
- hbrt/vm.go GetSym(): new helper. Generated code calls it with a
pointer to a package-level `*Symbol` slot so FindSymbol (which takes
the VM RWMutex + map lookup) runs at most once per symbol per
process. Nil results are intentionally NOT cached — an init-order
miss becomes a retry on the next call instead of a permanent sticky
failure.
- hbrt/thread.go pushPendingSym(): scalar fast slot for depth=1 call
nesting (common case). Nil syms still go through the slice so the
"empty vs stored nil" ambiguity can't produce a false pop.
- compiler/gengo/gengo.go: emit `t.PushSymbol(t.GetSym(&_sym_<file>_<NAME>, "NAME"))`
for every function call site, with a per-file prefix so multi-PRG
builds don't collide on identical symbol names.
Bugs fixed during bring-up
- pendingSymFast == nil was ambiguous ("unused" vs "nil stored"). Nil
syms now spill to the slice, preserving distinguishability.
- The old varName-reuse branch at the PushSymbol emit site skipped
the GetSym wrapper, emitting a raw `t.PushSymbol(varName)` against
an uninitialized package-level *Symbol. Every call path now funnels
through emitPushSymbol.
bench_sql deltas vs prior build
- B1 SELECT * 114 → 97 µs (15%)
- B4 GROUP_HAVING 584 → 554 µs (5%)
- B8 RECURSIVE CTE 150 → 141 µs (6%)
- B10 RANK PARTITION 310 → 296 µs (5%)
- B11 SUM OVER 335 → 320 µs (4%)
- B14 COUNT 295 → 281 µs (5%)
- B15 CTE+WIN+JOIN 1891 → 1826 µs (3%)
Verification
- go test ./... ALL PASS
- FiveSql2 test_sql1999 43/43
- tests/compat_harbour 56/56
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements hybrid execution model: keep AST tree-walk for SQL:2013+
features (Window, Recursive CTE, JOIN, aggregates) while compiling
simple SELECT hot paths to Go + pcode. See docs/FiveSql2-Hybrid-Plan.md
for the full architecture rationale (why not SQLite-style VDBE).
Hot path (single table, no joins/groups/aggregates):
- TryBuildFieldPositions: resolves SELECT column list to FieldPos
array once per query (bails to PRG loop on any complex expr).
- TryCompileWhere + SqlExprToPrg: walks WHERE AST, emits equivalent
PRG source, runs it through PcCompile to get a PcodeFunc.
- SqlScan RTL: Go-native scan loop — GoTop/EOF/Skip/GetValue
direct, ExecPcode per row for WHERE, result array pre-alloc.
WHERE compiler scope:
- ND_LIT numeric/logical/string (string literals AllTrim'd to match
SqlCmpEq CHAR-padding semantics; rejects embedded quotes/newlines)
- ND_COL: CHAR fields auto-wrapped with AllTrim(FieldGet(n)) based
on dbStruct() lookup cached once per query in aCompileStruct
- ND_BIN: = <> != < <= > >= AND OR + - * /
- ND_UNI: NOT -
- Anything else (ND_FN, ND_CASE, ND_SUB, ND_PAR, LIKE, IN, IS NULL,
BETWEEN, dates) returns NIL → falls back to PRG tree-walk.
Bench (50k rows, ~/tmp ext4):
Before After Speedup
Numeric WHERE ~150ms 11.7ms ~13x
String WHERE 119.3ms 10.5ms 11.4x
No WHERE - 14.6ms -
Raw RDD baseline 6.8ms 6.8ms 1.0x
Remaining gap to raw RDD (~1.5x) is structural: Value boxing, result
array construction, per-row ExecPcode frame overhead. Would need a
Value-pool or SoA refactor to close further.
Side fixes bundled:
- TSqlIndex:FindExclusive short-circuited. Originally called
dbInfo(DBI_FULLPATH)/DBI_SHARED which are unresolved symbols in
Five (dbInfo is a stub, DBI_* never defined). Panic'd with
"local variable index out of range: 0" whenever a standalone PRG
had a workarea Used before calling five_SQL. 43-test masked the
bug because it only reached FindExclusive with no open workareas.
Restore the scan once dbInfo lands in hbrtl.
- cmd/five/main.go: FIVE_KEEP_BUILD=1 env var keeps the temp Go
project around for debugging gengo output.
Validation:
- FiveSql2 43/43
- Harbour compat 51/51
- go test ./... ALL PASS
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the no-op Windows lock stub with actual kernel32 LockFileEx /
UnlockFileEx calls via syscall.LazyDLL (zero external dependency).
- LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY for non-blocking
semantics matching Clipper FLOCK() → .F.
- Same lock region layout as POSIX: header region for FLOCK, record
offsets for DBRLOCK — compatible across platforms
- Handles returned as syscall.Handle from os.File.Fd()
Note: full Windows cross-compile still blocked by unrelated issues
(mmap in cdx/ntx, termios in debugcli.go). The lock code itself
compiles cleanly with //go:build windows.
Also updates gap-analysis.md to reflect Windows lock status.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the FLOCK/DBRLOCK/DBRUNLOCK no-op stubs with actual
fcntl(F_SETLK) byte-range advisory locks, matching Harbour's
hb_fsLockLarge implementation.
Before: rtlDbRLock always returned .T. regardless of contention.
Multi-process writers could silently corrupt records.
After: Non-blocking POSIX byte-range locks per file descriptor.
Cross-process exclusion verified by a subprocess-spawning
Go test that witnesses BUSY vs OK transitions.
New files:
hbrdd/dbf/locks_posix.go fcntl F_WRLCK/F_UNLCK wrappers
hbrdd/dbf/locks_windows.go stub (TODO: LockFileEx)
hbrdd/dbf/lock_multi_test.go cross-process verification
docs/gap-analysis.md honest Harbour parity assessment
Modified:
hbrdd/dbf/dbf.go
- DBFArea gains fileLocked bool + lockedRecs map
- Close() calls releaseAllLocks() before dropping the fd
hbrtl/database.go
- rtlDbRLock / rtlDbRUnlock now delegate to DBFArea.LockRecord /
UnlockRecord instead of returning fixed .T./NIL
- New rtlFLock / rtlDbUnlock for FLOCK() / DBUNLOCK()
hbrtl/register.go
- FLOCK and DBUNLOCK symbols registered (were missing entirely)
compiler/analyzer/analyzer.go
- FLOCK / DBUNLOCK added to RTL known-function set
Lock region layout (non-overlapping on purpose):
FLOCK region [0, HeaderLen+1)
Record N region [RecordOffset(N), RecordLen)
So a workarea can hold FLOCK and multiple DBRLOCK simultaneously
on the same fd without conflict.
Design rationale (captured in locks_posix.go header):
* POSIX fcntl, not flock(2) — byte-range + NFS-safe
* Non-blocking F_SETLK — matches Clipper FLOCK() → .F. semantics
* Released explicitly on Close to avoid workarea-sharing races
* Windows falls back to no-op (TODO: LockFileEx)
Verification:
go test ./hbrdd/dbf/ -run TestFLockBlocksAcrossProcesses PASS
go test ./hbrdd/dbf/ -run TestRLockBlocksAcrossProcesses PASS
go test ./... ALL PASS
FiveSql2 43/43 100%
compat_harbour 51/51 100%
The gap-analysis doc (docs/gap-analysis.md) is a running inventory
of what works vs what's still missing vs Harbour 3.2, written for
users evaluating Five for production — not a sales pitch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive review as if evaluated by Google Go team:
- Architecture analysis (transpiler pipeline, gengo innovations)
- Performance evidence (6/10 categories faster than C)
- Correctness proof (82/82 + 77/77 + 18/18 + 47/47)
- Strategic value (5M xBase developer bridge to Go)
- Improvement roadmap (lazy GoTo, string fusion, CDX create)
- Market positioning (vs Harbour, xHarbour, Alaska xBase++)
Key quote: "Five demonstrates that Go is ready to be a universal
compilation target, not just a language for writing programs directly."
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete Harbour-compatible MEMVAR implementation:
- PUBLIC: global scope, persist until program end
- PRIVATE: function scope + called functions, auto-release on return
- Shadowing: PRIVATE can shadow PUBLIC, restored on scope exit
- Nested: multi-level PRIVATE scoping with save/restore stack
- Thread.PushMemvar/PopMemvar: stack-based memvar access
- Thread.DeclarePublic/DeclarePrivate: declaration helpers
- MacroEval: &cVar now looks up memvars (was returning string)
- Shutdown: Phase 4 clears all memvars on all threads
- Case-insensitive: all lookups uppercased
Tests: 12 tests including:
PUBLIC create/update, case-insensitive, PRIVATE basic,
shadow/restore, nested 3-level shadow, new var cleanup,
release, releaseAll, names, thread integration, macro access
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cmd/five/main.go:
#16: Merge goPath() → alias for findGoBin() (removed 10-line duplicate)
#17: Merge findProjectRoot() → alias for findFiveRoot()
New walkUpForGoMod() helper shared by both strategies
#33-34: Fix path injection in debugPRG
Was: string concat with unescaped path
Now: fmt.Sprintf(%q) for safe Go string escaping
#43: findProjectRoot aliased to findFiveRoot (removes 3rd copy)
Issues resolved: #16,17 (HIGH), #33,34,43 (MEDIUM)
Total fixed: 34/53
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Expanded from 180 lines to 450+ lines per language:
- Real failure stories: EU bank COBOL→Java (€200M), Brazil Clipper→Python (tax error)
- Deep Go analysis: 25 keywords vs 90+, no exceptions by design, hardware future
- AI paradox: code generation vs code understanding gap
- Detailed code comparisons: Go vs Five for discount calculation
- Five principles with battle scars from 30 years of xBase deployment
- Independence manifesto: zero-dependency code ownership
- Epilogue: what will future archaeologists find?
"The measure of a language is not what it can express,
but what it allows a stranger to understand."
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The founding document of Five, written as a manifesto:
- Why 30 years of xBase code must not be discarded
- Why Go is the right foundation for the next 50 years
- Why human-readable code matters more in the AI era
- What was designed: 5 principles, 5 architectures
- The vision: living code that bridges past and future
"Code fades, but thought endures."
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- five-syntax-en/ko: Add Math comparison table (Harbour RTL vs Go math)
- go_math_compare.prg: Detailed English comments explaining each section
- Example lists updated with go_math_compare.prg
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Five's DEFER is Go's defer in PRG syntax.
Same safety guarantee, but without if err != nil pollution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Five's hidden strength: PRG code is readable by non-developers.
When AI generates code, humans must verify it.
Five's xBase syntax makes this possible.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
