f4ed42556b
checkpoint: season-wide bug fix campaign + infra
...
Cumulative season's silent-bug hunting (~62 fixes) across the FiveSql2
SQL engine, the Five compiler/runtime, and the hbrdd RDD layer. Saved
as a single checkpoint before refactoring the parser to delegate xBase
command translation to the preprocessor.
Highlights:
FiveSql2 engine (_FiveSql2/src/)
- prefix-glob index attach -> explicit convention (<table>_pk.ntx,
<table>_uq.ntx, <table>.cdx) — fixes silent multi-row INSERT row-drop
- DROP/CREATE TABLE FErase chain extended (.cdx, .fsc, .fsv, .dbt, .fpt)
- COUNT(DISTINCT col) parsed + aggregated via hSeen hash
- UNION column-count mismatch returns SQL_ERR_GRAMMAR (was silent)
- DISTINCT + ORDER BY hidden-col leak fixed (trim before DISTINCT)
- Derived table FROM (SELECT...) + JOIN right-side derived
- Self-FK CASCADE depth 2+ via SqlGetSingleColPK pre-collect
- LAG/LEAD default arg uses SqlEvalRowExpr (handles -N const exprs)
- DATE literal round-trip validation (Feb 29 non-leap rejected)
- CREATE OR REPLACE VIEW; CREATE VIEW errors on already-exists
- AlterTable type dispatcher comma-wrapped (1-char type "A" no longer
matches CHARACTER)
Compiler / runtime
- gengo: HB_ -> FV_ prefix on emitted Go function names (Five identity)
- gengo split: emit_block.go, emit_stmt.go, folding.go extracted
- parser/stmtreg.go nudges
- hbrt: debug TUI/CLI restructure (debugcmd, debugkey, termios_*),
windows debug stubs collapsed
- thread/vm/value/class/pcinterp tightening from panic traces
RDD layer (hbrdd/)
- dbf: null bitmap support (null.go + null_test.go), mmap split
(mmap_posix.go / mmap_windows.go), byte-level numeric parse
- ntx/cdx: windows mmap parity
- workarea + mem RDD: cross-area state-bleed fixes
RTL (hbrtl/)
- errorlog rewrite with platform-specific FD (errorlog_fd_unix /
errorlog_fd_other)
- sqlscan, sqlhelpers, indexrtl, datetime extensions
Gates green at checkpoint:
- go test ./... : PASS
- FiveSql2 SQL:1999 : 43/43
- Harbour compat : 56/56
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-04-30 09:26:25 +09:00
c84cde6175
perf(fivesql2): Go-native SqlIsAggName — drop per-row substring scan
...
B4 GROUP+HAVING profile showed SqlIsAggName at ~9% of CPU —
SqlEvalFunc checks it for every function in every row, and the
PRG body was two string allocations + a substring scan:
RETURN ("," + c + ",") $ ("," + AGG_FUNCTIONS + ",")
Replace with a hash lookup against the existing aggFuncSet map
in hbrtl/sqlexpr.go (already populated for SqlExprHasAgg, same
AGG_FUNCTIONS list). Upper-casing skips the allocation when the
input is already upper, which it almost always is in practice.
Bench deltas (median of 3 steady runs, 1000 iters):
B4_GROUP_HAVING 447 → 418 us -6.5%
B14_COUNT 252 → 235 us -7%
B15_CTE_WIN_JOIN 1595 → 1577 us -1%
Other benches unchanged (no aggregate calls per row).
FiveSql2 43/43, Harbour compat 56/56.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-04-18 13:40:19 +09:00
dd270d5d9d
perf: RTL Go-native migration — 27 optimizations, DML up to 70-90x
...
Systematic pass through PRG hot paths, promoting them to Go RTL while
preserving Harbour/FiveSql2 semantics. Full log in
docs/RTL-Go-Native-Migration.md.
Bench (bench_sql) vs 2026-04-08 baseline
- B1 SELECT * 2,192 → 114 µs (19x)
- B6 INNER JOIN 9,291 → 233 µs (40x)
- B7 CTE simple 8,037 → 129 µs (62x)
- B9 ROW_NUMBER 3,705 → 265 µs (14x)
- B10 RANK PARTITION 4,748 → 309 µs (15x)
- B12 INSERT (WA cache) 4,319 → 63 µs (69x)
- B13 UPDATE (WA cache) 6,144 → 68 µs (90x)
- B15 CTE+WIN+JOIN 18,395 → 1,873 µs (10x)
Infrastructure
- HbHash O(1) Index preserving insertion order (Harbour KEEPORDER)
- HbDeepClone Go RTL (scalar-sharing, immutable hash keys)
- MEMRDD auto-imported via gengo; all Five programs get mem:name driver
- SQL plan + pcode caches (s_hPlanCache, s_hDmlPcodeCache)
- Opt-in SqlWACacheEnable — dbUseArea/Close/Commit batched for DML
SQL engine
- FiveSql2 lexer ported to Go (byte FSM) with combined automatic
template parameterization (literals → ?, concat queries share plan)
- Go RTL: SqlDistinct, SqlGroupRows, SqlWindowPartitions,
SqlWindowSortPartition, SqlWindowAssignRank, SqlComputeAggSimple,
SqlBulkInsert, SqlBulkUpdate, SqlExprHasAgg, SqlEvalHaving
- CTE / subquery / driving-table materialize paths use MEMRDD
- SqlCoerce/SqlCmp/SqlIsTrue helpers moved from PRG to Go
- SqlBulkUpdate defers Flush when WA cache active (APFS fsync was
dominant B13 cost — 1.6ms/call → gone)
Correctness fixes uncovered during migration
- ASort default path now sorts dates/logicals/timestamps (was no-op)
- ORDER BY default NULL placement matches PRG SqlRowCompare across
Go fast path; explicit NULLS FIRST/LAST honored by both paths
- SqlBulkUpdate respects EXCLUSIVE vs SHARED mode record locks
- SqlCmp/SqlCmpEq normalize NumInt vs Double (caught by test 6b)
Verification
- go test ./... ALL PASS
- FiveSql2 test_sql1999 43/43
- tests/compat_harbour 56/56 (+5 new: ASort dates/logicals,
AScan int cross-type)
- Regression test test_null_order.prg for ORDER BY NULL ordering
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-04-17 20:20:14 +09:00
