CharlesKWON
8aaed994f4
Implements hybrid execution model: keep AST tree-walk for SQL:2013+
features (Window, Recursive CTE, JOIN, aggregates) while compiling
simple SELECT hot paths to Go + pcode. See docs/FiveSql2-Hybrid-Plan.md
for the full architecture rationale (why not SQLite-style VDBE).
Hot path (single table, no joins/groups/aggregates):
- TryBuildFieldPositions: resolves SELECT column list to FieldPos
array once per query (bails to PRG loop on any complex expr).
- TryCompileWhere + SqlExprToPrg: walks WHERE AST, emits equivalent
PRG source, runs it through PcCompile to get a PcodeFunc.
- SqlScan RTL: Go-native scan loop — GoTop/EOF/Skip/GetValue
direct, ExecPcode per row for WHERE, result array pre-alloc.
WHERE compiler scope:
- ND_LIT numeric/logical/string (string literals AllTrim'd to match
SqlCmpEq CHAR-padding semantics; rejects embedded quotes/newlines)
- ND_COL: CHAR fields auto-wrapped with AllTrim(FieldGet(n)) based
on dbStruct() lookup cached once per query in aCompileStruct
- ND_BIN: = <> != < <= > >= AND OR + - * /
- ND_UNI: NOT -
- Anything else (ND_FN, ND_CASE, ND_SUB, ND_PAR, LIKE, IN, IS NULL,
BETWEEN, dates) returns NIL → falls back to PRG tree-walk.
Bench (50k rows, ~/tmp ext4):
Before After Speedup
Numeric WHERE ~150ms 11.7ms ~13x
String WHERE 119.3ms 10.5ms 11.4x
No WHERE - 14.6ms -
Raw RDD baseline 6.8ms 6.8ms 1.0x
Remaining gap to raw RDD (~1.5x) is structural: Value boxing, result
array construction, per-row ExecPcode frame overhead. Would need a
Value-pool or SoA refactor to close further.
Side fixes bundled:
- TSqlIndex:FindExclusive short-circuited. Originally called
dbInfo(DBI_FULLPATH)/DBI_SHARED which are unresolved symbols in
Five (dbInfo is a stub, DBI_* never defined). Panic'd with
"local variable index out of range: 0" whenever a standalone PRG
had a workarea Used before calling five_SQL. 43-test masked the
bug because it only reached FindExclusive with no open workareas.
Restore the scan once dbInfo lands in hbrtl.
- cmd/five/main.go: FIVE_KEEP_BUILD=1 env var keeps the temp Go
project around for debugging gengo output.
Validation:
- FiveSql2 43/43
- Harbour compat 51/51
- go test ./... ALL PASS
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
123 lines
3.3 KiB
Go
123 lines
3.3 KiB
Go
// Copyright (c) 2026 Charles KWON OhJun (charleskwonohjun@gmail.com)
|
||
// All rights reserved.
|
||
|
||
// Go-native SQL scan loop for FiveSql2 hot path.
|
||
//
|
||
// Motivation: FiveSql2 is a PRG-based SQL interpreter. For simple
|
||
// "SELECT cols FROM table WHERE cond" queries, the per-row cost is
|
||
// dominated by PRG interpreter overhead (AST tree walk, field name
|
||
// lookup, workarea switching). Moving just the inner scan loop to Go
|
||
// bypasses all that overhead and gets us ~15x speedup for the common
|
||
// case while keeping the rest of FiveSql2 untouched.
|
||
//
|
||
// The SQL engine remains responsible for:
|
||
// - Parsing SQL and building AST
|
||
// - Resolving field names to positions (column binding)
|
||
// - Compiling WHERE expression to pcode (via PcCompile)
|
||
// - GROUP BY, ORDER BY, aggregates (not per-row)
|
||
//
|
||
// This helper only handles the hot loop:
|
||
// - Full table scan (workarea already positioned)
|
||
// - Per-row WHERE evaluation via ExecPcode
|
||
// - Column extraction via cached field positions
|
||
// - Result array construction
|
||
|
||
package hbrtl
|
||
|
||
import (
|
||
"five/hbrdd"
|
||
"five/hbrt"
|
||
)
|
||
|
||
// SqlScan(aFieldPositions, pcWhere) → aRows
|
||
//
|
||
// Scans the current workarea top-to-bottom, evaluates pcWhere per row
|
||
// (nil = no filter), collects selected column values into rows.
|
||
//
|
||
// aFieldPositions: array of 1-based field positions to extract per row.
|
||
// Resolve once before calling (FieldPos cache is O(1)
|
||
// but still has PRG → Go call overhead).
|
||
// pcWhere: pcode function pointer from PcCompile, or NIL.
|
||
//
|
||
// Returns:
|
||
// Array of rows, each row = Array of field values.
|
||
//
|
||
// Notes on CHAR trimming: DBF character fields are space-padded. The
|
||
// caller decides whether to trim (via a SELECT-list AllTrim wrapper).
|
||
// We don't trim here — that's a semantic choice, and callers who need
|
||
// raw bytes shouldn't pay for a strings.TrimSpace().
|
||
func SqlScan(t *hbrt.Thread) {
|
||
t.Frame(2, 0)
|
||
defer t.EndProc()
|
||
|
||
// Parse arguments
|
||
fieldsVal := t.Local(1)
|
||
if !fieldsVal.IsArray() {
|
||
t.PushValue(hbrt.MakeArray(0))
|
||
t.RetValue()
|
||
return
|
||
}
|
||
fieldsArr := fieldsVal.AsArray().Items
|
||
nFields := len(fieldsArr)
|
||
|
||
whereVal := t.Local(2)
|
||
var whereFn *hbrt.PcodeFunc
|
||
if !whereVal.IsNil() {
|
||
if p := whereVal.AsPointer(); p != nil {
|
||
whereFn, _ = p.(*hbrt.PcodeFunc)
|
||
}
|
||
}
|
||
|
||
// Pre-convert field positions to []int (avoid Value->int per row)
|
||
fieldPos := make([]int, nFields)
|
||
for i := 0; i < nFields; i++ {
|
||
fieldPos[i] = int(fieldsArr[i].AsNumInt())
|
||
if fieldPos[i] < 1 {
|
||
fieldPos[i] = 1
|
||
}
|
||
}
|
||
|
||
wam, ok := t.WA.(*hbrdd.WorkAreaManager)
|
||
if !ok {
|
||
t.PushValue(hbrt.MakeArray(0))
|
||
t.RetValue()
|
||
return
|
||
}
|
||
area := wam.Current()
|
||
if area == nil {
|
||
t.PushValue(hbrt.MakeArray(0))
|
||
t.RetValue()
|
||
return
|
||
}
|
||
|
||
// Pre-allocate result: 50k × small-row header pressure matters
|
||
rows := make([]hbrt.Value, 0, 1024)
|
||
|
||
// Scan
|
||
area.GoTop()
|
||
for !area.EOF() {
|
||
// WHERE evaluation (if any)
|
||
keep := true
|
||
if whereFn != nil {
|
||
hbrt.ExecPcode(t, whereFn, nil)
|
||
keep = t.GetRetValue().AsBool()
|
||
}
|
||
|
||
if keep {
|
||
// Collect column values
|
||
row := make([]hbrt.Value, nFields)
|
||
for i := 0; i < nFields; i++ {
|
||
// GetValue is 0-based
|
||
v, _ := area.GetValue(fieldPos[i] - 1)
|
||
row[i] = v
|
||
}
|
||
rows = append(rows, hbrt.MakeArrayFrom(row))
|
||
}
|
||
|
||
area.Skip(1)
|
||
}
|
||
|
||
t.PushValue(hbrt.MakeArrayFrom(rows))
|
||
t.RetValue()
|
||
}
|