CharlesKWON
e75167c2e9
Wires the new SqlEach RTL into FiveSql2's front-end so users write
the SQL they know and opt into streaming with a familiar Harbour
code block — no manual RTL plumbing.
API:
/* Existing array form — unchanged, 43-test still green */
aR := five_SQL( "SELECT name FROM t" )
/* New block form — zero intermediate rows, 2x raw PRG */
five_SQL( "SELECT id, name FROM t WHERE salary > 50000", NIL,
{|nID, cName| Process(nID, cName)} )
Parameter order (cSQL, aParams, bBlock) keeps backward compatibility
with every existing call site. Passing NIL for aParams when only a
block is needed is standard Harbour idiom.
Routing:
* TFiveSQL:Execute now takes an optional bBlock parameter and
stores it on TSqlExecutor as ::bRowBlock.
* TSqlExecutor:RunSelect's existing Go fast path (same guards as
before: single table, no JOIN/GROUP/aggregate, plain column
projections, WHERE compilable via SqlExprToPrg) branches on
::bRowBlock:
- block present → SqlEach streams rows through the block
- block absent → SqlScan materializes into aRows (current path)
* Post-processing (GROUP BY / ORDER BY / window / DISTINCT / LIMIT)
runs on empty aRows when block mode fires — all are no-ops on
empty input, so the sequence stays harmless.
* RunSelect returns NIL (not {fields, rows}) when ::bRowBlock was
used — signals "streaming semantics, all work done in the block".
Complex queries (JOIN, GROUP BY, subquery, window, ORDER BY not
matchable by an index, LIMIT/OFFSET, etc.) still fall back to the
array path even when a block is supplied — those genuinely require
materialization. Block mode is a fast-path opt-in, not a semantic
change.
End-to-end bench (50k rows, steady state — includes the user-side
loop/block for every row):
Path Time Speedup vs raw
──────────────────────────────────────────────────────────────
Raw PRG DO WHILE !Eof() + WHERE sum 7.6ms 1.00x
five_SQL array + FOR 7.7ms ~same
five_SQL + block (new) 3.7ms 2.05x ← beats raw
──────────────────────────────────────────────────────────────
Raw PRG no WHERE 6.1ms 1.00x
five_SQL + block, no WHERE 2.9ms 2.10x ← beats raw
SQL now pays for itself on end-to-end timing — not just competitive
with hand-rolled RDD loops, but faster than them. The layered cost
of FieldGet's Frame+RTL-dispatch that hand-written loops incur per
call is gone; the block-callback path captures *dbf.DBFArea directly
via FastFieldGetter and uses PcOpFieldGet to bypass dispatch in the
compiled WHERE predicate.
Validation:
- FiveSql2 43/43 (array API unchanged)
- Harbour compat 51/51
- go test ./... ALL PASS
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FiveSql2 — SQL Engine for Harbour DBF/NTX/CDX
Pratt parser + SQL:1992-2023 full standard support Supports both NTX (Clipper) and CDX (FoxPro/ADS) indexes
Architecture
five_SQL("SELECT ...")
│
├── TSqlLexer Tokenizer
├── TSqlParser2 Pratt parser (data-driven operators)
├── TSqlExecutor Query executor (Volcano model)
│ ├── TSqlAlias Central alias manager (no collisions)
│ ├── TSqlIndex NTX/CDX index optimization (auto-detect)
│ ├── TSqlAgg GROUP BY / aggregation
│ ├── TSqlSort ORDER BY / DISTINCT
│ ├── TSqlDDL CREATE/DROP/ALTER TABLE/INDEX
│ └── TSqlTxn BEGIN/COMMIT/ROLLBACK
├── TSqlExpr AST nodes + expression evaluation
└── TSqlFunc 60+ scalar functions
Build & Test
export PATH="/path/to/harbour-core/bin/linux/gcc:$PATH"
export HB_INSTALL_PREFIX="/path/to/harbour-core"
make # Build all tests
make test # Run all 157 tests
make bench # Parser benchmark
make clean # Clean
SQL Standard Coverage
| Standard | Features | Tests |
|---|---|---|
| SQL:1992 | SELECT, JOIN, GROUP BY, HAVING, Subquery, CASE, CAST | 43 |
| SQL:1999 | CTE, Recursive CTE, Window Functions, MERGE | 10 |
| SQL:2003 | SIMILAR TO, GROUPING SETS, LATERAL, Window frames | 64 |
| SQL:2008 | FETCH/OFFSET, FOR UPDATE, Extended MERGE | (incl.) |
| SQL:2016 | JSON functions, LISTAGG | (incl.) |
| SQL:2023 | ANY_VALUE, GREATEST/LEAST, BOOL_AND/OR | (incl.) |
| Challenge | LeetCode-level complex queries | 15 |
| Extreme | Production analytics stress tests | 15 |
Adding New Operators
Edit TSqlParser2.prg, method InitInfixTables():
::hInfixTT[ TK_MYOP ] := { "<=>", 40, 41, ND_BIN }
One line. No structural changes needed.
Copyright
Copyright (c) 2025-2026 Charles KWON (Charles KWON OhJun) Email: charleskwonohjun@gmail.com All rights reserved.