fivedev/five
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d7a81af7db1cc2c88794957e8831a81f0a6ce596
five/_FiveSql2/src/TSqlExecutor.prg
CharlesKWON 5e4a1c5d72 refactor(FiveSql2): cross-session globals → Go atomic + RWMutex 
Completes the per-STATIC migration started in 5bba0c2. The
remaining three TSqlExecutor module STATICs (s_nSchemaVer,
s_nRCJSeq, s_hAutoInc) genuinely needed cross-connection
visibility — a CREATE TABLE on connection A MUST invalidate B's
plan cache, an RCJ alias MUST be unique across all live queries,
and an IDENTITY column MUST hand out monotonic values across all
writers. Moving them to TSqlSession (per-instance) would have
broken those semantics.

Solution: back them with Go-side primitives exposed via HB_FUNCs:

  s_nSchemaVer  →  atomic.Uint64 (SqlSchemaVer / SqlBumpSchemaVer)
  s_nRCJSeq     →  atomic.Uint64 (SqlNextRCJSeq, returns mod-100000)
  s_hAutoInc    →  sync.RWMutex + map[string][]string
                    (SqlSetAutoInc / SqlGetAutoIncFields)

Lives in `hbrtl/sqlglobals.go`. The PRG-side `FUNCTION
SqlSchemaVer() / SqlBumpSchemaVer() / SqlSetAutoInc() /
SqlGetAutoIncFields()` definitions in TSqlExecutor.prg are
deleted; the HB_FUNC dispatch takes their place. The single PRG
caller of `s_nRCJSeq` (in the RCJ helper around line 5600)
becomes `SqlNextRCJSeq()` and reads cleaner — the old
`s_nRCJSeq := (s_nRCJSeq + 1) % 100000` was both racy and a
non-atomic two-write update under multi-conn load.

The other module STATIC, `s_hAutoInc`, used to lazy-init on
first use (`IF s_hAutoInc == NIL ... := { => }`); two concurrent
first-CREATE TABLE calls hit "concurrent map writes" on that
branch. The Go RWMutex eliminates the race; reads still scale
(RLock) so the IDENTITY-lookup at INSERT time isn't a contention
hot-spot.

All six release gates green:
  go test ./...               ✓
  FiveSql2 SQL:1999 43/43     ✓
  Harbour compat 56/56        ✓
  std.ch 17/17                ✓
  FRB 7/7                     ✓
  pgserver integration 6/6    ✓

Concurrency stress (3-worker × 20):
  pre-Layer-1:    ~60% pass + occasional Go panic
  +Layer 1+2:     80% pass, no panics
  +3a:            80% pass
  +per-session 3 STATIC move:  90% pass
  +this commit:   ~75% pass (variability — Go map atomic + mutex
                  serialise the writers but the underlying
                  hbrdd multi-area mmap path still has its own
                  race, deferred to follow-up)

The next bottleneck is at the hbrdd workarea layer (multi-Area
instances per file each holding their own mmap snapshot), not at
the FiveSql2 STATIC level. That fix is its own commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 19:58:52 +09:00

6208 lines
222 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/*
* TSqlExecutor.prg — Main query executor with index optimization
*
* FiveSql — SQL Engine for Harbour DBF/NTX
*
* Copyright (c) 2025 Charles KWON (Charles KWON OhJun)
* Email: charleskwonohjun@gmail.com
*
* All rights reserved.
*/
#include "hbclass.ch"
#include "dbstruct.ch"
#include "dbinfo.ch"
#include "error.ch"
#include "FiveSqlDef.ch"
/* aOuterStack / lCteDiskSeen / hDmlPcodeCache live on TSqlSession
* now (per-instance) so concurrent pgserver connections don't share
* one set of Go package vars — that produced "concurrent map writes"
* panics on hDmlPcodeCache + stack-corruption on aOuterStack as
* soon as more than a couple of clients did DML at once. The
* STATICs below are cross-session counters (schema version, RCJ
* sequence) and the IDENTITY-tracking hash, which need cross-
* connection visibility — those remain global with mutex/atomic
* guards (TSqlAutoInc.go / SqlBumpSchemaVer).
*/
/* s_hAutoInc / s_nRCJSeq / s_nSchemaVer used to live here as
* module STATICs. They're now backed by Go-side atomics + an
* RWMutex-protected map in hbrtl/sqlglobals.go, accessed via
* HB_FUNCs (SqlSchemaVer / SqlBumpSchemaVer / SqlNextRCJSeq /
* SqlSetAutoInc / SqlGetAutoIncFields). Cross-connection-visible
* counters need atomic semantics across goroutines; PRG STATIC
* compiles to a Go package var with no synchronization, which
* raced under multi-pgserver-connection load.
*/
/* Per-plan DML pcode cache. Keyed by the plan-cache key that TFiveSQL
* uses (template key or cSQL text); value is a hash:
* { "set_fpos" => aFPos,
* "set_pc" => aValuePc, — parallel to set_fpos
* "where_pc" => pcWhere | NIL,
* "compiled" => .T. }
* RunUpdate populates on first hit, subsequent calls reuse. Compiled
* pcode depends on the target table's field layout; the plan-cache
* key carries a schema-version prefix (SqlSchemaVer) so DDL
* (ALTER / DROP / CREATE TABLE|INDEX|VIEW) invalidates this cache in
* one bump without iterating the hash. The DML cache piggybacks on
* the same cap as the plan cache (SQL_PLAN_CACHE_MAX from TFiveSQL).
* On overflow (size >= cap) the whole hash is wiped — coarser than
* LRU but we don't have insertion-order tracking on Five hashes. */
#define SQL_DML_PCODE_CACHE_MAX 1000
/* Compatibility shim — TFiveSQL.prg's plan-cache eviction path
* calls SqlDmlPcodeCacheReset() through the FUNCTION namespace
* because it doesn't have an ::oSession in scope (it has its own
* ::oSession on TFiveSQL). For embedded callers that fall through
* here, reset the default session's cache so the helper preserves
* its long-standing single-process semantics. pgserver callers
* invoke ::oSession:DmlPcodeCacheReset() directly. */
FUNCTION SqlDmlPcodeCacheReset( oSession )
IF oSession == NIL
oSession := SqlDefaultSession()
ENDIF
oSession:DmlPcodeCacheReset()
RETURN NIL
/* Schema version — bumped by every DDL completion. Used as a prefix
* on all SQL plan-cache / DML-pcode-cache keys so any DDL invalidates
* every plan that referenced the pre-DDL schema, without walking the
* hash. Old entries become unreachable (never looked up again) and
* are collected at process exit; DDL is rare enough in the target
* workload that the bounded leak is acceptable.
*
* STATIC is file-scoped, so cross-file access goes through the
* SqlSchemaVer / SqlBumpSchemaVer top-level functions below. */
/* s_nSchemaVer — see Go-side hbrtl/sqlglobals.go */
CLASS TSqlExecutor
DATA hQuery
DATA aParams
DATA oIndex AS OBJECT
DATA oAgg AS OBJECT
DATA oSort AS OBJECT
DATA oDDL AS OBJECT
DATA oTxn AS OBJECT
DATA oAlias AS OBJECT
DATA nDepth INIT 0
DATA aOpened INIT {}
DATA aTables INIT {}
DATA aCompileStruct
DATA bRowBlock /* optional code block — receives SELECT cols as params */
DATA aFetchCache /* pre-bound {nWA, nFPos} per SELECT expression, or NIL */
DATA cCacheKey /* plan-cache key set by TFiveSQL; used for DML pcode cache */
DATA aWrappedWindowCols INIT {} /* SELECT-col indices whose expr wraps ND_WINDOW */
DATA hSubCorrCache INIT { => } /* per-outer-key subquery result cache */
DATA aSubCacheSlots INIT {} /* list of {xSubNode, {id, aFreeVars}} */
DATA nSubCacheSeq INIT 0 /* monotonic ID for subqueries */
DATA aSemiJoinSlots INIT {} /* list of {xSubNode, semiJoinData | "NO"} */
DATA hRightMatched /* RecNo sets for RIGHT JOIN pass */
DATA hSubCache
/* Session container — carries txn log, plan cache, current user,
* role grants. Nested subquery executors inherit the parent's
* session so a child query's BEGIN/COMMIT operates on the same
* connection-scoped state. */
DATA oSession
METHOD New( hQuery, aParams, oSession ) CONSTRUCTOR
METHOD Run()
METHOD RunImpl()
METHOD RunSelect()
METHOD RunInsert()
METHOD RunUpdate()
METHOD RunDelete()
METHOD OpenTable( cTable, cAlias )
METHOD CloseOpened()
METHOD FetchRow( aExprs )
METHOD EvalExpr( xNode )
METHOD Resolve( cRef )
METHOD FindWA( cAlias )
METHOD JoinRecurse( aJoins, nIdx, xWhere, aRE, aRows, hHashTbl, aPushByLevel )
METHOD SplitAndClauses( xE, aOut )
METHOD BuildAliasLevelMap( aJoins )
METHOD ClauseMaxLevel( xClause, hAliasToLevel, nDefault )
METHOD EvalPushedAtLevel( aPushByLevel, nIdx )
METHOD RightJoinPass( aJoins, nIdx, aRE, aRows )
METHOD FetchRowNull( aRE, cInnerAlias )
METHOD ColBelongsTo( cColRef, cAlias )
METHOD PushOuter()
METHOD PopOuter()
METHOD ResolveFromOuter( cRef, cTblAlias, cField, lFound )
METHOD MakeError( nCode, cMsg )
METHOD HashJoin( nInnerWA, cInnerField, cOuterCol, xOnCond, aJoins, nIdx, xWhere, aRE, aRows, hHashTbl, aPushByLevel )
METHOD CacheSubquery( xSubExpr )
METHOD SnapshotAreaRecNos()
METHOD RestoreAreaRecNos( aSnap )
METHOD MaterializeCTE( aCTE )
METHOD MaterializeRecursiveCTE( aCTE )
METHOD ApplyWindowFunctions( aRows, aFN, aCols )
METHOD RunMerge()
METHOD RunTruncate()
METHOD TryGoJoin( aJoins, aResultExprs, nOuterWA )
METHOD TryBuildSortSpec( aOrderBy, aFieldNames )
METHOD TryBuildFieldPositions( aExprs )
METHOD TryCompileWhere( xWhere )
METHOD SqlExprToPrg( xNode )
METHOD BuildFetchCache( aExprs )
METHOD PreResolveColumns( xNode )
METHOD PreResolveCol( xNode )
METHOD SubqueryCached( xSubNode )
METHOD CollectFreeVars( hQ )
METHOD CollectExprFreeVars( xE, aLocalAliases, aFree )
METHOD ExistsViaSemiJoin( xSubNode, lNegate )
METHOD TryBuildSemiJoin( xSubNode )
ENDCLASS
METHOD New( hQuery, aParams, oSession ) CLASS TSqlExecutor
::hQuery := hQuery
::aParams := iif( aParams == NIL, {}, aParams )
/* Inherit caller's session if provided; otherwise fall back to
* the process-default session so embedded `five_SQL(cSQL)` callers
* and any TSqlExecutor created without the new arg keep working. */
IF oSession == NIL
::oSession := SqlDefaultSession()
ELSE
::oSession := oSession
ENDIF
::oIndex := TSqlIndex():New()
::oAgg := TSqlAgg():New()
::oSort := TSqlSort():New()
::oDDL := TSqlDDL():New()
::oTxn := TSqlTxn():New( ::oSession )
::oAlias := TSqlAlias():New()
::nDepth := 0
::aOpened := {}
::aTables := {}
/* Explicit fresh initialization — DATA INIT on hash/array literals
* can end up sharing the same instance across New() calls depending
* on the compile path, which would let one query's subquery cache
* leak into the next query's results. */
::hSubCache := { => }
::hSubCorrCache := { => }
::aSubCacheSlots := {}
::aSemiJoinSlots := {}
::nSubCacheSeq := 0
::hRightMatched := { => }
RETURN SELF
METHOD MakeError( nCode, cMsg ) CLASS TSqlExecutor
RETURN { { "__error__" }, { { nCode, cMsg, "" } } }
METHOD Run() CLASS TSqlExecutor
LOCAL aResult, lOldDel
IF ::hQuery == NIL
RETURN ::MakeError( SQL_ERR_SYNTAX, "Empty or invalid SQL" )
ENDIF
/* Save caller's SET DELETED state, force ON for the duration of
* this statement, restore on exit. SQL semantics treat marked-
* deleted rows as absent; forcing it here is required regardless
* of caller state. Restoring prevents a five_SQL() call from
* silently flipping the caller's setting.
* Set index 8 = Harbour SetDeleted slot (HB_SET_DELETED). Literal
* used because Five doesn't ship a set.ch with _SET_DELETED macro. */
lOldDel := Set( 8, .T. )
aResult := ::RunImpl()
Set( 8, lOldDel )
RETURN aResult
METHOD RunImpl() CLASS TSqlExecutor
LOCAL cType, aT, nP2
cType := ::hQuery[ "type" ]
DO CASE
CASE cType == "SELECT"
RETURN ::RunSelect()
CASE cType == "INSERT"
RETURN ::RunInsert()
CASE cType == "UPDATE"
RETURN ::RunUpdate()
CASE cType == "DELETE"
RETURN ::RunDelete()
CASE cType == "CREATE"
aT := ::hQuery[ "tokens" ]
nP2 := ::hQuery[ "pos" ]
IF ::oDDL:DDL_IsKW( aT, nP2, "TABLE" )
RETURN ::oDDL:CreateTable( aT, nP2 )
ELSEIF ::oDDL:DDL_IsKW( aT, nP2, "UNIQUE" ) .OR. ::oDDL:DDL_IsKW( aT, nP2, "INDEX" )
RETURN ::oDDL:CreateIndex( aT, nP2 )
ELSEIF ::oDDL:DDL_IsKW( aT, nP2, "VIEW" )
RETURN ::oDDL:CreateView( aT, nP2 )
ELSEIF ::oDDL:DDL_IsKW( aT, nP2, "OR" ) .AND. ;
::oDDL:DDL_IsKW( aT, nP2 + 1, "REPLACE" ) .AND. ;
::oDDL:DDL_IsKW( aT, nP2 + 2, "VIEW" )
/* `CREATE OR REPLACE VIEW v AS ...` — the OR/REPLACE prefix
* stays in the token stream so CreateView's own consumer
* picks them up. */
RETURN ::oDDL:CreateView( aT, nP2 )
ENDIF
RETURN ::MakeError( SQL_ERR_UNSUPPORTED, "CREATE: unsupported object" )
CASE cType == "DROP"
aT := ::hQuery[ "tokens" ]
nP2 := ::hQuery[ "pos" ]
IF ::oDDL:DDL_IsKW( aT, nP2, "TABLE" )
RETURN ::oDDL:DropTable( aT, nP2 )
ELSEIF ::oDDL:DDL_IsKW( aT, nP2, "INDEX" )
RETURN ::oDDL:DropIndex( aT, nP2 )
ELSEIF ::oDDL:DDL_IsKW( aT, nP2, "VIEW" )
RETURN ::oDDL:DropView( aT, nP2 )
ENDIF
RETURN ::MakeError( SQL_ERR_UNSUPPORTED, "DROP: unsupported object" )
CASE cType == "SET_COLLATION"
SqlSetCollation( ::hQuery[ "value" ] )
RETURN { { "result" }, { { "Collation set to " + ::hQuery[ "value" ] } } }
CASE cType == "ALTER"
aT := ::hQuery[ "tokens" ]
nP2 := ::hQuery[ "pos" ]
RETURN ::oDDL:AlterTable( aT, nP2 )
CASE cType == "BEGIN"
RETURN ::oTxn:Begin()
CASE cType == "COMMIT"
RETURN ::oTxn:Commit()
CASE cType == "ROLLBACK"
RETURN ::oTxn:Rollback()
CASE cType == "ROLLBACK_TO"
RETURN ::oTxn:RollbackTo( ::hQuery[ "savepoint" ] )
CASE cType == "SAVEPOINT"
RETURN ::oTxn:SetSavepoint( ::hQuery[ "name" ] )
CASE cType == "TRUNCATE"
RETURN ::RunTruncate()
CASE cType == "MERGE"
RETURN ::RunMerge()
ENDCASE
RETURN ::MakeError( SQL_ERR_UNSUPPORTED, "Unknown statement type: " + cType )
METHOD OpenTable( cTable, cAlias ) CLASS TSqlExecutor
LOCAL nWA, i, lFound
nWA := ::oIndex:OpenTable( cTable, cAlias, .T., .T. )
IF nWA > 0
/* When the WA cache is on, hand lifetime to the cache so CloseOpened
* leaves the mmap alive for the next query. Profile showed
* rtlDbCloseArea + munmap at ~30% of SELECT CPU prior to this branch.
*
* Skip caching when the alias is an AcquireTemp-generated "FA_####"
* token: those change every query (self-joins, nested depth), so
* caching them just leaks entries while delivering zero reuse. */
IF SqlWACacheIsEnabled() .AND. ! ( Left( cAlias, 3 ) == "FA_" )
SqlWACachePut( cAlias, nWA )
ELSE
AAdd( ::aOpened, cAlias )
ENDIF
/* Register with alias manager if not already tracked */
lFound := .F.
FOR i := 1 TO Len( ::oAlias:aSlots )
IF ::oAlias:aSlots[ i ][ 1 ] == cAlias
::oAlias:aSlots[ i ][ 4 ] := .T.
lFound := .T.
EXIT
ENDIF
NEXT
IF ! lFound
AAdd( ::oAlias:aSlots, { cAlias, Upper( cTable ), Upper( cAlias ), .T. } )
ENDIF
ENDIF
RETURN nWA
METHOD CloseOpened() CLASS TSqlExecutor
LOCAL i, nWA
FOR i := 1 TO Len( ::aOpened )
nWA := Select( ::aOpened[ i ] )
IF nWA > 0
dbSelectArea( nWA )
dbCloseArea()
ENDIF
NEXT
::aOpened := {}
::oAlias:aSlots := {}
RETURN NIL
METHOD PushOuter() CLASS TSqlExecutor
AAdd( ::oSession:aOuterStack, ::aTables )
RETURN NIL
METHOD PopOuter() CLASS TSqlExecutor
IF Len( ::oSession:aOuterStack ) > 0
ASize( ::oSession:aOuterStack, Len( ::oSession:aOuterStack ) - 1 )
ENDIF
RETURN NIL
METHOD FindWA( cAlias ) CLASS TSqlExecutor
LOCAL i, nWA, cA, cOrig, cReal
nWA := Select( cAlias )
IF nWA > 0 .AND. ( nWA )->( Used() )
RETURN nWA
ENDIF
FOR i := 1 TO Len( ::aTables )
cA := ::aTables[ i ][ 2 ]
IF Empty( cA )
cA := ::aTables[ i ][ 1 ]
ENDIF
cOrig := ""
IF Len( ::aTables[ i ] ) >= 3
cOrig := Upper( ::aTables[ i ][ 3 ] )
ENDIF
IF Upper( cA ) == cAlias .OR. cOrig == cAlias .OR. Upper( ::aTables[ i ][ 1 ] ) == cAlias
nWA := Select( cA )
IF nWA > 0 .AND. ( nWA )->( Used() )
RETURN nWA
ENDIF
ENDIF
NEXT
/* Fallback: check the alias manager for user alias mapping */
cReal := ::oAlias:RealAlias( cAlias )
IF ! Empty( cReal )
nWA := Select( cReal )
IF nWA > 0 .AND. ( nWA )->( Used() )
RETURN nWA
ENDIF
ENDIF
RETURN 0
/* PreResolveColumns walks an expression tree and caches {nWA, nFPos}
* on every ND_COL it can statically resolve. Called once per RunSelect
* entry (after tables are open and ::aTables is populated) so the
* per-row EvalExpr hot path can short-circuit Resolve()'s string
* parsing + alias lookup on every ND_COL visit.
*
* ND_SUB subtrees are intentionally skipped — they open their own
* scope at execution time (correlated/uncorrelated subqueries), so
* caching here with the outer query's workarea map would be wrong.
*
* Idempotent: re-invocation overwrites xNode[5] with the current
* resolution, so table reopens between runs (which may shift nWA)
* don't produce stale caches. */
METHOD PreResolveColumns( xNode ) CLASS TSqlExecutor
LOCAL i, j, xChild, nHi
IF xNode == NIL .OR. ValType( xNode ) != "A" .OR. Len( xNode ) < 1
RETURN Self
ENDIF
IF ValType( xNode[ 1 ] ) != "N"
RETURN Self
ENDIF
IF xNode[ 1 ] == ND_COL
::PreResolveCol( xNode )
RETURN Self
ENDIF
IF xNode[ 1 ] == ND_SUB
/* Subquery: leave its columns to resolve at its own scope. */
RETURN Self
ENDIF
/* Recurse into child slots 3..5. Each slot may be either:
* - a single node (array whose [1] is a numeric kind)
* - an array of nodes (CASE arms, FN args, LIST values) */
nHi := Len( xNode )
IF nHi > 5
nHi := 5
ENDIF
FOR i := 3 TO nHi
xChild := xNode[ i ]
IF ValType( xChild ) != "A" .OR. Len( xChild ) == 0
LOOP
ENDIF
IF ValType( xChild[ 1 ] ) == "N"
::PreResolveColumns( xChild )
ELSE
FOR j := 1 TO Len( xChild )
IF ValType( xChild[ j ] ) == "A"
::PreResolveColumns( xChild[ j ] )
ENDIF
NEXT
ENDIF
NEXT
RETURN Self
/* PreResolveCol — attempt to cache {nWA, nFPos} on a single ND_COL
* node. Leaves xNode[5] at NIL when resolution can't be determined
* statically (qualified to an unknown alias, no matching field,
* FIELD-list reference awaiting outer context) — EvalExpr then falls
* back to the full Resolve() path at runtime. */
METHOD PreResolveCol( xNode ) CLASS TSqlExecutor
LOCAL cRef, cField, cTblAlias, nDot, nWA, nFPos, i, cA, nSavedArea
xNode[ 5 ] := NIL /* clear prior cache */
cRef := xNode[ 2 ]
IF ValType( cRef ) != "C" .OR. cRef == "*"
RETURN Self
ENDIF
nDot := At( ".", cRef )
IF nDot > 0
cTblAlias := Upper( Left( cRef, nDot - 1 ) )
cField := Upper( SubStr( cRef, nDot + 1 ) )
ELSE
cField := Upper( cRef )
cTblAlias := ""
ENDIF
nSavedArea := Select()
IF ! Empty( cTblAlias )
nWA := ::FindWA( cTblAlias )
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
dbSelectArea( nSavedArea )
IF nFPos > 0
xNode[ 5 ] := { nWA, nFPos }
ENDIF
ENDIF
RETURN Self
ENDIF
/* Unqualified: first table where field exists wins (matches
* Resolve's iteration order). */
FOR i := 1 TO Len( ::aTables )
cA := ::aTables[ i ][ 2 ]
IF Empty( cA )
cA := ::aTables[ i ][ 1 ]
ENDIF
nWA := Select( cA )
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
IF nFPos > 0
dbSelectArea( nSavedArea )
xNode[ 5 ] := { nWA, nFPos }
RETURN Self
ENDIF
ENDIF
NEXT
dbSelectArea( nSavedArea )
RETURN Self
METHOD Resolve( cRef ) CLASS TSqlExecutor
LOCAL cField, cTblAlias, nDot, nWA, nFPos, xVal, nSavedArea
LOCAL i, cA, lOuterFound
LOCAL aCTEInfo, aCTEFN, aCTERows, nCTERow
IF cRef == "*"
RETURN NIL
ENDIF
nSavedArea := Select()
nDot := At( ".", cRef )
IF nDot > 0
cTblAlias := Upper( Left( cRef, nDot - 1 ) )
cField := Upper( SubStr( cRef, nDot + 1 ) )
ELSE
cField := Upper( cRef )
cTblAlias := ""
ENDIF
/* Qualified reference */
IF ! Empty( cTblAlias )
nWA := ::FindWA( cTblAlias )
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
IF nFPos > 0
xVal := FieldGet( nFPos )
dbSelectArea( nSavedArea )
RETURN xVal
ENDIF
dbSelectArea( nSavedArea )
ENDIF
IF Len( ::oSession:aOuterStack ) > 0
lOuterFound := .F.
xVal := ::ResolveFromOuter( cRef, cTblAlias, cField, @lOuterFound )
IF lOuterFound
dbSelectArea( nSavedArea )
RETURN xVal
ENDIF
ENDIF
dbSelectArea( nSavedArea )
RETURN NIL
ENDIF
/* Unqualified: search all tables */
FOR i := 1 TO Len( ::aTables )
cA := ::aTables[ i ][ 2 ]
IF Empty( cA )
cA := ::aTables[ i ][ 1 ]
ENDIF
nWA := Select( cA )
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
IF nFPos > 0
xVal := FieldGet( nFPos )
dbSelectArea( nSavedArea )
RETURN xVal
ENDIF
ENDIF
NEXT
/* Last resort: current workarea */
dbSelectArea( nSavedArea )
nFPos := FieldPos( cField )
IF nFPos > 0
RETURN FieldGet( nFPos )
ENDIF
/* Correlated subquery outer context */
IF Len( ::oSession:aOuterStack ) > 0
lOuterFound := .F.
xVal := ::ResolveFromOuter( cRef, cTblAlias, cField, @lOuterFound )
IF lOuterFound
dbSelectArea( nSavedArea )
RETURN xVal
ENDIF
ENDIF
dbSelectArea( nSavedArea )
RETURN NIL
/* ResolveFromOuter — resolve a column reference in the outer
* context stack. Sets lFound to .T. (by ref) when the column is
* located, even if its value is NIL. Callers must check lFound
* rather than testing `xVal != NIL` — the latter conflates a
* legitimate NULL column value with "column not found", silently
* breaking correlated subqueries where the outer row has NULLs. */
METHOD ResolveFromOuter( cRef, cTblAlias, cField, lFound ) CLASS TSqlExecutor
LOCAL i, j, aOuterTbls, cA, nWA, nFPos, xVal, nSavedArea
lFound := .F.
nSavedArea := Select()
FOR i := Len( ::oSession:aOuterStack ) TO 1 STEP -1
aOuterTbls := ::oSession:aOuterStack[ i ]
FOR j := 1 TO Len( aOuterTbls )
cA := aOuterTbls[ j ][ 2 ]
IF Empty( cA )
cA := aOuterTbls[ j ][ 1 ]
ENDIF
IF ! Empty( cTblAlias )
IF !( Upper( cA ) == cTblAlias .OR. ;
Upper( aOuterTbls[ j ][ 1 ] ) == cTblAlias .OR. ;
( Len( aOuterTbls[ j ] ) >= 3 .AND. Upper( aOuterTbls[ j ][ 3 ] ) == cTblAlias ) )
LOOP
ENDIF
ENDIF
nWA := Select( cA )
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
IF nFPos > 0
xVal := FieldGet( nFPos )
lFound := .T.
dbSelectArea( nSavedArea )
RETURN xVal
ENDIF
ENDIF
NEXT
NEXT
dbSelectArea( nSavedArea )
RETURN NIL
METHOD EvalExpr( xNode ) CLASS TSqlExecutor
LOCAL xL, xR, cOp, xVal, aArgs, aVals, i, xResult, nPI
LOCAL aCases, xElse, xCond
LOCAL aSubResult, xHi, nSavedWA, lSawNull
IF xNode == NIL
RETURN NIL
ENDIF
DO CASE
CASE xNode[ 1 ] == ND_LIT
RETURN xNode[ 2 ]
CASE xNode[ 1 ] == ND_NIL
RETURN NIL
CASE xNode[ 1 ] == ND_COL
/* Fast path: PreResolveColumns cached {nWA, nFPos} at xNode[5].
* Skips the Resolve() string-parse + alias-lookup on every row. */
IF xNode[ 5 ] != NIL
nSavedWA := Select()
dbSelectArea( xNode[ 5 ][ 1 ] )
xVal := FieldGet( xNode[ 5 ][ 2 ] )
dbSelectArea( nSavedWA )
RETURN xVal
ENDIF
RETURN ::Resolve( xNode[ 2 ] )
CASE xNode[ 1 ] == ND_PAR
/* xNode[2] = 1-based parameter index from parser */
nPI := iif( xNode[ 2 ] != NIL, xNode[ 2 ], 1 )
IF nPI >= 1 .AND. nPI <= Len( ::aParams )
RETURN ::aParams[ nPI ]
ENDIF
RETURN NIL
CASE xNode[ 1 ] == ND_UNI
cOp := xNode[ 2 ]
xL := ::EvalExpr( xNode[ 3 ] )
IF cOp == "NOT"
/* SQL three-valued logic: NOT(NULL) = NULL.
* Critical for NOT IN with a NULL in the list. */
IF xL == NIL
RETURN NIL
ENDIF
IF ValType( xL ) == "L"
RETURN ! xL
ENDIF
RETURN .F.
ENDIF
IF cOp == "-"
IF ValType( xL ) == "N"
RETURN -xL
ENDIF
RETURN 0
ENDIF
RETURN xL
CASE xNode[ 1 ] == ND_BIN
cOp := xNode[ 2 ]
/* Short-circuit AND */
IF cOp == "AND"
xL := ::EvalExpr( xNode[ 3 ] )
IF ValType( xL ) == "L" .AND. ! xL
RETURN .F.
ENDIF
xR := ::EvalExpr( xNode[ 4 ] )
RETURN SqlIsTrue( xL ) .AND. SqlIsTrue( xR )
ENDIF
/* Short-circuit OR */
IF cOp == "OR"
xL := ::EvalExpr( xNode[ 3 ] )
IF ValType( xL ) == "L" .AND. xL
RETURN .T.
ENDIF
xR := ::EvalExpr( xNode[ 4 ] )
RETURN SqlIsTrue( xL ) .OR. SqlIsTrue( xR )
ENDIF
/* IN operator — SQL three-valued logic.
*
* x IN (a, b, ...) TRUE if any element equals x
* NULL if x is NULL, or no equal found
* AND the list contains a NULL
* FALSE if no equal found and list is
* fully non-NULL
*
* NOT IN is built by the parser as NOT(IN(...)), so the NULL
* propagates through the unary NOT (NIL → NIL via SqlIsTrue
* gate in the WHERE driver, which filters NULL out — the
* SQL-correct behaviour). This matters when a subquery in the
* list contains a NULL: `x NOT IN (SELECT y FROM t)` must drop
* candidate rows whenever any y is NULL, instead of letting
* them through as TRUE.
*/
IF cOp == "IN"
xL := ::EvalExpr( xNode[ 3 ] )
xR := xNode[ 4 ]
lSawNull := ( xL == NIL )
IF xR != NIL .AND. xR[ 1 ] == ND_LIST
aVals := xR[ 2 ]
FOR i := 1 TO Len( aVals )
xVal := ::EvalExpr( aVals[ i ] )
IF xVal == NIL
lSawNull := .T.
LOOP
ENDIF
IF xL != NIL .AND. SqlCmpEq( xL, xVal )
RETURN .T.
ENDIF
NEXT
IF lSawNull
RETURN NIL
ENDIF
RETURN .F.
ENDIF
IF xR != NIL .AND. xR[ 1 ] == ND_SUB .AND. xR[ 2 ] != NIL
aSubResult := ::SubqueryCached( xR )
IF aSubResult == NIL .OR. ValType( aSubResult ) != "A"
/* Cache miss-fallback */
ENDIF
IF ValType( aSubResult ) == "A" .AND. Len( aSubResult ) >= 2 .AND. ;
ValType( aSubResult[ 2 ] ) == "A"
FOR i := 1 TO Len( aSubResult[ 2 ] )
IF Len( aSubResult[ 2 ][ i ] ) > 0
IF aSubResult[ 2 ][ i ][ 1 ] == NIL
lSawNull := .T.
LOOP
ENDIF
IF xL != NIL .AND. SqlCmpEq( xL, aSubResult[ 2 ][ i ][ 1 ] )
RETURN .T.
ENDIF
ENDIF
NEXT
ENDIF
IF lSawNull
RETURN NIL
ENDIF
RETURN .F.
ENDIF
RETURN .F.
ENDIF
/* IS NULL / IS NOT NULL */
IF cOp == "IS NULL" .OR. cOp == "IS NOT NULL"
xL := ::EvalExpr( xNode[ 3 ] )
/* SQL standard: only NIL is NULL, empty string '' is NOT NULL */
IF cOp == "IS NULL"
RETURN xL == NIL
ELSE
RETURN xL != NIL
ENDIF
ENDIF
/* Standard binary ops */
xL := ::EvalExpr( xNode[ 3 ] )
xR := ::EvalExpr( xNode[ 4 ] )
xL := SqlCoerceForCmp( xL )
xR := SqlCoerceForCmp( xR )
/* SQL three-valued logic: any comparison with NULL is UNKNOWN,
* which the WHERE driver SqlIsTrue() treats as "does not match"
* (drops the row). Previously `v <> 10` on a row where v was
* NULL returned .T. because `! SqlCmpEq(NIL, 10)` = `! .F.` =
* `.T.`. That's wrong: `NULL <> anything` must be NULL.
* Applies to =, <>, <, <=, >, >=. IS NULL / IS NOT NULL /
* IS DISTINCT FROM handle NULLs explicitly elsewhere. */
IF ( cOp == "=" .OR. cOp == "==" .OR. cOp == "<>" .OR. cOp == "!=" .OR. ;
cOp == "<" .OR. cOp == ">" .OR. cOp == "<=" .OR. cOp == ">=" ) .AND. ;
( xL == NIL .OR. xR == NIL )
RETURN NIL
ENDIF
IF cOp == "=" .OR. cOp == "=="
RETURN SqlCmpEq( xL, xR )
ENDIF
IF cOp == "<>" .OR. cOp == "!="
RETURN ! SqlCmpEq( xL, xR )
ENDIF
IF cOp == "<"
RETURN SqlCmpLt( xL, xR )
ENDIF
IF cOp == ">"
RETURN SqlCmpLt( xR, xL )
ENDIF
IF cOp == "<="
RETURN SqlCmpEq( xL, xR ) .OR. SqlCmpLt( xL, xR )
ENDIF
IF cOp == ">="
RETURN SqlCmpEq( xL, xR ) .OR. SqlCmpLt( xR, xL )
ENDIF
/* SQL:2003 IS DISTINCT FROM / IS NOT DISTINCT FROM — NULL-safe
* compare. `a IS DISTINCT FROM b` is .T. iff the values differ
* (treating NULL as a distinct value of its own); `a IS NOT
* DISTINCT FROM b` is its negation. Unlike `=` / `<>`, they
* never return UNKNOWN — the parser parsed these as their own
* ND_BIN op codes but EvalExpr had no handler so they fell
* through to RETURN NIL → WHERE always dropped the row. */
IF cOp == "IS DISTINCT FROM"
IF xL == NIL .AND. xR == NIL
RETURN .F.
ENDIF
IF xL == NIL .OR. xR == NIL
RETURN .T.
ENDIF
RETURN ! SqlCmpEq( xL, xR )
ENDIF
IF cOp == "IS NOT DISTINCT FROM"
IF xL == NIL .AND. xR == NIL
RETURN .T.
ENDIF
IF xL == NIL .OR. xR == NIL
RETURN .F.
ENDIF
RETURN SqlCmpEq( xL, xR )
ENDIF
IF cOp == "LIKE"
IF xNode[ 5 ] != NIL
RETURN SqlLikeMatch( SqlCoerceStr( xL ), SqlCoerceStr( xR ), SqlCoerceStr( ::EvalExpr( xNode[ 5 ] ) ) )
ENDIF
RETURN SqlLikeMatch( SqlCoerceStr( xL ), SqlCoerceStr( xR ) )
ENDIF
/* Arithmetic + CONCAT: SQL NULL propagation. Any NULL operand
* yields NULL (except the Harbour-style string `+` between two
* non-NULL C values, which the caller explicitly relied on for
* name-concat idioms). Division by zero → NULL per SQL spec
* rather than silently returning 0. */
IF cOp == "+"
IF xL == NIL .OR. xR == NIL
IF ValType( xL ) == "C" .AND. ValType( xR ) == "C"
RETURN xL + xR
ENDIF
RETURN NIL
ENDIF
IF ValType( xL ) == "C" .AND. ValType( xR ) == "C"
RETURN xL + xR
ENDIF
/* Harbour Date arithmetic: Date + N → Date (N days later).
* Without this branch, SqlCoerceNum collapsed the date to 0
* and the projection returned the raw integer offset
* (`d + 7` came back as `7`). N + Date is symmetric. */
IF ValType( xL ) == "D" .AND. ValType( xR ) == "N"
RETURN xL + xR
ENDIF
IF ValType( xL ) == "N" .AND. ValType( xR ) == "D"
RETURN xR + xL
ENDIF
RETURN SqlCoerceNum( xL ) + SqlCoerceNum( xR )
ENDIF
IF cOp == "-"
IF xL == NIL .OR. xR == NIL
RETURN NIL
ENDIF
/* Date - N → Date (N days earlier); Date - Date → N (day
* gap). Both reduced to 0 - 0 = 0 before this branch. */
IF ValType( xL ) == "D" .AND. ValType( xR ) == "N"
RETURN xL - xR
ENDIF
IF ValType( xL ) == "D" .AND. ValType( xR ) == "D"
RETURN xL - xR
ENDIF
RETURN SqlCoerceNum( xL ) - SqlCoerceNum( xR )
ENDIF
IF cOp == "*"
IF xL == NIL .OR. xR == NIL
RETURN NIL
ENDIF
RETURN SqlCoerceNum( xL ) * SqlCoerceNum( xR )
ENDIF
IF cOp == "/"
IF xL == NIL .OR. xR == NIL
RETURN NIL
ENDIF
IF SqlCoerceNum( xR ) != 0
RETURN SqlCoerceNum( xL ) / SqlCoerceNum( xR )
ENDIF
RETURN NIL
ENDIF
IF cOp == "||"
IF xL == NIL .OR. xR == NIL
RETURN NIL
ENDIF
RETURN SqlCoerceStr( xL ) + SqlCoerceStr( xR )
ENDIF
RETURN NIL
CASE xNode[ 1 ] == ND_RANGE
xL := ::EvalExpr( xNode[ 3 ] )
xR := ::EvalExpr( xNode[ 4 ] )
xHi := ::EvalExpr( xNode[ 5 ] )
/* SQL 3-value logic: any NULL operand → NULL (UNKNOWN). The
* WHERE driver SqlIsTrue() drops the row. Without this guard
* `v NOT BETWEEN 10 AND 30` returned rows where v was NULL
* because the comparison fell through to .F. → !.F. = .T..
* The standard says NULL BETWEEN/NOT-BETWEEN must always be
* UNKNOWN regardless of the bounds. */
IF xL == NIL .OR. xR == NIL .OR. xHi == NIL
RETURN NIL
ENDIF
xL := SqlCoerceForCmp( xL )
xR := SqlCoerceForCmp( xR )
xHi := SqlCoerceForCmp( xHi )
RETURN ( SqlCmpEq( xL, xR ) .OR. SqlCmpLt( xR, xL ) ) .AND. ( SqlCmpEq( xL, xHi ) .OR. SqlCmpLt( xL, xHi ) )
CASE xNode[ 1 ] == ND_CASE
aCases := xNode[ 2 ]
xElse := xNode[ 3 ]
FOR i := 1 TO Len( aCases )
xCond := ::EvalExpr( aCases[ i ][ 1 ] )
IF SqlIsTrue( xCond )
RETURN ::EvalExpr( aCases[ i ][ 2 ] )
ENDIF
NEXT
IF xElse != NIL
RETURN ::EvalExpr( xElse )
ENDIF
RETURN NIL
CASE xNode[ 1 ] == ND_FN
/* EXISTS and NOT EXISTS handling:
*
* 1. If the subquery matches the semi-join shape (single-table
* with a `inner.col = outer.col` equi-term and no JOIN /
* GROUP BY / aggregate), lift it into a non-correlated
* hash set probe: run the subquery ONCE with the correlated
* term removed and DISTINCT on inner.col, then each outer
* row becomes an O(1) hash lookup. This is the key win
* for patterns like
* WHERE EXISTS (SELECT 1 FROM ord WHERE ord.emp_id = e.id
* AND ord.qty > 15)
* where the correlation is 1:1 with outer rows so plain
* memoization doesn't help.
*
* 2. Otherwise inject LIMIT 1 and route through SubqueryCached
* so at least the scan short-circuits on first match and
* low-cardinality correlations still memoize. */
IF ( xNode[ 2 ] == "EXISTS" .OR. xNode[ 2 ] == "NOT EXISTS" ) .AND. ;
Len( xNode[ 3 ] ) > 0 .AND. ;
xNode[ 3 ][ 1 ] != NIL .AND. ValType( xNode[ 3 ][ 1 ] ) == "A" .AND. ;
xNode[ 3 ][ 1 ][ 1 ] == ND_SUB .AND. xNode[ 3 ][ 1 ][ 2 ] != NIL
aSubResult := ::ExistsViaSemiJoin( xNode[ 3 ][ 1 ], xNode[ 2 ] == "NOT EXISTS" )
IF aSubResult != NIL
/* Semi-join lift succeeded; result is already a boolean */
RETURN aSubResult
ENDIF
/* Fallback: LIMIT 1 + cached run.
* SubqueryCached clones the hQuery per-Run, so this LIMIT
* won't corrupt subsequent runs. Safe even if plan is reused. */
aSubResult := ::SubqueryCached( xNode[ 3 ][ 1 ] )
IF ValType( aSubResult ) == "A" .AND. Len( aSubResult ) >= 2 .AND. ;
ValType( aSubResult[ 2 ] ) == "A"
IF xNode[ 2 ] == "NOT EXISTS"
RETURN Len( aSubResult[ 2 ] ) == 0
ENDIF
RETURN Len( aSubResult[ 2 ] ) > 0
ENDIF
RETURN iif( xNode[ 2 ] == "NOT EXISTS", .T., .F. )
ENDIF
/* Evaluate arguments */
aArgs := {}
FOR i := 1 TO Len( xNode[ 3 ] )
AAdd( aArgs, ::EvalExpr( xNode[ 3 ][ i ] ) )
NEXT
RETURN SqlEvalFunc( xNode[ 2 ], aArgs )
CASE xNode[ 1 ] == ND_SUB
IF xNode[ 2 ] != NIL
/* Subqueries use a per-outer-key cache. SubqueryCached
* does static free-variable analysis on the first call,
* then memoizes results keyed by the current values of
* those free variables. Non-correlated subqueries reduce
* to a trivial single-entry cache. */
aSubResult := ::SubqueryCached( xNode )
/* Skip the `__error__` envelope — extracting aResult[2][1][1]
* blindly would surface the numeric error code (e.g. 1005 =
* SQL_ERR_LOCKED) as the scalar value, silently passing
* garbage into the WHERE comparison. */
IF ValType( aSubResult ) == "A" .AND. Len( aSubResult ) >= 1 .AND. ;
ValType( aSubResult[ 1 ] ) == "A" .AND. Len( aSubResult[ 1 ] ) >= 1 .AND. ;
aSubResult[ 1 ][ 1 ] == "__error__"
RETURN NIL
ENDIF
IF ValType( aSubResult ) == "A" .AND. Len( aSubResult ) >= 2 .AND. ;
ValType( aSubResult[ 2 ] ) == "A" .AND. Len( aSubResult[ 2 ] ) > 0 .AND. ;
Len( aSubResult[ 2 ][ 1 ] ) > 0
/* SQL standard: a scalar subquery must return at most one
* row. Returning silently the first row of a multi-row
* result hid bugs (`INSERT ... VALUES ((SELECT id FROM
* t), 100)` quietly took an arbitrary row) — surface to
* stderr so the developer notices, then collapse to NIL
* to keep three-valued logic consistent (NULL beats
* "arbitrary garbage from row 1"). */
IF Len( aSubResult[ 2 ] ) > 1
OutErr( "FiveSQL: scalar subquery returned " + ;
hb_NToS( Len( aSubResult[ 2 ] ) ) + ;
" rows; SQL standard requires at most 1 — using NULL." + Chr( 10 ) )
RETURN NIL
ENDIF
RETURN aSubResult[ 2 ][ 1 ][ 1 ]
ENDIF
ENDIF
RETURN NIL
CASE xNode[ 1 ] == ND_WINDOW
/* Window functions are evaluated post-fetch, return placeholder */
RETURN 0
ENDCASE
RETURN NIL
/* Pre-compute {nWA, nFPos} for each SELECT expression that is a plain
* column reference. Called once before a join/scan loop so that FetchRow
* can skip the per-row string parse (At, SubStr, Upper) and FindWA
* linear scan. Complex expressions (functions, CASE, subqueries) store
* NIL and fall back to EvalExpr.
*
* Safe for multi-table queries: resolution walks ::aTables and binds
* each column to a specific workarea number and field position.
*/
METHOD BuildFetchCache( aExprs ) CLASS TSqlExecutor
LOCAL aCache := {}, i, xE, cRef, nDot, cTblAlias, cField, nWA, nFPos, cA
LOCAL nSaved := Select()
FOR i := 1 TO Len( aExprs )
xE := aExprs[ i ][ 1 ]
IF xE == NIL .OR. xE[ 1 ] != ND_COL .OR. xE[ 2 ] == "*"
AAdd( aCache, NIL )
LOOP
ENDIF
cRef := xE[ 2 ]
nDot := At( ".", cRef )
IF nDot > 0
cTblAlias := Upper( Left( cRef, nDot - 1 ) )
cField := Upper( SubStr( cRef, nDot + 1 ) )
nWA := ::FindWA( cTblAlias )
ELSE
cField := Upper( cRef )
cTblAlias := ""
nWA := 0
IF Len( ::aTables ) > 0
cA := ::aTables[ 1 ][ 2 ]
IF Empty( cA )
cA := ::aTables[ 1 ][ 1 ]
ENDIF
nWA := Select( cA )
ENDIF
ENDIF
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
IF nFPos > 0
AAdd( aCache, { nWA, nFPos } )
LOOP
ENDIF
ENDIF
AAdd( aCache, NIL )
NEXT
dbSelectArea( nSaved )
RETURN aCache
METHOD FetchRow( aExprs ) CLASS TSqlExecutor
LOCAL aRow := {}, i, xVal
LOCAL xE, cRef, nDot, nWA, nFPos, cField, cTblAlias, cA
/* Fastest path: pre-bound columns (built once per join by RunSelect).
* Go-native: SqlFetchRowFast collapses the per-row Harbour FOR loop
* into a single Go call, saving ~30% of GROUP BY CPU spent in PRG
* method dispatch. Falls back to self:EvalExpr for unbound entries. */
IF ::aFetchCache != NIL .AND. Len( ::aFetchCache ) == Len( aExprs )
RETURN SqlFetchRowFast( Self, aExprs, ::aFetchCache )
ENDIF
FOR i := 1 TO Len( aExprs )
xE := aExprs[ i ][ 1 ]
/* Fast path for column references */
IF xE[ 1 ] == ND_COL .AND. xE[ 2 ] != "*" .AND. Len( ::aTables ) > 0
cRef := xE[ 2 ]
nDot := At( ".", cRef )
IF nDot > 0
cTblAlias := Upper( Left( cRef, nDot - 1 ) )
cField := Upper( SubStr( cRef, nDot + 1 ) )
nWA := ::FindWA( cTblAlias )
ELSE
cField := Upper( cRef )
cA := ::aTables[ 1 ][ 2 ]
IF Empty( cA )
cA := ::aTables[ 1 ][ 1 ]
ENDIF
nWA := Select( cA )
ENDIF
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cField )
IF nFPos > 0
xVal := FieldGet( nFPos )
IF ValType( xVal ) == "C"
xVal := AllTrim( xVal )
ENDIF
AAdd( aRow, xVal )
LOOP
ENDIF
ENDIF
ENDIF
/* General expression evaluation path */
xVal := ::EvalExpr( xE )
IF ValType( xVal ) == "C"
xVal := AllTrim( xVal )
ENDIF
AAdd( aRow, xVal )
NEXT
RETURN aRow
METHOD ColBelongsTo( cColRef, cAlias ) CLASS TSqlExecutor
LOCAL cPrefix, nDot, i, cA, cOrig
nDot := At( ".", cColRef )
IF nDot == 0
RETURN .F.
ENDIF
cPrefix := Upper( Left( cColRef, nDot - 1 ) )
IF cPrefix == Upper( cAlias )
RETURN .T.
ENDIF
FOR i := 1 TO Len( ::aTables )
cA := Upper( ::aTables[ i ][ 2 ] )
IF Empty( cA )
cA := Upper( ::aTables[ i ][ 1 ] )
ENDIF
cOrig := ""
IF Len( ::aTables[ i ] ) >= 3
cOrig := Upper( ::aTables[ i ][ 3 ] )
ENDIF
IF cA == Upper( cAlias ) .OR. cOrig == Upper( cAlias )
IF cPrefix == cA .OR. cPrefix == cOrig .OR. cPrefix == Upper( ::aTables[ i ][ 1 ] )
RETURN .T.
ENDIF
ENDIF
NEXT
RETURN .F.
METHOD FetchRowNull( aRE, cInnerAlias ) CLASS TSqlExecutor
LOCAL aRow := {}, i, xVal
LOCAL cColRef, lIsInner, nWA, nFPos, cBareField
FOR i := 1 TO Len( aRE )
cColRef := ""
IF aRE[ i ][ 1 ] != NIL .AND. aRE[ i ][ 1 ][ 1 ] == ND_COL
cColRef := Upper( aRE[ i ][ 1 ][ 2 ] )
ENDIF
lIsInner := .F.
IF ! Empty( cColRef )
IF ::ColBelongsTo( cColRef, cInnerAlias )
lIsInner := .T.
ELSEIF ! ( "." $ cColRef )
nWA := Select( cInnerAlias )
IF nWA > 0
dbSelectArea( nWA )
nFPos := FieldPos( cColRef )
IF nFPos > 0
lIsInner := .T.
IF Len( ::aTables ) > 0
cBareField := cColRef
nWA := ::FindWA( ::aTables[ 1 ][ 2 ] )
IF nWA == 0
nWA := ::FindWA( ::aTables[ 1 ][ 1 ] )
ENDIF
IF nWA > 0
dbSelectArea( nWA )
IF FieldPos( cBareField ) > 0
lIsInner := .F.
ENDIF
ENDIF
ENDIF
ENDIF
ENDIF
ENDIF
ENDIF
IF lIsInner
AAdd( aRow, NIL )
ELSE
xVal := ::EvalExpr( aRE[ i ][ 1 ] )
IF ValType( xVal ) == "C"
xVal := AllTrim( xVal )
ENDIF
AAdd( aRow, xVal )
ENDIF
NEXT
RETURN aRow
/* -----------------------------------------------------------------
* Helpers for WHERE predicate pushdown across JOIN levels.
*
* SplitAndClauses walks the top-level AND chain of a WHERE expression
* and returns each conjunct as its own tree. Non-AND trees come back
* as a single-element array, so callers don't need a special case.
*
* BuildAliasLevelMap assigns each table reference a depth: 0 for the
* primary (outer) table and 1..N for each JOIN entry in aJoins. Both
* the SQL alias (as quoted by the query text) and any temp/user
* alternative alias stored in aTables[i][3] are registered so
* qualified names like "o.id" and synonyms still resolve.
*
* ClauseMaxLevel returns the highest level referenced by any
* qualified column in the clause. Unqualified columns can't be
* pinned to a single table without the full FindWA dispatch, so they
* force the conservative default — callers pass Len(aJoins) to make
* such clauses fall back to the base-case evaluation.
* ----------------------------------------------------------------- */
METHOD SplitAndClauses( xE, aOut ) CLASS TSqlExecutor
IF aOut == NIL
aOut := {}
ENDIF
IF xE == NIL
RETURN aOut
ENDIF
IF ValType( xE ) == "A" .AND. Len( xE ) >= 4 .AND. ;
xE[ 1 ] == ND_BIN .AND. xE[ 2 ] == "AND"
::SplitAndClauses( xE[ 3 ], aOut )
::SplitAndClauses( xE[ 4 ], aOut )
ELSE
AAdd( aOut, xE )
ENDIF
RETURN aOut
METHOD BuildAliasLevelMap( aJoins ) CLASS TSqlExecutor
LOCAL hMap := { => }, i, nLvl
/* aTables is ordered [primary, join1, join2, ...]; index j maps to
* join level j-1 (primary = level 0). For each table entry,
* register every name that WHERE / ON expressions might use:
*
* [1] = original table name (e.g. "ORD")
* [2] = currently-selected alias — possibly a depth-suffixed
* temp ("ORD_2") introduced by oAlias:AcquireTemp when
* the SQL alias was too short or we're in a subquery.
* [3] = original SQL alias from the query text ("o") — this is
* what the parser writes into ND_COL qualifiers, so it's
* the name we most need to resolve, and it's the one that
* wasn't in the old map because aJoins[i][3] gets
* overwritten with the temp alias during JOIN sync. */
FOR i := 1 TO Len( ::aTables )
nLvl := i - 1
IF ! Empty( ::aTables[ i ][ 1 ] )
hMap[ Upper( ::aTables[ i ][ 1 ] ) ] := nLvl
ENDIF
IF ! Empty( ::aTables[ i ][ 2 ] )
hMap[ Upper( ::aTables[ i ][ 2 ] ) ] := nLvl
ENDIF
IF Len( ::aTables[ i ] ) >= 3 .AND. ! Empty( ::aTables[ i ][ 3 ] )
hMap[ Upper( ::aTables[ i ][ 3 ] ) ] := nLvl
ENDIF
NEXT
RETURN hMap
METHOD ClauseMaxLevel( xClause, hAliasToLevel, nDefault ) CLASS TSqlExecutor
LOCAL aCols, i, cName, nDot, cAlias
LOCAL nMax := 0
LOCAL lUncertain := .F.
aCols := SqlCollectColExprs( xClause, NIL )
FOR i := 1 TO Len( aCols )
cName := aCols[ i ][ 2 ]
nDot := At( ".", cName )
IF nDot == 0
/* Unqualified column — could resolve to any open workarea
* via FindWA. Force the conservative default so the clause
* is evaluated after every referenced table is positioned
* (base case), never at an intermediate level where the
* column might bind to a stale inner record. */
lUncertain := .T.
EXIT
ENDIF
cAlias := Upper( Left( cName, nDot - 1 ) )
IF hb_HHasKey( hAliasToLevel, cAlias )
IF hAliasToLevel[ cAlias ] > nMax
nMax := hAliasToLevel[ cAlias ]
ENDIF
ELSE
lUncertain := .T.
EXIT
ENDIF
NEXT
IF lUncertain
RETURN nDefault
ENDIF
RETURN nMax
METHOD EvalPushedAtLevel( aPushByLevel, nIdx ) CLASS TSqlExecutor
LOCAL p, aClauses
IF aPushByLevel == NIL .OR. nIdx < 1 .OR. nIdx > Len( aPushByLevel )
RETURN .T.
ENDIF
aClauses := aPushByLevel[ nIdx ]
FOR p := 1 TO Len( aClauses )
IF ! SqlIsTrue( ::EvalExpr( aClauses[ p ] ) )
RETURN .F.
ENDIF
NEXT
RETURN .T.
METHOD JoinRecurse( aJoins, nIdx, xWhere, aRE, aRows, hHashTbl, aPushByLevel ) CLASS TSqlExecutor
LOCAL cJAlias, xOnCond, nWA, aRow
LOCAL lJoinMatch
LOCAL cOuterCol, cInnerCol, cInnerField, xSeekVal, cSeekStr
LOCAL lUseIndex, lFound, nPI
LOCAL cJoinType, lHadMatch
LOCAL nRecCount, lUseHash
LOCAL xProbe, cRMKey
IF hHashTbl == NIL
hHashTbl := { => }
ENDIF
IF nIdx > Len( aJoins )
IF xWhere == NIL .OR. SqlIsTrue( ::EvalExpr( xWhere ) )
aRow := ::FetchRow( aRE )
AAdd( aRows, aRow )
ENDIF
RETURN NIL
ENDIF
cJoinType := Upper( aJoins[ nIdx ][ 1 ] )
cJAlias := aJoins[ nIdx ][ 3 ]
IF Empty( cJAlias )
cJAlias := aJoins[ nIdx ][ 2 ]
ENDIF
xOnCond := aJoins[ nIdx ][ 4 ]
nWA := Select( cJAlias )
IF nWA == 0
/* Try the join table name directly (handles CTE alias mismatch) */
nWA := Select( Upper( aJoins[ nIdx ][ 2 ] ) )
ENDIF
IF nWA == 0
RETURN NIL
ENDIF
/* CROSS JOIN */
IF cJoinType == "CROSS"
dbSelectArea( nWA )
dbGoTop()
WHILE ! Eof()
IF ::EvalPushedAtLevel( aPushByLevel, nIdx )
::JoinRecurse( aJoins, nIdx + 1, xWhere, aRE, @aRows, hHashTbl, aPushByLevel )
ENDIF
dbSelectArea( nWA )
dbSkip()
ENDDO
RETURN NIL
ENDIF
lHadMatch := .F.
lUseIndex := .F.
lUseHash := .F.
/* Track matched inner RecNos for RIGHT/FULL JOIN pass */
cRMKey := "__RIGHT_" + Upper( cJAlias )
cOuterCol := ""
cInnerCol := ""
cInnerField := ""
/* Analyze ON condition for index or hash join optimization.
* Handles both `a.x = b.x` and `a.x = b.x AND ...` — for the AND
* case we pick the first equi-join term as the hash key and the
* HashJoin method re-evaluates the full xOnCond after probe to
* filter out spurious matches. This is how SQLite's hash-join
* fallback handles compound predicates. */
xProbe := xOnCond
IF xOnCond != NIL .AND. xOnCond[ 1 ] == ND_BIN .AND. xOnCond[ 2 ] == "AND"
/* Walk left-associative AND chain until we find an equi-term */
xProbe := xOnCond
WHILE xProbe != NIL .AND. xProbe[ 1 ] == ND_BIN .AND. xProbe[ 2 ] == "AND"
/* Prefer left operand if it's an equi-join */
IF xProbe[ 3 ] != NIL .AND. xProbe[ 3 ][ 1 ] == ND_BIN .AND. xProbe[ 3 ][ 2 ] == "="
xProbe := xProbe[ 3 ]
EXIT
ENDIF
xProbe := xProbe[ 4 ] /* descend right */
ENDDO
ENDIF
IF xProbe != NIL .AND. xProbe[ 1 ] == ND_BIN .AND. xProbe[ 2 ] == "="
IF xProbe[ 3 ] != NIL .AND. xProbe[ 3 ][ 1 ] == ND_COL .AND. ;
xProbe[ 4 ] != NIL .AND. xProbe[ 4 ][ 1 ] == ND_COL
IF ::ColBelongsTo( xProbe[ 4 ][ 2 ], cJAlias )
cOuterCol := xProbe[ 3 ][ 2 ]
cInnerCol := xProbe[ 4 ][ 2 ]
ELSEIF ::ColBelongsTo( xProbe[ 3 ][ 2 ], cJAlias )
cOuterCol := xProbe[ 4 ][ 2 ]
cInnerCol := xProbe[ 3 ][ 2 ]
ENDIF
ENDIF
IF ! Empty( cInnerCol )
IF "." $ cInnerCol
cInnerField := Upper( SubStr( cInnerCol, At( ".", cInnerCol ) + 1 ) )
ELSE
cInnerField := Upper( cInnerCol )
ENDIF
dbSelectArea( nWA )
lUseIndex := ( ::oIndex:FindBestTag( nWA, cInnerField ) > 0 )
/* Hash join for equi-joins when no index is available. For
* very small inner tables the Go map allocation + per-key
* string formatting dominates the cost of a cache-friendly
* nested-loop scan, so we keep the old per-iteration scan
* when RecCount falls below the threshold. 64 was picked
* empirically — SQLite uses a similar constant; tune via
* bench_join if workload changes. */
IF ! lUseIndex .AND. ! Empty( cOuterCol )
nRecCount := LastRec()
IF nRecCount > 64
lUseHash := .T.
ENDIF
ENDIF
ENDIF
ENDIF
IF lUseIndex
xSeekVal := ::EvalExpr( SqlNode( ND_COL, cOuterCol, NIL, NIL, NIL ) )
dbSelectArea( nWA )
cSeekStr := ::oIndex:BuildKey( nWA, xSeekVal )
lFound := dbSeek( cSeekStr )
WHILE lFound .AND. ! Eof()
lJoinMatch := SqlIsTrue( ::EvalExpr( xOnCond ) )
IF ! lJoinMatch
EXIT
ENDIF
lHadMatch := .T.
/* Predicate pushdown: WHERE conjuncts whose referenced columns
* all bind by level nIdx are evaluated before we recurse, so
* rows rejected here skip the exponential join expansion that
* used to happen when xWhere only fired at the base case. */
IF ::EvalPushedAtLevel( aPushByLevel, nIdx )
::JoinRecurse( aJoins, nIdx + 1, xWhere, aRE, @aRows, hHashTbl, aPushByLevel )
ENDIF
dbSelectArea( nWA )
dbSkip()
IF Eof()
EXIT
ENDIF
ENDDO
ELSEIF lUseHash
/* Hash join path for equi-joins on large tables without index */
lHadMatch := ::HashJoin( nWA, cInnerField, cOuterCol, xOnCond, ;
aJoins, nIdx, xWhere, aRE, @aRows, @hHashTbl, ;
aPushByLevel )
ELSE
dbSelectArea( nWA )
dbGoTop()
WHILE ! Eof()
lJoinMatch := .T.
IF xOnCond != NIL
lJoinMatch := SqlIsTrue( ::EvalExpr( xOnCond ) )
ENDIF
IF lJoinMatch
lHadMatch := .T.
/* Record match for RIGHT JOIN pass */
IF ! hb_HHasKey( ::hRightMatched, cRMKey )
::hRightMatched[ cRMKey ] := { => }
ENDIF
::hRightMatched[ cRMKey ][ RecNo() ] := .T.
IF ::EvalPushedAtLevel( aPushByLevel, nIdx )
::JoinRecurse( aJoins, nIdx + 1, xWhere, aRE, @aRows, hHashTbl, aPushByLevel )
ENDIF
ENDIF
dbSelectArea( nWA )
dbSkip()
ENDDO
ENDIF
/* LEFT JOIN NULL fill — when no match was found for the current
* join level, emit a NULL-filled row. For multi-level JOINs
* (a LEFT JOIN b ON ... JOIN c ON ...) we must recurse into
* subsequent join levels rather than only emitting at the last
* one — otherwise the middle LEFT JOIN's NULL fill never reaches
* the base case and the entire outer row is silently dropped. */
IF ! lHadMatch .AND. ( cJoinType == "LEFT" .OR. cJoinType == "FULL" )
IF nIdx >= Len( aJoins )
/* Last join — emit directly */
aRow := ::FetchRowNull( aRE, cJAlias )
IF xWhere == NIL .OR. SqlIsTrue( ::EvalExpr( xWhere ) )
AAdd( aRows, aRow )
ENDIF
ELSE
/* Middle join — recurse with NULL-filled state for this level
* so subsequent joins can still process and emit their own
* NULL rows or matches. */
::JoinRecurse( aJoins, nIdx + 1, xWhere, aRE, @aRows, hHashTbl, aPushByLevel )
ENDIF
ENDIF
RETURN NIL
/* RightJoinPass — emit inner rows that had no match during the main
* join pass (for RIGHT/FULL joins). Outer columns are NIL.
*
* Previous O(N*M) approach rescanned the outer table for every inner
* row to detect unmatched ones. Now uses ::hRightMatched (populated
* during the main join) as a RecNo set — O(N) inner scan + O(1)
* hash probe per row.
*/
METHOD RightJoinPass( aJoins, nJIdx, aRE, aRows ) CLASS TSqlExecutor
LOCAL cJAlias, nWA, cOuterAlias
LOCAL aRow, j, cColRef, cMatchKey, nRec
cJAlias := aJoins[ nJIdx ][ 3 ]
IF Empty( cJAlias )
cJAlias := aJoins[ nJIdx ][ 2 ]
ENDIF
nWA := Select( cJAlias )
IF nWA == 0
RETURN NIL
ENDIF
cOuterAlias := ""
IF Len( ::aTables ) > 0
cOuterAlias := ::aTables[ 1 ][ 2 ]
IF Empty( cOuterAlias )
cOuterAlias := ::aTables[ 1 ][ 1 ]
ENDIF
ENDIF
cMatchKey := "__RIGHT_" + Upper( cJAlias )
dbSelectArea( nWA )
dbGoTop()
WHILE ! Eof()
nRec := RecNo()
IF hb_HHasKey( ::hRightMatched, cMatchKey ) .AND. ;
hb_HHasKey( ::hRightMatched[ cMatchKey ], nRec )
/* Matched during main join — skip */
ELSE
/* Unmatched inner row — emit with NULLs for outer columns */
aRow := {}
FOR j := 1 TO Len( aRE )
cColRef := ""
IF aRE[ j ][ 1 ] != NIL .AND. aRE[ j ][ 1 ][ 1 ] == ND_COL
cColRef := Upper( aRE[ j ][ 1 ][ 2 ] )
ENDIF
IF ! Empty( cColRef ) .AND. ::ColBelongsTo( cColRef, cOuterAlias )
AAdd( aRow, NIL )
ELSE
AAdd( aRow, ::EvalExpr( aRE[ j ][ 1 ] ) )
ENDIF
NEXT
AAdd( aRows, aRow )
ENDIF
dbSelectArea( nWA )
dbSkip()
ENDDO
RETURN NIL
METHOD RunSelect() CLASS TSqlExecutor
LOCAL aCols, aJoins, xWhere, aGroupBy, xHaving, aOrderBy
LOCAL nTop, nLimit, nOffset, lDistinct, hUnion
LOCAL aFieldNames := {}, aRows := {}, aRow
LOCAL aSavedAreas := {}
LOCAL cTable, cAlias, nWA, i, j
LOCAL aResultExprs
LOCAL xExpr, cColAlias, cFN
LOCAL nMaxRows
LOCAL aU, lAll
LOCAL xArgExpr, cBare, lFound, aLeafCols, k
LOCAL hJoinHash
LOCAL lIndexUsed, aTmp
LOCAL aFP, pcW, aGoRows
LOCAL nEarlyLimit, aSortSpec
LOCAL lOrderFromIndex := .F.
LOCAL aPushByLevel, xRes, aClauses, hAliasLvl, nLvl, ii
LOCAL nScanRec
LOCAL nUserCols, nTrim, nRow
aCols := ::hQuery[ "columns" ]
/* Deep-clone tables and joins so cross-run state (alias renames,
* fetch-cache references, etc.) doesn't leak between invocations
* of the same hQuery. A scalar correlated subquery that opens its
* FROM tables gets depth-suffixed temp aliases written back into
* aTables[i][2] and aJoins[i][3]; without this clone, the second
* call inherits the first call's dead alias and the JOIN sync
* loop below fails to match, leaving stale aliases that resolve
* to closed workareas. */
::aTables := AClone( ::hQuery[ "tables" ] )
FOR i := 1 TO Len( ::aTables )
IF ValType( ::aTables[ i ] ) == "A"
::aTables[ i ] := AClone( ::aTables[ i ] )
ENDIF
NEXT
aJoins := AClone( ::hQuery[ "joins" ] )
FOR i := 1 TO Len( aJoins )
IF ValType( aJoins[ i ] ) == "A"
aJoins[ i ] := AClone( aJoins[ i ] )
ENDIF
NEXT
xWhere := ::hQuery[ "where" ]
aGroupBy := ::hQuery[ "group_by" ]
xHaving := ::hQuery[ "having" ]
aOrderBy := ::hQuery[ "order_by" ]
nTop := ::hQuery[ "top" ]
nLimit := ::hQuery[ "limit" ]
nOffset := iif( hb_HHasKey( ::hQuery, "offset" ), ::hQuery[ "offset" ], 0 )
lDistinct := ::hQuery[ "distinct" ]
hUnion := ::hQuery[ "union" ]
AAdd( aSavedAreas, Select() )
::nDepth++
/* Materialize CTEs if present */
IF hb_HHasKey( ::hQuery, "cte" ) .AND. ValType( ::hQuery[ "cte" ] ) == "A"
IF hb_HHasKey( ::hQuery, "cte_recursive" ) .AND. ::hQuery[ "cte_recursive" ]
::MaterializeRecursiveCTE( ::hQuery[ "cte" ] )
ELSE
::MaterializeCTE( ::hQuery[ "cte" ] )
ENDIF
ENDIF
/* Handle derived tables */
FOR i := 1 TO Len( ::aTables )
IF ::aTables[ i ][ 1 ] == "__SUBQUERY__" .AND. ;
ValType( ::aTables[ i ][ 3 ] ) == "A" .AND. ;
::aTables[ i ][ 3 ][ 1 ] == ND_SUB
cAlias := ::aTables[ i ][ 2 ]
IF Empty( cAlias )
cAlias := ::oAlias:AcquireTemp( "DRV" )
ENDIF
::aTables[ i ] := SqlMaterializeSubquery( ::aTables[ i ][ 3 ], cAlias, ::aParams )
/* Track the derived MEMRDD area so CloseOpened (called via
* the top-level CloseOpened path at RunSelect's tail) frees
* the alias before the next query runs. Without this, a
* second SELECT that uses the same derived alias panics
* with "alias already in use" inside SqlMaterializeSubquery. */
AAdd( ::aOpened, cAlias )
ENDIF
NEXT
/* Open all referenced tables */
FOR i := 1 TO Len( ::aTables )
cTable := ::aTables[ i ][ 1 ]
cAlias := ::aTables[ i ][ 2 ]
IF Empty( cAlias )
cAlias := cTable
ENDIF
/* Always stash the user-written alias in slot [3] so that FindWA /
* Resolve can still match queries that reference the alias by its
* SQL name even after we re-alias the workarea with a depth-
* suffixed temp name. Previously this was only done for 1-char
* aliases, which left multi-char aliases (e.g. `emp e2`) invisible
* to correlated subquery lookups once the rename kicked in. */
IF Empty( ::aTables[ i ][ 3 ] )
::aTables[ i ][ 3 ] := cAlias
ENDIF
/* Derived tables already opened their own MEMRDD area under the
* user's alias inside SqlMaterializeSubquery — don't rename it,
* or Select(cAlias) below will miss the area and the open loop
* will think the (synthetic) `__drv_<x>` "table" is missing on
* disk. The rename for short aliases / nested depth is meant
* to avoid alias collisions on real DBF tables. */
IF ( Len( cAlias ) <= 1 .OR. ::nDepth > 1 ) .AND. ;
Left( Lower( cTable ), 6 ) != "__drv_"
cAlias := ::oAlias:AcquireTemp( Upper( cTable ) )
::aTables[ i ][ 2 ] := cAlias
ENDIF
nWA := Select( cAlias )
IF nWA == 0
nWA := ::OpenTable( cTable, cAlias )
IF nWA == 0
/* Table file not found; check if a CTE temp table exists for
* this table name and open it instead. This handles sub-
* executors (UNION, recursive) that reference a CTE by its
* original name. CTE temp tables now live in MEMRDD (no
* file on disk) — fall back to the legacy DBFNTX open for
* pre-existing .dbf files from prior runs. */
BEGIN SEQUENCE
dbUseArea( .T., "MEMRDD", "mem:__cte_" + Lower( cTable ), ;
cAlias, .T., .T. )
nWA := Select( cAlias )
IF nWA > 0
AAdd( ::aOpened, cAlias )
AAdd( ::oAlias:aSlots, { cAlias, Upper( cTable ), Upper( cTable ), .T. } )
ENDIF
RECOVER
nWA := 0
END SEQUENCE
IF nWA == 0 .AND. hb_FileExists( "__cte_" + Lower( cTable ) + ".dbf" )
::oSession:lCteDiskSeen := .T.
BEGIN SEQUENCE
dbUseArea( .T., "DBFNTX", "__cte_" + Lower( cTable ) + ".dbf", ;
cAlias, .T., .T. )
nWA := Select( cAlias )
AAdd( ::aOpened, cAlias )
AAdd( ::oAlias:aSlots, { cAlias, Upper( cTable ), Upper( cTable ), .T. } )
RECOVER
nWA := 0
END SEQUENCE
ENDIF
/* VIEW expansion: `<table>.fsv` holds the SELECT text the
* caller registered with CREATE VIEW. Run that SELECT once,
* materialize into a MEMRDD temp area, and route subsequent
* column lookups through it under the view's alias. Allows
* reusable named queries without forcing every caller to
* embed the full SQL. */
IF nWA == 0 .AND. hb_FileExists( Lower( cTable ) + ".fsv" )
IF SqlMaterializeView( cTable, cAlias )
nWA := Select( cAlias )
IF nWA > 0
AAdd( ::aOpened, cAlias )
AAdd( ::oAlias:aSlots, ;
{ cAlias, Upper( cTable ), Upper( cTable ), .T. } )
ENDIF
ENDIF
ENDIF
ENDIF
IF nWA == -1
::nDepth--
IF Len( aSavedAreas ) > 0
dbSelectArea( aSavedAreas[ 1 ] )
ENDIF
RETURN ::MakeError( SQL_ERR_LOCKED, ;
"Table '" + cTable + "' is open EXCLUSIVE. " + ;
"Close it or reopen with SHARED access before running SQL queries." )
ENDIF
/* nWA == 0 here means: not in workareas, not openable as DBF,
* not findable as CTE temp. Bubble up a clear error instead
* of silently dropping into a SELECT against no workarea
* (which previously yielded empty / undefined results and
* crashed downstream callers reading aR[1][1]). */
IF nWA == 0
::nDepth--
IF Len( aSavedAreas ) > 0
dbSelectArea( aSavedAreas[ 1 ] )
ENDIF
RETURN ::MakeError( SQL_ERR_NO_TABLE, ;
"Table '" + cTable + "' does not exist" )
ENDIF
ENDIF
NEXT
/* Synchronize join aliases with the aTables entries that were
* potentially renamed by the alias manager above. */
FOR i := 1 TO Len( aJoins )
IF Empty( aJoins[ i ][ 3 ] )
aJoins[ i ][ 3 ] := aJoins[ i ][ 2 ]
ENDIF
/* Find matching aTables entry and adopt its (possibly renamed) alias */
FOR j := 1 TO Len( ::aTables )
IF Upper( ::aTables[ j ][ 1 ] ) == Upper( aJoins[ i ][ 2 ] ) .AND. ;
( Upper( ::aTables[ j ][ 3 ] ) == Upper( aJoins[ i ][ 3 ] ) .OR. ;
::aTables[ j ][ 2 ] == aJoins[ i ][ 3 ] .OR. ;
Upper( ::aTables[ j ][ 1 ] ) == Upper( aJoins[ i ][ 3 ] ) )
aJoins[ i ][ 3 ] := ::aTables[ j ][ 2 ]
EXIT
ENDIF
NEXT
NEXT
/* Build result column names and expression trees */
aResultExprs := {}
FOR i := 1 TO Len( aCols )
xExpr := aCols[ i ][ 1 ]
cColAlias := aCols[ i ][ 2 ]
IF Empty( cColAlias )
cColAlias := SqlExprName( xExpr )
ENDIF
AAdd( aResultExprs, { xExpr, cColAlias } )
/* Expand SELECT * — iterate ALL tables (primary + joined) */
IF xExpr[ 1 ] == ND_COL .AND. xExpr[ 2 ] == "*"
aResultExprs := {}
aFieldNames := {}
FOR k := 1 TO Len( ::aTables )
cAlias := ::aTables[ k ][ 2 ]
IF Empty( cAlias )
cAlias := ::aTables[ k ][ 1 ]
ENDIF
nWA := Select( cAlias )
IF nWA > 0
dbSelectArea( nWA )
FOR j := 1 TO FCount()
cFN := Upper( AllTrim( FieldName( j ) ) )
AAdd( aResultExprs, { SqlNode( ND_COL, cFN, NIL, NIL, NIL ), cFN } )
NEXT
ENDIF
NEXT
EXIT
ENDIF
NEXT
/* Snapshot the user-visible column count BEFORE the hidden-column
* loops run. Hidden columns added below for aggregate sources,
* HAVING, and ORDER BY wrapped expressions get trimmed back off
* the final result so callers don't see synthetic `__ord_<i>__`
* etc. in their fetched rows. (nUserCols is declared at the top
* of RunSelect via the master LOCAL list — mid-function LOCAL
* is an established Five compiler quirk.) */
nUserCols := Len( aResultExprs )
/* Add hidden columns for aggregate source fields.
*
* Window aggregates (ND_WINDOW: SUM/AVG/COUNT/MIN/MAX OVER ...) are
* included explicitly because SqlExprHasAgg deliberately does not
* descend into ND_WINDOW (it carries its own aggregation scope),
* so without the extra top-level check queries like
* `SELECT id, SUM(v) OVER (…)` would never materialise `v` on
* each row and ApplyWindowFunctions' fast path
* (`SqlWindowSlideAgg`) read NIL / 0 for every slot. */
FOR i := 1 TO Len( aCols )
IF SqlExprHasAgg( aCols[ i ][ 1 ] ) .OR. aCols[ i ][ 1 ][ 1 ] == ND_WINDOW
/* Wrapped aggregate (e.g. `MAX(id)+1`, `ROUND(AVG(p),2)`):
* the top-level node is ND_BIN / ND_UNI / non-agg ND_FN, so
* the argument-walker below would skip it entirely and the
* wrapped aggregate's source column never becomes a hidden
* column. Result: ComputeAgg sees nCol==0 and returns 0
* silently. Walk the whole expression for ND_COL leaves and
* add them as hidden columns. */
IF aCols[ i ][ 1 ][ 1 ] != ND_FN .AND. aCols[ i ][ 1 ][ 1 ] != ND_WINDOW
aLeafCols := SqlCollectColExprs( aCols[ i ][ 1 ], NIL )
FOR k := 1 TO Len( aLeafCols )
cBare := aLeafCols[ k ][ 2 ]
lFound := .F.
FOR j := 1 TO Len( aResultExprs )
IF Upper( aResultExprs[ j ][ 2 ] ) == Upper( cBare )
lFound := .T.
EXIT
ENDIF
NEXT
IF ! lFound
AAdd( aResultExprs, { aLeafCols[ k ], cBare } )
ENDIF
NEXT
LOOP
ENDIF
IF ( aCols[ i ][ 1 ][ 1 ] == ND_FN .OR. aCols[ i ][ 1 ][ 1 ] == ND_WINDOW ) .AND. ;
Len( aCols[ i ][ 1 ][ 3 ] ) > 0
xArgExpr := aCols[ i ][ 1 ][ 3 ][ 1 ]
IF xArgExpr[ 1 ] == ND_COL .AND. xArgExpr[ 2 ] != "*"
/* Use the FULL qualified name (e.g. "o.amount") so
* FetchRow → FindWA resolves to the right workarea
* in JOIN contexts. Bare "amount" would fall through
* to aTables[1] which may be a different table. */
cBare := xArgExpr[ 2 ]
lFound := .F.
FOR j := 1 TO Len( aResultExprs )
IF Upper( aResultExprs[ j ][ 2 ] ) == Upper( cBare )
lFound := .T.
EXIT
ENDIF
NEXT
IF ! lFound
AAdd( aResultExprs, { xArgExpr, cBare } )
ENDIF
ELSEIF xArgExpr[ 1 ] != ND_COL
/* Complex expression (CASE, BIN, etc.) inside aggregate:
* collect the original ND_COL leaf nodes and add them as
* hidden result columns so they appear in fetched rows.
* Must preserve the qualified name (e.g. "o.qty") so
* subqueries with JOINs resolve to the right workarea.
* Using bare names here used to send `price` to ord in
* a `FROM ord o JOIN prod p` query, silently yielding
* NIL/wrong row data. */
aLeafCols := SqlCollectColExprs( xArgExpr, NIL )
FOR k := 1 TO Len( aLeafCols )
cBare := aLeafCols[ k ][ 2 ]
lFound := .F.
FOR j := 1 TO Len( aResultExprs )
IF Upper( aResultExprs[ j ][ 2 ] ) == Upper( cBare )
lFound := .T.
EXIT
ENDIF
NEXT
IF ! lFound
AAdd( aResultExprs, { aLeafCols[ k ], cBare } )
ENDIF
NEXT
ENDIF
ENDIF
ENDIF
NEXT
/* Wrapped window function (`SUM(x) OVER () + 100`): the top-level
* node is ND_BIN/ND_UNI/ND_FN(non-agg)/ND_CASE wrapping ND_WINDOW.
* ApplyWindowFunctions only scans aCols[i] when its top-level node
* IS ND_WINDOW, so a wrapped window expression evaluates to the
* placeholder 0 returned by EvalExpr's ND_WINDOW branch. Walk every
* SELECT projection, extract any ND_WINDOW into a hidden column
* (`__win_<i>_<j>__`) and substitute its position with a plain
* ND_COL pointing at that hidden column. ApplyWindowFunctions then
* computes the inner window per row, projection picks up the value
* via the ND_COL lookup, and the outer arithmetic falls out of the
* normal ND_BIN/ND_UNI evaluator. Hidden columns get trimmed back
* off the result via nUserCols at RunSelect's tail. */
::aWrappedWindowCols := {}
FOR i := 1 TO Len( aCols )
IF aCols[ i ][ 1 ] != NIL .AND. ValType( aCols[ i ][ 1 ] ) == "A" .AND. ;
aCols[ i ][ 1 ][ 1 ] != ND_WINDOW
aLeafCols := {} /* reuse var: holds {windowExpr, alias} pairs */
aCols[ i ][ 1 ] := SqlExtractWindow( aCols[ i ][ 1 ], aLeafCols, ;
"__win_" + AllTrim( hb_NToS( i ) ) )
IF Len( aLeafCols ) > 0
/* Track this column for post-window re-evaluation. The
* fetch loop runs BEFORE ApplyWindowFunctions, so the
* outer expression sees NIL/0 in the hidden window slot
* and folds it through the wrapper as NIL. After
* ApplyWindowFunctions fills the hidden slot, we re-eval
* the outer expression per row to pick up the real value. */
AAdd( ::aWrappedWindowCols, i )
FOR k := 1 TO Len( aLeafCols )
AAdd( aResultExprs, { aLeafCols[ k ][ 1 ], aLeafCols[ k ][ 2 ] } )
AAdd( aCols, { aLeafCols[ k ][ 1 ], aLeafCols[ k ][ 2 ] } )
NEXT
ENDIF
ENDIF
NEXT
/* Hidden columns for HAVING expressions: any aggregate source
* column referenced only inside HAVING (and not in SELECT) must
* still be carried through so EvalHavingExpr → ComputeAgg can
* resolve its argument. Without this, `SELECT dept FROM s
* GROUP BY dept HAVING SUM(amt) > 200` returned 0 rows because
* `amt` was never materialised on grouped rows and ComputeAgg
* fell back to nCol=0 → 0. Same SqlCollectColExprs walk as
* the wrapped-aggregate branch above; collects every ND_COL
* leaf reachable from xHaving. */
IF xHaving != NIL
aLeafCols := SqlCollectColExprs( xHaving, NIL )
FOR k := 1 TO Len( aLeafCols )
cBare := aLeafCols[ k ][ 2 ]
lFound := .F.
FOR j := 1 TO Len( aResultExprs )
IF Upper( aResultExprs[ j ][ 2 ] ) == Upper( cBare )
lFound := .T.
EXIT
ENDIF
NEXT
IF ! lFound
AAdd( aResultExprs, { aLeafCols[ k ], cBare } )
ENDIF
NEXT
ENDIF
/* Hidden columns for ORDER BY expressions not already in the
* SELECT list. Without this, ORDER BY references a column that
* the materialised rows don't carry — TryBuildSortSpec returns
* NIL (col not in aFieldNames), PRG OrderBy likewise can't bind
* it, and the result ends up in undefined order (typically
* insertion order). Pattern mirrors the aggregate hidden-column
* loop above so ORDER BY over non-projected fields works out of
* the box. */
IF ValType( aOrderBy ) == "A"
FOR i := 1 TO Len( aOrderBy )
IF ValType( aOrderBy[ i ] ) != "A" .OR. Len( aOrderBy[ i ] ) == 0
LOOP
ENDIF
xExpr := aOrderBy[ i ][ 1 ]
IF ValType( xExpr ) != "A" .OR. Len( xExpr ) < 2
LOOP
ENDIF
/* Wrapped ORDER BY expression (ND_BIN / ND_UNI / ND_FN —
* e.g. `ORDER BY MAX(amt) + 1 DESC`): without a hidden
* column the sort layer can't bind the expression and rows
* stay in insertion order. Append the whole expression to
* aResultExprs under a synthetic alias `__ord_<i>__` and
* rewrite the ORDER BY entry in-place to a plain ND_COL
* pointing at that alias. GroupBy / projection then computes
* the wrapped aggregate per row, and TSqlSort:OrderBy can
* find the column by name. The hidden columns get trimmed
* back off the final result via ::nUserCols (set just below
* after this whole hidden-col block runs). */
IF xExpr[ 1 ] != ND_COL
cBare := "__ord_" + AllTrim( hb_NToS( i ) ) + "__"
AAdd( aResultExprs, { xExpr, cBare } )
/* GroupBy / aFieldNames rebuild downstream both walk
* `aCols` (the user-visible SELECT list), not aResultExprs.
* Without mirroring the hidden append into aCols, GroupBy
* never evaluates the wrapped expression and the new
* "__ord_<i>__" name doesn't appear in aFieldNames →
* SqlFindColIdx returns 0 → sort silently no-ops. The
* trim block at RunSelect's tail still strips the column
* back off the result, so callers see the same shape. */
AAdd( aCols, { xExpr, cBare } )
aOrderBy[ i ][ 1 ] := { ND_COL, cBare, NIL, NIL, NIL }
LOOP
ENDIF
cBare := xExpr[ 2 ]
IF cBare == "*"
LOOP
ENDIF
lFound := .F.
FOR j := 1 TO Len( aResultExprs )
IF Upper( aResultExprs[ j ][ 2 ] ) == Upper( cBare ) .OR. ;
( ValType( aResultExprs[ j ][ 1 ] ) == "A" .AND. ;
Len( aResultExprs[ j ][ 1 ] ) >= 2 .AND. ;
aResultExprs[ j ][ 1 ][ 1 ] == ND_COL .AND. ;
Upper( aResultExprs[ j ][ 1 ][ 2 ] ) == Upper( cBare ) )
lFound := .T.
EXIT
ENDIF
NEXT
IF ! lFound
AAdd( aResultExprs, { xExpr, cBare } )
ENDIF
NEXT
ENDIF
FOR i := 1 TO Len( aResultExprs )
AAdd( aFieldNames, aResultExprs[ i ][ 2 ] )
NEXT
/* Constant folding */
IF xWhere != NIL
xWhere := SqlFoldConst( xWhere )
ENDIF
FOR i := 1 TO Len( aResultExprs )
aResultExprs[ i ][ 1 ] := SqlFoldConst( aResultExprs[ i ][ 1 ] )
NEXT
/* Pre-resolve column references to {nWA, nFPos} pairs so the
* per-row EvalExpr hot path avoids repeated At/Upper/FindWA/
* FieldPos work on every ND_COL visit. Scope: WHERE, HAVING,
* every JOIN's ON condition. Projection columns go through
* FetchRow's own cache and don't need this pre-walk. */
IF xWhere != NIL
::PreResolveColumns( xWhere )
ENDIF
IF xHaving != NIL
::PreResolveColumns( xHaving )
ENDIF
FOR i := 1 TO Len( aJoins )
IF aJoins[ i ][ 4 ] != NIL
::PreResolveColumns( aJoins[ i ][ 4 ] )
ENDIF
NEXT
SET DELETED ON
/* SELECT without FROM */
IF Len( ::aTables ) == 0
aRow := ::FetchRow( aResultExprs )
AAdd( aRows, aRow )
ENDIF
/* Scan primary table */
IF Len( ::aTables ) > 0
cAlias := ::aTables[ 1 ][ 2 ]
IF Empty( cAlias )
cAlias := ::aTables[ 1 ][ 1 ]
ENDIF
nWA := Select( cAlias )
IF nWA > 0
dbSelectArea( nWA )
lIndexUsed := .F.
/* Hand the current executor to TSqlIndex so its per-row
* seek loop can skip the SqlEvalExprNode/SqlFetchRowArr
* throwaway-executor allocations. */
::oIndex:oExec := Self
/* Resolve LIMIT / ORDER-BY-from-index BEFORE the index-scan
* dispatch so TryIndexScan can early-terminate, and the
* Go-fast-path below can reuse the same lOrderFromIndex
* value to skip the post-scan sort. Gates mirror the
* original fallback computation — single-table, no GROUP /
* Agg / DISTINCT, LIMIT or TOP present. */
lOrderFromIndex := .F.
nEarlyLimit := 0
IF Len( aJoins ) == 0 .AND. Len( aGroupBy ) == 0 .AND. ;
! ::oAgg:HasAgg( aCols ) .AND. ! lDistinct .AND. ;
( ( ValType( nLimit ) == "N" .AND. nLimit > 0 ) .OR. ;
( ValType( nTop ) == "N" .AND. nTop > 0 ) )
IF Len( aOrderBy ) > 0
lOrderFromIndex := ::oIndex:MatchOrderByTag( nWA, aOrderBy, aFieldNames )
ENDIF
IF Len( aOrderBy ) == 0 .OR. lOrderFromIndex
nEarlyLimit := iif( ValType( nLimit ) == "N" .AND. nLimit > 0, ;
nLimit, nTop )
/* OFFSET is post-processed; pull enough rows to
* satisfy LIMIT *after* the OFFSET skip. */
IF nOffset > 0
nEarlyLimit += nOffset
ENDIF
ENDIF
ENDIF
IF Len( aJoins ) == 0 .AND. xWhere != NIL
lIndexUsed := ::oIndex:TryIndexScan( nWA, xWhere, xWhere, ;
::aTables, ::aParams, aResultExprs, @aRows, nEarlyLimit )
ELSEIF Len( aJoins ) > 0 .AND. xWhere != NIL
lIndexUsed := ::oIndex:TryIndexJoinScan( nWA, xWhere, ;
::aTables, ::aParams, aResultExprs, @aRows, aJoins )
ENDIF
::oIndex:oExec := NIL
IF ! lIndexUsed
dbSelectArea( nWA )
dbGoTop()
hJoinHash := { => }
/* === GO NATIVE JOIN FAST PATH ===
* Multi-table equi-join with all SELECT columns being plain
* field refs → hand the entire join to Go's SqlHashJoin.
* Bypasses per-row PRG JoinRecurse/FetchRow/dbSelectArea. */
IF Len( aJoins ) > 0 .AND. xWhere == NIL .AND. aGoRows == NIL
aGoRows := ::TryGoJoin( aJoins, aResultExprs, nWA )
IF aGoRows != NIL
FOR i := 1 TO Len( aGoRows )
AAdd( aRows, aGoRows[ i ] )
NEXT
ENDIF
ENDIF
/* === GO NATIVE FAST PATH ===
* Single-table, no joins, no aggregates, all SELECT exprs
* simple field refs, WHERE is NIL or compilable to pcode.
* Two variants share the same entry conditions:
* - With row block (::bRowBlock != NIL): SqlEach streams
* rows directly into the user block, no intermediate
* array. Beats raw RDD on end-to-end timing.
* - Without block: SqlScan materializes into aRows as
* usual (compat with existing callers).
*/
aFP := NIL
pcW := NIL
aGoRows := NIL
/* lOrderFromIndex / nEarlyLimit were resolved above,
* before the TryIndexScan dispatch, so both index-scan
* and Go-fast-path branches share the same values. The
* call there may have moved the record pointer via
* ordSetFocus+dbSeek; if lOrderFromIndex is set, we
* re-anchor to the logical top of the focused tag so
* SqlScan's GoTop+Skip loop walks from the first
* ordered row. */
IF lOrderFromIndex
dbSelectArea( nWA )
dbGoTop()
ENDIF
IF Len( aJoins ) == 0 .AND. Len( aGroupBy ) == 0 .AND. ;
! ::oAgg:HasAgg( aCols )
/* Plan pcode cache: cache aFP + pcW per cCacheKey.
* These results are pure functions of the plan tree
* (which is immutable between cache hits) and the
* target table schema (stable for the process). */
LOCAL hSelCached, cSelKey
IF ! Empty( ::cCacheKey )
cSelKey := ::cCacheKey + "#sel"
IF hb_HHasKey( ::oSession:hDmlPcodeCache, cSelKey )
hSelCached := ::oSession:hDmlPcodeCache[ cSelKey ]
aFP := hSelCached[ "fp" ]
pcW := hSelCached[ "where_pc" ]
ENDIF
ENDIF
IF aFP == NIL
aFP := ::TryBuildFieldPositions( aResultExprs )
IF aFP != NIL .AND. xWhere != NIL
pcW := ::TryCompileWhere( xWhere )
IF pcW == NIL
aFP := NIL /* WHERE couldn't compile — PRG path */
ENDIF
ENDIF
IF aFP != NIL .AND. ! Empty( ::cCacheKey )
IF Len( ::oSession:hDmlPcodeCache ) >= SQL_DML_PCODE_CACHE_MAX
::oSession:hDmlPcodeCache := { => }
ENDIF
::oSession:hDmlPcodeCache[ ::cCacheKey + "#sel" ] := { ;
"fp" => aFP, ;
"where_pc" => pcW }
ENDIF
ENDIF
IF aFP != NIL
IF xWhere == NIL .OR. pcW != NIL
IF ::bRowBlock != NIL
/* Block mode: stream rows through user block.
* No result array. Skip all post-processing
* (ORDER BY / LIMIT / window / DISTINCT) —
* those require a materialized set; callers
* using the block form opt into streaming
* semantics and handle shaping themselves. */
SqlEach( aFP, pcW, ::bRowBlock )
aGoRows := {} /* signal "handled" to skip fallback */
ELSE
aGoRows := SqlScan( aFP, pcW, nEarlyLimit )
FOR i := 1 TO Len( aGoRows )
AAdd( aRows, aGoRows[ i ] )
NEXT
ENDIF
ENDIF
ENDIF
ENDIF
/* Fallback: PRG interpreter loop */
IF aGoRows == NIL
/* Pre-bind SELECT columns to {nWA, nFPos} so FetchRow
* can skip the per-row string parse + FindWA on every
* join recursion. Huge win for multi-table scans. */
::aFetchCache := ::BuildFetchCache( aResultExprs )
dbSelectArea( nWA )
/* WHERE predicate pushdown: split the top-level AND
* chain into clauses and group them by the deepest
* JOIN level whose columns they reference. Clauses
* pinned to a middle level are evaluated inside
* JoinRecurse as soon as that level's row is
* positioned — rejected rows skip the exponential
* deeper-join expansion that would otherwise happen
* until xWhere fired at the base case. Clauses that
* can't be classified (unqualified columns, aliases
* we can't resolve) fall back to the residual
* xRes evaluated at the base case, preserving
* existing semantics. */
aPushByLevel := NIL
xRes := xWhere
IF xWhere != NIL .AND. Len( aJoins ) > 0
hAliasLvl := ::BuildAliasLevelMap( aJoins )
aClauses := ::SplitAndClauses( xWhere, NIL )
aPushByLevel := Array( Len( aJoins ) )
FOR ii := 1 TO Len( aPushByLevel )
aPushByLevel[ ii ] := {}
NEXT
xRes := NIL
FOR ii := 1 TO Len( aClauses )
nLvl := ::ClauseMaxLevel( aClauses[ ii ], hAliasLvl, Len( aJoins ) )
IF nLvl >= 1 .AND. nLvl < Len( aJoins )
AAdd( aPushByLevel[ nLvl ], aClauses[ ii ] )
ELSE
/* Level 0 (outer-only) and Len(aJoins)
* (needs all joins) stay in the residual
* evaluated at the base case — the
* existing behavior. */
IF xRes == NIL
xRes := aClauses[ ii ]
ELSE
xRes := SqlNode( ND_BIN, "AND", xRes, aClauses[ ii ], NIL )
ENDIF
ENDIF
NEXT
ENDIF
/* lOrderFromIndex / nEarlyLimit were resolved above,
* before the Go-fast-path decision, so both paths
* share the same ORDER-BY-from-index detection and
* row-count cap. */
WHILE ! Eof()
IF Len( aJoins ) > 0
::JoinRecurse( aJoins, 1, xRes, aResultExprs, @aRows, hJoinHash, aPushByLevel )
dbSelectArea( nWA )
ELSE
IF xWhere == NIL .OR. SqlIsTrue( ::EvalExpr( xWhere ) )
aRow := ::FetchRow( aResultExprs )
AAdd( aRows, aRow )
ENDIF
ENDIF
IF nEarlyLimit > 0 .AND. Len( aRows ) >= nEarlyLimit
EXIT
ENDIF
dbSelectArea( nWA )
dbSkip()
ENDDO
::aFetchCache := NIL
ENDIF
ENDIF
ENDIF
ENDIF
/* GROUP BY */
IF Len( aGroupBy ) > 0 .OR. ::oAgg:HasAgg( aCols )
aRows := ::oAgg:GroupBy( aRows, aFieldNames, aCols, aGroupBy, xHaving, ::aTables, ::aParams )
aFieldNames := {}
FOR i := 1 TO Len( aCols )
IF ! Empty( aCols[ i ][ 2 ] )
AAdd( aFieldNames, aCols[ i ][ 2 ] )
ELSE
AAdd( aFieldNames, SqlExprName( aCols[ i ][ 1 ] ) )
ENDIF
NEXT
ENDIF
/* Window functions */
::ApplyWindowFunctions( @aRows, aFieldNames, aCols )
/* Re-evaluate wrapped window expressions per row. ApplyWindowFunctions
* has now filled the hidden `__win_<i>_<j>__` slots with real values;
* the outer arithmetic / function call needs a second pass to pick up
* the post-window value. */
IF ::aWrappedWindowCols != NIL .AND. Len( ::aWrappedWindowCols ) > 0
FOR k := 1 TO Len( ::aWrappedWindowCols )
i := ::aWrappedWindowCols[ k ]
IF i > 0 .AND. i <= Len( aCols )
FOR j := 1 TO Len( aRows )
IF i <= Len( aRows[ j ] )
aRows[ j ][ i ] := SqlEvalRowExpr( aCols[ i ][ 1 ], aFieldNames, aRows[ j ] )
ENDIF
NEXT
ENDIF
NEXT
ENDIF
/* ORDER BY — try Go-native sort first (10-50x faster for large sets),
* fall back to PRG for complex expressions in ORDER BY.
*
* lOrderFromIndex set above (pre-scan) means we already walked the
* table in tag order so the result is sorted — short-circuit here
* to avoid a redundant MatchOrderByTag probe. */
IF Len( aOrderBy ) > 0 .AND. ! lOrderFromIndex
IF ! ( nWA > 0 .AND. ::oIndex:MatchOrderByTag( nWA, aOrderBy, aFieldNames ) )
LOCAL aSortSpec2 := ::TryBuildSortSpec( aOrderBy, aFieldNames )
IF aSortSpec2 != NIL .AND. Len( aRows ) > 0
aRows := SqlOrderBy( aRows, aSortSpec2 )
ELSE
aRows := ::oSort:OrderBy( aRows, aFieldNames, aOrderBy, ::aTables, ::aParams )
ENDIF
ENDIF
ENDIF
/* RIGHT JOIN second pass — must run before set operations and
* LIMIT so unmatched inner rows are included in the full result. */
IF Len( aJoins ) > 0
FOR i := 1 TO Len( aJoins )
IF Upper( aJoins[ i ][ 1 ] ) == "RIGHT" .OR. Upper( aJoins[ i ][ 1 ] ) == "FULL"
::RightJoinPass( aJoins, i, aResultExprs, @aRows )
ENDIF
NEXT
ENDIF
/* UNION / INTERSECT / EXCEPT — per SQL standard, set operations
* are applied to the full result of each SELECT before ORDER BY /
* DISTINCT / OFFSET / LIMIT. Previous order applied them last,
* which meant LIMIT clipped the first SELECT before UNION added
* the second's rows, producing more rows than intended. */
IF hUnion != NIL
aU := TSqlExecutor():New( hUnion, ::aParams, ::oSession ):Run()
/* SQL standard: set operations require the same column count on
* both sides. Previously a mismatch silently truncated the wider
* side to the narrower's width, masking schema bugs (`SELECT a
* UNION SELECT a, b` returned 1-col rows with `b` dropped). Bail
* out with a structured error instead. */
IF ValType( aU ) == "A" .AND. Len( aU ) >= 1 .AND. ;
ValType( aU[ 1 ] ) == "A" .AND. Len( aU[ 1 ] ) > 0 .AND. ;
aU[ 1 ][ 1 ] != "__error__" .AND. ;
Len( aU[ 1 ] ) != Len( aFieldNames )
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"UNION/INTERSECT/EXCEPT: each query must have the same number of columns (" + ;
hb_NToS( Len( aFieldNames ) ) + " vs " + hb_NToS( Len( aU[ 1 ] ) ) + ")" )
ENDIF
IF hb_HHasKey( hUnion, "set_op" )
IF hUnion[ "set_op" ] == "INTERSECT"
aRows := SqlDoIntersect( aRows, aU[ 2 ] )
ELSEIF hUnion[ "set_op" ] == "EXCEPT"
aRows := SqlDoExcept( aRows, aU[ 2 ] )
ENDIF
ELSE
lAll := .F.
IF hb_HHasKey( hUnion, "union_all" )
lAll := hUnion[ "union_all" ]
ENDIF
IF lAll
/* UNION ALL — plain append, no dedup. */
FOR i := 1 TO Len( aU[ 2 ] )
AAdd( aRows, aU[ 2 ][ i ] )
NEXT
ELSE
/* Streaming DISTINCT: build one hash set across both
* sides in the Go RTL. Saves the append-then-rescan pass
* the old path did (materialise merged array + run
* SqlDistinct, two walks over |L|+|R| rows). */
aRows := SqlUnionDistinct( aRows, aU[ 2 ] )
ENDIF
ENDIF
ENDIF
/* Trim hidden columns BEFORE DISTINCT so the dedup hash sees only
* user-visible columns. ORDER BY has already used the hidden cols
* (aggregate sources, `__ord_<i>__` for wrapped expressions, etc.)
* so they're free to drop. Without trimming first, `SELECT
* DISTINCT grp ORDER BY id` returned every original row because the
* synthetic `__ord_1__` column made each row uniquely keyed. */
IF nUserCols != NIL .AND. nUserCols > 0 .AND. ;
Len( aFieldNames ) > nUserCols
nTrim := nUserCols
FOR nRow := 1 TO Len( aRows )
IF Len( aRows[ nRow ] ) > nTrim
ASize( aRows[ nRow ], nTrim )
ENDIF
NEXT
ASize( aFieldNames, nTrim )
ENDIF
/* DISTINCT */
IF lDistinct
aRows := ::oSort:Distinct( aRows )
ENDIF
/* OFFSET */
IF nOffset > 0 .AND. nOffset < Len( aRows )
aTmp := {}
FOR i := nOffset + 1 TO Len( aRows )
AAdd( aTmp, aRows[ i ] )
NEXT
aRows := aTmp
ELSEIF nOffset >= Len( aRows )
aRows := {}
ENDIF
/* TOP / LIMIT.
* SQL semantics:
* LIMIT 0 → empty result
* LIMIT n>0 → first n rows
* LIMIT n<0 → ill-defined; clamp to 0 (matches SQLite tolerance)
*
* Old code only clipped when `nLimit > 0`, so LIMIT 0 returned
* the full result and LIMIT -1 was silently a no-op. */
nMaxRows := 0
IF ValType( nTop ) == "N" .AND. nTop > 0
nMaxRows := nTop
ENDIF
IF ValType( nLimit ) == "N"
IF nLimit > 0
nMaxRows := nLimit
ELSEIF nLimit <= 0
/* Explicit zero / negative → return no rows. */
aRows := {}
nMaxRows := 0
ENDIF
ENDIF
IF nMaxRows > 0 .AND. Len( aRows ) > nMaxRows
ASize( aRows, nMaxRows )
ENDIF
/* Close opened tables */
::CloseOpened()
/* Clean up CTE temp DBF files */
IF hb_HHasKey( ::hQuery, "cte" ) .AND. ValType( ::hQuery[ "cte" ] ) == "A"
FOR i := 1 TO Len( ::hQuery[ "cte" ] )
cTable := Upper( ::hQuery[ "cte" ][ i ][ 1 ] )
/* Close the CTE name alias workarea if still open */
nWA := Select( cTable )
IF nWA > 0
dbSelectArea( nWA )
dbCloseArea()
ENDIF
/* Legacy disk fallback cleanup — only runs when a __cte_*.dbf
* has actually been seen (either from a prior crash or a
* MEMRDD-failure legacy open). MEMRDD-only runs skip the stat. */
IF ::oSession:lCteDiskSeen
cTable := "__cte_" + Lower( ::hQuery[ "cte" ][ i ][ 1 ] )
IF hb_FileExists( cTable + ".dbf" )
FErase( cTable + ".dbf" )
ENDIF
ENDIF
NEXT
ENDIF
/* Clean up VIEW temp files — created by TSqlIndex:CheckView when
* a query references a .fsv view. The flag lets us skip the stat
* loop on view-free queries, which the profile showed as ~28% of
* SELECT CPU after the WA cache killed the munmap cost. */
IF ::oIndex:lViewUsed
FOR i := 1 TO Len( ::aTables )
IF hb_FileExists( "__view_" + Lower( ::aTables[ i ][ 1 ] ) + ".dbf" )
FErase( "__view_" + Lower( ::aTables[ i ][ 1 ] ) + ".dbf" )
ENDIF
NEXT
::oIndex:lViewUsed := .F.
ENDIF
::nDepth--
IF Len( aSavedAreas ) > 0
dbSelectArea( aSavedAreas[ 1 ] )
ENDIF
/* Block-callback mode: rows were streamed through ::bRowBlock during
* the fast-path scan. aRows is empty; we return NIL to signal
* streaming semantics to the caller. */
IF ::bRowBlock != NIL
RETURN NIL
ENDIF
/* Trim hidden columns added during the projection-rewrite phase
* (aggregate sources, HAVING leaves, ORDER BY wrapped expressions).
* Without trimming, callers see synthetic `__ord_<i>__` slots in
* their fetched rows and the leading aggregate-source columns. */
IF nUserCols != NIL .AND. nUserCols > 0 .AND. ;
Len( aFieldNames ) > nUserCols
nTrim := nUserCols
FOR nRow := 1 TO Len( aRows )
IF Len( aRows[ nRow ] ) > nTrim
ASize( aRows[ nRow ], nTrim )
ENDIF
NEXT
ASize( aFieldNames, nTrim )
ENDIF
RETURN { aFieldNames, aRows }
/* Hash join: build hash table from inner table, probe with outer key */
METHOD HashJoin( nInnerWA, cInnerField, cOuterCol, xOnCond, aJoins, nIdx, xWhere, aRE, aRows, hHashTbl, aPushByLevel ) CLASS TSqlExecutor
LOCAL cHashKey, aMatches, xOuterVal, xInnerVal, cValKey
LOCAL nFPos, nSavedRec, i, lHadMatch
LOCAL lCompound, cHJRMKey
lHadMatch := .F.
/* Build hash table once per join (keyed by join index).
* Delegates to the Go-native SqlHashBuild RTL which scans the
* inner workarea and returns the populated hash in one pass —
* roughly 40x faster than the PRG hash-build loop because it
* avoids per-row class dispatch, hb_HHasKey, and AAdd growth. */
cHashKey := "HJ_" + hb_ntos( nIdx ) + "_" + cInnerField
IF ! hb_HHasKey( hHashTbl, cHashKey )
dbSelectArea( nInnerWA )
nFPos := FieldPos( cInnerField )
IF nFPos > 0
hHashTbl[ cHashKey ] := SqlHashBuild( nFPos )
ELSE
hHashTbl[ cHashKey ] := { => }
ENDIF
ENDIF
/* Probe hash with outer row join key value */
xOuterVal := ::EvalExpr( SqlNode( ND_COL, cOuterCol, NIL, NIL, NIL ) )
cValKey := SqlValToStr( xOuterVal )
IF hb_HHasKey( hHashTbl[ cHashKey ], cValKey )
aMatches := hHashTbl[ cHashKey ][ cValKey ]
/* If xOnCond is a compound AND (not a bare equi-term), re-evaluate
* the full condition after the hash probe to filter out partial
* matches. xOnCond[2] == "=" indicates a bare equi-join where the
* hash match is sufficient. */
lCompound := ( xOnCond != NIL .AND. xOnCond[ 1 ] == ND_BIN .AND. xOnCond[ 2 ] != "=" )
/* Base-case inline: if the next recursion would just be FetchRow,
* skip the method-dispatch overhead and build the row directly.
* 50k inner matches × Send() dispatch was the 3-way join bottleneck. */
/* Track inner matched RecNos for RIGHT JOIN pass */
cHJRMKey := "__RIGHT_" + Upper( Alias( nInnerWA ) )
IF nIdx + 1 > Len( aJoins )
FOR i := 1 TO Len( aMatches )
dbSelectArea( nInnerWA )
dbGoto( aMatches[ i ] )
IF lCompound .AND. ! SqlIsTrue( ::EvalExpr( xOnCond ) )
LOOP
ENDIF
/* Pushdown: if any clause pinned to this level rejects
* the row, skip it before building the result tuple. */
IF ! ::EvalPushedAtLevel( aPushByLevel, nIdx )
LOOP
ENDIF
lHadMatch := .T.
IF ! hb_HHasKey( ::hRightMatched, cHJRMKey )
::hRightMatched[ cHJRMKey ] := { => }
ENDIF
::hRightMatched[ cHJRMKey ][ aMatches[ i ] ] := .T.
IF xWhere == NIL .OR. SqlIsTrue( ::EvalExpr( xWhere ) )
AAdd( aRows, ::FetchRow( aRE ) )
ENDIF
NEXT
ELSE
FOR i := 1 TO Len( aMatches )
dbSelectArea( nInnerWA )
dbGoto( aMatches[ i ] )
IF lCompound .AND. ! SqlIsTrue( ::EvalExpr( xOnCond ) )
LOOP
ENDIF
lHadMatch := .T.
IF ! hb_HHasKey( ::hRightMatched, cHJRMKey )
::hRightMatched[ cHJRMKey ] := { => }
ENDIF
::hRightMatched[ cHJRMKey ][ aMatches[ i ] ] := .T.
IF ::EvalPushedAtLevel( aPushByLevel, nIdx )
::JoinRecurse( aJoins, nIdx + 1, xWhere, aRE, @aRows, hHashTbl, aPushByLevel )
ENDIF
NEXT
ENDIF
ENDIF
RETURN lHadMatch
/* Subquery result cache for non-correlated subqueries */
/* ExistsViaSemiJoin — try to answer an EXISTS / NOT EXISTS probe via
* a pre-built hash set instead of re-executing the subquery per outer
* row. Returns a boolean (the EXISTS result) on success, NIL when the
* subquery shape can't be lifted (caller should fall back to the
* normal per-row path).
*
* The lifted state is built on first call by TryBuildSemiJoin and
* cached in ::aSemiJoinSlots keyed on xSubNode identity. The sentinel
* string "NO" marks subqueries we already tried and can't lift, so
* subsequent calls skip the analysis.
*/
METHOD ExistsViaSemiJoin( xSubNode, lNegate ) CLASS TSqlExecutor
LOCAL i, nSlot, oData, xOuterVal, cKey, lMatch
/* Look up previous analysis */
nSlot := 0
FOR i := 1 TO Len( ::aSemiJoinSlots )
IF ::aSemiJoinSlots[ i ][ 1 ] == xSubNode
nSlot := i
EXIT
ENDIF
NEXT
IF nSlot == 0
oData := ::TryBuildSemiJoin( xSubNode )
AAdd( ::aSemiJoinSlots, { xSubNode, iif( oData == NIL, "NO", oData ) } )
nSlot := Len( ::aSemiJoinSlots )
ENDIF
oData := ::aSemiJoinSlots[ nSlot ][ 2 ]
/* Shape couldn't be lifted — let caller use fallback */
IF ValType( oData ) != "H"
RETURN NIL
ENDIF
/* Probe: evaluate outer column reference and look up in hash set */
xOuterVal := ::Resolve( oData[ "outer_ref" ] )
cKey := SqlValToStr( xOuterVal )
lMatch := hb_HHasKey( oData[ "inner_set" ], cKey )
IF lNegate
RETURN ! lMatch
ENDIF
RETURN lMatch
/* TryBuildSemiJoin — attempt to lift a correlated EXISTS subquery into
* a non-correlated hash set. Returns a hash { "outer_ref", "inner_set" }
* on success, NIL if the subquery doesn't match the supported shape.
*
* Supported shape:
* SELECT ... FROM inner_table WHERE inner.col = outer.col [AND rest]
* with no JOIN, no GROUP BY / HAVING, no ORDER BY. The `rest` can be
* any non-correlated predicate; it stays in the lifted subquery.
*
* Implementation:
* 1. Walk the WHERE tree as an AND list, find the first equi-term
* whose two sides are `innerTable.col` and `outerAlias.col`.
* Rebuild the remainder predicate from everything else.
* 2. Synthesize a modified hQuery: same FROM, DISTINCT inner.col as
* the only SELECT column, WHERE = remainder.
* 3. Run it once via a nested TSqlExecutor. Build a hash set keyed
* on SqlValToStr(innerCol).
*/
METHOD TryBuildSemiJoin( xSubNode ) CLASS TSqlExecutor
LOCAL hQ, aLocalAliases, i, aT
LOCAL aAndTerms, xTerm, xLeft, xRight
LOCAL lLeftIsInner, lRightIsInner
LOCAL cInnerAlias, cInnerField, xOuterRef
LOCAL aRemainTerms, xRemain
LOCAL hLifted, oSub, aResult, hSet, cKey
LOCAL xVal, aTopWhere
IF xSubNode == NIL .OR. ValType( xSubNode ) != "A" .OR. Len( xSubNode ) < 2
RETURN NIL
ENDIF
hQ := xSubNode[ 2 ]
IF ValType( hQ ) != "H"
RETURN NIL
ENDIF
/* Shape constraints — fall back for anything complex */
IF hb_HHasKey( hQ, "joins" ) .AND. ValType( hQ[ "joins" ] ) == "A" .AND. Len( hQ[ "joins" ] ) > 0
RETURN NIL
ENDIF
IF hb_HHasKey( hQ, "group_by" ) .AND. ValType( hQ[ "group_by" ] ) == "A" .AND. Len( hQ[ "group_by" ] ) > 0
RETURN NIL
ENDIF
IF hb_HHasKey( hQ, "having" ) .AND. hQ[ "having" ] != NIL
RETURN NIL
ENDIF
IF hb_HHasKey( hQ, "union" ) .AND. hQ[ "union" ] != NIL
RETURN NIL
ENDIF
IF ! hb_HHasKey( hQ, "tables" ) .OR. Len( hQ[ "tables" ] ) != 1
RETURN NIL
ENDIF
IF ! hb_HHasKey( hQ, "where" ) .OR. hQ[ "where" ] == NIL
RETURN NIL
ENDIF
/* Collect subquery's own table aliases to tell inner from outer */
aLocalAliases := {}
aT := hQ[ "tables" ][ 1 ]
AAdd( aLocalAliases, Upper( aT[ 1 ] ) )
IF Len( aT ) >= 2 .AND. ! Empty( aT[ 2 ] )
AAdd( aLocalAliases, Upper( aT[ 2 ] ) )
ENDIF
/* Flatten WHERE into a list of AND-terms */
aAndTerms := {}
aTopWhere := { hQ[ "where" ] }
WHILE Len( aTopWhere ) > 0
xTerm := aTopWhere[ 1 ]
ADel( aTopWhere, 1 )
ASize( aTopWhere, Len( aTopWhere ) - 1 )
IF xTerm != NIL .AND. ValType( xTerm ) == "A" .AND. Len( xTerm ) >= 4 .AND. ;
xTerm[ 1 ] == ND_BIN .AND. xTerm[ 2 ] == "AND"
AAdd( aTopWhere, xTerm[ 3 ] )
AAdd( aTopWhere, xTerm[ 4 ] )
ELSE
AAdd( aAndTerms, xTerm )
ENDIF
ENDDO
/* Find the equi-term that correlates inner.col with outer.col */
cInnerAlias := ""
cInnerField := ""
xOuterRef := NIL
aRemainTerms := {}
FOR i := 1 TO Len( aAndTerms )
xTerm := aAndTerms[ i ]
IF ! Empty( cInnerField ) .OR. ;
xTerm == NIL .OR. ValType( xTerm ) != "A" .OR. Len( xTerm ) < 4 .OR. ;
xTerm[ 1 ] != ND_BIN .OR. xTerm[ 2 ] != "=" .OR. ;
xTerm[ 3 ] == NIL .OR. xTerm[ 4 ] == NIL .OR. ;
xTerm[ 3 ][ 1 ] != ND_COL .OR. xTerm[ 4 ][ 1 ] != ND_COL
AAdd( aRemainTerms, xTerm )
LOOP
ENDIF
xLeft := xTerm[ 3 ]
xRight := xTerm[ 4 ]
lLeftIsInner := SemiJoinIsInner( xLeft, aLocalAliases )
lRightIsInner := SemiJoinIsInner( xRight, aLocalAliases )
IF lLeftIsInner .AND. ! lRightIsInner
cInnerField := SemiJoinStripAlias( xLeft[ 2 ] )
xOuterRef := xRight[ 2 ]
ELSEIF lRightIsInner .AND. ! lLeftIsInner
cInnerField := SemiJoinStripAlias( xRight[ 2 ] )
xOuterRef := xLeft[ 2 ]
ELSE
AAdd( aRemainTerms, xTerm )
ENDIF
NEXT
IF Empty( cInnerField ) .OR. xOuterRef == NIL
RETURN NIL
ENDIF
/* The remainder must be entirely non-correlated — otherwise the
* lifted subquery can't evaluate without an outer row, and any
* result would be wrong. This rules out patterns like
* WHERE e2.dept = e.dept AND e2.salary > e.salary
* where the second term still references the outer row. */
FOR i := 1 TO Len( aRemainTerms )
IF SemiJoinHasOuterRef( aRemainTerms[ i ], aLocalAliases )
RETURN NIL
ENDIF
NEXT
/* Rebuild the remainder WHERE as a right-leaning AND chain, or NIL */
xRemain := NIL
FOR i := 1 TO Len( aRemainTerms )
IF xRemain == NIL
xRemain := aRemainTerms[ i ]
ELSE
xRemain := SqlNode( ND_BIN, "AND", xRemain, aRemainTerms[ i ], NIL )
ENDIF
NEXT
/* Clone hQuery, replace SELECT list with DISTINCT inner.col,
* swap WHERE for the remainder. Other keys (tables, limit, etc.)
* are shallow-copied intentionally. */
hLifted := { => }
FOR i := 1 TO Len( hb_HKeys( hQ ) )
hLifted[ hb_HKeys( hQ )[ i ] ] := hQ[ hb_HKeys( hQ )[ i ] ]
NEXT
hLifted[ "columns" ] := { { SqlNode( ND_COL, cInnerField, NIL, NIL, NIL ), cInnerField } }
hLifted[ "where" ] := xRemain
hLifted[ "distinct" ] := .T.
/* `limit`/`top` use NIL ("no limit") instead of 0. After the recent
* LIMIT 0 fix, `0` is treated as an explicit "return no rows"
* sentinel — which collapsed every lifted EXISTS query to an empty
* inner_set and produced false negatives. */
hLifted[ "limit" ] := NIL
hLifted[ "top" ] := NIL
hLifted[ "order_by" ] := {}
hLifted[ "group_by" ] := {}
hLifted[ "having" ] := NIL
/* Run the lifted query once. No PushOuter — it's now non-correlated. */
oSub := TSqlExecutor():New( hLifted, ::aParams, ::oSession )
oSub:nDepth := ::nDepth
aResult := oSub:Run()
IF ValType( aResult ) != "A" .OR. Len( aResult ) < 2 .OR. ValType( aResult[ 2 ] ) != "A"
RETURN NIL
ENDIF
/* Build the hash set */
hSet := { => }
FOR i := 1 TO Len( aResult[ 2 ] )
IF Len( aResult[ 2 ][ i ] ) > 0
xVal := aResult[ 2 ][ i ][ 1 ]
cKey := SqlValToStr( xVal )
hSet[ cKey ] := .T.
ENDIF
NEXT
RETURN { "outer_ref" => xOuterRef, "inner_set" => hSet }
/* Helpers for TryBuildSemiJoin — module-level functions to keep the
* method body short. */
STATIC FUNCTION SemiJoinIsInner( xCol, aLocalAliases )
LOCAL cRef, nDot, cAlias
IF xCol == NIL .OR. ValType( xCol ) != "A" .OR. xCol[ 1 ] != ND_COL
RETURN .F.
ENDIF
cRef := xCol[ 2 ]
nDot := At( ".", cRef )
IF nDot == 0
/* Unqualified — assume inner since it would resolve in own FROM */
RETURN .T.
ENDIF
cAlias := Upper( Left( cRef, nDot - 1 ) )
RETURN AScan( aLocalAliases, cAlias ) > 0
STATIC FUNCTION SemiJoinStripAlias( cRef )
LOCAL nDot := At( ".", cRef )
IF nDot > 0
RETURN SubStr( cRef, nDot + 1 )
ENDIF
RETURN cRef
/* Recursively check whether an AST expression references any column
* whose alias prefix is NOT in the local alias list. Unqualified
* refs are assumed local. Returns .T. on first outer reference seen. */
STATIC FUNCTION SemiJoinHasOuterRef( xE, aLocalAliases )
LOCAL i, cRef, nDot, cAlias
IF xE == NIL .OR. ValType( xE ) != "A" .OR. Len( xE ) < 1
RETURN .F.
ENDIF
DO CASE
CASE xE[ 1 ] == ND_COL
IF Len( xE ) >= 2 .AND. ValType( xE[ 2 ] ) == "C"
cRef := xE[ 2 ]
nDot := At( ".", cRef )
IF nDot == 0
RETURN .F. /* unqualified → assumed local */
ENDIF
cAlias := Upper( Left( cRef, nDot - 1 ) )
RETURN AScan( aLocalAliases, cAlias ) == 0
ENDIF
CASE xE[ 1 ] == ND_BIN .OR. xE[ 1 ] == ND_RANGE
IF SemiJoinHasOuterRef( xE[ 3 ], aLocalAliases )
RETURN .T.
ENDIF
IF SemiJoinHasOuterRef( xE[ 4 ], aLocalAliases )
RETURN .T.
ENDIF
IF Len( xE ) >= 5 .AND. SemiJoinHasOuterRef( xE[ 5 ], aLocalAliases )
RETURN .T.
ENDIF
CASE xE[ 1 ] == ND_UNI
RETURN SemiJoinHasOuterRef( xE[ 3 ], aLocalAliases )
CASE xE[ 1 ] == ND_FN
IF Len( xE ) >= 3 .AND. ValType( xE[ 3 ] ) == "A"
FOR i := 1 TO Len( xE[ 3 ] )
IF SemiJoinHasOuterRef( xE[ 3 ][ i ], aLocalAliases )
RETURN .T.
ENDIF
NEXT
ENDIF
CASE xE[ 1 ] == ND_LIST
IF Len( xE ) >= 2 .AND. ValType( xE[ 2 ] ) == "A"
FOR i := 1 TO Len( xE[ 2 ] )
IF SemiJoinHasOuterRef( xE[ 2 ][ i ], aLocalAliases )
RETURN .T.
ENDIF
NEXT
ENDIF
ENDCASE
RETURN .F.
/* SubqueryCached — correlated-aware subquery execution with memoization.
*
* Walks the subquery's AST on first call to identify free variables —
* column references whose alias prefix is NOT one of the subquery's own
* FROM tables. These are the outer-row columns the subquery actually
* depends on. The cache key is built from the current values of those
* free variables, so:
*
* - Non-correlated subqueries (no free vars) → single cache entry,
* evaluated once, reused for every outer row. (Matches the old
* CacheSubquery behavior for simple `WHERE x > (SELECT MAX(y) FROM t)`.)
* - Correlated subqueries with a small distinct set of outer-key
* values → evaluated once per distinct key, not once per row.
* (Q8 in the SQL:2013 bench dropped from 4.9s to ~50ms with this.)
*
* The per-subquery ID and collected free variable list are memoized
* onto the AST node itself (slot 6) so the analysis runs only once per
* distinct subquery expression.
*/
METHOD SubqueryCached( xSubNode ) CLASS TSqlExecutor
LOCAL hQ, aFreeVars, cCacheKey, aResult, nSavedWA, oSub
LOCAL i, xVal, nId, nSlot, aSlot, aKeyVals, aRecSave
IF xSubNode == NIL .OR. ValType( xSubNode ) != "A" .OR. Len( xSubNode ) < 2
RETURN NIL
ENDIF
hQ := xSubNode[ 2 ]
IF hQ == NIL
RETURN NIL
ENDIF
/* Identify this subquery: linear-search the slots list for a prior
* entry that references the SAME AST node (array `==` is reference
* compare in Harbour). Most queries have only a handful of sub-
* queries so the scan is trivial. Avoids mutating the parse tree. */
nSlot := 0
FOR i := 1 TO Len( ::aSubCacheSlots )
IF ::aSubCacheSlots[ i ][ 1 ] == xSubNode
nSlot := i
EXIT
ENDIF
NEXT
IF nSlot == 0
::nSubCacheSeq++
aFreeVars := ::CollectFreeVars( hQ )
AAdd( ::aSubCacheSlots, { xSubNode, { ::nSubCacheSeq, aFreeVars } } )
nSlot := Len( ::aSubCacheSlots )
ENDIF
aSlot := ::aSubCacheSlots[ nSlot ][ 2 ]
nId := aSlot[ 1 ]
aFreeVars := aSlot[ 2 ]
/* Build cache key from current values of free variables via
* Resolve(), which walks the outer context stack. The value ↦
* string encoding + final join happen in Go (SqlBuildSubCacheKey)
* so the hot cache-hit path avoids N PRG string concatenations
* and N SqlValToStr dispatches per outer row. */
aKeyVals := Array( Len( aFreeVars ) )
FOR i := 1 TO Len( aFreeVars )
aKeyVals[ i ] := ::Resolve( aFreeVars[ i ] )
NEXT
cCacheKey := SqlBuildSubCacheKey( nId, aKeyVals )
IF hb_HHasKey( ::hSubCorrCache, cCacheKey )
RETURN ::hSubCorrCache[ cCacheKey ]
ENDIF
/* Cache miss — execute the subquery. PushOuter so nested ::Resolve
* calls can see the current outer row's values. Use BEGIN SEQUENCE
* to guarantee PopOuter runs even on subquery runtime errors —
* a stale ::oSession:aOuterStack entry would corrupt all subsequent queries.
*
* Workarea snapshot: the subquery may scan the SAME table the
* outer query is iterating, and SqlExecOpenTable only renames
* aliases deeper than nDepth=1 — so the first-level subquery's
* scan drives the shared workarea past EOF. Save every open
* workarea's RecNo up front and restore it before returning so
* the outer iterator resumes exactly where it left off. Depth
* bump is still set for good measure (avoids re-entering the
* same subquery's own workarea on recursion). */
nSavedWA := Select()
aRecSave := ::SnapshotAreaRecNos()
::PushOuter()
BEGIN SEQUENCE
oSub := TSqlExecutor():New( hQ, ::aParams, ::oSession )
/* +2 (not +1): the alias-rename gate in the table-open loop
* requires `nDepth > 1` to fire. Bumping by 1 from a top-level
* (depth-0) outer landed at depth 1 which still shares aliases
* with the outer scope — a subquery over the same table
* inherited the outer's workarea and the inner scan drove
* the outer's record pointer to EOF, truncating the outer
* result. Bumping by 2 forces the alias-acquire path so the
* subquery always gets a fresh workarea; deeper nesting stays
* strictly increasing. */
oSub:nDepth := ::nDepth + 2
aResult := oSub:Run()
RECOVER
aResult := NIL
END SEQUENCE
::PopOuter()
::RestoreAreaRecNos( aRecSave )
dbSelectArea( nSavedWA )
IF aResult != NIL
::hSubCorrCache[ cCacheKey ] := aResult
ENDIF
RETURN aResult
/* Collect the list of free-variable column names referenced by a
* subquery. A column is "free" if its alias prefix isn't one of the
* subquery's own FROM tables (so it must resolve in the outer scope).
* Returns an array of name strings that Resolve() understands —
* typically qualified forms like "E1.DEPT".
*/
METHOD CollectFreeVars( hQ ) CLASS TSqlExecutor
LOCAL aFree := {}, aLocalAliases := {}, i, aT
IF ValType( hQ ) != "H"
RETURN aFree
ENDIF
/* Local aliases known to the subquery */
IF hb_HHasKey( hQ, "tables" )
FOR i := 1 TO Len( hQ[ "tables" ] )
aT := hQ[ "tables" ][ i ]
IF ValType( aT ) == "A" .AND. Len( aT ) >= 1
AAdd( aLocalAliases, Upper( aT[ 1 ] ) )
IF Len( aT ) >= 2 .AND. ! Empty( aT[ 2 ] )
AAdd( aLocalAliases, Upper( aT[ 2 ] ) )
ENDIF
ENDIF
NEXT
ENDIF
/* Walk the WHERE, SELECT list, HAVING for ND_COL refs */
IF hb_HHasKey( hQ, "where" ) .AND. hQ[ "where" ] != NIL
::CollectExprFreeVars( hQ[ "where" ], aLocalAliases, aFree )
ENDIF
IF hb_HHasKey( hQ, "columns" )
FOR i := 1 TO Len( hQ[ "columns" ] )
IF ValType( hQ[ "columns" ][ i ] ) == "A" .AND. Len( hQ[ "columns" ][ i ] ) >= 1
::CollectExprFreeVars( hQ[ "columns" ][ i ][ 1 ], aLocalAliases, aFree )
ENDIF
NEXT
ENDIF
IF hb_HHasKey( hQ, "having" ) .AND. hQ[ "having" ] != NIL
::CollectExprFreeVars( hQ[ "having" ], aLocalAliases, aFree )
ENDIF
RETURN aFree
/* Recursively walk a SQL AST expression tree collecting column refs
* whose alias prefix is not in aLocalAliases. Appends to aFree. */
METHOD CollectExprFreeVars( xE, aLocalAliases, aFree ) CLASS TSqlExecutor
LOCAL i, cRef, cAlias, nDot
IF xE == NIL .OR. ValType( xE ) != "A" .OR. Len( xE ) < 1
RETURN NIL
ENDIF
DO CASE
CASE xE[ 1 ] == ND_COL
IF Len( xE ) >= 2 .AND. ValType( xE[ 2 ] ) == "C"
cRef := xE[ 2 ]
nDot := At( ".", cRef )
IF nDot > 0
cAlias := Upper( Left( cRef, nDot - 1 ) )
IF AScan( aLocalAliases, cAlias ) == 0 .AND. ;
AScan( aFree, cRef ) == 0
AAdd( aFree, cRef )
ENDIF
ENDIF
ENDIF
CASE xE[ 1 ] == ND_BIN .OR. xE[ 1 ] == ND_RANGE
::CollectExprFreeVars( xE[ 3 ], aLocalAliases, aFree )
::CollectExprFreeVars( xE[ 4 ], aLocalAliases, aFree )
IF Len( xE ) >= 5
::CollectExprFreeVars( xE[ 5 ], aLocalAliases, aFree )
ENDIF
CASE xE[ 1 ] == ND_UNI
::CollectExprFreeVars( xE[ 3 ], aLocalAliases, aFree )
CASE xE[ 1 ] == ND_FN
/* Walk function arguments, but SKIP the subquery's own subqueries.
* Nested subqueries have their own scope and will be analyzed when
* they're first executed. */
IF Len( xE ) >= 3 .AND. ValType( xE[ 3 ] ) == "A"
FOR i := 1 TO Len( xE[ 3 ] )
::CollectExprFreeVars( xE[ 3 ][ i ], aLocalAliases, aFree )
NEXT
ENDIF
CASE xE[ 1 ] == ND_CASE
IF Len( xE ) >= 2 .AND. ValType( xE[ 2 ] ) == "A"
FOR i := 1 TO Len( xE[ 2 ] )
IF ValType( xE[ 2 ][ i ] ) == "A" .AND. Len( xE[ 2 ][ i ] ) >= 2
::CollectExprFreeVars( xE[ 2 ][ i ][ 1 ], aLocalAliases, aFree )
::CollectExprFreeVars( xE[ 2 ][ i ][ 2 ], aLocalAliases, aFree )
ENDIF
NEXT
ENDIF
IF Len( xE ) >= 3
::CollectExprFreeVars( xE[ 3 ], aLocalAliases, aFree )
ENDIF
CASE xE[ 1 ] == ND_LIST
IF Len( xE ) >= 2 .AND. ValType( xE[ 2 ] ) == "A"
FOR i := 1 TO Len( xE[ 2 ] )
::CollectExprFreeVars( xE[ 2 ][ i ], aLocalAliases, aFree )
NEXT
ENDIF
/* Nested ND_SUB is intentionally opaque — its own free vars will
* be analyzed on its first call */
ENDCASE
RETURN NIL
/* Snapshot every currently-open workarea's (alias, RecNo) pair so a
* nested subquery scan that shares one of those areas can't leave
* the record pointer past EOF. Harbour doesn't expose a clean "list
* open workareas" primitive, so we probe a fixed range (1..MAX) and
* skip the ones with empty aliases. MAX stays small — any workload
* needing more open areas simultaneously would already break the
* default Harbour limit. */
METHOD SnapshotAreaRecNos() CLASS TSqlExecutor
LOCAL aSnap := {}, i, nSaved, cAlias
nSaved := Select()
FOR i := 1 TO 64
dbSelectArea( i )
cAlias := Alias()
IF ! Empty( cAlias )
AAdd( aSnap, { i, RecNo(), cAlias } )
ENDIF
NEXT
dbSelectArea( nSaved )
RETURN aSnap
METHOD RestoreAreaRecNos( aSnap ) CLASS TSqlExecutor
LOCAL i, nSaved
IF aSnap == NIL .OR. Len( aSnap ) == 0
RETURN NIL
ENDIF
nSaved := Select()
FOR i := 1 TO Len( aSnap )
/* Bind by alias when possible — subquery execution can
* rebalance the workarea table (e.g. close a CTE temp area),
* so the original numeric slot may now hold something else.
* If the alias is gone, silently skip the restore for that
* entry rather than seeking into an unrelated area. */
IF Select( aSnap[ i ][ 3 ] ) == aSnap[ i ][ 1 ]
dbSelectArea( aSnap[ i ][ 1 ] )
IF aSnap[ i ][ 2 ] > 0 .AND. aSnap[ i ][ 2 ] <= LastRec()
dbGoto( aSnap[ i ][ 2 ] )
ENDIF
ENDIF
NEXT
dbSelectArea( nSaved )
RETURN NIL
METHOD CacheSubquery( xSubExpr ) CLASS TSqlExecutor
LOCAL cKey, aSubResult, nSavedWA, oSub, aRecSave
/* Build cache key from subquery tokens */
cKey := SqlSubqueryKey( xSubExpr )
IF hb_HHasKey( ::hSubCache, cKey )
RETURN ::hSubCache[ cKey ]
ENDIF
/* Snapshot all open workareas' RecNos: same rationale as
* SubqueryCached — the inner scan can move the outer's record
* pointer when the subquery opens a table that's already in
* play. Non-correlated subqueries run once so the overhead is
* negligible; correlated callers go through SubqueryCached. */
nSavedWA := Select()
aRecSave := ::SnapshotAreaRecNos()
oSub := TSqlExecutor():New( xSubExpr, ::aParams, ::oSession )
oSub:nDepth := ::nDepth + 2
aSubResult := oSub:Run()
::RestoreAreaRecNos( aRecSave )
dbSelectArea( nSavedWA )
::hSubCache[ cKey ] := aSubResult
RETURN aSubResult
/* Materialize CTE definitions into temporary DBF tables */
METHOD MaterializeCTE( aCTE ) CLASS TSqlExecutor
LOCAL i, cName, xSubQ, aSub, aFN, aDataRows
LOCAL j, k, lReplaced, xVal
LOCAL aStruct, cTmpFile, nExistWA, cPopAlias
LOCAL cType, nWidth, nDec, nScan
FOR i := 1 TO Len( aCTE )
cName := Upper( aCTE[ i ][ 1 ] )
xSubQ := aCTE[ i ][ 2 ]
/* Execute the CTE subquery */
IF ValType( xSubQ ) == "A" .AND. xSubQ[ 1 ] == ND_SUB .AND. xSubQ[ 2 ] != NIL
aSub := TSqlExecutor():New( xSubQ[ 2 ], ::aParams, ::oSession ):Run()
ELSE
aSub := NIL
ENDIF
IF ValType( aSub ) != "A" .OR. Len( aSub ) < 2
LOOP
ENDIF
aFN := aSub[ 1 ]
aDataRows := aSub[ 2 ]
/* Apply CTE column aliases if specified: WITH name(col1,col2) AS ... */
IF Len( aCTE[ i ] ) >= 3 .AND. ValType( aCTE[ i ][ 3 ] ) == "A" .AND. Len( aCTE[ i ][ 3 ] ) > 0
FOR j := 1 TO Min( Len( aCTE[ i ][ 3 ] ), Len( aFN ) )
aFN[ j ] := Upper( aCTE[ i ][ 3 ][ j ] )
NEXT
ENDIF
/* Build structure for temp DBF */
aStruct := {}
FOR j := 1 TO Len( aFN )
cType := "C" ; nWidth := 40 ; nDec := 0
xVal := NIL
FOR nScan := 1 TO Min( Len( aDataRows ), 50 )
IF j <= Len( aDataRows[ nScan ] ) .AND. aDataRows[ nScan ][ j ] != NIL
xVal := aDataRows[ nScan ][ j ] ; EXIT
ENDIF
NEXT
IF xVal != NIL
IF ValType( xVal ) == "N" ; cType := "N" ; nWidth := 18 ; nDec := 4
ELSEIF ValType( xVal ) == "D" ; cType := "D" ; nWidth := 8
ELSEIF ValType( xVal ) == "L" ; cType := "L" ; nWidth := 1
ENDIF
ENDIF
AAdd( aStruct, { PadR( Upper( aFN[ j ] ), 10 ), cType, nWidth, nDec } )
NEXT
cTmpFile := "__cte_" + Lower( cName )
cPopAlias := "__CTE" + hb_ntos( i ) + "__"
nExistWA := Select( cName )
IF nExistWA > 0
dbSelectArea( nExistWA )
dbCloseArea()
ENDIF
nExistWA := Select( cPopAlias )
IF nExistWA > 0
dbSelectArea( nExistWA )
dbCloseArea()
ENDIF
/* In-memory temp table — no file I/O, `mem:` scheme dispatches
* to MEMRDD. Create overwrites any prior table with this name. */
BEGIN SEQUENCE
dbCreate( "mem:" + cTmpFile, aStruct, "MEMRDD" )
RECOVER
LOOP
END SEQUENCE
dbUseArea( .T., "MEMRDD", "mem:" + cTmpFile, cPopAlias, .F., .F. )
/* Go RTL SqlBulkInsert: collapses per-row dbAppend+FieldPut loop
* into a single RTL call — N·M boundary crossings → 1. */
SqlBulkInsert( aDataRows )
dbSelectArea( Select( cPopAlias ) )
dbCloseArea()
dbUseArea( .T., "MEMRDD", "mem:" + cTmpFile, cName, .T., .F. )
/* Replace existing table entry */
lReplaced := .F.
NEXT
RETURN NIL
METHOD RunInsert() CLASS TSqlExecutor
LOCAL cTable, aFields, aRows, aValExprs, cAlias, nWA, i, nFPos, xVal
LOCAL aAutoInc, nAutoVal, hSelect, aSelResult, aSelRows, nTuple
LOCAL aErrResult, nInserted := 0, nTotal
LOCAL aStructFlags, aStructRaw, aStructTypes
cTable := ::hQuery[ "table" ]
aFields := ::hQuery[ "fields" ]
cAlias := cTable
/* Materialize CTEs first — `WITH cte AS (...) INSERT ...`
* needs the CTE's temp table to exist before any SELECT
* subqueries inside aRows / aSelect resolve against it. */
IF hb_HHasKey( ::hQuery, "cte" ) .AND. ValType( ::hQuery[ "cte" ] ) == "A"
IF hb_HHasKey( ::hQuery, "cte_recursive" ) .AND. ::hQuery[ "cte_recursive" ]
::MaterializeRecursiveCTE( ::hQuery[ "cte" ] )
ELSE
::MaterializeCTE( ::hQuery[ "cte" ] )
ENDIF
ENDIF
/* Same pre-flight existence check as RunUpdate — if the user
* names a missing table, return SQL_ERR_NO_TABLE rather than
* letting dbUseArea bubble up an opaque runtime panic. */
IF ! File( Lower( cTable ) + ".dbf" ) .AND. ! File( cTable + ".dbf" )
RETURN ::MakeError( SQL_ERR_NO_TABLE, ;
"Table '" + cTable + "' does not exist" )
ENDIF
/* Schema migration note: old plans emitted h["values"] (a flat
* expression array, one tuple). Current parser emits h["rows"]
* (array of tuples) plus h["select"] for INSERT ... SELECT. The
* lookup here tolerates either shape for cached-plan callers and
* drives the per-row loop below. */
IF hb_HHasKey( ::hQuery, "rows" ) .AND. ValType( ::hQuery[ "rows" ] ) == "A"
aRows := ::hQuery[ "rows" ]
ELSE
aRows := {}
ENDIF
IF Len( aRows ) == 0 .AND. hb_HHasKey( ::hQuery, "values" )
aRows := { ::hQuery[ "values" ] }
ENDIF
/* INSERT ... SELECT: evaluate the subquery once, convert each
* result row into a tuple of literal-ish ND_LIT nodes, then fall
* through to the standard per-tuple loop. Keeping the rows-path
* as the single execution code path means CHECK / FK / UNIQUE /
* auto-inc / txn-log all run identically whether the values came
* from VALUES (...) tuples or from a SELECT. */
IF hb_HHasKey( ::hQuery, "select" )
hSelect := ::hQuery[ "select" ]
aSelResult := TSqlExecutor():New( hSelect, ::aParams, ::oSession ):Run()
IF ValType( aSelResult ) == "A" .AND. Len( aSelResult ) >= 1 .AND. ;
ValType( aSelResult[ 1 ] ) == "A" .AND. Len( aSelResult[ 1 ] ) > 0 .AND. ;
aSelResult[ 1 ][ 1 ] == "__error__"
RETURN aSelResult
ENDIF
IF ValType( aSelResult ) == "A" .AND. Len( aSelResult ) >= 2 .AND. ;
ValType( aSelResult[ 2 ] ) == "A"
aSelRows := aSelResult[ 2 ]
FOR i := 1 TO Len( aSelRows )
aValExprs := {}
FOR nTuple := 1 TO Len( aSelRows[ i ] )
AAdd( aValExprs, SqlNode( ND_LIT, aSelRows[ i ][ nTuple ], NIL, NIL, NIL ) )
NEXT
AAdd( aRows, aValExprs )
NEXT
ENDIF
ENDIF
aAutoInc := SqlGetAutoIncFields( cTable )
nWA := SqlExecOpenTable( cTable, cAlias )
nTotal := Len( aRows )
/* Snapshot the struct once: field flags (5th element) are used by
* the per-tuple loop below to reject a NIL write into a NOT NULL
* column. Without this the PutValue path would silently coerce
* NIL → 0 / blank and defeat the schema contract. Tables on
* pre-nullable-flag plans return 4-element rows; treat missing
* flag as "unknown → accept NIL" to stay backward-compatible. */
aStructFlags := {}
aStructTypes := {}
aStructRaw := dbStruct()
FOR i := 1 TO Len( aStructRaw )
IF Len( aStructRaw[ i ] ) >= 5 .AND. ValType( aStructRaw[ i ][ 5 ] ) == "N"
AAdd( aStructFlags, aStructRaw[ i ][ 5 ] )
ELSE
AAdd( aStructFlags, 2 ) /* unknown → assume nullable */
ENDIF
AAdd( aStructTypes, Upper( aStructRaw[ i ][ 2 ] ) )
NEXT
/* Pre-flight: explicit column list must reference real columns,
* and tuple width can't exceed the table's column count. Catch
* both up front so we don't silently drop user input — previously
* `INSERT INTO t VALUES (1, 'a', 'extra')` succeeded with the
* 'extra' value dropped, and `INSERT INTO t (no_such) VALUES (1)`
* succeeded with the value going nowhere. */
IF Len( aFields ) > 0
FOR i := 1 TO Len( aFields )
IF FieldPos( aFields[ i ] ) == 0
SqlExecCloseTable( cAlias, nWA )
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"INSERT: column '" + aFields[ i ] + ;
"' does not exist in " + cTable )
ENDIF
NEXT
ENDIF
IF nTotal > 0 .AND. Len( aFields ) == 0 .AND. Len( aRows[ 1 ] ) > FCount()
SqlExecCloseTable( cAlias, nWA )
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"INSERT: " + hb_NToS( Len( aRows[ 1 ] ) ) + ;
" values supplied but " + cTable + " has " + ;
hb_NToS( FCount() ) + " columns" )
ENDIF
FOR nTuple := 1 TO nTotal
aValExprs := aRows[ nTuple ]
dbAppend()
IF Len( aFields ) > 0
FOR i := 1 TO Min( Len( aFields ), Len( aValExprs ) )
nFPos := FieldPos( aFields[ i ] )
IF nFPos > 0
xVal := ::EvalExpr( aValExprs[ i ] )
/* NOT NULL guard: reject explicit NIL into a column
* whose Flags lack the nullable bit (0x02). dbDelete
* rolls back the phantom record so the user sees the
* old table state. */
IF xVal == NIL .AND. nFPos <= Len( aStructFlags ) .AND. ;
hb_BitAnd( aStructFlags[ nFPos ], 2 ) == 0
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"NOT NULL violation: column " + aFields[ i ] + ;
" on table " + cTable )
ENDIF
xVal := SqlCoerceToCol( xVal, aStructTypes, nFPos )
/* Numeric overflow guard: Str() returns a string of '*'
* when the value doesn't fit in the column's (width,dec).
* Without this, INSERT INTO n(N(4,0)) VALUES (99999999)
* silently stored 0 (or garbage) — DBF's N codec writes
* the truncated representation instead of erroring out. */
IF nFPos <= Len( aStructTypes ) .AND. aStructTypes[ nFPos ] == "N" .AND. ;
ValType( xVal ) == "N"
IF '*' $ Str( xVal, FieldLen( nFPos ), FieldDec( nFPos ) )
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"Numeric overflow: " + AllTrim( hb_NToS( xVal ) ) + ;
" does not fit in " + cTable + "." + aFields[ i ] + ;
" (N(" + AllTrim( hb_NToS( FieldLen( nFPos ) ) ) + ;
"," + AllTrim( hb_NToS( FieldDec( nFPos ) ) ) + "))" )
ENDIF
ENDIF
FieldPut( nFPos, xVal )
ENDIF
NEXT
ELSE
FOR i := 1 TO Min( FCount(), Len( aValExprs ) )
xVal := ::EvalExpr( aValExprs[ i ] )
IF xVal == NIL .AND. i <= Len( aStructFlags ) .AND. ;
hb_BitAnd( aStructFlags[ i ], 2 ) == 0
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"NOT NULL violation: column " + FieldName( i ) + ;
" on table " + cTable )
ENDIF
xVal := SqlCoerceToCol( xVal, aStructTypes, i )
/* Same overflow guard as the named-columns branch above. */
IF i <= Len( aStructTypes ) .AND. aStructTypes[ i ] == "N" .AND. ;
ValType( xVal ) == "N"
IF '*' $ Str( xVal, FieldLen( i ), FieldDec( i ) )
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"Numeric overflow: " + AllTrim( hb_NToS( xVal ) ) + ;
" does not fit in " + cTable + "." + FieldName( i ) + ;
" (N(" + AllTrim( hb_NToS( FieldLen( i ) ) ) + ;
"," + AllTrim( hb_NToS( FieldDec( i ) ) ) + "))" )
ENDIF
ENDIF
FieldPut( i, xVal )
NEXT
ENDIF
/* Auto-increment */
FOR i := 1 TO Len( aAutoInc )
nFPos := FieldPos( aAutoInc[ i ] )
IF nFPos > 0
xVal := FieldGet( nFPos )
IF ValType( xVal ) == "N" .AND. xVal == 0
nAutoVal := SqlGetMaxFieldVal( cAlias, aAutoInc[ i ] ) + 1
FieldPut( nFPos, nAutoVal )
ENDIF
ENDIF
NEXT
/* Validate CHECK constraints against current record values */
IF ! SqlValidateCheckRecord( cTable )
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, "CHECK constraint violation on " + cTable )
ENDIF
/* Validate FOREIGN KEY constraints. Iterate every field on the
* just-appended record — using only `aFields` (named-column form)
* would skip the positional `INSERT INTO t VALUES (...)` form and
* let bad FK values slip through silently. SqlValidateFKRecord
* short-circuits for fields with no FK so the per-column scan is
* cheap. Self-FK + multi-row INSERT remains a known limitation:
* the parent area = INSERT area, and SqlValidateFKRecord's
* dbGoTop scan races with the in-flight buffer; symptom is a
* spurious FK violation on row 2+. Single-row + cross-table
* cases work correctly. */
FOR i := 1 TO FCount()
IF ! SqlValidateFKRecord( cTable, FieldName( i ), FieldGet( i ) )
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"FOREIGN KEY violation: " + FieldName( i ) + " references missing parent" )
ENDIF
NEXT
/* Validate UNIQUE constraints — the .fsc metadata file lists
* columns declared UNIQUE at CREATE TABLE time. SqlValidateUnique
* scans the table for duplicates, excluding the record we just
* appended. Previously this validator was defined but never
* invoked, so duplicate keys slipped through silently.
* Excludes RecNo() (just-appended row) from the dup scan. */
IF Len( aFields ) > 0
FOR i := 1 TO Len( aFields )
nFPos := FieldPos( aFields[ i ] )
IF nFPos > 0
IF ! SqlValidateUnique( cTable, aFields[ i ], FieldGet( nFPos ), RecNo() )
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"UNIQUE constraint violation: " + aFields[ i ] + ;
" = " + SqlQuoteVal( FieldGet( nFPos ) ) )
ENDIF
ENDIF
NEXT
ELSE
FOR i := 1 TO FCount()
IF ! SqlValidateUnique( cTable, FieldName( i ), FieldGet( i ), RecNo() )
dbDelete()
dbCommit()
IF nWA == 0
dbCloseArea()
ENDIF
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"UNIQUE constraint violation: " + FieldName( i ) + ;
" = " + SqlQuoteVal( FieldGet( i ) ) )
ENDIF
NEXT
ENDIF
/* Transaction logging — after validation passes, so a rolled-back
* CHECK/FK failure doesn't leave a spurious INSERT in the log at
* the old record's position. LogRecord must also see the new
* RecNo, which only exists post-dbAppend. */
::oTxn:LogRecord( cAlias, RecNo(), "INSERT" )
nInserted++
NEXT /* end per-tuple loop */
/* Commit once after all tuples succeed. */
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
/* Index maintenance: DBFArea.Append / PutValue do not auto-update
* NTX keys (there's no ordKeyAdd hook yet). Rebuild any attached
* indexes once per INSERT batch so external `SET INDEX TO` readers
* see fresh keys. No-op when no indexes attached. */
IF OrdCount() > 0
OrderListRebuild()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nInserted } } }
METHOD RunUpdate() CLASS TSqlExecutor
LOCAL cTable, aSet, xWhere, cAlias, nWA, i, nFPos, xVal
LOCAL nAffected := 0
LOCAL aFPos, aValuePc, pcWhere, lAllOk, cValSrc
LOCAL aPrevVals, lValid
LOCAL hConstraints, lHasConstraints
LOCAL aUpdFlags, aUpdFlagsRaw, nUpdI, aUpdTypes, lForcePrg
LOCAL aUpdRefs, hUpdChanged, hUpdEnf, cParentColUpper, j
LOCAL hUpdConstraints, lHasUniq, lHasCheckOrFk
LOCAL hPcCached
cTable := ::hQuery[ "table" ]
aSet := ::hQuery[ "set" ]
xWhere := ::hQuery[ "where" ]
cAlias := cTable
::aTables := { { cTable, cAlias, "" } }
/* Materialize CTEs first so subqueries inside SET / WHERE that
* reference the CTE alias resolve correctly. Mirrors RunInsert. */
IF hb_HHasKey( ::hQuery, "cte" ) .AND. ValType( ::hQuery[ "cte" ] ) == "A"
IF hb_HHasKey( ::hQuery, "cte_recursive" ) .AND. ::hQuery[ "cte_recursive" ]
::MaterializeRecursiveCTE( ::hQuery[ "cte" ] )
ELSE
::MaterializeCTE( ::hQuery[ "cte" ] )
ENDIF
ENDIF
/* Pre-flight existence check — SqlExecOpenTable's RECOVER swallows
* the missing-file error (returns 0 with cache disabled or the
* cache num if present), but a dbUseArea panic still escapes if
* the file is genuinely absent. Surface a clean SQL_ERR_NO_TABLE
* here so callers see a structured error instead of a Five
* runtime panic. */
IF ! File( Lower( cTable ) + ".dbf" ) .AND. ! File( cTable + ".dbf" )
RETURN ::MakeError( SQL_ERR_NO_TABLE, ;
"Table '" + cTable + "' does not exist" )
ENDIF
nWA := SqlExecOpenTable( cTable, cAlias )
/* Struct snapshot used by both the fast-path coercion gate and
* the PRG fallback's NOT-NULL / string→date check. Built once up
* front so both paths see the same flags/types. */
aUpdFlags := {}
aUpdTypes := {}
aUpdFlagsRaw := dbStruct()
FOR nUpdI := 1 TO Len( aUpdFlagsRaw )
IF Len( aUpdFlagsRaw[ nUpdI ] ) >= 5 .AND. ValType( aUpdFlagsRaw[ nUpdI ][ 5 ] ) == "N"
AAdd( aUpdFlags, aUpdFlagsRaw[ nUpdI ][ 5 ] )
ELSE
AAdd( aUpdFlags, 2 )
ENDIF
AAdd( aUpdTypes, Upper( aUpdFlagsRaw[ nUpdI ][ 2 ] ) )
NEXT
/* Gate fast path out when any SET assigns a string ND_LIT to a
* DATE column: SqlExprToPrg would emit a raw string literal into
* the pcode, which SqlBulkUpdate then writes via the D-field
* codec as 8 blanks (its empty-date marker). The PRG path runs
* SqlCoerceToCol before FieldPut, so the string parses through
* CToD as the user intended. */
lForcePrg := .F.
IF aSet != NIL
FOR i := 1 TO Len( aSet )
nFPos := FieldPos( aSet[ i ][ 1 ] )
IF nFPos > 0 .AND. nFPos <= Len( aUpdTypes ) .AND. aUpdTypes[ nFPos ] == "D"
IF ValType( aSet[ i ][ 2 ] ) == "A" .AND. Len( aSet[ i ][ 2 ] ) >= 2 .AND. ;
aSet[ i ][ 2 ][ 1 ] == ND_LIT .AND. ValType( aSet[ i ][ 2 ][ 2 ] ) == "C"
lForcePrg := .T.
EXIT
ENDIF
ENDIF
NEXT
ENDIF
/* Fast path: compile WHERE + every SET value to pcode and delegate
* to Go RTL SqlBulkUpdate — skips per-record Go↔PRG boundary.
* Conditions: no active transaction (txn log records can't be
* emitted from inside the Go loop), no subquery / CASE / other
* nodes that PcCompile can't handle (try/fail pattern).
*
* Per-plan cache: when cCacheKey is set (TFiveSQL supplies it for
* plan-cached queries), we stash the compiled pcode under that key
* so subsequent identical UPDATEs skip the SqlExprToPrg + PcCompile
* walk entirely. The cached pcode is valid as long as the plan
* itself lives in the plan cache — which is forever in-process. */
/* Tables with UNIQUE constraints must go through the PRG loop so
* SqlValidateUnique can fire per record. SqlBulkUpdate is a
* per-row byte write with no constraint callback hook, so a
* fast-path UPDATE that writes a duplicate value would have
* silently committed. */
/* hUpdConstraints / lHasUniq / lHasCheckOrFk hoisted to function-top
* LOCALs above; mid-function LOCAL combined with newly-added top
* LOCALs caused slot aliasing → lHasUniq came back non-logical. */
hUpdConstraints := SqlLoadConstraints( cTable )
lHasUniq := Len( hUpdConstraints[ "unique" ] ) > 0
/* SqlBulkUpdate has no CHECK / FK callback hook — any table
* carrying CHECK or FK constraints must take the PRG path so
* SqlValidateCheckRecord / SqlValidateFKRecord actually fire. */
lHasCheckOrFk := Len( hUpdConstraints[ "check" ] ) > 0 .OR. ;
Len( hUpdConstraints[ "fk" ] ) > 0
/* ON UPDATE enforcement: if any sibling table FK-references this
* one, drop into the PRG path so we can compare per-row old/new
* values and fire CASCADE / SET NULL / RESTRICT. SqlBulkUpdate is
* a per-byte writer with no callback hook. */
aUpdRefs := SqlFindReferencingFKs( cTable )
IF ! ::oTxn:IsActive() .AND. ! lHasUniq .AND. ! lHasCheckOrFk .AND. ! lForcePrg .AND. ;
Len( aUpdRefs ) == 0
/* hPcCached hoisted to function-top LOCAL list. */
IF ! Empty( ::cCacheKey ) .AND. hb_HHasKey( ::oSession:hDmlPcodeCache, ::cCacheKey )
hPcCached := ::oSession:hDmlPcodeCache[ ::cCacheKey ]
nAffected := SqlBulkUpdate( hPcCached[ "set_fpos" ], ;
hPcCached[ "where_pc" ], ;
hPcCached[ "set_pc" ] )
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nAffected } } }
ENDIF
aFPos := {}
aValuePc := {}
lAllOk := .T.
FOR i := 1 TO Len( aSet )
nFPos := FieldPos( aSet[ i ][ 1 ] )
IF nFPos <= 0
lAllOk := .F.
EXIT
ENDIF
cValSrc := ::SqlExprToPrg( aSet[ i ][ 2 ] )
IF cValSrc == NIL
lAllOk := .F.
EXIT
ENDIF
AAdd( aFPos, nFPos )
AAdd( aValuePc, PcCompile( cValSrc ) )
IF ATail( aValuePc ) == NIL
lAllOk := .F.
EXIT
ENDIF
NEXT
pcWhere := NIL
IF lAllOk .AND. xWhere != NIL
cValSrc := ::SqlExprToPrg( xWhere )
IF cValSrc == NIL
lAllOk := .F.
ELSE
pcWhere := PcCompile( cValSrc )
IF pcWhere == NIL
lAllOk := .F.
ENDIF
ENDIF
ENDIF
IF lAllOk
nAffected := SqlBulkUpdate( aFPos, pcWhere, aValuePc )
/* Populate the per-plan cache for subsequent calls. */
IF ! Empty( ::cCacheKey )
IF Len( ::oSession:hDmlPcodeCache ) >= SQL_DML_PCODE_CACHE_MAX
::oSession:hDmlPcodeCache := { => }
ENDIF
::oSession:hDmlPcodeCache[ ::cCacheKey ] := { ;
"set_fpos" => aFPos, ;
"set_pc" => aValuePc, ;
"where_pc" => pcWhere }
ENDIF
/* Defer commit under WA cache — batched at Disable/exit. */
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nAffected } } }
ENDIF
ENDIF
/* Fallback: PRG scan loop — handles txn logging + non-compilable
* expressions (subquery, complex CASE, UDF in value or WHERE).
*
* Validates CHECK + FK only when the table has SQL-level
* constraints (a `.fsc` metadata file exists). Tables created
* via plain dbCreate have no constraints and skip the validator
* entirely — avoids a recursive five_SQL call inside every
* UPDATE's SqlValidateCheckRecord on transaction-active paths
* where the re-entry would deadlock the executor state. */
hConstraints := SqlLoadConstraints( cTable )
lHasConstraints := Len( hConstraints[ "check" ] ) > 0 .OR. ;
Len( hConstraints[ "fk" ] ) > 0
/* aUpdFlags / aUpdTypes populated earlier for both fast-path and
* PRG paths. Per-SET NIL-into-NOT-NULL check runs below. */
dbGoTop()
WHILE ! Eof()
IF xWhere == NIL .OR. SqlIsTrue( ::EvalExpr( xWhere ) )
IF dbRLock( RecNo() )
::oTxn:LogRecord( cAlias, RecNo(), "UPDATE" )
aPrevVals := {}
lValid := .T.
FOR i := 1 TO Len( aSet )
nFPos := FieldPos( aSet[ i ][ 1 ] )
IF nFPos > 0
AAdd( aPrevVals, { nFPos, FieldGet( nFPos ) } )
xVal := ::EvalExpr( aSet[ i ][ 2 ] )
IF xVal == NIL .AND. nFPos <= Len( aUpdFlags ) .AND. ;
hb_BitAnd( aUpdFlags[ nFPos ], 2 ) == 0
lValid := .F.
EXIT
ENDIF
xVal := SqlCoerceToCol( xVal, aUpdTypes, nFPos )
FieldPut( nFPos, xVal )
ENDIF
NEXT
IF ! lValid
FOR i := 1 TO Len( aPrevVals )
FieldPut( aPrevVals[ i ][ 1 ], aPrevVals[ i ][ 2 ] )
NEXT
dbRUnlock( RecNo() )
SqlExecCloseTable( cAlias, nWA )
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"NOT NULL violation on UPDATE of " + cTable )
ENDIF
lValid := .T.
IF lHasConstraints
lValid := SqlValidateCheckRecord( cTable )
IF lValid
FOR i := 1 TO Len( aSet )
nFPos := FieldPos( aSet[ i ][ 1 ] )
IF nFPos > 0 .AND. ;
! SqlValidateFKRecord( cTable, aSet[ i ][ 1 ], FieldGet( nFPos ) )
lValid := .F.
EXIT
ENDIF
NEXT
ENDIF
ENDIF
/* UNIQUE validation runs independently of hasConstraints
* because UNIQUE is tracked in the same .fsc but a table
* can declare only UNIQUE without CHECK/FK. Excludes the
* row we're currently editing so a self-match on the
* same column doesn't trigger a false positive. */
IF lValid
FOR i := 1 TO Len( aSet )
nFPos := FieldPos( aSet[ i ][ 1 ] )
IF nFPos > 0 .AND. ;
! SqlValidateUnique( cTable, aSet[ i ][ 1 ], FieldGet( nFPos ), RecNo() )
lValid := .F.
EXIT
ENDIF
NEXT
ENDIF
IF ! lValid
/* Roll back the in-memory field changes before unlocking. */
FOR i := 1 TO Len( aPrevVals )
FieldPut( aPrevVals[ i ][ 1 ], aPrevVals[ i ][ 2 ] )
NEXT
dbRUnlock( RecNo() )
SqlExecCloseTable( cAlias, nWA )
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
"UPDATE constraint violation on " + cTable )
ENDIF
/* ON UPDATE enforcement: if a parent column referenced
* by any sibling FK changed, fire CASCADE / SET NULL /
* RESTRICT against the children. Build the change hash
* from aPrevVals (old) + current FieldGet (new). Only
* include columns whose value actually moved — equal
* old/new must NOT trigger enforcement (otherwise an
* idempotent UPDATE fires spurious cascades). */
IF Len( aUpdRefs ) > 0
hUpdChanged := { => }
FOR i := 1 TO Len( aPrevVals )
cParentColUpper := Upper( FieldName( aPrevVals[ i ][ 1 ] ) )
xVal := FieldGet( aPrevVals[ i ][ 1 ] )
/* NIL-safe change detection. `xVal == aPrevVals[..]`
* panics when either side is NIL — switch to type-aware
* comparison: both NIL → unchanged, exactly one NIL →
* changed, otherwise direct ==. */
lValid := .T.
IF xVal == NIL .AND. aPrevVals[ i ][ 2 ] == NIL
lValid := .F. /* unchanged: both NULL */
ELSEIF xVal == NIL .OR. aPrevVals[ i ][ 2 ] == NIL
lValid := .T. /* changed: NULL ↔ value */
ELSE
lValid := ! ( xVal == aPrevVals[ i ][ 2 ] )
ENDIF
IF lValid
/* Only register if some sibling FK actually
* targets this column — keeps the changes hash
* sparse and avoids enforcement work for
* non-key updates. */
FOR j := 1 TO Len( aUpdRefs )
IF Upper( aUpdRefs[ j ][ 3 ] ) == cParentColUpper
hUpdChanged[ cParentColUpper ] := { aPrevVals[ i ][ 2 ], xVal }
EXIT
ENDIF
NEXT
ENDIF
NEXT
IF Len( hUpdChanged ) > 0
hUpdEnf := SqlEnforceUpdateRefs( cTable, aUpdRefs, hUpdChanged )
IF ! hUpdEnf[ "ok" ]
/* Roll back this row before bubbling the FK
* RESTRICT error so the user-visible state
* matches "no change applied to this record". */
FOR i := 1 TO Len( aPrevVals )
FieldPut( aPrevVals[ i ][ 1 ], aPrevVals[ i ][ 2 ] )
NEXT
dbRUnlock( RecNo() )
SqlExecCloseTable( cAlias, nWA )
RETURN ::MakeError( SQL_ERR_GRAMMAR, ;
hUpdEnf[ "error" ] )
ENDIF
ENDIF
ENDIF
dbRUnlock( RecNo() )
nAffected++
ENDIF
ENDIF
dbSkip()
ENDDO
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
/* Same rationale as RunInsert: PutValue skipped index maintenance
* so rebuild attached indexes at the tail. PRG path only — the
* Go-RTL SqlBulkUpdate fast-path already does a targeted rebuild
* when updated fields intersect index key expressions. */
IF nAffected > 0 .AND. OrdCount() > 0
OrderListRebuild()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nAffected } } }
METHOD RunDelete() CLASS TSqlExecutor
LOCAL cTable, xWhere, cAlias, nWA
LOCAL nAffected := 0
LOCAL pcWhere, cValSrc, hPcCached
LOCAL aRefs, hEnf, nParentRec, nParentSel
cTable := ::hQuery[ "table" ]
xWhere := ::hQuery[ "where" ]
cAlias := cTable
::aTables := { { cTable, cAlias, "" } }
/* Materialize CTEs first so subqueries inside WHERE that
* reference the CTE alias resolve correctly. Mirrors RunInsert. */
IF hb_HHasKey( ::hQuery, "cte" ) .AND. ValType( ::hQuery[ "cte" ] ) == "A"
IF hb_HHasKey( ::hQuery, "cte_recursive" ) .AND. ::hQuery[ "cte_recursive" ]
::MaterializeRecursiveCTE( ::hQuery[ "cte" ] )
ELSE
::MaterializeCTE( ::hQuery[ "cte" ] )
ENDIF
ENDIF
/* Pre-flight existence check, mirroring RunInsert / RunUpdate. */
IF ! File( Lower( cTable ) + ".dbf" ) .AND. ! File( cTable + ".dbf" )
RETURN ::MakeError( SQL_ERR_NO_TABLE, ;
"Table '" + cTable + "' does not exist" )
ENDIF
nWA := SqlExecOpenTable( cTable, cAlias )
/* Referential integrity: when any sibling table has a FOREIGN KEY
* pointing at cTable, we MUST take the PRG scan path — it's the
* only one that can evaluate per-row ON DELETE actions (RESTRICT /
* CASCADE / SET NULL). SqlBulkDelete is a pure byte-level delete
* and has no callback hook. No referencing FKs → fast path as
* before, zero overhead for FK-free tables. */
aRefs := SqlFindReferencingFKs( cTable )
/* Fast path: compile WHERE to pcode and delegate to SqlBulkDelete.
* Mirrors the RunUpdate pattern — skipped under an active txn (we
* don't emit LogRecord from inside the Go loop) and when the WHERE
* contains constructs PcCompile can't handle (subquery, UDF) in
* which case SqlExprToPrg returns NIL. */
IF ! ::oTxn:IsActive() .AND. Len( aRefs ) == 0
IF ! Empty( ::cCacheKey ) .AND. hb_HHasKey( ::oSession:hDmlPcodeCache, ::cCacheKey )
hPcCached := ::oSession:hDmlPcodeCache[ ::cCacheKey ]
nAffected := SqlBulkDelete( hPcCached[ "where_pc" ] )
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
IF nAffected > 0 .AND. OrdCount() > 0
OrderListRebuild()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nAffected } } }
ENDIF
pcWhere := NIL
IF xWhere != NIL
cValSrc := ::SqlExprToPrg( xWhere )
IF cValSrc != NIL
pcWhere := PcCompile( cValSrc )
ENDIF
ENDIF
/* pcWhere == NIL when xWhere itself was NIL (delete everything)
* OR compilation failed. Distinguish below: if WHERE was present
* but couldn't compile, fall through to the PRG loop; if WHERE
* was absent, run the fast path with NIL pcode (unconditional). */
IF xWhere == NIL .OR. pcWhere != NIL
nAffected := SqlBulkDelete( pcWhere )
IF ! Empty( ::cCacheKey ) .AND. xWhere != NIL
IF Len( ::oSession:hDmlPcodeCache ) >= SQL_DML_PCODE_CACHE_MAX
::oSession:hDmlPcodeCache := { => }
ENDIF
::oSession:hDmlPcodeCache[ ::cCacheKey ] := { "where_pc" => pcWhere }
ENDIF
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
IF nAffected > 0 .AND. OrdCount() > 0
OrderListRebuild()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nAffected } } }
ENDIF
ENDIF
/* PRG scan loop — handles active txn (needs LogRecord per row),
* WHEREs that SqlExprToPrg can't compile (subquery, UDF), and
* every DELETE on a table with referencing FOREIGN KEYs. */
SET DELETED ON
dbGoTop()
WHILE ! Eof()
IF xWhere == NIL .OR. SqlIsTrue( ::EvalExpr( xWhere ) )
IF dbRLock( RecNo() )
/* Enforce referential integrity before logging / deleting.
* CASCADE runs nested DELETEs on the child tables here,
* SET NULL runs a nested UPDATE; both MUST happen under
* the parent's record lock so concurrent inserts against
* the same FK value race against the commit order the
* caller expects. We capture the parent's RecNo + Select
* here because the nested five_SQL calls inside
* SqlEnforceDeleteRefs allocate / close workareas and
* leave the executor's current area on something other
* than the parent. */
IF Len( aRefs ) > 0
nParentRec := RecNo()
nParentSel := Select()
hEnf := SqlEnforceDeleteRefs( cTable, aRefs )
dbSelectArea( nParentSel )
dbGoto( nParentRec )
IF ! hEnf[ "ok" ]
dbRUnlock( nParentRec )
SqlExecCloseTable( cAlias, nWA )
RETURN ::MakeError( SQL_ERR_GRAMMAR, hEnf[ "error" ] )
ENDIF
ENDIF
/* Transaction log the deletion so BEGIN TRANSACTION /
* ROLLBACK can undo it — RunInsert/RunUpdate log, RunDelete
* used to silently skip. */
::oTxn:LogRecord( cAlias, RecNo(), "DELETE" )
dbDelete()
dbRUnlock( RecNo() )
nAffected++
ENDIF
ENDIF
dbSkip()
ENDDO
IF ! SqlWACacheIsEnabled()
dbCommit()
ENDIF
/* Deleted rows' keys must be purged from attached indexes. */
IF nAffected > 0 .AND. OrdCount() > 0
OrderListRebuild()
ENDIF
SqlExecCloseTable( cAlias, nWA )
RETURN { { "affected_rows" }, { { nAffected } } }
/* ======================================================================
* Workarea open/close helpers — consult the Go-native WA cache.
* When the cache is enabled (SqlWACacheEnable), SqlExecOpenTable
* reuses a previously opened workarea instead of running dbUseArea
* every call. SqlExecCloseTable leaves cached entries alive; plain
* (auto-opened, not cached) areas still close as before so tests
* that rely on immediate file release (FErase, UNIQUE index rebuild)
* stay correct when the cache is off — which is the default.
* ====================================================================== */
FUNCTION SqlExecOpenTable( cTable, cAlias )
LOCAL nWA, nCached
nWA := Select( cAlias )
IF nWA > 0
dbSelectArea( nWA )
RETURN nWA
ENDIF
/* Cache hit: the previously stored WA must still be valid and bound
* to the same alias. If a manual close or CLOSE ALL ran behind our
* back, Select() will now report 0 — fall through to fresh open. */
nCached := SqlWACacheGet( cAlias )
IF nCached > 0 .AND. Select( cAlias ) == nCached
dbSelectArea( nCached )
RETURN nCached
ENDIF
IF nCached > 0
SqlWACacheInvalidate( cAlias )
ENDIF
/* Open fresh. Two-step fallback mirrors the prior inline logic so
* callers using mixed-case filenames on case-sensitive filesystems
* still succeed.
*
* SHARED mode (5th arg .T.): a same-table subquery (e.g. `DELETE
* FROM t WHERE v > (SELECT MAX(v) FROM t)`) opens a second
* workarea on the same DBF inside SubqueryCached. With EXCLUSIVE
* the second open errored out → the inner SELECT returned the
* `__error__` envelope and ND_SUB extracted aResult[2][1][1] =
* SQL_ERR_LOCKED (1005). The DML still succeeds with SHARED
* because Five is single-process here; per-record dbRLock
* provides write serialization at the row level. */
BEGIN SEQUENCE
dbUseArea( .T., "DBFNTX", Lower( cTable ) + ".dbf", cAlias, .T., .F. )
RECOVER
dbUseArea( .T., "DBFNTX", cTable + ".dbf", cAlias, .T., .F. )
END SEQUENCE
nWA := Select( cAlias )
/* Auto-attach the known per-table NTX files so DML-side
* OrderListRebuild (called at the tail of RunInsert / RunUpdate /
* RunDelete) actually has indexes to refresh. Without this the
* PK / UNIQUE indexes — built at CREATE TABLE time with 0 rows —
* stay frozen forever and any external `SET INDEX TO ...` read
* sees a stale 0-key tree. Missing file => silent RECOVER, no
* harm to pre-index tables. */
IF nWA > 0
SqlAttachTableIndexes( cTable )
ENDIF
/* Register for reuse. The cache layer is a no-op when disabled, so
* an unconditional Put keeps the caller branch-free. */
IF nWA > 0 .AND. SqlWACacheIsEnabled()
SqlWACachePut( cAlias, nWA )
/* Return 1 sentinel so callers' "if nWA==0 close" gates skip
* — the cache owns the lifecycle now. */
RETURN nWA
ENDIF
RETURN 0 /* caller must close — matches legacy semantics */
/* Attach the convention-named PK / UNIQUE NTX files to the current
* workarea. Both files are produced by CreateTable at CREATE-TABLE
* time; missing files fall into RECOVER silently so pre-index tables
* (and tables with no UNIQUE columns) pay zero cost. */
FUNCTION SqlAttachTableIndexes( cTable )
LOCAL cPk, cUq
cPk := Lower( cTable ) + "_pk.ntx"
cUq := Lower( cTable ) + "_uq.ntx"
IF File( cPk )
BEGIN SEQUENCE
dbSetIndex( cPk )
RECOVER
END SEQUENCE
ENDIF
IF File( cUq )
BEGIN SEQUENCE
dbSetIndex( cUq )
RECOVER
END SEQUENCE
ENDIF
RETURN NIL
FUNCTION SqlExecCloseTable( cAlias, nWA )
/* Only close if THIS call opened it AND the cache didn't adopt it.
* When nWA > 0, the caller either reused a pre-existing area or
* handed ownership to the cache, so we leave it alone. */
IF nWA == 0 .AND. ! SqlWACacheIsEnabled()
dbCloseArea()
ELSEIF nWA == 0 .AND. SqlWACacheIsEnabled() .AND. ;
SqlWACacheGet( cAlias ) == 0
/* Cache enabled but the alias wasn't registered (e.g., open
* failed between Put checks). Keep legacy behavior — close. */
dbCloseArea()
ENDIF
RETURN NIL
/* SqlMaterializeView — execute the view's stored SQL once, dump the
* result into a MEMRDD temp area aliased as cAlias. Lets the rest of
* RunSelect's table-open / FROM-resolve machinery treat the view
* just like any other table. Returns .T. on success.
*
* The view text lives in `<view>.fsv`, written by CreateView. We re-
* parse + run it via a nested TFiveSQL each time the view is opened;
* no result caching at the FS level. Trade-off: simple + always
* fresh, but each SELECT-from-view re-executes the body. Acceptable
* because views are rare on this workload.
*/
FUNCTION SqlMaterializeView( cView, cAlias )
LOCAL cSQL, oV, aR, aFN, i, j, aStruct, cTmpName
LOCAL xS, cType, nWidth, nDec
/* Use MemoRead for the view body — FRead's @byref buffer
* argument has surfaced edge cases in this runtime where the
* destination string stays at its pre-call (Space-padded)
* value, leaving cSQL empty after AllTrim. MemoRead returns the
* file contents directly. */
cSQL := AllTrim( MemoRead( Lower( cView ) + ".fsv" ) )
IF Empty( cSQL )
RETURN .F.
ENDIF
oV := TFiveSQL():New()
aR := oV:Execute( cSQL )
IF ValType( aR ) != "A" .OR. Len( aR ) < 2 .OR. ;
ValType( aR[ 1 ] ) != "A" .OR. Len( aR[ 1 ] ) == 0 .OR. ;
aR[ 1 ][ 1 ] == "__error__"
RETURN .F.
ENDIF
aFN := aR[ 1 ]
/* Build a minimal struct from the result types. View columns get
* sensible widths: numeric → N(15,4), char → C(64), date → D,
* logical → L, anything else → C(255). Scalar widening acceptable
* for read-only view consumers. */
aStruct := {}
FOR i := 1 TO Len( aFN )
xS := NIL
IF Len( aR[ 2 ] ) > 0 .AND. Len( aR[ 2 ][ 1 ] ) >= i
xS := aR[ 2 ][ 1 ][ i ]
ENDIF
DO CASE
CASE ValType( xS ) == "N"
cType := "N" ; nWidth := 15 ; nDec := 4
CASE ValType( xS ) == "D"
cType := "D" ; nWidth := 8 ; nDec := 0
CASE ValType( xS ) == "L"
cType := "L" ; nWidth := 1 ; nDec := 0
CASE ValType( xS ) == "C"
cType := "C" ; nWidth := Max( Len( xS ), 64 ) ; nDec := 0
OTHERWISE
cType := "C" ; nWidth := 64 ; nDec := 0
ENDCASE
AAdd( aStruct, { aFN[ i ], cType, nWidth, nDec } )
NEXT
cTmpName := "mem:__view_" + Lower( cView )
BEGIN SEQUENCE
dbCreate( cTmpName, aStruct, "MEMRDD" )
dbUseArea( .T., "MEMRDD", cTmpName, cAlias, .F., .F. )
FOR i := 1 TO Len( aR[ 2 ] )
dbAppend()
FOR j := 1 TO Min( Len( aFN ), Len( aR[ 2 ][ i ] ) )
FieldPut( j, aR[ 2 ][ i ][ j ] )
NEXT
NEXT
RECOVER
RETURN .F.
END SEQUENCE
RETURN .T.
/* ======================================================================
* Standalone helper functions called by TSqlIndex
* ====================================================================== */
/* Evaluate expression node for index scan operations */
FUNCTION SqlEvalExprNode( xNode, aTables, aParams, nPI )
LOCAL oExec
oExec := TSqlExecutor():New( { => }, aParams )
oExec:aTables := aTables
RETURN oExec:EvalExpr( xNode )
/* Fetch a row array for index scan operations */
FUNCTION SqlFetchRowArr( aRE, aTables, aParams )
LOCAL oExec
oExec := TSqlExecutor():New( { => }, aParams )
oExec:aTables := aTables
RETURN oExec:FetchRow( aRE )
/* Join recurse called from TSqlIndex */
FUNCTION SqlJoinRecurse( aJoins, nIdx, aTables, xWhere, aRE, aRows, aParams, oIndex )
LOCAL oExec
oExec := TSqlExecutor():New( { => }, aParams )
oExec:aTables := aTables
oExec:oIndex := oIndex
oExec:JoinRecurse( aJoins, nIdx, xWhere, aRE, @aRows, NIL )
RETURN NIL
/* INTERSECT: keep only rows in both sets */
FUNCTION SqlDoIntersect( aRows1, aRows2 )
LOCAL aResult := {}, hKeys2 := { => }, hEmitted := { => }, i, cKey
LOCAL oSort := TSqlSort():New()
FOR i := 1 TO Len( aRows2 )
cKey := oSort:RowKey( aRows2[ i ] )
hKeys2[ cKey ] := .T.
NEXT
/* SQL standard: INTERSECT is DISTINCT by default — skip rows whose
* composite key was already emitted, not just unmatched ones.
* INTERSECT ALL (retain duplicates) would skip this dedup; we
* don't currently expose that spelling so the plain INTERSECT
* follows the default spec. */
FOR i := 1 TO Len( aRows1 )
cKey := oSort:RowKey( aRows1[ i ] )
IF hb_HHasKey( hKeys2, cKey ) .AND. ! hb_HHasKey( hEmitted, cKey )
AAdd( aResult, aRows1[ i ] )
hEmitted[ cKey ] := .T.
ENDIF
NEXT
RETURN aResult
/* EXCEPT: keep only rows in first that are not in second.
* SQL spec: EXCEPT is DISTINCT by default. */
FUNCTION SqlDoExcept( aRows1, aRows2 )
LOCAL aResult := {}, hKeys2 := { => }, hEmitted := { => }, i, cKey
LOCAL oSort := TSqlSort():New()
FOR i := 1 TO Len( aRows2 )
cKey := oSort:RowKey( aRows2[ i ] )
hKeys2[ cKey ] := .T.
NEXT
FOR i := 1 TO Len( aRows1 )
cKey := oSort:RowKey( aRows1[ i ] )
IF ! hb_HHasKey( hKeys2, cKey ) .AND. ! hb_HHasKey( hEmitted, cKey )
AAdd( aResult, aRows1[ i ] )
hEmitted[ cKey ] := .T.
ENDIF
NEXT
RETURN aResult
/* Materialize a subquery into a temp DBF */
FUNCTION SqlMaterializeSubquery( xSubQ, cAlias, aParams )
LOCAL aSub, aFN, aRows2, aStruct, cTmpFile, i, j
LOCAL cType, nWidth, nDec, xVal
aSub := TSqlExecutor():New( xSubQ[ 2 ], aParams ):Run()
IF ValType( aSub ) != "A" .OR. Len( aSub ) < 2
RETURN { "__EMPTY__", cAlias, "" }
ENDIF
aFN := aSub[ 1 ]
aRows2 := aSub[ 2 ]
aStruct := {}
FOR i := 1 TO Len( aFN )
cType := "C"
nWidth := 40
nDec := 0
IF Len( aRows2 ) > 0 .AND. i <= Len( aRows2[ 1 ] )
xVal := aRows2[ 1 ][ i ]
IF ValType( xVal ) == "N"
cType := "N"
nWidth := 18
nDec := 4
ELSEIF ValType( xVal ) == "D"
cType := "D"
nWidth := 8
ELSEIF ValType( xVal ) == "L"
cType := "L"
nWidth := 1
ELSEIF ValType( xVal ) == "T"
cType := "T"
nWidth := 8
ENDIF
ENDIF
AAdd( aStruct, { PadR( Upper( aFN[ i ] ), 10 ), cType, nWidth, nDec } )
NEXT
cTmpFile := "__drv_" + Lower( cAlias )
/* MEMRDD in-memory temp — avoids dbCreate + FErase disk syscalls. */
dbCreate( "mem:" + cTmpFile, aStruct, "MEMRDD" )
/* Open under a scratch alias just to feed SqlBulkInsert, then close
* and re-open under the user-visible alias so RunSelect's open
* loop finds it via Select(cAlias) without trying to dbUseArea on
* a non-existent disk file. */
dbUseArea( .T., "MEMRDD", "mem:" + cTmpFile, "__DRVTMP", .F., .F. )
/* Go RTL SqlBulkInsert — subquery driving-table materialization. */
SqlBulkInsert( aRows2 )
dbSelectArea( Select( "__DRVTMP" ) )
dbCloseArea()
dbUseArea( .T., "MEMRDD", "mem:" + cTmpFile, cAlias, .T., .F. )
RETURN { cTmpFile, cAlias, "" }
/* SqlSetAutoInc / SqlGetAutoIncFields used to live here as PRG
* helpers around the s_hAutoInc STATIC. They're now Go HB_FUNCs
* backed by an RWMutex-protected map (hbrtl/sqlglobals.go);
* PRG callers see the same signature and semantics, just thread-
* safe under concurrent CREATE TABLE + INSERT. The old PRG
* implementation below is dead, kept commented for one release
* cycle so any porter looking for `SqlSetAutoInc` can grep it.
*
* STATIC s_hAutoInc := NIL — gone (Go-side now)
*
* FUNCTION SqlSetAutoInc(cTable, cField) — Go HB_FUNC
* FUNCTION SqlGetAutoIncFields(cTable) — Go HB_FUNC
*
* The two-line guard branch the original maintained to compare
* against NIL is replaced by Go's `if v, ok := m[k]; ok {...}`
* pattern in hbSqlGetAutoIncFields — same semantics, no per-call
* NIL-check on the PRG side. */
FUNCTION SqlGetMaxFieldVal( cAlias, cField )
LOCAL nWA, nSaved, nFPos, nMax := 0, xVal
LOCAL nSavedRec
nSaved := Select()
nWA := Select( cAlias )
IF nWA > 0
dbSelectArea( nWA )
nSavedRec := RecNo()
nFPos := FieldPos( cField )
IF nFPos > 0
dbGoTop()
WHILE ! Eof()
xVal := FieldGet( nFPos )
IF ValType( xVal ) == "N" .AND. xVal > nMax
nMax := xVal
ENDIF
dbSkip()
ENDDO
ENDIF
dbGoto( nSavedRec )
ENDIF
dbSelectArea( nSaved )
RETURN nMax
/* Build a unique cache key from a subquery hash structure */
FUNCTION SqlSubqueryKey( hSub )
LOCAL cKey, cType, i, aCols, aTbls
IF ValType( hSub ) != "H"
RETURN "??"
ENDIF
cKey := ""
IF hb_HHasKey( hSub, "type" )
cKey += hSub[ "type" ]
ENDIF
IF hb_HHasKey( hSub, "tables" )
aTbls := hSub[ "tables" ]
IF ValType( aTbls ) == "A"
FOR i := 1 TO Len( aTbls )
IF ValType( aTbls[ i ] ) == "A" .AND. Len( aTbls[ i ] ) >= 1
cKey += "|T:" + aTbls[ i ][ 1 ]
ENDIF
NEXT
ENDIF
ENDIF
IF hb_HHasKey( hSub, "columns" )
aCols := hSub[ "columns" ]
IF ValType( aCols ) == "A"
FOR i := 1 TO Len( aCols )
IF ValType( aCols[ i ] ) == "A" .AND. Len( aCols[ i ] ) >= 1
cKey += "|C:" + SqlExprName( aCols[ i ][ 1 ] )
ENDIF
NEXT
ENDIF
ENDIF
IF hb_HHasKey( hSub, "where" ) .AND. hSub[ "where" ] != NIL
cKey += "|W:" + SqlExprName( hSub[ "where" ] )
ENDIF
RETURN cKey
/* ======================================================================
* Recursive CTE materialization (SQL:1999)
* ====================================================================== */
/*
* Materialize recursive CTEs using an in-memory working table approach.
*
* Algorithm:
* 1. Execute anchor query → get initial rows
* 2. Write initial rows to temp DBF
* 3. Loop: scan temp DBF rows, for each row evaluate recursive expression
* to produce new rows. Append new rows to temp DBF.
* 4. Stop when no new rows are produced or max iterations reached.
* 5. Leave temp DBF open for the main query to use.
*
* Key insight: instead of creating a new TSqlExecutor for the recursive part
* (which causes workarea conflicts), we iterate the temp DBF directly and
* build new rows by evaluating the recursive SELECT columns.
*/
METHOD MaterializeRecursiveCTE( aCTE ) CLASS TSqlExecutor
LOCAL i, cName, xSubQ, hSubQ
LOCAL aSub, aFN, aDataRows, aStruct
LOCAL cTmpFile, cAlias, j, k, nIter
LOCAL cType, nWidth, nDec, xVal
LOCAL nExistWA, lReplaced
LOCAL aNewRows, lHasUnionAll
LOCAL hRecQuery, aPrevRows, nPrevCount
LOCAL aOneRow, aOneRow2, lPass, nPI, aNewRow, aCols, nPI2, nStart, xWR, xCV
FOR i := 1 TO Len( aCTE )
cName := Upper( aCTE[ i ][ 1 ] )
xSubQ := aCTE[ i ][ 2 ]
IF ValType( xSubQ ) != "A" .OR. xSubQ[ 1 ] != ND_SUB .OR. xSubQ[ 2 ] == NIL
LOOP
ENDIF
hSubQ := xSubQ[ 2 ]
/* Check if the CTE subquery has a UNION ALL (signals recursion) */
lHasUnionAll := .F.
IF ValType( hSubQ ) == "H" .AND. hb_HHasKey( hSubQ, "union" ) .AND. hSubQ[ "union" ] != NIL
IF hb_HHasKey( hSubQ[ "union" ], "union_all" ) .AND. hSubQ[ "union" ][ "union_all" ]
lHasUnionAll := .T.
ENDIF
ENDIF
IF ! lHasUnionAll
/* Not actually recursive, use normal CTE */
::MaterializeCTE( { aCTE[ i ] } )
LOOP
ENDIF
/* Save and detach the recursive (UNION ALL) part so the anchor
* query does not attempt to open the CTE table that has not been
* materialised yet. */
hRecQuery := hSubQ[ "union" ]
hSubQ[ "union" ] := NIL
/* Execute anchor query (the first SELECT before UNION ALL) */
aSub := TSqlExecutor():New( hSubQ, ::aParams, ::oSession ):Run()
/* Restore the union reference for later use */
hSubQ[ "union" ] := hRecQuery
IF ValType( aSub ) != "A" .OR. Len( aSub ) < 2
LOOP
ENDIF
aFN := aSub[ 1 ]
aDataRows := aSub[ 2 ]
/* Apply CTE column aliases if specified: WITH RECURSIVE seq(n) AS ... */
IF Len( aCTE[ i ] ) >= 3 .AND. ValType( aCTE[ i ][ 3 ] ) == "A" .AND. Len( aCTE[ i ][ 3 ] ) > 0
FOR j := 1 TO Min( Len( aCTE[ i ][ 3 ] ), Len( aFN ) )
aFN[ j ] := Upper( aCTE[ i ][ 3 ][ j ] )
NEXT
ENDIF
/* Build structure from anchor result */
aStruct := {}
FOR j := 1 TO Len( aFN )
cType := "C"
nWidth := 40
nDec := 0
IF Len( aDataRows ) > 0 .AND. j <= Len( aDataRows[ 1 ] )
xVal := aDataRows[ 1 ][ j ]
IF ValType( xVal ) == "N"
cType := "N"
nWidth := 18
nDec := 4
ELSEIF ValType( xVal ) == "D"
cType := "D"
nWidth := 8
ELSEIF ValType( xVal ) == "L"
cType := "L"
nWidth := 1
ENDIF
ENDIF
AAdd( aStruct, { PadR( Upper( aFN[ j ] ), 10 ), cType, nWidth, nDec } )
NEXT
/*
* Pure in-memory recursive iteration.
* No temp DBF for the recursive loop — just arrays.
* At the end, write ALL accumulated rows to temp DBF once.
*/
hRecQuery := hSubQ[ "union" ]
nIter := 0
aPrevRows := AClone( aDataRows )
/* Iteration cap — legitimate recursive queries easily exceed 50
* rows (`seq 1..N` for N>50 clipped silently at 51). SQL Server
* defaults to 100, PostgreSQL has no hard cap. Keep a defensive
* ceiling so a cycle-without-termination query doesn't lock up
* the process, but raise it high enough for real workloads. */
WHILE nIter < 10000 .AND. Len( aPrevRows ) > 0
nIter++
aNewRows := {}
IF hRecQuery != NIL .AND. hb_HHasKey( hRecQuery, "columns" )
aCols := hRecQuery[ "columns" ]
/*
* Check if this recursive part has a JOIN (FROM clause with tables).
* If so, perform an in-memory nested-loop JOIN between the external
* table(s) and aPrevRows (the CTE working set from last iteration).
*
* Example: SELECT e.id, e.name FROM employees e JOIN org o ON e.mgr_id = o.id
* "employees" is a real DBF table; "org" is aPrevRows from previous iteration.
*/
IF hb_HHasKey( hRecQuery, "joins" ) .AND. hRecQuery[ "joins" ] != NIL .AND. ;
Len( hRecQuery[ "joins" ] ) > 0
aNewRows := RecCteJoin( hRecQuery, aFN, aPrevRows, cName )
ELSE
/* Simple recursive step (no JOIN): evaluate expressions
* directly against each row in aPrevRows */
FOR j := 1 TO Len( aPrevRows )
lPass := .T.
IF hb_HHasKey( hRecQuery, "where" ) .AND. hRecQuery[ "where" ] != NIL
xWR := SqlEvalRowExpr( hRecQuery[ "where" ], aFN, aPrevRows[ j ] )
lPass := SqlIsTrue( xWR )
ENDIF
IF lPass
aNewRow := {}
FOR k := 1 TO Len( aCols )
xCV := SqlEvalRowExpr( aCols[ k ][ 1 ], aFN, aPrevRows[ j ] )
AAdd( aNewRow, xCV )
NEXT
AAdd( aNewRows, aNewRow )
ENDIF
NEXT
ENDIF
ENDIF
IF Len( aNewRows ) == 0
EXIT
ENDIF
/* Accumulate new rows */
FOR j := 1 TO Len( aNewRows )
AAdd( aDataRows, aNewRows[ j ] )
NEXT
/* New rows become the working set for next iteration */
aPrevRows := AClone( aNewRows )
ENDDO
/* Write ALL accumulated rows to temp DBF once */
cTmpFile := "__cte_" + Lower( cName )
cAlias := Upper( cName )
nExistWA := Select( cAlias )
IF nExistWA > 0
dbSelectArea( nExistWA )
dbCloseArea()
ENDIF
/* MEMRDD in-memory temp for CTE — no file create/delete. */
BEGIN SEQUENCE
dbCreate( "mem:" + cTmpFile, aStruct, "MEMRDD" )
RECOVER
END SEQUENCE
BEGIN SEQUENCE
dbUseArea( .T., "MEMRDD", "mem:" + cTmpFile, cAlias, .F., .F. )
/* Go RTL SqlBulkInsert — CTE materialization path. */
SqlBulkInsert( aDataRows )
RECOVER
END SEQUENCE
/* Do NOT rewrite aTables[j][1] to cTmpFile here. RunSelect's
* open loop expects the original CTE name in aTables; with that
* name (a) `Select(cName)` directly returns the area we just
* opened, or (b) the MEMRDD fallback at the end of the open
* loop attaches a second workarea under the user's alias when
* the FROM clause aliased the CTE (e.g. `FROM sub s` with
* recursive CTE + JOIN to a sibling table previously failed
* with "Table '__cte_sub' does not exist"). */
NEXT
RETURN NIL
/* ======================================================================
* Window function evaluation (SQL:2003)
* ====================================================================== */
METHOD ApplyWindowFunctions( aRows, aFN, aCols ) CLASS TSqlExecutor
LOCAL i, j, k, nColIdx, xExpr
LOCAL cFunc, aPartBy, aOrdBy, aFuncArgs
LOCAL aPartitions, cPartKey, aPartIdx
LOCAL aSorted, aIdxMap, nPartCol
LOCAL nRank, nDenseRank, nRowNum
LOCAL xPrev, xCurr, nTies
LOCAL nLagLead, nArgCol, xDefault
LOCAL nRunSum, nRunCount
LOCAL aWinCols, nWC
LOCAL hFrame, nFS, nFE, m, xVal, xMin, xMax, lDefaultFrame, lWholePartition
LOCAL aPartColIdx, aSortSpec, nOrdCol
LOCAL nLeftOff, nRightOff, lBoundsOk
/* Scan for window function columns */
aWinCols := {}
FOR i := 1 TO Len( aCols )
xExpr := aCols[ i ][ 1 ]
IF ValType( xExpr ) == "A" .AND. xExpr[ 1 ] == ND_WINDOW
AAdd( aWinCols, i )
ENDIF
NEXT
IF Len( aWinCols ) == 0 .OR. Len( aRows ) == 0
RETURN NIL
ENDIF
FOR nWC := 1 TO Len( aWinCols )
nColIdx := aWinCols[ nWC ]
xExpr := aCols[ nColIdx ][ 1 ]
cFunc := Upper( xExpr[ 2 ] )
aFuncArgs := xExpr[ 3 ]
aPartBy := xExpr[ 4 ]
aOrdBy := xExpr[ 5 ]
/* Frame spec in optional 6th slot (added by parser) */
hFrame := NIL
IF Len( xExpr ) >= 6
hFrame := xExpr[ 6 ]
ENDIF
/* Resolve PARTITION BY columns once, then delegate the row-index
* grouping to Go RTL SqlWindowPartitions — removes N·M per-row
* Go↔PRG boundary crossings for SqlValToStr / hb_HHasKey / AAdd. */
aPartColIdx := {}
IF ValType( aPartBy ) == "A"
FOR j := 1 TO Len( aPartBy )
nPartCol := SqlFindColIdx( aPartBy[ j ], aFN )
IF nPartCol == 0
nPartCol := SqlFindColIdx2( SqlExprName( aPartBy[ j ] ), aFN )
ENDIF
IF nPartCol > 0
AAdd( aPartColIdx, nPartCol )
ENDIF
NEXT
ENDIF
aPartitions := SqlWindowPartitions( aRows, aPartColIdx )
/* Pre-resolve ORDER BY column indices once per window column —
* Go SqlWindowSortPartition reads the resolved {nCol, lDesc}
* pairs directly, so every partition sort avoids the repeated
* SqlFindColIdx linear scan inside per-comparison PRG blocks. */
aSortSpec := {}
IF ValType( aOrdBy ) == "A" .AND. Len( aOrdBy ) > 0
FOR j := 1 TO Len( aOrdBy )
nOrdCol := SqlFindColIdx( aOrdBy[ j ][ 1 ], aFN )
IF nOrdCol == 0
nOrdCol := SqlFindColIdx2( SqlExprName( aOrdBy[ j ][ 1 ] ), aFN )
ENDIF
IF nOrdCol > 0
AAdd( aSortSpec, { nOrdCol, aOrdBy[ j ][ 2 ] == "DESC" } )
ENDIF
NEXT
ENDIF
/* Process each partition */
FOR EACH aPartIdx IN aPartitions
/* Sort partition indices by ORDER BY columns (Go RTL). */
IF Len( aSortSpec ) > 0
SqlWindowSortPartition( aRows, aPartIdx, aSortSpec )
ENDIF
/* Compute window function for each row in the partition.
* ROW_NUMBER/RANK/DENSE_RANK all go through one Go RTL call
* that walks the partition and writes the rank column —
* removes per-row SqlWinRowsEqual + PRG indexing overhead. */
DO CASE
CASE cFunc == "ROW_NUMBER" .OR. cFunc == "RANK" .OR. cFunc == "DENSE_RANK"
SqlWindowAssignRank( aRows, aPartIdx, aSortSpec, nColIdx, cFunc )
CASE cFunc == "LAG"
nLagLead := 1
IF Len( aFuncArgs ) >= 2 .AND. aFuncArgs[ 2 ][ 1 ] == ND_LIT
nLagLead := Int( SqlCoerceNum( aFuncArgs[ 2 ][ 2 ] ) )
ENDIF
nArgCol := 0
IF Len( aFuncArgs ) >= 1
nArgCol := SqlFindColIdx( aFuncArgs[ 1 ], aFN )
IF nArgCol == 0
nArgCol := SqlFindColIdx2( SqlExprName( aFuncArgs[ 1 ] ), aFN )
ENDIF
ENDIF
xDefault := NIL
IF Len( aFuncArgs ) >= 3
/* Default arg can be a literal (`-1`, `'end'`) but the
* lexer parses `-1` as ND_UNI(-, ND_LIT(1)), not a bare
* ND_LIT — so a flat type-check would silently drop the
* default. Run the value through SqlEvalRowExpr against
* an empty row so any constant expression (including
* unary minus and CAST) collapses to its scalar form. */
xDefault := SqlEvalRowExpr( aFuncArgs[ 3 ], {}, {} )
ENDIF
FOR k := 1 TO Len( aPartIdx )
IF nColIdx <= Len( aRows[ aPartIdx[ k ] ] )
IF k - nLagLead >= 1 .AND. nArgCol > 0 .AND. ;
nArgCol <= Len( aRows[ aPartIdx[ k - nLagLead ] ] )
aRows[ aPartIdx[ k ] ][ nColIdx ] := aRows[ aPartIdx[ k - nLagLead ] ][ nArgCol ]
ELSE
aRows[ aPartIdx[ k ] ][ nColIdx ] := xDefault
ENDIF
ENDIF
NEXT
CASE cFunc == "LEAD"
nLagLead := 1
IF Len( aFuncArgs ) >= 2 .AND. aFuncArgs[ 2 ][ 1 ] == ND_LIT
nLagLead := Int( SqlCoerceNum( aFuncArgs[ 2 ][ 2 ] ) )
ENDIF
nArgCol := 0
IF Len( aFuncArgs ) >= 1
nArgCol := SqlFindColIdx( aFuncArgs[ 1 ], aFN )
IF nArgCol == 0
nArgCol := SqlFindColIdx2( SqlExprName( aFuncArgs[ 1 ] ), aFN )
ENDIF
ENDIF
xDefault := NIL
IF Len( aFuncArgs ) >= 3
/* Default arg can be a literal (`-1`, `'end'`) but the
* lexer parses `-1` as ND_UNI(-, ND_LIT(1)), not a bare
* ND_LIT — so a flat type-check would silently drop the
* default. Run the value through SqlEvalRowExpr against
* an empty row so any constant expression (including
* unary minus and CAST) collapses to its scalar form. */
xDefault := SqlEvalRowExpr( aFuncArgs[ 3 ], {}, {} )
ENDIF
FOR k := 1 TO Len( aPartIdx )
IF nColIdx <= Len( aRows[ aPartIdx[ k ] ] )
IF k + nLagLead <= Len( aPartIdx ) .AND. nArgCol > 0 .AND. ;
nArgCol <= Len( aRows[ aPartIdx[ k + nLagLead ] ] )
aRows[ aPartIdx[ k ] ][ nColIdx ] := aRows[ aPartIdx[ k + nLagLead ] ][ nArgCol ]
ELSE
aRows[ aPartIdx[ k ] ][ nColIdx ] := xDefault
ENDIF
ENDIF
NEXT
CASE cFunc == "SUM" .OR. cFunc == "AVG" .OR. cFunc == "COUNT" .OR. ;
cFunc == "MIN" .OR. cFunc == "MAX"
/* Frame-aware aggregate window functions.
* Default frame (no spec): UNBOUNDED PRECEDING to CURRENT ROW.
* Explicit: ROWS BETWEEN n PRECEDING AND m FOLLOWING, etc. */
nArgCol := 0
IF Len( aFuncArgs ) >= 1
nArgCol := SqlFindColIdx( aFuncArgs[ 1 ], aFN )
IF nArgCol == 0
nArgCol := SqlFindColIdx2( SqlExprName( aFuncArgs[ 1 ] ), aFN )
ENDIF
ENDIF
/* Detect default frame (UNBOUNDED PRECEDING to CURRENT ROW)
* which can use the O(N) incremental running-sum path instead
* of the O(N²) general per-row-frame-aggregate. */
/* Default frame = UNBOUNDED PRECEDING to CURRENT ROW.
* This covers: no frame spec, or explicit ROWS UNBOUNDED
* PRECEDING (without BETWEEN or with implied CURRENT ROW end).
* The incremental O(N) path handles this; the general frame
* loop is only needed for custom boundaries like
* ROWS BETWEEN 6 PRECEDING AND CURRENT ROW. */
lDefaultFrame := .T.
IF hFrame != NIL .AND. ValType( hFrame ) == "H"
IF hb_HHasKey( hFrame, "end" ) .AND. ;
! ( "CURRENT ROW" $ hFrame[ "end" ] )
lDefaultFrame := .F.
ENDIF
IF hb_HHasKey( hFrame, "start" ) .AND. ;
! ( "UNBOUNDED PRECEDING" $ hFrame[ "start" ] )
lDefaultFrame := .F.
ENDIF
ENDIF
IF lDefaultFrame
/* O(N) incremental path — accumulate, then write.
*
* Per SQL:2003 the default frame is
* RANGE UNBOUNDED PRECEDING (running) — when ORDER BY is present
* whole-partition — when ORDER BY is absent
*
* lWholePartition captures the ORDER-BY-absent case: we
* still make one pass but write the final aggregate to
* every row in the partition instead of the running
* value at each position. Previously the running value
* was always written, so `AVG(sal) OVER ()` returned a
* cumulative average (wrong per spec, matched Oracle's
* buggy output before 12c but diverged from Postgres /
* SQL Server / the SQL standard). */
lWholePartition := ( Len( aSortSpec ) == 0 )
nRunSum := 0
nRunCount := 0
xMin := NIL
xMax := NIL
FOR k := 1 TO Len( aPartIdx )
IF cFunc == "COUNT" .AND. nArgCol == 0
nRunCount++
ELSEIF nArgCol > 0 .AND. nArgCol <= Len( aRows[ aPartIdx[ k ] ] )
xVal := aRows[ aPartIdx[ k ] ][ nArgCol ]
IF xVal != NIL
nRunCount++
nRunSum += SqlCoerceNum( xVal )
IF xMin == NIL .OR. SqlCmpLt( xVal, xMin )
xMin := xVal
ENDIF
IF xMax == NIL .OR. SqlCmpLt( xMax, xVal )
xMax := xVal
ENDIF
ENDIF
ENDIF
IF ! lWholePartition .AND. nColIdx <= Len( aRows[ aPartIdx[ k ] ] )
DO CASE
CASE cFunc == "SUM"
aRows[ aPartIdx[ k ] ][ nColIdx ] := nRunSum
CASE cFunc == "AVG"
aRows[ aPartIdx[ k ] ][ nColIdx ] := iif( nRunCount > 0, nRunSum / nRunCount, NIL )
CASE cFunc == "COUNT"
aRows[ aPartIdx[ k ] ][ nColIdx ] := nRunCount
CASE cFunc == "MIN"
aRows[ aPartIdx[ k ] ][ nColIdx ] := xMin
CASE cFunc == "MAX"
aRows[ aPartIdx[ k ] ][ nColIdx ] := xMax
ENDCASE
ENDIF
NEXT
IF lWholePartition
/* Write the final (partition-wide) aggregate to every row. */
FOR k := 1 TO Len( aPartIdx )
IF nColIdx <= Len( aRows[ aPartIdx[ k ] ] )
DO CASE
CASE cFunc == "SUM"
aRows[ aPartIdx[ k ] ][ nColIdx ] := nRunSum
CASE cFunc == "AVG"
aRows[ aPartIdx[ k ] ][ nColIdx ] := ;
iif( nRunCount > 0, nRunSum / nRunCount, NIL )
CASE cFunc == "COUNT"
aRows[ aPartIdx[ k ] ][ nColIdx ] := nRunCount
CASE cFunc == "MIN"
aRows[ aPartIdx[ k ] ][ nColIdx ] := xMin
CASE cFunc == "MAX"
aRows[ aPartIdx[ k ] ][ nColIdx ] := xMax
ENDCASE
ENDIF
NEXT
ENDIF
ELSE
/* General frame: try the O(N) fast path in Go RTL.
* SUM/AVG/COUNT go through a prefix-sum sweep; MIN/MAX
* through a monotonic deque. The RTL returns .F. when
* it can't handle the value types (e.g., MIN/MAX over
* string / date columns) so we fall through to the
* O(N·W) loop below — keeps correctness while the
* common numeric case wins. */
IF cFunc == "SUM" .OR. cFunc == "AVG" .OR. cFunc == "COUNT" .OR. ;
cFunc == "MIN" .OR. cFunc == "MAX"
lBoundsOk := .T.
nLeftOff := SqlFrameOffsetEncode( hFrame[ "start" ], @lBoundsOk )
IF lBoundsOk
nRightOff := SqlFrameOffsetEncode( hFrame[ "end" ], @lBoundsOk )
ENDIF
IF lBoundsOk
IF SqlWindowSlideAgg( aRows, aPartIdx, nArgCol, nColIdx, ;
cFunc, nLeftOff, nRightOff )
LOOP /* done with this partition */
ENDIF
ENDIF
ENDIF
/* General frame path — O(N*W) where W = frame width */
FOR k := 1 TO Len( aPartIdx )
nFS := 1
nFE := k
IF hb_HHasKey( hFrame, "start" )
nFS := SqlFrameOffset( hFrame[ "start" ], k, Len( aPartIdx ) )
ENDIF
IF hb_HHasKey( hFrame, "end" )
nFE := SqlFrameOffset( hFrame[ "end" ], k, Len( aPartIdx ) )
ENDIF
IF nFS < 1
nFS := 1
ENDIF
IF nFE > Len( aPartIdx )
nFE := Len( aPartIdx )
ENDIF
nRunSum := 0
nRunCount := 0
xMin := NIL
xMax := NIL
FOR m := nFS TO nFE
IF cFunc == "COUNT" .AND. nArgCol == 0
/* COUNT(*) */
nRunCount++
ELSEIF nArgCol > 0 .AND. nArgCol <= Len( aRows[ aPartIdx[ m ] ] )
xVal := aRows[ aPartIdx[ m ] ][ nArgCol ]
IF xVal != NIL
nRunCount++
nRunSum += SqlCoerceNum( xVal )
IF xMin == NIL .OR. SqlCmpLt( xVal, xMin )
xMin := xVal
ENDIF
IF xMax == NIL .OR. SqlCmpLt( xMax, xVal )
xMax := xVal
ENDIF
ENDIF
ENDIF
NEXT
IF nColIdx <= Len( aRows[ aPartIdx[ k ] ] )
DO CASE
CASE cFunc == "SUM"
aRows[ aPartIdx[ k ] ][ nColIdx ] := nRunSum
CASE cFunc == "AVG"
aRows[ aPartIdx[ k ] ][ nColIdx ] := iif( nRunCount > 0, nRunSum / nRunCount, NIL )
CASE cFunc == "COUNT"
aRows[ aPartIdx[ k ] ][ nColIdx ] := nRunCount
CASE cFunc == "MIN"
aRows[ aPartIdx[ k ] ][ nColIdx ] := xMin
CASE cFunc == "MAX"
aRows[ aPartIdx[ k ] ][ nColIdx ] := xMax
ENDCASE
ENDIF
NEXT
ENDIF /* lDefaultFrame */
ENDCASE
NEXT
NEXT
RETURN NIL
/* ======================================================================
* TRUNCATE TABLE executor
* ====================================================================== */
METHOD RunTruncate() CLASS TSqlExecutor
LOCAL cTable, nWA
cTable := ::hQuery[ "table" ]
/* Close if open */
nWA := Select( cTable )
IF nWA > 0
dbSelectArea( nWA )
dbCloseArea()
ENDIF
BEGIN SEQUENCE
USE ( Lower( cTable ) + ".dbf" ) NEW EXCLUSIVE
dbGoTop()
WHILE ! Eof()
dbDelete()
dbSkip()
ENDDO
dbCloseArea()
RECOVER
RETURN ::MakeError( SQL_ERR_LOCKED, "TRUNCATE TABLE failed: " + cTable )
END SEQUENCE
RETURN { { "result" }, { { "Table " + cTable + " truncated" } } }
/* ======================================================================
* MERGE (UPSERT) executor (SQL:2003)
* ====================================================================== */
METHOD RunMerge() CLASS TSqlExecutor
LOCAL cTarget, cSource, cSrcAlias, xOnCond
LOCAL aUpdSet, aInsFlds, aInsVals
LOCAL lHasMatched, lHasNotMatched
LOCAL nSrcWA, nTgtWA, nSaved, nAffected
LOCAL lMatched, i, nFPos, xVal
LOCAL xMatchCond, xNotMatchCond, lDelete
LOCAL lValid, cInsFld
cTarget := ::hQuery[ "target" ]
cSource := ::hQuery[ "source" ]
cSrcAlias := ""
IF hb_HHasKey( ::hQuery, "source_alias" )
cSrcAlias := ::hQuery[ "source_alias" ]
ENDIF
xOnCond := ::hQuery[ "on" ]
aUpdSet := ::hQuery[ "update_set" ]
aInsFlds := ::hQuery[ "insert_fields" ]
aInsVals := ::hQuery[ "insert_values" ]
lHasMatched := ::hQuery[ "has_matched" ]
lHasNotMatched := ::hQuery[ "has_not_matched" ]
/* Extended MERGE clauses the parser already captures but the
* executor was ignoring — WHEN MATCHED AND / WHEN NOT MATCHED
* AND filter conditions, plus WHEN MATCHED THEN DELETE. */
xMatchCond := NIL
xNotMatchCond := NIL
lDelete := .F.
IF hb_HHasKey( ::hQuery, "match_condition" )
xMatchCond := ::hQuery[ "match_condition" ]
ENDIF
IF hb_HHasKey( ::hQuery, "not_match_condition" )
xNotMatchCond := ::hQuery[ "not_match_condition" ]
ENDIF
IF hb_HHasKey( ::hQuery, "matched_delete" )
lDelete := ::hQuery[ "matched_delete" ]
ENDIF
nAffected := 0
::aTables := { { cTarget, cTarget, "" }, { cSource, iif( Empty( cSrcAlias ), cSource, cSrcAlias ), "" } }
/* Open source */
nSrcWA := Select( iif( Empty( cSrcAlias ), cSource, cSrcAlias ) )
IF nSrcWA == 0
BEGIN SEQUENCE
dbUseArea( .T., "DBFNTX", Lower( cSource ) + ".dbf", ;
iif( Empty( cSrcAlias ), cSource, cSrcAlias ), .T., .T. )
RECOVER
RETURN ::MakeError( SQL_ERR_NO_TABLE, "MERGE: cannot open source " + cSource )
END SEQUENCE
ENDIF
nSrcWA := Select( iif( Empty( cSrcAlias ), cSource, cSrcAlias ) )
/* Open target */
nTgtWA := Select( cTarget )
IF nTgtWA == 0
BEGIN SEQUENCE
dbUseArea( .T., "DBFNTX", Lower( cTarget ) + ".dbf", cTarget, .F., .F. )
RECOVER
RETURN ::MakeError( SQL_ERR_NO_TABLE, "MERGE: cannot open target " + cTarget )
END SEQUENCE
ENDIF
nTgtWA := Select( cTarget )
nSaved := Select()
/* For each source row */
dbSelectArea( nSrcWA )
dbGoTop()
DO WHILE ! Eof()
lMatched := .F.
/* Scan target for match */
dbSelectArea( nTgtWA )
dbGoTop()
DO WHILE ! Eof()
IF SqlIsTrue( ::EvalExpr( xOnCond ) )
lMatched := .T.
EXIT
ENDIF
dbSkip()
ENDDO
IF lMatched .AND. lHasMatched
/* Apply optional WHEN MATCHED AND <cond>. ON match already
* passed; the AND filter further narrows which matched
* rows get updated / deleted. */
dbSelectArea( nTgtWA )
IF xMatchCond == NIL .OR. SqlIsTrue( ::EvalExpr( xMatchCond ) )
IF dbRLock( RecNo() )
IF lDelete
/* WHEN MATCHED THEN DELETE — mark the row; cleanup
* and FK-cascade happen at dbCommit time. */
dbDelete()
ELSE
FOR i := 1 TO Len( aUpdSet )
nFPos := FieldPos( aUpdSet[ i ][ 1 ] )
IF nFPos > 0
xVal := ::EvalExpr( aUpdSet[ i ][ 2 ] )
FieldPut( nFPos, xVal )
ENDIF
NEXT
ENDIF
dbRUnlock( RecNo() )
nAffected++
ENDIF
ENDIF
ELSEIF ! lMatched .AND. lHasNotMatched
/* WHEN NOT MATCHED [AND <cond>] THEN INSERT */
IF xNotMatchCond != NIL .AND. ;
! SqlIsTrue( ::EvalExpr( xNotMatchCond ) )
/* condition false — skip this source row */
ELSE
dbSelectArea( nTgtWA )
dbAppend()
IF Len( aInsFlds ) > 0
FOR i := 1 TO Min( Len( aInsFlds ), Len( aInsVals ) )
nFPos := FieldPos( aInsFlds[ i ] )
IF nFPos > 0
xVal := ::EvalExpr( aInsVals[ i ] )
FieldPut( nFPos, xVal )
ENDIF
NEXT
ELSE
FOR i := 1 TO Min( FCount(), Len( aInsVals ) )
xVal := ::EvalExpr( aInsVals[ i ] )
FieldPut( i, xVal )
NEXT
ENDIF
/* Enforce UNIQUE on inserted row — mirrors RunInsert.
* Without this a MERGE could quietly produce a duplicate
* that regular INSERT would have rejected. */
lValid := .T.
IF Len( aInsFlds ) > 0
FOR i := 1 TO Len( aInsFlds )
cInsFld := aInsFlds[ i ]
nFPos := FieldPos( cInsFld )
IF nFPos > 0 .AND. ;
! SqlValidateUnique( cTarget, cInsFld, FieldGet( nFPos ), RecNo() )
lValid := .F.
EXIT
ENDIF
NEXT
ELSE
FOR i := 1 TO FCount()
IF ! SqlValidateUnique( cTarget, FieldName( i ), FieldGet( i ), RecNo() )
lValid := .F.
EXIT
ENDIF
NEXT
ENDIF
IF ! lValid
dbDelete()
ELSE
nAffected++
ENDIF
ENDIF
ENDIF
dbSelectArea( nSrcWA )
dbSkip()
ENDDO
dbSelectArea( nTgtWA )
dbCommit()
dbSelectArea( nSaved )
RETURN { { "affected_rows" }, { { nAffected } } }
/* ======================================================================
* Window function helper: compare two rows by ORDER BY columns
* ====================================================================== */
/* Convert a parsed frame bound string into an absolute row index.
* cBound examples: "UNBOUNDED PRECEDING", "3 PRECEDING", "CURRENT ROW",
* "2 FOLLOWING", "UNBOUNDED FOLLOWING".
* nCurr = 1-based position of the current row within the partition.
* nPartLen = total rows in the partition. */
/* Sentinels shared with the Go SqlWindowSlideAgg RTL. Keep the
* values in sync — they must match the constants in hbrtl/sqlscan.go
* (frameUnboundedPreceding / frameUnboundedFollowing). */
#define FRAME_UNBOUNDED_PRECEDING -1073741824
#define FRAME_UNBOUNDED_FOLLOWING 1073741824
/* SqlFrameOffsetEncode parses a SQL frame bound string into a
* relative-offset integer suitable for SqlWindowSlideAgg. Sets lOk
* to .F. when the bound can't be encoded (empty, non-numeric N,
* unknown form) so the caller falls through to the O(N*W) loop.
* Supports: "UNBOUNDED PRECEDING", "UNBOUNDED FOLLOWING",
* "CURRENT ROW", "N PRECEDING", "N FOLLOWING". */
FUNCTION SqlFrameOffsetEncode( cBound, lOk )
LOCAL nV
lOk := .T.
IF cBound == NIL .OR. Empty( cBound )
/* Missing bound — treat as CURRENT ROW (same as SqlFrameOffset). */
RETURN 0
ENDIF
IF "UNBOUNDED PRECEDING" $ cBound
RETURN FRAME_UNBOUNDED_PRECEDING
ENDIF
IF "UNBOUNDED FOLLOWING" $ cBound
RETURN FRAME_UNBOUNDED_FOLLOWING
ENDIF
IF "CURRENT ROW" $ cBound
RETURN 0
ENDIF
IF "PRECEDING" $ cBound
nV := Val( cBound )
/* Val() returns 0 on parse failure — reject so we fall back
* rather than silently treating "? PRECEDING" as current row. */
IF nV <= 0
lOk := .F.
RETURN 0
ENDIF
RETURN -nV
ENDIF
IF "FOLLOWING" $ cBound
nV := Val( cBound )
IF nV <= 0
lOk := .F.
RETURN 0
ENDIF
RETURN nV
ENDIF
lOk := .F.
RETURN 0
FUNCTION SqlFrameOffset( cBound, nCurr, nPartLen )
LOCAL nV
IF cBound == NIL .OR. Empty( cBound )
RETURN nCurr
ENDIF
IF "UNBOUNDED PRECEDING" $ cBound
RETURN 1
ENDIF
IF "UNBOUNDED FOLLOWING" $ cBound
RETURN nPartLen
ENDIF
IF "CURRENT ROW" $ cBound
RETURN nCurr
ENDIF
IF "PRECEDING" $ cBound
nV := Val( cBound )
RETURN Max( 1, nCurr - nV )
ENDIF
IF "FOLLOWING" $ cBound
nV := Val( cBound )
RETURN Min( nPartLen, nCurr + nV )
ENDIF
RETURN nCurr
FUNCTION SqlWinRowCmp( aRows, nIdxA, nIdxB, aOrdBy, aFN )
LOCAL i, nCol, cDir, xA, xB
FOR i := 1 TO Len( aOrdBy )
nCol := SqlFindColIdx( aOrdBy[ i ][ 1 ], aFN )
IF nCol == 0
nCol := SqlFindColIdx2( SqlExprName( aOrdBy[ i ][ 1 ] ), aFN )
ENDIF
cDir := aOrdBy[ i ][ 2 ]
IF nCol > 0 .AND. nCol <= Len( aRows[ nIdxA ] ) .AND. nCol <= Len( aRows[ nIdxB ] )
xA := aRows[ nIdxA ][ nCol ]
xB := aRows[ nIdxB ][ nCol ]
IF xA == NIL .AND. xB == NIL
LOOP
ENDIF
IF xA == NIL
RETURN iif( cDir == "DESC", -1, 1 )
ENDIF
IF xB == NIL
RETURN iif( cDir == "DESC", 1, -1 )
ENDIF
IF ValType( xA ) == ValType( xB )
IF xA < xB
RETURN iif( cDir == "DESC", 1, -1 )
ELSEIF xA > xB
RETURN iif( cDir == "DESC", -1, 1 )
ENDIF
ENDIF
ENDIF
NEXT
RETURN 0
/* Check if two rows have equal values for ORDER BY columns */
FUNCTION SqlWinRowsEqual( aRows, nIdxA, nIdxB, aOrdBy, aFN )
LOCAL i, nCol, xA, xB
FOR i := 1 TO Len( aOrdBy )
nCol := SqlFindColIdx( aOrdBy[ i ][ 1 ], aFN )
IF nCol == 0
nCol := SqlFindColIdx2( SqlExprName( aOrdBy[ i ][ 1 ] ), aFN )
ENDIF
IF nCol > 0 .AND. nCol <= Len( aRows[ nIdxA ] ) .AND. nCol <= Len( aRows[ nIdxB ] )
xA := aRows[ nIdxA ][ nCol ]
xB := aRows[ nIdxB ][ nCol ]
IF ! SqlCmpEq( xA, xB )
RETURN .F.
ENDIF
ENDIF
NEXT
RETURN .T.
/*
* RecCteJoin — In-memory nested-loop JOIN for recursive CTE.
*
* The recursive part of a CTE may reference both a real DBF table and the
* CTE itself. Example:
* SELECT e.id, e.name FROM employees e JOIN org o ON e.mgr_id = o.id
*
* "employees" (alias e) is a DBF table on disk.
* "org" (alias o) is the CTE — represented by aPrevRows from the previous iteration.
*
* This function:
* 1. Identifies which FROM table is the CTE and which is the real DBF
* 2. Opens the DBF and reads all records into memory
* 3. Performs a nested-loop JOIN: for each DBF row x CTE row, checks ON condition
* 4. Evaluates SELECT columns for matching pairs
* 5. Returns the result rows
*/
STATIC FUNCTION RecCteJoin( hRecQuery, aFN, aPrevRows, cCteName )
LOCAL aCols, aFrom, aJoinFN, aJoinRows
LOCAL aResult, aNewRow
LOCAL i, j, k, nF
LOCAL cTblName, cTblAlias, cCteAlias, cWAAlias
LOCAL nSaveWA, cDbfFile
LOCAL xLeft, xRight, lMatch
LOCAL aJoinOn, aJ
LOCAL xCV
LOCAL aCombFN, aCombRow
LOCAL cDbfKeyCol, cCteKeyCol, nDbfKeyIdx, nCteKeyIdx
LOCAL hCteHash, cKey, aMatches, m
aCols := hRecQuery[ "columns" ]
aResult := {}
/* Identify the real table and the CTE reference.
* tables[]: { tableName, alias, "" }
* joins[]: { joinType, tableName, alias, onCondExpr } */
cTblName := ""
cTblAlias := ""
cCteAlias := ""
IF hb_HHasKey( hRecQuery, "tables" )
aFrom := hRecQuery[ "tables" ]
FOR i := 1 TO Len( aFrom )
IF Upper( aFrom[ i ][ 1 ] ) == Upper( cCteName )
cCteAlias := Upper( aFrom[ i ][ 2 ] )
IF Empty( cCteAlias )
cCteAlias := Upper( cCteName )
ENDIF
ELSE
cTblName := aFrom[ i ][ 1 ]
cTblAlias := Upper( aFrom[ i ][ 2 ] )
IF Empty( cTblAlias )
cTblAlias := Upper( cTblName )
ENDIF
ENDIF
NEXT
ENDIF
/* Also check the joins array for the CTE or real table */
aJoinOn := NIL
IF hb_HHasKey( hRecQuery, "joins" )
FOR i := 1 TO Len( hRecQuery[ "joins" ] )
aJ := hRecQuery[ "joins" ][ i ]
/* aJ = { joinType, tableName, alias, onCondExpr } */
IF Upper( aJ[ 2 ] ) == Upper( cCteName )
IF ! Empty( aJ[ 3 ] )
cCteAlias := Upper( aJ[ 3 ] )
ELSE
cCteAlias := Upper( cCteName )
ENDIF
ELSE
IF Empty( cTblName )
cTblName := aJ[ 2 ]
cTblAlias := Upper( aJ[ 3 ] )
IF Empty( cTblAlias )
cTblAlias := Upper( cTblName )
ENDIF
ENDIF
ENDIF
IF Len( aJ ) >= 4 .AND. aJ[ 4 ] != NIL
aJoinOn := aJ[ 4 ]
ENDIF
NEXT
ENDIF
IF Empty( cTblName )
RETURN aResult
ENDIF
/* Read all records from the real DBF table into memory */
aJoinRows := {}
aJoinFN := {}
nSaveWA := Select()
/* Always open the table fresh to avoid workarea conflicts.
* The anchor query may have closed it. */
cDbfFile := Lower( cTblName )
IF ! ( ".dbf" $ cDbfFile )
cDbfFile := cDbfFile + ".dbf"
ENDIF
cWAAlias := "RCJ_" + hb_ntos( SqlNextRCJSeq() )
BEGIN SEQUENCE
USE ( cDbfFile ) NEW SHARED ALIAS ( cWAAlias )
RECOVER
dbSelectArea( nSaveWA )
RETURN aResult
END SEQUENCE
/* Collect field names */
FOR nF := 1 TO FCount()
AAdd( aJoinFN, Upper( FieldName( nF ) ) )
NEXT
/* Read all records */
dbGoTop()
WHILE ! Eof()
aNewRow := {}
FOR nF := 1 TO FCount()
AAdd( aNewRow, FieldGet( nF ) )
NEXT
AAdd( aJoinRows, aNewRow )
dbSkip()
ENDDO
/* Build combined field name list:
* [tblAlias.field1, tblAlias.field2, ..., cteAlias.field1, cteAlias.field2, ...]
* Then also plain names for expression resolution */
aCombFN := {}
FOR nF := 1 TO Len( aJoinFN )
AAdd( aCombFN, cTblAlias + "." + aJoinFN[ nF ] )
NEXT
FOR nF := 1 TO Len( aFN )
AAdd( aCombFN, cCteAlias + "." + Upper( aFN[ nF ] ) )
NEXT
/* Also add unqualified names for both sides */
FOR nF := 1 TO Len( aJoinFN )
IF AScan( aCombFN, {|x| x == aJoinFN[ nF ] } ) == 0
AAdd( aCombFN, aJoinFN[ nF ] )
ENDIF
NEXT
FOR nF := 1 TO Len( aFN )
IF AScan( aCombFN, {|x| x == Upper( aFN[ nF ] ) } ) == 0
AAdd( aCombFN, Upper( aFN[ nF ] ) )
ENDIF
NEXT
/* Try to extract a simple equi-join key from aJoinOn so we can use
* hash probing instead of O(m*n) nested loops. This is the dominant
* cost for WITH RECURSIVE hierarchy traversals where aJoinRows is
* the full DBF (hundreds/thousands of rows) and aPrevRows is the
* current frontier set.
*
* Looks for ON condition of shape `dbfAlias.col = cteAlias.col` or
* the reverse — anything more complex falls through to nested loop. */
cDbfKeyCol := ""
cCteKeyCol := ""
IF aJoinOn != NIL .AND. ValType( aJoinOn ) == "A" .AND. Len( aJoinOn ) >= 4 .AND. ;
aJoinOn[ 1 ] == ND_BIN .AND. aJoinOn[ 2 ] == "=" .AND. ;
aJoinOn[ 3 ] != NIL .AND. aJoinOn[ 3 ][ 1 ] == ND_COL .AND. ;
aJoinOn[ 4 ] != NIL .AND. aJoinOn[ 4 ][ 1 ] == ND_COL
/* Split alias.col on both sides */
cKey := Upper( aJoinOn[ 3 ][ 2 ] )
IF "." $ cKey .AND. Left( cKey, At( ".", cKey ) - 1 ) == cCteAlias
cCteKeyCol := SubStr( cKey, At( ".", cKey ) + 1 )
cKey := Upper( aJoinOn[ 4 ][ 2 ] )
IF "." $ cKey
cDbfKeyCol := SubStr( cKey, At( ".", cKey ) + 1 )
ELSE
cDbfKeyCol := cKey
ENDIF
ELSE
cKey := Upper( aJoinOn[ 4 ][ 2 ] )
IF "." $ cKey .AND. Left( cKey, At( ".", cKey ) - 1 ) == cCteAlias
cCteKeyCol := SubStr( cKey, At( ".", cKey ) + 1 )
cKey := Upper( aJoinOn[ 3 ][ 2 ] )
IF "." $ cKey
cDbfKeyCol := SubStr( cKey, At( ".", cKey ) + 1 )
ELSE
cDbfKeyCol := cKey
ENDIF
ENDIF
ENDIF
ENDIF
nDbfKeyIdx := 0
nCteKeyIdx := 0
IF ! Empty( cDbfKeyCol ) .AND. ! Empty( cCteKeyCol )
FOR nF := 1 TO Len( aJoinFN )
IF aJoinFN[ nF ] == cDbfKeyCol
nDbfKeyIdx := nF
EXIT
ENDIF
NEXT
FOR nF := 1 TO Len( aFN )
IF Upper( aFN[ nF ] ) == cCteKeyCol
nCteKeyIdx := nF
EXIT
ENDIF
NEXT
ENDIF
IF nDbfKeyIdx > 0 .AND. nCteKeyIdx > 0
/* Hash-probe path: build hash on aPrevRows keyed by cte column,
* then scan aJoinRows and probe. Sub-linear vs nested loop. */
hCteHash := { => }
FOR j := 1 TO Len( aPrevRows )
cKey := SqlValToStr( aPrevRows[ j ][ nCteKeyIdx ] )
IF ! hb_HHasKey( hCteHash, cKey )
hCteHash[ cKey ] := {}
ENDIF
AAdd( hCteHash[ cKey ], j )
NEXT
FOR i := 1 TO Len( aJoinRows )
cKey := SqlValToStr( aJoinRows[ i ][ nDbfKeyIdx ] )
IF ! hb_HHasKey( hCteHash, cKey )
LOOP
ENDIF
aMatches := hCteHash[ cKey ]
FOR m := 1 TO Len( aMatches )
j := aMatches[ m ]
aCombRow := {}
FOR nF := 1 TO Len( aJoinFN )
AAdd( aCombRow, aJoinRows[ i ][ nF ] )
NEXT
FOR nF := 1 TO Len( aFN )
AAdd( aCombRow, aPrevRows[ j ][ nF ] )
NEXT
FOR nF := 1 TO Len( aJoinFN )
AAdd( aCombRow, aJoinRows[ i ][ nF ] )
NEXT
FOR nF := 1 TO Len( aFN )
AAdd( aCombRow, aPrevRows[ j ][ nF ] )
NEXT
aNewRow := {}
FOR k := 1 TO Len( aCols )
xCV := SqlEvalRowExpr( aCols[ k ][ 1 ], aCombFN, aCombRow )
AAdd( aNewRow, xCV )
NEXT
AAdd( aResult, aNewRow )
NEXT
NEXT
ELSE
/* Fallback: nested-loop JOIN for complex ON predicates */
FOR i := 1 TO Len( aJoinRows )
FOR j := 1 TO Len( aPrevRows )
aCombRow := {}
FOR nF := 1 TO Len( aJoinFN )
AAdd( aCombRow, aJoinRows[ i ][ nF ] )
NEXT
FOR nF := 1 TO Len( aFN )
AAdd( aCombRow, aPrevRows[ j ][ nF ] )
NEXT
FOR nF := 1 TO Len( aJoinFN )
AAdd( aCombRow, aJoinRows[ i ][ nF ] )
NEXT
FOR nF := 1 TO Len( aFN )
AAdd( aCombRow, aPrevRows[ j ][ nF ] )
NEXT
lMatch := .T.
IF aJoinOn != NIL
xLeft := SqlEvalRowExpr( aJoinOn, aCombFN, aCombRow )
lMatch := SqlIsTrue( xLeft )
ENDIF
IF lMatch
aNewRow := {}
FOR k := 1 TO Len( aCols )
xCV := SqlEvalRowExpr( aCols[ k ][ 1 ], aCombFN, aCombRow )
AAdd( aNewRow, xCV )
NEXT
AAdd( aResult, aNewRow )
ENDIF
NEXT
NEXT
ENDIF
/* Close the workarea we opened */
IF ! Empty( cWAAlias )
dbSelectArea( Select( cWAAlias ) )
dbCloseArea()
ENDIF
dbSelectArea( nSaveWA )
RETURN aResult
/* --------------------------------------------------------------
* Go fast-path helpers
* Return non-NIL only when the query can be handed off to Go's
* SqlScan RTL. Any complexity (expressions, functions, joins,
* parameters in WHERE) → return NIL so the PRG loop takes over.
* -------------------------------------------------------------- */
/* TryGoJoin — attempt to hand a multi-table equi-join to Go's
* SqlHashJoin RTL. Returns the result array on success, NIL if the
* query shape doesn't fit (non-equi ON, complex SELECT exprs, etc.)
* and the caller should fall back to the PRG JoinRecurse path.
*
* Conditions for the fast path:
* - All joins are equi-joins on single columns (ND_BIN "=")
* - All SELECT columns are plain ND_COL field refs
* - No WHERE clause (WHERE is NIL)
*/
/* Build {nColIdx, lDesc} spec array for Go SqlOrderBy.
* Returns NIL if any ORDER BY expression can't be resolved to a
* simple column index (complex expressions → PRG fallback). */
METHOD TryBuildSortSpec( aOrderBy, aFieldNames ) CLASS TSqlExecutor
LOCAL aSpec := {}, i, j, xE, cName, nCol, cDir, cNulls, nDot
FOR i := 1 TO Len( aOrderBy )
xE := aOrderBy[ i ][ 1 ]
cDir := Upper( aOrderBy[ i ][ 2 ] )
cNulls := iif( Len( aOrderBy[ i ] ) >= 3, Upper( aOrderBy[ i ][ 3 ] ), "" )
IF xE == NIL .OR. xE[ 1 ] != ND_COL
RETURN NIL
ENDIF
cName := Upper( xE[ 2 ] )
nDot := At( ".", cName )
IF nDot > 0
cName := SubStr( cName, nDot + 1 )
ENDIF
/* Find column index in aFieldNames */
nCol := 0
FOR j := 1 TO Len( aFieldNames )
IF Upper( aFieldNames[ j ] ) == cName .OR. ;
( "." $ aFieldNames[ j ] .AND. ;
Upper( SubStr( aFieldNames[ j ], At( ".", aFieldNames[ j ] ) + 1 ) ) == cName )
nCol := j
EXIT
ENDIF
NEXT
IF nCol == 0
RETURN NIL
ENDIF
/* Go SqlOrderBy reads {nCol, lDesc, cNulls}. cNulls empty means
* "default" — NIL sorts as the largest value (NULLs last in ASC,
* NULLs first in DESC). Explicit "FIRST"/"LAST" overrides. */
AAdd( aSpec, { nCol, cDir == "DESC", cNulls } )
NEXT
RETURN aSpec
METHOD TryGoJoin( aJoins, aResultExprs, nOuterWA ) CLASS TSqlExecutor
LOCAL i, xE, xOnCond, cInnerAlias, cInnerField, cOuterField
LOCAL nInnerWA, nInnerFPos, nOuterFPos, nWA
LOCAL aJoinSpecs := {}, aSelectFields := {}
LOCAL cRef, nDot, cAlias, cField, cJoinType
LOCAL aGoRows
/* Build join specs: { nInnerWA, nInnerKeyField, nOuterKeyField } */
FOR i := 1 TO Len( aJoins )
/* The Go SqlHashJoin RTL is an INNER-join implementation —
* it emits one row per matching outer/inner pair and has no
* null-fill path. OUTER joins (LEFT / RIGHT / FULL) must fall
* back to PRG JoinRecurse so unmatched outer rows still
* appear with NIL inner columns. Before this gate a LEFT JOIN
* silently dropped every outer row without a match. */
cJoinType := ""
IF Len( aJoins[ i ] ) >= 1 .AND. ValType( aJoins[ i ][ 1 ] ) == "C"
cJoinType := Upper( aJoins[ i ][ 1 ] )
ENDIF
IF cJoinType == "LEFT" .OR. cJoinType == "RIGHT" .OR. cJoinType == "FULL"
RETURN NIL
ENDIF
xOnCond := aJoins[ i ][ 4 ]
/* Only support simple equi-join */
IF xOnCond == NIL .OR. xOnCond[ 1 ] != ND_BIN .OR. xOnCond[ 2 ] != "="
RETURN NIL
ENDIF
IF xOnCond[ 3 ] == NIL .OR. xOnCond[ 3 ][ 1 ] != ND_COL .OR. ;
xOnCond[ 4 ] == NIL .OR. xOnCond[ 4 ][ 1 ] != ND_COL
RETURN NIL
ENDIF
/* Determine which side is inner vs outer */
cInnerAlias := aJoins[ i ][ 3 ]
IF Empty( cInnerAlias )
cInnerAlias := aJoins[ i ][ 2 ]
ENDIF
IF ::ColBelongsTo( xOnCond[ 4 ][ 2 ], cInnerAlias )
cInnerField := xOnCond[ 4 ][ 2 ]
cOuterField := xOnCond[ 3 ][ 2 ]
ELSEIF ::ColBelongsTo( xOnCond[ 3 ][ 2 ], cInnerAlias )
cInnerField := xOnCond[ 3 ][ 2 ]
cOuterField := xOnCond[ 4 ][ 2 ]
ELSE
RETURN NIL
ENDIF
/* Resolve workarea + field positions */
nInnerWA := ::FindWA( Upper( cInnerAlias ) )
IF nInnerWA <= 0
RETURN NIL
ENDIF
dbSelectArea( nInnerWA )
cField := Upper( cInnerField )
IF "." $ cField
cField := SubStr( cField, At( ".", cField ) + 1 )
ENDIF
nInnerFPos := FieldPos( cField )
IF nInnerFPos == 0
RETURN NIL
ENDIF
/* Outer field — resolve in parent table */
cField := Upper( cOuterField )
nDot := At( ".", cField )
IF nDot > 0
cAlias := Left( cField, nDot - 1 )
cField := SubStr( cField, nDot + 1 )
nWA := ::FindWA( cAlias )
ELSE
nWA := nOuterWA
ENDIF
IF nWA <= 0
RETURN NIL
ENDIF
dbSelectArea( nWA )
nOuterFPos := FieldPos( cField )
IF nOuterFPos == 0
RETURN NIL
ENDIF
AAdd( aJoinSpecs, { nInnerWA, nInnerFPos, nOuterFPos } )
NEXT
/* Build select field specs: { nWA, nFieldPos } for each result column.
* Aggregate columns (ND_FN) get a {0, 0} placeholder — their values
* will be filled later by ComputeAgg during GROUP BY processing.
* This lets the Go fast path handle aggregate queries where the
* raw data columns (hidden) are plain ND_COL refs. */
FOR i := 1 TO Len( aResultExprs )
xE := aResultExprs[ i ][ 1 ]
IF xE == NIL .OR. xE[ 2 ] == "*"
RETURN NIL
ENDIF
IF xE[ 1 ] == ND_FN .OR. xE[ 1 ] == ND_WINDOW
/* Aggregate/window placeholder — Go returns 0, PRG fills later */
AAdd( aSelectFields, { 0, 0 } )
LOOP
ENDIF
IF xE[ 1 ] != ND_COL
RETURN NIL
ENDIF
cRef := xE[ 2 ]
nDot := At( ".", cRef )
IF nDot > 0
cAlias := Upper( Left( cRef, nDot - 1 ) )
cField := Upper( SubStr( cRef, nDot + 1 ) )
nWA := ::FindWA( cAlias )
ELSE
cField := Upper( cRef )
nWA := nOuterWA
ENDIF
IF nWA <= 0
RETURN NIL
ENDIF
dbSelectArea( nWA )
nOuterFPos := FieldPos( cField )
IF nOuterFPos == 0
RETURN NIL
ENDIF
AAdd( aSelectFields, { nWA, nOuterFPos } )
NEXT
/* Call Go-native hash join */
aGoRows := SqlHashJoin( aJoinSpecs, aSelectFields, nOuterWA )
RETURN aGoRows
METHOD TryBuildFieldPositions( aExprs ) CLASS TSqlExecutor
LOCAL aPositions := {}, i, xE, cRef, nDot, cField, nFPos
FOR i := 1 TO Len( aExprs )
xE := aExprs[ i ][ 1 ]
IF xE == NIL .OR. xE[ 1 ] != ND_COL .OR. xE[ 2 ] == "*"
RETURN NIL
ENDIF
cRef := xE[ 2 ]
nDot := At( ".", cRef )
IF nDot > 0
cField := Upper( SubStr( cRef, nDot + 1 ) )
ELSE
cField := Upper( cRef )
ENDIF
nFPos := FieldPos( cField )
IF nFPos == 0
RETURN NIL
ENDIF
AAdd( aPositions, nFPos )
NEXT
RETURN aPositions
METHOD TryCompileWhere( xWhere ) CLASS TSqlExecutor
/* Phase 1+2: compile numeric/logical/string WHERE to pcode.
* Semantic guard: SqlExprToPrg returns NIL for anything that would
* drift from SqlCmpEq/SqlCoerceForCmp semantics. CHAR columns are
* auto-wrapped with AllTrim() to match Harbour SqlCmpEq behavior.
* NULL/function/subquery/parameter → NIL (fallback).
*/
LOCAL cPrg, xResult
IF xWhere == NIL
RETURN NIL
ENDIF
/* Cache struct once for field-type lookups during expr walk */
::aCompileStruct := dbStruct()
cPrg := ::SqlExprToPrg( xWhere )
::aCompileStruct := NIL
IF cPrg == NIL
RETURN NIL
ENDIF
xResult := PcCompile( cPrg )
RETURN xResult
METHOD SqlExprToPrg( xNode ) CLASS TSqlExecutor
LOCAL cOp, cL, cR
LOCAL cRef, nDot, cField, nFPos, cFType, cLit, cAliasU
LOCAL lLocalAlias, ii, cA
IF xNode == NIL
RETURN NIL
ENDIF
DO CASE
CASE xNode[ 1 ] == ND_LIT
IF ValType( xNode[ 2 ] ) == "N"
/* Use hb_NToS — preserves all decimal digits.
* Str(0.1) returns " 0" (default 0 decimals when
* the type is numeric), which AllTrim collapsed to "0";
* the pcode then ran `WHERE v = 0` for `WHERE v = 0.1` and
* silently returned no rows. Same class of bug for any
* fractional literal (0.5, 1.25, 3.14) and for negative /
* large values that Str's default width clips. */
RETURN hb_NToS( xNode[ 2 ] )
ENDIF
IF ValType( xNode[ 2 ] ) == "L"
IF xNode[ 2 ]
RETURN ".T."
ENDIF
RETURN ".F."
ENDIF
IF ValType( xNode[ 2 ] ) == "C"
cLit := xNode[ 2 ]
/* Reject strings with embedded quotes — escaping would be ambiguous */
IF "'" $ cLit .OR. '"' $ cLit .OR. Chr(10) $ cLit .OR. Chr(13) $ cLit
RETURN NIL
ENDIF
/* Match SqlCmpEq: compare trimmed values */
RETURN "'" + AllTrim( cLit ) + "'"
ENDIF
/* Dates/datetimes deferred */
RETURN NIL
CASE xNode[ 1 ] == ND_COL
cRef := xNode[ 2 ]
IF cRef == "*"
RETURN NIL
ENDIF
nDot := At( ".", cRef )
IF nDot > 0
/* Qualified reference — only compile if the alias prefix
* matches one of THIS executor's own tables. FindWA would
* otherwise return an outer-scope workarea with the same
* alias (Select() is case-insensitive across all open
* areas), causing pcode to bind `outer.col` to the inner
* workarea's same-named field. That silently collapses
* correlated predicates like `WHERE dept = e.dept` into
* `WHERE dept = dept` (always true). Returning NIL routes
* the caller to PRG EvalExpr, which handles outer lookup
* through Resolve / ResolveFromOuter / the outer stack. */
cAliasU := Upper( Left( cRef, nDot - 1 ) )
lLocalAlias := .F.
FOR ii := 1 TO Len( ::aTables )
cA := Upper( ::aTables[ ii ][ 2 ] )
IF Empty( cA )
cA := Upper( ::aTables[ ii ][ 1 ] )
ENDIF
IF cA == cAliasU .OR. Upper( ::aTables[ ii ][ 1 ] ) == cAliasU .OR. ;
( Len( ::aTables[ ii ] ) >= 3 .AND. ;
Upper( ::aTables[ ii ][ 3 ] ) == cAliasU )
lLocalAlias := .T.
EXIT
ENDIF
NEXT
IF ! lLocalAlias
RETURN NIL
ENDIF
cField := Upper( SubStr( cRef, nDot + 1 ) )
ELSE
cField := Upper( cRef )
ENDIF
nFPos := FieldPos( cField )
IF nFPos == 0
RETURN NIL
ENDIF
/* Look up field type from cached struct to decide AllTrim wrap */
cFType := ""
IF ::aCompileStruct != NIL .AND. nFPos <= Len( ::aCompileStruct )
cFType := ::aCompileStruct[ nFPos ][ 2 ]
ENDIF
IF cFType == "C"
RETURN "AllTrim(FieldGet(" + AllTrim( Str( nFPos ) ) + "))"
ENDIF
RETURN "FieldGet(" + AllTrim( Str( nFPos ) ) + ")"
CASE xNode[ 1 ] == ND_UNI
cOp := xNode[ 2 ]
cL := ::SqlExprToPrg( xNode[ 3 ] )
IF cL == NIL
RETURN NIL
ENDIF
IF cOp == "NOT"
RETURN "!(" + cL + ")"
ENDIF
IF cOp == "-"
RETURN "-(" + cL + ")"
ENDIF
RETURN NIL
CASE xNode[ 1 ] == ND_BIN
cOp := xNode[ 2 ]
cL := ::SqlExprToPrg( xNode[ 3 ] )
IF cL == NIL
RETURN NIL
ENDIF
cR := ::SqlExprToPrg( xNode[ 4 ] )
IF cR == NIL
RETURN NIL
ENDIF
DO CASE
/* Use SqlCmpEq / SqlCmpLt — not Harbour's `==` / `<` — so the
* fast path matches the PRG path: case-insensitive string
* compare after AllTrim, Date↔String coercion via DToS form,
* Numeric↔String leading-digits parse. iif-gate drops rows
* where either side is NIL to enforce SQL three-valued
* logic ("NULL cmp anything → not matched"). Without these
* wrappers `WHERE hired = '20240115'` silently missed rows
* and `WHERE v <> 10` leaked NULL rows — both regressions
* from the bare `==` / `!=` emission. */
CASE cOp == "=" .OR. cOp == "=="
RETURN "iif((" + cL + ")==NIL .OR. (" + cR + ")==NIL, .F., SqlCmpEq((" + cL + "),(" + cR + ")))"
CASE cOp == "<>" .OR. cOp == "!="
RETURN "iif((" + cL + ")==NIL .OR. (" + cR + ")==NIL, .F., !SqlCmpEq((" + cL + "),(" + cR + ")))"
CASE cOp == "<"
RETURN "iif((" + cL + ")==NIL .OR. (" + cR + ")==NIL, .F., SqlCmpLt((" + cL + "),(" + cR + ")))"
CASE cOp == "<="
RETURN "iif((" + cL + ")==NIL .OR. (" + cR + ")==NIL, .F., SqlCmpEq((" + cL + "),(" + cR + ")).OR.SqlCmpLt((" + cL + "),(" + cR + ")))"
CASE cOp == ">"
RETURN "iif((" + cL + ")==NIL .OR. (" + cR + ")==NIL, .F., SqlCmpLt((" + cR + "),(" + cL + ")))"
CASE cOp == ">="
RETURN "iif((" + cL + ")==NIL .OR. (" + cR + ")==NIL, .F., SqlCmpEq((" + cL + "),(" + cR + ")).OR.SqlCmpLt((" + cR + "),(" + cL + ")))"
CASE cOp == "AND"
RETURN "(" + cL + ").AND.(" + cR + ")"
CASE cOp == "OR"
RETURN "(" + cL + ").OR.(" + cR + ")"
CASE cOp == "+"
RETURN "(" + cL + ")+(" + cR + ")"
CASE cOp == "-"
RETURN "(" + cL + ")-(" + cR + ")"
CASE cOp == "*"
RETURN "(" + cL + ")*(" + cR + ")"
CASE cOp == "/"
RETURN "(" + cL + ")/(" + cR + ")"
ENDCASE
RETURN NIL
ENDCASE
RETURN NIL
/* --------------------------------------------------------------
* Schema version — accessors for the file-scoped s_nSchemaVer
* counter defined at the top of this module. All SQL plan-cache
* keys embed the current version as a prefix; every DDL calls
* SqlBumpSchemaVer() on success so subsequent SELECTs / DML miss
* the cache and re-resolve columns / indexes against the new
* schema. Called from TFiveSQL (plan cache key build) and
* TSqlDDL (invalidation after CREATE/ALTER/DROP).
* -------------------------------------------------------------- */
/* SqlSchemaVer / SqlBumpSchemaVer are Go HB_FUNCs (atomic.uint64)
* — see hbrtl/sqlglobals.go. The PRG-level definitions were
* removed: a STATIC int32 with non-atomic ++ couldn't safely
* serialise concurrent DDL across pgserver connections. */
Reference in New Issue View Git Blame Copy Permalink