From 83877f7a7b9a4599695880ec3d0d7842dd2f1252 Mon Sep 17 00:00:00 2001 From: Viktor Szakats Date: Tue, 22 Dec 2009 22:11:48 +0000 Subject: [PATCH] 2009-12-22 23:10 UTC+0100 Viktor Szakats (harbour.01 syenar.hu) * doc/whatsnew.txt ! Fixed date of 2.0.0 release. [TOMERGE 2.0] * external/sqlite3/sqlite3.c * external/sqlite3/sqlite3.h + sqlite upgraded to 3.6.21 (from 3.6.20) --- harbour/ChangeLog | 11 + harbour/doc/whatsnew.txt | 2 +- harbour/external/sqlite3/sqlite3.c | 20267 +++++++++++++-------------- harbour/external/sqlite3/sqlite3.h | 23 +- 4 files changed, 9661 insertions(+), 10642 deletions(-) diff --git a/harbour/ChangeLog b/harbour/ChangeLog index 20903dfb22..14c6625fd8 100644 --- a/harbour/ChangeLog +++ b/harbour/ChangeLog @@ -17,6 +17,17 @@ past entries belonging to author(s): Viktor Szakats. */ +2009-12-22 23:10 UTC+0100 Viktor Szakats (harbour.01 syenar.hu) + * doc/whatsnew.txt + ! Fixed date of 2.0.0 release. + [TOMERGE 2.0] + + * external/sqlite3/sqlite3.c + * external/sqlite3/sqlite3.h + + sqlite upgraded to 3.6.21 (from 3.6.20) + +---------- 2.0.0 ---------- + 2009-12-22 22:00 UTC+0100 Viktor Szakats (harbour.01 syenar.hu) * harbour-win-spec * harbour-wce-spec diff --git a/harbour/doc/whatsnew.txt b/harbour/doc/whatsnew.txt index c532b0ef58..72d8301ad7 100644 --- a/harbour/doc/whatsnew.txt +++ b/harbour/doc/whatsnew.txt @@ -12,7 +12,7 @@ The license applies to all entries on or after Version 1.1.0. */ -Version 2.0.0 (2009-12-??) +Version 2.0.0 (2009-12-22) Note ============================= diff --git a/harbour/external/sqlite3/sqlite3.c b/harbour/external/sqlite3/sqlite3.c index 433c657f4b..3f61fe6940 100644 --- a/harbour/external/sqlite3/sqlite3.c +++ b/harbour/external/sqlite3/sqlite3.c @@ -1,6 +1,6 @@ /****************************************************************************** ** This file is an amalgamation of many separate C source files from SQLite -** version 3.6.20. By combining all the individual C code files into this +** version 3.6.21. By combining all the individual C code files into this ** single large file, the entire code can be compiled as a one translation ** unit. This allows many compilers to do optimizations that would not be ** possible if the files were compiled separately. Performance improvements @@ -9,7 +9,7 @@ ** ** This file is all you need to compile SQLite. To use SQLite in other ** programs, you need this file and the "sqlite3.h" header file that defines -** the programming interface to the SQLite library. (If you do not have +** the programming interface to the SQLite library. (If you do not have ** the "sqlite3.h" header file at hand, you will find a copy embedded within ** the text of this file. Search for "Begin file sqlite3.h" to find the start ** of the embedded sqlite3.h header file.) Additional code files may be needed @@ -91,10 +91,8 @@ ** May you share freely, never taking more than you give. ** ************************************************************************* -** -** This file defines various limits of what SQLite can process. ** -** @(#) $Id: sqliteLimit.h,v 1.10 2009/01/10 16:15:09 danielk1977 Exp $ +** This file defines various limits of what SQLite can process. */ /* @@ -141,9 +139,9 @@ #endif /* -** The maximum depth of an expression tree. This is limited to -** some extent by SQLITE_MAX_SQL_LENGTH. But sometime you might -** want to place more severe limits on the complexity of an +** The maximum depth of an expression tree. This is limited to +** some extent by SQLITE_MAX_SQL_LENGTH. But sometime you might +** want to place more severe limits on the complexity of an ** expression. ** ** A value of 0 used to mean that the limit was not enforced. @@ -215,8 +213,8 @@ ** ** If this limit is changed, then the compiled library is technically ** incompatible with an SQLite library compiled with a different limit. If -** a process operating on a database with a page-size of 65536 bytes -** crashes, then an instance of SQLite compiled with the default page-size +** a process operating on a database with a page-size of 65536 bytes +** crashes, then an instance of SQLite compiled with the default page-size ** limit will not be able to rollback the aborted transaction. This could ** lead to database corruption. */ @@ -275,16 +273,12 @@ ** Maximum depth of recursion for triggers. ** ** A value of 1 means that a trigger program will not be able to itself -** fire any triggers. A value of 0 means that no trigger programs at all +** fire any triggers. A value of 0 means that no trigger programs at all ** may be executed. */ #ifndef SQLITE_MAX_TRIGGER_DEPTH -#if defined(SQLITE_SMALL_STACK) -# define SQLITE_MAX_TRIGGER_DEPTH 10 -#else # define SQLITE_MAX_TRIGGER_DEPTH 1000 #endif -#endif /************** End of sqliteLimit.h *****************************************/ /************** Continuing where we left off in sqliteInt.h ******************/ @@ -318,14 +312,14 @@ /* ** This macro is used to "hide" some ugliness in casting an int ** value to a ptr value under the MSVC 64-bit compiler. Casting -** non 64-bit values to ptr types results in a "hard" error with -** the MSVC 64-bit compiler which this attempts to avoid. +** non 64-bit values to ptr types results in a "hard" error with +** the MSVC 64-bit compiler which this attempts to avoid. ** ** A simple compiler pragma or casting sequence could not be found ** to correct this in all situations, so this macro was introduced. ** ** It could be argued that the intptr_t type could be used in this -** case, but that type is not available on all compilers, or +** case, but that type is not available on all compilers, or ** requires the #include of specific headers which differs between ** platforms. ** @@ -363,7 +357,7 @@ /* ** The SQLITE_DEFAULT_MEMSTATUS macro must be defined as either 0 or 1. -** It determines whether or not the features related to +** It determines whether or not the features related to ** SQLITE_CONFIG_MEMSTATUS are available by default or not. This value can ** be overridden at runtime using the sqlite3_config() API. */ @@ -436,12 +430,12 @@ ** option is set. Thus NDEBUG becomes an opt-in rather than an opt-out ** feature. */ -#if !defined(NDEBUG) && !defined(SQLITE_DEBUG) +#if !defined(NDEBUG) && !defined(SQLITE_DEBUG) # define NDEBUG 1 #endif /* -** The testcase() macro is used to aid in coverage testing. When +** The testcase() macro is used to aid in coverage testing. When ** doing coverage testing, the condition inside the argument to ** testcase() must be evaluated both true and false in order to ** get full branch coverage. The testcase() macro is inserted @@ -487,7 +481,7 @@ SQLITE_PRIVATE void sqlite3Coverage(int); #endif /* -** The ALWAYS and NEVER macros surround boolean expressions which +** The ALWAYS and NEVER macros surround boolean expressions which ** are intended to always be true or false, respectively. Such ** expressions could be omitted from the code completely. But they ** are included in a few cases in order to enhance the resilience @@ -649,9 +643,9 @@ extern "C" { ** ** Requirements: [H10011] [H10014] */ -#define SQLITE_VERSION "3.6.20" -#define SQLITE_VERSION_NUMBER 3006020 -#define SQLITE_SOURCE_ID "2009-11-04 13:30:02 eb7a544fe49d1626bacecfe53ddc03fe082e3243" +#define SQLITE_VERSION "3.6.21" +#define SQLITE_VERSION_NUMBER 3006021 +#define SQLITE_SOURCE_ID "2009-12-07 16:39:13 1ed88e9d01e9eda5cbc622e7614277f29bcc551c" /* ** CAPI3REF: Run-Time Library Version Numbers {H10020} @@ -693,7 +687,7 @@ SQLITE_API int sqlite3_libversion_number(void); ** SQLite can be compiled with or without mutexes. When ** the [SQLITE_THREADSAFE] C preprocessor macro is 1 or 2, mutexes ** are enabled and SQLite is threadsafe. When the -** [SQLITE_THREADSAFE] macro is 0, +** [SQLITE_THREADSAFE] macro is 0, ** the mutexes are omitted. Without the mutexes, it is not safe ** to use SQLite concurrently from more than one thread. ** @@ -1028,7 +1022,7 @@ SQLITE_API int sqlite3_exec( /* ** CAPI3REF: OS Interface Open File Handle {H11110} ** -** An [sqlite3_file] object represents an open file in the +** An [sqlite3_file] object represents an open file in the ** [sqlite3_vfs | OS interface layer]. Individual OS interface ** implementations will ** want to subclass this object by appending additional fields @@ -1050,7 +1044,7 @@ struct sqlite3_file { ** This object defines the methods used to perform various operations ** against the open file represented by the [sqlite3_file] object. ** -** If the xOpen method sets the sqlite3_file.pMethods element +** If the xOpen method sets the sqlite3_file.pMethods element ** to a non-NULL pointer, then the sqlite3_io_methods.xClose method ** may be invoked even if the xOpen reported that it failed. The ** only way to prevent a call to xClose following a failed xOpen @@ -1220,14 +1214,14 @@ typedef struct sqlite3_mutex sqlite3_mutex; ** the [sqlite3_file] can safely store a pointer to the ** filename if it needs to remember the filename for some reason. ** If the zFilename parameter is xOpen is a NULL pointer then xOpen -** must invent its own temporary name for the file. Whenever the +** must invent its own temporary name for the file. Whenever the ** xFilename parameter is NULL it will also be the case that the ** flags parameter will include [SQLITE_OPEN_DELETEONCLOSE]. ** ** The flags argument to xOpen() includes all bits set in ** the flags argument to [sqlite3_open_v2()]. Or if [sqlite3_open()] ** or [sqlite3_open16()] is used, then flags includes at least -** [SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]. +** [SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]. ** If xOpen() opens a file read-only then it sets *pOutFlags to ** include [SQLITE_OPEN_READONLY]. Other bits in *pOutFlags may be set. ** @@ -1267,10 +1261,10 @@ typedef struct sqlite3_mutex sqlite3_mutex; ** The [SQLITE_OPEN_EXCLUSIVE] flag is always used in conjunction ** with the [SQLITE_OPEN_CREATE] flag, which are both directly ** analogous to the O_EXCL and O_CREAT flags of the POSIX open() -** API. The SQLITE_OPEN_EXCLUSIVE flag, when paired with the +** API. The SQLITE_OPEN_EXCLUSIVE flag, when paired with the ** SQLITE_OPEN_CREATE, is used to indicate that file should always ** be created, and that it is an error if it already exists. -** It is not used to indicate the file should be opened +** It is not used to indicate the file should be opened ** for exclusive access. ** ** At least szOsFile bytes of memory are allocated by SQLite @@ -1355,7 +1349,7 @@ struct sqlite3_vfs { ** The sqlite3_initialize() routine initializes the ** SQLite library. The sqlite3_shutdown() routine ** deallocates any resources that were allocated by sqlite3_initialize(). -** This routines are designed to aid in process initialization and +** These routines are designed to aid in process initialization and ** shutdown on embedded systems. Workstation applications using ** SQLite normally do not need to invoke either of these routines. ** @@ -1474,7 +1468,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_config(int, ...); ** [database connection] (specified in the first argument). The ** sqlite3_db_config() interface can only be used immediately after ** the database connection is created using [sqlite3_open()], -** [sqlite3_open16()], or [sqlite3_open_v2()]. +** [sqlite3_open16()], or [sqlite3_open_v2()]. ** ** The second argument to sqlite3_db_config(D,V,...) is the ** configuration verb - an integer code that indicates what @@ -1498,7 +1492,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_db_config(sqlite3*, int op, ...); ** This object is used in only one place in the SQLite interface. ** A pointer to an instance of this object is the argument to ** [sqlite3_config()] when the configuration option is -** [SQLITE_CONFIG_MALLOC] or [SQLITE_CONFIG_GETMALLOC]. +** [SQLITE_CONFIG_MALLOC] or [SQLITE_CONFIG_GETMALLOC]. ** By creating an instance of this object ** and passing it to [sqlite3_config]([SQLITE_CONFIG_MALLOC]) ** during configuration, an application can specify an alternative @@ -1534,7 +1528,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_db_config(sqlite3*, int op, ...); ** allocators round up memory allocations at least to the next multiple ** of 8. Some allocators round up to a larger multiple or to a power of 2. ** Every memory allocation request coming in through [sqlite3_malloc()] -** or [sqlite3_realloc()] first calls xRoundup. If xRoundup returns 0, +** or [sqlite3_realloc()] first calls xRoundup. If xRoundup returns 0, ** that causes the corresponding memory allocation to fail. ** ** The xInit method initializes the memory allocator. (For example, @@ -1626,9 +1620,9 @@ struct sqlite3_mem_methods { ** tracks memory usage, for example. ** **
SQLITE_CONFIG_MEMSTATUS
-**
This option takes single argument of type int, interpreted as a -** boolean, which enables or disables the collection of memory allocation -** statistics. When disabled, the following SQLite interfaces become +**
This option takes single argument of type int, interpreted as a +** boolean, which enables or disables the collection of memory allocation +** statistics. When disabled, the following SQLite interfaces become ** non-operational: **
    **
  • [sqlite3_memory_used()] @@ -1658,7 +1652,7 @@ struct sqlite3_mem_methods { ** **
    SQLITE_CONFIG_PAGECACHE
    **
    This option specifies a static memory buffer that SQLite can use for -** the database page cache with the default page cache implemenation. +** the database page cache with the default page cache implemenation. ** This configuration should not be used if an application-define page ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option. ** There are three arguments to this option: A pointer to 8-byte aligned @@ -1673,7 +1667,7 @@ struct sqlite3_mem_methods { ** memory needs for the first N pages that it adds to cache. If additional ** page cache memory is needed beyond what is provided by this option, then ** SQLite goes to [sqlite3_malloc()] for the additional storage space. -** The implementation might use one or more of the N buffers to hold +** The implementation might use one or more of the N buffers to hold ** memory accounting information. The pointer in the first argument must ** be aligned to an 8-byte boundary or subsequent behavior of SQLite ** will be undefined.
    @@ -1741,7 +1735,7 @@ struct sqlite3_mem_methods { #define SQLITE_CONFIG_MEMSTATUS 9 /* boolean */ #define SQLITE_CONFIG_MUTEX 10 /* sqlite3_mutex_methods* */ #define SQLITE_CONFIG_GETMUTEX 11 /* sqlite3_mutex_methods* */ -/* previously SQLITE_CONFIG_CHUNKALLOC 12 which is now unused. */ +/* previously SQLITE_CONFIG_CHUNKALLOC 12 which is now unused. */ #define SQLITE_CONFIG_LOOKASIDE 13 /* int int */ #define SQLITE_CONFIG_PCACHE 14 /* sqlite3_pcache_methods* */ #define SQLITE_CONFIG_GETPCACHE 15 /* sqlite3_pcache_methods* */ @@ -1762,7 +1756,7 @@ struct sqlite3_mem_methods { ** **
    **
    SQLITE_DBCONFIG_LOOKASIDE
    -**
    This option takes three additional arguments that determine the +**
    This option takes three additional arguments that determine the ** [lookaside memory allocator] configuration for the [database connection]. ** The first argument (the third parameter to [sqlite3_db_config()] is a ** pointer to an memory buffer to use for lookaside memory. @@ -1825,6 +1819,9 @@ SQLITE_API int sqlite3_extended_result_codes(sqlite3*, int onoff); ** For the purposes of this routine, an [INSERT] is considered to ** be successful even if it is subsequently rolled back. ** +** This function is accessible to SQL statements via the +** [last_insert_rowid() SQL function]. +** ** Requirements: ** [H12221] [H12223] ** @@ -1859,7 +1856,7 @@ SQLITE_API sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*); ** mechanisms do not count as direct row changes. ** ** A "trigger context" is a scope of execution that begins and -** ends with the script of a [CREATE TRIGGER | trigger]. +** ends with the script of a [CREATE TRIGGER | trigger]. ** Most SQL statements are ** evaluated outside of any trigger. This is the "top level" ** trigger context. If a trigger fires from the top level, a @@ -1882,8 +1879,8 @@ SQLITE_API sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*); ** However, the number returned does not include changes ** caused by subtriggers since those have their own context. ** -** See also the [sqlite3_total_changes()] interface and the -** [count_changes pragma]. +** See also the [sqlite3_total_changes()] interface, the +** [count_changes pragma], and the [changes() SQL function]. ** ** Requirements: ** [H12241] [H12243] @@ -1899,19 +1896,19 @@ SQLITE_API int sqlite3_changes(sqlite3*); ** ** This function returns the number of row changes caused by [INSERT], ** [UPDATE] or [DELETE] statements since the [database connection] was opened. -** The count includes all changes from all [CREATE TRIGGER | trigger] +** The count includes all changes from all [CREATE TRIGGER | trigger] ** contexts and changes made by [foreign key actions]. However, ** the count does not include changes used to implement [REPLACE] constraints, ** do rollbacks or ABORT processing, or [DROP TABLE] processing. The ** count does not include rows of views that fire an [INSTEAD OF trigger], -** though if the INSTEAD OF trigger makes changes of its own, those changes +** though if the INSTEAD OF trigger makes changes of its own, those changes ** are counted. ** The changes are counted as soon as the statement that makes them is ** completed (when the statement handle is passed to [sqlite3_reset()] or ** [sqlite3_finalize()]). ** -** See also the [sqlite3_changes()] interface and the -** [count_changes pragma]. +** See also the [sqlite3_changes()] interface, the +** [count_changes pragma], and the [total_changes() SQL function]. ** ** Requirements: ** [H12261] [H12263] @@ -1947,7 +1944,7 @@ SQLITE_API int sqlite3_total_changes(sqlite3*); ** ** The sqlite3_interrupt(D) call is in effect until all currently running ** SQL statements on [database connection] D complete. Any new SQL statements -** that are started after the sqlite3_interrupt() call and before the +** that are started after the sqlite3_interrupt() call and before the ** running statements reaches zero are interrupted as if they had been ** running prior to the sqlite3_interrupt() call. New SQL statements ** that are started after the running statement count reaches zero are @@ -1985,7 +1982,7 @@ SQLITE_API void sqlite3_interrupt(sqlite3*); ** These routines do not parse the SQL statements thus ** will not detect syntactically incorrect SQL. ** -** If SQLite has not been initialized using [sqlite3_initialize()] prior +** If SQLite has not been initialized using [sqlite3_initialize()] prior ** to invoking sqlite3_complete16() then sqlite3_initialize() is invoked ** automatically by sqlite3_complete16(). If that initialization fails, ** then the return value from sqlite3_complete16() will be non-zero @@ -2062,7 +2059,7 @@ SQLITE_API int sqlite3_complete16(const void *sql); ** The busy callback should not take any actions which modify the ** database connection that invoked the busy handler. Any such actions ** result in undefined behavior. -** +** ** Requirements: ** [H12311] [H12312] [H12314] [H12316] [H12318] ** @@ -2418,7 +2415,7 @@ SQLITE_API void sqlite3_randomness(int N, void *P); ** requested is ok. When the callback returns [SQLITE_DENY], the ** [sqlite3_prepare_v2()] or equivalent call that triggered the ** authorizer will fail with an error message explaining that -** access is denied. +** access is denied. ** ** The first parameter to the authorizer callback is a copy of the third ** parameter to the sqlite3_set_authorizer() interface. The second parameter @@ -2465,7 +2462,7 @@ SQLITE_API void sqlite3_randomness(int N, void *P); ** database connections for the meaning of "modify" in this paragraph. ** ** When [sqlite3_prepare_v2()] is used to prepare a statement, the -** statement might be re-prepared during [sqlite3_step()] due to a +** statement might be re-prepared during [sqlite3_step()] due to a ** schema change. Hence, the application should ensure that the ** correct authorizer callback remains in place during the [sqlite3_step()]. ** @@ -2631,7 +2628,7 @@ SQLITE_API void sqlite3_progress_handler(sqlite3*, int, int(*)(void*), void*); ** The sqlite3_open_v2() interface works like sqlite3_open() ** except that it accepts two additional parameters for additional control ** over the new database connection. The flags parameter can take one of -** the following three values, optionally combined with the +** the following three values, optionally combined with the ** [SQLITE_OPEN_NOMUTEX], [SQLITE_OPEN_FULLMUTEX], [SQLITE_OPEN_SHAREDCACHE], ** and/or [SQLITE_OPEN_PRIVATECACHE] flags: ** @@ -2719,7 +2716,7 @@ SQLITE_API int sqlite3_open_v2( ** associated with a [database connection]. If a prior API call failed ** but the most recent API call succeeded, the return value from ** sqlite3_errcode() is undefined. The sqlite3_extended_errcode() -** interface is the same except that it always returns the +** interface is the same except that it always returns the ** [extended result code] even when extended result codes are ** disabled. ** @@ -2789,9 +2786,9 @@ typedef struct sqlite3_stmt sqlite3_stmt; ** new limit for that construct. The function returns the old limit. ** ** If the new limit is a negative number, the limit is unchanged. -** For the limit category of SQLITE_LIMIT_XYZ there is a +** For the limit category of SQLITE_LIMIT_XYZ there is a ** [limits | hard upper bound] -** set by a compile-time C preprocessor macro named +** set by a compile-time C preprocessor macro named ** [limits | SQLITE_MAX_XYZ]. ** (The "_LIMIT_" in the name is changed to "_MAX_".) ** Attempts to increase a limit above its hard upper bound are @@ -2925,7 +2922,7 @@ SQLITE_API int sqlite3_limit(sqlite3*, int id, int newVal); ** In the "v2" interfaces, the prepared statement ** that is returned (the [sqlite3_stmt] object) contains a copy of the ** original SQL text. This causes the [sqlite3_step()] interface to -** behave a differently in three ways: +** behave differently in three ways: ** **
      **
    1. @@ -2951,9 +2948,9 @@ SQLITE_API int sqlite3_limit(sqlite3*, int id, int newVal); **
    2. ** ^If the value of a [parameter | host parameter] in the WHERE clause might ** change the query plan for a statement, then the statement may be -** automatically recompiled (as if there had been a schema change) on the first -** [sqlite3_step()] call following any change to the -** [sqlite3_bind_text | bindings] of the [parameter]. +** automatically recompiled (as if there had been a schema change) on the first +** [sqlite3_step()] call following any change to the +** [sqlite3_bind_text | bindings] of the [parameter]. **
      3. **
    ** @@ -3022,7 +3019,7 @@ SQLITE_API const char *sqlite3_sql(sqlite3_stmt *pStmt); ** sqlite3_value object but no mutex is held for an unprotected ** sqlite3_value object. If SQLite is compiled to be single-threaded ** (with [SQLITE_THREADSAFE=0] and with [sqlite3_threadsafe()] returning 0) -** or if SQLite is run in one of reduced mutex modes +** or if SQLite is run in one of reduced mutex modes ** [SQLITE_CONFIG_SINGLETHREAD] or [SQLITE_CONFIG_MULTITHREAD] ** then there is no distinction between protected and unprotected ** sqlite3_value objects and they can be used interchangeably. However, @@ -3754,7 +3751,7 @@ SQLITE_API int sqlite3_reset(sqlite3_stmt *pStmt); ** nArg parameter is a better match than a function implementation with ** a negative nArg. A function where the preferred text encoding ** matches the database encoding is a better -** match than a function where the encoding is different. +** match than a function where the encoding is different. ** A function where the encoding difference is between UTF16le and UTF16be ** is a closer match than a function where the encoding difference is ** between UTF8 and UTF16. @@ -3762,7 +3759,7 @@ SQLITE_API int sqlite3_reset(sqlite3_stmt *pStmt); ** Built-in functions may be overloaded by new application-defined functions. ** The first application-defined function with a given name overrides all ** built-in functions in the same [database connection] with the same name. -** Subsequent application-defined functions of the same name only override +** Subsequent application-defined functions of the same name only override ** prior application-defined functions that are an exact match for the ** number of parameters and preferred encoding. ** @@ -3814,7 +3811,7 @@ SQLITE_API int sqlite3_create_function16( ** DEPRECATED ** ** These functions are [deprecated]. In order to maintain -** backwards compatibility with older code, these functions continue +** backwards compatibility with older code, these functions continue ** to be supported. However, new applications should avoid ** the use of these functions. To help encourage people to avoid ** using these functions, we are not going to tell you what they do. @@ -4194,24 +4191,24 @@ SQLITE_API void sqlite3_result_zeroblob(sqlite3_context*, int n); ** [H16624] [H16627] [H16630] */ SQLITE_API int sqlite3_create_collation( - sqlite3*, - const char *zName, - int eTextRep, + sqlite3*, + const char *zName, + int eTextRep, void*, int(*xCompare)(void*,int,const void*,int,const void*) ); SQLITE_API int sqlite3_create_collation_v2( - sqlite3*, - const char *zName, - int eTextRep, + sqlite3*, + const char *zName, + int eTextRep, void*, int(*xCompare)(void*,int,const void*,int,const void*), void(*xDestroy)(void*) ); SQLITE_API int sqlite3_create_collation16( - sqlite3*, + sqlite3*, const void *zName, - int eTextRep, + int eTextRep, void*, int(*xCompare)(void*,int,const void*,int,const void*) ); @@ -4246,12 +4243,12 @@ SQLITE_API int sqlite3_create_collation16( ** [H16702] [H16704] [H16706] */ SQLITE_API int sqlite3_collation_needed( - sqlite3*, - void*, + sqlite3*, + void*, void(*)(void*,sqlite3*,int eTextRep,const char*) ); SQLITE_API int sqlite3_collation_needed16( - sqlite3*, + sqlite3*, void*, void(*)(void*,sqlite3*,int eTextRep,const void*) ); @@ -4320,7 +4317,7 @@ SQLITE_API int sqlite3_sleep(int); ** The [temp_store_directory pragma] may modify this variable and cause ** it to point to memory obtained from [sqlite3_malloc]. Furthermore, ** the [temp_store_directory pragma] always assumes that any string -** that this variable points to is held in memory obtained from +** that this variable points to is held in memory obtained from ** [sqlite3_malloc] and the pragma may attempt to free that memory ** using [sqlite3_free]. ** Hence, if this variable is modified directly, either it should be @@ -4484,7 +4481,7 @@ SQLITE_API void *sqlite3_rollback_hook(sqlite3*, void(*)(void *), void*); ** [H12971] [H12973] [H12975] [H12977] [H12979] [H12981] [H12983] [H12986] */ SQLITE_API void *sqlite3_update_hook( - sqlite3*, + sqlite3*, void(*)(void *,int ,char const *,char const *,sqlite3_int64), void* ); @@ -4672,6 +4669,8 @@ SQLITE_API int sqlite3_table_column_metadata( ** {H12606} Extension loading must be enabled using ** [sqlite3_enable_load_extension()] prior to calling this API, ** otherwise an error will be returned. +** +** See also the [load_extension() SQL function]. */ SQLITE_API int sqlite3_load_extension( sqlite3 *db, /* Load the extension into this database connection */ @@ -4763,8 +4762,8 @@ typedef struct sqlite3_module sqlite3_module; ** KEYWORDS: sqlite3_module {virtual table module} ** EXPERIMENTAL ** -** This structure, sometimes called a a "virtual table module", -** defines the implementation of a [virtual tables]. +** This structure, sometimes called a a "virtual table module", +** defines the implementation of a [virtual tables]. ** This structure consists mostly of methods for the module. ** ** A virtual table module is created by filling in a persistent @@ -4898,7 +4897,7 @@ struct sqlite3_index_info { ** preexisting [virtual table] for the module. ** ** The module name is registered on the [database connection] specified -** by the first parameter. The name of the module is given by the +** by the first parameter. The name of the module is given by the ** second parameter. The third parameter is a pointer to ** the implementation of the [virtual table module]. The fourth ** parameter is an arbitrary client data pointer that is passed through @@ -4920,10 +4919,10 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_create_module( ** EXPERIMENTAL ** ** This routine is identical to the [sqlite3_create_module()] method, -** except that it has an extra parameter to specify +** except that it has an extra parameter to specify ** a destructor function for the client data pointer. SQLite will ** invoke the destructor function (if it is not NULL) when SQLite -** no longer needs the pClientData pointer. +** no longer needs the pClientData pointer. */ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_create_module_v2( sqlite3 *db, /* SQLite connection to register module with */ @@ -4998,7 +4997,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_declare_vtab(sqlite3*, const char *zS ** EXPERIMENTAL ** ** Virtual tables can provide alternative implementations of functions -** using the [xFindFunction] method of the [virtual table module]. +** using the [xFindFunction] method of the [virtual table module]. ** But global versions of those functions ** must exist in order to be overloaded. ** @@ -5051,8 +5050,8 @@ typedef struct sqlite3_blob sqlite3_blob; ** ** If the flags parameter is non-zero, then the BLOB is opened for read ** and write access. If it is zero, the BLOB is opened for read access. -** It is not possible to open a column that is part of an index or primary -** key for writing. ^If [foreign key constraints] are enabled, it is +** It is not possible to open a column that is part of an index or primary +** key for writing. ^If [foreign key constraints] are enabled, it is ** not possible to open a column that is part of a [child key] for writing. ** ** Note that the database name is not the filename that contains @@ -5137,7 +5136,7 @@ SQLITE_API int sqlite3_blob_close(sqlite3_blob *); /* ** CAPI3REF: Return The Size Of An Open BLOB {H17840} ** -** Returns the size in bytes of the BLOB accessible via the +** Returns the size in bytes of the BLOB accessible via the ** successfully opened [BLOB handle] in its only argument. The ** incremental blob I/O routines can only read or overwriting existing ** blob content; they cannot change the size of a blob. @@ -5513,7 +5512,7 @@ SQLITE_API int sqlite3_mutex_notheld(sqlite3_mutex*); /* ** CAPI3REF: Retrieve the mutex for a database connection {H17002} ** -** This interface returns a pointer the [sqlite3_mutex] object that +** This interface returns a pointer the [sqlite3_mutex] object that ** serializes access to the [database connection] given in the argument ** when the [threading mode] is Serialized. ** If the [threading mode] is Single-thread or Multi-thread then this @@ -5643,12 +5642,12 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_status(int op, int *pCurrent, int *pH **
    This parameter records the largest memory allocation request ** handed to [sqlite3_malloc()] or [sqlite3_realloc()] (or their ** internal equivalents). Only the value returned in the -** *pHighwater parameter to [sqlite3_status()] is of interest. +** *pHighwater parameter to [sqlite3_status()] is of interest. ** The value written into the *pCurrent parameter is undefined.
    ** **
    SQLITE_STATUS_PAGECACHE_USED
    **
    This parameter returns the number of pages used out of the -** [pagecache memory allocator] that was configured using +** [pagecache memory allocator] that was configured using ** [SQLITE_CONFIG_PAGECACHE]. The ** value returned is in pages, not in bytes.
    ** @@ -5664,7 +5663,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_status(int op, int *pCurrent, int *pH **
    SQLITE_STATUS_PAGECACHE_SIZE
    **
    This parameter records the largest memory allocation request ** handed to [pagecache memory allocator]. Only the value returned in the -** *pHighwater parameter to [sqlite3_status()] is of interest. +** *pHighwater parameter to [sqlite3_status()] is of interest. ** The value written into the *pCurrent parameter is undefined.
    ** **
    SQLITE_STATUS_SCRATCH_USED
    @@ -5688,7 +5687,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_status(int op, int *pCurrent, int *pH **
    SQLITE_STATUS_SCRATCH_SIZE
    **
    This parameter records the largest memory allocation request ** handed to [scratch memory allocator]. Only the value returned in the -** *pHighwater parameter to [sqlite3_status()] is of interest. +** *pHighwater parameter to [sqlite3_status()] is of interest. ** The value written into the *pCurrent parameter is undefined.
    ** **
    SQLITE_STATUS_PARSER_STACK
    @@ -5712,7 +5711,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_status(int op, int *pCurrent, int *pH ** CAPI3REF: Database Connection Status {H17500} ** EXPERIMENTAL ** -** This interface is used to retrieve runtime status information +** This interface is used to retrieve runtime status information ** about a single [database connection]. The first argument is the ** database connection object to be interrogated. The second argument ** is the parameter to interrogate. Currently, the only allowed value @@ -5761,13 +5760,13 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_db_status(sqlite3*, int op, int *pCur ** statements. For example, if the number of table steps greatly exceeds ** the number of table searches or result rows, that would tend to indicate ** that the prepared statement is using a full table scan rather than -** an index. +** an index. ** ** This interface is used to retrieve and reset counter values from ** a [prepared statement]. The first argument is the prepared statement ** object to be interrogated. The second argument ** is an integer code for a specific [SQLITE_STMTSTATUS_SORT | counter] -** to be interrogated. +** to be interrogated. ** The current value of the requested counter is returned. ** If the resetFlg is true, then the counter is reset to zero after this ** interface call returns. @@ -5788,7 +5787,7 @@ SQLITE_API SQLITE_EXPERIMENTAL int sqlite3_stmt_status(sqlite3_stmt*, int op,int **
    SQLITE_STMTSTATUS_FULLSCAN_STEP
    **
    This is the number of times that SQLite has stepped forward in ** a table as part of a full table scan. Large numbers for this counter -** may indicate opportunities for performance improvement through +** may indicate opportunities for performance improvement through ** careful use of indices.
    ** **
    SQLITE_STMTSTATUS_SORT
    @@ -5821,14 +5820,14 @@ typedef struct sqlite3_pcache sqlite3_pcache; ** EXPERIMENTAL ** ** The [sqlite3_config]([SQLITE_CONFIG_PCACHE], ...) interface can -** register an alternative page cache implementation by passing in an -** instance of the sqlite3_pcache_methods structure. The majority of the -** heap memory used by SQLite is used by the page cache to cache data read -** from, or ready to be written to, the database file. By implementing a -** custom page cache using this API, an application can control more -** precisely the amount of memory consumed by SQLite, the way in which -** that memory is allocated and released, and the policies used to -** determine exactly which parts of a database file are cached and for +** register an alternative page cache implementation by passing in an +** instance of the sqlite3_pcache_methods structure. The majority of the +** heap memory used by SQLite is used by the page cache to cache data read +** from, or ready to be written to, the database file. By implementing a +** custom page cache using this API, an application can control more +** precisely the amount of memory consumed by SQLite, the way in which +** that memory is allocated and released, and the policies used to +** determine exactly which parts of a database file are cached and for ** how long. ** ** The contents of the sqlite3_pcache_methods structure are copied to an @@ -5839,11 +5838,11 @@ typedef struct sqlite3_pcache sqlite3_pcache; ** The xInit() method is called once for each call to [sqlite3_initialize()] ** (usually only once during the lifetime of the process). It is passed ** a copy of the sqlite3_pcache_methods.pArg value. It can be used to set -** up global structures and mutexes required by the custom page cache -** implementation. +** up global structures and mutexes required by the custom page cache +** implementation. ** -** The xShutdown() method is called from within [sqlite3_shutdown()], -** if the application invokes this API. It can be used to clean up +** The xShutdown() method is called from within [sqlite3_shutdown()], +** if the application invokes this API. It can be used to clean up ** any outstanding resources before process shutdown, if required. ** ** SQLite holds a [SQLITE_MUTEX_RECURSIVE] mutex when it invokes @@ -5884,11 +5883,11 @@ typedef struct sqlite3_pcache sqlite3_pcache; ** ** The xPagecount() method should return the number of pages currently ** stored in the cache. -** -** The xFetch() method is used to fetch a page and return a pointer to it. +** +** The xFetch() method is used to fetch a page and return a pointer to it. ** A 'page', in this context, is a buffer of szPage bytes aligned at an ** 8-byte boundary. The page to be fetched is determined by the key. The -** mimimum key value is 1. After it has been retrieved using xFetch, the page +** mimimum key value is 1. After it has been retrieved using xFetch, the page ** is considered to be "pinned". ** ** If the requested page is already in the page cache, then the page cache @@ -5915,14 +5914,14 @@ typedef struct sqlite3_pcache sqlite3_pcache; ** ** xUnpin() is called by SQLite with a pointer to a currently pinned page ** as its second argument. If the third parameter, discard, is non-zero, -** then the page should be evicted from the cache. In this case SQLite +** then the page should be evicted from the cache. In this case SQLite ** assumes that the next time the page is retrieved from the cache using ** the xFetch() method, it will be zeroed. If the discard parameter is ** zero, then the page is considered to be unpinned. The cache implementation ** may choose to evict unpinned pages at any time. ** -** The cache is not required to perform any reference counting. A single -** call to xUnpin() unpins the page regardless of the number of prior calls +** The cache is not required to perform any reference counting. A single +** call to xUnpin() unpins the page regardless of the number of prior calls ** to xFetch(). ** ** The xRekey() method is used to change the key value associated with the @@ -5977,25 +5976,25 @@ typedef struct sqlite3_backup sqlite3_backup; ** ** This API is used to overwrite the contents of one database with that ** of another. It is useful either for creating backups of databases or -** for copying in-memory databases to or from persistent files. +** for copying in-memory databases to or from persistent files. ** ** See Also: [Using the SQLite Online Backup API] ** -** Exclusive access is required to the destination database for the +** Exclusive access is required to the destination database for the ** duration of the operation. However the source database is only ** read-locked while it is actually being read, it is not locked ** continuously for the entire operation. Thus, the backup may be ** performed on a live database without preventing other users from ** writing to the database for an extended period of time. -** -** To perform a backup operation: +** +** To perform a backup operation: **
      **
    1. sqlite3_backup_init() is called once to initialize the -** backup, -**
    2. sqlite3_backup_step() is called one or more times to transfer +** backup, +**
    3. sqlite3_backup_step() is called one or more times to transfer ** the data between the two databases, and finally -**
    4. sqlite3_backup_finish() is called to release all resources -** associated with the backup operation. +**
    5. sqlite3_backup_finish() is called to release all resources +** associated with the backup operation. **
    ** There should be exactly one call to sqlite3_backup_finish() for each ** successful call to sqlite3_backup_init(). @@ -6003,34 +6002,34 @@ typedef struct sqlite3_backup sqlite3_backup; ** sqlite3_backup_init() ** ** The first two arguments passed to [sqlite3_backup_init()] are the database -** handle associated with the destination database and the database name +** handle associated with the destination database and the database name ** used to attach the destination database to the handle. The database name ** is "main" for the main database, "temp" for the temporary database, or ** the name specified as part of the [ATTACH] statement if the destination is -** an attached database. The third and fourth arguments passed to +** an attached database. The third and fourth arguments passed to ** sqlite3_backup_init() identify the [database connection] ** and database name used -** to access the source database. The values passed for the source and +** to access the source database. The values passed for the source and ** destination [database connection] parameters must not be the same. ** ** If an error occurs within sqlite3_backup_init(), then NULL is returned -** and an error code and error message written into the [database connection] +** and an error code and error message written into the [database connection] ** passed as the first argument. They may be retrieved using the ** [sqlite3_errcode()], [sqlite3_errmsg()], and [sqlite3_errmsg16()] functions. ** Otherwise, if successful, a pointer to an [sqlite3_backup] object is ** returned. This pointer may be used with the sqlite3_backup_step() and -** sqlite3_backup_finish() functions to perform the specified backup +** sqlite3_backup_finish() functions to perform the specified backup ** operation. ** ** sqlite3_backup_step() ** -** Function [sqlite3_backup_step()] is used to copy up to nPage pages between -** the source and destination databases, where nPage is the value of the +** Function [sqlite3_backup_step()] is used to copy up to nPage pages between +** the source and destination databases, where nPage is the value of the ** second parameter passed to sqlite3_backup_step(). If nPage is a negative -** value, all remaining source pages are copied. If the required pages are -** succesfully copied, but there are still more pages to copy before the -** backup is complete, it returns [SQLITE_OK]. If no error occured and there -** are no more pages to copy, then [SQLITE_DONE] is returned. If an error +** value, all remaining source pages are copied. If the required pages are +** succesfully copied, but there are still more pages to copy before the +** backup is complete, it returns [SQLITE_OK]. If no error occured and there +** are no more pages to copy, then [SQLITE_DONE] is returned. If an error ** occurs, then an SQLite error code is returned. As well as [SQLITE_OK] and ** [SQLITE_DONE], a call to sqlite3_backup_step() may return [SQLITE_READONLY], ** [SQLITE_NOMEM], [SQLITE_BUSY], [SQLITE_LOCKED], or an @@ -6043,8 +6042,8 @@ typedef struct sqlite3_backup sqlite3_backup; ** ** If sqlite3_backup_step() cannot obtain a required file-system lock, then ** the [sqlite3_busy_handler | busy-handler function] -** is invoked (if one is specified). If the -** busy-handler returns non-zero before the lock is available, then +** is invoked (if one is specified). If the +** busy-handler returns non-zero before the lock is available, then ** [SQLITE_BUSY] is returned to the caller. In this case the call to ** sqlite3_backup_step() can be retried later. If the source ** [database connection] @@ -6052,16 +6051,16 @@ typedef struct sqlite3_backup sqlite3_backup; ** is called, then [SQLITE_LOCKED] is returned immediately. Again, in this ** case the call to sqlite3_backup_step() can be retried later on. If ** [SQLITE_IOERR_ACCESS | SQLITE_IOERR_XXX], [SQLITE_NOMEM], or -** [SQLITE_READONLY] is returned, then -** there is no point in retrying the call to sqlite3_backup_step(). These -** errors are considered fatal. At this point the application must accept -** that the backup operation has failed and pass the backup operation handle +** [SQLITE_READONLY] is returned, then +** there is no point in retrying the call to sqlite3_backup_step(). These +** errors are considered fatal. At this point the application must accept +** that the backup operation has failed and pass the backup operation handle ** to the sqlite3_backup_finish() to release associated resources. ** ** Following the first call to sqlite3_backup_step(), an exclusive lock is -** obtained on the destination file. It is not released until either -** sqlite3_backup_finish() is called or the backup operation is complete -** and sqlite3_backup_step() returns [SQLITE_DONE]. Additionally, each time +** obtained on the destination file. It is not released until either +** sqlite3_backup_finish() is called or the backup operation is complete +** and sqlite3_backup_step() returns [SQLITE_DONE]. Additionally, each time ** a call to sqlite3_backup_step() is made a [shared lock] is obtained on ** the source database file. This lock is released before the ** sqlite3_backup_step() call returns. Because the source database is not @@ -6069,14 +6068,14 @@ typedef struct sqlite3_backup sqlite3_backup; ** through the backup procedure. If the source database is modified by an ** external process or via a database connection other than the one being ** used by the backup operation, then the backup will be transparently -** restarted by the next call to sqlite3_backup_step(). If the source +** restarted by the next call to sqlite3_backup_step(). If the source ** database is modified by the using the same database connection as is used -** by the backup operation, then the backup database is transparently +** by the backup operation, then the backup database is transparently ** updated at the same time. ** ** sqlite3_backup_finish() ** -** Once sqlite3_backup_step() has returned [SQLITE_DONE], or when the +** Once sqlite3_backup_step() has returned [SQLITE_DONE], or when the ** application wishes to abandon the backup operation, the [sqlite3_backup] ** object should be passed to sqlite3_backup_finish(). This releases all ** resources associated with the backup operation. If sqlite3_backup_step() @@ -6120,7 +6119,7 @@ typedef struct sqlite3_backup sqlite3_backup; ** from within other threads. ** ** However, the application must guarantee that the destination database -** connection handle is not passed to any other API (by any thread) after +** connection handle is not passed to any other API (by any thread) after ** sqlite3_backup_init() is called and before the corresponding call to ** sqlite3_backup_finish(). Unfortunately SQLite does not currently check ** for this, if the application does use the destination [database connection] @@ -6132,11 +6131,11 @@ typedef struct sqlite3_backup sqlite3_backup; ** Furthermore, if running in [shared cache mode], the application must ** guarantee that the shared cache used by the destination database ** is not accessed while the backup is running. In practice this means -** that the application must guarantee that the file-system file being +** that the application must guarantee that the file-system file being ** backed up to is not accessed by any connection within the process, ** not just the specific connection that was passed to sqlite3_backup_init(). ** -** The [sqlite3_backup] object itself is partially threadsafe. Multiple +** The [sqlite3_backup] object itself is partially threadsafe. Multiple ** threads may safely make multiple concurrent calls to sqlite3_backup_step(). ** However, the sqlite3_backup_remaining() and sqlite3_backup_pagecount() ** APIs are not strictly speaking threadsafe. If they are invoked at the @@ -6161,8 +6160,8 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** When running in shared-cache mode, a database operation may fail with ** an [SQLITE_LOCKED] error if the required locks on the shared-cache or ** individual tables within the shared-cache cannot be obtained. See -** [SQLite Shared-Cache Mode] for a description of shared-cache locking. -** This API may be used to register a callback that SQLite will invoke +** [SQLite Shared-Cache Mode] for a description of shared-cache locking. +** This API may be used to register a callback that SQLite will invoke ** when the connection currently holding the required lock relinquishes it. ** This API is only available if the library was compiled with the ** [SQLITE_ENABLE_UNLOCK_NOTIFY] C-preprocessor symbol defined. @@ -6170,14 +6169,14 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** See Also: [Using the SQLite Unlock Notification Feature]. ** ** Shared-cache locks are released when a database connection concludes -** its current transaction, either by committing it or rolling it back. +** its current transaction, either by committing it or rolling it back. ** ** When a connection (known as the blocked connection) fails to obtain a ** shared-cache lock and SQLITE_LOCKED is returned to the caller, the ** identity of the database connection (the blocking connection) that -** has locked the required resource is stored internally. After an +** has locked the required resource is stored internally. After an ** application receives an SQLITE_LOCKED error, it may call the -** sqlite3_unlock_notify() method with the blocked connection handle as +** sqlite3_unlock_notify() method with the blocked connection handle as ** the first argument to register for a callback that will be invoked ** when the blocking connections current transaction is concluded. The ** callback is invoked from within the [sqlite3_step] or [sqlite3_close] @@ -6191,15 +6190,15 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** ** If the blocked connection is attempting to obtain a write-lock on a ** shared-cache table, and more than one other connection currently holds -** a read-lock on the same table, then SQLite arbitrarily selects one of +** a read-lock on the same table, then SQLite arbitrarily selects one of ** the other connections to use as the blocking connection. ** -** There may be at most one unlock-notify callback registered by a +** There may be at most one unlock-notify callback registered by a ** blocked connection. If sqlite3_unlock_notify() is called when the ** blocked connection already has a registered unlock-notify callback, ** then the new callback replaces the old. If sqlite3_unlock_notify() is ** called with a NULL pointer as its second argument, then any existing -** unlock-notify callback is cancelled. The blocked connections +** unlock-notify callback is cancelled. The blocked connections ** unlock-notify callback may also be canceled by closing the blocked ** connection using [sqlite3_close()]. ** @@ -6212,7 +6211,7 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** ** Callback Invocation Details ** -** When an unlock-notify callback is registered, the application provides a +** When an unlock-notify callback is registered, the application provides a ** single void* pointer that is passed to the callback when it is invoked. ** However, the signature of the callback function allows SQLite to pass ** it an array of void* context pointers. The first argument passed to @@ -6225,12 +6224,12 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** same callback function, then instead of invoking the callback function ** multiple times, it is invoked once with the set of void* context pointers ** specified by the blocked connections bundled together into an array. -** This gives the application an opportunity to prioritize any actions +** This gives the application an opportunity to prioritize any actions ** related to the set of unblocked database connections. ** ** Deadlock Detection ** -** Assuming that after registering for an unlock-notify callback a +** Assuming that after registering for an unlock-notify callback a ** database waits for the callback to be issued before taking any further ** action (a reasonable assumption), then using this API may cause the ** application to deadlock. For example, if connection X is waiting for @@ -6253,7 +6252,7 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** ** The "DROP TABLE" Exception ** -** When a call to [sqlite3_step()] returns SQLITE_LOCKED, it is almost +** When a call to [sqlite3_step()] returns SQLITE_LOCKED, it is almost ** always appropriate to call sqlite3_unlock_notify(). There is however, ** one exception. When executing a "DROP TABLE" or "DROP INDEX" statement, ** SQLite checks if there are any currently executing SELECT statements @@ -6266,7 +6265,7 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p); ** One way around this problem is to check the extended error code returned ** by an sqlite3_step() call. If there is a blocking connection, then the ** extended error code is set to SQLITE_LOCKED_SHAREDCACHE. Otherwise, in -** the special "DROP TABLE/INDEX" case, the extended error code is just +** the special "DROP TABLE/INDEX" case, the extended error code is just ** SQLITE_LOCKED. */ SQLITE_API int sqlite3_unlock_notify( @@ -6282,7 +6281,7 @@ SQLITE_API int sqlite3_unlock_notify( ** ** The [sqlite3_strnicmp()] API allows applications and extensions to ** compare the contents of two buffers containing UTF-8 strings in a -** case-indendent fashion, using the same definition of case independence +** case-indendent fashion, using the same definition of case independence ** that SQLite uses internally when comparing identifiers. */ SQLITE_API int sqlite3_strnicmp(const char *, const char *, int); @@ -6318,8 +6317,6 @@ SQLITE_API int sqlite3_strnicmp(const char *, const char *, int); ************************************************************************* ** This is the header file for the generic hash-table implemenation ** used in SQLite. -** -** $Id: hash.h,v 1.15 2009/05/02 13:29:38 drh Exp $ */ #ifndef _SQLITE_HASH_H_ #define _SQLITE_HASH_H_ @@ -6344,7 +6341,7 @@ typedef struct HashElem HashElem; ** element pointed to plus the next _ht.count-1 elements in the list. ** ** Hash.htsize and Hash.ht may be zero. In that case lookup is done -** by a linear search of the global list. For small tables, the +** by a linear search of the global list. For small tables, the ** Hash.ht table is never allocated because if there are few elements ** in the table, it is faster to do a linear search than to manage ** the hash table. @@ -6359,7 +6356,7 @@ struct Hash { } *ht; }; -/* Each element in the hash table is an instance of the following +/* Each element in the hash table is an instance of the following ** structure. All elements are stored on a single doubly-linked list. ** ** Again, this structure is intended to be opaque, but it can't really @@ -6595,7 +6592,7 @@ SQLITE_PRIVATE void sqlite3HashClear(Hash*); /* ** OMIT_TEMPDB is set to 1 if SQLITE_OMIT_TEMPDB is defined, or 0 -** afterward. Having this macro allows us to cause the C compiler +** afterward. Having this macro allows us to cause the C compiler ** to omit code used by TEMP tables without messy #ifndef statements. */ #ifdef SQLITE_OMIT_TEMPDB @@ -6748,7 +6745,7 @@ SQLITE_PRIVATE const int sqlite3one; #define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) #define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) -/* +/* ** Round up a number to the next larger multiple of 8. This is used ** to force 8-byte alignment on 64-bit architectures. */ @@ -6760,14 +6757,24 @@ SQLITE_PRIVATE const int sqlite3one; #define ROUNDDOWN8(x) ((x)&~7) /* -** Assert that the pointer X is aligned to an 8-byte boundary. +** Assert that the pointer X is aligned to an 8-byte boundary. This +** macro is used only within assert() to verify that the code gets +** all alignment restrictions correct. +** +** Except, if SQLITE_4_BYTE_ALIGNED_MALLOC is defined, then the +** underlying malloc() implemention might return us 4-byte aligned +** pointers. In that case, only verify 4-byte alignment. */ -#define EIGHT_BYTE_ALIGNMENT(X) ((((char*)(X) - (char*)0)&7)==0) +#ifdef SQLITE_4_BYTE_ALIGNED_MALLOC +# define EIGHT_BYTE_ALIGNMENT(X) ((((char*)(X) - (char*)0)&3)==0) +#else +# define EIGHT_BYTE_ALIGNMENT(X) ((((char*)(X) - (char*)0)&7)==0) +#endif /* ** An instance of the following structure is used to store the busy-handler -** callback for a given sqlite handle. +** callback for a given sqlite handle. ** ** The sqlite.busyHandler member of the sqlite struct contains the busy ** callback for the database handle. Each pager opened via the sqlite @@ -6832,16 +6839,16 @@ struct BusyHandler { SQLITE_API int sqlite3_wsd_init(int N, int J); SQLITE_API void *sqlite3_wsd_find(void *K, int L); #else - #define SQLITE_WSD + #define SQLITE_WSD #define GLOBAL(t,v) v #define sqlite3GlobalConfig sqlite3Config #endif /* ** The following macros are used to suppress compiler warnings and to -** make it clear to human readers when a function parameter is deliberately +** make it clear to human readers when a function parameter is deliberately ** left unused within the body of a function. This usually happens when -** a function is called via a function pointer. For example the +** a function is called via a function pointer. For example the ** implementation of an SQL aggregate step callback may not use the ** parameter indicating the number of arguments passed to the aggregate, ** if it knows that this is enforced elsewhere. @@ -6902,7 +6909,7 @@ typedef struct WhereInfo WhereInfo; typedef struct WhereLevel WhereLevel; /* -** Defer sourcing vdbe.h and btree.h until after the "u8" and +** Defer sourcing vdbe.h and btree.h until after the "u8" and ** "BusyHandler" typedefs. vdbe.h also requires a few of the opaque ** pointer types (i.e. FuncDef) defined above. */ @@ -6922,8 +6929,6 @@ typedef struct WhereLevel WhereLevel; ** This header file defines the interface that the sqlite B-Tree file ** subsystem. See comments in the source code for a detailed description ** of what each interface routine does. -** -** @(#) $Id: btree.h,v 1.120 2009/07/22 00:35:24 drh Exp $ */ #ifndef _BTREE_H_ #define _BTREE_H_ @@ -7033,7 +7038,7 @@ SQLITE_PRIVATE int sqlite3BtreeUpdateMeta(Btree*, int idx, u32 value); /* ** The second parameter to sqlite3BtreeGetMeta or sqlite3BtreeUpdateMeta -** should be one of the following values. The integer values are assigned +** should be one of the following values. The integer values are assigned ** to constants so that the offset of the corresponding field in an ** SQLite database header may be found using the following formula: ** @@ -7060,6 +7065,7 @@ SQLITE_PRIVATE int sqlite3BtreeCursor( BtCursor *pCursor /* Space to write cursor structure */ ); SQLITE_PRIVATE int sqlite3BtreeCursorSize(void); +SQLITE_PRIVATE void sqlite3BtreeCursorZero(BtCursor*); SQLITE_PRIVATE int sqlite3BtreeCloseCursor(BtCursor*); SQLITE_PRIVATE int sqlite3BtreeMovetoUnpacked( @@ -7117,7 +7123,7 @@ SQLITE_PRIVATE void sqlite3BtreeCursorList(Btree*); SQLITE_PRIVATE void sqlite3BtreeEnter(Btree*); SQLITE_PRIVATE void sqlite3BtreeEnterAll(sqlite3*); #else -# define sqlite3BtreeEnter(X) +# define sqlite3BtreeEnter(X) # define sqlite3BtreeEnterAll(X) #endif @@ -7171,8 +7177,6 @@ SQLITE_PRIVATE int sqlite3BtreeHoldsAllMutexes(sqlite3*); ** This header defines the interface to the virtual database engine ** or VDBE. The VDBE implements an abstract machine that runs a ** simple program to access and modify the underlying database. -** -** $Id: vdbe.h,v 1.142 2009/07/24 17:58:53 danielk1977 Exp $ */ #ifndef _SQLITE_VDBE_H_ #define _SQLITE_VDBE_H_ @@ -7200,7 +7204,7 @@ typedef struct SubProgram SubProgram; struct VdbeOp { u8 opcode; /* What operation to perform */ signed char p4type; /* One of the P4_xxx constants for p4 */ - u8 opflags; /* Not currently used */ + u8 opflags; /* Mask of the OPFLG_* flags in opcodes.h */ u8 p5; /* Fifth parameter is an unsigned character */ int p1; /* First operand */ int p2; /* Second parameter (often the jump destination) */ @@ -7286,7 +7290,7 @@ typedef struct VdbeOpList VdbeOpList; #define P4_KEYINFO_STATIC (-17) /* -** The Vdbe.aColName array contains 5n Mem structures, where n is the +** The Vdbe.aColName array contains 5n Mem structures, where n is the ** number of columns of data returned by the statement. */ #define COLNAME_NAME 0 @@ -7306,7 +7310,7 @@ typedef struct VdbeOpList VdbeOpList; /* ** The following macro converts a relative address in the p2 field -** of a VdbeOp structure into a negative number so that +** of a VdbeOp structure into a negative number so that ** sqlite3VdbeAddOpList() knows that the address is relative. Calling ** the macro again restores the address. */ @@ -7478,22 +7482,23 @@ typedef struct VdbeOpList VdbeOpList; #define OPFLG_IN1 0x0004 /* in1: P1 is an input */ #define OPFLG_IN2 0x0008 /* in2: P2 is an input */ #define OPFLG_IN3 0x0010 /* in3: P3 is an input */ -#define OPFLG_OUT3 0x0020 /* out3: P3 is an output */ +#define OPFLG_OUT2 0x0020 /* out2: P2 is an output */ +#define OPFLG_OUT3 0x0040 /* out3: P3 is an output */ #define OPFLG_INITIALIZER {\ -/* 0 */ 0x00, 0x01, 0x01, 0x04, 0x04, 0x10, 0x00, 0x02,\ -/* 8 */ 0x02, 0x02, 0x02, 0x02, 0x00, 0x00, 0x04, 0x04,\ -/* 16 */ 0x00, 0x00, 0x00, 0x04, 0x04, 0x05, 0x04, 0x00,\ +/* 0 */ 0x00, 0x01, 0x05, 0x04, 0x04, 0x10, 0x00, 0x02,\ +/* 8 */ 0x02, 0x02, 0x02, 0x02, 0x00, 0x00, 0x24, 0x24,\ +/* 16 */ 0x00, 0x00, 0x00, 0x24, 0x04, 0x05, 0x04, 0x00,\ /* 24 */ 0x00, 0x01, 0x05, 0x05, 0x00, 0x00, 0x00, 0x02,\ /* 32 */ 0x00, 0x00, 0x00, 0x02, 0x10, 0x00, 0x00, 0x00,\ /* 40 */ 0x00, 0x00, 0x00, 0x11, 0x11, 0x11, 0x11, 0x08,\ /* 48 */ 0x11, 0x11, 0x11, 0x11, 0x02, 0x02, 0x00, 0x00,\ /* 56 */ 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x01, 0x01,\ -/* 64 */ 0x01, 0x01, 0x01, 0x08, 0x2c, 0x2c, 0x00, 0x02,\ -/* 72 */ 0x11, 0x05, 0x05, 0x15, 0x15, 0x15, 0x15, 0x15,\ -/* 80 */ 0x15, 0x11, 0x2c, 0x2c, 0x2c, 0x2c, 0x2c, 0x2c,\ -/* 88 */ 0x2c, 0x2c, 0x2c, 0x2c, 0x02, 0x04, 0x02, 0x00,\ +/* 64 */ 0x01, 0x01, 0x01, 0x08, 0x4c, 0x4c, 0x00, 0x02,\ +/* 72 */ 0x01, 0x05, 0x05, 0x15, 0x15, 0x15, 0x15, 0x15,\ +/* 80 */ 0x15, 0x01, 0x4c, 0x4c, 0x4c, 0x4c, 0x4c, 0x4c,\ +/* 88 */ 0x4c, 0x4c, 0x4c, 0x4c, 0x02, 0x24, 0x02, 0x00,\ /* 96 */ 0x02, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\ -/* 104 */ 0x08, 0x21, 0x15, 0x01, 0x02, 0x00, 0x01, 0x08,\ +/* 104 */ 0x0c, 0x45, 0x15, 0x01, 0x02, 0x00, 0x01, 0x08,\ /* 112 */ 0x05, 0x05, 0x05, 0x00, 0x00, 0x00, 0x01, 0x00,\ /* 120 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01,\ /* 128 */ 0x00, 0x00, 0x02, 0x02, 0x00, 0x00, 0x00, 0x00,\ @@ -7513,6 +7518,7 @@ SQLITE_PRIVATE int sqlite3VdbeAddOp1(Vdbe*,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int); +SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp); SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1); SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2); @@ -7545,6 +7551,9 @@ SQLITE_PRIVATE VdbeOp *sqlite3VdbeTakeOpArray(Vdbe*, int*, int*); SQLITE_PRIVATE void sqlite3VdbeProgramDelete(sqlite3 *, SubProgram *, int); SQLITE_PRIVATE sqlite3_value *sqlite3VdbeGetValue(Vdbe*, int, u8); SQLITE_PRIVATE void sqlite3VdbeSetVarmask(Vdbe*, int); +#ifndef SQLITE_OMIT_TRACE +SQLITE_PRIVATE char *sqlite3VdbeExpandSql(Vdbe*, const char*); +#endif SQLITE_PRIVATE UnpackedRecord *sqlite3VdbeRecordUnpack(KeyInfo*,int,const void*,char*,int); SQLITE_PRIVATE void sqlite3VdbeDeleteUnpackedRecord(UnpackedRecord*); @@ -7581,16 +7590,14 @@ SQLITE_PRIVATE void sqlite3VdbeNoopComment(Vdbe*, const char*, ...); ** This header file defines the interface that the sqlite page cache ** subsystem. The page cache subsystem reads and writes a file a page ** at a time and provides a journal for rollback. -** -** @(#) $Id: pager.h,v 1.104 2009/07/24 19:01:19 drh Exp $ */ #ifndef _PAGER_H_ #define _PAGER_H_ /* -** Default maximum size for persistent journal files. A negative -** value means no limit. This value may be overridden using the +** Default maximum size for persistent journal files. A negative +** value means no limit. This value may be overridden using the ** sqlite3PagerJournalSizeLimit() API. See also "PRAGMA journal_size_limit". */ #ifndef SQLITE_DEFAULT_JOURNAL_SIZE_LIMIT @@ -7616,9 +7623,9 @@ typedef struct PgHdr DbPage; /* ** Page number PAGER_MJ_PGNO is never used in an SQLite database (it is ** reserved for working around a windows/posix incompatibility). It is -** used in the journal to signify that the remainder of the journal file +** used in the journal to signify that the remainder of the journal file ** is devoted to storing a master journal name - there are no more pages to -** roll back. See comments for function writeMasterJournal() in pager.c +** roll back. See comments for function writeMasterJournal() in pager.c ** for details. */ #define PAGER_MJ_PGNO(x) ((Pgno)((PENDING_BYTE/((x)->pageSize))+1)) @@ -7650,11 +7657,11 @@ typedef struct PgHdr DbPage; /* ** The remainder of this file contains the declarations of the functions -** that make up the Pager sub-system API. See source code comments for +** that make up the Pager sub-system API. See source code comments for ** a detailed description of each routine. */ -/* Open and close a Pager connection. */ +/* Open and close a Pager connection. */ SQLITE_PRIVATE int sqlite3PagerOpen( sqlite3_vfs*, Pager **ppPager, @@ -7678,7 +7685,7 @@ SQLITE_PRIVATE int sqlite3PagerJournalMode(Pager *, int); SQLITE_PRIVATE i64 sqlite3PagerJournalSizeLimit(Pager *, i64); SQLITE_PRIVATE sqlite3_backup **sqlite3PagerBackupPtr(Pager*); -/* Functions used to obtain and release page references. */ +/* Functions used to obtain and release page references. */ SQLITE_PRIVATE int sqlite3PagerAcquire(Pager *pPager, Pgno pgno, DbPage **ppPage, int clrFlag); #define sqlite3PagerGet(A,B,C) sqlite3PagerAcquire(A,B,C,0) SQLITE_PRIVATE DbPage *sqlite3PagerLookup(Pager *pPager, Pgno pgno); @@ -7690,8 +7697,8 @@ SQLITE_PRIVATE int sqlite3PagerWrite(DbPage*); SQLITE_PRIVATE void sqlite3PagerDontWrite(DbPage*); SQLITE_PRIVATE int sqlite3PagerMovepage(Pager*,DbPage*,Pgno,int); SQLITE_PRIVATE int sqlite3PagerPageRefcount(DbPage*); -SQLITE_PRIVATE void *sqlite3PagerGetData(DbPage *); -SQLITE_PRIVATE void *sqlite3PagerGetExtra(DbPage *); +SQLITE_PRIVATE void *sqlite3PagerGetData(DbPage *); +SQLITE_PRIVATE void *sqlite3PagerGetExtra(DbPage *); /* Functions used to manage pager transactions and savepoints. */ SQLITE_PRIVATE int sqlite3PagerPagecount(Pager*, int*); @@ -7751,9 +7758,7 @@ SQLITE_PRIVATE void sqlite3PagerRefdump(Pager*); ** ************************************************************************* ** This header file defines the interface that the sqlite page cache -** subsystem. -** -** @(#) $Id: pcache.h,v 1.20 2009/07/25 11:46:49 danielk1977 Exp $ +** subsystem. */ #ifndef _PCACHE_H_ @@ -7826,7 +7831,7 @@ SQLITE_PRIVATE void sqlite3PcacheSetPageSize(PCache *, int); SQLITE_PRIVATE int sqlite3PcacheSize(void); /* One release per successful fetch. Page is pinned until released. -** Reference counted. +** Reference counted. */ SQLITE_PRIVATE int sqlite3PcacheFetch(PCache*, Pgno, int createFlag, PgHdr**); SQLITE_PRIVATE void sqlite3PcacheRelease(PgHdr*); @@ -7867,7 +7872,7 @@ SQLITE_PRIVATE int sqlite3PcachePagecount(PCache*); #if defined(SQLITE_CHECK_PAGES) || defined(SQLITE_DEBUG) /* Iterate through all dirty pages currently stored in the cache. This -** interface is only available if SQLITE_CHECK_PAGES is defined when the +** interface is only available if SQLITE_CHECK_PAGES is defined when the ** library is built. */ SQLITE_PRIVATE void sqlite3PcacheIterateDirty(PCache *pCache, void (*xIter)(PgHdr *)); @@ -7920,8 +7925,6 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); ** ** This header file is #include-ed by sqliteInt.h and thus ends up ** being included by every source file. -** -** $Id: os.h,v 1.108 2009/02/05 16:31:46 drh Exp $ */ #ifndef _SQLITE_OS_H_ #define _SQLITE_OS_H_ @@ -7929,8 +7932,8 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); /* ** Figure out if we are dealing with Unix, Windows, or some other ** operating system. After the following block of preprocess macros, -** all of SQLITE_OS_UNIX, SQLITE_OS_WIN, SQLITE_OS_OS2, and SQLITE_OS_OTHER -** will defined to either 1 or 0. One of the four will be 1. The other +** all of SQLITE_OS_UNIX, SQLITE_OS_WIN, SQLITE_OS_OS2, and SQLITE_OS_OTHER +** will defined to either 1 or 0. One of the four will be 1. The other ** three will be 0. */ #if defined(SQLITE_OS_OTHER) @@ -8032,10 +8035,10 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); ** 2006-10-31: The default prefix used to be "sqlite_". But then ** Mcafee started using SQLite in their anti-virus product and it ** started putting files with the "sqlite" name in the c:/temp folder. -** This annoyed many windows users. Those users would then do a +** This annoyed many windows users. Those users would then do a ** Google search for "sqlite", find the telephone numbers of the ** developers and call to wake them up at night and complain. -** For this reason, the default name prefix is changed to be "sqlite" +** For this reason, the default name prefix is changed to be "sqlite" ** spelled backwards. So the temp files are still identified, but ** anybody smart enough to figure out the code is also likely smart ** enough to know that calling the developer will not help get rid @@ -8076,9 +8079,9 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); ** UnlockFile(). ** ** LockFile() prevents not just writing but also reading by other processes. -** A SHARED_LOCK is obtained by locking a single randomly-chosen -** byte out of a specific range of bytes. The lock byte is obtained at -** random so two separate readers can probably access the file at the +** A SHARED_LOCK is obtained by locking a single randomly-chosen +** byte out of a specific range of bytes. The lock byte is obtained at +** random so two separate readers can probably access the file at the ** same time, unless they are unlucky and choose the same lock byte. ** An EXCLUSIVE_LOCK is obtained by locking all bytes in the range. ** There can only be one writer. A RESERVED_LOCK is obtained by locking @@ -8097,7 +8100,7 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); ** The following #defines specify the range of bytes used for locking. ** SHARED_SIZE is the number of bytes available in the pool from which ** a random byte is selected for a shared lock. The pool of bytes for -** shared locks begins at SHARED_FIRST. +** shared locks begins at SHARED_FIRST. ** ** The same locking strategy and ** byte ranges are used for Unix. This leaves open the possiblity of having @@ -8113,7 +8116,7 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); ** that all locks will fit on a single page even at the minimum page size. ** PENDING_BYTE defines the beginning of the locks. By default PENDING_BYTE ** is set high so that we don't have to allocate an unused page except -** for very large databases. But one should test the page skipping logic +** for very large databases. But one should test the page skipping logic ** by setting PENDING_BYTE low and running the entire regression suite. ** ** Changing the value of PENDING_BYTE results in a subtly incompatible @@ -8133,8 +8136,8 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void); */ SQLITE_PRIVATE int sqlite3OsInit(void); -/* -** Functions for accessing sqlite3_file methods +/* +** Functions for accessing sqlite3_file methods */ SQLITE_PRIVATE int sqlite3OsClose(sqlite3_file*); SQLITE_PRIVATE int sqlite3OsRead(sqlite3_file*, void*, int amt, i64 offset); @@ -8150,8 +8153,8 @@ SQLITE_PRIVATE int sqlite3OsFileControl(sqlite3_file*,int,void*); SQLITE_PRIVATE int sqlite3OsSectorSize(sqlite3_file *id); SQLITE_PRIVATE int sqlite3OsDeviceCharacteristics(sqlite3_file *id); -/* -** Functions for accessing sqlite3_vfs methods +/* +** Functions for accessing sqlite3_vfs methods */ SQLITE_PRIVATE int sqlite3OsOpen(sqlite3_vfs *, const char *, sqlite3_file*, int, int *); SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *, const char *, int); @@ -8168,7 +8171,7 @@ SQLITE_PRIVATE int sqlite3OsSleep(sqlite3_vfs *, int); SQLITE_PRIVATE int sqlite3OsCurrentTime(sqlite3_vfs *, double*); /* -** Convenience functions for opening and closing files using +** Convenience functions for opening and closing files using ** sqlite3_malloc() to obtain space for the file-handle structure. */ SQLITE_PRIVATE int sqlite3OsOpenMalloc(sqlite3_vfs *, const char *, sqlite3_file **, int,int*); @@ -8200,8 +8203,6 @@ SQLITE_PRIVATE int sqlite3OsCloseFree(sqlite3_file *); ** NOTE: source files should *not* #include this header file directly. ** Source files should #include the sqliteInt.h file and let that file ** include this one indirectly. -** -** $Id: mutex.h,v 1.9 2008/10/07 15:25:48 drh Exp $ */ @@ -8278,10 +8279,10 @@ struct Db { ** ** If there are no virtual tables configured in this schema, the ** Schema.db variable is set to NULL. After the first virtual table -** has been added, it is set to point to the database connection +** has been added, it is set to point to the database connection ** used to create the connection. Once a virtual table has been -** added to the Schema structure and the Schema.db variable populated, -** only that database connection may use the Schema to prepare +** added to the Schema structure and the Schema.db variable populated, +** only that database connection may use the Schema to prepare ** statements. */ struct Schema { @@ -8301,7 +8302,7 @@ struct Schema { }; /* -** These macros can be used to test, set, or clear bits in the +** These macros can be used to test, set, or clear bits in the ** Db.flags field. */ #define DbHasProperty(D,I,P) (((D)->aDb[I].pSchema->flags&(P))==(P)) @@ -8437,9 +8438,9 @@ struct sqlite3 { void *pTraceArg; /* Argument to the trace function */ void (*xProfile)(void*,const char*,u64); /* Profiling function */ void *pProfileArg; /* Argument to profile function */ - void *pCommitArg; /* Argument to xCommitCallback() */ + void *pCommitArg; /* Argument to xCommitCallback() */ int (*xCommitCallback)(void*); /* Invoked at every commit. */ - void *pRollbackArg; /* Argument to xRollbackCallback() */ + void *pRollbackArg; /* Argument to xRollbackCallback() */ void (*xRollbackCallback)(void*); /* Invoked at every commit. */ void *pUpdateArg; void (*xUpdateCallback)(void*,int, const char*,const char*,sqlite_int64); @@ -8483,8 +8484,8 @@ struct sqlite3 { i64 nDeferredCons; /* Net deferred constraints this transaction. */ #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY - /* The following variables are all protected by the STATIC_MASTER - ** mutex, not by sqlite3.mutex. They are used by code in notify.c. + /* The following variables are all protected by the STATIC_MASTER + ** mutex, not by sqlite3.mutex. They are used by code in notify.c. ** ** When X.pUnlockConnection==Y, that means that X is waiting for Y to ** unlock so that it can proceed. @@ -8526,7 +8527,7 @@ struct sqlite3 { #define SQLITE_SqlTrace 0x00000200 /* Debug print SQL as it executes */ #define SQLITE_VdbeListing 0x00000400 /* Debug listings of VDBE programs */ #define SQLITE_WriteSchema 0x00000800 /* OK to update SQLITE_MASTER */ -#define SQLITE_NoReadlock 0x00001000 /* Readlocks are omitted when +#define SQLITE_NoReadlock 0x00001000 /* Readlocks are omitted when ** accessing read-only databases */ #define SQLITE_IgnoreChecks 0x00002000 /* Do not enforce check constraints */ #define SQLITE_ReadUncommitted 0x00004000 /* For shared-cache mode */ @@ -8578,16 +8579,17 @@ struct FuncDef { #define SQLITE_FUNC_NEEDCOLL 0x08 /* sqlite3GetFuncCollSeq() might be called */ #define SQLITE_FUNC_PRIVATE 0x10 /* Allowed for internal use only */ #define SQLITE_FUNC_COUNT 0x20 /* Built-in count(*) aggregate */ +#define SQLITE_FUNC_COALESCE 0x40 /* Built-in coalesce() or ifnull() function */ /* ** The following three macros, FUNCTION(), LIKEFUNC() and AGGREGATE() are ** used to create the initializers for the FuncDef structures. ** ** FUNCTION(zName, nArg, iArg, bNC, xFunc) -** Used to create a scalar function definition of a function zName +** Used to create a scalar function definition of a function zName ** implemented by C function xFunc that accepts nArg arguments. The ** value passed as iArg is cast to a (void*) and made available -** as the user-data (sqlite3_user_data()) for the function. If +** as the user-data (sqlite3_user_data()) for the function. If ** argument bNC is true, then the SQLITE_FUNC_NEEDCOLL flag is set. ** ** AGGREGATE(zName, nArg, iArg, bNC, xStep, xFinal) @@ -8597,8 +8599,8 @@ struct FuncDef { ** FUNCTION(). ** ** LIKEFUNC(zName, nArg, pArg, flags) -** Used to create a scalar function definition of a function zName -** that accepts nArg arguments and is implemented by a call to C +** Used to create a scalar function definition of a function zName +** that accepts nArg arguments and is implemented by a call to C ** function likeFunc. Argument pArg is cast to a (void *) and made ** available as the function user-data (sqlite3_user_data()). The ** FuncDef.flags variable is set to the value passed as the flags @@ -8716,7 +8718,7 @@ struct CollSeq { ** ** These used to have mnemonic name like 'i' for SQLITE_AFF_INTEGER and ** 't' for SQLITE_AFF_TEXT. But we can save a little space and improve -** the speed a little by numbering the values consecutively. +** the speed a little by numbering the values consecutively. ** ** But rather than start with 0 or 1, we begin with 'a'. That way, ** when multiple affinity types are concatenated into a string and @@ -8735,7 +8737,7 @@ struct CollSeq { /* ** The SQLITE_AFF_MASK values masks off the significant bits of an -** affinity value. +** affinity value. */ #define SQLITE_AFF_MASK 0x67 @@ -8749,20 +8751,20 @@ struct CollSeq { /* ** An object of this type is created for each virtual table present in -** the database schema. +** the database schema. ** ** If the database schema is shared, then there is one instance of this ** structure for each database connection (sqlite3*) that uses the shared ** schema. This is because each database connection requires its own unique -** instance of the sqlite3_vtab* handle used to access the virtual table -** implementation. sqlite3_vtab* handles can not be shared between -** database connections, even when the rest of the in-memory database +** instance of the sqlite3_vtab* handle used to access the virtual table +** implementation. sqlite3_vtab* handles can not be shared between +** database connections, even when the rest of the in-memory database ** schema is shared, as the implementation often stores the database ** connection handle passed to it via the xConnect() or xCreate() method ** during initialization internally. This database connection handle may -** then used by the virtual table implementation to access real tables -** within the database. So that they appear as part of the callers -** transaction, these accesses need to be made via the same database +** then used by the virtual table implementation to access real tables +** within the database. So that they appear as part of the callers +** transaction, these accesses need to be made via the same database ** connection as that used to execute SQL operations on the virtual table. ** ** All VTable objects that correspond to a single table in a shared @@ -8774,19 +8776,19 @@ struct CollSeq { ** sqlite3_vtab* handle in the compiled query. ** ** When an in-memory Table object is deleted (for example when the -** schema is being reloaded for some reason), the VTable objects are not -** deleted and the sqlite3_vtab* handles are not xDisconnect()ed +** schema is being reloaded for some reason), the VTable objects are not +** deleted and the sqlite3_vtab* handles are not xDisconnect()ed ** immediately. Instead, they are moved from the Table.pVTable list to ** another linked list headed by the sqlite3.pDisconnect member of the -** corresponding sqlite3 structure. They are then deleted/xDisconnected +** corresponding sqlite3 structure. They are then deleted/xDisconnected ** next time a statement is prepared using said sqlite3*. This is done ** to avoid deadlock issues involving multiple sqlite3.mutex mutexes. ** Refer to comments above function sqlite3VtabUnlockList() for an ** explanation as to why it is safe to add an entry to an sqlite3.pDisconnect ** list without holding the corresponding sqlite3.mutex mutex. ** -** The memory for objects of this type is always allocated by -** sqlite3DbMalloc(), using the connection handle stored in VTable.db as +** The memory for objects of this type is always allocated by +** sqlite3DbMalloc(), using the connection handle stored in VTable.db as ** the first argument. */ struct VTable { @@ -8821,10 +8823,10 @@ struct VTable { ** in sqlite.aDb[]. 0 is for the main database and 1 is for the file that ** holds temporary tables and indices. If TF_Ephemeral is set ** then the table is stored in a file that is automatically deleted -** when the VDBE cursor to the table is closed. In this case Table.tnum +** when the VDBE cursor to the table is closed. In this case Table.tnum ** refers VDBE cursor number that holds the table open, not to the root ** page number. Transient tables are used to hold the results of a -** sub-query that appears instead of a real table name in the FROM clause +** sub-query that appears instead of a real table name in the FROM clause ** of a SELECT statement. */ struct Table { @@ -8939,7 +8941,7 @@ struct FKey { ** key is set to NULL. CASCADE means that a DELETE or UPDATE of the ** referenced table row is propagated into the row that holds the ** foreign key. -** +** ** The following symbolic values are used to record which type ** of action to take. */ @@ -8960,7 +8962,7 @@ struct FKey { /* ** An instance of the following structure is passed as the first -** argument to sqlite3VdbeKeyCompare and is used to control the +** argument to sqlite3VdbeKeyCompare and is used to control the ** comparison of the two index keys. */ struct KeyInfo { @@ -9017,7 +9019,7 @@ struct UnpackedRecord { ** In the Table structure describing Ex1, nCol==3 because there are ** three columns in the table. In the Index structure describing ** Ex2, nColumn==2 since 2 of the 3 columns of Ex1 are indexed. -** The value of aiColumn is {2, 0}. aiColumn[0]==2 because the +** The value of aiColumn is {2, 0}. aiColumn[0]==2 because the ** first column to be indexed (c3) has an index of 2 in Ex1.aCol[]. ** The second column to be indexed (c1) has an index of 0 in ** Ex1.aCol[], hence Ex2.aiColumn[1]==0. @@ -9025,7 +9027,7 @@ struct UnpackedRecord { ** The Index.onError field determines whether or not the indexed columns ** must be unique and what to do if they are not. When Index.onError=OE_None, ** it means this is not a unique index. Otherwise it is a unique index -** and the value of Index.onError indicate the which conflict resolution +** and the value of Index.onError indicate the which conflict resolution ** algorithm to employ whenever an attempt is made to insert a non-unique ** element. */ @@ -9047,7 +9049,7 @@ struct Index { }; /* -** Each sample stored in the sqlite_stat2 table is represented in memory +** Each sample stored in the sqlite_stat2 table is represented in memory ** using a structure of this type. */ struct IndexSample { @@ -9127,7 +9129,7 @@ struct AggInfo { ** the option is available (at compile-time). */ #if SQLITE_MAX_VARIABLE_NUMBER<=32767 -typedef i64 ynVar; +typedef i16 ynVar; #else typedef int ynVar; #endif @@ -9142,9 +9144,9 @@ typedef int ynVar; ** to represent the greater-than-or-equal-to operator in the expression ** tree. ** -** If the expression is an SQL literal (TK_INTEGER, TK_FLOAT, TK_BLOB, +** If the expression is an SQL literal (TK_INTEGER, TK_FLOAT, TK_BLOB, ** or TK_STRING), then Expr.token contains the text of the SQL literal. If -** the expression is a variable (TK_VARIABLE), then Expr.token contains the +** the expression is a variable (TK_VARIABLE), then Expr.token contains the ** variable name. Finally, if the expression is an SQL function (TK_FUNCTION), ** then Expr.token contains the name of the function. ** @@ -9155,7 +9157,7 @@ typedef int ynVar; ** a CASE expression or an IN expression of the form " IN (, ...)". ** Expr.x.pSelect is used if the expression is a sub-select or an expression of ** the form " IN (SELECT ...)". If the EP_xIsSelect bit is set in the -** Expr.flags mask, then Expr.x.pSelect is valid. Otherwise, Expr.x.pList is +** Expr.flags mask, then Expr.x.pSelect is valid. Otherwise, Expr.x.pList is ** valid. ** ** An expression of the form ID or ID.ID refers to a column in a table. @@ -9166,8 +9168,8 @@ typedef int ynVar; ** value is also stored in the Expr.iAgg column in the aggregate so that ** it can be accessed after all aggregates are computed. ** -** If the expression is an unbound variable marker (a question mark -** character '?' in the original SQL) then the Expr.iTable holds the index +** If the expression is an unbound variable marker (a question mark +** character '?' in the original SQL) then the Expr.iTable holds the index ** number for that variable. ** ** If the expression is a subquery then Expr.iColumn holds an integer @@ -9206,7 +9208,7 @@ struct Expr { /* If the EP_TokenOnly flag is set in the Expr.flags mask, then no ** space is allocated for the fields below this point. An attempt to - ** access them will result in a segfault or malfunction. + ** access them will result in a segfault or malfunction. *********************************************************************/ Expr *pLeft; /* Left subnode */ @@ -9278,7 +9280,7 @@ struct Expr { #endif /* -** These macros can be used to test, set, or clear bits in the +** These macros can be used to test, set, or clear bits in the ** Expr.flags field. */ #define ExprHasProperty(E,P) (((E)->flags&(P))==(P)) @@ -9287,8 +9289,8 @@ struct Expr { #define ExprClearProperty(E,P) (E)->flags&=~(P) /* -** Macros to determine the number of bytes required by a normal Expr -** struct, an Expr struct with the EP_Reduced flag set in Expr.flags +** Macros to determine the number of bytes required by a normal Expr +** struct, an Expr struct with the EP_Reduced flag set in Expr.flags ** and an Expr struct with the EP_TokenOnly flag set. */ #define EXPR_FULLSIZE sizeof(Expr) /* Full size */ @@ -9296,7 +9298,7 @@ struct Expr { #define EXPR_TOKENONLYSIZE offsetof(Expr,pLeft) /* Fewer features */ /* -** Flags passed to the sqlite3ExprDup() function. See the header comment +** Flags passed to the sqlite3ExprDup() function. See the header comment ** above sqlite3ExprDup() for details. */ #define EXPRDUP_REDUCE 0x0001 /* Used reduced-size Expr nodes */ @@ -9532,12 +9534,12 @@ struct WhereInfo { ** pEList corresponds to the result set of a SELECT and is NULL for ** other statements. ** -** NameContexts can be nested. When resolving names, the inner-most +** NameContexts can be nested. When resolving names, the inner-most ** context is searched first. If no match is found, the next outer ** context is checked. If there is still no match, the next context ** is checked. This process continues until either a match is found ** or all contexts are check. When a match is found, the nRef member of -** the context containing the match is incremented. +** the context containing the match is incremented. ** ** Each subquery gets a new NameContext. The pNext field points to the ** NameContext in the parent query. Thus the process of scanning the @@ -9642,7 +9644,7 @@ struct SelectDest { }; /* -** During code generation of statements that do inserts into AUTOINCREMENT +** During code generation of statements that do inserts into AUTOINCREMENT ** tables, the following information is attached to the Table.u.autoInc.p ** pointer of each autoincrement table to record some side information that ** the code generator needs. We have to keep per-table autoincrement @@ -9665,7 +9667,7 @@ struct AutoincInfo { #endif /* -** At least one instance of the following structure is created for each +** At least one instance of the following structure is created for each ** trigger that may be fired while parsing an INSERT, UPDATE or DELETE ** statement. All such objects are stored in the linked list headed at ** Parse.pTriggerPrg and deleted once statement compilation has been @@ -9677,15 +9679,16 @@ struct AutoincInfo { ** The Parse.pTriggerPrg list never contains two entries with the same ** values for both pTrigger and orconf. ** -** The TriggerPrg.oldmask variable is set to a mask of old.* columns -** accessed (or set to 0 for triggers fired as a result of INSERT -** statements). +** The TriggerPrg.aColmask[0] variable is set to a mask of old.* columns +** accessed (or set to 0 for triggers fired as a result of INSERT +** statements). Similarly, the TriggerPrg.aColmask[1] variable is set to +** a mask of new.* columns used by the program. */ struct TriggerPrg { Trigger *pTrigger; /* Trigger this program was coded from */ int orconf; /* Default ON CONFLICT policy */ SubProgram *pProgram; /* Program implementing pTrigger/orconf */ - u32 oldmask; /* Mask of old.* columns accessed */ + u32 aColmask[2]; /* Masks of old.*, new.* columns accessed */ TriggerPrg *pNext; /* Next entry in Parse.pTriggerPrg list */ }; @@ -9699,7 +9702,7 @@ struct TriggerPrg { ** is constant but the second part is reset at the beginning and end of ** each recursion. ** -** The nTableLock and aTableLock variables are only used if the shared-cache +** The nTableLock and aTableLock variables are only used if the shared-cache ** feature is enabled (if sqlite3Tsd()->useSharedData is true). They are ** used to store the set of table-locks required by the statement being ** compiled. Function sqlite3TableLock() is used to add entries to the @@ -9757,6 +9760,7 @@ struct Parse { Parse *pToplevel; /* Parse structure for main program (or NULL) */ Table *pTriggerTab; /* Table triggers are being coded for */ u32 oldmask; /* Mask of old.* columns referenced */ + u32 newmask; /* Mask of new.* columns referenced */ u8 eTriggerOp; /* TK_UPDATE, TK_INSERT or TK_DELETE */ u8 eOrconf; /* Default ON CONFLICT policy for trigger steps */ u8 disableTriggers; /* True to disable triggers */ @@ -9817,10 +9821,10 @@ struct AuthContext { /* * Each trigger present in the database schema is stored as an instance of - * struct Trigger. + * struct Trigger. * * Pointers to instances of struct Trigger are stored in two ways. - * 1. In the "trigHash" hash table (part of the sqlite3* that represents the + * 1. In the "trigHash" hash table (part of the sqlite3* that represents the * database). This allows Trigger structures to be retrieved by name. * 2. All triggers associated with a single table form a linked list, using the * pNext member of struct Trigger. A pointer to the first element of the @@ -9846,7 +9850,7 @@ struct Trigger { /* ** A trigger is either a BEFORE or an AFTER trigger. The following constants -** determine which. +** determine which. ** ** If there are multiple triggers, you might of some BEFORE and some AFTER. ** In that cases, the constants below can be ORed together. @@ -9856,15 +9860,15 @@ struct Trigger { /* * An instance of struct TriggerStep is used to store a single SQL statement - * that is a part of a trigger-program. + * that is a part of a trigger-program. * * Instances of struct TriggerStep are stored in a singly linked list (linked - * using the "pNext" member) referenced by the "step_list" member of the + * using the "pNext" member) referenced by the "step_list" member of the * associated struct Trigger instance. The first element of the linked list is * the first step of the trigger-program. - * + * * The "op" member indicates whether this is a "DELETE", "INSERT", "UPDATE" or - * "SELECT" statement. The meanings of the other members is determined by the + * "SELECT" statement. The meanings of the other members is determined by the * value of "op" as follows: * * (op == TK_INSERT) @@ -9874,7 +9878,7 @@ struct Trigger { * target -> A token holding the quoted name of the table to insert into. * pExprList -> If this is an INSERT INTO ... VALUES ... statement, then * this stores values to be inserted. Otherwise NULL. - * pIdList -> If this is an INSERT INTO ... () VALUES ... + * pIdList -> If this is an INSERT INTO ... () VALUES ... * statement, then this stores the column-names to be * inserted into. * @@ -9882,7 +9886,7 @@ struct Trigger { * target -> A token holding the quoted name of the table to delete from. * pWhere -> The WHERE clause of the DELETE statement if one is specified. * Otherwise NULL. - * + * * (op == TK_UPDATE) * target -> A token holding the quoted name of the table to update rows of. * pWhere -> The WHERE clause of the UPDATE statement if one is specified. @@ -9890,7 +9894,7 @@ struct Trigger { * pExprList -> A list of the columns to update and the expressions to update * them to. See sqlite3Update() documentation of "pChanges" * argument. - * + * */ struct TriggerStep { u8 op; /* One of TK_DELETE, TK_UPDATE, TK_INSERT, TK_SELECT */ @@ -9908,7 +9912,7 @@ struct TriggerStep { /* ** The following structure contains information used by the sqliteFix... ** routines as they walk the parse tree to make database references -** explicit. +** explicit. */ typedef struct DbFixer DbFixer; struct DbFixer { @@ -10106,7 +10110,7 @@ SQLITE_PRIVATE int sqlite3MemoryAlarm(void (*)(void*, sqlite3_int64, int), void* #ifdef SQLITE_USE_ALLOCA # define sqlite3StackAllocRaw(D,N) alloca(N) # define sqlite3StackAllocZero(D,N) memset(alloca(N), 0, N) -# define sqlite3StackFree(D,P) +# define sqlite3StackFree(D,P) #else # define sqlite3StackAllocRaw(D,N) sqlite3DbMallocRaw(D,N) # define sqlite3StackAllocZero(D,N) sqlite3DbMallocZero(D,N) @@ -10135,6 +10139,9 @@ SQLITE_PRIVATE void sqlite3StatusSet(int, int); SQLITE_PRIVATE int sqlite3IsNaN(double); SQLITE_PRIVATE void sqlite3VXPrintf(StrAccum*, int, const char*, va_list); +#ifndef SQLITE_OMIT_TRACE +SQLITE_PRIVATE void sqlite3XPrintf(StrAccum*, const char*, ...); +#endif SQLITE_PRIVATE char *sqlite3MPrintf(sqlite3*,const char*, ...); SQLITE_PRIVATE char *sqlite3VMPrintf(sqlite3*,const char*, va_list); SQLITE_PRIVATE char *sqlite3MAppendf(sqlite3*,char*,const char*,...); @@ -10162,7 +10169,6 @@ SQLITE_PRIVATE Expr *sqlite3PExpr(Parse*, int, Expr*, Expr*, const Token*); SQLITE_PRIVATE Expr *sqlite3ExprAnd(sqlite3*,Expr*, Expr*); SQLITE_PRIVATE Expr *sqlite3ExprFunction(Parse*,ExprList*, Token*); SQLITE_PRIVATE void sqlite3ExprAssignVarNumber(Parse*, Expr*); -SQLITE_PRIVATE void sqlite3ExprClear(sqlite3*, Expr*); SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3*, Expr*); SQLITE_PRIVATE ExprList *sqlite3ExprListAppend(Parse*,ExprList*,Expr*); SQLITE_PRIVATE void sqlite3ExprListSetName(Parse*,ExprList*,Token*,int); @@ -10292,6 +10298,9 @@ SQLITE_PRIVATE int sqlite3ExprIsConstant(Expr*); SQLITE_PRIVATE int sqlite3ExprIsConstantNotJoin(Expr*); SQLITE_PRIVATE int sqlite3ExprIsConstantOrFunction(Expr*); SQLITE_PRIVATE int sqlite3ExprIsInteger(Expr*, int*); +SQLITE_PRIVATE int sqlite3ExprCanBeNull(const Expr*); +SQLITE_PRIVATE void sqlite3ExprCodeIsNullJump(Vdbe*, const Expr*, int, int); +SQLITE_PRIVATE int sqlite3ExprNeedsNoAffinityChange(const Expr*, char); SQLITE_PRIVATE int sqlite3IsRowid(const char*); SQLITE_PRIVATE void sqlite3GenerateRowDelete(Parse*, Table*, int, int, int, Trigger *, int); SQLITE_PRIVATE void sqlite3GenerateRowIndexDelete(Parse*, Table*, int, int*); @@ -10349,7 +10358,7 @@ SQLITE_PRIVATE TriggerStep *sqlite3TriggerUpdateStep(sqlite3*,Token*,ExprList* SQLITE_PRIVATE TriggerStep *sqlite3TriggerDeleteStep(sqlite3*,Token*, Expr*); SQLITE_PRIVATE void sqlite3DeleteTrigger(sqlite3*, Trigger*); SQLITE_PRIVATE void sqlite3UnlinkAndDeleteTrigger(sqlite3*,int,const char*); -SQLITE_PRIVATE u32 sqlite3TriggerOldmask(Parse*,Trigger*,ExprList*,Table*,int); +SQLITE_PRIVATE u32 sqlite3TriggerColmask(Parse*,Trigger*,ExprList*,int,int,Table*,int); # define sqlite3ParseToplevel(p) ((p)->pToplevel ? (p)->pToplevel : (p)) #else # define sqlite3TriggersExist(B,C,D,E,F) 0 @@ -10360,7 +10369,7 @@ SQLITE_PRIVATE u32 sqlite3TriggerOldmask(Parse*,Trigger*,ExprList*,Table*,int) # define sqlite3CodeRowTriggerDirect(A,B,C,D,E,F) # define sqlite3TriggerList(X, Y) 0 # define sqlite3ParseToplevel(p) p -# define sqlite3TriggerOldmask(A,B,C,D,E) 0 +# define sqlite3TriggerColmask(A,B,C,D,E,F,G) 0 #endif SQLITE_PRIVATE int sqlite3JoinType(Parse*, Token*, Token*, Token*); @@ -10399,7 +10408,7 @@ SQLITE_PRIVATE int sqlite3Utf8Read(const u8*, const u8**); ** Routines to read and write variable-length integers. These used to ** be defined locally, but now we use the varint routines in the util.c ** file. Code should use the MACRO forms below, as the Varint32 versions -** are coded to assume the single byte case is already handled (which +** are coded to assume the single byte case is already handled (which ** the MACRO form does). */ SQLITE_PRIVATE int sqlite3PutVarint(unsigned char*, u64); @@ -10452,7 +10461,7 @@ SQLITE_PRIVATE void sqlite3VdbeSetChanges(sqlite3 *, int); SQLITE_PRIVATE const void *sqlite3ValueText(sqlite3_value*, u8); SQLITE_PRIVATE int sqlite3ValueBytes(sqlite3_value*, u8); -SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8, +SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8, void(*)(void*)); SQLITE_PRIVATE void sqlite3ValueFree(sqlite3_value*); SQLITE_PRIVATE sqlite3_value *sqlite3ValueNew(sqlite3 *); @@ -10463,6 +10472,7 @@ SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *, u8, char *, int, int *); SQLITE_PRIVATE int sqlite3ValueFromExpr(sqlite3 *, Expr *, u8, u8, sqlite3_value **); SQLITE_PRIVATE void sqlite3ValueApplyAffinity(sqlite3_value *, u8, u8); #ifndef SQLITE_AMALGAMATION +SQLITE_PRIVATE const unsigned char sqlite3OpcodeProperty[]; SQLITE_PRIVATE const unsigned char sqlite3UpperToLower[]; SQLITE_PRIVATE const unsigned char sqlite3CtypeMap[]; SQLITE_PRIVATE SQLITE_WSD struct Sqlite3Config sqlite3Config; @@ -10500,7 +10510,7 @@ SQLITE_PRIVATE void sqlite3SchemaFree(void *); SQLITE_PRIVATE Schema *sqlite3SchemaGet(sqlite3 *, Btree *); SQLITE_PRIVATE int sqlite3SchemaToIndex(sqlite3 *db, Schema *); SQLITE_PRIVATE KeyInfo *sqlite3IndexKeyinfo(Parse *, Index *); -SQLITE_PRIVATE int sqlite3CreateFunc(sqlite3 *, const char *, int, int, void *, +SQLITE_PRIVATE int sqlite3CreateFunc(sqlite3 *, const char *, int, int, void *, void (*)(sqlite3_context*,int,sqlite3_value **), void (*)(sqlite3_context*,int,sqlite3_value **), void (*)(sqlite3_context*)); SQLITE_PRIVATE int sqlite3ApiExit(sqlite3 *db, int); @@ -10549,7 +10559,7 @@ SQLITE_PRIVATE int sqlite3Utf8To8(unsigned char*); # define sqlite3VtabRollback(X) # define sqlite3VtabCommit(X) # define sqlite3VtabInSync(db) 0 -# define sqlite3VtabLock(X) +# define sqlite3VtabLock(X) # define sqlite3VtabUnlock(X) # define sqlite3VtabUnlockList(X) #else @@ -10573,6 +10583,7 @@ SQLITE_PRIVATE int sqlite3VtabCallDestroy(sqlite3*, int, const char *); SQLITE_PRIVATE int sqlite3VtabBegin(sqlite3 *, VTable *); SQLITE_PRIVATE FuncDef *sqlite3VtabOverloadFunction(sqlite3 *,FuncDef*, int nArg, Expr*); SQLITE_PRIVATE void sqlite3InvalidFunction(sqlite3_context*,int,sqlite3_value**); +SQLITE_PRIVATE int sqlite3VdbeParameterIndex(Vdbe*, const char*, int); SQLITE_PRIVATE int sqlite3TransferBindings(sqlite3_stmt *, sqlite3_stmt *); SQLITE_PRIVATE int sqlite3Reprepare(Vdbe*); SQLITE_PRIVATE void sqlite3ExprListCheckLength(Parse*, ExprList*, const char*); @@ -10584,7 +10595,7 @@ SQLITE_PRIVATE VTable *sqlite3GetVTable(sqlite3*, Table*); ** no-op macros if OMIT_FOREIGN_KEY is defined. In this case no foreign ** key functionality is available. If OMIT_TRIGGER is defined but ** OMIT_FOREIGN_KEY is not, only some of the functions are no-oped. In -** this case foreign keys are parsed, but no other functionality is +** this case foreign keys are parsed, but no other functionality is ** provided (enforcement of FK constraints requires the triggers sub-system). */ #if !defined(SQLITE_OMIT_FOREIGN_KEY) && !defined(SQLITE_OMIT_TRIGGER) @@ -10674,7 +10685,7 @@ SQLITE_PRIVATE void sqlite3ParserTrace(FILE*, char *); /* ** If the SQLITE_ENABLE IOTRACE exists then the global variable ** sqlite3IoTrace is a pointer to a printf-like routine used to -** print I/O tracing messages. +** print I/O tracing messages. */ #ifdef SQLITE_ENABLE_IOTRACE # define IOTRACE(A) if( sqlite3IoTrace ){ sqlite3IoTrace A; } @@ -10704,9 +10715,8 @@ SQLITE_PRIVATE void (*sqlite3IoTrace)(const char*,...); ** This file contains definitions of global variables and contants. */ - /* An array to map all upper-case characters into their corresponding -** lower-case character. +** lower-case character. ** ** SQLite only considers US-ASCII (or EBCDIC) characters. We do not ** handle case conversions for the UTF character set since the tables @@ -10760,6 +10770,7 @@ SQLITE_PRIVATE const unsigned char sqlite3UpperToLower[] = { ** isalnum() 0x06 ** isxdigit() 0x08 ** toupper() 0x20 +** SQLite identifier character 0x40 ** ** Bit 0x20 is set if the mapped character requires translation to upper ** case. i.e. if the character is a lower-case ASCII character. @@ -10771,6 +10782,11 @@ SQLITE_PRIVATE const unsigned char sqlite3UpperToLower[] = { ** Standard function tolower() is implemented using the sqlite3UpperToLower[] ** array. tolower() is used more often than toupper() by SQLite. ** +** Bit 0x40 is set if the character non-alphanumeric and can be used in an +** SQLite identifier. Identifiers are alphanumerics, "_", "$", and any +** non-ASCII UTF character. Hence the test for whether or not a character is +** part of an identifier is 0x46. +** ** SQLite's versions are identical to the standard versions assuming a ** locale of "C". They are implemented as macros in sqliteInt.h. */ @@ -10780,7 +10796,7 @@ SQLITE_PRIVATE const unsigned char sqlite3CtypeMap[256] = { 0x00, 0x01, 0x01, 0x01, 0x01, 0x01, 0x00, 0x00, /* 08..0f ........ */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 10..17 ........ */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 18..1f ........ */ - 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 20..27 !"#$%&' */ + 0x01, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, /* 20..27 !"#$%&' */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 28..2f ()*+,-./ */ 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, 0x0c, /* 30..37 01234567 */ 0x0c, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 38..3f 89:;<=>? */ @@ -10788,29 +10804,29 @@ SQLITE_PRIVATE const unsigned char sqlite3CtypeMap[256] = { 0x00, 0x0a, 0x0a, 0x0a, 0x0a, 0x0a, 0x0a, 0x02, /* 40..47 @ABCDEFG */ 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, /* 48..4f HIJKLMNO */ 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, /* 50..57 PQRSTUVW */ - 0x02, 0x02, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, /* 58..5f XYZ[\]^_ */ + 0x02, 0x02, 0x02, 0x00, 0x00, 0x00, 0x00, 0x40, /* 58..5f XYZ[\]^_ */ 0x00, 0x2a, 0x2a, 0x2a, 0x2a, 0x2a, 0x2a, 0x22, /* 60..67 `abcdefg */ 0x22, 0x22, 0x22, 0x22, 0x22, 0x22, 0x22, 0x22, /* 68..6f hijklmno */ 0x22, 0x22, 0x22, 0x22, 0x22, 0x22, 0x22, 0x22, /* 70..77 pqrstuvw */ 0x22, 0x22, 0x22, 0x00, 0x00, 0x00, 0x00, 0x00, /* 78..7f xyz{|}~. */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 80..87 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 88..8f ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 90..97 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 98..9f ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* a0..a7 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* a8..af ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* b0..b7 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* b8..bf ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* 80..87 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* 88..8f ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* 90..97 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* 98..9f ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* a0..a7 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* a8..af ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* b0..b7 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* b8..bf ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* c0..c7 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* c8..cf ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* d0..d7 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* d8..df ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* e0..e7 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* e8..ef ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* f0..f7 ........ */ - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 /* f8..ff ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* c0..c7 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* c8..cf ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* d0..d7 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* d8..df ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* e0..e7 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* e8..ef ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, /* f0..f7 ........ */ + 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40 /* f8..ff ........ */ }; #endif @@ -10879,6 +10895,14 @@ SQLITE_PRIVATE SQLITE_WSD FuncDefHash sqlite3GlobalFunctions; */ SQLITE_PRIVATE int sqlite3PendingByte = 0x40000000; +/* +** Properties of opcodes. The OPFLG_INITIALIZER macro is +** created by mkopcodeh.awk during compilation. Data is obtained +** from the comments following the "case OP_xxxx:" statements in +** the vdbe.c file. +*/ +SQLITE_PRIVATE const unsigned char sqlite3OpcodeProperty[] = OPFLG_INITIALIZER; + /************** End of global.c **********************************************/ /************** Begin file status.c ******************************************/ /* @@ -10895,8 +10919,6 @@ SQLITE_PRIVATE int sqlite3PendingByte = 0x40000000; ** ** This module implements the sqlite3_status() interface and related ** functionality. -** -** $Id: status.c,v 1.9 2008/09/02 00:52:52 drh Exp $ */ /* @@ -11017,18 +11039,16 @@ SQLITE_API int sqlite3_db_status( ** ************************************************************************* ** This file contains the C functions that implement date and time -** functions for SQLite. +** functions for SQLite. ** ** There is only one exported symbol in this file - the function ** sqlite3RegisterDateTimeFunctions() found at the bottom of the file. ** All other code has file scope. ** -** $Id: date.c,v 1.107 2009/05/03 20:23:53 drh Exp $ -** ** SQLite processes all times and dates as Julian Day numbers. The ** dates and times are stored as the number of days since noon ** in Greenwich on November 24, 4714 B.C. according to the Gregorian -** calendar system. +** calendar system. ** ** 1970-01-01 00:00:00 is JD 2440587.5 ** 2000-01-01 00:00:00 is JD 2451544.5 @@ -11058,14 +11078,14 @@ SQLITE_API int sqlite3_db_status( /* ** On recent Windows platforms, the localtime_s() function is available -** as part of the "Secure CRT". It is essentially equivalent to -** localtime_r() available under most POSIX platforms, except that the +** as part of the "Secure CRT". It is essentially equivalent to +** localtime_r() available under most POSIX platforms, except that the ** order of the parameters is reversed. ** ** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx. ** ** If the user has not indicated to use localtime_r() or localtime_s() -** already, check for an MSVC build environment that provides +** already, check for an MSVC build environment that provides ** localtime_s(). */ #if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \ @@ -11334,7 +11354,7 @@ static void setDateTimeToCurrent(sqlite3_context *context, DateTime *p){ ** The following are acceptable forms for the input string: ** ** YYYY-MM-DD HH:MM:SS.FFF +/-HH:MM -** DDDD.DD +** DDDD.DD ** now ** ** In the first form, the +/-HH:MM is always optional. The fractional @@ -11344,8 +11364,8 @@ static void setDateTimeToCurrent(sqlite3_context *context, DateTime *p){ ** as there is a year and date. */ static int parseDateOrTime( - sqlite3_context *context, - const char *zDate, + sqlite3_context *context, + const char *zDate, DateTime *p ){ int isRealNum; /* Return from sqlite3IsNumber(). Not used */ @@ -11721,9 +11741,9 @@ static int parseModifier(const char *zMod, DateTime *p){ ** then assume a default value of "now" for argv[0]. */ static int isDate( - sqlite3_context *context, - int argc, - sqlite3_value **argv, + sqlite3_context *context, + int argc, + sqlite3_value **argv, DateTime *p ){ int i; @@ -12127,8 +12147,6 @@ SQLITE_PRIVATE void sqlite3RegisterDateTimeFunctions(void){ ** ** This file contains OS interface code that is common to all ** architectures. -** -** $Id: os.c,v 1.127 2009/07/27 11:41:21 danielk1977 Exp $ */ #define _SQLITE_OS_C_ 1 #undef _SQLITE_OS_C_ @@ -12140,7 +12158,7 @@ SQLITE_PRIVATE void sqlite3RegisterDateTimeFunctions(void){ ** So we test the effects of a malloc() failing and the sqlite3OsXXX() ** function returning SQLITE_IOERR_NOMEM using the DO_OS_MALLOC_TEST macro. ** -** The following functions are instrumented for malloc() failure +** The following functions are instrumented for malloc() failure ** testing: ** ** sqlite3OsOpen() @@ -12220,10 +12238,10 @@ SQLITE_PRIVATE int sqlite3OsDeviceCharacteristics(sqlite3_file *id){ ** VFS methods. */ SQLITE_PRIVATE int sqlite3OsOpen( - sqlite3_vfs *pVfs, - const char *zPath, - sqlite3_file *pFile, - int flags, + sqlite3_vfs *pVfs, + const char *zPath, + sqlite3_file *pFile, + int flags, int *pFlagsOut ){ int rc; @@ -12240,20 +12258,21 @@ SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *pVfs, const char *zPath, int dir return pVfs->xDelete(pVfs, zPath, dirSync); } SQLITE_PRIVATE int sqlite3OsAccess( - sqlite3_vfs *pVfs, - const char *zPath, - int flags, + sqlite3_vfs *pVfs, + const char *zPath, + int flags, int *pResOut ){ DO_OS_MALLOC_TEST(0); return pVfs->xAccess(pVfs, zPath, flags, pResOut); } SQLITE_PRIVATE int sqlite3OsFullPathname( - sqlite3_vfs *pVfs, - const char *zPath, - int nPathOut, + sqlite3_vfs *pVfs, + const char *zPath, + int nPathOut, char *zPathOut ){ + zPathOut[0] = 0; return pVfs->xFullPathname(pVfs, zPath, nPathOut, zPathOut); } #ifndef SQLITE_OMIT_LOAD_EXTENSION @@ -12281,9 +12300,9 @@ SQLITE_PRIVATE int sqlite3OsCurrentTime(sqlite3_vfs *pVfs, double *pTimeOut){ } SQLITE_PRIVATE int sqlite3OsOpenMalloc( - sqlite3_vfs *pVfs, - const char *zFile, - sqlite3_file **ppFile, + sqlite3_vfs *pVfs, + const char *zFile, + sqlite3_file **ppFile, int flags, int *pOutFlags ){ @@ -12425,21 +12444,17 @@ SQLITE_API int sqlite3_vfs_unregister(sqlite3_vfs *pVfs){ ** ************************************************************************* ** -** $Id: fault.c,v 1.11 2008/09/02 00:52:52 drh Exp $ -*/ - -/* -** This file contains code to support the concept of "benign" +** This file contains code to support the concept of "benign" ** malloc failures (when the xMalloc() or xRealloc() method of the ** sqlite3_mem_methods structure fails to allocate a block of memory -** and returns 0). +** and returns 0). ** ** Most malloc failures are non-benign. After they occur, SQLite ** abandons the current operation and returns an error code (usually ** SQLITE_NOMEM) to the user. However, sometimes a fault is not necessarily -** fatal. For example, if a malloc fails while resizing a hash table, this -** is completely recoverable simply by not carrying out the resize. The -** hash table will continue to function normally. So a malloc failure +** fatal. For example, if a malloc fails while resizing a hash table, this +** is completely recoverable simply by not carrying out the resize. The +** hash table will continue to function normally. So a malloc failure ** during a hash table resize is a benign fault. */ @@ -12523,8 +12538,6 @@ SQLITE_PRIVATE void sqlite3EndBenignMalloc(void){ ** here always fail. SQLite will not operate with these drivers. These ** are merely placeholders. Real drivers must be substituted using ** sqlite3_config() before SQLite will operate. -** -** $Id: mem0.c,v 1.1 2008/10/28 18:58:20 drh Exp $ */ /* @@ -12587,8 +12600,6 @@ SQLITE_PRIVATE void sqlite3MemSetDefault(void){ ** ** This file contains implementations of the low-level memory allocation ** routines specified in the sqlite3_mem_methods object. -** -** $Id: mem1.c,v 1.30 2009/03/23 04:33:33 danielk1977 Exp $ */ /* @@ -12736,8 +12747,6 @@ SQLITE_PRIVATE void sqlite3MemSetDefault(void){ ** ** This file contains implementations of the low-level memory allocation ** routines specified in the sqlite3_mem_methods object. -** -** $Id: mem2.c,v 1.45 2009/03/23 04:33:33 danielk1977 Exp $ */ /* @@ -12797,7 +12806,7 @@ struct MemBlockHdr { ** when this module is combined with other in the amalgamation. */ static struct { - + /* ** Mutex to control access to the memory allocation subsystem. */ @@ -12808,7 +12817,7 @@ static struct { */ struct MemBlockHdr *pFirst; struct MemBlockHdr *pLast; - + /* ** The number of levels of backtrace to save in new allocations. */ @@ -12821,7 +12830,7 @@ static struct { int nTitle; /* Bytes of zTitle to save. Includes '\0' and padding */ char zTitle[100]; /* The title text */ - /* + /* ** sqlite3MallocDisallow() increments the following counter. ** sqlite3MallocAllow() decrements it. */ @@ -12880,7 +12889,7 @@ static struct MemBlockHdr *sqlite3MemsysGetHeader(void *pAllocation){ pU8 = (u8*)pAllocation; assert( pInt[nReserve/sizeof(int)]==(int)REARGUARD ); /* This checks any of the "extra" bytes allocated due - ** to rounding up to an 8 byte boundary to ensure + ** to rounding up to an 8 byte boundary to ensure ** they haven't been overwritten. */ while( nReserve-- > p->iSize ) assert( pU8[nReserve]==0x65 ); @@ -12982,7 +12991,7 @@ static void *sqlite3MemMalloc(int nByte){ p = (void*)pInt; } sqlite3_mutex_leave(mem.mutex); - return p; + return p; } /* @@ -13017,15 +13026,15 @@ static void sqlite3MemFree(void *pPrior){ memset(z, 0x2b, sizeof(void*)*pHdr->nBacktraceSlots + sizeof(*pHdr) + pHdr->iSize + sizeof(int) + pHdr->nTitle); free(z); - sqlite3_mutex_leave(mem.mutex); + sqlite3_mutex_leave(mem.mutex); } /* ** Change the size of an existing memory allocation. ** ** For this debugging implementation, we *always* make a copy of the -** allocation into a new place in memory. In this way, if the -** higher level code is using pointer to the old allocation, it is +** allocation into a new place in memory. In this way, if the +** higher level code is using pointer to the old allocation, it is ** much more likely to break and we are much more liking to find ** the error. */ @@ -13102,7 +13111,7 @@ SQLITE_PRIVATE void sqlite3MemdebugSync(){ } /* -** Open the file indicated and write a log of all unfreed memory +** Open the file indicated and write a log of all unfreed memory ** allocations into that log. */ SQLITE_PRIVATE void sqlite3MemdebugDump(const char *zFilename){ @@ -13119,7 +13128,7 @@ SQLITE_PRIVATE void sqlite3MemdebugDump(const char *zFilename){ for(pHdr=mem.pFirst; pHdr; pHdr=pHdr->pNext){ char *z = (char*)pHdr; z -= pHdr->nBacktraceSlots*sizeof(void*) + pHdr->nTitle; - fprintf(out, "**** %lld bytes at %p from %s ****\n", + fprintf(out, "**** %lld bytes at %p from %s ****\n", pHdr->iSize, &pHdr[1], pHdr->nTitle ? z : "???"); if( pHdr->nBacktrace ){ fflush(out); @@ -13132,7 +13141,7 @@ SQLITE_PRIVATE void sqlite3MemdebugDump(const char *zFilename){ fprintf(out, "COUNTS:\n"); for(i=0; i= M*(1 + log2(n)/2) - n + 1 @@ -13904,7 +13911,7 @@ SQLITE_PRIVATE const sqlite3_mem_methods *sqlite3MemGetMemsys3(void){ */ /* -** This version of the memory allocator is used only when +** This version of the memory allocator is used only when ** SQLITE_ENABLE_MEMSYS5 is defined. */ #ifdef SQLITE_ENABLE_MEMSYS5 @@ -13949,7 +13956,7 @@ static SQLITE_WSD struct Mem5Global { int szAtom; /* Smallest possible allocation in bytes */ int nBlock; /* Number of szAtom sized blocks in zPool */ u8 *zPool; /* Memory available to be allocated */ - + /* ** Mutex to control access to the memory allocation subsystem. */ @@ -13966,7 +13973,7 @@ static SQLITE_WSD struct Mem5Global { u32 maxOut; /* Maximum instantaneous currentOut */ u32 maxCount; /* Maximum instantaneous currentCount */ u32 maxRequest; /* Largest allocation (exclusive of internal frag) */ - + /* ** Lists of free blocks. aiFreelist[0] is a list of free blocks of ** size mem5.szAtom. aiFreelist[1] holds blocks of size szAtom*2. @@ -14064,7 +14071,7 @@ static int memsys5Size(void *p){ /* ** Find the first entry on the freelist iLogsize. Unlink that -** entry and return its index. +** entry and return its index. */ static int memsys5UnlinkFirst(int iLogsize){ int i; @@ -14153,7 +14160,7 @@ static void memsys5FreeUnsafe(void *pOld){ u32 size, iLogsize; int iBlock; - /* Set iBlock to the index of the block pointed to by pOld in + /* Set iBlock to the index of the block pointed to by pOld in ** the array of mem5.szAtom byte blocks pointed to by mem5.zPool. */ iBlock = ((u8 *)pOld-mem5.zPool)/mem5.szAtom; @@ -14212,7 +14219,7 @@ static void *memsys5Malloc(int nBytes){ p = memsys5MallocUnsafe(nBytes); memsys5Leave(); } - return (void*)p; + return (void*)p; } /* @@ -14225,14 +14232,14 @@ static void memsys5Free(void *pPrior){ assert( pPrior!=0 ); memsys5Enter(); memsys5FreeUnsafe(pPrior); - memsys5Leave(); + memsys5Leave(); } /* ** Change the size of an existing memory allocation. ** ** The outer layer memory allocator prevents this routine from -** being called with pPrior==0. +** being called with pPrior==0. ** ** nBytes is always a value obtained from a prior call to ** memsys5Round(). Hence nBytes is always a non-negative power @@ -14366,7 +14373,7 @@ static void memsys5Shutdown(void *NotUsed){ #ifdef SQLITE_TEST /* -** Open the file indicated and write a log of all unfreed memory +** Open the file indicated and write a log of all unfreed memory ** allocations into that log. */ SQLITE_PRIVATE void sqlite3Memsys5Dump(const char *zFilename){ @@ -14408,7 +14415,7 @@ SQLITE_PRIVATE void sqlite3Memsys5Dump(const char *zFilename){ #endif /* -** This routine is the only routine in this file with external +** This routine is the only routine in this file with external ** linkage. It returns a pointer to a static sqlite3_mem_methods ** struct populated with the memsys5 methods. */ @@ -14444,9 +14451,6 @@ SQLITE_PRIVATE const sqlite3_mem_methods *sqlite3MemGetMemsys5(void){ ** This file contains the C functions that implement mutexes. ** ** This file contains code that is common across all mutex implementations. - -** -** $Id: mutex.c,v 1.31 2009/07/16 18:21:18 drh Exp $ */ #if defined(SQLITE_DEBUG) && !defined(SQLITE_MUTEX_OMIT) @@ -14463,12 +14467,12 @@ static SQLITE_WSD int mutexIsInit = 0; /* ** Initialize the mutex system. */ -SQLITE_PRIVATE int sqlite3MutexInit(void){ +SQLITE_PRIVATE int sqlite3MutexInit(void){ int rc = SQLITE_OK; if( sqlite3GlobalConfig.bCoreMutex ){ if( !sqlite3GlobalConfig.mutex.xMutexAlloc ){ /* If the xMutexAlloc method has not been set, then the user did not - ** install a mutex implementation via sqlite3_config() prior to + ** install a mutex implementation via sqlite3_config() prior to ** sqlite3_initialize() being called. This block copies pointers to ** the default implementation into the sqlite3GlobalConfig structure. */ @@ -14558,7 +14562,7 @@ SQLITE_API int sqlite3_mutex_try(sqlite3_mutex *p){ /* ** The sqlite3_mutex_leave() routine exits a mutex that was previously -** entered by the same thread. The behavior is undefined if the mutex +** entered by the same thread. The behavior is undefined if the mutex ** is not currently entered. If a NULL pointer is passed as an argument ** this function is a no-op. */ @@ -14611,8 +14615,6 @@ SQLITE_API int sqlite3_mutex_notheld(sqlite3_mutex *p){ ** If compiled with SQLITE_DEBUG, then additional logic is inserted ** that does error checking on mutexes to make sure they are being ** called correctly. -** -** $Id: mutex_noop.c,v 1.3 2008/12/05 17:17:08 drh Exp $ */ @@ -14685,7 +14687,7 @@ static int debugMutexEnd(void){ return SQLITE_OK; } /* ** The sqlite3_mutex_alloc() routine allocates a new ** mutex and returns a pointer to it. If it returns NULL -** that means that a mutex could not be allocated. +** that means that a mutex could not be allocated. */ static sqlite3_mutex *debugMutexAlloc(int id){ static sqlite3_mutex aStatic[6]; @@ -14785,8 +14787,6 @@ SQLITE_PRIVATE sqlite3_mutex_methods *sqlite3DefaultMutex(void){ ** ************************************************************************* ** This file contains the C functions that implement mutexes for OS/2 -** -** $Id: mutex_os2.c,v 1.11 2008/11/22 19:50:54 pweilbacher Exp $ */ /* @@ -14822,7 +14822,7 @@ static int os2MutexEnd(void){ return SQLITE_OK; } /* ** The sqlite3_mutex_alloc() routine allocates a new ** mutex and returns a pointer to it. If it returns NULL -** that means that a mutex could not be allocated. +** that means that a mutex could not be allocated. ** SQLite will unwind its stack and return an error. The argument ** to sqlite3_mutex_alloc() is one of these integer constants: ** @@ -14934,6 +14934,39 @@ static void os2MutexFree(sqlite3_mutex *p){ sqlite3_free( p ); } +#ifdef SQLITE_DEBUG +/* +** The sqlite3_mutex_held() and sqlite3_mutex_notheld() routine are +** intended for use inside assert() statements. +*/ +static int os2MutexHeld(sqlite3_mutex *p){ + TID tid; + PID pid; + ULONG ulCount; + PTIB ptib; + if( p!=0 ) { + DosQueryMutexSem(p->mutex, &pid, &tid, &ulCount); + } else { + DosGetInfoBlocks(&ptib, NULL); + tid = ptib->tib_ptib2->tib2_ultid; + } + return p==0 || (p->nRef!=0 && p->owner==tid); +} +static int os2MutexNotheld(sqlite3_mutex *p){ + TID tid; + PID pid; + ULONG ulCount; + PTIB ptib; + if( p!= 0 ) { + DosQueryMutexSem(p->mutex, &pid, &tid, &ulCount); + } else { + DosGetInfoBlocks(&ptib, NULL); + tid = ptib->tib_ptib2->tib2_ultid; + } + return p==0 || p->nRef==0 || p->owner!=tid; +} +#endif + /* ** The sqlite3_mutex_enter() and sqlite3_mutex_try() routines attempt ** to enter a mutex. If another thread is already within the mutex, @@ -14994,39 +15027,6 @@ static void os2MutexLeave(sqlite3_mutex *p){ DosReleaseMutexSem(p->mutex); } -#ifdef SQLITE_DEBUG -/* -** The sqlite3_mutex_held() and sqlite3_mutex_notheld() routine are -** intended for use inside assert() statements. -*/ -static int os2MutexHeld(sqlite3_mutex *p){ - TID tid; - PID pid; - ULONG ulCount; - PTIB ptib; - if( p!=0 ) { - DosQueryMutexSem(p->mutex, &pid, &tid, &ulCount); - } else { - DosGetInfoBlocks(&ptib, NULL); - tid = ptib->tib_ptib2->tib2_ultid; - } - return p==0 || (p->nRef!=0 && p->owner==tid); -} -static int os2MutexNotheld(sqlite3_mutex *p){ - TID tid; - PID pid; - ULONG ulCount; - PTIB ptib; - if( p!= 0 ) { - DosQueryMutexSem(p->mutex, &pid, &tid, &ulCount); - } else { - DosGetInfoBlocks(&ptib, NULL); - tid = ptib->tib_ptib2->tib2_ultid; - } - return p==0 || p->nRef==0 || p->owner!=tid; -} -#endif - SQLITE_PRIVATE sqlite3_mutex_methods *sqlite3DefaultMutex(void){ static sqlite3_mutex_methods sMutex = { os2MutexInit, @@ -15060,8 +15060,6 @@ SQLITE_PRIVATE sqlite3_mutex_methods *sqlite3DefaultMutex(void){ ** ************************************************************************* ** This file contains the C functions that implement mutexes for pthreads -** -** $Id: mutex_unix.c,v 1.16 2008/12/08 18:19:18 drh Exp $ */ /* @@ -15100,7 +15098,7 @@ struct sqlite3_mutex { ** there might be race conditions that can cause these routines to ** deliver incorrect results. In particular, if pthread_equal() is ** not an atomic operation, then these routines might delivery -** incorrect results. On most platforms, pthread_equal() is a +** incorrect results. On most platforms, pthread_equal() is a ** comparison of two integers and is therefore atomic. But we are ** told that HPUX is not such a platform. If so, then these routines ** will not always work correctly on HPUX. @@ -15163,7 +15161,7 @@ static int pthreadMutexEnd(void){ return SQLITE_OK; } ** ** Note that if one of the dynamic mutex parameters (SQLITE_MUTEX_FAST ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc() -** returns a different mutex on every call. But for the static +** returns a different mutex on every call. But for the static ** mutex types, the same mutex is returned on every call that has ** the same type number. */ @@ -15249,7 +15247,7 @@ static void pthreadMutexEnter(sqlite3_mutex *p){ ** is atomic - that it cannot be deceived into thinking self ** and p->owner are equal if p->owner changes between two values ** that are not equal to self while the comparison is taking place. - ** This implementation also assumes a coherent cache - that + ** This implementation also assumes a coherent cache - that ** separate processes cannot read different values from the same ** address at the same time. If either of these two conditions ** are not met, then the mutexes will fail and problems will result. @@ -15289,7 +15287,7 @@ static int pthreadMutexTry(sqlite3_mutex *p){ ** is atomic - that it cannot be deceived into thinking self ** and p->owner are equal if p->owner changes between two values ** that are not equal to self while the comparison is taking place. - ** This implementation also assumes a coherent cache - that + ** This implementation also assumes a coherent cache - that ** separate processes cannot read different values from the same ** address at the same time. If either of these two conditions ** are not met, then the mutexes will fail and problems will result. @@ -15391,8 +15389,6 @@ SQLITE_PRIVATE sqlite3_mutex_methods *sqlite3DefaultMutex(void){ ** ************************************************************************* ** This file contains the C functions that implement mutexes for win32 -** -** $Id: mutex_w32.c,v 1.18 2009/08/10 03:23:21 shane Exp $ */ /* @@ -15423,9 +15419,9 @@ struct sqlite3_mutex { ** the LockFileEx() API. ** ** mutexIsNT() is only used for the TryEnterCriticalSection() API call, -** which is only available if your application was compiled with +** which is only available if your application was compiled with ** _WIN32_WINNT defined to a value >= 0x0400. Currently, the only -** call to TryEnterCriticalSection() is #ifdef'ed out, so #ifdef +** call to TryEnterCriticalSection() is #ifdef'ed out, so #ifdef ** this out as well. */ #if 0 @@ -15471,7 +15467,7 @@ static int winMutex_isInit = 0; */ static long winMutex_lock = 0; -static int winMutexInit(void){ +static int winMutexInit(void){ /* The first to increment to 1 does actual initialization */ if( InterlockedCompareExchange(&winMutex_lock, 1, 0)==0 ){ int i; @@ -15485,11 +15481,11 @@ static int winMutexInit(void){ Sleep(1); } } - return SQLITE_OK; + return SQLITE_OK; } -static int winMutexEnd(void){ - /* The first to decrement to 0 does actual shutdown +static int winMutexEnd(void){ + /* The first to decrement to 0 does actual shutdown ** (which should be the last to shutdown.) */ if( InterlockedCompareExchange(&winMutex_lock, 0, 1)==1 ){ if( winMutex_isInit==1 ){ @@ -15500,7 +15496,7 @@ static int winMutexEnd(void){ winMutex_isInit = 0; } } - return SQLITE_OK; + return SQLITE_OK; } /* @@ -15541,7 +15537,7 @@ static int winMutexEnd(void){ ** ** Note that if one of the dynamic mutex parameters (SQLITE_MUTEX_FAST ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc() -** returns a different mutex on every call. But for the static +** returns a different mutex on every call. But for the static ** mutex types, the same mutex is returned on every call that has ** the same type number. */ @@ -15552,7 +15548,7 @@ static sqlite3_mutex *winMutexAlloc(int iType){ case SQLITE_MUTEX_FAST: case SQLITE_MUTEX_RECURSIVE: { p = sqlite3MallocZero( sizeof(*p) ); - if( p ){ + if( p ){ p->id = iType; InitializeCriticalSection(&p->mutex); } @@ -15598,7 +15594,7 @@ static void winMutexFree(sqlite3_mutex *p){ static void winMutexEnter(sqlite3_mutex *p){ assert( p->id==SQLITE_MUTEX_RECURSIVE || winMutexNotheld(p) ); EnterCriticalSection(&p->mutex); - p->owner = GetCurrentThreadId(); + p->owner = GetCurrentThreadId(); p->nRef++; } static int winMutexTry(sqlite3_mutex *p){ @@ -15607,7 +15603,7 @@ static int winMutexTry(sqlite3_mutex *p){ /* ** The sqlite3_mutex_try() routine is very rarely used, and when it ** is used it is merely an optimization. So it is OK for it to always - ** fail. + ** fail. ** ** The TryEnterCriticalSection() interface is only available on WinNT. ** And some windows compilers complain if you try to use it without @@ -15678,8 +15674,6 @@ SQLITE_PRIVATE sqlite3_mutex_methods *sqlite3DefaultMutex(void){ ************************************************************************* ** ** Memory allocation functions used throughout sqlite. -** -** $Id: malloc.c,v 1.66 2009/07/17 11:44:07 drh Exp $ */ /* @@ -15688,7 +15682,7 @@ SQLITE_PRIVATE sqlite3_mutex_methods *sqlite3DefaultMutex(void){ ** limit. */ static void softHeapLimitEnforcer( - void *NotUsed, + void *NotUsed, sqlite3_int64 NotUsed2, int allocSize ){ @@ -15697,7 +15691,7 @@ static void softHeapLimitEnforcer( } /* -** Set the soft heap-size limit for the library. Passing a zero or +** Set the soft heap-size limit for the library. Passing a zero or ** negative value indicates no limit. */ SQLITE_API void sqlite3_soft_heap_limit(int n){ @@ -15876,7 +15870,7 @@ SQLITE_API int sqlite3_memory_alarm( #endif /* -** Trigger the alarm +** Trigger the alarm */ static void sqlite3MallocAlarm(int nByte){ void (*xCallback)(void*,sqlite3_int64,int); @@ -15991,7 +15985,7 @@ SQLITE_PRIVATE void *sqlite3ScratchMalloc(int n){ if( sqlite3GlobalConfig.szScratch= + if( sqlite3StatusValue(SQLITE_STATUS_MEMORY_USED)+nNew-nOld >= mem0.alarmThreshold ){ sqlite3MallocAlarm(nNew-nOld); } @@ -16183,7 +16177,7 @@ SQLITE_API void *sqlite3_realloc(void *pOld, int n){ /* ** Allocate and zero memory. -*/ +*/ SQLITE_PRIVATE void *sqlite3MallocZero(int n){ void *p = sqlite3Malloc(n); if( p ){ @@ -16298,9 +16292,9 @@ SQLITE_PRIVATE void *sqlite3DbReallocOrFree(sqlite3 *db, void *p, int n){ } /* -** Make a copy of a string in memory obtained from sqliteMalloc(). These +** Make a copy of a string in memory obtained from sqliteMalloc(). These ** functions call sqlite3MallocRaw() directly instead of sqliteMalloc(). This -** is because when memory debugging is turned on, these two functions are +** is because when memory debugging is turned on, these two functions are ** called via macros that record the current file and line number in the ** ThreadData structure. */ @@ -16350,13 +16344,13 @@ SQLITE_PRIVATE void sqlite3SetString(char **pz, sqlite3 *db, const char *zFormat /* -** This function must be called before exiting any API function (i.e. +** This function must be called before exiting any API function (i.e. ** returning control to the user) that has called sqlite3_malloc or ** sqlite3_realloc. ** ** The returned value is normally a copy of the second argument to this ** function. However, if a malloc() failure has occurred since the previous -** invocation SQLITE_NOMEM is returned instead. +** invocation SQLITE_NOMEM is returned instead. ** ** If the first argument, db, is not NULL and a malloc() error has occurred, ** then the connection error-code (the value returned by sqlite3_errcode()) @@ -16364,7 +16358,7 @@ SQLITE_PRIVATE void sqlite3SetString(char **pz, sqlite3 *db, const char *zFormat */ SQLITE_PRIVATE int sqlite3ApiExit(sqlite3* db, int rc){ /* If the db handle is not NULL, then we must hold the connection handle - ** mutex here. Otherwise the read (and possible write) of db->mallocFailed + ** mutex here. Otherwise the read (and possible write) of db->mallocFailed ** is unsafe, as is the call to sqlite3Error(). */ assert( !db || sqlite3_mutex_held(db->mutex) ); @@ -16385,8 +16379,6 @@ SQLITE_PRIVATE int sqlite3ApiExit(sqlite3* db, int rc){ ** an historical reference. Most of the "enhancements" have been backed ** out so that the functionality is now the same as standard printf(). ** -** $Id: printf.c,v 1.104 2009/06/03 01:24:54 drh Exp $ -** ************************************************************************** ** ** The following modules is an enhanced replacement for the "printf" subroutines @@ -16659,7 +16651,7 @@ SQLITE_PRIVATE void sqlite3VXPrintf( break; } /* Find out what flags are present */ - flag_leftjustify = flag_plussign = flag_blanksign = + flag_leftjustify = flag_plussign = flag_blanksign = flag_alternateform = flag_altform2 = flag_zeropad = 0; done = 0; do{ @@ -17026,14 +17018,15 @@ SQLITE_PRIVATE void sqlite3VXPrintf( case etSQLESCAPE: case etSQLESCAPE2: case etSQLESCAPE3: { - int i, j, n, isnull; + int i, j, k, n, isnull; int needQuote; char ch; char q = ((xtype==etSQLESCAPE3)?'"':'\''); /* Quote character */ char *escarg = va_arg(ap,char*); isnull = escarg==0; if( isnull ) escarg = (xtype==etSQLESCAPE2 ? "NULL" : "(NULL)"); - for(i=n=0; (ch=escarg[i])!=0; i++){ + k = precision; + for(i=n=0; (ch=escarg[i])!=0 && k!=0; i++, k--){ if( ch==q ) n++; } needQuote = !isnull && xtype==etSQLESCAPE2; @@ -17049,15 +17042,17 @@ SQLITE_PRIVATE void sqlite3VXPrintf( } j = 0; if( needQuote ) bufpt[j++] = q; - for(i=0; (ch=escarg[i])!=0; i++){ - bufpt[j++] = ch; + k = i; + for(i=0; i=0 && precision=0 && precisionenc==SQLITE_UTF16LE ){ /* UTF-16 Little-endian -> UTF-8 */ while( zIn UTF-8 */ while( zIn0 ) { scale *= 1.0e+22; e -= 22; } @@ -19069,7 +19069,7 @@ SQLITE_PRIVATE int sqlite3PutVarint(unsigned char *p, u64 v){ v >>= 7; } return 9; - } + } n = 0; do{ buf[n++] = (u8)((v & 0x7f) | 0x80); @@ -19272,8 +19272,8 @@ SQLITE_PRIVATE u8 sqlite3GetVarint(const unsigned char *p, u64 *v){ ** If the varint stored in p[0] is larger than can fit in a 32-bit unsigned ** integer, then set *v to 0xffffffff. ** -** A MACRO version, getVarint32, is provided which inlines the -** single-byte case. All code should use the MACRO version as +** A MACRO version, getVarint32, is provided which inlines the +** single-byte case. All code should use the MACRO version as ** this function assumes the single-byte case has already been handled. */ SQLITE_PRIVATE u8 sqlite3GetVarint32(const unsigned char *p, u32 *v){ @@ -19476,10 +19476,10 @@ SQLITE_PRIVATE void *sqlite3HexToBlob(sqlite3 *db, const char *z, int n){ ** open and is not being used by another thread. By changing the value ** to SQLITE_MAGIC_BUSY we indicate that the connection is in use. ** sqlite3SafetyOff() below will change the value back to SQLITE_MAGIC_OPEN -** when the API exits. +** when the API exits. ** ** This routine is a attempt to detect if two threads use the -** same sqlite* pointer at the same time. There is a race +** same sqlite* pointer at the same time. There is a race ** condition so it is possible that the error is not detected. ** But usually the problem will be seen. The result will be an ** error which can be used to debug the application that is @@ -19542,7 +19542,7 @@ SQLITE_PRIVATE int sqlite3SafetyCheckOk(sqlite3 *db){ u32 magic; if( db==0 ) return 0; magic = db->magic; - if( magic!=SQLITE_MAGIC_OPEN + if( magic!=SQLITE_MAGIC_OPEN #ifdef SQLITE_DEBUG && magic!=SQLITE_MAGIC_BUSY #endif @@ -19576,8 +19576,6 @@ SQLITE_PRIVATE int sqlite3SafetyCheckSickOrOk(sqlite3 *db){ ************************************************************************* ** This is the implementation of generic hash-tables ** used in SQLite. -** -** $Id: hash.c,v 1.38 2009/05/09 23:29:12 drh Exp $ */ /* Turn bulk memory into a hash table object by initializing the @@ -19719,7 +19717,7 @@ static HashElem *findElementGivenHash( count = pH->count; } while( count-- && ALWAYS(elem) ){ - if( elem->nKey==nKey && sqlite3StrNICmp(elem->pKey,pKey,nKey)==0 ){ + if( elem->nKey==nKey && sqlite3StrNICmp(elem->pKey,pKey,nKey)==0 ){ return elem; } elem = elem->next; @@ -19737,7 +19735,7 @@ static void removeElementGivenHash( ){ struct _ht *pEntry; if( elem->prev ){ - elem->prev->next = elem->next; + elem->prev->next = elem->next; }else{ pH->first = elem->next; } @@ -20013,8 +20011,6 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){ ****************************************************************************** ** ** This file contains code that is specific to OS/2. -** -** $Id: os_os2.c,v 1.63 2008/12/10 19:26:24 drh Exp $ */ @@ -20076,8 +20072,6 @@ SQLITE_PRIVATE const char *sqlite3OpcodeName(int i){ ** ** This file should be #included by the os_*.c files only. It is not a ** general purpose header file. -** -** $Id: os_common.h,v 1.38 2009/02/24 18:40:50 danielk1977 Exp $ */ #ifndef _OS_COMMON_H_ #define _OS_COMMON_H_ @@ -20118,8 +20112,8 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; */ #ifdef SQLITE_PERFORMANCE_TRACE -/* -** hwtime.h contains inline assembler code for implementing +/* +** hwtime.h contains inline assembler code for implementing ** high-performance timing routines. */ /************** Include hwtime.h in the middle of os_common.h ****************/ @@ -20138,8 +20132,6 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; ** ** This file contains inline asm code for retrieving "high-performance" ** counters for x86 class CPUs. -** -** $Id: hwtime.h,v 1.3 2008/08/01 14:33:15 shane Exp $ */ #ifndef _HWTIME_H_ #define _HWTIME_H_ @@ -20179,7 +20171,7 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; __asm__ __volatile__ ("rdtsc" : "=A" (val)); return val; } - + #elif (defined(__GNUC__) && defined(__ppc__)) __inline__ sqlite_uint64 sqlite3Hwtime(void){ @@ -20988,8 +20980,8 @@ static int os2Open( char *zNameCp; char zTmpname[CCHMAXPATH+1]; /* Buffer to hold name of temp file */ - /* If the second argument to this function is NULL, generate a - ** temporary file name to use + /* If the second argument to this function is NULL, generate a + ** temporary file name to use */ if( !zName ){ int rc = getTempname(CCHMAXPATH+1, zTmpname); @@ -21414,7 +21406,7 @@ SQLITE_API int sqlite3_os_end(void){ ** Styles 4, 5, and 7 are only available of SQLITE_ENABLE_LOCKING_STYLE ** is defined to 1. The SQLITE_ENABLE_LOCKING_STYLE also enables automatic ** selection of the appropriate locking style based on the filesystem -** where the database is located. +** where the database is located. */ #if !defined(SQLITE_ENABLE_LOCKING_STYLE) # if defined(__APPLE__) @@ -21425,7 +21417,7 @@ SQLITE_API int sqlite3_os_end(void){ #endif /* -** Define the OS_VXWORKS pre-processor macro to 1 if building on +** Define the OS_VXWORKS pre-processor macro to 1 if building on ** vxworks, or 0 otherwise. */ #ifndef OS_VXWORKS @@ -21512,7 +21504,7 @@ SQLITE_API int sqlite3_os_end(void){ #define MAX_PATHNAME 512 /* -** Only set the lastErrno if the error code is a real error and not +** Only set the lastErrno if the error code is a real error and not ** a normal expected return code of SQLITE_BUSY or SQLITE_OK */ #define IS_LOCK_ERROR(x) ((x != SQLITE_OK) && (x != SQLITE_BUSY)) @@ -21563,14 +21555,14 @@ struct unixFile { ** whenever any part of the database changes. An assertion fault will ** occur if a file is updated without also updating the transaction ** counter. This test is made to avoid new problems similar to the - ** one described by ticket #3584. + ** one described by ticket #3584. */ unsigned char transCntrChng; /* True if the transaction counter changed */ unsigned char dbUpdate; /* True if any part of database file changed */ unsigned char inNormalWrite; /* True if in a normal write operation */ #endif #ifdef SQLITE_TEST - /* In test mode, increase the size of this structure a bit so that + /* In test mode, increase the size of this structure a bit so that ** it is larger than the struct CrashFile defined in test6.c. */ char aPadding[32]; @@ -21605,8 +21597,6 @@ struct unixFile { ** ** This file should be #included by the os_*.c files only. It is not a ** general purpose header file. -** -** $Id: os_common.h,v 1.38 2009/02/24 18:40:50 danielk1977 Exp $ */ #ifndef _OS_COMMON_H_ #define _OS_COMMON_H_ @@ -21647,8 +21637,8 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; */ #ifdef SQLITE_PERFORMANCE_TRACE -/* -** hwtime.h contains inline assembler code for implementing +/* +** hwtime.h contains inline assembler code for implementing ** high-performance timing routines. */ /************** Include hwtime.h in the middle of os_common.h ****************/ @@ -21667,8 +21657,6 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; ** ** This file contains inline asm code for retrieving "high-performance" ** counters for x86 class CPUs. -** -** $Id: hwtime.h,v 1.3 2008/08/01 14:33:15 shane Exp $ */ #ifndef _HWTIME_H_ #define _HWTIME_H_ @@ -21708,7 +21696,7 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; __asm__ __volatile__ ("rdtsc" : "=A" (val)); return val; } - + #elif (defined(__GNUC__) && defined(__ppc__)) __inline__ sqlite_uint64 sqlite3Hwtime(void){ @@ -21851,11 +21839,11 @@ SQLITE_API int sqlite3_open_file_count = 0; /* ** Helper functions to obtain and relinquish the global mutex. The ** global mutex is used to protect the unixOpenCnt, unixLockInfo and -** vxworksFileId objects used by this file, all of which may be +** vxworksFileId objects used by this file, all of which may be ** shared by multiple threads. ** -** Function unixMutexHeld() is used to assert() that the global mutex -** is held when required. This function is only used as part of assert() +** Function unixMutexHeld() is used to assert() that the global mutex +** is held when required. This function is only used as part of assert() ** statements. e.g. ** ** unixEnterMutex() @@ -21958,49 +21946,49 @@ static int lockTrace(int fd, int op, struct flock *p){ ** This routine translates a standard POSIX errno code into something ** useful to the clients of the sqlite3 functions. Specifically, it is ** intended to translate a variety of "try again" errors into SQLITE_BUSY -** and a variety of "please close the file descriptor NOW" errors into +** and a variety of "please close the file descriptor NOW" errors into ** SQLITE_IOERR -** +** ** Errors during initialization of locks, or file system support for locks, ** should handle ENOLCK, ENOTSUP, EOPNOTSUPP separately. */ static int sqliteErrorFromPosixError(int posixError, int sqliteIOErr) { switch (posixError) { - case 0: + case 0: return SQLITE_OK; - + case EAGAIN: case ETIMEDOUT: case EBUSY: case EINTR: - case ENOLCK: - /* random NFS retry error, unless during file system support + case ENOLCK: + /* random NFS retry error, unless during file system support * introspection, in which it actually means what it says */ return SQLITE_BUSY; - - case EACCES: + + case EACCES: /* EACCES is like EAGAIN during locking operations, but not any other time*/ - if( (sqliteIOErr == SQLITE_IOERR_LOCK) || - (sqliteIOErr == SQLITE_IOERR_UNLOCK) || + if( (sqliteIOErr == SQLITE_IOERR_LOCK) || + (sqliteIOErr == SQLITE_IOERR_UNLOCK) || (sqliteIOErr == SQLITE_IOERR_RDLOCK) || (sqliteIOErr == SQLITE_IOERR_CHECKRESERVEDLOCK) ){ return SQLITE_BUSY; } /* else fall through */ - case EPERM: + case EPERM: return SQLITE_PERM; - + case EDEADLK: return SQLITE_IOERR_BLOCKED; - + #if EOPNOTSUPP!=ENOTSUP - case EOPNOTSUPP: - /* something went terribly awry, unless during file system support + case EOPNOTSUPP: + /* something went terribly awry, unless during file system support * introspection, in which it actually means what it says */ #endif #ifdef ENOTSUP - case ENOTSUP: - /* invalid fd, unless during file system support introspection, in which + case ENOTSUP: + /* invalid fd, unless during file system support introspection, in which * it actually means what it says */ #endif case EIO: @@ -22013,8 +22001,8 @@ static int sqliteErrorFromPosixError(int posixError, int sqliteIOErr) { case ESTALE: case ENOSYS: /* these should force the client to close the file and reconnect */ - - default: + + default: return sqliteIOErr; } } @@ -22030,7 +22018,7 @@ static int sqliteErrorFromPosixError(int posixError, int sqliteIOErr) { ** ** A pointer to an instance of the following structure can be used as a ** unique file ID in VxWorks. Each instance of this structure contains -** a copy of the canonical filename. There is also a reference count. +** a copy of the canonical filename. There is also a reference count. ** The structure is reclaimed when the number of pointers to it drops to ** zero. ** @@ -22046,7 +22034,7 @@ struct vxworksFileId { }; #if OS_VXWORKS -/* +/* ** All unique filenames are held on a linked list headed by this ** variable: */ @@ -22118,7 +22106,7 @@ static struct vxworksFileId *vxworksFindFileId(const char *zAbsoluteName){ */ unixEnterMutex(); for(pCandidate=vxworksFileList; pCandidate; pCandidate=pCandidate->pNext){ - if( pCandidate->nName==n + if( pCandidate->nName==n && memcmp(pCandidate->zCanonicalName, pNew->zCanonicalName, n)==0 ){ sqlite3_free(pNew); @@ -22211,7 +22199,7 @@ static void vxworksReleaseFileId(struct vxworksFileId *pId){ ** cnt>0 means there are cnt shared locks on the file. ** ** Any attempt to lock or unlock a file first checks the locking -** structure. The fcntl() system call is only invoked to set a +** structure. The fcntl() system call is only invoked to set a ** POSIX lock if the internal lock structure transitions between ** a locked and an unlocked state. ** @@ -22264,7 +22252,7 @@ static void vxworksReleaseFileId(struct vxworksFileId *pId){ ** another as long as we are running on a system that supports threads ** overriding each others locks (which is now the most common behavior) ** or if no locks are held. But the unixFile.pLock field needs to be -** recomputed because its key includes the thread-id. See the +** recomputed because its key includes the thread-id. See the ** transferOwnership() function below for additional information */ #if SQLITE_THREADSAFE && defined(__linux__) @@ -22404,9 +22392,9 @@ struct threadTestData { ** by the main thread in testThreadLockingBehavior() conflicts with a ** hypothetical write-lock obtained by this thread on the same file. ** -** The write-lock is not actually acquired, as this is not possible if +** The write-lock is not actually acquired, as this is not possible if ** the file is open in read-only mode (see ticket #3472). -*/ +*/ static void *threadLockingTest(void *pArg){ struct threadTestData *pData = (struct threadTestData*)pArg; pData->result = fcntl(pData->fd, F_GETLK, &pData->lock); @@ -22418,7 +22406,7 @@ static void *threadLockingTest(void *pArg){ #if SQLITE_THREADSAFE && defined(__linux__) /* ** This procedure attempts to determine whether or not threads -** can override each others locks then sets the +** can override each others locks then sets the ** threadsOverrideEachOthersLocks variable appropriately. */ static void testThreadLockingBehavior(int fd_orig){ @@ -22731,9 +22719,9 @@ static int unixCheckReservedLock(sqlite3_file *id, int *pResOut){ } } #endif - + unixLeaveMutex(); - OSTRACE4("TEST WR-LOCK %d %d %d\n", pFile->h, rc, reserved); + OSTRACE4("TEST WR-LOCK %d %d %d (unix)\n", pFile->h, rc, reserved); *pResOut = reserved; return rc; @@ -22837,7 +22825,7 @@ static int unixLock(sqlite3_file *id, int locktype){ ** ** A process may only obtain a RESERVED lock after it has a SHARED lock. ** A RESERVED lock is implemented by grabbing a write-lock on the - ** 'reserved byte'. + ** 'reserved byte'. ** ** A process may only obtain a PENDING lock after it has obtained a ** SHARED lock. A PENDING lock is implemented by obtaining a write-lock @@ -22851,7 +22839,7 @@ static int unixLock(sqlite3_file *id, int locktype){ ** implemented by obtaining a write-lock on the entire 'shared byte ** range'. Since all other locks require a read-lock on one of the bytes ** within this range, this ensures that no other locks are held on the - ** database. + ** database. ** ** The reason a single byte cannot be used instead of the 'shared byte ** range' is that some versions of windows do not support read-locks. By @@ -22866,7 +22854,7 @@ static int unixLock(sqlite3_file *id, int locktype){ int tErrno; assert( pFile ); - OSTRACE7("LOCK %d %s was %s(%s,%d) pid=%d\n", pFile->h, + OSTRACE7("LOCK %d %s was %s(%s,%d) pid=%d (unix)\n", pFile->h, locktypeName(locktype), locktypeName(pFile->locktype), locktypeName(pLock->locktype), pLock->cnt , getpid()); @@ -22875,7 +22863,7 @@ static int unixLock(sqlite3_file *id, int locktype){ ** unixEnterMutex() hasn't been called yet. */ if( pFile->locktype>=locktype ){ - OSTRACE3("LOCK %d %s ok (already held)\n", pFile->h, + OSTRACE3("LOCK %d %s ok (already held) (unix)\n", pFile->h, locktypeName(locktype)); return SQLITE_OK; } @@ -22905,7 +22893,7 @@ static int unixLock(sqlite3_file *id, int locktype){ /* If some thread using this PID has a lock via a different unixFile* ** handle that precludes the requested lock, return BUSY. */ - if( (pFile->locktype!=pLock->locktype && + if( (pFile->locktype!=pLock->locktype && (pLock->locktype>=PENDING_LOCK || locktype>SHARED_LOCK)) ){ rc = SQLITE_BUSY; @@ -22916,7 +22904,7 @@ static int unixLock(sqlite3_file *id, int locktype){ ** has a SHARED or RESERVED lock, then increment reference counts and ** return SQLITE_OK. */ - if( locktype==SHARED_LOCK && + if( locktype==SHARED_LOCK && (pLock->locktype==SHARED_LOCK || pLock->locktype==RESERVED_LOCK) ){ assert( locktype==SHARED_LOCK ); assert( pFile->locktype==0 ); @@ -22934,7 +22922,7 @@ static int unixLock(sqlite3_file *id, int locktype){ */ lock.l_len = 1L; lock.l_whence = SEEK_SET; - if( locktype==SHARED_LOCK + if( locktype==SHARED_LOCK || (locktype==EXCLUSIVE_LOCK && pFile->locktypeh, F_SETLK, &lock)!=0 ){ if( s != -1 ){ /* This could happen with a network mount */ - tErrno = errno; - rc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_UNLOCK); + tErrno = errno; + rc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_UNLOCK); if( IS_LOCK_ERROR(rc) ){ pFile->lastErrno = tErrno; } @@ -23016,7 +23004,7 @@ static int unixLock(sqlite3_file *id, int locktype){ } } } - + #ifndef NDEBUG /* Set up the transaction-counter change checking flags when @@ -23045,7 +23033,7 @@ static int unixLock(sqlite3_file *id, int locktype){ end_lock: unixLeaveMutex(); - OSTRACE4("LOCK %d %s %s\n", pFile->h, locktypeName(locktype), + OSTRACE4("LOCK %d %s %s (unix)\n", pFile->h, locktypeName(locktype), rc==SQLITE_OK ? "ok" : "failed"); return rc; } @@ -23056,9 +23044,9 @@ end_lock: ** cleared and SQLITE_OK returned. ** ** Otherwise, if an error occurs, then successfully closed file descriptor -** entries are removed from the list, and SQLITE_IOERR_CLOSE returned. +** entries are removed from the list, and SQLITE_IOERR_CLOSE returned. ** not deleted and SQLITE_IOERR_CLOSE returned. -*/ +*/ static int closePendingFds(unixFile *pFile){ int rc = SQLITE_OK; struct unixOpenCnt *pOpen = pFile->pOpen; @@ -23109,7 +23097,7 @@ static int unixUnlock(sqlite3_file *id, int locktype){ int tErrno; /* Error code from system call errors */ assert( pFile ); - OSTRACE7("UNLOCK %d %d was %d(%d,%d) pid=%d\n", pFile->h, locktype, + OSTRACE7("UNLOCK %d %d was %d(%d,%d) pid=%d (unix)\n", pFile->h, locktype, pFile->locktype, pFile->pLock->locktype, pFile->pLock->cnt, getpid()); assert( locktype<=SHARED_LOCK ); @@ -23211,7 +23199,7 @@ static int unixUnlock(sqlite3_file *id, int locktype){ } } } - + end_unlock: unixLeaveMutex(); if( rc==SQLITE_OK ) pFile->locktype = locktype; @@ -23219,7 +23207,7 @@ end_unlock: } /* -** This function performs the parts of the "close file" operation +** This function performs the parts of the "close file" operation ** common to all locking schemes. It closes the directory and file ** handles, if they are valid, and sets all fields of the unixFile ** structure to 0. @@ -23276,7 +23264,7 @@ static int unixClose(sqlite3_file *id){ if( pFile->pOpen && pFile->pOpen->nLock ){ /* If there are outstanding locks, do not actually close the file just ** yet because that would clear those locks. Instead, add the file - ** descriptor to pOpen->pUnused list. It will be automatically closed + ** descriptor to pOpen->pUnused list. It will be automatically closed ** when the last lock is cleared. */ setPendingFd(pFile); @@ -23377,7 +23365,7 @@ static int dotlockCheckReservedLock(sqlite3_file *id, int *pResOut) { unixFile *pFile = (unixFile*)id; SimulateIOError( return SQLITE_IOERR_CHECKRESERVEDLOCK; ); - + assert( pFile ); /* Check if a thread in this process holds such a lock */ @@ -23390,7 +23378,7 @@ static int dotlockCheckReservedLock(sqlite3_file *id, int *pResOut) { const char *zLockFile = (const char*)pFile->lockingContext; reserved = access(zLockFile, 0)==0; } - OSTRACE4("TEST WR-LOCK %d %d %d\n", pFile->h, rc, reserved); + OSTRACE4("TEST WR-LOCK %d %d %d (dotlock)\n", pFile->h, rc, reserved); *pResOut = reserved; return rc; } @@ -23440,7 +23428,7 @@ static int dotlockLock(sqlite3_file *id, int locktype) { #endif return SQLITE_OK; } - + /* grab an exclusive lock */ fd = open(zLockFile,O_RDONLY|O_CREAT|O_EXCL,0600); if( fd<0 ){ @@ -23455,12 +23443,12 @@ static int dotlockLock(sqlite3_file *id, int locktype) { } } return rc; - } + } if( close(fd) ){ pFile->lastErrno = errno; rc = SQLITE_IOERR_CLOSE; } - + /* got it, set the type and return ok */ pFile->locktype = locktype; return rc; @@ -23480,10 +23468,10 @@ static int dotlockUnlock(sqlite3_file *id, int locktype) { char *zLockFile = (char *)pFile->lockingContext; assert( pFile ); - OSTRACE5("UNLOCK %d %d was %d pid=%d\n", pFile->h, locktype, + OSTRACE5("UNLOCK %d %d was %d pid=%d (dotlock)\n", pFile->h, locktype, pFile->locktype, getpid()); assert( locktype<=SHARED_LOCK ); - + /* no-op if possible */ if( pFile->locktype==locktype ){ return SQLITE_OK; @@ -23496,7 +23484,7 @@ static int dotlockUnlock(sqlite3_file *id, int locktype) { pFile->locktype = SHARED_LOCK; return SQLITE_OK; } - + /* To fully unlock the database, delete the lock file */ assert( locktype==NO_LOCK ); if( unlink(zLockFile) ){ @@ -23508,7 +23496,7 @@ static int dotlockUnlock(sqlite3_file *id, int locktype) { if( IS_LOCK_ERROR(rc) ){ pFile->lastErrno = tErrno; } - return rc; + return rc; } pFile->locktype = NO_LOCK; return SQLITE_OK; @@ -23557,16 +23545,16 @@ static int flockCheckReservedLock(sqlite3_file *id, int *pResOut){ int rc = SQLITE_OK; int reserved = 0; unixFile *pFile = (unixFile*)id; - + SimulateIOError( return SQLITE_IOERR_CHECKRESERVEDLOCK; ); - + assert( pFile ); - + /* Check if a thread in this process holds such a lock */ if( pFile->locktype>SHARED_LOCK ){ reserved = 1; } - + /* Otherwise see if some other process holds it. */ if( !reserved ){ /* attempt to get the lock */ @@ -23577,7 +23565,7 @@ static int flockCheckReservedLock(sqlite3_file *id, int *pResOut){ if ( lrc ) { int tErrno = errno; /* unlock failed with an error */ - lrc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_UNLOCK); + lrc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_UNLOCK); if( IS_LOCK_ERROR(lrc) ){ pFile->lastErrno = tErrno; rc = lrc; @@ -23587,14 +23575,14 @@ static int flockCheckReservedLock(sqlite3_file *id, int *pResOut){ int tErrno = errno; reserved = 1; /* someone else might have it reserved */ - lrc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_LOCK); + lrc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_LOCK); if( IS_LOCK_ERROR(lrc) ){ pFile->lastErrno = tErrno; rc = lrc; } } } - OSTRACE4("TEST WR-LOCK %d %d %d\n", pFile->h, rc, reserved); + OSTRACE4("TEST WR-LOCK %d %d %d (flock)\n", pFile->h, rc, reserved); #ifdef SQLITE_IGNORE_FLOCK_LOCK_ERRORS if( (rc & SQLITE_IOERR) == SQLITE_IOERR ){ @@ -23641,15 +23629,15 @@ static int flockLock(sqlite3_file *id, int locktype) { assert( pFile ); - /* if we already have a lock, it is exclusive. + /* if we already have a lock, it is exclusive. ** Just adjust level and punt on outta here. */ if (pFile->locktype > NO_LOCK) { pFile->locktype = locktype; return SQLITE_OK; } - + /* grab an exclusive lock */ - + if (flock(pFile->h, LOCK_EX | LOCK_NB)) { int tErrno = errno; /* didn't get, must be busy */ @@ -23661,7 +23649,7 @@ static int flockLock(sqlite3_file *id, int locktype) { /* got it, set the type and return ok */ pFile->locktype = locktype; } - OSTRACE4("LOCK %d %s %s\n", pFile->h, locktypeName(locktype), + OSTRACE4("LOCK %d %s %s (flock)\n", pFile->h, locktypeName(locktype), rc==SQLITE_OK ? "ok" : "failed"); #ifdef SQLITE_IGNORE_FLOCK_LOCK_ERRORS if( (rc & SQLITE_IOERR) == SQLITE_IOERR ){ @@ -23681,23 +23669,23 @@ static int flockLock(sqlite3_file *id, int locktype) { */ static int flockUnlock(sqlite3_file *id, int locktype) { unixFile *pFile = (unixFile*)id; - + assert( pFile ); - OSTRACE5("UNLOCK %d %d was %d pid=%d\n", pFile->h, locktype, + OSTRACE5("UNLOCK %d %d was %d pid=%d (flock)\n", pFile->h, locktype, pFile->locktype, getpid()); assert( locktype<=SHARED_LOCK ); - + /* no-op if possible */ if( pFile->locktype==locktype ){ return SQLITE_OK; } - + /* shared can just be set because we always have an exclusive */ if (locktype==SHARED_LOCK) { pFile->locktype = locktype; return SQLITE_OK; } - + /* no, really, unlock. */ int rc = flock(pFile->h, LOCK_UN); if (rc) { @@ -23711,7 +23699,7 @@ static int flockUnlock(sqlite3_file *id, int locktype) { r = SQLITE_BUSY; } #endif /* SQLITE_IGNORE_FLOCK_LOCK_ERRORS */ - + return r; } else { pFile->locktype = NO_LOCK; @@ -23758,14 +23746,14 @@ static int semCheckReservedLock(sqlite3_file *id, int *pResOut) { unixFile *pFile = (unixFile*)id; SimulateIOError( return SQLITE_IOERR_CHECKRESERVEDLOCK; ); - + assert( pFile ); /* Check if a thread in this process holds such a lock */ if( pFile->locktype>SHARED_LOCK ){ reserved = 1; } - + /* Otherwise see if some other process holds it. */ if( !reserved ){ sem_t *pSem = pFile->pOpen->pSem; @@ -23785,7 +23773,7 @@ static int semCheckReservedLock(sqlite3_file *id, int *pResOut) { sem_post(pSem); } } - OSTRACE4("TEST WR-LOCK %d %d %d\n", pFile->h, rc, reserved); + OSTRACE4("TEST WR-LOCK %d %d %d (sem)\n", pFile->h, rc, reserved); *pResOut = reserved; return rc; @@ -23826,14 +23814,14 @@ static int semLock(sqlite3_file *id, int locktype) { sem_t *pSem = pFile->pOpen->pSem; int rc = SQLITE_OK; - /* if we already have a lock, it is exclusive. + /* if we already have a lock, it is exclusive. ** Just adjust level and punt on outta here. */ if (pFile->locktype > NO_LOCK) { pFile->locktype = locktype; rc = SQLITE_OK; goto sem_end_lock; } - + /* lock semaphore now but bail out when already locked. */ if( sem_trywait(pSem)==-1 ){ rc = SQLITE_BUSY; @@ -23860,21 +23848,21 @@ static int semUnlock(sqlite3_file *id, int locktype) { assert( pFile ); assert( pSem ); - OSTRACE5("UNLOCK %d %d was %d pid=%d\n", pFile->h, locktype, + OSTRACE5("UNLOCK %d %d was %d pid=%d (sem)\n", pFile->h, locktype, pFile->locktype, getpid()); assert( locktype<=SHARED_LOCK ); - + /* no-op if possible */ if( pFile->locktype==locktype ){ return SQLITE_OK; } - + /* shared can just be set because we always have an exclusive */ if (locktype==SHARED_LOCK) { pFile->locktype = locktype; return SQLITE_OK; } - + /* no, really unlock. */ if ( sem_post(pSem)==-1 ) { int rc, tErrno = errno; @@ -23882,7 +23870,7 @@ static int semUnlock(sqlite3_file *id, int locktype) { if( IS_LOCK_ERROR(rc) ){ pFile->lastErrno = tErrno; } - return rc; + return rc; } pFile->locktype = NO_LOCK; return SQLITE_OK; @@ -23948,7 +23936,7 @@ struct ByteRangeLockPB2 /* ** This is a utility for setting or clearing a bit-range lock on an ** AFP filesystem. -** +** ** Return SQLITE_OK on success, SQLITE_BUSY on failure. */ static int afpSetLock( @@ -23960,14 +23948,14 @@ static int afpSetLock( ){ struct ByteRangeLockPB2 pb; int err; - + pb.unLockFlag = setLockFlag ? 0 : 1; pb.startEndFlag = 0; pb.offset = offset; - pb.length = length; + pb.length = length; pb.fd = pFile->h; - - OSTRACE6("AFPSETLOCK [%s] for %d%s in range %llx:%llx\n", + + OSTRACE6("AFPSETLOCK [%s] for %d%s in range %llx:%llx\n", (setLockFlag?"ON":"OFF"), pFile->h, (pb.fd==-1?"[testval-1]":""), offset, length); err = fsctl(path, afpfsByteRangeLock2FSCTL, &pb, 0); @@ -24001,22 +23989,22 @@ static int afpCheckReservedLock(sqlite3_file *id, int *pResOut){ int rc = SQLITE_OK; int reserved = 0; unixFile *pFile = (unixFile*)id; - + SimulateIOError( return SQLITE_IOERR_CHECKRESERVEDLOCK; ); - + assert( pFile ); afpLockingContext *context = (afpLockingContext *) pFile->lockingContext; - + /* Check if a thread in this process holds such a lock */ if( pFile->locktype>SHARED_LOCK ){ reserved = 1; } - + /* Otherwise see if some other process holds it. */ if( !reserved ){ /* lock the RESERVED byte */ - int lrc = afpSetLock(context->dbPath, pFile, RESERVED_BYTE, 1,1); + int lrc = afpSetLock(context->dbPath, pFile, RESERVED_BYTE, 1,1); if( SQLITE_OK==lrc ){ /* if we succeeded in taking the reserved lock, unlock it to restore ** the original state */ @@ -24029,9 +24017,9 @@ static int afpCheckReservedLock(sqlite3_file *id, int *pResOut){ rc=lrc; } } - - OSTRACE4("TEST WR-LOCK %d %d %d\n", pFile->h, rc, reserved); - + + OSTRACE4("TEST WR-LOCK %d %d %d (afp)\n", pFile->h, rc, reserved); + *pResOut = reserved; return rc; } @@ -24064,9 +24052,9 @@ static int afpLock(sqlite3_file *id, int locktype){ int rc = SQLITE_OK; unixFile *pFile = (unixFile*)id; afpLockingContext *context = (afpLockingContext *) pFile->lockingContext; - + assert( pFile ); - OSTRACE5("LOCK %d %s was %s pid=%d\n", pFile->h, + OSTRACE5("LOCK %d %s was %s pid=%d (afp)\n", pFile->h, locktypeName(locktype), locktypeName(pFile->locktype), getpid()); /* If there is already a lock of this type or more restrictive on the @@ -24074,7 +24062,7 @@ static int afpLock(sqlite3_file *id, int locktype){ ** unixEnterMutex() hasn't been called yet. */ if( pFile->locktype>=locktype ){ - OSTRACE3("LOCK %d %s ok (already held)\n", pFile->h, + OSTRACE3("LOCK %d %s ok (already held) (afp)\n", pFile->h, locktypeName(locktype)); return SQLITE_OK; } @@ -24084,7 +24072,7 @@ static int afpLock(sqlite3_file *id, int locktype){ assert( pFile->locktype!=NO_LOCK || locktype==SHARED_LOCK ); assert( locktype!=PENDING_LOCK ); assert( locktype!=RESERVED_LOCK || pFile->locktype==SHARED_LOCK ); - + /* This mutex is needed because pFile->pLock is shared across threads */ unixEnterMutex(); @@ -24096,12 +24084,12 @@ static int afpLock(sqlite3_file *id, int locktype){ unixLeaveMutex(); return rc; } - + /* A PENDING lock is needed before acquiring a SHARED lock and before ** acquiring an EXCLUSIVE lock. For the SHARED lock, the PENDING will ** be released. */ - if( locktype==SHARED_LOCK + if( locktype==SHARED_LOCK || (locktype==EXCLUSIVE_LOCK && pFile->locktypesharedByte = (lk & 0x7fffffff)%(SHARED_SIZE - 1); - lrc1 = afpSetLock(context->dbPath, pFile, + lrc1 = afpSetLock(context->dbPath, pFile, SHARED_FIRST+context->sharedByte, 1, 1); if( IS_LOCK_ERROR(lrc1) ){ lrc1Errno = pFile->lastErrno; } /* Drop the temporary PENDING lock */ lrc2 = afpSetLock(context->dbPath, pFile, PENDING_BYTE, 1, 0); - + if( IS_LOCK_ERROR(lrc1) ) { pFile->lastErrno = lrc1Errno; rc = lrc1; @@ -24157,43 +24145,43 @@ static int afpLock(sqlite3_file *id, int locktype){ } if (!failed && locktype == EXCLUSIVE_LOCK) { /* Acquire an EXCLUSIVE lock */ - - /* Remove the shared lock before trying the range. we'll need to + + /* Remove the shared lock before trying the range. we'll need to ** reestablish the shared lock if we can't get the afpUnlock */ if( !(failed = afpSetLock(context->dbPath, pFile, SHARED_FIRST + context->sharedByte, 1, 0)) ){ int failed2 = SQLITE_OK; /* now attemmpt to get the exclusive lock range */ - failed = afpSetLock(context->dbPath, pFile, SHARED_FIRST, + failed = afpSetLock(context->dbPath, pFile, SHARED_FIRST, SHARED_SIZE, 1); - if( failed && (failed2 = afpSetLock(context->dbPath, pFile, + if( failed && (failed2 = afpSetLock(context->dbPath, pFile, SHARED_FIRST + context->sharedByte, 1, 1)) ){ /* Can't reestablish the shared lock. Sqlite can't deal, this is ** a critical I/O error */ - rc = ((failed & SQLITE_IOERR) == SQLITE_IOERR) ? failed2 : + rc = ((failed & SQLITE_IOERR) == SQLITE_IOERR) ? failed2 : SQLITE_IOERR_LOCK; goto afp_end_lock; - } + } }else{ - rc = failed; + rc = failed; } } if( failed ){ rc = failed; } } - + if( rc==SQLITE_OK ){ pFile->locktype = locktype; }else if( locktype==EXCLUSIVE_LOCK ){ pFile->locktype = PENDING_LOCK; } - + afp_end_lock: unixLeaveMutex(); - OSTRACE4("LOCK %d %s %s\n", pFile->h, locktypeName(locktype), + OSTRACE4("LOCK %d %s %s (afp)\n", pFile->h, locktypeName(locktype), rc==SQLITE_OK ? "ok" : "failed"); return rc; } @@ -24211,7 +24199,7 @@ static int afpUnlock(sqlite3_file *id, int locktype) { afpLockingContext *pCtx = (afpLockingContext *) pFile->lockingContext; assert( pFile ); - OSTRACE5("UNLOCK %d %d was %d pid=%d\n", pFile->h, locktype, + OSTRACE5("UNLOCK %d %d was %d pid=%d (afp)\n", pFile->h, locktype, pFile->locktype, getpid()); assert( locktype<=SHARED_LOCK ); @@ -24223,7 +24211,7 @@ static int afpUnlock(sqlite3_file *id, int locktype) { } unixEnterMutex(); if( pFile->locktype>SHARED_LOCK ){ - + if( pFile->locktype==EXCLUSIVE_LOCK ){ rc = afpSetLock(pCtx->dbPath, pFile, SHARED_FIRST, SHARED_SIZE, 0); if( rc==SQLITE_OK && locktype==SHARED_LOCK ){ @@ -24234,7 +24222,7 @@ static int afpUnlock(sqlite3_file *id, int locktype) { } if( rc==SQLITE_OK && pFile->locktype>=PENDING_LOCK ){ rc = afpSetLock(pCtx->dbPath, pFile, PENDING_BYTE, 1, 0); - } + } if( rc==SQLITE_OK && pFile->locktype>=RESERVED_LOCK ){ rc = afpSetLock(pCtx->dbPath, pFile, RESERVED_BYTE, 1, 0); } @@ -24262,7 +24250,7 @@ static int afpUnlock(sqlite3_file *id, int locktype) { } /* -** Close a file & cleanup AFP specific locking context +** Close a file & cleanup AFP specific locking context */ static int afpClose(sqlite3_file *id) { if( id ){ @@ -24299,7 +24287,7 @@ static int afpClose(sqlite3_file *id) { /****************************************************************************** **************** Non-locking sqlite3_file methods ***************************** ** -** The next division contains implementations for all methods of the +** The next division contains implementations for all methods of the ** sqlite3_file object other than the locking methods. The locking ** methods were defined in divisions above (one locking method per ** division). Those methods that are common to all locking modes @@ -24307,7 +24295,7 @@ static int afpClose(sqlite3_file *id) { */ /* -** Seek to the offset passed as the second argument, then read cnt +** Seek to the offset passed as the second argument, then read cnt ** bytes into pBuf. Return the number of bytes actually read. ** ** NB: If you define USE_PREAD or USE_PREAD64, then it might also @@ -24336,7 +24324,7 @@ static int seekAndRead(unixFile *id, sqlite3_int64 offset, void *pBuf, int cnt){ if( newOffset == -1 ){ ((unixFile*)id)->lastErrno = errno; }else{ - ((unixFile*)id)->lastErrno = 0; + ((unixFile*)id)->lastErrno = 0; } return -1; } @@ -24356,8 +24344,8 @@ static int seekAndRead(unixFile *id, sqlite3_int64 offset, void *pBuf, int cnt){ ** wrong. */ static int unixRead( - sqlite3_file *id, - void *pBuf, + sqlite3_file *id, + void *pBuf, int amt, sqlite3_int64 offset ){ @@ -24369,7 +24357,7 @@ static int unixRead( ** file), the bytes in the locking range should never be read or written. */ assert( pFile->pUnused==0 || offset>=PENDING_BYTE+512 - || offset+amt<=PENDING_BYTE + || offset+amt<=PENDING_BYTE ); got = seekAndRead(pFile, offset, pBuf, amt); @@ -24407,7 +24395,7 @@ static int seekAndWrite(unixFile *id, i64 offset, const void *pBuf, int cnt){ if( newOffset == -1 ){ ((unixFile*)id)->lastErrno = errno; }else{ - ((unixFile*)id)->lastErrno = 0; + ((unixFile*)id)->lastErrno = 0; } return -1; } @@ -24428,10 +24416,10 @@ static int seekAndWrite(unixFile *id, i64 offset, const void *pBuf, int cnt){ ** or some other error code on failure. */ static int unixWrite( - sqlite3_file *id, - const void *pBuf, + sqlite3_file *id, + const void *pBuf, int amt, - sqlite3_int64 offset + sqlite3_int64 offset ){ unixFile *pFile = (unixFile*)id; int wrote = 0; @@ -24442,7 +24430,7 @@ static int unixWrite( ** file), the bytes in the locking range should never be read or written. */ assert( pFile->pUnused==0 || offset>=PENDING_BYTE+512 - || offset+amt<=PENDING_BYTE + || offset+amt<=PENDING_BYTE ); #ifndef NDEBUG @@ -24530,8 +24518,8 @@ SQLITE_API int sqlite3_fullsync_count = 0; ** ** SQLite sets the dataOnly flag if the size of the file is unchanged. ** The idea behind dataOnly is that it should only write the file content -** to disk, not the inode. We only set dataOnly if the file size is -** unchanged since the file size is part of the inode. However, +** to disk, not the inode. We only set dataOnly if the file size is +** unchanged since the file size is part of the inode. However, ** Ted Ts'o tells us that fdatasync() will also write the inode if the ** file size has changed. The only real difference between fdatasync() ** and fsync(), Ted tells us, is that fdatasync() will not flush the @@ -24545,7 +24533,7 @@ static int full_fsync(int fd, int fullSync, int dataOnly){ int rc; /* The following "ifdef/elif/else/" block has the same structure as - ** the one below. It is replicated here solely to avoid cluttering + ** the one below. It is replicated here solely to avoid cluttering ** up the real code with the UNUSED_PARAMETER() macros. */ #ifdef SQLITE_NO_SYNC @@ -24559,7 +24547,7 @@ static int full_fsync(int fd, int fullSync, int dataOnly){ UNUSED_PARAMETER(dataOnly); #endif - /* Record the number of times that we do a normal fsync() and + /* Record the number of times that we do a normal fsync() and ** FULLSYNC. This is used during testing to verify that this procedure ** gets called with the correct arguments. */ @@ -24580,16 +24568,16 @@ static int full_fsync(int fd, int fullSync, int dataOnly){ rc = 1; } /* If the FULLFSYNC failed, fall back to attempting an fsync(). - ** It shouldn't be possible for fullfsync to fail on the local + ** It shouldn't be possible for fullfsync to fail on the local ** file system (on OSX), so failure indicates that FULLFSYNC - ** isn't supported for this file system. So, attempt an fsync - ** and (for now) ignore the overhead of a superfluous fcntl call. - ** It'd be better to detect fullfsync support once and avoid + ** isn't supported for this file system. So, attempt an fsync + ** and (for now) ignore the overhead of a superfluous fcntl call. + ** It'd be better to detect fullfsync support once and avoid ** the fcntl call every time sync is called. */ if( rc ) rc = fsync(fd); -#else +#else rc = fdatasync(fd); #if OS_VXWORKS if( rc==-1 && errno==ENOTSUP ){ @@ -24687,6 +24675,19 @@ static int unixTruncate(sqlite3_file *id, i64 nByte){ ((unixFile*)id)->lastErrno = errno; return SQLITE_IOERR_TRUNCATE; }else{ +#ifndef NDEBUG + /* If we are doing a normal write to a database file (as opposed to + ** doing a hot-journal rollback or a write to some file other than a + ** normal database file) and we truncate the file to zero length, + ** that effectively updates the change counter. This might happen + ** when restoring a database using the backup API from a zero-length + ** source. + */ + if( ((unixFile*)id)->inNormalWrite && nByte==0 ){ + ((unixFile*)id)->transCntrChng = 1; + } +#endif + return SQLITE_OK; } } @@ -24924,7 +24925,7 @@ static const sqlite3_io_methods *posixWflIoFinderImpl(const char*z, unixFile*p){ p->fileFlags = SQLITE_WHOLE_FILE_LOCKING; return &posixIoMethods; } -static const sqlite3_io_methods +static const sqlite3_io_methods *(*const posixWflIoFinder)(const char*,unixFile *p) = posixWflIoFinderImpl; /* @@ -24953,8 +24954,8 @@ IOMETHODS( #if defined(__APPLE__) && SQLITE_ENABLE_LOCKING_STYLE -/* -** This "finder" function attempts to determine the best locking strategy +/* +** This "finder" function attempts to determine the best locking strategy ** for the database file "filePath". It then returns the sqlite3_io_methods ** object that implements that strategy. ** @@ -25000,8 +25001,8 @@ static const sqlite3_io_methods *autolockIoFinderImpl( } /* Default case. Handles, amongst others, "nfs". - ** Test byte-range lock using fcntl(). If the call succeeds, - ** assume that the file-system supports POSIX style locks. + ** Test byte-range lock using fcntl(). If the call succeeds, + ** assume that the file-system supports POSIX style locks. */ lockInfo.l_len = 1; lockInfo.l_start = 0; @@ -25014,14 +25015,14 @@ static const sqlite3_io_methods *autolockIoFinderImpl( return &dotlockIoMethods; } } -static const sqlite3_io_methods +static const sqlite3_io_methods *(*const autolockIoFinder)(const char*,unixFile*) = autolockIoFinderImpl; #endif /* defined(__APPLE__) && SQLITE_ENABLE_LOCKING_STYLE */ #if OS_VXWORKS && SQLITE_ENABLE_LOCKING_STYLE -/* -** This "finder" function attempts to determine the best locking strategy +/* +** This "finder" function attempts to determine the best locking strategy ** for the database file "filePath". It then returns the sqlite3_io_methods ** object that implements that strategy. ** @@ -25052,7 +25053,7 @@ static const sqlite3_io_methods *autolockIoFinderImpl( return &semIoMethods; } } -static const sqlite3_io_methods +static const sqlite3_io_methods *(*const autolockIoFinder)(const char*,unixFile*) = autolockIoFinderImpl; #endif /* OS_VXWORKS && SQLITE_ENABLE_LOCKING_STYLE */ @@ -25089,12 +25090,12 @@ static int fillInUnixFile( assert( pNew->pLock==NULL ); assert( pNew->pOpen==NULL ); - /* Parameter isDelete is only used on vxworks. Express this explicitly + /* Parameter isDelete is only used on vxworks. Express this explicitly ** here to prevent compiler warnings about unused parameters. */ UNUSED_PARAMETER(isDelete); - OSTRACE3("OPEN %-3d %s\n", h, zFilename); + OSTRACE3("OPEN %-3d %s\n", h, zFilename); pNew->h = h; pNew->dirfd = dirfd; SET_THREADID(pNew); @@ -25165,14 +25166,14 @@ static int fillInUnixFile( srandomdev(); unixEnterMutex(); rc = findLockInfo(pNew, NULL, &pNew->pOpen); - unixLeaveMutex(); + unixLeaveMutex(); } } #endif else if( pLockingStyle == &dotlockIoMethods ){ /* Dotfile locking uses the file path so it needs to be included in - ** the dotlockLockingContext + ** the dotlockLockingContext */ char *zLockFile; int nFilename; @@ -25209,7 +25210,7 @@ static int fillInUnixFile( unixLeaveMutex(); } #endif - + pNew->lastErrno = 0; #if OS_VXWORKS if( rc!=SQLITE_OK ){ @@ -25283,7 +25284,7 @@ static int getTempname(int nBuf, char *zBuf){ /* It's odd to simulate an io-error here, but really this is just ** using the io-error infrastructure to test that SQLite handles this - ** function failing. + ** function failing. */ SimulateIOError( return SQLITE_IOERR ); @@ -25291,7 +25292,7 @@ static int getTempname(int nBuf, char *zBuf){ if (NULL == azDirs[1]) { azDirs[1] = getenv("TMPDIR"); } - + for(i=0; i= (size_t)nBuf ){ @@ -25330,7 +25331,7 @@ static int proxyTransformUnixFile(unixFile*, const char*); #endif /* -** Search for an unused file descriptor that was opened on the database +** Search for an unused file descriptor that was opened on the database ** file (not a journal or master-journal file) identified by pathname ** zPath with SQLITE_OPEN_XXX flags matching those passed as the second ** argument to this function. @@ -25339,7 +25340,7 @@ static int proxyTransformUnixFile(unixFile*, const char*); ** but the associated file descriptor could not be closed because some ** other file descriptor open on the same file is holding a file-lock. ** Refer to comments in the unixClose() function and the lengthy comment -** describing "Posix Advisory Locking" at the start of this file for +** describing "Posix Advisory Locking" at the start of this file for ** further details. Also, ticket #4018. ** ** If a suitable file descriptor is found, then it is returned. If no @@ -25350,8 +25351,8 @@ static UnixUnusedFd *findReusableFd(const char *zPath, int flags){ /* Do not search for an unused file descriptor on vxworks. Not because ** vxworks would not benefit from the change (it might, we're not sure), - ** but because no way to test it is currently available. It is better - ** not to risk breaking vxworks support for the sake of such an obscure + ** but because no way to test it is currently available. It is better + ** not to risk breaking vxworks support for the sake of such an obscure ** feature. */ #if !OS_VXWORKS struct stat sStat; /* Results of stat() call */ @@ -25388,7 +25389,7 @@ static UnixUnusedFd *findReusableFd(const char *zPath, int flags){ /* ** Open the file zPath. -** +** ** Previously, the SQLite OS layer used three functions in place of this ** one: ** @@ -25399,13 +25400,13 @@ static UnixUnusedFd *findReusableFd(const char *zPath, int flags){ ** These calls correspond to the following combinations of flags: ** ** ReadWrite() -> (READWRITE | CREATE) -** ReadOnly() -> (READONLY) +** ReadOnly() -> (READONLY) ** OpenExclusive() -> (READWRITE | CREATE | EXCLUSIVE) ** ** The old OpenExclusive() accepted a boolean argument - "delFlag". If ** true, the file was configured to be automatically deleted when the -** file handle closed. To achieve the same effect using this new -** interface, add the DELETEONCLOSE flag to those specified above for +** file handle closed. To achieve the same effect using this new +** interface, add the DELETEONCLOSE flag to those specified above for ** OpenExclusive(). */ static int unixOpen( @@ -25433,7 +25434,7 @@ static int unixOpen( ** a file-descriptor on the directory too. The first time unixSync() ** is called the directory file descriptor will be fsync()ed and close()d. */ - int isOpenDirectory = (isCreate && + int isOpenDirectory = (isCreate && (eType==SQLITE_OPEN_MASTER_JOURNAL || eType==SQLITE_OPEN_MAIN_JOURNAL) ); @@ -25443,9 +25444,9 @@ static int unixOpen( char zTmpname[MAX_PATHNAME+1]; const char *zName = zPath; - /* Check the following statements are true: + /* Check the following statements are true: ** - ** (a) Exactly one of the READWRITE and READONLY flags must be set, and + ** (a) Exactly one of the READWRITE and READONLY flags must be set, and ** (b) if CREATE is set, then READWRITE must also be set, and ** (c) if EXCLUSIVE is set, then CREATE must also be set. ** (d) if DELETEONCLOSE is set, then CREATE must also be set. @@ -25462,9 +25463,9 @@ static int unixOpen( assert( (!isDelete && zName) || eType!=SQLITE_OPEN_MASTER_JOURNAL ); /* Assert that the upper layer has set one of the "file-type" flags. */ - assert( eType==SQLITE_OPEN_MAIN_DB || eType==SQLITE_OPEN_TEMP_DB - || eType==SQLITE_OPEN_MAIN_JOURNAL || eType==SQLITE_OPEN_TEMP_JOURNAL - || eType==SQLITE_OPEN_SUBJOURNAL || eType==SQLITE_OPEN_MASTER_JOURNAL + assert( eType==SQLITE_OPEN_MAIN_DB || eType==SQLITE_OPEN_TEMP_DB + || eType==SQLITE_OPEN_MAIN_JOURNAL || eType==SQLITE_OPEN_TEMP_JOURNAL + || eType==SQLITE_OPEN_SUBJOURNAL || eType==SQLITE_OPEN_MASTER_JOURNAL || eType==SQLITE_OPEN_TRANSIENT_DB ); @@ -25494,7 +25495,7 @@ static int unixOpen( /* Determine the value of the flags parameter passed to POSIX function ** open(). These must be calculated even if open() is not called, as - ** they may be stored as part of the file handle and used by the + ** they may be stored as part of the file handle and used by the ** 'conch file' locking functions later on. */ if( isReadonly ) openFlags |= O_RDONLY; if( isReadWrite ) openFlags |= O_RDWR; @@ -25566,7 +25567,7 @@ static int unixOpen( char *envforce = getenv("SQLITE_FORCE_PROXY_LOCKING"); int useProxy = 0; - /* SQLITE_FORCE_PROXY_LOCKING==1 means force always use proxy, 0 means + /* SQLITE_FORCE_PROXY_LOCKING==1 means force always use proxy, 0 means ** never use proxy, NULL means use proxy for non-local files only. */ if( envforce!=NULL ){ useProxy = atoi(envforce)>0; @@ -25599,7 +25600,7 @@ static int unixOpen( } } #endif - + rc = fillInUnixFile(pVfs, fd, dirfd, pFile, zPath, noLock, isDelete); open_finished: if( rc!=SQLITE_OK ){ @@ -25685,9 +25686,9 @@ static int unixAccess( /* ** Turn a relative pathname into a full pathname. The relative path ** is stored as a nul-terminated string in the buffer pointed to by -** zPath. +** zPath. ** -** zOut points to a buffer of at least sqlite3_vfs.mxPathname bytes +** zOut points to a buffer of at least sqlite3_vfs.mxPathname bytes ** (in this case, MAX_PATHNAME bytes). The full-path is written to ** this buffer before returning. */ @@ -25752,7 +25753,7 @@ static void unixDlError(sqlite3_vfs *NotUsed, int nBuf, char *zBufOut){ unixLeaveMutex(); } static void (*unixDlSym(sqlite3_vfs *NotUsed, void *p, const char*zSym))(void){ - /* + /* ** GCC with -pedantic-errors says that C90 does not allow a void* to be ** cast into a pointer to a function. And yet the library dlsym() routine ** returns a void* which is really a pointer to a function. So how do we @@ -25762,7 +25763,7 @@ static void (*unixDlSym(sqlite3_vfs *NotUsed, void *p, const char*zSym))(void){ ** parameters void* and const char* and returning a pointer to a function. ** We initialize x by assigning it a pointer to the dlsym() function. ** (That assignment requires a cast.) Then we call the function that - ** x points to. + ** x points to. ** ** This work-around is unlikely to work correctly on any system where ** you really cannot cast a function pointer into void*. But then, on the @@ -25940,7 +25941,7 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ ** address in the shared range is taken for a SHARED lock, the entire ** shared range is taken for an EXCLUSIVE lock): ** -** PENDING_BYTE 0x40000000 +** PENDING_BYTE 0x40000000 ** RESERVED_BYTE 0x40000001 ** SHARED_RANGE 0x40000002 -> 0x40000200 ** @@ -25958,7 +25959,7 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ ** To address the performance and cache coherency issues, proxy file locking ** changes the way database access is controlled by limiting access to a ** single host at a time and moving file locks off of the database file -** and onto a proxy file on the local file system. +** and onto a proxy file on the local file system. ** ** ** Using proxy locks @@ -25983,19 +25984,19 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ ** actual proxy file name is generated from the name and path of the ** database file. For example: ** -** For database path "/Users/me/foo.db" +** For database path "/Users/me/foo.db" ** The lock path will be "/sqliteplocks/_Users_me_foo.db:auto:") ** ** Once a lock proxy is configured for a database connection, it can not ** be removed, however it may be switched to a different proxy path via ** the above APIs (assuming the conch file is not being held by another -** connection or process). +** connection or process). ** ** ** How proxy locking works ** ----------------------- ** -** Proxy file locking relies primarily on two new supporting files: +** Proxy file locking relies primarily on two new supporting files: ** ** * conch file to limit access to the database file to a single host ** at a time @@ -26027,11 +26028,11 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ ** /Library/Caches/.com.apple.sqliteConchHostId). ** ** Requesting the lock proxy does not immediately take the conch, it is -** only taken when the first request to lock database file is made. +** only taken when the first request to lock database file is made. ** This matches the semantics of the traditional locking behavior, where ** opening a connection to a database file does not take a lock on it. -** The shared lock and an open file descriptor are maintained until -** the connection to the database is closed. +** The shared lock and an open file descriptor are maintained until +** the connection to the database is closed. ** ** The proxy file and the lock file are never deleted so they only need ** to be created the first time they are used. @@ -26045,7 +26046,7 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ ** automatically configured for proxy locking, lock files are ** named automatically using the same logic as ** PRAGMA lock_proxy_file=":auto:" -** +** ** SQLITE_PROXY_DEBUG ** ** Enables the logging of error messages during host id file @@ -26064,8 +26065,8 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ ** ** Permissions to use when creating a directory for storing the ** lock proxy files, only used when LOCKPROXYDIR is not set. -** -** +** +** ** As mentioned above, when compiled with SQLITE_PREFER_PROXY_LOCKING, ** setting the environment variable SQLITE_FORCE_PROXY_LOCKING to 1 will ** force proxy locking to be used for every database file opened, and 0 @@ -26075,7 +26076,7 @@ static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){ */ /* -** Proxy locking is only available on MacOSX +** Proxy locking is only available on MacOSX */ #if defined(__APPLE__) && SQLITE_ENABLE_LOCKING_STYLE @@ -26085,7 +26086,7 @@ SQLITE_API int sqlite3_hostid_num = 0; #endif /* -** The proxyLockingContext has the path and file structures for the remote +** The proxyLockingContext has the path and file structures for the remote ** and local proxy files in it */ typedef struct proxyLockingContext proxyLockingContext; @@ -26100,8 +26101,8 @@ struct proxyLockingContext { sqlite3_io_methods const *pOldMethod; /* Original I/O methods for close */ }; -/* HOSTIDLEN and CONCHLEN both include space for the string -** terminating nul +/* HOSTIDLEN and CONCHLEN both include space for the string +** terminating nul */ #define HOSTIDLEN 128 #define CONCHLEN (MAXPATHLEN+HOSTIDLEN+1) @@ -26114,7 +26115,7 @@ struct proxyLockingContext { static int proxyGenerateHostID(char *pHostID){ int pid, fd, len; unsigned char *key = (unsigned char *)pHostID; - + memset(key, 0, HOSTIDLEN); len = 0; fd = open("/dev/urandom", O_RDONLY); @@ -26129,7 +26130,7 @@ static int proxyGenerateHostID(char *pHostID){ pid = getpid(); memcpy(&key[sizeof(t)], &pid, sizeof(pid)); } - + #ifdef MAKE_PRETTY_HOSTID { int i; @@ -26149,7 +26150,7 @@ static int proxyGenerateHostID(char *pHostID){ } /* writes the host id path to path, path should be an pre-allocated buffer -** with enough space for a path +** with enough space for a path */ static void proxyGetHostIDPath(char *path, size_t len){ strlcpy(path, HOSTIDPATH, len); @@ -26163,12 +26164,12 @@ static void proxyGetHostIDPath(char *path, size_t len){ OSTRACE3("GETHOSTIDPATH %s pid=%d\n", path, getpid()); } -/* get the host ID from a sqlite hostid file stored in the -** user-specific tmp directory, create the ID if it's not there already +/* get the host ID from a sqlite hostid file stored in the +** user-specific tmp directory, create the ID if it's not there already */ static int proxyGetHostID(char *pHostID, int *pError){ int fd; - char path[MAXPATHLEN]; + char path[MAXPATHLEN]; size_t len; int rc=SQLITE_OK; @@ -26177,7 +26178,7 @@ static int proxyGetHostID(char *pHostID, int *pError){ fd = open(path, O_CREAT|O_WRONLY|O_EXCL, 0644); if( fd<0 ){ int err=errno; - + if( err!=EEXIST ){ #ifdef SQLITE_PROXY_DEBUG /* set the sqlite error message instead */ fprintf(stderr, "sqlite error creating host ID file %s: %s\n", @@ -26247,7 +26248,7 @@ static int proxyGetLockPath(const char *dbPath, char *lPath, size_t maxLen){ }else{ OSTRACE3("GETLOCKPATH mkdir %s pid=%d\n", lPath, getpid()); } - + } # else len = strlcpy(lPath, "/tmp/", maxLen); @@ -26257,7 +26258,7 @@ static int proxyGetLockPath(const char *dbPath, char *lPath, size_t maxLen){ if( lPath[len-1]!='/' ){ len = strlcat(lPath, "/", maxLen); } - + /* transform the db path to a unique cache name */ dbLen = (int)strlen(dbPath); for( i=0; ilockingContext; - + proxyLockingContext *pCtx = (proxyLockingContext *)pFile->lockingContext; + if( pCtx->conchHeld>0 ){ return SQLITE_OK; }else{ @@ -26356,7 +26357,7 @@ static int proxyTakeConch(unixFile *pFile){ if( rc!=SQLITE_OK ){ goto end_takeconch; } - + readRc = unixRead((sqlite3_file *)conchFile, conchValue, CONCHLEN, 0); if( readRc!=SQLITE_IOERR_SHORT_READ ){ if( readRc!=SQLITE_OK ){ @@ -26381,20 +26382,20 @@ static int proxyTakeConch(unixFile *pFile){ } } }else{ - /* a short read means we're "creating" the conch (even though it could + /* a short read means we're "creating" the conch (even though it could ** have been user-intervention), if we acquire the exclusive lock, ** we'll try to match the current on-disk permissions of the database */ syncPerms = 1; } - + /* either conch was emtpy or didn't match */ if( !pCtx->lockProxyPath ){ proxyGetLockPath(pCtx->dbPath, lockPath, MAXPATHLEN); tLockPath = lockPath; strlcpy(&testValue[HOSTIDLEN], lockPath, MAXPATHLEN); } - + /* update conch with host and path (this will fail if other process ** has a shared lock already) */ rc = conchFile->pMethod->xLock((sqlite3_file*)conchFile, EXCLUSIVE_LOCK); @@ -26417,14 +26418,14 @@ static int proxyTakeConch(unixFile *pFile){ } }else{ int code = errno; - fprintf(stderr, "STAT FAILED[%d] with %d %s\n", + fprintf(stderr, "STAT FAILED[%d] with %d %s\n", err, code, strerror(code)); #endif } } } conchFile->pMethod->xUnlock((sqlite3_file*)conchFile, SHARED_LOCK); - + end_takeconch: OSTRACE2("TRANSPROXY: CLOSE %d\n", pFile->h); if( rc==SQLITE_OK && pFile->openFlags ){ @@ -26483,7 +26484,7 @@ static int proxyReleaseConch(unixFile *pFile){ pCtx = (proxyLockingContext *)pFile->lockingContext; conchFile = pCtx->conchFile; OSTRACE4("RELEASECONCH %d for %s pid=%d\n", conchFile->h, - (pCtx->lockProxyPath ? pCtx->lockProxyPath : ":auto:"), + (pCtx->lockProxyPath ? pCtx->lockProxyPath : ":auto:"), getpid()); pCtx->conchHeld = 0; rc = conchFile->pMethod->xUnlock((sqlite3_file*)conchFile, NO_LOCK); @@ -26509,13 +26510,13 @@ static int proxyCreateConchPathname(char *dbPath, char **pConchPath){ char *conchPath; /* buffer in which to construct conch name */ /* Allocate space for the conch filename and initialize the name to - ** the name of the original database file. */ + ** the name of the original database file. */ *pConchPath = conchPath = (char *)sqlite3_malloc(len + 8); if( conchPath==0 ){ return SQLITE_NOMEM; } memcpy(conchPath, dbPath, len+1); - + /* now insert a "." before the last / character */ for( i=(len-1); i>=0; i-- ){ if( conchPath[i]=='/' ){ @@ -26538,7 +26539,7 @@ static int proxyCreateConchPathname(char *dbPath, char **pConchPath){ /* Takes a fully configured proxy locking-style unix file and switches -** the local lock file path +** the local lock file path */ static int switchLockProxyPath(unixFile *pFile, const char *path) { proxyLockingContext *pCtx = (proxyLockingContext*)pFile->lockingContext; @@ -26547,7 +26548,7 @@ static int switchLockProxyPath(unixFile *pFile, const char *path) { if( pFile->locktype!=NO_LOCK ){ return SQLITE_BUSY; - } + } /* nothing to do if the path is NULL, :auto: or matches the existing path */ if( !path || path[0]=='\0' || !strcmp(path, ":auto:") || @@ -26565,7 +26566,7 @@ static int switchLockProxyPath(unixFile *pFile, const char *path) { sqlite3_free(oldPath); pCtx->lockProxyPath = sqlite3DbStrDup(0, path); } - + return rc; } @@ -26579,7 +26580,7 @@ static int switchLockProxyPath(unixFile *pFile, const char *path) { static int proxyGetDbPathForUnixFile(unixFile *pFile, char *dbPath){ #if defined(__APPLE__) if( pFile->pMethod == &afpIoMethods ){ - /* afp style keeps a reference to the db path in the filePath field + /* afp style keeps a reference to the db path in the filePath field ** of the struct */ assert( (int)strlen((char*)pFile->lockingContext)<=MAXPATHLEN ); strcpy(dbPath, ((afpLockingContext *)pFile->lockingContext)->dbPath); @@ -26599,9 +26600,9 @@ static int proxyGetDbPathForUnixFile(unixFile *pFile, char *dbPath){ } /* -** Takes an already filled in unix file and alters it so all file locking +** Takes an already filled in unix file and alters it so all file locking ** will be performed on the local proxy lock file. The following fields -** are preserved in the locking context so that they can be restored and +** are preserved in the locking context so that they can be restored and ** the unix structure properly cleaned up at close time: ** ->lockingContext ** ->pMethod @@ -26611,7 +26612,7 @@ static int proxyTransformUnixFile(unixFile *pFile, const char *path) { char dbPath[MAXPATHLEN+1]; /* Name of the database file */ char *lockPath=NULL; int rc = SQLITE_OK; - + if( pFile->locktype!=NO_LOCK ){ return SQLITE_BUSY; } @@ -26621,7 +26622,7 @@ static int proxyTransformUnixFile(unixFile *pFile, const char *path) { }else{ lockPath=(char *)path; } - + OSTRACE4("TRANSPROXY %d for %s pid=%d\n", pFile->h, (lockPath ? lockPath : ":auto:"), getpid()); @@ -26634,13 +26635,13 @@ static int proxyTransformUnixFile(unixFile *pFile, const char *path) { rc = proxyCreateConchPathname(dbPath, &pCtx->conchFilePath); if( rc==SQLITE_OK ){ rc = proxyCreateUnixFile(pCtx->conchFilePath, &pCtx->conchFile); - } + } if( rc==SQLITE_OK && lockPath ){ pCtx->lockProxyPath = sqlite3DbStrDup(0, lockPath); } if( rc==SQLITE_OK ){ - /* all memory is allocated, proxys are created and assigned, + /* all memory is allocated, proxys are created and assigned, ** switch the locking context and pMethod then return. */ pCtx->dbPath = sqlite3DbStrDup(0, dbPath); @@ -26649,12 +26650,12 @@ static int proxyTransformUnixFile(unixFile *pFile, const char *path) { pCtx->pOldMethod = pFile->pMethod; pFile->pMethod = &proxyIoMethods; }else{ - if( pCtx->conchFile ){ + if( pCtx->conchFile ){ rc = pCtx->conchFile->pMethod->xClose((sqlite3_file *)pCtx->conchFile); if( rc ) return rc; sqlite3_free(pCtx->conchFile); } - sqlite3_free(pCtx->conchFilePath); + sqlite3_free(pCtx->conchFilePath); sqlite3_free(pCtx); } OSTRACE3("TRANSPROXY %d %s\n", pFile->h, @@ -26699,9 +26700,9 @@ static int proxyFileControl(sqlite3_file *id, int op, void *pArg){ }else{ const char *proxyPath = (const char *)pArg; if( isProxyStyle ){ - proxyLockingContext *pCtx = + proxyLockingContext *pCtx = (proxyLockingContext*)pFile->lockingContext; - if( !strcmp(pArg, ":auto:") + if( !strcmp(pArg, ":auto:") || (pCtx->lockProxyPath && !strncmp(pCtx->lockProxyPath, proxyPath, MAXPATHLEN)) ){ @@ -26814,7 +26815,7 @@ static int proxyClose(sqlite3_file *id) { unixFile *lockProxy = pCtx->lockProxy; unixFile *conchFile = pCtx->conchFile; int rc = SQLITE_OK; - + if( lockProxy ){ rc = lockProxy->pMethod->xUnlock((sqlite3_file*)lockProxy, NO_LOCK); if( rc ) return rc; @@ -26851,7 +26852,7 @@ static int proxyClose(sqlite3_file *id) { ** The proxy locking style is intended for use with AFP filesystems. ** And since AFP is only supported on MacOSX, the proxy locking is also ** restricted to MacOSX. -** +** ** ******************* End of the proxy lock implementation ********************** ******************************************************************************/ @@ -26869,8 +26870,8 @@ static int proxyClose(sqlite3_file *id) { ** necessarily been initialized when this routine is called, and so they ** should not be used. */ -SQLITE_API int sqlite3_os_init(void){ - /* +SQLITE_API int sqlite3_os_init(void){ + /* ** The following macro defines an initializer for an sqlite3_vfs object. ** The name of the VFS is NAME. The pAppData is a pointer to a pointer ** to the "finder" function. (pAppData is a pointer to a pointer because @@ -26886,7 +26887,7 @@ SQLITE_API int sqlite3_os_init(void){ ** ** Most finders simply return a pointer to a fixed sqlite3_io_methods ** object. But the "autolockIoFinder" available on MacOSX does a little - ** more than that; it looks at the filesystem type that hosts the + ** more than that; it looks at the filesystem type that hosts the ** database file and tries to choose an locking method appropriate for ** that filesystem time. */ @@ -26947,7 +26948,7 @@ SQLITE_API int sqlite3_os_init(void){ for(i=0; i<(sizeof(aVfs)/sizeof(sqlite3_vfs)); i++){ sqlite3_vfs_register(&aVfs[i], i==0); } - return SQLITE_OK; + return SQLITE_OK; } /* @@ -26957,10 +26958,10 @@ SQLITE_API int sqlite3_os_init(void){ ** to release dynamically allocated objects. But not on unix. ** This routine is a no-op for unix. */ -SQLITE_API int sqlite3_os_end(void){ - return SQLITE_OK; +SQLITE_API int sqlite3_os_end(void){ + return SQLITE_OK; } - + #endif /* SQLITE_OS_UNIX */ /************** End of os_unix.c *********************************************/ @@ -27044,8 +27045,6 @@ SQLITE_API int sqlite3_os_end(void){ ** ** This file should be #included by the os_*.c files only. It is not a ** general purpose header file. -** -** $Id: os_common.h,v 1.38 2009/02/24 18:40:50 danielk1977 Exp $ */ #ifndef _OS_COMMON_H_ #define _OS_COMMON_H_ @@ -27086,8 +27085,8 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; */ #ifdef SQLITE_PERFORMANCE_TRACE -/* -** hwtime.h contains inline assembler code for implementing +/* +** hwtime.h contains inline assembler code for implementing ** high-performance timing routines. */ /************** Include hwtime.h in the middle of os_common.h ****************/ @@ -27106,8 +27105,6 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; ** ** This file contains inline asm code for retrieving "high-performance" ** counters for x86 class CPUs. -** -** $Id: hwtime.h,v 1.3 2008/08/01 14:33:15 shane Exp $ */ #ifndef _HWTIME_H_ #define _HWTIME_H_ @@ -27147,7 +27144,7 @@ SQLITE_PRIVATE int sqlite3OSTrace = 0; __asm__ __volatile__ ("rdtsc" : "=A" (val)); return val; } - + #elif (defined(__GNUC__) && defined(__ppc__)) __inline__ sqlite_uint64 sqlite3Hwtime(void){ @@ -27253,7 +27250,7 @@ SQLITE_API int sqlite3_open_file_count = 0; ** Some microsoft compilers lack this definition. */ #ifndef INVALID_FILE_ATTRIBUTES -# define INVALID_FILE_ATTRIBUTES ((DWORD)-1) +# define INVALID_FILE_ATTRIBUTES ((DWORD)-1) #endif /* @@ -27292,7 +27289,7 @@ struct winFile { DWORD sectorSize; /* Sector size of the device file is on */ #if SQLITE_OS_WINCE WCHAR *zDeleteOnClose; /* Name of file to delete when closing */ - HANDLE hMutex; /* Mutex used to control access to shared lock */ + HANDLE hMutex; /* Mutex used to control access to shared lock */ HANDLE hShared; /* Shared memory segment used for locking */ winceLock local; /* Locks obtained by this instance of winFile */ winceLock *shared; /* Global shared lock memory for the file */ @@ -27351,7 +27348,7 @@ static int sqlite3_os_type = 0; #endif /* SQLITE_OS_WINCE */ /* -** Convert a UTF-8 string to microsoft unicode (UTF-16?). +** Convert a UTF-8 string to microsoft unicode (UTF-16?). ** ** Space to hold the returned string is obtained from malloc. */ @@ -27397,7 +27394,7 @@ static char *unicodeToUtf8(const WCHAR *zWideFilename){ /* ** Convert an ansi string to microsoft unicode, based on the ** current codepage settings for file apis. -** +** ** Space to hold the returned string is obtained ** from malloc. */ @@ -27463,7 +27460,7 @@ SQLITE_API char *sqlite3_win32_mbcs_to_utf8(const char *zFilename){ } /* -** Convert UTF-8 to multibyte character string. Space to hold the +** Convert UTF-8 to multibyte character string. Space to hold the ** returned string is obtained from malloc(). */ static char *utf8ToMbcs(const char *zFilename){ @@ -27561,17 +27558,17 @@ static BOOL winceCreateLock(const char *zFilename, winFile *pFile){ /* Acquire the mutex before continuing */ winceMutexAcquire(pFile->hMutex); - - /* Since the names of named mutexes, semaphores, file mappings etc are + + /* Since the names of named mutexes, semaphores, file mappings etc are ** case-sensitive, take advantage of that by uppercasing the mutex name ** and using that as the shared filemapping name. */ CharUpperW(zName); pFile->hShared = CreateFileMappingW(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, sizeof(winceLock), - zName); + zName); - /* Set a flag that indicates we're the first to create the memory so it + /* Set a flag that indicates we're the first to create the memory so it ** must be zero-initialized */ if (GetLastError() == ERROR_ALREADY_EXISTS){ bInit = FALSE; @@ -27581,7 +27578,7 @@ static BOOL winceCreateLock(const char *zFilename, winFile *pFile){ /* If we succeeded in making the shared memory handle, map it. */ if (pFile->hShared){ - pFile->shared = (winceLock*)MapViewOfFile(pFile->hShared, + pFile->shared = (winceLock*)MapViewOfFile(pFile->hShared, FILE_MAP_READ|FILE_MAP_WRITE, 0, 0, sizeof(winceLock)); /* If mapping failed, close the shared memory handle and erase it */ if (!pFile->shared){ @@ -27598,7 +27595,7 @@ static BOOL winceCreateLock(const char *zFilename, winFile *pFile){ pFile->hMutex = NULL; return FALSE; } - + /* Initialize the shared memory if we're supposed to */ if (bInit) { ZeroMemory(pFile->shared, sizeof(winceLock)); @@ -27636,13 +27633,13 @@ static void winceDestroyLock(winFile *pFile){ CloseHandle(pFile->hShared); /* Done with the mutex */ - winceMutexRelease(pFile->hMutex); + winceMutexRelease(pFile->hMutex); CloseHandle(pFile->hMutex); pFile->hMutex = NULL; } } -/* +/* ** An implementation of the LockFile() API of windows for wince */ static BOOL winceLockFile( @@ -27828,7 +27825,7 @@ static int winClose(sqlite3_file *id){ int cnt = 0; while( DeleteFileW(pFile->zDeleteOnClose)==0 - && GetFileAttributesW(pFile->zDeleteOnClose)!=0xffffffff + && GetFileAttributesW(pFile->zDeleteOnClose)!=0xffffffff && cnt++ < WINCE_DELETION_ATTEMPTS ){ Sleep(100); /* Wait a little before trying again */ @@ -28041,7 +28038,7 @@ static int getReadLock(winFile *pFile){ ovlp.hEvent = 0; res = LockFileEx(pFile->h, LOCKFILE_FAIL_IMMEDIATELY, 0, SHARED_SIZE, 0, &ovlp); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. */ #if SQLITE_OS_WINCE==0 }else{ @@ -28064,7 +28061,7 @@ static int unlockReadLock(winFile *pFile){ int res; if( isNT() ){ res = UnlockFile(pFile->h, SHARED_FIRST, 0, SHARED_SIZE, 0); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. */ #if SQLITE_OS_WINCE==0 }else{ @@ -28363,7 +28360,7 @@ static void *convertUtf8Filename(const char *zFilename){ void *zConverted = 0; if( isNT() ){ zConverted = utf8ToUnicode(zFilename); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. */ #if SQLITE_OS_WINCE==0 }else{ @@ -28398,7 +28395,7 @@ static int getTempname(int nBuf, char *zBuf){ }else{ return SQLITE_NOMEM; } -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28427,7 +28424,7 @@ static int getTempname(int nBuf, char *zBuf){ } zBuf[j] = 0; OSTRACE2("TEMP FILENAME: %s\n", zBuf); - return SQLITE_OK; + return SQLITE_OK; } /* @@ -28459,7 +28456,7 @@ static int getLastErrorMsg(int nBuf, char *zBuf){ /* free the system buffer allocated by FormatMessage */ LocalFree(zTempWide); } -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28518,8 +28515,8 @@ static int winOpen( assert( id!=0 ); UNUSED_PARAMETER(pVfs); - /* If the second argument to this function is NULL, generate a - ** temporary file name to use + /* If the second argument to this function is NULL, generate a + ** temporary file name to use */ if( !zUtf8Name ){ int rc = getTempname(MAX_PATH+1, zTmpname); @@ -28540,8 +28537,8 @@ static int winOpen( }else{ dwDesiredAccess = GENERIC_READ; } - /* SQLITE_OPEN_EXCLUSIVE is used to make sure that a new file is - ** created. SQLite doesn't use it to indicate "exclusive access" + /* SQLITE_OPEN_EXCLUSIVE is used to make sure that a new file is + ** created. SQLite doesn't use it to indicate "exclusive access" ** as it is usually understood. */ assert(!(flags & SQLITE_OPEN_EXCLUSIVE) || (flags & SQLITE_OPEN_CREATE)); @@ -28583,7 +28580,7 @@ static int winOpen( dwFlagsAndAttributes, NULL ); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28602,7 +28599,7 @@ static int winOpen( if( h==INVALID_HANDLE_VALUE ){ free(zConverted); if( flags & SQLITE_OPEN_READWRITE ){ - return winOpen(pVfs, zName, id, + return winOpen(pVfs, zName, id, ((flags|SQLITE_OPEN_READONLY)&~SQLITE_OPEN_READWRITE), pOutFlags); }else{ return SQLITE_CANTOPEN; @@ -28675,7 +28672,7 @@ static int winDelete( || ((error = GetLastError()) == ERROR_ACCESS_DENIED)) && (++cnt < MX_DELETION_ATTEMPTS) && (Sleep(100), 1) ); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28691,7 +28688,7 @@ static int winDelete( } free(zConverted); OSTRACE2("DELETE \"%s\"\n", zFilename); - return ( (rc == INVALID_FILE_ATTRIBUTES) + return ( (rc == INVALID_FILE_ATTRIBUTES) && (error == ERROR_FILE_NOT_FOUND)) ? SQLITE_OK : SQLITE_IOERR_DELETE; } @@ -28713,7 +28710,7 @@ static int winAccess( } if( isNT() ){ attr = GetFileAttributesW((WCHAR*)zConverted); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28750,7 +28747,7 @@ static int winFullPathname( int nFull, /* Size of output buffer in bytes */ char *zFull /* Output buffer */ ){ - + #if defined(__CYGWIN__) UNUSED_PARAMETER(nFull); cygwin_conv_to_full_win32_path(zRelative, zFull); @@ -28782,7 +28779,7 @@ static int winFullPathname( free(zConverted); zOut = unicodeToUtf8(zTemp); free(zTemp); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28876,7 +28873,7 @@ static int getSectorSize( } } #endif - return (int) bytesPerSector; + return (int) bytesPerSector; } #ifndef SQLITE_OMIT_LOAD_EXTENSION @@ -28897,7 +28894,7 @@ static void *winDlOpen(sqlite3_vfs *pVfs, const char *zFilename){ } if( isNT() ){ h = LoadLibraryW((WCHAR*)zConverted); -/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. +/* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed. ** Since the ASCII version of these Windows API do not exist for WINCE, ** it's important to not reference them for WINCE builds. */ @@ -28997,22 +28994,22 @@ SQLITE_API int sqlite3_current_time = 0; */ int winCurrentTime(sqlite3_vfs *pVfs, double *prNow){ FILETIME ft; - /* FILETIME structure is a 64-bit value representing the number of - 100-nanosecond intervals since January 1, 1601 (= JD 2305813.5). + /* FILETIME structure is a 64-bit value representing the number of + 100-nanosecond intervals since January 1, 1601 (= JD 2305813.5). */ sqlite3_int64 timeW; /* Whole days */ sqlite3_int64 timeF; /* Fractional Days */ /* Number of 100-nanosecond intervals in a single day */ - static const sqlite3_int64 ntuPerDay = + static const sqlite3_int64 ntuPerDay = 10000000*(sqlite3_int64)86400; /* Number of 100-nanosecond intervals in half of a day */ - static const sqlite3_int64 ntuPerHalfDay = + static const sqlite3_int64 ntuPerHalfDay = 10000000*(sqlite3_int64)43200; /* 2^32 - to avoid use of LL and warnings in gcc */ - static const sqlite3_int64 max32BitValue = + static const sqlite3_int64 max32BitValue = (sqlite3_int64)2000000000 + (sqlite3_int64)2000000000 + (sqlite3_int64)294967296; #if SQLITE_OS_WINCE @@ -29088,7 +29085,7 @@ SQLITE_API int sqlite3_os_init(void){ 0, /* pNext */ "win32", /* zName */ 0, /* pAppData */ - + winOpen, /* xOpen */ winDelete, /* xDelete */ winAccess, /* xAccess */ @@ -29104,9 +29101,9 @@ SQLITE_API int sqlite3_os_init(void){ }; sqlite3_vfs_register(&winVfs, 1); - return SQLITE_OK; + return SQLITE_OK; } -SQLITE_API int sqlite3_os_end(void){ +SQLITE_API int sqlite3_os_end(void){ return SQLITE_OK; } @@ -29133,8 +29130,8 @@ SQLITE_API int sqlite3_os_end(void){ ** property. Usually only a few pages are meet either condition. ** So the bitmap is usually sparse and has low cardinality. ** But sometimes (for example when during a DROP of a large table) most -** or all of the pages in a database can get journalled. In those cases, -** the bitmap becomes dense with high cardinality. The algorithm needs +** or all of the pages in a database can get journalled. In those cases, +** the bitmap becomes dense with high cardinality. The algorithm needs ** to handle both cases well. ** ** The size of the bitmap is fixed when the object is created. @@ -29149,19 +29146,17 @@ SQLITE_API int sqlite3_os_end(void){ ** Bitvec object is the number of pages in the database file at the ** start of a transaction, and is thus usually less than a few thousand, ** but can be as large as 2 billion for a really big database. -** -** @(#) $Id: bitvec.c,v 1.17 2009/07/25 17:33:26 drh Exp $ */ /* Size of the Bitvec structure in bytes. */ #define BITVEC_SZ (sizeof(void*)*128) /* 512 on 32bit. 1024 on 64bit */ -/* Round the union size down to the nearest pointer boundary, since that's how +/* Round the union size down to the nearest pointer boundary, since that's how ** it will be aligned within the Bitvec struct. */ #define BITVEC_USIZE (((BITVEC_SZ-(3*sizeof(u32)))/sizeof(Bitvec*))*sizeof(Bitvec*)) -/* Type of the array "element" for the bitmap representation. -** Should be a power of 2, and ideally, evenly divide into BITVEC_USIZE. +/* Type of the array "element" for the bitmap representation. +** Should be a power of 2, and ideally, evenly divide into BITVEC_USIZE. ** Setting this to the "natural word" size of your CPU may improve ** performance. */ #define BITVEC_TELEM u8 @@ -29174,12 +29169,12 @@ SQLITE_API int sqlite3_os_end(void){ /* Number of u32 values in hash table. */ #define BITVEC_NINT (BITVEC_USIZE/sizeof(u32)) -/* Maximum number of entries in hash table before +/* Maximum number of entries in hash table before ** sub-dividing and re-hashing. */ #define BITVEC_MXHASH (BITVEC_NINT/2) /* Hashing function for the aHash representation. -** Empirical testing showed that the *37 multiplier -** (an arbitrary prime)in the hash function provided +** Empirical testing showed that the *37 multiplier +** (an arbitrary prime)in the hash function provided ** no fewer collisions than the no-op *1. */ #define BITVEC_HASH(X) (((X)*1)%BITVEC_NINT) @@ -29225,7 +29220,7 @@ struct Bitvec { /* ** Create a new bitmap object able to handle bits between 0 and iSize, -** inclusive. Return a pointer to the new object. Return NULL if +** inclusive. Return a pointer to the new object. Return NULL if ** malloc fails. */ SQLITE_PRIVATE Bitvec *sqlite3BitvecCreate(u32 iSize){ @@ -29479,7 +29474,7 @@ SQLITE_PRIVATE int sqlite3BitvecBuiltinTest(int sz, int *aOp){ break; } case 3: - case 4: + case 4: default: { nx = 2; sqlite3_randomness(sizeof(i), &i); @@ -29538,8 +29533,6 @@ bitvec_end: ** ************************************************************************* ** This file implements that page cache. -** -** @(#) $Id: pcache.c,v 1.47 2009/07/25 11:46:49 danielk1977 Exp $ */ /* @@ -29666,7 +29659,7 @@ static void pcacheUnpin(PgHdr *p){ /*************************************************** General Interfaces ****** ** -** Initialize and shutdown the page cache subsystem. Neither of these +** Initialize and shutdown the page cache subsystem. Neither of these ** functions are threadsafe. */ SQLITE_PRIVATE int sqlite3PcacheInitialize(void){ @@ -29688,8 +29681,8 @@ SQLITE_PRIVATE int sqlite3PcacheSize(void){ return sizeof(PCache); } /* ** Create a new PCache object. Storage space to hold the object -** has already been allocated and is passed in as the p pointer. -** The caller discovers how much space needs to be allocated by +** has already been allocated and is passed in as the p pointer. +** The caller discovers how much space needs to be allocated by ** calling sqlite3PcacheSize(). */ SQLITE_PRIVATE void sqlite3PcacheOpen( @@ -29761,14 +29754,14 @@ SQLITE_PRIVATE int sqlite3PcacheFetch( if( !pPage && eCreate==1 ){ PgHdr *pPg; - /* Find a dirty page to write-out and recycle. First try to find a + /* Find a dirty page to write-out and recycle. First try to find a ** page that does not require a journal-sync (one with PGHDR_NEED_SYNC - ** cleared), but if that is not possible settle for any other + ** cleared), but if that is not possible settle for any other ** unreferenced dirty page. */ expensive_assert( pcacheCheckSynced(pCache) ); - for(pPg=pCache->pSynced; - pPg && (pPg->nRef || (pPg->flags&PGHDR_NEED_SYNC)); + for(pPg=pCache->pSynced; + pPg && (pPg->nRef || (pPg->flags&PGHDR_NEED_SYNC)); pPg=pPg->pDirtyPrev ); if( !pPg ){ @@ -29905,7 +29898,7 @@ SQLITE_PRIVATE void sqlite3PcacheClearSyncFlags(PCache *pCache){ } /* -** Change the page number of page p to newPgno. +** Change the page number of page p to newPgno. */ SQLITE_PRIVATE void sqlite3PcacheMove(PgHdr *p, Pgno newPgno){ PCache *pCache = p->pCache; @@ -29956,7 +29949,7 @@ SQLITE_PRIVATE void sqlite3PcacheClose(PCache *pCache){ } } -/* +/* ** Discard the contents of the cache. */ SQLITE_PRIVATE void sqlite3PcacheClear(PCache *pCache){ @@ -30044,7 +30037,7 @@ SQLITE_PRIVATE PgHdr *sqlite3PcacheDirtyList(PCache *pCache){ return pcacheSortDirtyList(pCache->pDirty); } -/* +/* ** Return the total number of referenced pages held by the cache. */ SQLITE_PRIVATE int sqlite3PcacheRefCount(PCache *pCache){ @@ -30058,7 +30051,7 @@ SQLITE_PRIVATE int sqlite3PcachePageRefcount(PgHdr *p){ return p->nRef; } -/* +/* ** Return the total number of pages in the cache. */ SQLITE_PRIVATE int sqlite3PcachePagecount(PCache *pCache){ @@ -30121,8 +30114,6 @@ SQLITE_PRIVATE void sqlite3PcacheIterateDirty(PCache *pCache, void (*xIter)(PgHd ** of the SQLITE_CONFIG_PAGECACHE and sqlite3_release_memory() features. ** If the default page cache implementation is overriden, then neither of ** these two features are available. -** -** @(#) $Id: pcache1.c,v 1.19 2009/07/17 11:44:07 drh Exp $ */ @@ -30130,12 +30121,12 @@ typedef struct PCache1 PCache1; typedef struct PgHdr1 PgHdr1; typedef struct PgFreeslot PgFreeslot; -/* Pointers to structures of this type are cast and returned as +/* Pointers to structures of this type are cast and returned as ** opaque sqlite3_pcache* handles */ struct PCache1 { /* Cache configuration parameters. Page size (szPage) and the purgeable - ** flag (bPurgeable) are set when the cache is created. nMax may be + ** flag (bPurgeable) are set when the cache is created. nMax may be ** modified at any time by a call to the pcache1CacheSize() method. ** The global mutex must be held when accessing nMax. */ @@ -30145,7 +30136,7 @@ struct PCache1 { unsigned int nMax; /* Configured "cache_size" value */ /* Hash table of all pages. The following variables may only be accessed - ** when the accessor is holding the global mutex (see pcache1EnterMutex() + ** when the accessor is holding the global mutex (see pcache1EnterMutex() ** and pcache1LeaveMutex()). */ unsigned int nRecyclable; /* Number of pages in the LRU list */ @@ -30157,9 +30148,9 @@ struct PCache1 { }; /* -** Each cache entry is represented by an instance of the following -** structure. A buffer of PgHdr1.pCache->szPage bytes is allocated -** directly before this structure in memory (see the PGHDR1_TO_PAGE() +** Each cache entry is represented by an instance of the following +** structure. A buffer of PgHdr1.pCache->szPage bytes is allocated +** directly before this structure in memory (see the PGHDR1_TO_PAGE() ** macro below). */ struct PgHdr1 { @@ -30228,7 +30219,7 @@ static SQLITE_WSD struct PCacheGlobal { /******** Page Allocation/SQLITE_CONFIG_PCACHE Related Functions **************/ /* -** This function is called during initialization if a static buffer is +** This function is called during initialization if a static buffer is ** supplied to use for the page-cache by passing the SQLITE_CONFIG_PAGECACHE ** verb to sqlite3_config(). Parameter pBuf points to an allocation large ** enough to contain 'n' buffers of 'sz' bytes each. @@ -30252,8 +30243,8 @@ SQLITE_PRIVATE void sqlite3PCacheBufferSetup(void *pBuf, int sz, int n){ /* ** Malloc function used within this file to allocate space from the buffer -** configured using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no -** such buffer exists or there is no space left in it, this function falls +** configured using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no +** such buffer exists or there is no space left in it, this function falls ** back to sqlite3Malloc(). */ static void *pcache1Alloc(int nByte){ @@ -30268,8 +30259,8 @@ static void *pcache1Alloc(int nByte){ }else{ /* Allocate a new buffer using sqlite3Malloc. Before doing so, exit the - ** global pcache mutex and unlock the pager-cache object pCache. This is - ** so that if the attempt to allocate a new buffer causes the the + ** global pcache mutex and unlock the pager-cache object pCache. This is + ** so that if the attempt to allocate a new buffer causes the the ** configured soft-heap-limit to be breached, it will be possible to ** reclaim memory from this pager-cache. */ @@ -30406,7 +30397,7 @@ static int pcache1ResizeHash(PCache1 *p){ } /* -** This function is used internally to remove the page pPage from the +** This function is used internally to remove the page pPage from the ** global LRU list, if is part of it. If pPage is not part of the global ** LRU list, then this function is a no-op. ** @@ -30435,7 +30426,7 @@ static void pcache1PinPage(PgHdr1 *pPage){ /* -** Remove the page supplied as an argument from the hash table +** Remove the page supplied as an argument from the hash table ** (PCache1.apHash structure) that it is currently stored in. ** ** The global mutex must be held when this function is called. @@ -30467,21 +30458,21 @@ static void pcache1EnforceMaxPage(void){ } /* -** Discard all pages from cache pCache with a page number (key value) -** greater than or equal to iLimit. Any pinned pages that meet this +** Discard all pages from cache pCache with a page number (key value) +** greater than or equal to iLimit. Any pinned pages that meet this ** criteria are unpinned before they are discarded. ** ** The global mutex must be held when this function is called. */ static void pcache1TruncateUnsafe( - PCache1 *pCache, - unsigned int iLimit + PCache1 *pCache, + unsigned int iLimit ){ TESTONLY( unsigned int nPage = 0; ) /* Used to assert pCache->nPage is correct */ unsigned int h; assert( sqlite3_mutex_held(pcache1.mutex) ); for(h=0; hnHash; h++){ - PgHdr1 **pp = &pCache->apHash[h]; + PgHdr1 **pp = &pCache->apHash[h]; PgHdr1 *pPage; while( (pPage = *pp)!=0 ){ if( pPage->iKey>=iLimit ){ @@ -30517,7 +30508,7 @@ static int pcache1Init(void *NotUsed){ /* ** Implementation of the sqlite3_pcache.xShutdown method. -** Note that the static mutex allocated in xInit does +** Note that the static mutex allocated in xInit does ** not need to be freed. */ static void pcache1Shutdown(void *NotUsed){ @@ -30550,7 +30541,7 @@ static sqlite3_pcache *pcache1Create(int szPage, int bPurgeable){ } /* -** Implementation of the sqlite3_pcache.xCachesize method. +** Implementation of the sqlite3_pcache.xCachesize method. ** ** Configure the cache_size limit for a cache. */ @@ -30566,7 +30557,7 @@ static void pcache1Cachesize(sqlite3_pcache *p, int nMax){ } /* -** Implementation of the sqlite3_pcache.xPagecount method. +** Implementation of the sqlite3_pcache.xPagecount method. */ static int pcache1Pagecount(sqlite3_pcache *p){ int n; @@ -30577,13 +30568,13 @@ static int pcache1Pagecount(sqlite3_pcache *p){ } /* -** Implementation of the sqlite3_pcache.xFetch method. +** Implementation of the sqlite3_pcache.xFetch method. ** ** Fetch a page by key value. ** ** Whether or not a new page may be allocated by this function depends on ** the value of the createFlag argument. 0 means do not allocate a new -** page. 1 means allocate a new page if space is easily available. 2 +** page. 1 means allocate a new page if space is easily available. 2 ** means to try really hard to allocate a new page. ** ** For a non-purgeable cache (a cache used as the storage for an in-memory @@ -30594,7 +30585,7 @@ static int pcache1Pagecount(sqlite3_pcache *p){ ** There are three different approaches to obtaining space for a page, ** depending on the value of parameter createFlag (which may be 0, 1 or 2). ** -** 1. Regardless of the value of createFlag, the cache is searched for a +** 1. Regardless of the value of createFlag, the cache is searched for a ** copy of the requested page. If one is found, it is returned. ** ** 2. If createFlag==0 and the page is not already in the cache, NULL is @@ -30606,13 +30597,13 @@ static int pcache1Pagecount(sqlite3_pcache *p){ ** (a) the number of pages pinned by the cache is greater than ** PCache1.nMax, or ** (b) the number of pages pinned by the cache is greater than -** the sum of nMax for all purgeable caches, less the sum of -** nMin for all other purgeable caches. +** the sum of nMax for all purgeable caches, less the sum of +** nMin for all other purgeable caches. ** ** 4. If none of the first three conditions apply and the cache is marked ** as purgeable, and if one of the following is true: ** -** (a) The number of pages allocated for the cache is already +** (a) The number of pages allocated for the cache is already ** PCache1.nMax, or ** ** (b) The number of pages allocated for all purgeable caches is @@ -30621,7 +30612,7 @@ static int pcache1Pagecount(sqlite3_pcache *p){ ** ** then attempt to recycle a page from the LRU list. If it is the right ** size, return the recycled buffer. Otherwise, free the buffer and -** proceed to step 5. +** proceed to step 5. ** ** 5. Otherwise, allocate and return a new page buffer. */ @@ -30673,8 +30664,8 @@ static void *pcache1Fetch(sqlite3_pcache *p, unsigned int iKey, int createFlag){ } } - /* Step 5. If a usable page buffer has still not been found, - ** attempt to allocate a new one. + /* Step 5. If a usable page buffer has still not been found, + ** attempt to allocate a new one. */ if( !pPage ){ pPage = pcache1AllocPage(pCache); @@ -30710,11 +30701,11 @@ fetch_out: static void pcache1Unpin(sqlite3_pcache *p, void *pPg, int reuseUnlikely){ PCache1 *pCache = (PCache1 *)p; PgHdr1 *pPage = PAGE_TO_PGHDR1(pCache, pPg); - + assert( pPage->pCache==pCache ); pcache1EnterMutex(); - /* It is an error to call this function if the page is already + /* It is an error to call this function if the page is already ** part of the global LRU list. */ assert( pPage->pLruPrev==0 && pPage->pLruNext==0 ); @@ -30725,7 +30716,7 @@ static void pcache1Unpin(sqlite3_pcache *p, void *pPg, int reuseUnlikely){ pcache1FreePage(pPage); }else{ /* Add the page to the global LRU list. Normally, the page is added to - ** the head of the list (last page to be recycled). However, if the + ** the head of the list (last page to be recycled). However, if the ** reuseUnlikely flag passed to this function is true, the page is added ** to the tail of the list (first page to be recycled). */ @@ -30744,7 +30735,7 @@ static void pcache1Unpin(sqlite3_pcache *p, void *pPg, int reuseUnlikely){ } /* -** Implementation of the sqlite3_pcache.xRekey method. +** Implementation of the sqlite3_pcache.xRekey method. */ static void pcache1Rekey( sqlite3_pcache *p, @@ -30755,7 +30746,7 @@ static void pcache1Rekey( PCache1 *pCache = (PCache1 *)p; PgHdr1 *pPage = PAGE_TO_PGHDR1(pCache, pPg); PgHdr1 **pp; - unsigned int h; + unsigned int h; assert( pPage->iKey==iOld ); assert( pPage->pCache==pCache ); @@ -30772,15 +30763,7 @@ static void pcache1Rekey( pPage->iKey = iNew; pPage->pNext = pCache->apHash[h]; pCache->apHash[h] = pPage; - - /* The xRekey() interface is only used to move pages earlier in the - ** database file (in order to move all free pages to the end of the - ** file where they can be truncated off.) Hence, it is not possible - ** for the new page number to be greater than the largest previously - ** fetched page. But we retain the following test in case xRekey() - ** begins to be used in different ways in the future. - */ - if( NEVER(iNew>pCache->iMaxKey) ){ + if( iNew>pCache->iMaxKey ){ pCache->iMaxKey = iNew; } @@ -30788,7 +30771,7 @@ static void pcache1Rekey( } /* -** Implementation of the sqlite3_pcache.xTruncate method. +** Implementation of the sqlite3_pcache.xTruncate method. ** ** Discard all unpinned pages in the cache with a page number equal to ** or greater than parameter iLimit. Any pinned pages with a page number @@ -30805,7 +30788,7 @@ static void pcache1Truncate(sqlite3_pcache *p, unsigned int iLimit){ } /* -** Implementation of the sqlite3_pcache.xDestroy method. +** Implementation of the sqlite3_pcache.xDestroy method. ** ** Destroy a cache allocated using pcache1Create(). */ @@ -30850,7 +30833,7 @@ SQLITE_PRIVATE void sqlite3PCacheSetDefault(void){ ** by the current thread may be sqlite3_free()ed. ** ** nReq is the number of bytes of memory required. Once this much has -** been released, the function returns. The return value is the total number +** been released, the function returns. The return value is the total number ** of bytes of memory released. */ SQLITE_PRIVATE int sqlite3PcacheReleaseMemory(int nReq){ @@ -30932,7 +30915,7 @@ SQLITE_PRIVATE void sqlite3PcacheStats( ** extracts the least value from the RowSet. ** ** The INSERT primitive might allocate additional memory. Memory is -** allocated in chunks so most INSERTs do no allocation. There is an +** allocated in chunks so most INSERTs do no allocation. There is an ** upper bound on the size of allocated memory. No memory is freed ** until DESTROY. ** @@ -30956,8 +30939,6 @@ SQLITE_PRIVATE void sqlite3PcacheStats( ** ** There is an added cost of O(N) when switching between TEST and ** SMALLEST primitives. -** -** $Id: rowset.c,v 1.7 2009/05/22 01:00:13 drh Exp $ */ @@ -30975,7 +30956,7 @@ SQLITE_PRIVATE void sqlite3PcacheStats( /* ** Each entry in a RowSet is an instance of the following object. */ -struct RowSetEntry { +struct RowSetEntry { i64 v; /* ROWID value for this entry */ struct RowSetEntry *pRight; /* Right subtree (larger entries) or list */ struct RowSetEntry *pLeft; /* Left subtree (smaller entries) */ @@ -31017,7 +30998,7 @@ struct RowSet { ** ** It must be the case that N is sufficient to make a Rowset. If not ** an assertion fault occurs. -** +** ** If N is larger than the minimum, use the surplus as an initial ** allocation of entries available to be filled. */ @@ -31097,7 +31078,7 @@ SQLITE_PRIVATE void sqlite3RowSetInsert(RowSet *p, i64 rowid){ /* ** Merge two lists of RowSetEntry objects. Remove duplicates. ** -** The input lists are connected via pRight pointers and are +** The input lists are connected via pRight pointers and are ** assumed to each already be in sorted order. */ static struct RowSetEntry *rowSetMerge( @@ -31135,7 +31116,7 @@ static struct RowSetEntry *rowSetMerge( /* ** Sort all elements on the pEntry list of the RowSet into ascending order. -*/ +*/ static void rowSetSort(RowSet *p){ unsigned int i; struct RowSetEntry *pEntry; @@ -31275,7 +31256,7 @@ static void rowSetToList(RowSet *p){ ** 0 if the RowSet is already empty. ** ** After this routine has been called, the sqlite3RowSetInsert() -** routine may not be called again. +** routine may not be called again. */ SQLITE_PRIVATE int sqlite3RowSetNext(RowSet *p, i64 *pRowid){ rowSetToList(p); @@ -31333,15 +31314,13 @@ SQLITE_PRIVATE int sqlite3RowSetTest(RowSet *pRowSet, u8 iBatch, sqlite3_int64 i ** ************************************************************************* ** This is the implementation of the page cache subsystem or "pager". -** +** ** The pager is used to access a database disk file. It implements ** atomic commit and rollback through the use of a journal file that ** is separate from the database file. The pager also implements file ** locking to prevent two processes from writing the same database ** file simultaneously, or one process from reading the database while ** another is writing. -** -** @(#) $Id: pager.c,v 1.629 2009/08/10 17:48:57 drh Exp $ */ #ifndef SQLITE_OMIT_DISKIO @@ -31358,7 +31337,7 @@ int sqlite3PagerTrace=1; /* True to enable tracing */ /* ** The following two macros are used within the PAGERTRACE() macros above -** to print out file-descriptors. +** to print out file-descriptors. ** ** PAGERID() takes a pointer to a Pager struct as its argument. The ** associated file-descriptor is returned. FILEHANDLEID() takes an sqlite3_file @@ -31371,7 +31350,7 @@ int sqlite3PagerTrace=1; /* True to enable tracing */ ** The page cache as a whole is always in one of the following ** states: ** -** PAGER_UNLOCK The page cache is not currently reading or +** PAGER_UNLOCK The page cache is not currently reading or ** writing the database file. There is no ** data held in memory. This is the initial ** state. @@ -31397,7 +31376,7 @@ int sqlite3PagerTrace=1; /* True to enable tracing */ ** after all dirty pages have been written to the ** database file and the file has been synced to ** disk. All that remains to do is to remove or -** truncate the journal file and the transaction +** truncate the journal file and the transaction ** will be committed. ** ** The page cache comes up in PAGER_UNLOCK. The first time a @@ -31436,7 +31415,7 @@ int sqlite3PagerTrace=1; /* True to enable tracing */ #endif /* -** The maximum allowed sector size. 64KiB. If the xSectorsize() method +** The maximum allowed sector size. 64KiB. If the xSectorsize() method ** returns a value larger than this, then MAX_SECTOR_SIZE is used instead. ** This could conceivably cause corruption following a power failure on ** such a system. This is currently an undocumented limit. @@ -31451,7 +31430,7 @@ int sqlite3PagerTrace=1; /* True to enable tracing */ ** ** When a savepoint is created, the PagerSavepoint.iHdrOffset field is ** set to 0. If a journal-header is written into the main journal while -** the savepoint is active, then iHdrOffset is set to the byte offset +** the savepoint is active, then iHdrOffset is set to the byte offset ** immediately following the last journal record written into the main ** journal before the journal-header. This is required during savepoint ** rollback (see pagerPlaybackSavepoint()). @@ -31496,18 +31475,18 @@ struct PagerSavepoint { ** ** changeCountDone ** -** This boolean variable is used to make sure that the change-counter -** (the 4-byte header field at byte offset 24 of the database file) is -** not updated more often than necessary. +** This boolean variable is used to make sure that the change-counter +** (the 4-byte header field at byte offset 24 of the database file) is +** not updated more often than necessary. ** -** It is set to true when the change-counter field is updated, which +** It is set to true when the change-counter field is updated, which ** can only happen if an exclusive lock is held on the database file. -** It is cleared (set to false) whenever an exclusive lock is +** It is cleared (set to false) whenever an exclusive lock is ** relinquished on the database file. Each time a transaction is committed, ** The changeCountDone flag is inspected. If it is true, the work of ** updating the change-counter is omitted for the current transaction. ** -** This mechanism means that when running in exclusive mode, a connection +** This mechanism means that when running in exclusive mode, a connection ** need only update the change-counter once, for the first transaction ** committed. ** @@ -31521,15 +31500,15 @@ struct PagerSavepoint { ** ** journalStarted ** -** This flag is set whenever the the main journal is synced. +** This flag is set whenever the the main journal is synced. ** -** The point of this flag is that it must be set after the +** The point of this flag is that it must be set after the ** first journal header in a journal file has been synced to disk. -** After this has happened, new pages appended to the database +** After this has happened, new pages appended to the database ** do not need the PGHDR_NEED_SYNC flag set, as they do not need ** to wait for a journal sync before they can be written out to ** the database file (see function pager_write()). -** +** ** setMaster ** ** This variable is used to ensure that the master journal file name @@ -31541,7 +31520,7 @@ struct PagerSavepoint { ** then attempts to upgrade to an exclusive lock. If this attempt ** fails, then SQLITE_BUSY may be returned to the user and the user ** may attempt to commit the transaction again later (calling -** CommitPhaseOne() again). This flag is used to ensure that the +** CommitPhaseOne() again). This flag is used to ensure that the ** master journal name is only written to the journal file the first ** time CommitPhaseOne() is called. ** @@ -31577,12 +31556,12 @@ struct Pager { /* The following block contains those class members that are dynamically ** modified during normal operations. The other variables in this structure ** are either constant throughout the lifetime of the pager, or else - ** used to store configuration parameters that affect the way the pager + ** used to store configuration parameters that affect the way the pager ** operates. ** ** The 'state' variable is described in more detail along with the ** descriptions of the values it may take - PAGER_UNLOCK etc. Many of the - ** other variables in this block are described in the comment directly + ** other variables in this block are described in the comment directly ** above this class definition. */ u8 state; /* PAGER_UNLOCK, _SHARED, _RESERVED, etc. */ @@ -31688,7 +31667,7 @@ static const unsigned char aJournalMagic[] = { #define JOURNAL_PG_SZ(pPager) ((pPager->pageSize) + 8) /* -** The journal header size for this pager. This is usually the same +** The journal header size for this pager. This is usually the same ** size as a single disk sector. See also setSectorSize(). */ #define JOURNAL_HDR_SZ(pPager) (pPager->sectorSize) @@ -31710,7 +31689,7 @@ static const unsigned char aJournalMagic[] = { */ #define PAGER_MAX_PGNO 2147483647 -#ifndef NDEBUG +#ifndef NDEBUG /* ** Usage: ** @@ -31897,8 +31876,8 @@ static void checkPage(PgHdr *pPg){ /* ** When this is called the journal file for pager pPager must be open. -** This function attempts to read a master journal file name from the -** end of the file and, if successful, copies it into memory supplied +** This function attempts to read a master journal file name from the +** end of the file and, if successful, copies it into memory supplied ** by the caller. See comments above writeMasterJournal() for the format ** used to store a master journal file name at the end of a journal file. ** @@ -31914,7 +31893,7 @@ static void checkPage(PgHdr *pPg){ ** nul-terminator byte is appended to the buffer following the master ** journal file name. ** -** If it is determined that no master journal file name is present +** If it is determined that no master journal file name is present ** zMaster[0] is set to 0 and SQLITE_OK returned. ** ** If an error occurs while reading from the journal file, an SQLite @@ -31932,7 +31911,7 @@ static int readMasterJournal(sqlite3_file *pJrnl, char *zMaster, u32 nMaster){ if( SQLITE_OK!=(rc = sqlite3OsFileSize(pJrnl, &szJ)) || szJ<16 || SQLITE_OK!=(rc = read32bits(pJrnl, szJ-16, &len)) - || len>=nMaster + || len>=nMaster || SQLITE_OK!=(rc = read32bits(pJrnl, szJ-12, &cksum)) || SQLITE_OK!=(rc = sqlite3OsRead(pJrnl, aMagic, 8, szJ-8)) || memcmp(aMagic, aJournalMagic, 8) @@ -31954,13 +31933,13 @@ static int readMasterJournal(sqlite3_file *pJrnl, char *zMaster, u32 nMaster){ len = 0; } zMaster[len] = '\0'; - + return SQLITE_OK; } /* -** Return the offset of the sector boundary at or immediately -** following the value in pPager->journalOff, assuming a sector +** Return the offset of the sector boundary at or immediately +** following the value in pPager->journalOff, assuming a sector ** size of pPager->sectorSize bytes. ** ** i.e for a sector size of 512: @@ -31971,7 +31950,7 @@ static int readMasterJournal(sqlite3_file *pJrnl, char *zMaster, u32 nMaster){ ** 512 512 ** 100 512 ** 2000 2048 -** +** */ static i64 journalHdrOffset(Pager *pPager){ i64 offset = 0; @@ -31993,12 +31972,12 @@ static i64 journalHdrOffset(Pager *pPager){ ** ** If doTruncate is non-zero or the Pager.journalSizeLimit variable is ** set to 0, then truncate the journal file to zero bytes in size. Otherwise, -** zero the 28-byte header at the start of the journal file. In either case, -** if the pager is not in no-sync mode, sync the journal file immediately +** zero the 28-byte header at the start of the journal file. In either case, +** if the pager is not in no-sync mode, sync the journal file immediately ** after writing or truncating it. ** ** If Pager.journalSizeLimit is set to a positive, non-zero value, and -** following the truncation or zeroing described above the size of the +** following the truncation or zeroing described above the size of the ** journal file in bytes is larger than this value, then truncate the ** journal file to Pager.journalSizeLimit bytes. The journal file does ** not need to be synced following this operation. @@ -32023,8 +32002,8 @@ static int zeroJournalHdr(Pager *pPager, int doTruncate){ rc = sqlite3OsSync(pPager->jfd, SQLITE_SYNC_DATAONLY|pPager->sync_flags); } - /* At this point the transaction is committed but the write lock - ** is still held on the file. If there is a size limit configured for + /* At this point the transaction is committed but the write lock + ** is still held on the file. If there is a size limit configured for ** the persistent journal and the journal file currently consumes more ** space than that limit allows for, truncate it now. There is no need ** to sync the file following this operation. @@ -32052,7 +32031,7 @@ static int zeroJournalHdr(Pager *pPager, int doTruncate){ ** - 4 bytes: Initial database page count. ** - 4 bytes: Sector size used by the process that wrote this journal. ** - 4 bytes: Database page size. -** +** ** Followed by (JOURNAL_HDR_SZ - 28) bytes of unused space. */ static int writeJournalHdr(Pager *pPager){ @@ -32068,8 +32047,8 @@ static int writeJournalHdr(Pager *pPager){ nHeader = JOURNAL_HDR_SZ(pPager); } - /* If there are active savepoints and any of them were created - ** since the most recent journal header was written, update the + /* If there are active savepoints and any of them were created + ** since the most recent journal header was written, update the ** PagerSavepoint.iHdrOffset fields now. */ for(ii=0; iinSavepoint; ii++){ @@ -32080,10 +32059,10 @@ static int writeJournalHdr(Pager *pPager){ pPager->journalHdr = pPager->journalOff = journalHdrOffset(pPager); - /* + /* ** Write the nRec Field - the number of page records that follow this ** journal header. Normally, zero is written to this value at this time. - ** After the records are added to the journal (and the journal synced, + ** After the records are added to the journal (and the journal synced, ** if in full-sync mode), the zero is overwritten with the true number ** of records (see syncJournal()). ** @@ -32102,7 +32081,7 @@ static int writeJournalHdr(Pager *pPager){ */ assert( isOpen(pPager->fd) || pPager->noSync ); if( (pPager->noSync) || (pPager->journalMode==PAGER_JOURNALMODE_MEMORY) - || (sqlite3OsDeviceCharacteristics(pPager->fd)&SQLITE_IOCAP_SAFE_APPEND) + || (sqlite3OsDeviceCharacteristics(pPager->fd)&SQLITE_IOCAP_SAFE_APPEND) ){ memcpy(zHeader, aJournalMagic, sizeof(aJournalMagic)); put32bits(&zHeader[sizeof(aJournalMagic)], 0xffffffff); @@ -32110,7 +32089,7 @@ static int writeJournalHdr(Pager *pPager){ memset(zHeader, 0, sizeof(aJournalMagic)+4); } - /* The random check-hash initialiser */ + /* The random check-hash initialiser */ sqlite3_randomness(sizeof(pPager->cksumInit), &pPager->cksumInit); put32bits(&zHeader[sizeof(aJournalMagic)+4], pPager->cksumInit); /* The initial database size */ @@ -32129,23 +32108,23 @@ static int writeJournalHdr(Pager *pPager){ memset(&zHeader[sizeof(aJournalMagic)+20], 0, nHeader-(sizeof(aJournalMagic)+20)); - /* In theory, it is only necessary to write the 28 bytes that the - ** journal header consumes to the journal file here. Then increment the - ** Pager.journalOff variable by JOURNAL_HDR_SZ so that the next + /* In theory, it is only necessary to write the 28 bytes that the + ** journal header consumes to the journal file here. Then increment the + ** Pager.journalOff variable by JOURNAL_HDR_SZ so that the next ** record is written to the following sector (leaving a gap in the file ** that will be implicitly filled in by the OS). ** - ** However it has been discovered that on some systems this pattern can + ** However it has been discovered that on some systems this pattern can ** be significantly slower than contiguously writing data to the file, - ** even if that means explicitly writing data to the block of + ** even if that means explicitly writing data to the block of ** (JOURNAL_HDR_SZ - 28) bytes that will not be used. So that is what - ** is done. + ** is done. ** - ** The loop is required here in case the sector-size is larger than the + ** The loop is required here in case the sector-size is larger than the ** database page size. Since the zHeader buffer is only Pager.pageSize ** bytes in size, more than one call to sqlite3OsWrite() may be required ** to populate the entire journal header sector. - */ + */ for(nWrite=0; rc==SQLITE_OK&&nWritejournalHdr, nHeader)) rc = sqlite3OsWrite(pPager->jfd, zHeader, nHeader, pPager->journalOff); @@ -32235,23 +32214,23 @@ static int readJournalHdr( /* Check that the values read from the page-size and sector-size fields ** are within range. To be 'in range', both values need to be a power - ** of two greater than or equal to 512, and not greater than their + ** of two greater than or equal to 512 or 32, and not greater than their ** respective compile time maximum limits. */ - if( iPageSize<512 || iSectorSize<512 + if( iPageSize<512 || iSectorSize<32 || iPageSize>SQLITE_MAX_PAGE_SIZE || iSectorSize>MAX_SECTOR_SIZE - || ((iPageSize-1)&iPageSize)!=0 || ((iSectorSize-1)&iSectorSize)!=0 + || ((iPageSize-1)&iPageSize)!=0 || ((iSectorSize-1)&iSectorSize)!=0 ){ - /* If the either the page-size or sector-size in the journal-header is - ** invalid, then the process that wrote the journal-header must have - ** crashed before the header was synced. In this case stop reading + /* If the either the page-size or sector-size in the journal-header is + ** invalid, then the process that wrote the journal-header must have + ** crashed before the header was synced. In this case stop reading ** the journal file here. */ return SQLITE_DONE; } - /* Update the page-size to match the value read from the journal. - ** Use a testcase() macro to make sure that malloc failure within + /* Update the page-size to match the value read from the journal. + ** Use a testcase() macro to make sure that malloc failure within ** PagerSetPagesize() is tested. */ iPageSize16 = (u16)iPageSize; @@ -32259,7 +32238,7 @@ static int readJournalHdr( testcase( rc!=SQLITE_OK ); assert( rc!=SQLITE_OK || iPageSize16==(u16)iPageSize ); - /* Update the assumed sector-size to match the value used by + /* Update the assumed sector-size to match the value used by ** the process that created this journal. If this journal was ** created by a process other than this one, then this routine ** is being called from within pager_playback(). The local value @@ -32289,7 +32268,7 @@ static int readJournalHdr( ** The master journal page checksum is the sum of the bytes in the master ** journal name, where each byte is interpreted as a signed 8-bit integer. ** -** If zMaster is a NULL pointer (occurs for a single database transaction), +** If zMaster is a NULL pointer (occurs for a single database transaction), ** this call is a no-op. */ static int writeMasterJournal(Pager *pPager, const char *zMaster){ @@ -32300,8 +32279,8 @@ static int writeMasterJournal(Pager *pPager, const char *zMaster){ u32 cksum = 0; /* Checksum of string zMaster */ if( !zMaster || pPager->setMaster - || pPager->journalMode==PAGER_JOURNALMODE_MEMORY - || pPager->journalMode==PAGER_JOURNALMODE_OFF + || pPager->journalMode==PAGER_JOURNALMODE_MEMORY + || pPager->journalMode==PAGER_JOURNALMODE_OFF ){ return SQLITE_OK; } @@ -32336,16 +32315,16 @@ static int writeMasterJournal(Pager *pPager, const char *zMaster){ pPager->journalOff += (nMaster+20); pPager->needSync = !pPager->noSync; - /* If the pager is in peristent-journal mode, then the physical + /* If the pager is in peristent-journal mode, then the physical ** journal-file may extend past the end of the master-journal name - ** and 8 bytes of magic data just written to the file. This is + ** and 8 bytes of magic data just written to the file. This is ** dangerous because the code to rollback a hot-journal file - ** will not be able to find the master-journal name to determine - ** whether or not the journal is hot. + ** will not be able to find the master-journal name to determine + ** whether or not the journal is hot. ** - ** Easiest thing to do in this scenario is to truncate the journal + ** Easiest thing to do in this scenario is to truncate the journal ** file to the required size. - */ + */ if( SQLITE_OK==(rc = sqlite3OsFileSize(pPager->jfd, &jrnlSize)) && jrnlSize>pPager->journalOff ){ @@ -32356,7 +32335,7 @@ static int writeMasterJournal(Pager *pPager, const char *zMaster){ /* ** Find a page in the hash table given its page number. Return -** a pointer to the page or NULL if the requested page is not +** a pointer to the page or NULL if the requested page is not ** already in memory. */ static PgHdr *pager_lookup(Pager *pPager, Pgno pgno){ @@ -32403,7 +32382,7 @@ static void releaseAllSavepoints(Pager *pPager){ } /* -** Set the bit number pgno in the PagerSavepoint.pInSavepoint +** Set the bit number pgno in the PagerSavepoint.pInSavepoint ** bitvecs of all open savepoints. Return SQLITE_OK if successful ** or SQLITE_NOMEM if a malloc failure occurs. */ @@ -32426,7 +32405,7 @@ static int addToSavepointBitvecs(Pager *pPager, Pgno pgno){ ** Unlock the database file. This function is a no-op if the pager ** is in exclusive mode. ** -** If the pager is currently in error state, discard the contents of +** If the pager is currently in error state, discard the contents of ** the cache and reset the Pager structure internal state. If there is ** an open journal-file, then the next time a shared-lock is obtained ** on the pager file (by this or any other process), it will be @@ -32471,23 +32450,24 @@ static void pager_unlock(Pager *pPager){ pPager->changeCountDone = 0; pPager->state = PAGER_UNLOCK; + pPager->dbModified = 0; } } /* ** This function should be called when an IOERR, CORRUPT or FULL error -** may have occurred. The first argument is a pointer to the pager -** structure, the second the error-code about to be returned by a pager -** API function. The value returned is a copy of the second argument -** to this function. +** may have occurred. The first argument is a pointer to the pager +** structure, the second the error-code about to be returned by a pager +** API function. The value returned is a copy of the second argument +** to this function. ** ** If the second argument is SQLITE_IOERR, SQLITE_CORRUPT, or SQLITE_FULL ** the error becomes persistent. Until the persisten error is cleared, -** subsequent API calls on this Pager will immediately return the same +** subsequent API calls on this Pager will immediately return the same ** error code. ** -** A persistent error indicates that the contents of the pager-cache -** cannot be trusted. This state can be cleared by completely discarding +** A persistent error indicates that the contents of the pager-cache +** cannot be trusted. This state can be cleared by completely discarding ** the contents of the pager-cache. If a transaction was active when ** the persistent error occurred, then the rollback journal may need ** to be replayed to restore the contents of the database file (as if @@ -32508,10 +32488,10 @@ static int pager_error(Pager *pPager, int rc){ } /* -** Execute a rollback if a transaction is active and unlock the -** database file. +** Execute a rollback if a transaction is active and unlock the +** database file. ** -** If the pager has already entered the error state, do not attempt +** If the pager has already entered the error state, do not attempt ** the rollback at this time. Instead, pager_unlock() is called. The ** call to pager_unlock() will discard all in-memory pages, unlock ** the database file and clear the error state. If this means that @@ -32520,7 +32500,7 @@ static int pager_error(Pager *pPager, int rc){ ** roll it back. ** ** If the pager has not already entered the error state, but an IO or -** malloc error occurs during a rollback, then this will itself cause +** malloc error occurs during a rollback, then this will itself cause ** the pager to enter the error state. Which will be cleared by the ** call to pager_unlock(), as described above. */ @@ -32534,26 +32514,26 @@ static void pagerUnlockAndRollback(Pager *pPager){ } /* -** This routine ends a transaction. A transaction is usually ended by -** either a COMMIT or a ROLLBACK operation. This routine may be called +** This routine ends a transaction. A transaction is usually ended by +** either a COMMIT or a ROLLBACK operation. This routine may be called ** after rollback of a hot-journal, or if an error occurs while opening ** the journal file or writing the very first journal-header of a ** database transaction. -** +** ** If the pager is in PAGER_SHARED or PAGER_UNLOCK state when this ** routine is called, it is a no-op (returns SQLITE_OK). ** ** Otherwise, any active savepoints are released. ** -** If the journal file is open, then it is "finalized". Once a journal -** file has been finalized it is not possible to use it to roll back a +** If the journal file is open, then it is "finalized". Once a journal +** file has been finalized it is not possible to use it to roll back a ** transaction. Nor will it be considered to be a hot-journal by this ** or any other database connection. Exactly how a journal is finalized ** depends on whether or not the pager is running in exclusive mode and ** the current journal-mode (Pager.journalMode value), as follows: ** ** journalMode==MEMORY -** Journal file descriptor is simply closed. This destroys an +** Journal file descriptor is simply closed. This destroys an ** in-memory journal. ** ** journalMode==TRUNCATE @@ -32582,7 +32562,7 @@ static void pagerUnlockAndRollback(Pager *pPager){ ** ** SQLITE_OK is returned if no error occurs. If an error occurs during ** any of the IO operations to finalize the journal file or unlock the -** database then the IO error code is returned to the user. If the +** database then the IO error code is returned to the user. If the ** operation to finalize the journal file fails, then the code still ** tries to unlock the database file if not in exclusive mode. If the ** unlock operation fails as well, then the first error code related @@ -32613,7 +32593,7 @@ static int pager_end_transaction(Pager *pPager, int hasMaster){ } pPager->journalOff = 0; pPager->journalStarted = 0; - }else if( pPager->exclusiveMode + }else if( pPager->exclusiveMode || pPager->journalMode==PAGER_JOURNALMODE_PERSIST ){ rc = zeroJournalHdr(pPager, hasMaster); @@ -32625,8 +32605,8 @@ static int pager_end_transaction(Pager *pPager, int hasMaster){ ** a hot-journal was just rolled back. In this case the journal ** file should be closed and deleted. If this connection writes to ** the database file, it will do so using an in-memory journal. */ - assert( pPager->journalMode==PAGER_JOURNALMODE_DELETE - || pPager->journalMode==PAGER_JOURNALMODE_MEMORY + assert( pPager->journalMode==PAGER_JOURNALMODE_DELETE + || pPager->journalMode==PAGER_JOURNALMODE_MEMORY ); sqlite3OsClose(pPager->jfd); if( !pPager->tempFile ){ @@ -32655,7 +32635,7 @@ static int pager_end_transaction(Pager *pPager, int hasMaster){ pPager->needSync = 0; pPager->dbModified = 0; - /* TODO: Is this optimal? Why is the db size invalidated here + /* TODO: Is this optimal? Why is the db size invalidated here ** when the database file is not unlocked? */ pPager->dbOrigSize = 0; sqlite3PcacheTruncate(pPager->pPCache, pPager->dbSize); @@ -32668,10 +32648,10 @@ static int pager_end_transaction(Pager *pPager, int hasMaster){ /* ** Parameter aData must point to a buffer of pPager->pageSize bytes -** of data. Compute and return a checksum based ont the contents of the +** of data. Compute and return a checksum based ont the contents of the ** page of data and the current value of pPager->cksumInit. ** -** This is not a real checksum. It is really just the sum of the +** This is not a real checksum. It is really just the sum of the ** random initial value (pPager->cksumInit) and every 200th byte ** of the page data, starting with byte offset (pPager->pageSize%200). ** Each byte is interpreted as an 8-bit unsigned integer. @@ -32679,8 +32659,8 @@ static int pager_end_transaction(Pager *pPager, int hasMaster){ ** Changing the formula used to compute this checksum results in an ** incompatible journal file format. ** -** If journal corruption occurs due to a power failure, the most likely -** scenario is that one end or the other of the record will be changed. +** If journal corruption occurs due to a power failure, the most likely +** scenario is that one end or the other of the record will be changed. ** It is much less likely that the two ends of the journal record will be ** correct and the middle be corrupt. Thus, this "checksum" scheme, ** though fast and simple, catches the mostly likely kind of corruption. @@ -32722,7 +32702,7 @@ static u32 pager_cksum(Pager *pPager, const u8 *aData){ ** is successfully read from the (sub-)journal file but appears to be ** corrupted, SQLITE_DONE is returned. Data is considered corrupted in ** two circumstances: -** +** ** * If the record page-number is illegal (0 or PAGER_MJ_PGNO), or ** * If the record is being rolled back from the main journal file ** and the checksum field does not match the record content. @@ -32745,7 +32725,7 @@ static int pager_playback_one_page( PgHdr *pPg; /* An existing page in the cache */ Pgno pgno; /* The page number of a page in journal */ u32 cksum; /* Checksum used for sanity checking */ - u8 *aData; /* Temporary storage for the page */ + char *aData; /* Temporary storage for the page */ sqlite3_file *jfd; /* The file descriptor for the journal file */ assert( (isMainJrnl&~1)==0 ); /* isMainJrnl is 0 or 1 */ @@ -32753,7 +32733,7 @@ static int pager_playback_one_page( assert( isMainJrnl || pDone ); /* pDone always used on sub-journals */ assert( isSavepnt || pDone==0 ); /* pDone never used on non-savepoint */ - aData = (u8*)pPager->pTmpSpace; + aData = pPager->pTmpSpace; assert( aData ); /* Temp storage must have already been allocated */ /* Read the page number and page data from the journal or sub-journal @@ -32762,7 +32742,7 @@ static int pager_playback_one_page( jfd = isMainJrnl ? pPager->jfd : pPager->sjfd; rc = read32bits(jfd, *pOffset, &pgno); if( rc!=SQLITE_OK ) return rc; - rc = sqlite3OsRead(jfd, aData, pPager->pageSize, (*pOffset)+4); + rc = sqlite3OsRead(jfd, (u8*)aData, pPager->pageSize, (*pOffset)+4); if( rc!=SQLITE_OK ) return rc; *pOffset += pPager->pageSize + 4 + isMainJrnl*4; @@ -32781,7 +32761,7 @@ static int pager_playback_one_page( if( isMainJrnl ){ rc = read32bits(jfd, (*pOffset)-4, &cksum); if( rc ) return rc; - if( !isSavepnt && pager_cksum(pPager, aData)!=cksum ){ + if( !isSavepnt && pager_cksum(pPager, (u8*)aData)!=cksum ){ return SQLITE_DONE; } } @@ -32827,8 +32807,8 @@ static int pager_playback_one_page( pPg = pager_lookup(pPager, pgno); assert( pPg || !MEMDB ); PAGERTRACE(("PLAYBACK %d page %d hash(%08x) %s\n", - PAGERID(pPager), pgno, pager_datahash(pPager->pageSize, aData), - (isMainJrnl?"main-journal":"sub-journal") + PAGERID(pPager), pgno, pager_datahash(pPager->pageSize, (u8*)aData), + (isMainJrnl?"main-journal":"sub-journal") )); if( (pPager->state>=PAGER_EXCLUSIVE) && (pPg==0 || 0==(pPg->flags&PGHDR_NEED_SYNC)) @@ -32836,30 +32816,30 @@ static int pager_playback_one_page( && !isUnsync ){ i64 ofst = (pgno-1)*(i64)pPager->pageSize; - rc = sqlite3OsWrite(pPager->fd, aData, pPager->pageSize, ofst); + rc = sqlite3OsWrite(pPager->fd, (u8*)aData, pPager->pageSize, ofst); if( pgno>pPager->dbFileSize ){ pPager->dbFileSize = pgno; } if( pPager->pBackup ){ CODEC1(pPager, aData, pgno, 3, rc=SQLITE_NOMEM); - sqlite3BackupUpdate(pPager->pBackup, pgno, aData); - CODEC1(pPager, aData, pgno, 0, rc=SQLITE_NOMEM); + sqlite3BackupUpdate(pPager->pBackup, pgno, (u8*)aData); + CODEC2(pPager, aData, pgno, 7, rc=SQLITE_NOMEM, aData); } }else if( !isMainJrnl && pPg==0 ){ /* If this is a rollback of a savepoint and data was not written to ** the database and the page is not in-memory, there is a potential - ** problem. When the page is next fetched by the b-tree layer, it - ** will be read from the database file, which may or may not be - ** current. + ** problem. When the page is next fetched by the b-tree layer, it + ** will be read from the database file, which may or may not be + ** current. ** ** There are a couple of different ways this can happen. All are quite - ** obscure. When running in synchronous mode, this can only happen + ** obscure. When running in synchronous mode, this can only happen ** if the page is on the free-list at the start of the transaction, then ** populated, then moved using sqlite3PagerMovepage(). ** ** The solution is to add an in-memory page to the cache containing - ** the data just read from the sub-journal. Mark the page as dirty - ** and if the pager requires a journal-sync, then mark the page as + ** the data just read from the sub-journal. Mark the page as dirty + ** and if the pager requires a journal-sync, then mark the page as ** requiring a journal-sync before it is written. */ assert( isSavepnt ); @@ -32878,16 +32858,16 @@ static int pager_playback_one_page( */ void *pData; pData = pPg->pData; - memcpy(pData, aData, pPager->pageSize); + memcpy(pData, (u8*)aData, pPager->pageSize); pPager->xReiniter(pPg); if( isMainJrnl && (!isSavepnt || *pOffset<=pPager->journalHdr) ){ - /* If the contents of this page were just restored from the main - ** journal file, then its content must be as they were when the + /* If the contents of this page were just restored from the main + ** journal file, then its content must be as they were when the ** transaction was first opened. In this case we can mark the page ** as clean, since there will be no need to write it out to the. ** ** There is one exception to this rule. If the page is being rolled - ** back as part of a savepoint (or statement) rollback from an + ** back as part of a savepoint (or statement) rollback from an ** unsynced portion of the main journal file, then it is not safe ** to mark the page as clean. This is because marking the page as ** clean will clear the PGHDR_NEED_SYNC flag. Since the page is @@ -32923,26 +32903,26 @@ static int pager_playback_one_page( ** This routine checks if it is possible to delete the master journal file, ** and does so if it is. ** -** Argument zMaster may point to Pager.pTmpSpace. So that buffer is not +** Argument zMaster may point to Pager.pTmpSpace. So that buffer is not ** available for use within this function. ** -** When a master journal file is created, it is populated with the names -** of all of its child journals, one after another, formatted as utf-8 -** encoded text. The end of each child journal file is marked with a +** When a master journal file is created, it is populated with the names +** of all of its child journals, one after another, formatted as utf-8 +** encoded text. The end of each child journal file is marked with a ** nul-terminator byte (0x00). i.e. the entire contents of a master journal ** file for a transaction involving two databases might be: ** ** "/home/bill/a.db-journal\x00/home/bill/b.db-journal\x00" ** -** A master journal file may only be deleted once all of its child +** A master journal file may only be deleted once all of its child ** journals have been rolled back. ** -** This function reads the contents of the master-journal file into +** This function reads the contents of the master-journal file into ** memory and loops through each of the child journal names. For ** each child journal, it checks if: ** ** * if the child journal exists, and if so -** * if the child journal contains a reference to master journal +** * if the child journal contains a reference to master journal ** file zMaster ** ** If a child journal can be found that matches both of the criteria @@ -32952,12 +32932,12 @@ static int pager_playback_one_page( ** ** If an IO error within this function, an error code is returned. This ** function allocates memory by calling sqlite3Malloc(). If an allocation -** fails, SQLITE_NOMEM is returned. Otherwise, if no IO or malloc errors +** fails, SQLITE_NOMEM is returned. Otherwise, if no IO or malloc errors ** occur, SQLITE_OK is returned. ** ** TODO: This function allocates a single block of memory to load ** the entire contents of the master journal file. This could be -** a couple of kilobytes or so - potentially larger than the page +** a couple of kilobytes or so - potentially larger than the page ** size. */ static int pager_delmaster(Pager *pPager, const char *zMaster){ @@ -32990,7 +32970,7 @@ static int pager_delmaster(Pager *pPager, const char *zMaster){ int nMasterPtr = pVfs->mxPathname+1; /* Load the entire master journal file into space obtained from - ** sqlite3_malloc() and pointed to by zMasterJournal. + ** sqlite3_malloc() and pointed to by zMasterJournal. */ zMasterJournal = sqlite3Malloc((int)nMasterJournal + nMasterPtr + 1); if( !zMasterJournal ){ @@ -33036,13 +33016,13 @@ static int pager_delmaster(Pager *pPager, const char *zMaster){ zJournal += (sqlite3Strlen30(zJournal)+1); } } - + rc = sqlite3OsDelete(pVfs, zMaster, 0); delmaster_out: if( zMasterJournal ){ sqlite3_free(zMasterJournal); - } + } if( pMaster ){ sqlite3OsClose(pMaster); assert( !isOpen(pJournal) ); @@ -33053,7 +33033,7 @@ delmaster_out: /* -** This function is used to change the actual size of the database +** This function is used to change the actual size of the database ** file in the file-system. This only happens when committing a transaction, ** or rolling back a transaction (including rolling back a hot-journal). ** @@ -33063,10 +33043,10 @@ delmaster_out: ** on disk is currently larger than nPage pages, then use the VFS ** xTruncate() method to truncate it. ** -** Or, it might might be the case that the file on disk is smaller than -** nPage pages. Some operating system implementations can get confused if -** you try to truncate a file to some size that is larger than it -** currently is, so detect this case and write a single zero byte to +** Or, it might might be the case that the file on disk is smaller than +** nPage pages. Some operating system implementations can get confused if +** you try to truncate a file to some size that is larger than it +** currently is, so detect this case and write a single zero byte to ** the end of the new file instead. ** ** If successful, return SQLITE_OK. If an IO error occurs while modifying @@ -33096,15 +33076,15 @@ static int pager_truncate(Pager *pPager, Pgno nPage){ /* ** Set the value of the Pager.sectorSize variable for the given ** pager based on the value returned by the xSectorSize method -** of the open database file. The sector size will be used used -** to determine the size and alignment of journal header and +** of the open database file. The sector size will be used used +** to determine the size and alignment of journal header and ** master journal pointers within created journal files. ** ** For temporary files the effective sector size is always 512 bytes. ** ** Otherwise, for non-temporary files, the effective sector size is -** the value returned by the xSectorSize() method rounded up to 512 if -** it is less than 512, or rounded down to MAX_SECTOR_SIZE if it +** the value returned by the xSectorSize() method rounded up to 32 if +** it is less than 32, or rounded down to MAX_SECTOR_SIZE if it ** is greater than MAX_SECTOR_SIZE. */ static void setSectorSize(Pager *pPager){ @@ -33117,7 +33097,7 @@ static void setSectorSize(Pager *pPager){ */ pPager->sectorSize = sqlite3OsSectorSize(pPager->fd); } - if( pPager->sectorSize<512 ){ + if( pPager->sectorSize<32 ){ pPager->sectorSize = 512; } if( pPager->sectorSize>MAX_SECTOR_SIZE ){ @@ -33128,15 +33108,15 @@ static void setSectorSize(Pager *pPager){ /* ** Playback the journal and thus restore the database file to -** the state it was in before we started making changes. +** the state it was in before we started making changes. ** -** The journal file format is as follows: +** The journal file format is as follows: ** ** (1) 8 byte prefix. A copy of aJournalMagic[]. ** (2) 4 byte big-endian integer which is the number of valid page records ** in the journal. If this value is 0xffffffff, then compute the ** number of page records from the journal size. -** (3) 4 byte big-endian integer which is the initial value for the +** (3) 4 byte big-endian integer which is the initial value for the ** sanity checksum. ** (4) 4 byte integer which is the number of pages to truncate the ** database to during a rollback. @@ -33165,7 +33145,7 @@ static void setSectorSize(Pager *pPager){ ** from the file size. This value is used when the user selects the ** no-sync option for the journal. A power failure could lead to corruption ** in this case. But for things like temporary table (which will be -** deleted when the power is restored) we don't care. +** deleted when the power is restored) we don't care. ** ** If the file opened as the journal file is not a well-formed ** journal file then all pages up to the first corrupted page are rolled @@ -33177,7 +33157,7 @@ static void setSectorSize(Pager *pPager){ ** and an error code is returned. ** ** The isHot parameter indicates that we are trying to rollback a journal -** that might be a hot journal. Or, it could be that the journal is +** that might be a hot journal. Or, it could be that the journal is ** preserved because of JOURNALMODE_PERSIST or JOURNALMODE_TRUNCATE. ** If the journal really is hot, reset the pager cache prior rolling ** back any content. If the journal is merely persistent, no reset is @@ -33226,9 +33206,9 @@ static int pager_playback(Pager *pPager, int isHot){ pPager->journalOff = 0; needPagerReset = isHot; - /* This loop terminates either when a readJournalHdr() or - ** pager_playback_one_page() call returns SQLITE_DONE or an IO error - ** occurs. + /* This loop terminates either when a readJournalHdr() or + ** pager_playback_one_page() call returns SQLITE_DONE or an IO error + ** occurs. */ while( 1 ){ int isUnsync = 0; @@ -33239,7 +33219,7 @@ static int pager_playback(Pager *pPager, int isHot){ ** This indicates nothing more needs to be rolled back. */ rc = readJournalHdr(pPager, isHot, szJ, &nRec, &mxPg); - if( rc!=SQLITE_OK ){ + if( rc!=SQLITE_OK ){ if( rc==SQLITE_DONE ){ rc = SQLITE_OK; } @@ -33267,7 +33247,7 @@ static int pager_playback(Pager *pPager, int isHot){ ** chunk of the journal contains zero pages to be rolled back. But ** when doing a ROLLBACK and the nRec==0 chunk is the last chunk in ** the journal, it means that the journal might contain additional - ** pages that need to be rolled back and that the number of pages + ** pages that need to be rolled back and that the number of pages ** should be computed based on the journal file size. */ if( nRec==0 && !isHot && @@ -33287,7 +33267,7 @@ static int pager_playback(Pager *pPager, int isHot){ pPager->dbSize = mxPg; } - /* Copy original pages out of the journal and back into the + /* Copy original pages out of the journal and back into the ** database file and/or page cache. */ for(u=0; ufd,SQLITE_FCNTL_DB_UNCHANGED,0)>=SQLITE_OK ); - /* If this playback is happening automatically as a result of an IO or - ** malloc error that occurred after the change-counter was updated but - ** before the transaction was committed, then the change-counter - ** modification may just have been reverted. If this happens in exclusive + /* If this playback is happening automatically as a result of an IO or + ** malloc error that occurred after the change-counter was updated but + ** before the transaction was committed, then the change-counter + ** modification may just have been reverted. If this happens in exclusive ** mode, then subsequent transactions performed by the connection will not ** update the change-counter at all. This may lead to cache inconsistency ** problems for other processes at some point in the future. So, just @@ -33364,21 +33344,21 @@ end_playback: /* ** Playback savepoint pSavepoint. Or, if pSavepoint==NULL, then playback -** the entire master journal file. The case pSavepoint==NULL occurs when -** a ROLLBACK TO command is invoked on a SAVEPOINT that is a transaction +** the entire master journal file. The case pSavepoint==NULL occurs when +** a ROLLBACK TO command is invoked on a SAVEPOINT that is a transaction ** savepoint. ** -** When pSavepoint is not NULL (meaning a non-transaction savepoint is +** When pSavepoint is not NULL (meaning a non-transaction savepoint is ** being rolled back), then the rollback consists of up to three stages, ** performed in the order specified: ** ** * Pages are played back from the main journal starting at byte -** offset PagerSavepoint.iOffset and continuing to +** offset PagerSavepoint.iOffset and continuing to ** PagerSavepoint.iHdrOffset, or to the end of the main journal ** file if PagerSavepoint.iHdrOffset is zero. ** ** * If PagerSavepoint.iHdrOffset is not zero, then pages are played -** back starting from the journal header immediately following +** back starting from the journal header immediately following ** PagerSavepoint.iHdrOffset to the end of the main journal file. ** ** * Pages are then played back from the sub-journal file, starting @@ -33394,7 +33374,7 @@ end_playback: ** journal file. There is no need for a bitvec in this case. ** ** In either case, before playback commences the Pager.dbSize variable -** is reset to the value that it held at the start of the savepoint +** is reset to the value that it held at the start of the savepoint ** (or transaction). No page with a page-number greater than this value ** is played back. If one is encountered it is simply skipped. */ @@ -33414,7 +33394,7 @@ static int pagerPlaybackSavepoint(Pager *pPager, PagerSavepoint *pSavepoint){ } } - /* Set the database size back to the value it was before the savepoint + /* Set the database size back to the value it was before the savepoint ** being reverted was opened. */ pPager->dbSize = pSavepoint ? pSavepoint->nOrig : pPager->dbOrigSize; @@ -33461,7 +33441,7 @@ static int pagerPlaybackSavepoint(Pager *pPager, PagerSavepoint *pSavepoint){ ** test is related to ticket #2565. See the discussion in the ** pager_playback() function for additional information. */ - if( nJRec==0 + if( nJRec==0 && pPager->journalHdr+JOURNAL_HDR_SZ(pPager)==pPager->journalOff ){ nJRec = (u32)((szJ - pPager->journalOff)/JOURNAL_PG_SZ(pPager)); @@ -33539,7 +33519,7 @@ SQLITE_PRIVATE void sqlite3PagerSetSafetyLevel(Pager *pPager, int level, int bFu /* ** The following global variable is incremented whenever the library ** attempts to open a temporary file. This information is used for -** testing and analysis only. +** testing and analysis only. */ #ifdef SQLITE_TEST SQLITE_API int sqlite3_opentemp_count = 0; @@ -33548,8 +33528,8 @@ SQLITE_API int sqlite3_opentemp_count = 0; /* ** Open a temporary file. ** -** Write the file descriptor into *pFile. Return SQLITE_OK on success -** or some other error code if we fail. The OS will automatically +** Write the file descriptor into *pFile. Return SQLITE_OK on success +** or some other error code if we fail. The OS will automatically ** delete the temporary file when it is closed. ** ** The flags passed to the VFS layer xOpen() call are those specified @@ -33581,9 +33561,9 @@ static int pagerOpentemp( /* ** Set the busy handler function. ** -** The pager invokes the busy-handler if sqlite3OsLock() returns +** The pager invokes the busy-handler if sqlite3OsLock() returns ** SQLITE_BUSY when trying to upgrade from no-lock to a SHARED lock, -** or when trying to upgrade from a RESERVED lock to an EXCLUSIVE +** or when trying to upgrade from a RESERVED lock to an EXCLUSIVE ** lock. It does *not* invoke the busy handler when upgrading from ** SHARED to RESERVED, or when upgrading from SHARED to EXCLUSIVE ** (which occurs during hot-journal rollback). Summary: @@ -33595,7 +33575,7 @@ static int pagerOpentemp( ** SHARED_LOCK -> EXCLUSIVE_LOCK | No ** RESERVED_LOCK -> EXCLUSIVE_LOCK | Yes ** -** If the busy-handler callback returns non-zero, the lock is +** If the busy-handler callback returns non-zero, the lock is ** retried. If it returns zero, then the SQLITE_BUSY error is ** returned to the caller of the pager API function. */ @@ -33603,7 +33583,7 @@ SQLITE_PRIVATE void sqlite3PagerSetBusyhandler( Pager *pPager, /* Pager object */ int (*xBusyHandler)(void *), /* Pointer to busy-handler function */ void *pBusyHandlerArg /* Argument to pass to xBusyHandler */ -){ +){ pPager->xBusyHandler = xBusyHandler; pPager->pBusyHandlerArg = pBusyHandlerArg; } @@ -33624,16 +33604,16 @@ static void pagerReportSize(Pager *pPager){ #endif /* -** Change the page size used by the Pager object. The new page size +** Change the page size used by the Pager object. The new page size ** is passed in *pPageSize. ** ** If the pager is in the error state when this function is called, it -** is a no-op. The value returned is the error state error code (i.e. +** is a no-op. The value returned is the error state error code (i.e. ** one of SQLITE_IOERR, SQLITE_CORRUPT or SQLITE_FULL). ** ** Otherwise, if all of the following are true: ** -** * the new page size (value of *pPageSize) is valid (a power +** * the new page size (value of *pPageSize) is valid (a power ** of two between 512 and SQLITE_MAX_PAGE_SIZE, inclusive), and ** ** * there are no outstanding page references, and @@ -33643,14 +33623,14 @@ static void pagerReportSize(Pager *pPager){ ** ** then the pager object page size is set to *pPageSize. ** -** If the page size is changed, then this function uses sqlite3PagerMalloc() -** to obtain a new Pager.pTmpSpace buffer. If this allocation attempt -** fails, SQLITE_NOMEM is returned and the page size remains unchanged. +** If the page size is changed, then this function uses sqlite3PagerMalloc() +** to obtain a new Pager.pTmpSpace buffer. If this allocation attempt +** fails, SQLITE_NOMEM is returned and the page size remains unchanged. ** In all other cases, SQLITE_OK is returned. ** ** If the page size is not changed, either because one of the enumerated ** conditions above is not true, the pager was in error state when this -** function was called, or because the memory allocation attempt failed, +** function was called, or because the memory allocation attempt failed, ** then *pPageSize is set to the old, retained page size before returning. */ SQLITE_PRIVATE int sqlite3PagerSetPagesize(Pager *pPager, u16 *pPageSize, int nReserve){ @@ -33660,8 +33640,8 @@ SQLITE_PRIVATE int sqlite3PagerSetPagesize(Pager *pPager, u16 *pPageSize, int nR u16 pageSize = *pPageSize; assert( pageSize==0 || (pageSize>=512 && pageSize<=SQLITE_MAX_PAGE_SIZE) ); if( (pPager->memDb==0 || pPager->dbSize==0) - && sqlite3PcacheRefCount(pPager->pPCache)==0 - && pageSize && pageSize!=pPager->pageSize + && sqlite3PcacheRefCount(pPager->pPCache)==0 + && pageSize && pageSize!=pPager->pageSize ){ char *pNew = (char *)sqlite3PageMalloc(pageSize); if( !pNew ){ @@ -33696,7 +33676,7 @@ SQLITE_PRIVATE void *sqlite3PagerTempSpace(Pager *pPager){ } /* -** Attempt to set the maximum database page count if mxPage is positive. +** Attempt to set the maximum database page count if mxPage is positive. ** Make no changes if mxPage is zero or negative. And never reduce the ** maximum page count below the current size of the database. ** @@ -33736,11 +33716,11 @@ void enable_simulated_io_errors(void){ /* ** Read the first N bytes from the beginning of the file into memory -** that pDest points to. +** that pDest points to. ** ** If the pager was opened on a transient file (zFilename==""), or ** opened on a file less than N bytes in size, the output buffer is -** zeroed and SQLITE_OK returned. The rationale for this is that this +** zeroed and SQLITE_OK returned. The rationale for this is that this ** function is used to read database headers, and a new transient or ** zero sized database has a header than consists entirely of zeroes. ** @@ -33763,9 +33743,9 @@ SQLITE_PRIVATE int sqlite3PagerReadFileheader(Pager *pPager, int N, unsigned cha } /* -** Return the total number of pages in the database file associated +** Return the total number of pages in the database file associated ** with pPager. Normally, this is calculated as (/). -** However, if the file is between 1 and bytes in size, then +** However, if the file is between 1 and bytes in size, then ** this is considered a 1 page file. ** ** If the pager is in error state when this function is called, then the @@ -33809,7 +33789,7 @@ SQLITE_PRIVATE int sqlite3PagerPagecount(Pager *pPager, int *pnPage){ } } - /* If the current number of pages in the file is greater than the + /* If the current number of pages in the file is greater than the ** configured maximum pager number, increase the allowed limit so ** that the file can be read. */ @@ -33830,13 +33810,13 @@ SQLITE_PRIVATE int sqlite3PagerPagecount(Pager *pPager, int *pnPage){ ** a similar or greater lock is already held, this function is a no-op ** (returning SQLITE_OK immediately). ** -** Otherwise, attempt to obtain the lock using sqlite3OsLock(). Invoke -** the busy callback if the lock is currently not available. Repeat -** until the busy callback returns false or until the attempt to +** Otherwise, attempt to obtain the lock using sqlite3OsLock(). Invoke +** the busy callback if the lock is currently not available. Repeat +** until the busy callback returns false or until the attempt to ** obtain the lock succeeds. ** ** Return SQLITE_OK on success and an error code if we cannot obtain -** the lock. If the lock is obtained successfully, set the Pager.state +** the lock. If the lock is obtained successfully, set the Pager.state ** variable to locktype before returning. */ static int pager_wait_on_lock(Pager *pPager, int locktype){ @@ -33847,10 +33827,13 @@ static int pager_wait_on_lock(Pager *pPager, int locktype){ assert( PAGER_RESERVED==RESERVED_LOCK ); assert( PAGER_EXCLUSIVE==EXCLUSIVE_LOCK ); - /* If the file is currently unlocked then the size must be unknown */ + /* If the file is currently unlocked then the size must be unknown. It + ** must not have been modified at this point. + */ assert( pPager->state>=PAGER_SHARED || pPager->dbSizeValid==0 ); + assert( pPager->state>=PAGER_SHARED || pPager->dbModified==0 ); - /* Check that this is either a no-op (because the requested lock is + /* Check that this is either a no-op (because the requested lock is ** already held, or one of the transistions that the busy-handler ** may be invoked during, according to the comment above ** sqlite3PagerSetBusyhandler(). @@ -33875,10 +33858,10 @@ static int pager_wait_on_lock(Pager *pPager, int locktype){ } /* -** Function assertTruncateConstraint(pPager) checks that one of the +** Function assertTruncateConstraint(pPager) checks that one of the ** following is true for all dirty pages currently in the page-cache: ** -** a) The page number is less than or equal to the size of the +** a) The page number is less than or equal to the size of the ** current database image, in pages, OR ** ** b) if the page content were written at this time, it would not @@ -33891,9 +33874,9 @@ static int pager_wait_on_lock(Pager *pPager, int locktype){ ** the database file. If a savepoint transaction were rolled back after ** this happened, the correct behaviour would be to restore the current ** content of the page. However, since this content is not present in either -** the database file or the portion of the rollback journal and +** the database file or the portion of the rollback journal and ** sub-journal rolled back the content could not be restored and the -** database image would become corrupt. It is therefore fortunate that +** database image would become corrupt. It is therefore fortunate that ** this circumstance cannot arise. */ #if defined(SQLITE_DEBUG) @@ -33909,9 +33892,9 @@ static void assertTruncateConstraint(Pager *pPager){ #endif /* -** Truncate the in-memory database file image to nPage pages. This -** function does not actually modify the database file on disk. It -** just sets the internal state of the pager object so that the +** Truncate the in-memory database file image to nPage pages. This +** function does not actually modify the database file on disk. It +** just sets the internal state of the pager object so that the ** truncation will be done when the current transaction is committed. */ SQLITE_PRIVATE void sqlite3PagerTruncateImage(Pager *pPager, Pgno nPage){ @@ -33932,7 +33915,7 @@ SQLITE_PRIVATE void sqlite3PagerTruncateImage(Pager *pPager, Pgno nPage){ ** result in a coredump. ** ** This function always succeeds. If a transaction is active an attempt -** is made to roll it back. If an error occurs during the rollback +** is made to roll it back. If an error occurs during the rollback ** a hot journal may be left in the filesystem but no error is returned ** to the caller. */ @@ -33945,7 +33928,7 @@ SQLITE_PRIVATE int sqlite3PagerClose(Pager *pPager){ if( MEMDB ){ pager_unlock(pPager); }else{ - /* Set Pager.journalHdr to -1 for the benefit of the pager_playback() + /* Set Pager.journalHdr to -1 for the benefit of the pager_playback() ** call which may be made from within pagerUnlockAndRollback(). If it ** is not -1, then the unsynced portion of an open journal file may ** be played back into the database. If a power failure occurs while @@ -34007,7 +33990,7 @@ SQLITE_PRIVATE void sqlite3PagerRef(DbPage *pPg){ ** been written following it. If the pager is operating in full-sync ** mode, then the journal file is synced before this field is updated. ** -** * If the device does not support the SEQUENTIAL property, then +** * If the device does not support the SEQUENTIAL property, then ** journal file is synced. ** ** Or, in pseudo-code: @@ -34016,14 +33999,14 @@ SQLITE_PRIVATE void sqlite3PagerRef(DbPage *pPg){ ** if( NOT SAFE_APPEND ){ ** if( ) xSync(); ** -** } +** } ** if( NOT SEQUENTIAL ) xSync(); ** } ** ** The Pager.needSync flag is never be set for temporary files, or any ** file operating in no-sync mode (Pager.noSync set to non-zero). ** -** If successful, this routine clears the PGHDR_NEED_SYNC flag of every +** If successful, this routine clears the PGHDR_NEED_SYNC flag of every ** page currently held in memory before returning SQLITE_OK. If an IO ** error is encountered, then the IO error code is returned to the caller. */ @@ -34041,10 +34024,10 @@ static int syncJournal(Pager *pPager){ ** mode, then the journal file may at this point actually be larger ** than Pager.journalOff bytes. If the next thing in the journal ** file happens to be a journal-header (written as part of the - ** previous connections transaction), and a crash or power-failure - ** occurs after nRec is updated but before this connection writes - ** anything else to the journal file (or commits/rolls back its - ** transaction), then SQLite may become confused when doing the + ** previous connections transaction), and a crash or power-failure + ** occurs after nRec is updated but before this connection writes + ** anything else to the journal file (or commits/rolls back its + ** transaction), then SQLite may become confused when doing the ** hot-journal rollback following recovery. It may roll back all ** of this connections data, then proceed to rolling back the old, ** out-of-date data that follows it. Database corruption. @@ -34054,7 +34037,7 @@ static int syncJournal(Pager *pPager){ ** byte to the start of it to prevent it from being recognized. ** ** Variable iNextHdrOffset is set to the offset at which this - ** problematic header will occur, if it exists. aMagic is used + ** problematic header will occur, if it exists. aMagic is used ** as a temporary buffer to inspect the first couple of bytes of ** the potential journal header. */ @@ -34081,7 +34064,7 @@ static int syncJournal(Pager *pPager){ ** it as a candidate for rollback. ** ** This is not required if the persistent media supports the - ** SAFE_APPEND property. Because in this case it is not possible + ** SAFE_APPEND property. Because in this case it is not possible ** for garbage data to be appended to the file, the nRec field ** is populated with 0xFFFFFFFF when the journal header is written ** and never needs to be updated. @@ -34101,14 +34084,14 @@ static int syncJournal(Pager *pPager){ if( 0==(iDc&SQLITE_IOCAP_SEQUENTIAL) ){ PAGERTRACE(("SYNC journal of %d\n", PAGERID(pPager))); IOTRACE(("JSYNC %p\n", pPager)) - rc = sqlite3OsSync(pPager->jfd, pPager->sync_flags| + rc = sqlite3OsSync(pPager->jfd, pPager->sync_flags| (pPager->sync_flags==SQLITE_SYNC_FULL?SQLITE_SYNC_DATAONLY:0) ); if( rc!=SQLITE_OK ) return rc; } } - /* The journal file was just successfully synced. Set Pager.needSync + /* The journal file was just successfully synced. Set Pager.needSync ** to zero and clear the PGHDR_NEED_SYNC flag on all pagess. */ pPager->needSync = 0; @@ -34130,9 +34113,9 @@ static int syncJournal(Pager *pPager){ ** is called. Before writing anything to the database file, this lock ** is upgraded to an EXCLUSIVE lock. If the lock cannot be obtained, ** SQLITE_BUSY is returned and no data is written to the database file. -** +** ** If the pager is a temp-file pager and the actual file-system file -** is not yet open, it is created and opened before any data is +** is not yet open, it is created and opened before any data is ** written out. ** ** Once the lock has been upgraded and, if necessary, the file opened, @@ -34147,7 +34130,7 @@ static int syncJournal(Pager *pPager){ ** in Pager.dbFileVers[] is updated to match the new value stored in ** the database file. ** -** If everything is successful, SQLITE_OK is returned. If an IO error +** If everything is successful, SQLITE_OK is returned. If an IO error ** occurs, an IO error code is returned. Or, if the EXCLUSIVE lock cannot ** be obtained, SQLITE_BUSY is returned. */ @@ -34195,11 +34178,13 @@ static int pager_write_pagelist(PgHdr *pList){ ** any such pages to the file. ** ** Also, do not write out any page that has the PGHDR_DONT_WRITE flag - ** set (set by sqlite3PagerDontWrite()). + ** set (set by sqlite3PagerDontWrite()). Note that if compiled with + ** SQLITE_SECURE_DELETE the PGHDR_DONT_WRITE bit is never set and so + ** the second test is always true. */ if( pgno<=pPager->dbSize && 0==(pList->flags&PGHDR_DONT_WRITE) ){ i64 offset = (pgno-1)*(i64)pPager->pageSize; /* Offset to write */ - char *pData; /* Data to write */ + char *pData; /* Data to write */ /* Encode the database */ CODEC2(pPager, pList->pData, pgno, 6, return SQLITE_NOMEM, pData); @@ -34208,8 +34193,8 @@ static int pager_write_pagelist(PgHdr *pList){ rc = sqlite3OsWrite(pPager->fd, pData, pPager->pageSize, offset); /* If page 1 was just written, update Pager.dbFileVers to match - ** the value now stored in the database file. If writing this - ** page caused the database file to grow, update dbFileSize. + ** the value now stored in the database file. If writing this + ** page caused the database file to grow, update dbFileSize. */ if( pgno==1 ){ memcpy(&pPager->dbFileVers, &pData[24], sizeof(pPager->dbFileVers)); @@ -34239,15 +34224,15 @@ static int pager_write_pagelist(PgHdr *pList){ } /* -** Append a record of the current state of page pPg to the sub-journal. -** It is the callers responsibility to use subjRequiresPage() to check +** Append a record of the current state of page pPg to the sub-journal. +** It is the callers responsibility to use subjRequiresPage() to check ** that it is really required before calling this function. ** ** If successful, set the bit corresponding to pPg->pgno in the bitvecs ** for all open savepoints before returning. ** ** This function returns SQLITE_OK if everything is successful, an IO -** error code if the attempt to write to the sub-journal fails, or +** error code if the attempt to write to the sub-journal fails, or ** SQLITE_NOMEM if a malloc fails while setting a bit in a savepoint ** bitvec. */ @@ -34261,7 +34246,7 @@ static int subjournalPage(PgHdr *pPg){ CODEC2(pPager, pData, pPg->pgno, 7, return SQLITE_NOMEM, pData2); PAGERTRACE(("STMT-JOURNAL %d page %d\n", PAGERID(pPager), pPg->pgno)); - + assert( pageInJournal(pPg) || pPg->pgno>pPager->dbOrigSize ); rc = write32bits(pPager->sjfd, offset, pPg->pgno); if( rc==SQLITE_OK ){ @@ -34281,14 +34266,14 @@ static int subjournalPage(PgHdr *pPg){ ** This function is called by the pcache layer when it has reached some ** soft memory limit. The first argument is a pointer to a Pager object ** (cast as a void*). The pager is always 'purgeable' (not an in-memory -** database). The second argument is a reference to a page that is +** database). The second argument is a reference to a page that is ** currently dirty but has no outstanding references. The page -** is always associated with the Pager object passed as the first +** is always associated with the Pager object passed as the first ** argument. ** ** The job of this function is to make pPg clean by writing its contents ** out to the database file, if possible. This may involve syncing the -** journal file. +** journal file. ** ** If successful, sqlite3PcacheMakeClean() is called on the page and ** SQLITE_OK returned. If an IO error occurs while trying to make the @@ -34325,7 +34310,7 @@ static int pagerStress(void *p, PgHdr *pPg){ /* Sync the journal file if required. */ if( pPg->flags&PGHDR_NEED_SYNC ){ rc = syncJournal(pPager); - if( rc==SQLITE_OK && pPager->fullSync && + if( rc==SQLITE_OK && pPager->fullSync && !(pPager->journalMode==PAGER_JOURNALMODE_MEMORY) && !(sqlite3OsDeviceCharacteristics(pPager->fd)&SQLITE_IOCAP_SAFE_APPEND) ){ @@ -34356,9 +34341,9 @@ static int pagerStress(void *p, PgHdr *pPg){ ** was when the transaction started, not as it was when "SAVEPOINT sp" ** was executed. ** - ** The solution is to write the current data for page X into the + ** The solution is to write the current data for page X into the ** sub-journal file now (if it is not already there), so that it will - ** be restored to its current value when the "ROLLBACK TO sp" is + ** be restored to its current value when the "ROLLBACK TO sp" is ** executed. */ if( NEVER( @@ -34391,8 +34376,8 @@ static int pagerStress(void *p, PgHdr *pPg){ ** The zFilename argument is the path to the database file to open. ** If zFilename is NULL then a randomly-named temporary file is created ** and used as the file to be cached. Temporary files are be deleted -** automatically when they are closed. If zFilename is ":memory:" then -** all information is held in cache. It is never written to disk. +** automatically when they are closed. If zFilename is ":memory:" then +** all information is held in cache. It is never written to disk. ** This can be used to implement an in-memory database. ** ** The nExtra parameter specifies the number of bytes of space allocated @@ -34404,13 +34389,13 @@ static int pagerStress(void *p, PgHdr *pPg){ ** of the PAGER_OMIT_JOURNAL and PAGER_NO_READLOCK flags. ** ** The vfsFlags parameter is a bitmask to pass to the flags parameter -** of the xOpen() method of the supplied VFS when opening files. +** of the xOpen() method of the supplied VFS when opening files. ** -** If the pager object is allocated and the specified file opened +** If the pager object is allocated and the specified file opened ** successfully, SQLITE_OK is returned and *ppPager set to point to ** the new pager object. If an error occurs, *ppPager is set to NULL ** and error code returned. This function may return SQLITE_NOMEM -** (sqlite3Malloc() is used to allocate memory), SQLITE_CANTOPEN or +** (sqlite3Malloc() is used to allocate memory), SQLITE_CANTOPEN or ** various SQLITE_IO_XXX errors. */ SQLITE_PRIVATE int sqlite3PagerOpen( @@ -34438,10 +34423,10 @@ SQLITE_PRIVATE int sqlite3PagerOpen( /* Figure out how much space is required for each journal file-handle ** (there are two of them, the main journal and the sub-journal). This - ** is the maximum space required for an in-memory journal file handle + ** is the maximum space required for an in-memory journal file handle ** and a regular journal file-handle. Note that a "regular journal-handle" ** may be a wrapper capable of caching the first portion of the journal - ** file in memory to implement the atomic-write optimization (see + ** file in memory to implement the atomic-write optimization (see ** source file journal.c). */ if( sqlite3JournalSize(pVfs)>sqlite3MemJournalSize() ){ @@ -34491,7 +34476,7 @@ SQLITE_PRIVATE int sqlite3PagerOpen( } /* Allocate memory for the Pager structure, PCache object, the - ** three file descriptors, the database file name and the journal + ** three file descriptors, the database file name and the journal ** file name. The layout in memory is as follows: ** ** Pager object (sizeof(Pager) bytes) @@ -34506,7 +34491,7 @@ SQLITE_PRIVATE int sqlite3PagerOpen( ROUND8(sizeof(*pPager)) + /* Pager structure */ ROUND8(pcacheSize) + /* PCache object */ ROUND8(pVfs->szOsFile) + /* The main db file */ - journalFileSize * 2 + /* The two journal files */ + journalFileSize * 2 + /* The two journal files */ nPathname + 1 + /* zFilename */ nPathname + 8 + 1 /* zJournal */ ); @@ -34583,13 +34568,13 @@ SQLITE_PRIVATE int sqlite3PagerOpen( ** This branch is also run for an in-memory database. An in-memory ** database is the same as a temp-file that is never written out to ** disk and uses an in-memory rollback journal. - */ + */ tempFile = 1; pPager->state = PAGER_EXCLUSIVE; readOnly = (vfsFlags&SQLITE_OPEN_READONLY); } - /* The following call to PagerSetPagesize() serves to set the value of + /* The following call to PagerSetPagesize() serves to set the value of ** Pager.pageSize and to allocate the Pager.pTmpSpace buffer. */ if( rc==SQLITE_OK ){ @@ -34598,7 +34583,7 @@ SQLITE_PRIVATE int sqlite3PagerOpen( testcase( rc!=SQLITE_OK ); } - /* If an error occurred in either of the blocks above, free the + /* If an error occurred in either of the blocks above, free the ** Pager structure and close the file. */ if( rc!=SQLITE_OK ){ @@ -34631,10 +34616,10 @@ SQLITE_PRIVATE int sqlite3PagerOpen( assert( pPager->state == (tempFile ? PAGER_EXCLUSIVE : PAGER_UNLOCK) ); /* pPager->errMask = 0; */ pPager->tempFile = (u8)tempFile; - assert( tempFile==PAGER_LOCKINGMODE_NORMAL + assert( tempFile==PAGER_LOCKINGMODE_NORMAL || tempFile==PAGER_LOCKINGMODE_EXCLUSIVE ); assert( PAGER_LOCKINGMODE_EXCLUSIVE==1 ); - pPager->exclusiveMode = (u8)tempFile; + pPager->exclusiveMode = (u8)tempFile; pPager->changeCountDone = pPager->tempFile; pPager->memDb = (u8)memDb; pPager->readOnly = (u8)readOnly; @@ -34668,7 +34653,7 @@ SQLITE_PRIVATE int sqlite3PagerOpen( /* ** This function is called after transitioning from PAGER_UNLOCK to ** PAGER_SHARED state. It tests if there is a hot journal present in -** the file-system for the given pager. A hot journal is one that +** the file-system for the given pager. A hot journal is one that ** needs to be played back. According to this function, a hot-journal ** file exists if the following criteria are met: ** @@ -34687,10 +34672,10 @@ SQLITE_PRIVATE int sqlite3PagerOpen( ** at the end of the file. If there is, and that master journal file ** does not exist, then the journal file is not really hot. In this ** case this routine will return a false-positive. The pager_playback() -** routine will discover that the journal file is not really hot and -** will not roll it back. +** routine will discover that the journal file is not really hot and +** will not roll it back. ** -** If a hot-journal file is found to exist, *pExists is set to 1 and +** If a hot-journal file is found to exist, *pExists is set to 1 and ** SQLITE_OK returned. If no hot-journal file is present, *pExists is ** set to 0 and SQLITE_OK returned. If an IO error occurs while trying ** to determine whether or not a hot-journal file exists, the IO error @@ -34713,7 +34698,7 @@ static int hasHotJournal(Pager *pPager, int *pExists){ int locked; /* True if some process holds a RESERVED lock */ /* Race condition here: Another process might have been holding the - ** the RESERVED lock and have a journal open at the sqlite3OsAccess() + ** the RESERVED lock and have a journal open at the sqlite3OsAccess() ** call above, but then delete the journal and drop the lock before ** we get to the following sqlite3OsCheckReservedLock() call. If that ** is the case, this routine might think there is a hot journal when @@ -34725,7 +34710,7 @@ static int hasHotJournal(Pager *pPager, int *pExists){ int nPage; /* Check the size of the database file. If it consists of 0 pages, - ** then delete the journal file. See the header comment above for + ** then delete the journal file. See the header comment above for ** the reasoning here. Delete the obsolete journal file under ** a RESERVED lock to avoid race conditions and to avoid violating ** [H33020]. @@ -34743,7 +34728,7 @@ static int hasHotJournal(Pager *pPager, int *pExists){ /* The journal file exists and no other connection has a reserved ** or greater lock on the database file. Now check that there is ** at least one non-zero bytes at the start of the journal file. - ** If there is, then we consider this journal to be hot. If not, + ** If there is, then we consider this journal to be hot. If not, ** it can be ignored. */ int f = SQLITE_OPEN_READONLY|SQLITE_OPEN_MAIN_JOURNAL; @@ -34778,7 +34763,7 @@ static int hasHotJournal(Pager *pPager, int *pExists){ } /* -** Read the content for page pPg out of the database file and into +** Read the content for page pPg out of the database file and into ** pPg->pData. A shared lock or greater must be held on the database ** file before this function is called. ** @@ -34834,7 +34819,7 @@ static int readDbPage(PgHdr *pPg){ ** on the database file), then an attempt is made to obtain a ** SHARED lock on the database file. Immediately after obtaining ** the SHARED lock, the file-system is checked for a hot-journal, -** which is played back if present. Following any hot-journal +** which is played back if present. Following any hot-journal ** rollback, the contents of the cache are validated by checking ** the 'change-counter' field of the database file header and ** discarded if they are found to be invalid. @@ -34907,12 +34892,12 @@ SQLITE_PRIVATE int sqlite3PagerSharedLock(Pager *pPager){ ** important that a RESERVED lock is not obtained on the way to the ** EXCLUSIVE lock. If it were, another process might open the ** database file, detect the RESERVED lock, and conclude that the - ** database is safe to read while this process is still rolling the + ** database is safe to read while this process is still rolling the ** hot-journal back. - ** + ** ** Because the intermediate RESERVED lock is not requested, any - ** other process attempting to access the database file will get to - ** this point in the code and fail to obtain its own EXCLUSIVE lock + ** other process attempting to access the database file will get to + ** this point in the code and fail to obtain its own EXCLUSIVE lock ** on the database file. */ if( pPager->statestate = PAGER_EXCLUSIVE; } - - /* Open the journal for read/write access. This is because in + + /* Open the journal for read/write access. This is because in ** exclusive-access mode the file descriptor will be kept open and ** possibly used for a transaction later on. On some systems, the ** OsTruncate() call used in exclusive-access mode also requires @@ -34945,9 +34930,9 @@ SQLITE_PRIVATE int sqlite3PagerSharedLock(Pager *pPager){ sqlite3OsClose(pPager->jfd); } }else{ - /* If the journal does not exist, it usually means that some - ** other connection managed to get in and roll it back before - ** this connection obtained the exclusive lock above. Or, it + /* If the journal does not exist, it usually means that some + ** other connection managed to get in and roll it back before + ** this connection obtained the exclusive lock above. Or, it ** may mean that the pager was in the error-state when this ** function was called and the journal file does not exist. */ rc = pager_end_transaction(pPager, 0); @@ -34963,7 +34948,7 @@ SQLITE_PRIVATE int sqlite3PagerSharedLock(Pager *pPager){ pPager->journalOff = 0; pPager->setMaster = 0; pPager->journalHdr = 0; - + /* Playback and delete the journal. Drop the database write ** lock and reacquire the read lock. Purge the cache before ** playing back the hot-journal so that we don't end up with @@ -34993,8 +34978,8 @@ SQLITE_PRIVATE int sqlite3PagerSharedLock(Pager *pPager){ ** a 32-bit counter that is incremented with each change. The ** other bytes change randomly with each file change when ** a codec is in use. - ** - ** There is a vanishingly small chance that a change will not be + ** + ** There is a vanishingly small chance that a change will not be ** detected. The chance of an undetected change is so small that ** it can be neglected. */ @@ -35039,10 +35024,10 @@ SQLITE_PRIVATE int sqlite3PagerSharedLock(Pager *pPager){ ** Except, in locking_mode=EXCLUSIVE when there is nothing to in ** the rollback journal, the unlock is not performed and there is ** nothing to rollback, so this routine is a no-op. -*/ +*/ static void pagerUnlockIfUnused(Pager *pPager){ if( (sqlite3PcacheRefCount(pPager->pPCache)==0) - && (!pPager->exclusiveMode || pPager->journalOff>0) + && (!pPager->exclusiveMode || pPager->journalOff>0) ){ pagerUnlockAndRollback(pPager); } @@ -35050,25 +35035,25 @@ static void pagerUnlockIfUnused(Pager *pPager){ /* ** Acquire a reference to page number pgno in pager pPager (a page -** reference has type DbPage*). If the requested reference is +** reference has type DbPage*). If the requested reference is ** successfully obtained, it is copied to *ppPage and SQLITE_OK returned. ** -** If the requested page is already in the cache, it is returned. +** If the requested page is already in the cache, it is returned. ** Otherwise, a new page object is allocated and populated with data ** read from the database file. In some cases, the pcache module may ** choose not to allocate a new page object and may reuse an existing ** object with no outstanding references. ** -** The extra data appended to a page is always initialized to zeros the -** first time a page is loaded into memory. If the page requested is +** The extra data appended to a page is always initialized to zeros the +** first time a page is loaded into memory. If the page requested is ** already in the cache when this function is called, then the extra ** data is left as it was when the page object was last used. ** -** If the database image is smaller than the requested page or if a -** non-zero value is passed as the noContent parameter and the -** requested page is not already stored in the cache, then no -** actual disk read occurs. In this case the memory image of the -** page is initialized to all zeros. +** If the database image is smaller than the requested page or if a +** non-zero value is passed as the noContent parameter and the +** requested page is not already stored in the cache, then no +** actual disk read occurs. In this case the memory image of the +** page is initialized to all zeros. ** ** If noContent is true, it means that we do not care about the contents ** of the page. This occurs in two seperate scenarios: @@ -35114,7 +35099,7 @@ SQLITE_PRIVATE int sqlite3PagerAcquire( return SQLITE_CORRUPT_BKPT; } - /* If the pager is in the error state, return an error immediately. + /* If the pager is in the error state, return an error immediately. ** Otherwise, request the page from the PCache layer. */ if( pPager->errCode!=SQLITE_OK && pPager->errCode!=SQLITE_FULL ){ rc = pPager->errCode; @@ -35140,7 +35125,7 @@ SQLITE_PRIVATE int sqlite3PagerAcquire( return SQLITE_OK; }else{ - /* The pager cache has created a new page. Its content needs to + /* The pager cache has created a new page. Its content needs to ** be initialized. */ int nMax; @@ -35160,16 +35145,16 @@ SQLITE_PRIVATE int sqlite3PagerAcquire( goto pager_acquire_err; } - if( nMax<(int)pgno || MEMDB || noContent ){ + if( MEMDB || nMax<(int)pgno || noContent ){ if( pgno>pPager->mxPgno ){ rc = SQLITE_FULL; goto pager_acquire_err; } if( noContent ){ /* Failure to set the bits in the InJournal bit-vectors is benign. - ** It merely means that we might do some extra work to journal a - ** page that does not need to be journaled. Nevertheless, be sure - ** to test the case where a malloc error occurs while trying to set + ** It merely means that we might do some extra work to journal a + ** page that does not need to be journaled. Nevertheless, be sure + ** to test the case where a malloc error occurs while trying to set ** a bit in a bit vector. */ sqlite3BeginBenignMalloc(); @@ -35212,14 +35197,14 @@ pager_acquire_err: /* ** Acquire a page if it is already in the in-memory cache. Do ** not read the page from disk. Return a pointer to the page, -** or 0 if the page is not in cache. Also, return 0 if the +** or 0 if the page is not in cache. Also, return 0 if the ** pager is in PAGER_UNLOCK state when this function is called, ** or if the pager is in an error state other than SQLITE_FULL. ** ** See also sqlite3PagerGet(). The difference between this routine ** and sqlite3PagerGet() is that _get() will go to the disk and read ** in the page if the page is not already in cache. This routine -** returns NULL if the page is not in cache or if a disk I/O error +** returns NULL if the page is not in cache or if a disk I/O error ** has ever happened. */ SQLITE_PRIVATE DbPage *sqlite3PagerLookup(Pager *pPager, Pgno pgno){ @@ -35253,8 +35238,8 @@ SQLITE_PRIVATE void sqlite3PagerUnref(DbPage *pPg){ ** sub-journal file is open too. If the main journal is not open, ** this function is a no-op. ** -** SQLITE_OK is returned if everything goes according to plan. -** An SQLITE_IOERR_XXX error code is returned if a call to +** SQLITE_OK is returned if everything goes according to plan. +** An SQLITE_IOERR_XXX error code is returned if a call to ** sqlite3OsOpen() fails. */ static int openSubJournal(Pager *pPager){ @@ -35271,24 +35256,24 @@ static int openSubJournal(Pager *pPager){ /* ** This function is called at the start of every write transaction. -** There must already be a RESERVED or EXCLUSIVE lock on the database +** There must already be a RESERVED or EXCLUSIVE lock on the database ** file when this routine is called. ** ** Open the journal file for pager pPager and write a journal header ** to the start of it. If there are active savepoints, open the sub-journal -** as well. This function is only used when the journal file is being -** opened to write a rollback log for a transaction. It is not used +** as well. This function is only used when the journal file is being +** opened to write a rollback log for a transaction. It is not used ** when opening a hot journal file to roll it back. ** ** If the journal file is already open (as it may be in exclusive mode), ** then this function just writes a journal header to the start of the -** already open file. +** already open file. ** ** Whether or not the journal file is opened by this function, the ** Pager.pInJournal bitvec structure is allocated. ** -** Return SQLITE_OK if everything is successful. Otherwise, return -** SQLITE_NOMEM if the attempt to allocate Pager.pInJournal fails, or +** Return SQLITE_OK if everything is successful. Otherwise, return +** SQLITE_NOMEM if the attempt to allocate Pager.pInJournal fails, or ** an IO error code if opening or writing the journal file fails. */ static int pager_open_journal(Pager *pPager){ @@ -35299,7 +35284,7 @@ static int pager_open_journal(Pager *pPager){ assert( pPager->useJournal ); assert( pPager->journalMode!=PAGER_JOURNALMODE_OFF ); assert( pPager->pInJournal==0 ); - + /* If already in the error state, this function is a no-op. But on ** the other hand, this routine is never called if we are already in ** an error state. */ @@ -35323,7 +35308,7 @@ static int pager_open_journal(Pager *pPager){ }else{ const int flags = /* VFS flags to open journal file */ SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE| - (pPager->tempFile ? + (pPager->tempFile ? (SQLITE_OPEN_DELETEONCLOSE|SQLITE_OPEN_TEMP_JOURNAL): (SQLITE_OPEN_MAIN_JOURNAL) ); @@ -35339,7 +35324,7 @@ static int pager_open_journal(Pager *pPager){ } - /* Write the first journal header to the journal file and open + /* Write the first journal header to the journal file and open ** the sub-journal if necessary. */ if( rc==SQLITE_OK ){ @@ -35365,17 +35350,17 @@ static int pager_open_journal(Pager *pPager){ } /* -** Begin a write-transaction on the specified pager object. If a +** Begin a write-transaction on the specified pager object. If a ** write-transaction has already been opened, this function is a no-op. ** ** If the exFlag argument is false, then acquire at least a RESERVED ** lock on the database file. If exFlag is true, then acquire at least -** an EXCLUSIVE lock. If such a lock is already held, no locking +** an EXCLUSIVE lock. If such a lock is already held, no locking ** functions need be called. ** -** If this is not a temporary or in-memory file and, the journal file is -** opened if it has not been already. For a temporary file, the opening -** of the journal file is deferred until there is an actual need to +** If this is not a temporary or in-memory file and, the journal file is +** opened if it has not been already. For a temporary file, the opening +** of the journal file is deferred until there is an actual need to ** write to the journal. TODO: Why handle temporary files differently? ** ** If the journal file is opened (or if it is already open), then a @@ -35386,7 +35371,7 @@ static int pager_open_journal(Pager *pPager){ ** has no effect if the sub-journal is already opened (as it may be when ** running in exclusive mode) or if the transaction does not require a ** sub-journal. If the subjInMemory argument is zero, then any required -** sub-journal is implemented in-memory if pPager is an in-memory database, +** sub-journal is implemented in-memory if pPager is an in-memory database, ** or using a temporary file otherwise. */ SQLITE_PRIVATE int sqlite3PagerBegin(Pager *pPager, int exFlag, int subjInMemory){ @@ -35419,7 +35404,7 @@ SQLITE_PRIVATE int sqlite3PagerBegin(Pager *pPager, int exFlag, int subjInMemory }else if( isOpen(pPager->jfd) && pPager->journalOff==0 ){ /* This happens when the pager was in exclusive-access mode the last ** time a (read or write) transaction was successfully concluded - ** by this connection. Instead of deleting the journal file it was + ** by this connection. Instead of deleting the journal file it was ** kept open and either was truncated to 0 bytes or its header was ** overwritten with zeros. */ @@ -35444,9 +35429,9 @@ SQLITE_PRIVATE int sqlite3PagerBegin(Pager *pPager, int exFlag, int subjInMemory } /* -** Mark a single data page as writeable. The page is written into the +** Mark a single data page as writeable. The page is written into the ** main journal or sub-journal as required. If the page is written into -** one of the journals, the corresponding bit is set in the +** one of the journals, the corresponding bit is set in the ** Pager.pInJournal bitvec and the PagerSavepoint.pInSavepoint bitvecs ** of any open savepoints as appropriate. */ @@ -35499,7 +35484,7 @@ static int pager_write(PgHdr *pPg){ if( rc!=SQLITE_OK ) return rc; } pPager->dbModified = 1; - + /* The transaction journal now exists and we have a RESERVED or an ** EXCLUSIVE lock on the main database file. Write the current page to ** the transaction journal if it is not there already. @@ -35525,11 +35510,11 @@ static int pager_write(PgHdr *pPg){ rc = write32bits(pPager->jfd, pPager->journalOff, cksum); pPager->journalOff += 4; } - IOTRACE(("JOUT %p %d %lld %d\n", pPager, pPg->pgno, + IOTRACE(("JOUT %p %d %lld %d\n", pPager, pPg->pgno, pPager->journalOff, pPager->pageSize)); PAGER_INCR(sqlite3_pager_writej_count); PAGERTRACE(("JOURNAL %d page %d needSync=%d hash(%08x)\n", - PAGERID(pPager), pPg->pgno, + PAGERID(pPager), pPg->pgno, ((pPg->flags&PGHDR_NEED_SYNC)?1:0), pager_pagehash(pPg))); /* Even if an IO or diskfull error occurred while journalling the @@ -35544,7 +35529,7 @@ static int pager_write(PgHdr *pPg){ pPager->needSync = 1; } - /* An error has occurred writing to the journal file. The + /* An error has occurred writing to the journal file. The ** transaction will be rolled back by the layer above. */ if( rc!=SQLITE_OK ){ @@ -35571,7 +35556,7 @@ static int pager_write(PgHdr *pPg){ ((pPg->flags&PGHDR_NEED_SYNC)?1:0))); } } - + /* If the statement journal is open and the page is not in it, ** then write the current page to the statement journal. Note that ** the statement journal format differs from the standard journal format @@ -35592,9 +35577,9 @@ static int pager_write(PgHdr *pPg){ } /* -** Mark a data page as writeable. This routine must be called before -** making changes to a page. The caller must check the return value -** of this function and be careful not to change any page data unless +** Mark a data page as writeable. This routine must be called before +** making changes to a page. The caller must check the return value +** of this function and be careful not to change any page data unless ** this routine returns SQLITE_OK. ** ** The difference between this function and pager_write() is that this @@ -35667,7 +35652,7 @@ SQLITE_PRIVATE int sqlite3PagerWrite(DbPage *pDbPage){ } } - /* If the PGHDR_NEED_SYNC flag is set for any of the nPage pages + /* If the PGHDR_NEED_SYNC flag is set for any of the nPage pages ** starting at pg1, then it needs to be set for all of them. Because ** writing to any of these nPage pages may damage the others, the ** journal file must contain sync()ed copies of all of them @@ -35704,6 +35689,7 @@ SQLITE_PRIVATE int sqlite3PagerIswriteable(DbPage *pPg){ } #endif +#ifndef SQLITE_SECURE_DELETE /* ** A call to this routine tells the pager that it is not necessary to ** write the information on page pPg back to the disk, even though @@ -35715,7 +35701,7 @@ SQLITE_PRIVATE int sqlite3PagerIswriteable(DbPage *pPg){ ** on the given page is unused. The pager marks the page as clean so ** that it does not get written to disk. ** -** Tests show that this optimization can quadruple the speed of large +** Tests show that this optimization can quadruple the speed of large ** DELETE operations. */ SQLITE_PRIVATE void sqlite3PagerDontWrite(PgHdr *pPg){ @@ -35729,13 +35715,14 @@ SQLITE_PRIVATE void sqlite3PagerDontWrite(PgHdr *pPg){ #endif } } +#endif /* !defined(SQLITE_SECURE_DELETE) */ /* -** This routine is called to increment the value of the database file -** change-counter, stored as a 4-byte big-endian integer starting at +** This routine is called to increment the value of the database file +** change-counter, stored as a 4-byte big-endian integer starting at ** byte offset 24 of the pager file. ** -** If the isDirectMode flag is zero, then this is done by calling +** If the isDirectMode flag is zero, then this is done by calling ** sqlite3PagerWrite() on page 1, then modifying the contents of the ** page data. In this case the file will be updated when the current ** transaction is committed. @@ -35743,7 +35730,7 @@ SQLITE_PRIVATE void sqlite3PagerDontWrite(PgHdr *pPg){ ** The isDirectMode flag may only be non-zero if the library was compiled ** with the SQLITE_ENABLE_ATOMIC_WRITE macro defined. In this case, ** if isDirect is non-zero, then the database file is updated directly -** by writing an updated version of page 1 using a call to the +** by writing an updated version of page 1 using a call to the ** sqlite3OsWrite() function. */ static int pager_incr_changecounter(Pager *pPager, int isDirectMode){ @@ -35768,7 +35755,7 @@ static int pager_incr_changecounter(Pager *pPager, int isDirectMode){ #endif assert( pPager->state>=PAGER_RESERVED ); - if( !pPager->changeCountDone && ALWAYS(pPager->dbSize>0) ){ + if( !pPager->changeCountDone && pPager->dbSize>0 ){ PgHdr *pPgHdr; /* Reference to page 1 */ u32 change_counter; /* Initial value of change-counter field */ @@ -35779,7 +35766,7 @@ static int pager_incr_changecounter(Pager *pPager, int isDirectMode){ assert( pPgHdr==0 || rc==SQLITE_OK ); /* If page one was fetched successfully, and this function is not - ** operating in direct-mode, make page 1 writable. When not in + ** operating in direct-mode, make page 1 writable. When not in ** direct mode, page 1 is always held in cache and hence the PagerGet() ** above is always successful - hence the ALWAYS on rc==SQLITE_OK. */ @@ -35840,12 +35827,12 @@ SQLITE_PRIVATE int sqlite3PagerSync(Pager *pPager){ ** ** * The database file change-counter is updated, ** * the journal is synced (unless the atomic-write optimization is used), -** * all dirty pages are written to the database file, +** * all dirty pages are written to the database file, ** * the database file is truncated (if required), and -** * the database file synced. +** * the database file synced. ** -** The only thing that remains to commit the transaction is to finalize -** (delete, truncate or zero the first part of) the journal file (or +** The only thing that remains to commit the transaction is to finalize +** (delete, truncate or zero the first part of) the journal file (or ** delete the master journal file if specified). ** ** Note that if zMaster==NULL, this does not overwrite a previous value @@ -35871,7 +35858,7 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne( ** coding errors by repeating the prior error. */ if( NEVER(pPager->errCode) ) return pPager->errCode; - PAGERTRACE(("DATABASE SYNC: File=%s zMaster=%s nSize=%d\n", + PAGERTRACE(("DATABASE SYNC: File=%s zMaster=%s nSize=%d\n", pPager->zFilename, zMaster, pPager->dbSize)); if( MEMDB && pPager->dbModified ){ @@ -35884,11 +35871,11 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne( /* The following block updates the change-counter. Exactly how it ** does this depends on whether or not the atomic-update optimization - ** was enabled at compile time, and if this transaction meets the - ** runtime criteria to use the operation: + ** was enabled at compile time, and if this transaction meets the + ** runtime criteria to use the operation: ** ** * The file-system supports the atomic-write property for - ** blocks of size page-size, and + ** blocks of size page-size, and ** * This commit is not part of a multi-file transaction, and ** * Exactly one page has been modified and store in the journal file. ** @@ -35898,7 +35885,7 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne( ** is not applicable to this transaction, call sqlite3JournalCreate() ** to make sure the journal file has actually been created, then call ** pager_incr_changecounter() to update the change-counter in indirect - ** mode. + ** mode. ** ** Otherwise, if the optimization is both enabled and applicable, ** then call pager_incr_changecounter() to update the change-counter @@ -35908,15 +35895,15 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne( #ifdef SQLITE_ENABLE_ATOMIC_WRITE PgHdr *pPg; assert( isOpen(pPager->jfd) || pPager->journalMode==PAGER_JOURNALMODE_OFF ); - if( !zMaster && isOpen(pPager->jfd) - && pPager->journalOff==jrnlBufferSize(pPager) + if( !zMaster && isOpen(pPager->jfd) + && pPager->journalOff==jrnlBufferSize(pPager) && pPager->dbSize>=pPager->dbFileSize && (0==(pPg = sqlite3PcacheDirtyList(pPager->pPCache)) || 0==pPg->pDirty) ){ - /* Update the db file change counter via the direct-write method. The - ** following call will modify the in-memory representation of page 1 - ** to include the updated change counter and then write page 1 - ** directly to the database file. Because of the atomic-write + /* Update the db file change counter via the direct-write method. The + ** following call will modify the in-memory representation of page 1 + ** to include the updated change counter and then write page 1 + ** directly to the database file. Because of the atomic-write ** property of the host file-system, this is safe. */ rc = pager_incr_changecounter(pPager, 1); @@ -35935,22 +35922,22 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne( ** being discarded by the truncation must be written to the journal ** file. This can only happen in auto-vacuum mode. ** - ** Before reading the pages with page numbers larger than the + ** Before reading the pages with page numbers larger than the ** current value of Pager.dbSize, set dbSize back to the value ** that it took at the start of the transaction. Otherwise, the - ** calls to sqlite3PagerGet() return zeroed pages instead of + ** calls to sqlite3PagerGet() return zeroed pages instead of ** reading data from the database file. ** ** When journal_mode==OFF the dbOrigSize is always zero, so this ** block never runs if journal_mode=OFF. */ #ifndef SQLITE_OMIT_AUTOVACUUM - if( pPager->dbSizedbOrigSize + if( pPager->dbSizedbOrigSize && ALWAYS(pPager->journalMode!=PAGER_JOURNALMODE_OFF) ){ Pgno i; /* Iterator variable */ const Pgno iSkip = PAGER_MJ_PGNO(pPager); /* Pending lock page */ - const Pgno dbSize = pPager->dbSize; /* Database image size */ + const Pgno dbSize = pPager->dbSize; /* Database image size */ pPager->dbSize = pPager->dbOrigSize; for( i=dbSize+1; i<=pPager->dbOrigSize; i++ ){ if( !sqlite3BitvecTest(pPager->pInJournal, i) && i!=iSkip ){ @@ -35961,13 +35948,13 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseOne( sqlite3PagerUnref(pPage); if( rc!=SQLITE_OK ) goto commit_phase_one_exit; } - } + } pPager->dbSize = dbSize; } #endif - /* Write the master journal name into the journal file. If a master - ** journal file name has already been written to the journal file, + /* Write the master journal name into the journal file. If a master + ** journal file name has already been written to the journal file, ** or if zMaster is NULL (no master journal), then this call is a no-op. */ rc = writeMasterJournal(pPager, zMaster); @@ -36015,12 +36002,12 @@ commit_phase_one_exit: /* ** When this function is called, the database file has been completely ** updated to reflect the changes made by the current transaction and -** synced to disk. The journal file still exists in the file-system +** synced to disk. The journal file still exists in the file-system ** though, and if a failure occurs at this point it will eventually ** be used as a hot-journal and the current transaction rolled back. ** -** This function finalizes the journal file, either by deleting, -** truncating or partially zeroing it, so that it cannot be used +** This function finalizes the journal file, either by deleting, +** truncating or partially zeroing it, so that it cannot be used ** for hot-journal rollback. Once this is done the transaction is ** irrevocably committed. ** @@ -36045,14 +36032,14 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseTwo(Pager *pPager){ ** this transaction, the pager is running in exclusive-mode and is ** using persistent journals, then this function is a no-op. ** - ** The start of the journal file currently contains a single journal + ** The start of the journal file currently contains a single journal ** header with the nRec field set to 0. If such a journal is used as ** a hot-journal during hot-journal rollback, 0 changes will be made - ** to the database file. So there is no need to zero the journal + ** to the database file. So there is no need to zero the journal ** header. Since the pager is in exclusive mode, there is no need ** to drop any locks either. */ - if( pPager->dbModified==0 && pPager->exclusiveMode + if( pPager->dbModified==0 && pPager->exclusiveMode && pPager->journalMode==PAGER_JOURNALMODE_PERSIST ){ assert( pPager->journalOff==JOURNAL_HDR_SZ(pPager) ); @@ -36070,7 +36057,7 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseTwo(Pager *pPager){ ** ** This function performs two tasks: ** -** 1) It rolls back the journal file, restoring all database file and +** 1) It rolls back the journal file, restoring all database file and ** in-memory cache pages to the state they were in when the transaction ** was opened, and ** 2) It finalizes the journal file, so that it is not used for hot @@ -36082,24 +36069,24 @@ SQLITE_PRIVATE int sqlite3PagerCommitPhaseTwo(Pager *pPager){ ** then only (2) is performed. In this case there is no journal file ** to roll back. ** -** * If in an error state other than SQLITE_FULL, then task (1) is +** * If in an error state other than SQLITE_FULL, then task (1) is ** performed. If successful, task (2). Regardless of the outcome ** of either, the error state error code is returned to the caller ** (i.e. either SQLITE_IOERR or SQLITE_CORRUPT). ** ** * If the pager is in PAGER_RESERVED state, then attempt (1). Whether ** or not (1) is succussful, also attempt (2). If successful, return -** SQLITE_OK. Otherwise, enter the error state and return the first -** error code encountered. +** SQLITE_OK. Otherwise, enter the error state and return the first +** error code encountered. ** -** In this case there is no chance that the database was written to. -** So is safe to finalize the journal file even if the playback +** In this case there is no chance that the database was written to. +** So is safe to finalize the journal file even if the playback ** (operation 1) failed. However the pager must enter the error state ** as the contents of the in-memory cache are now suspect. ** ** * Finally, if in PAGER_EXCLUSIVE state, then attempt (1). Only ** attempt (2) if (1) is successful. Return SQLITE_OK if successful, -** otherwise enter the error state and return the error code from the +** otherwise enter the error state and return the error code from the ** failing operation. ** ** In this case the database file may have been written to. So if the @@ -36135,7 +36122,7 @@ SQLITE_PRIVATE int sqlite3PagerRollback(Pager *pPager){ } /* If an error occurs during a ROLLBACK, we can no longer trust the pager - ** cache. So call pager_error() on the way out to make any error + ** cache. So call pager_error() on the way out to make any error ** persistent. */ rc = pager_error(pPager, rc); @@ -36199,7 +36186,7 @@ SQLITE_PRIVATE int sqlite3PagerIsMemdb(Pager *pPager){ ** to make up the difference. If the number of savepoints is already ** equal to nSavepoint, then this function is a no-op. ** -** If a memory allocation fails, SQLITE_NOMEM is returned. If an error +** If a memory allocation fails, SQLITE_NOMEM is returned. If an error ** occurs while opening the sub-journal file, then an IO error code is ** returned. Otherwise, SQLITE_OK. */ @@ -36211,13 +36198,13 @@ SQLITE_PRIVATE int sqlite3PagerOpenSavepoint(Pager *pPager, int nSavepoint){ int ii; /* Iterator variable */ PagerSavepoint *aNew; /* New Pager.aSavepoint array */ - /* Either there is no active journal or the sub-journal is open or + /* Either there is no active journal or the sub-journal is open or ** the journal is always stored in memory */ assert( pPager->nSavepoint==0 || isOpen(pPager->sjfd) || pPager->journalMode==PAGER_JOURNALMODE_MEMORY ); /* Grow the Pager.aSavepoint array using realloc(). Return SQLITE_NOMEM - ** if the allocation fails. Otherwise, zero the new portion in case a + ** if the allocation fails. Otherwise, zero the new portion in case a ** malloc failure occurs while populating it in the for(...) loop below. */ aNew = (PagerSavepoint *)sqlite3Realloc( @@ -36256,7 +36243,7 @@ SQLITE_PRIVATE int sqlite3PagerOpenSavepoint(Pager *pPager, int nSavepoint){ /* ** This function is called to rollback or release (commit) a savepoint. -** The savepoint to release or rollback need not be the most recently +** The savepoint to release or rollback need not be the most recently ** created savepoint. ** ** Parameter op is always either SAVEPOINT_ROLLBACK or SAVEPOINT_RELEASE. @@ -36264,26 +36251,26 @@ SQLITE_PRIVATE int sqlite3PagerOpenSavepoint(Pager *pPager, int nSavepoint){ ** index iSavepoint. If it is SAVEPOINT_ROLLBACK, then rollback all changes ** that have occurred since the specified savepoint was created. ** -** The savepoint to rollback or release is identified by parameter +** The savepoint to rollback or release is identified by parameter ** iSavepoint. A value of 0 means to operate on the outermost savepoint ** (the first created). A value of (Pager.nSavepoint-1) means operate ** on the most recently created savepoint. If iSavepoint is greater than ** (Pager.nSavepoint-1), then this function is a no-op. ** ** If a negative value is passed to this function, then the current -** transaction is rolled back. This is different to calling +** transaction is rolled back. This is different to calling ** sqlite3PagerRollback() because this function does not terminate -** the transaction or unlock the database, it just restores the -** contents of the database to its original state. +** the transaction or unlock the database, it just restores the +** contents of the database to its original state. ** -** In any case, all savepoints with an index greater than iSavepoint +** In any case, all savepoints with an index greater than iSavepoint ** are destroyed. If this is a release operation (op==SAVEPOINT_RELEASE), ** then savepoint iSavepoint is also destroyed. ** ** This function may return SQLITE_NOMEM if a memory allocation fails, -** or an IO error code if an IO error occurs while rolling back a +** or an IO error code if an IO error occurs while rolling back a ** savepoint. If no errors occur, SQLITE_OK is returned. -*/ +*/ SQLITE_PRIVATE int sqlite3PagerSavepoint(Pager *pPager, int op, int iSavepoint){ int rc = SQLITE_OK; @@ -36295,7 +36282,7 @@ SQLITE_PRIVATE int sqlite3PagerSavepoint(Pager *pPager, int op, int iSavepoint){ int nNew; /* Number of remaining savepoints after this op. */ /* Figure out how many savepoints will still be active after this - ** operation. Store this value in nNew. Then free resources associated + ** operation. Store this value in nNew. Then free resources associated ** with any savepoints that are destroyed by this operation. */ nNew = iSavepoint + (op==SAVEPOINT_ROLLBACK); @@ -36314,8 +36301,8 @@ SQLITE_PRIVATE int sqlite3PagerSavepoint(Pager *pPager, int op, int iSavepoint){ rc = pagerPlaybackSavepoint(pPager, pSavepoint); assert(rc!=SQLITE_DONE); } - - /* If this is a release of the outermost savepoint, truncate + + /* If this is a release of the outermost savepoint, truncate ** the sub-journal to zero bytes in size. */ if( nNew==0 && op==SAVEPOINT_RELEASE && isOpen(pPager->sjfd) ){ assert( rc==SQLITE_OK ); @@ -36376,7 +36363,7 @@ static void sqlite3PagerSetCodec( void *pCodec ){ if( pPager->xCodecFree ) pPager->xCodecFree(pPager->pCodec); - pPager->xCodec = xCodec; + pPager->xCodec = pPager->memDb ? 0 : xCodec; pPager->xCodecSizeChng = xCodecSizeChng; pPager->xCodecFree = xCodecFree; pPager->pCodec = pCodec; @@ -36406,8 +36393,8 @@ static void *sqlite3PagerGetCodec(Pager *pPager){ ** transaction is active). ** ** If the fourth argument, isCommit, is non-zero, then this page is being -** moved as part of a database reorganization just before the transaction -** is being committed. In this case, it is guaranteed that the database page +** moved as part of a database reorganization just before the transaction +** is being committed. In this case, it is guaranteed that the database page ** pPg refers to will not be written to again within this transaction. ** ** This function may return SQLITE_NOMEM or an IO error code if an error @@ -36421,8 +36408,16 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i assert( pPg->nRef>0 ); + /* In order to be able to rollback, an in-memory database must journal + ** the page we are moving from. + */ + if( MEMDB ){ + rc = sqlite3PagerWrite(pPg); + if( rc ) return rc; + } + /* If the page being moved is dirty and has not been saved by the latest - ** savepoint, then save the current contents of the page into the + ** savepoint, then save the current contents of the page into the ** sub-journal now. This is required to handle the following scenario: ** ** BEGIN; @@ -36439,14 +36434,14 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i ** one or more savepoint bitvecs. This is the reason this function ** may return SQLITE_NOMEM. */ - if( pPg->flags&PGHDR_DIRTY + if( pPg->flags&PGHDR_DIRTY && subjRequiresPage(pPg) && SQLITE_OK!=(rc = subjournalPage(pPg)) ){ return rc; } - PAGERTRACE(("MOVE %d page %d (needSync=%d) moves to %d\n", + PAGERTRACE(("MOVE %d page %d (needSync=%d) moves to %d\n", PAGERID(pPager), pPg->pgno, (pPg->flags&PGHDR_NEED_SYNC)?1:0, pgno)); IOTRACE(("MOVE %p %d %d\n", pPager, pPg->pgno, pgno)) @@ -36454,7 +36449,7 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i ** be written to, store pPg->pgno in local variable needSyncPgno. ** ** If the isCommit flag is set, there is no need to remember that - ** the journal needs to be sync()ed before database page pPg->pgno + ** the journal needs to be sync()ed before database page pPg->pgno ** can be written to. The caller has already promised not to write to it. */ if( (pPg->flags&PGHDR_NEED_SYNC) && !isCommit ){ @@ -36465,8 +36460,8 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i } /* If the cache contains a page with page-number pgno, remove it - ** from its hash chain. Also, if the PgHdr.needSync was set for - ** page pgno before the 'move' operation, it needs to be retained + ** from its hash chain. Also, if the PgHdr.needSync was set for + ** page pgno before the 'move' operation, it needs to be retained ** for the page moved there. */ pPg->flags &= ~PGHDR_NEED_SYNC; @@ -36474,7 +36469,14 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i assert( !pPgOld || pPgOld->nRef==1 ); if( pPgOld ){ pPg->flags |= (pPgOld->flags&PGHDR_NEED_SYNC); - sqlite3PcacheDrop(pPgOld); + if( MEMDB ){ + /* Do not discard pages from an in-memory database since we might + ** need to rollback later. Just move the page out of the way. */ + assert( pPager->dbSizeValid ); + sqlite3PcacheMove(pPgOld, pPager->dbSize+1); + }else{ + sqlite3PcacheDrop(pPgOld); + } } origPgno = pPg->pgno; @@ -36483,11 +36485,11 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i pPager->dbModified = 1; if( needSyncPgno ){ - /* If needSyncPgno is non-zero, then the journal file needs to be + /* If needSyncPgno is non-zero, then the journal file needs to be ** sync()ed before any data is written to database file page needSyncPgno. - ** Currently, no such page exists in the page-cache and the + ** Currently, no such page exists in the page-cache and the ** "is journaled" bitvec flag has been set. This needs to be remedied by - ** loading the page into the pager-cache and setting the PgHdr.needSync + ** loading the page into the pager-cache and setting the PgHdr.needSync ** flag. ** ** If the attempt to load the page into the page-cache fails, (due @@ -36519,18 +36521,12 @@ SQLITE_PRIVATE int sqlite3PagerMovepage(Pager *pPager, DbPage *pPg, Pgno pgno, i /* ** For an in-memory database, make sure the original page continues - ** to exist, in case the transaction needs to roll back. We allocate - ** the page now, instead of at rollback, because we can better deal - ** with an out-of-memory error now. Ticket #3761. + ** to exist, in case the transaction needs to roll back. Use pPgOld + ** as the original page since it has already been allocated. */ if( MEMDB ){ - DbPage *pNew; - rc = sqlite3PagerAcquire(pPager, origPgno, &pNew, 1); - if( rc!=SQLITE_OK ){ - sqlite3PcacheMove(pPg, origPgno); - return rc; - } - sqlite3PagerUnref(pNew); + sqlite3PcacheMove(pPgOld, origPgno); + sqlite3PagerUnref(pPgOld); } return SQLITE_OK; @@ -36546,7 +36542,7 @@ SQLITE_PRIVATE void *sqlite3PagerGetData(DbPage *pPg){ } /* -** Return a pointer to the Pager.nExtra bytes of "extra" space +** Return a pointer to the Pager.nExtra bytes of "extra" space ** allocated along with the specified page. */ SQLITE_PRIVATE void *sqlite3PagerGetExtra(DbPage *pPg){ @@ -36555,7 +36551,7 @@ SQLITE_PRIVATE void *sqlite3PagerGetExtra(DbPage *pPg){ /* ** Get/set the locking-mode for this pager. Parameter eMode must be one -** of PAGER_LOCKINGMODE_QUERY, PAGER_LOCKINGMODE_NORMAL or +** of PAGER_LOCKINGMODE_QUERY, PAGER_LOCKINGMODE_NORMAL or ** PAGER_LOCKINGMODE_EXCLUSIVE. If the parameter is not _QUERY, then ** the locking-mode is set to the value specified. ** @@ -36601,11 +36597,11 @@ SQLITE_PRIVATE int sqlite3PagerJournalMode(Pager *pPager, int eMode){ || eMode==PAGER_JOURNALMODE_DELETE || eMode==PAGER_JOURNALMODE_TRUNCATE || eMode==PAGER_JOURNALMODE_PERSIST - || eMode==PAGER_JOURNALMODE_OFF + || eMode==PAGER_JOURNALMODE_OFF || eMode==PAGER_JOURNALMODE_MEMORY ); assert( PAGER_JOURNALMODE_QUERY<0 ); if( eMode>=0 - && (!MEMDB || eMode==PAGER_JOURNALMODE_MEMORY + && (!MEMDB || eMode==PAGER_JOURNALMODE_MEMORY || eMode==PAGER_JOURNALMODE_OFF) && !pPager->dbModified && (!isOpen(pPager->jfd) || 0==pPager->journalOff) @@ -36657,8 +36653,6 @@ SQLITE_PRIVATE sqlite3_backup **sqlite3PagerBackupPtr(Pager *pPager){ ** ************************************************************************* ** -** $Id: btmutex.c,v 1.17 2009/07/20 12:33:33 drh Exp $ -** ** This file contains code used to implement mutexes on Btree objects. ** This code really belongs in btree.c. But btree.c is getting too ** big and we want to break it down some. This packaged seemed like @@ -36677,8 +36671,6 @@ SQLITE_PRIVATE sqlite3_backup **sqlite3PagerBackupPtr(Pager *pPager){ ** May you share freely, never taking more than you give. ** ************************************************************************* -** $Id: btreeInt.h,v 1.52 2009/07/15 17:25:46 drh Exp $ -** ** This file implements a external (disk-based) database using BTrees. ** For a detailed discussion of BTrees, refer to ** @@ -36699,16 +36691,16 @@ SQLITE_PRIVATE sqlite3_backup **sqlite3PagerBackupPtr(Pager *pPager){ ** on Ptr(N) and its subpages have values greater than Key(N-1). And ** so forth. ** -** Finding a particular key requires reading O(log(M)) pages from the +** Finding a particular key requires reading O(log(M)) pages from the ** disk where M is the number of entries in the tree. ** -** In this implementation, a single file can hold one or more separate +** In this implementation, a single file can hold one or more separate ** BTrees. Each BTree is identified by the index of its root page. The ** key and data for any entry are combined to form the "payload". A ** fixed amount of payload can be carried directly on the database ** page. If the payload is larger than the preset amount then surplus ** bytes are stored on overflow pages. The payload for an entry -** and the preceding pointer are combined to form a "Cell". Each +** and the preceding pointer are combined to form a "Cell". Each ** page has a small header which contains the Ptr(N) pointer and other ** information such as the size of key and data. ** @@ -36726,7 +36718,7 @@ SQLITE_PRIVATE sqlite3_backup **sqlite3PagerBackupPtr(Pager *pPager){ ** ** OFFSET SIZE DESCRIPTION ** 0 16 Header string: "SQLite format 3\000" -** 16 2 Page size in bytes. +** 16 2 Page size in bytes. ** 18 1 File format write version ** 19 1 File format read version ** 20 1 Bytes of unused space at the end of each page @@ -36837,7 +36829,7 @@ SQLITE_PRIVATE sqlite3_backup **sqlite3PagerBackupPtr(Pager *pPager){ ** contiguous or in order, but cell pointers are contiguous and in order. ** ** Cell content makes use of variable length integers. A variable -** length integer is 1 to 9 bytes where the lower 7 bits of each +** length integer is 1 to 9 bytes where the lower 7 bits of each ** byte are used. The integer consists of all bytes that have bit 8 set and ** the first byte with bit 8 clear. The most significant byte of the integer ** appears first. A variable-length integer may not be more than 9 bytes long. @@ -36908,7 +36900,7 @@ typedef struct BtLock BtLock; ** -DSQLITE_FILE_HEADER="..." on the compiler command-line. The ** header must be exactly 16 bytes including the zero-terminator so ** the string itself should be 15 characters long. If you change -** the header, then your custom library will not be able to read +** the header, then your custom library will not be able to read ** databases generated by the standard tools and the standard tools ** will not be able to read databases created by your custom library. */ @@ -36971,7 +36963,7 @@ struct MemPage { /* ** A linked list of the following structures is stored at BtShared.pLock. -** Locks are added (or upgraded from READ_LOCK to WRITE_LOCK) when a cursor +** Locks are added (or upgraded from READ_LOCK to WRITE_LOCK) when a cursor ** is opened on the table with root page BtShared.iTable. Locks are removed ** from this list when a transaction is committed or rolled back, or when ** a btree handle is closed. @@ -36995,7 +36987,7 @@ struct BtLock { ** see the internals of this structure and only deals with pointers to ** this structure. ** -** For some database files, the same underlying database cache might be +** For some database files, the same underlying database cache might be ** shared between multiple connections. In that case, each connection ** has it own instance of this object. But each instance of this object ** points to the same BtShared object. The database cache and the @@ -37003,7 +36995,7 @@ struct BtLock { ** the BtShared object. ** ** All fields in this structure are accessed under sqlite3.mutex. -** The pBt pointer itself may not be changed while there exists cursors +** The pBt pointer itself may not be changed while there exists cursors ** in the referenced BtShared that point back to this Btree since those ** cursors have to do go through this Btree to find their BtShared and ** they often do so without holding sqlite3.mutex. @@ -37036,7 +37028,7 @@ struct Btree { /* ** An instance of this object represents a single database file. -** +** ** A single database file can be in use as the same time by two ** or more database connections. When two or more connections are ** sharing the same database file, each connection has it own @@ -37143,7 +37135,7 @@ struct CellInfo { ** particular database connection identified BtCursor.pBtree.db. ** ** Fields in this structure are accessed under the BtShared.mutex -** found at self->pBt->mutex. +** found at self->pBt->mutex. */ struct BtCursor { Btree *pBtree; /* The Btree to which this cursor belongs */ @@ -37176,14 +37168,14 @@ struct BtCursor { ** Cursor points to a valid entry. getPayload() etc. may be called. ** ** CURSOR_INVALID: -** Cursor does not point to a valid entry. This can happen (for example) +** Cursor does not point to a valid entry. This can happen (for example) ** because the table is empty or because BtreeCursorFirst() has not been ** called. ** ** CURSOR_REQUIRESEEK: -** The table that this cursor was opened on still exists, but has been +** The table that this cursor was opened on still exists, but has been ** modified since the cursor was last used. The cursor position is saved -** in variables BtCursor.pKey and BtCursor.nKey. When a cursor is in +** in variables BtCursor.pKey and BtCursor.nKey. When a cursor is in ** this state, restoreCursorPosition() can be called to attempt to ** seek the cursor to the saved position. ** @@ -37199,13 +37191,13 @@ struct BtCursor { #define CURSOR_REQUIRESEEK 2 #define CURSOR_FAULT 3 -/* +/* ** The database page the PENDING_BYTE occupies. This page is never used. */ # define PENDING_BYTE_PAGE(pBt) PAGER_MJ_PGNO(pBt) /* -** These macros define the location of the pointer-map entry for a +** These macros define the location of the pointer-map entry for a ** database page. The first argument to each is the number of usable ** bytes on each page of the database (often 1024). The second is the ** page number to look up in the pointer map. @@ -37240,10 +37232,10 @@ struct BtCursor { ** PTRMAP_ROOTPAGE: The database page is a root-page. The page-number is not ** used in this case. ** -** PTRMAP_FREEPAGE: The database page is an unused (free) page. The page-number +** PTRMAP_FREEPAGE: The database page is an unused (free) page. The page-number ** is not used in this case. ** -** PTRMAP_OVERFLOW1: The database page is the first page in a list of +** PTRMAP_OVERFLOW1: The database page is the first page in a list of ** overflow pages. The page number identifies the page that ** contains the cell with a pointer to this overflow page. ** @@ -37265,13 +37257,13 @@ struct BtCursor { */ #define btreeIntegrity(p) \ assert( p->pBt->inTransaction!=TRANS_NONE || p->pBt->nTransaction==0 ); \ - assert( p->pBt->inTransaction>=p->inTrans ); + assert( p->pBt->inTransaction>=p->inTrans ); /* ** The ISAUTOVACUUM macro is used within balance_nonroot() to determine ** if the database supports auto-vacuum or not. Because it is used -** within an expression that is an argument to another macro +** within an expression that is an argument to another macro ** (sqliteMallocRaw), it is not possible to use conditional compilation. ** So, this macro is defined instead. */ @@ -37544,7 +37536,7 @@ SQLITE_PRIVATE int sqlite3BtreeHoldsAllMutexes(sqlite3 *db){ #endif /* NDEBUG */ /* -** Add a new Btree pointer to a BtreeMutexArray. +** Add a new Btree pointer to a BtreeMutexArray. ** if the pointer can possibly be shared with ** another database connection. ** @@ -37662,8 +37654,6 @@ SQLITE_PRIVATE void sqlite3BtreeEnterAll(sqlite3 *db){ ** May you share freely, never taking more than you give. ** ************************************************************************* -** $Id: btree.c,v 1.705 2009/08/10 03:57:58 shane Exp $ -** ** This file implements a external (disk-based) database using BTrees. ** See the header comment on "btreeInt.h" for additional information. ** Including a description of file format and an overview of operation. @@ -37692,7 +37682,7 @@ int sqlite3BtreeTrace=1; /* True to enable tracing */ /* ** A list of BtShared objects that are eligible for participation ** in shared cache. This variable has file scope during normal builds, -** but the test harness needs to access it so we make it global for +** but the test harness needs to access it so we make it global for ** test builds. ** ** Access to this variable is protected by SQLITE_MUTEX_STATIC_MASTER. @@ -37727,7 +37717,7 @@ SQLITE_API int sqlite3_enable_shared_cache(int enable){ ** manipulate entries in the BtShared.pLock linked list used to store ** shared-cache table level locks. If the library is compiled with the ** shared-cache feature disabled, then there is only ever one user - ** of each BtShared structure and so this locking is not necessary. + ** of each BtShared structure and so this locking is not necessary. ** So define the lock related functions as no-ops. */ #define querySharedCacheTableLock(a,b,c) SQLITE_OK @@ -37744,15 +37734,15 @@ SQLITE_API int sqlite3_enable_shared_cache(int enable){ /* **** This function is only used as part of an assert() statement. *** ** -** Check to see if pBtree holds the required locks to read or write to the +** Check to see if pBtree holds the required locks to read or write to the ** table with root page iRoot. Return 1 if it does and 0 if not. ** -** For example, when writing to a table with root-page iRoot via +** For example, when writing to a table with root-page iRoot via ** Btree connection pBtree: ** ** assert( hasSharedCacheTableLock(pBtree, iRoot, 0, WRITE_LOCK) ); ** -** When writing to an index that resides in a sharable database, the +** When writing to an index that resides in a sharable database, the ** caller should have first obtained a lock specifying the root page of ** the corresponding table. This makes things a bit more complicated, ** as this module treats each table as a separate structure. To determine @@ -37774,7 +37764,7 @@ static int hasSharedCacheTableLock( BtLock *pLock; /* If this database is not shareable, or if the client is reading - ** and has the read-uncommitted flag set, then no lock is required. + ** and has the read-uncommitted flag set, then no lock is required. ** Return true immediately. */ if( (pBtree->sharable==0) @@ -37808,13 +37798,13 @@ static int hasSharedCacheTableLock( iTab = iRoot; } - /* Search for the required lock. Either a write-lock on root-page iTab, a + /* Search for the required lock. Either a write-lock on root-page iTab, a ** write-lock on the schema table, or (if the client is reading) a ** read-lock on iTab will suffice. Return 1 if any of these are found. */ for(pLock=pBtree->pBt->pLock; pLock; pLock=pLock->pNext){ - if( pLock->pBtree==pBtree + if( pLock->pBtree==pBtree && (pLock->iTable==iTab || (pLock->eLock==WRITE_LOCK && pLock->iTable==1)) - && pLock->eLock>=eLockType + && pLock->eLock>=eLockType ){ return 1; } @@ -37847,7 +37837,7 @@ static int hasSharedCacheTableLock( static int hasReadConflicts(Btree *pBtree, Pgno iRoot){ BtCursor *p; for(p=pBtree->pBt->pCursor; p; p=p->pNext){ - if( p->pgnoRoot==iRoot + if( p->pgnoRoot==iRoot && p->pBtree!=pBtree && 0==(p->pBtree->db->flags & SQLITE_ReadUncommitted) ){ @@ -37859,7 +37849,7 @@ static int hasReadConflicts(Btree *pBtree, Pgno iRoot){ #endif /* #ifdef SQLITE_DEBUG */ /* -** Query to see if Btree handle p may obtain a lock of type eLock +** Query to see if Btree handle p may obtain a lock of type eLock ** (READ_LOCK or WRITE_LOCK) on the table with root-page iTab. Return ** SQLITE_OK if the lock may be obtained (by calling ** setSharedCacheTableLock()), or SQLITE_LOCKED if not. @@ -37872,14 +37862,14 @@ static int querySharedCacheTableLock(Btree *p, Pgno iTab, u8 eLock){ assert( eLock==READ_LOCK || eLock==WRITE_LOCK ); assert( p->db!=0 ); assert( !(p->db->flags&SQLITE_ReadUncommitted)||eLock==WRITE_LOCK||iTab==1 ); - + /* If requesting a write-lock, then the Btree must have an open write - ** transaction on this file. And, obviously, for this to be so there + ** transaction on this file. And, obviously, for this to be so there ** must be an open write transaction on the file itself. */ assert( eLock==READ_LOCK || (p==pBt->pWriter && p->inTrans==TRANS_WRITE) ); assert( eLock==READ_LOCK || pBt->inTransaction==TRANS_WRITE ); - + /* This routine is a no-op if the shared-cache is not enabled */ if( !p->sharable ){ return SQLITE_OK; @@ -37894,7 +37884,7 @@ static int querySharedCacheTableLock(Btree *p, Pgno iTab, u8 eLock){ } for(pIter=pBt->pLock; pIter; pIter=pIter->pNext){ - /* The condition (pIter->eLock!=eLock) in the following if(...) + /* The condition (pIter->eLock!=eLock) in the following if(...) ** statement is a simplification of: ** ** (eLock==WRITE_LOCK || pIter->eLock==WRITE_LOCK) @@ -37921,7 +37911,7 @@ static int querySharedCacheTableLock(Btree *p, Pgno iTab, u8 eLock){ #ifndef SQLITE_OMIT_SHARED_CACHE /* ** Add a lock on the table with root-page iTable to the shared-btree used -** by Btree handle p. Parameter eLock must be either READ_LOCK or +** by Btree handle p. Parameter eLock must be either READ_LOCK or ** WRITE_LOCK. ** ** This function assumes the following: @@ -37933,7 +37923,7 @@ static int querySharedCacheTableLock(Btree *p, Pgno iTab, u8 eLock){ ** with the requested lock (i.e. querySharedCacheTableLock() has ** already been called and returned SQLITE_OK). ** -** SQLITE_OK is returned if the lock is added successfully. SQLITE_NOMEM +** SQLITE_OK is returned if the lock is added successfully. SQLITE_NOMEM ** is returned if a malloc attempt fails. */ static int setSharedCacheTableLock(Btree *p, Pgno iTable, u8 eLock){ @@ -37947,11 +37937,11 @@ static int setSharedCacheTableLock(Btree *p, Pgno iTable, u8 eLock){ /* A connection with the read-uncommitted flag set will never try to ** obtain a read-lock using this function. The only read-lock obtained - ** by a connection in read-uncommitted mode is on the sqlite_master + ** by a connection in read-uncommitted mode is on the sqlite_master ** table, and that lock is obtained in BtreeBeginTrans(). */ assert( 0==(p->db->flags&SQLITE_ReadUncommitted) || eLock==WRITE_LOCK ); - /* This function should only be called on a sharable b-tree after it + /* This function should only be called on a sharable b-tree after it ** has been determined that no other b-tree holds a conflicting lock. */ assert( p->sharable ); assert( SQLITE_OK==querySharedCacheTableLock(p, iTable, eLock) ); @@ -37996,7 +37986,7 @@ static int setSharedCacheTableLock(Btree *p, Pgno iTable, u8 eLock){ ** Release all the table locks (locks obtained via calls to ** the setSharedCacheTableLock() procedure) held by Btree object p. ** -** This function assumes that Btree p has an open read or write +** This function assumes that Btree p has an open read or write ** transaction. If it does not, then the BtShared.isPending variable ** may be incorrectly cleared. */ @@ -38029,7 +38019,7 @@ static void clearAllSharedCacheTableLocks(Btree *p){ pBt->isExclusive = 0; pBt->isPending = 0; }else if( pBt->nTransaction==2 ){ - /* This function is called when Btree p is concluding its + /* This function is called when Btree p is concluding its ** transaction. If there currently exists a writer, and p is not ** that writer, then the number of locks held by connections other ** than the writer must be about to drop to zero. In this case @@ -38133,8 +38123,8 @@ static void invalidateIncrblobCursors( #endif /* SQLITE_OMIT_INCRBLOB */ /* -** Set bit pgno of the BtShared.pHasContent bitvec. This is called -** when a page that previously contained data becomes a free-list leaf +** Set bit pgno of the BtShared.pHasContent bitvec. This is called +** when a page that previously contained data becomes a free-list leaf ** page. ** ** The BtShared.pHasContent bitvec exists to work around an obscure @@ -38160,7 +38150,7 @@ static void invalidateIncrblobCursors( ** may be lost. In the event of a rollback, it may not be possible ** to restore the database to its original configuration. ** -** The solution is the BtShared.pHasContent bitvec. Whenever a page is +** The solution is the BtShared.pHasContent bitvec. Whenever a page is ** moved to become a free-list leaf page, the corresponding bit is ** set in the bitvec. Whenever a leaf page is extracted from the free-list, ** optimization 2 above is omitted if the corresponding bit is already @@ -38207,11 +38197,11 @@ static void btreeClearHasContent(BtShared *pBt){ } /* -** Save the current cursor position in the variables BtCursor.nKey +** Save the current cursor position in the variables BtCursor.nKey ** and BtCursor.pKey. The cursor's state is set to CURSOR_REQUIRESEEK. ** ** The caller must ensure that the cursor is valid (has eState==CURSOR_VALID) -** prior to calling this routine. +** prior to calling this routine. */ static int saveCursorPosition(BtCursor *pCur){ int rc; @@ -38226,7 +38216,7 @@ static int saveCursorPosition(BtCursor *pCur){ /* If this is an intKey table, then the above call to BtreeKeySize() ** stores the integer key in pCur->nKey. In this case this value is ** all that is required. Otherwise, if pCur is not open on an intKey - ** table, then malloc space for and store the pCur->nKey bytes of key + ** table, then malloc space for and store the pCur->nKey bytes of key ** data. */ if( 0==pCur->apPage[0]->intKey ){ @@ -38268,7 +38258,7 @@ static int saveAllCursors(BtShared *pBt, Pgno iRoot, BtCursor *pExcept){ assert( sqlite3_mutex_held(pBt->mutex) ); assert( pExcept==0 || pExcept->pBt==pBt ); for(p=pBt->pCursor; p; p=p->pNext){ - if( p!=pExcept && (0==iRoot || p->pgnoRoot==iRoot) && + if( p!=pExcept && (0==iRoot || p->pgnoRoot==iRoot) && p->eState==CURSOR_VALID ){ int rc = saveCursorPosition(p); if( SQLITE_OK!=rc ){ @@ -38322,9 +38312,9 @@ static int btreeMoveto( /* ** Restore the cursor to the position it was in (or as close to as possible) -** when saveCursorPosition() was called. Note that this call deletes the +** when saveCursorPosition() was called. Note that this call deletes the ** saved position info stored by saveCursorPosition(), so there can be -** at most one effective restoreCursorPosition() call after each +** at most one effective restoreCursorPosition() call after each ** saveCursorPosition(). */ static int btreeRestoreCursorPosition(BtCursor *pCur){ @@ -38385,7 +38375,7 @@ static Pgno ptrmapPageno(BtShared *pBt, Pgno pgno){ assert( sqlite3_mutex_held(pBt->mutex) ); nPagesPerMapPage = (pBt->usableSize/5)+1; iPtrMap = (pgno-2)/nPagesPerMapPage; - ret = (iPtrMap*nPagesPerMapPage) + 2; + ret = (iPtrMap*nPagesPerMapPage) + 2; if( ret==PENDING_BYTE_PAGE(pBt) ){ ret++; } @@ -38519,8 +38509,8 @@ static u8 *findOverflowCell(MemPage *pPage, int iCell){ /* ** Parse a cell content block and fill in the CellInfo structure. There -** are two versions of this function. btreeParseCell() takes a -** cell index as the second argument and btreeParseCellPtr() +** are two versions of this function. btreeParseCell() takes a +** cell index as the second argument and btreeParseCellPtr() ** takes a pointer to the body of the cell as its second argument. ** ** Within this file, the parseCell() macro can be called instead of @@ -38634,7 +38624,7 @@ static u16 cellSizePtr(MemPage *pPage, u8 *pCell){ nSize = 0; } - /* pIter now points at the 64-bit integer key value, a variable length + /* pIter now points at the 64-bit integer key value, a variable length ** integer. The following block moves pIter to point at the first byte ** past the end of the key value. */ pEnd = &pIter[9]; @@ -38740,7 +38730,7 @@ static int defragmentPage(MemPage *pPage){ testcase( pc==iCellLast ); #if !defined(SQLITE_ENABLE_OVERSIZE_CELL_CHECK) /* These conditions have already been verified in btreeInitPage() - ** if SQLITE_ENABLE_OVERSIZE_CELL_CHECK is defined + ** if SQLITE_ENABLE_OVERSIZE_CELL_CHECK is defined */ if( pciCellLast ){ return SQLITE_CORRUPT_BKPT; @@ -38797,14 +38787,16 @@ static int allocateSpace(MemPage *pPage, int nByte, int *pIdx){ int top; /* First byte of cell content area */ int gap; /* First byte of gap between cell pointers and cell content */ int rc; /* Integer return code */ - + int usableSize; /* Usable size of the page */ + assert( sqlite3PagerIswriteable(pPage->pDbPage) ); assert( pPage->pBt ); assert( sqlite3_mutex_held(pPage->pBt->mutex) ); assert( nByte>=0 ); /* Minimum cell size is 4 */ assert( pPage->nFree>=nByte ); assert( pPage->nOverflow==0 ); - assert( nBytepBt->usableSize-8 ); + usableSize = pPage->pBt->usableSize; + assert( nByte < usableSize-8 ); nFrag = data[hdr+7]; assert( pPage->cellOffset == hdr + 12 - 4*pPage->leaf ); @@ -38821,13 +38813,17 @@ static int allocateSpace(MemPage *pPage, int nByte, int *pIdx){ if( rc ) return rc; top = get2byte(&data[hdr+5]); }else if( gap+2<=top ){ - /* Search the freelist looking for a free slot big enough to satisfy - ** the request. The allocation is made from the first free slot in + /* Search the freelist looking for a free slot big enough to satisfy + ** the request. The allocation is made from the first free slot in ** the list that is large enough to accomadate it. */ int pc, addr; for(addr=hdr+1; (pc = get2byte(&data[addr]))>0; addr=pc){ - int size = get2byte(&data[pc+2]); /* Size of free slot */ + int size; /* Size of the free slot */ + if( pc>usableSize-4 || pc=nByte ){ int x = size - nByte; testcase( x==4 ); @@ -38837,6 +38833,8 @@ static int allocateSpace(MemPage *pPage, int nByte, int *pIdx){ ** fragmented bytes within the page. */ memcpy(&data[addr], &data[pc], 2); data[hdr+7] = (u8)(nFrag + x); + }else if( size+pc > usableSize ){ + return SQLITE_CORRUPT_BKPT; }else{ /* The slot remains on the free-list. Reduce its size to account ** for the portion used by the new allocation. */ @@ -38894,7 +38892,7 @@ static int freeSpace(MemPage *pPage, int start, int size){ assert( size>=0 ); /* Minimum cell size is 4 */ #ifdef SQLITE_SECURE_DELETE - /* Overwrite deleted information with zeros when the SECURE_DELETE + /* Overwrite deleted information with zeros when the SECURE_DELETE ** option is enabled at compile-time */ memset(&data[start], 0, size); #endif @@ -39003,7 +39001,7 @@ static int decodeFlags(MemPage *pPage, int flagByte){ ** Initialize the auxiliary information for a disk block. ** ** Return SQLITE_OK on success. If we see that the page does -** not contain a well-formed database page, then return +** not contain a well-formed database page, then return ** SQLITE_CORRUPT. Note that a return of SQLITE_OK does not ** guarantee that the page is well-formed. It only shows that ** we failed to detect any corruption. @@ -39047,10 +39045,10 @@ static int btreeInitPage(MemPage *pPage){ testcase( pPage->nCell==MX_CELL(pBt) ); /* A malformed database page might cause us to read past the end - ** of page when parsing a cell. + ** of page when parsing a cell. ** ** The following block of code checks early to see if a cell extends - ** past the end of a page boundary and causes SQLITE_CORRUPT to be + ** past the end of a page boundary and causes SQLITE_CORRUPT to be ** returned if it does. */ iCellFirst = cellOffset + 2*pPage->nCell; @@ -39075,7 +39073,7 @@ static int btreeInitPage(MemPage *pPage){ } } if( !pPage->leaf ) iCellLast++; - } + } #endif /* Compute the total free space on the page */ @@ -39085,14 +39083,14 @@ static int btreeInitPage(MemPage *pPage){ u16 next, size; if( pciCellLast ){ /* Start of free block is off the page */ - return SQLITE_CORRUPT_BKPT; + return SQLITE_CORRUPT_BKPT; } next = get2byte(&data[pc]); size = get2byte(&data[pc+2]); if( (next>0 && next<=pc+size+3) || pc+size>usableSize ){ /* Free blocks must be in ascending order. And the last byte of ** the free-block must lie on the database page. */ - return SQLITE_CORRUPT_BKPT; + return SQLITE_CORRUPT_BKPT; } nFree = nFree + size; pc = next; @@ -39106,7 +39104,7 @@ static int btreeInitPage(MemPage *pPage){ ** area, according to the page header, lies within the page. */ if( nFree>usableSize ){ - return SQLITE_CORRUPT_BKPT; + return SQLITE_CORRUPT_BKPT; } pPage->nFree = (u16)(nFree - iCellFirst); pPage->isInit = 1; @@ -39129,7 +39127,9 @@ static void zeroPage(MemPage *pPage, int flags){ assert( sqlite3PagerGetData(pPage->pDbPage) == data ); assert( sqlite3PagerIswriteable(pPage->pDbPage) ); assert( sqlite3_mutex_held(pBt->mutex) ); - /*memset(&data[hdr], 0, pBt->usableSize - hdr);*/ +#ifdef SQLITE_SECURE_DELETE + memset(&data[hdr], 0, pBt->usableSize - hdr); +#endif data[hdr] = (char)flags; first = hdr + 8 + 4*((flags&PTF_LEAF)==0 ?1:0); memset(&data[hdr+1], 0, 4); @@ -39158,7 +39158,7 @@ static MemPage *btreePageFromDbPage(DbPage *pDbPage, Pgno pgno, BtShared *pBt){ pPage->pBt = pBt; pPage->pgno = pgno; pPage->hdrOffset = pPage->pgno==1 ? 100 : 0; - return pPage; + return pPage; } /* @@ -39218,7 +39218,7 @@ static Pgno pagerPagecount(BtShared *pBt){ /* ** Get a page from the pager and initialize it. This routine is just a -** convenience wrapper around separate calls to btreeGetPage() and +** convenience wrapper around separate calls to btreeGetPage() and ** btreeInitPage(). ** ** If an error occurs, then the value *ppPage is set to is undefined. It @@ -39258,7 +39258,6 @@ static int getAndInitPage( */ static void releasePage(MemPage *pPage){ if( pPage ){ - assert( pPage->nOverflow==0 || sqlite3PagerPageRefcount(pPage->pDbPage)>1 ); assert( pPage->aData ); assert( pPage->pBt ); assert( sqlite3PagerGetExtra(pPage->pDbPage) == (void*)pPage ); @@ -39307,7 +39306,7 @@ static int btreeInvokeBusyHandler(void *pArg){ /* ** Open a database file. -** +** ** zFilename is the name of the database file. If zFilename is NULL ** a new database with a random name is created. This randomly named ** database file will be deleted when sqlite3BtreeClose() is called. @@ -39335,9 +39334,9 @@ SQLITE_PRIVATE int sqlite3BtreeOpen( u8 nReserve; /* Byte of unused space on each page */ unsigned char zDbHeader[100]; /* Database header content */ - /* Set the variable isMemdb to true for an in-memory database, or + /* Set the variable isMemdb to true for an in-memory database, or ** false for a file-based database. This symbol is only required if - ** either of the shared-data or autovacuum features are compiled + ** either of the shared-data or autovacuum features are compiled ** into the library. */ #if !defined(SQLITE_OMIT_SHARED_CACHE) || !defined(SQLITE_OMIT_AUTOVACUUM) @@ -39429,7 +39428,7 @@ SQLITE_PRIVATE int sqlite3BtreeOpen( assert( sizeof(u32)==4 ); assert( sizeof(u16)==2 ); assert( sizeof(Pgno)==4 ); - + pBt = sqlite3MallocZero( sizeof(*pBt) ); if( pBt==0 ){ rc = SQLITE_NOMEM; @@ -39446,7 +39445,7 @@ SQLITE_PRIVATE int sqlite3BtreeOpen( pBt->db = db; sqlite3PagerSetBusyhandler(pBt->pPager, btreeInvokeBusyHandler, pBt); p->pBt = pBt; - + pBt->pCursor = 0; pBt->pPage1 = 0; pBt->readOnly = sqlite3PagerIsreadonly(pBt->pPager); @@ -39479,7 +39478,7 @@ SQLITE_PRIVATE int sqlite3BtreeOpen( if( rc ) goto btree_open_out; pBt->usableSize = pBt->pageSize - nReserve; assert( (pBt->pageSize & 7)==0 ); /* 8-byte alignment of pageSize */ - + #if !defined(SQLITE_OMIT_SHARED_CACHE) && !defined(SQLITE_OMIT_DISKIO) /* Add the new BtShared object to the linked list sharable BtShareds. */ @@ -39593,7 +39592,7 @@ static int removeFromSharingList(BtShared *pBt){ } /* -** Make sure pBt->pTmpSpace points to an allocation of +** Make sure pBt->pTmpSpace points to an allocation of ** MX_CELL_SIZE(pBt) bytes. */ static void allocateTempSpace(BtShared *pBt){ @@ -39637,7 +39636,7 @@ SQLITE_PRIVATE int sqlite3BtreeClose(Btree *p){ sqlite3BtreeLeave(p); /* If there are still other outstanding references to the shared-btree - ** structure, return now. The remainder of this procedure cleans + ** structure, return now. The remainder of this procedure cleans ** up the shared-btree. */ assert( p->wantToLock==0 && p->locked==0 ); @@ -39718,7 +39717,7 @@ SQLITE_PRIVATE int sqlite3BtreeSetSafetyLevel(Btree *p, int level, int fullSync) SQLITE_PRIVATE int sqlite3BtreeSyncDisabled(Btree *p){ BtShared *pBt = p->pBt; int rc; - assert( sqlite3_mutex_held(p->db->mutex) ); + assert( sqlite3_mutex_held(p->db->mutex) ); sqlite3BtreeEnter(p); assert( pBt && pBt->pPager ); rc = sqlite3PagerNosync(pBt->pPager); @@ -39729,7 +39728,7 @@ SQLITE_PRIVATE int sqlite3BtreeSyncDisabled(Btree *p){ #if !defined(SQLITE_OMIT_PAGER_PRAGMAS) || !defined(SQLITE_OMIT_VACUUM) /* ** Change the default pages size and the number of reserved bytes per page. -** Or, if the page size has already been fixed, return SQLITE_READONLY +** Or, if the page size has already been fixed, return SQLITE_READONLY ** without changing anything. ** ** The page size must be a power of 2 between 512 and 65536. If the page @@ -39811,7 +39810,7 @@ SQLITE_PRIVATE int sqlite3BtreeMaxPageCount(Btree *p, int mxPage){ /* ** Change the 'auto-vacuum' property of the database. If the 'autoVacuum' ** parameter is non-zero, then auto-vacuum mode is enabled. If zero, it -** is disabled. The default value for the auto-vacuum property is +** is disabled. The default value for the auto-vacuum property is ** determined by the SQLITE_DEFAULT_AUTOVACUUM macro. */ SQLITE_PRIVATE int sqlite3BtreeSetAutoVacuum(Btree *p, int autoVacuum){ @@ -39835,7 +39834,7 @@ SQLITE_PRIVATE int sqlite3BtreeSetAutoVacuum(Btree *p, int autoVacuum){ } /* -** Return the value of the 'auto-vacuum' property. If auto-vacuum is +** Return the value of the 'auto-vacuum' property. If auto-vacuum is ** enabled 1 is returned. Otherwise 0. */ SQLITE_PRIVATE int sqlite3BtreeGetAutoVacuum(Btree *p){ @@ -39862,7 +39861,7 @@ SQLITE_PRIVATE int sqlite3BtreeGetAutoVacuum(Btree *p){ ** SQLITE_OK is returned on success. If the file is not a ** well-formed database file, then SQLITE_CORRUPT is returned. ** SQLITE_BUSY is returned if the database is locked. SQLITE_NOMEM -** is returned if we run out of memory. +** is returned if we run out of memory. */ static int lockBtree(BtShared *pBt){ int rc; @@ -39877,7 +39876,7 @@ static int lockBtree(BtShared *pBt){ if( rc!=SQLITE_OK ) return rc; /* Do some checking to help insure the file we opened really is - ** a valid database file. + ** a valid database file. */ rc = sqlite3PagerPagecount(pBt->pPager, &nPage); if( rc!=SQLITE_OK ){ @@ -39969,7 +39968,7 @@ page1_init_failed: /* ** If there are no outstanding cursors and we are not in the middle ** of a transaction but there is a read lock on the database, then -** this routine unrefs the first page of the database file which +** this routine unrefs the first page of the database file which ** has the effect of releasing the read lock. ** ** If there is a transaction in progress, this routine is a no-op. @@ -39998,11 +39997,8 @@ static int newDatabase(BtShared *pBt){ int nPage; assert( sqlite3_mutex_held(pBt->mutex) ); - /* The database size has already been measured and cached, so failure - ** is impossible here. If the original size measurement failed, then - ** processing aborts before entering this routine. */ rc = sqlite3PagerPagecount(pBt->pPager, &nPage); - if( NEVER(rc!=SQLITE_OK) || nPage>0 ){ + if( rc!=SQLITE_OK || nPage>0 ){ return rc; } pP1 = pBt->pPage1; @@ -40041,8 +40037,8 @@ static int newDatabase(BtShared *pBt){ ** upgraded to exclusive by calling this routine a second time - the ** exclusivity flag only works for a new transaction. ** -** A write-transaction must be started before attempting any -** changes to the database. None of the following routines +** A write-transaction must be started before attempting any +** changes to the database. None of the following routines ** will work unless a transaction is started first: ** ** sqlite3BtreeCreateTable() @@ -40056,7 +40052,7 @@ static int newDatabase(BtShared *pBt){ ** If an initial attempt to acquire the lock fails because of lock contention ** and the database was previously unlocked, then invoke the busy handler ** if there is one. But if there was previously a read-lock, do not -** invoke the busy handler - just return SQLITE_BUSY. SQLITE_BUSY is +** invoke the busy handler - just return SQLITE_BUSY. SQLITE_BUSY is ** returned when there is already a read-lock in order to avoid a deadlock. ** ** Suppose there are two processes A and B. A has a read lock and B has @@ -40090,7 +40086,7 @@ SQLITE_PRIVATE int sqlite3BtreeBeginTrans(Btree *p, int wrflag){ } #ifndef SQLITE_OMIT_SHARED_CACHE - /* If another database handle has already opened a write transaction + /* If another database handle has already opened a write transaction ** on this shared-btree structure and a second write transaction is ** requested, return SQLITE_LOCKED. */ @@ -40112,8 +40108,8 @@ SQLITE_PRIVATE int sqlite3BtreeBeginTrans(Btree *p, int wrflag){ } #endif - /* Any read-only or read-write transaction implies a read-lock on - ** page 1. So if some other shared-cache client already has a write-lock + /* Any read-only or read-write transaction implies a read-lock on + ** page 1. So if some other shared-cache client already has a write-lock ** on page 1, the transaction cannot be opened. */ rc = querySharedCacheTableLock(p, MASTER_ROOT, READ_LOCK); if( SQLITE_OK!=rc ) goto trans_begun; @@ -40122,7 +40118,7 @@ SQLITE_PRIVATE int sqlite3BtreeBeginTrans(Btree *p, int wrflag){ /* Call lockBtree() until either pBt->pPage1 is populated or ** lockBtree() returns something other than SQLITE_OK. lockBtree() ** may return SQLITE_OK but leave pBt->pPage1 set to 0 if after - ** reading page 1 it discovers that the page-size of the database + ** reading page 1 it discovers that the page-size of the database ** file is not pBt->pageSize. In this case lockBtree() will update ** pBt->pageSize to the page-size of the file on disk. */ @@ -40138,7 +40134,7 @@ SQLITE_PRIVATE int sqlite3BtreeBeginTrans(Btree *p, int wrflag){ } } } - + if( rc!=SQLITE_OK ){ unlockBtreeIfUnused(pBt); } @@ -40233,7 +40229,7 @@ set_child_ptrmaps_out: ** that it points to iTo. Parameter eType describes the type of pointer to ** be modified, as follows: ** -** PTRMAP_BTREE: pPage is a btree-page. The pointer points at a child +** PTRMAP_BTREE: pPage is a btree-page. The pointer points at a child ** page of pPage. ** ** PTRMAP_OVERFLOW1: pPage is a btree-page. The pointer points at an overflow @@ -40277,9 +40273,9 @@ static int modifyPagePointer(MemPage *pPage, Pgno iFrom, Pgno iTo, u8 eType){ } } } - + if( i==nCell ){ - if( eType!=PTRMAP_BTREE || + if( eType!=PTRMAP_BTREE || get4byte(&pPage->aData[pPage->hdrOffset+8])!=iFrom ){ return SQLITE_CORRUPT_BKPT; } @@ -40293,11 +40289,11 @@ static int modifyPagePointer(MemPage *pPage, Pgno iFrom, Pgno iTo, u8 eType){ /* -** Move the open database page pDbPage to location iFreePage in the +** Move the open database page pDbPage to location iFreePage in the ** database. The pDbPage reference remains valid. ** ** The isCommit flag indicates that there is no need to remember that -** the journal needs to be sync()ed before database page pDbPage->pgno +** the journal needs to be sync()ed before database page pDbPage->pgno ** can be written to. The caller has already promised not to write to that ** page. */ @@ -40314,13 +40310,13 @@ static int relocatePage( Pager *pPager = pBt->pPager; int rc; - assert( eType==PTRMAP_OVERFLOW2 || eType==PTRMAP_OVERFLOW1 || + assert( eType==PTRMAP_OVERFLOW2 || eType==PTRMAP_OVERFLOW1 || eType==PTRMAP_BTREE || eType==PTRMAP_ROOTPAGE ); assert( sqlite3_mutex_held(pBt->mutex) ); assert( pDbPage->pBt==pBt ); /* Move page iDbPage from its current location to page number iFreePage */ - TRACE(("AUTOVACUUM: Moving %d to free page %d (ptr page %d type %d)\n", + TRACE(("AUTOVACUUM: Moving %d to free page %d (ptr page %d type %d)\n", iDbPage, iFreePage, iPtrPage, eType)); rc = sqlite3PagerMovepage(pPager, pDbPage->pDbPage, iFreePage, isCommit); if( rc!=SQLITE_OK ){ @@ -40382,14 +40378,14 @@ static int allocateBtreePage(BtShared *, MemPage **, Pgno *, Pgno, u8); ** return SQLITE_OK. If there is no work to do (and therefore no ** point in calling this function again), return SQLITE_DONE. ** -** More specificly, this function attempts to re-organize the +** More specificly, this function attempts to re-organize the ** database so that the last page of the file currently in use ** is no longer in use. ** ** If the nFin parameter is non-zero, this function assumes ** that the caller will keep calling incrVacuumStep() until ** it returns SQLITE_DONE or an error, and that nFin is the -** number of pages the database file will contain after this +** number of pages the database file will contain after this ** process is complete. If nFin is zero, it is assumed that ** incrVacuumStep() will be called a finite amount of times ** which may or may not empty the freelist. A full autovacuum @@ -40423,7 +40419,7 @@ static int incrVacuumStep(BtShared *pBt, Pgno nFin, Pgno iLastPg){ if( nFin==0 ){ /* Remove the page from the files free-list. This is not required ** if nFin is non-zero. In that case, the free-list will be - ** truncated to zero after this function returns, so it doesn't + ** truncated to zero after this function returns, so it doesn't ** matter if it still contains some garbage entries. */ Pgno iFreePg; @@ -40461,7 +40457,7 @@ static int incrVacuumStep(BtShared *pBt, Pgno nFin, Pgno iLastPg){ releasePage(pFreePg); }while( nFin!=0 && iFreePg>nFin ); assert( iFreePgpDbPage); if( rc==SQLITE_OK ){ rc = relocatePage(pBt, pLastPg, eType, iPtrPage, iFreePg, nFin!=0); @@ -40501,7 +40497,7 @@ static int incrVacuumStep(BtShared *pBt, Pgno nFin, Pgno iLastPg){ ** ** If the incremental vacuum is finished after this function has run, ** SQLITE_DONE is returned. If it is not finished, but no error occurred, -** SQLITE_OK is returned. Otherwise an SQLite error code. +** SQLITE_OK is returned. Otherwise an SQLite error code. */ SQLITE_PRIVATE int sqlite3BtreeIncrVacuum(Btree *p){ int rc; @@ -40524,7 +40520,7 @@ SQLITE_PRIVATE int sqlite3BtreeIncrVacuum(Btree *p){ ** is commited for an auto-vacuum database. ** ** If SQLITE_OK is returned, then *pnTrunc is set to the number of pages -** the database file should be truncated to during the commit process. +** the database file should be truncated to during the commit process. ** i.e. the database has been reorganized so that only the first *pnTrunc ** pages are in use. */ @@ -40605,7 +40601,7 @@ static int autoVacuumCommit(BtShared *pBt){ ** ** Otherwise, sync the database file for the btree pBt. zMaster points to ** the name of a master journal file that should be written into the -** individual journal file, or is NULL, indicating no master journal file +** individual journal file, or is NULL, indicating no master journal file ** (single database transaction). ** ** When this is called, the master journal should already have been @@ -40650,8 +40646,8 @@ static void btreeEndTransaction(Btree *p){ downgradeAllSharedCacheTableLocks(p); p->inTrans = TRANS_READ; }else{ - /* If the handle had any kind of transaction open, decrement the - ** transaction count of the shared btree. If the transaction count + /* If the handle had any kind of transaction open, decrement the + ** transaction count of the shared btree. If the transaction count ** reaches 0, set the shared state to TRANS_NONE. The unlockBtreeIfUnused() ** call below will unlock the pager. */ if( p->inTrans!=TRANS_NONE ){ @@ -40662,7 +40658,7 @@ static void btreeEndTransaction(Btree *p){ } } - /* Set the current transaction state to TRANS_NONE and unlock the + /* Set the current transaction state to TRANS_NONE and unlock the ** pager if this call closed the only read or write transaction. */ p->inTrans = TRANS_NONE; unlockBtreeIfUnused(pBt); @@ -40692,7 +40688,7 @@ SQLITE_PRIVATE int sqlite3BtreeCommitPhaseTwo(Btree *p){ sqlite3BtreeEnter(p); btreeIntegrity(p); - /* If the handle has a write-transaction open, commit the shared-btrees + /* If the handle has a write-transaction open, commit the shared-btrees ** transaction and set the shared state to TRANS_READ. */ if( p->inTrans==TRANS_WRITE ){ @@ -40741,7 +40737,7 @@ static int countWriteCursors(BtShared *pBt){ BtCursor *pCur; int r = 0; for(pCur=pBt->pCursor; pCur; pCur=pCur->pNext){ - if( pCur->wrFlag && pCur->eState!=CURSOR_FAULT ) r++; + if( pCur->wrFlag && pCur->eState!=CURSOR_FAULT ) r++; } return r; } @@ -40800,9 +40796,9 @@ SQLITE_PRIVATE int sqlite3BtreeRollback(Btree *p){ if( rc!=SQLITE_OK ){ /* This is a horrible situation. An IO or malloc() error occurred whilst ** trying to save cursor positions. If this is an automatic rollback (as - ** the result of a constraint, malloc() failure or IO error) then + ** the result of a constraint, malloc() failure or IO error) then ** the cache may be internally inconsistent (not contain valid trees) so - ** we cannot simply return the error to the caller. Instead, abort + ** we cannot simply return the error to the caller. Instead, abort ** all queries that may be using any of the cursors that failed to save. */ sqlite3BtreeTripAllCursors(p, rc); @@ -40836,8 +40832,8 @@ SQLITE_PRIVATE int sqlite3BtreeRollback(Btree *p){ /* ** Start a statement subtransaction. The subtransaction can can be rolled -** back independently of the main transaction. You must start a transaction -** before starting a subtransaction. The subtransaction is ended automatically +** back independently of the main transaction. You must start a transaction +** before starting a subtransaction. The subtransaction is ended automatically ** if the main transaction commits or rolls back. ** ** Statement subtransactions are used around individual SQL statements @@ -40878,11 +40874,11 @@ SQLITE_PRIVATE int sqlite3BtreeBeginStmt(Btree *p, int iStatement){ /* ** The second argument to this function, op, is always SAVEPOINT_ROLLBACK ** or SAVEPOINT_RELEASE. This function either releases or rolls back the -** savepoint identified by parameter iSavepoint, depending on the value +** savepoint identified by parameter iSavepoint, depending on the value ** of op. ** ** Normally, iSavepoint is greater than or equal to zero. However, if op is -** SAVEPOINT_ROLLBACK, then iSavepoint may also be -1. In this case the +** SAVEPOINT_ROLLBACK, then iSavepoint may also be -1. In this case the ** contents of the entire transaction are rolled back. This is different ** from a normal transaction rollback, as no locks are released and the ** transaction remains open. @@ -40932,8 +40928,8 @@ SQLITE_PRIVATE int sqlite3BtreeSavepoint(Btree *p, int op, int iSavepoint){ ** root page of a b-tree. If it is not, then the cursor acquired ** will not work correctly. ** -** It is assumed that the sqlite3BtreeCursorSize() bytes of memory -** pointed to by pCur have been zeroed by the caller. +** It is assumed that the sqlite3BtreeCursorZero() has been called +** on pCur to initialize the memory space prior to invoking this routine. */ static int btreeCursor( Btree *p, /* The btree */ @@ -40947,9 +40943,9 @@ static int btreeCursor( assert( sqlite3BtreeHoldsMutex(p) ); assert( wrFlag==0 || wrFlag==1 ); - /* The following assert statements verify that if this is a sharable - ** b-tree database, the connection is holding the required table locks, - ** and that no other connection has any open cursor that conflicts with + /* The following assert statements verify that if this is a sharable + ** b-tree database, the connection is holding the required table locks, + ** and that no other connection has any open cursor that conflicts with ** this lock. */ assert( hasSharedCacheTableLock(p, iTable, pKeyInfo!=0, wrFlag+1) ); assert( wrFlag==0 || !hasReadConflicts(p, iTable) ); @@ -41006,7 +41002,19 @@ SQLITE_PRIVATE int sqlite3BtreeCursor( ** this routine. */ SQLITE_PRIVATE int sqlite3BtreeCursorSize(void){ - return sizeof(BtCursor); + return ROUND8(sizeof(BtCursor)); +} + +/* +** Initialize memory that will be converted into a BtCursor object. +** +** The simple approach here would be to memset() the entire object +** to zero. But it turns out that the apPage[] and aiIdx[] arrays +** do not need to be zeroed and they are large, so we can save a lot +** of run-time by skipping the initialization of those elements. +*/ +SQLITE_PRIVATE void sqlite3BtreeCursorZero(BtCursor *p){ + memset(p, 0, offsetof(BtCursor, iPage)); } /* @@ -41132,14 +41140,14 @@ SQLITE_PRIVATE int sqlite3BtreeCursorIsValid(BtCursor *pCur){ /* ** Set *pSize to the size of the buffer needed to hold the value of ** the key for the current entry. If the cursor is not pointing -** to a valid entry, *pSize is set to 0. +** to a valid entry, *pSize is set to 0. ** ** For a table with the INTKEY flag set, this routine returns the key ** itself, not the number of bytes in the key. ** ** The caller must position the cursor prior to invoking this routine. -** -** This routine cannot fail. It always returns SQLITE_OK. +** +** This routine cannot fail. It always returns SQLITE_OK. */ SQLITE_PRIVATE int sqlite3BtreeKeySize(BtCursor *pCur, i64 *pSize){ assert( cursorHoldsMutex(pCur) ); @@ -41175,15 +41183,15 @@ SQLITE_PRIVATE int sqlite3BtreeDataSize(BtCursor *pCur, u32 *pSize){ /* ** Given the page number of an overflow page in the database (parameter -** ovfl), this function finds the page number of the next page in the +** ovfl), this function finds the page number of the next page in the ** linked list of overflow pages. If possible, it uses the auto-vacuum -** pointer-map data instead of reading the content of page ovfl to do so. +** pointer-map data instead of reading the content of page ovfl to do so. ** ** If an error occurs an SQLite error code is returned. Otherwise: ** -** The page number of the next overflow page in the linked list is -** written to *pPgnoNext. If page ovfl is the last page in its linked -** list, *pPgnoNext is set to zero. +** The page number of the next overflow page in the linked list is +** written to *pPgnoNext. If page ovfl is the last page in its linked +** list, *pPgnoNext is set to zero. ** ** If ppPage is not NULL, and a reference to the MemPage object corresponding ** to page number pOvfl was obtained, then *ppPage is set to point to that @@ -41207,9 +41215,9 @@ static int getOverflowPage( #ifndef SQLITE_OMIT_AUTOVACUUM /* Try to find the next page in the overflow list using the - ** autovacuum pointer-map pages. Guess that the next page in - ** the overflow list is page number (ovfl+1). If that guess turns - ** out to be wrong, fall back to loading the data of page + ** autovacuum pointer-map pages. Guess that the next page in + ** the overflow list is page number (ovfl+1). If that guess turns + ** out to be wrong, fall back to loading the data of page ** number ovfl to determine the next page number. */ if( pBt->autoVacuum ){ @@ -41296,7 +41304,7 @@ static int copyPayload( ** ** If the BtCursor.isIncrblobHandle flag is set, and the current ** cursor entry uses one or more overflow pages, this function -** allocates space for and lazily popluates the overflow page-list +** allocates space for and lazily popluates the overflow page-list ** cache array (BtCursor.aOverflow). Subsequent calls use this ** cache to make seeking to the supplied offset more efficient. ** @@ -41313,7 +41321,7 @@ static int accessPayload( BtCursor *pCur, /* Cursor pointing to entry to read from */ u32 offset, /* Begin reading this far into payload */ u32 amt, /* Read this many bytes */ - unsigned char *pBuf, /* Write the bytes into this buffer */ + unsigned char *pBuf, /* Write the bytes into this buffer */ int eOp /* zero to read. non-zero to write. */ ){ unsigned char *aPayload; @@ -41332,7 +41340,7 @@ static int accessPayload( aPayload = pCur->info.pCell + pCur->info.nHeader; nKey = (pPage->intKey ? 0 : (int)pCur->info.nKey); - if( NEVER(offset+amt > nKey+pCur->info.nData) + if( NEVER(offset+amt > nKey+pCur->info.nData) || &aPayload[pCur->info.nLocal] > &pPage->aData[pBt->usableSize] ){ /* Trying to read or write past the end of the data is an error */ @@ -41408,7 +41416,7 @@ static int accessPayload( #ifndef SQLITE_OMIT_INCRBLOB if( pCur->aOverflow && pCur->aOverflow[iIdx+1] ){ nextPage = pCur->aOverflow[iIdx+1]; - } else + } else #endif rc = getOverflowPage(pBt, nextPage, 0, &nextPage); offset -= ovflSize; @@ -41491,7 +41499,7 @@ SQLITE_PRIVATE int sqlite3BtreeData(BtCursor *pCur, u32 offset, u32 amt, void *p } /* -** Return a pointer to payload information from the entry that the +** Return a pointer to payload information from the entry that the ** pCur cursor is pointing to. The pointer is to the beginning of ** the key if skipKey==0 and it points to the beginning of data if ** skipKey==1. The number of bytes of available key/data is written @@ -41618,7 +41626,7 @@ static int moveToChild(BtCursor *pCur, u32 newPgno){ #ifndef NDEBUG /* -** Page pParent is an internal (non-leaf) tree page. This function +** Page pParent is an internal (non-leaf) tree page. This function ** asserts that page number iChild is the left-child if the iIdx'th ** cell in page pParent. Or, if iIdx is equal to the total number of ** cells in pParent, that page number iChild is the right-child of @@ -41633,7 +41641,7 @@ static void assertParentIndex(MemPage *pParent, int iIdx, Pgno iChild){ } } #else -# define assertParentIndex(x,y,z) +# define assertParentIndex(x,y,z) #endif /* @@ -41650,8 +41658,8 @@ static void moveToParent(BtCursor *pCur){ assert( pCur->iPage>0 ); assert( pCur->apPage[pCur->iPage] ); assertParentIndex( - pCur->apPage[pCur->iPage-1], - pCur->aiIdx[pCur->iPage-1], + pCur->apPage[pCur->iPage-1], + pCur->aiIdx[pCur->iPage-1], pCur->apPage[pCur->iPage]->pgno ); releasePage(pCur->apPage[pCur->iPage]); @@ -41665,19 +41673,19 @@ static void moveToParent(BtCursor *pCur){ ** ** If the table has a virtual root page, then the cursor is moved to point ** to the virtual root page instead of the actual root page. A table has a -** virtual root page when the actual root page contains no cells and a +** virtual root page when the actual root page contains no cells and a ** single child page. This can only happen with the table rooted at page 1. ** -** If the b-tree structure is empty, the cursor state is set to +** If the b-tree structure is empty, the cursor state is set to ** CURSOR_INVALID. Otherwise, the cursor is set to point to the first ** cell located on the root (or virtual root) page and the cursor state ** is set to CURSOR_VALID. ** ** If this function returns successfully, it may be assumed that the -** page-header flags indicate that the [virtual] root-page is the expected +** page-header flags indicate that the [virtual] root-page is the expected ** kind of b-tree page (i.e. if when opening the cursor the caller did not ** specify a KeyInfo structure the flags byte is set to 0x05 or 0x0D, -** indicating a table b-tree, or if the caller did specify a KeyInfo +** indicating a table b-tree, or if the caller did specify a KeyInfo ** structure the flags byte is set to 0x02 or 0x0A, indicating an index ** b-tree). */ @@ -41725,8 +41733,8 @@ static int moveToRoot(BtCursor *pCur){ /* Assert that the root page is of the correct type. This must be the ** case as the call to this function that loaded the root-page (either - ** this call or a previous invocation) would have detected corruption - ** if the assumption were not true, and it is not possible for the flags + ** this call or a previous invocation) would have detected corruption + ** if the assumption were not true, and it is not possible for the flags ** byte to have been modified while this cursor is holding a reference ** to the page. */ pRoot = pCur->apPage[0]; @@ -41832,14 +41840,14 @@ SQLITE_PRIVATE int sqlite3BtreeFirst(BtCursor *pCur, int *pRes){ */ SQLITE_PRIVATE int sqlite3BtreeLast(BtCursor *pCur, int *pRes){ int rc; - + assert( cursorHoldsMutex(pCur) ); assert( sqlite3_mutex_held(pCur->pBtree->db->mutex) ); /* If the cursor already points to the last entry, this is a no-op. */ if( CURSOR_VALID==pCur->eState && pCur->atLast ){ #ifdef SQLITE_DEBUG - /* This block serves to assert() that the cursor really does point + /* This block serves to assert() that the cursor really does point ** to the last entry in the b-tree. */ int ii; for(ii=0; iiiPage; ii++){ @@ -41866,10 +41874,10 @@ SQLITE_PRIVATE int sqlite3BtreeLast(BtCursor *pCur, int *pRes){ return rc; } -/* Move the cursor so that it points to an entry near the key +/* Move the cursor so that it points to an entry near the key ** specified by pIdxKey or intKey. Return a success code. ** -** For INTKEY tables, the intKey parameter is used. pIdxKey +** For INTKEY tables, the intKey parameter is used. pIdxKey ** must be NULL. For index tables, pIdxKey is used and intKey ** is ignored. ** @@ -41879,7 +41887,7 @@ SQLITE_PRIVATE int sqlite3BtreeLast(BtCursor *pCur, int *pRes){ ** before or after the key. ** ** An integer is written into *pRes which is the result of -** comparing the key with the entry to which the cursor is +** comparing the key with the entry to which the cursor is ** pointing. The meaning of the integer written into ** *pRes is as follows: ** @@ -41910,8 +41918,8 @@ SQLITE_PRIVATE int sqlite3BtreeMovetoUnpacked( /* If the cursor is already positioned at the point we are trying ** to move to, then just return without doing any work */ - if( pCur->eState==CURSOR_VALID && pCur->validNKey - && pCur->apPage[0]->intKey + if( pCur->eState==CURSOR_VALID && pCur->validNKey + && pCur->apPage[0]->intKey ){ if( pCur->info.nKey==intKey ){ *pRes = 0; @@ -41984,9 +41992,9 @@ SQLITE_PRIVATE int sqlite3BtreeMovetoUnpacked( /* The maximum supported page-size is 32768 bytes. This means that ** the maximum number of record bytes stored on an index B-Tree ** page is at most 8198 bytes, which may be stored as a 2-byte - ** varint. This information is used to attempt to avoid parsing - ** the entire cell by checking for the cases where the record is - ** stored entirely within the b-tree page by inspecting the first + ** varint. This information is used to attempt to avoid parsing + ** the entire cell by checking for the cases where the record is + ** stored entirely within the b-tree page by inspecting the first ** 2 bytes of the cell. */ int nCell = pCell[0]; @@ -41995,10 +42003,10 @@ SQLITE_PRIVATE int sqlite3BtreeMovetoUnpacked( ** single byte varint and the record fits entirely on the main ** b-tree page. */ c = sqlite3VdbeRecordCompare(nCell, (void*)&pCell[1], pIdxKey); - }else if( !(pCell[1] & 0x80) + }else if( !(pCell[1] & 0x80) && (nCell = ((nCell&0x7f)<<7) + pCell[1])<=pPage->maxLocal ){ - /* The record-size field is a 2 byte varint and the record + /* The record-size field is a 2 byte varint and the record ** fits entirely on the main b-tree page. */ c = sqlite3VdbeRecordCompare(nCell, (void*)&pCell[2], pIdxKey); }else{ @@ -42227,19 +42235,19 @@ SQLITE_PRIVATE int sqlite3BtreePrevious(BtCursor *pCur, int *pRes){ ** an error. *ppPage and *pPgno are undefined in the event of an error. ** Do not invoke sqlite3PagerUnref() on *ppPage if an error is returned. ** -** If the "nearby" parameter is not 0, then a (feeble) effort is made to +** If the "nearby" parameter is not 0, then a (feeble) effort is made to ** locate a page close to the page number "nearby". This can be used in an ** attempt to keep related pages close to each other in the database file, ** which in turn can make database access faster. ** -** If the "exact" parameter is not 0, and the page-number nearby exists +** If the "exact" parameter is not 0, and the page-number nearby exists ** anywhere on the free-list, then it is guarenteed to be returned. This ** is only used by auto-vacuum databases when allocating a new table. */ static int allocateBtreePage( - BtShared *pBt, - MemPage **ppPage, - Pgno *pPgno, + BtShared *pBt, + MemPage **ppPage, + Pgno *pPgno, Pgno nearby, u8 exact ){ @@ -42263,7 +42271,7 @@ static int allocateBtreePage( /* There are pages on the freelist. Reuse one of those pages. */ Pgno iTrunk; u8 searchList = 0; /* If the free-list must be searched for 'nearby' */ - + /* If the 'exact' parameter was true and a query of the pointer-map ** shows that the page 'nearby' is somewhere on the free-list, then ** the entire-list will be searched for that page. @@ -42313,8 +42321,8 @@ static int allocateBtreePage( k = get4byte(&pTrunk->aData[4]); if( k==0 && !searchList ){ - /* The trunk has no leaves and the list is not being searched. - ** So extract the trunk page itself and use it as the newly + /* The trunk has no leaves and the list is not being searched. + ** So extract the trunk page itself and use it as the newly ** allocated page */ assert( pPrevTrunk==0 ); rc = sqlite3PagerWrite(pTrunk->pDbPage); @@ -42349,13 +42357,13 @@ static int allocateBtreePage( memcpy(&pPrevTrunk->aData[0], &pTrunk->aData[0], 4); } }else{ - /* The trunk page is required by the caller but it contains + /* The trunk page is required by the caller but it contains ** pointers to free-list leaves. The first leaf becomes a trunk ** page in this case. */ MemPage *pNewTrunk; Pgno iNewTrunk = get4byte(&pTrunk->aData[8]); - if( iNewTrunk>mxPage ){ + if( iNewTrunk>mxPage ){ rc = SQLITE_CORRUPT_BKPT; goto end_allocate_page; } @@ -42504,12 +42512,12 @@ end_allocate_page: } /* -** This function is used to add page iPage to the database file free-list. +** This function is used to add page iPage to the database file free-list. ** It is assumed that the page is not already a part of the free-list. ** ** The value passed as the second argument to this function is optional. -** If the caller happens to have a pointer to the MemPage object -** corresponding to page iPage handy, it may pass it as the second value. +** If the caller happens to have a pointer to the MemPage object +** corresponding to page iPage handy, it may pass it as the second value. ** Otherwise, it may pass NULL. ** ** If a pointer to a MemPage object is passed as the second argument, @@ -42517,7 +42525,7 @@ end_allocate_page: */ static int freePage2(BtShared *pBt, MemPage *pMemPage, Pgno iPage){ MemPage *pTrunk = 0; /* Free-list trunk page */ - Pgno iTrunk = 0; /* Page number of free-list trunk page */ + Pgno iTrunk = 0; /* Page number of free-list trunk page */ MemPage *pPage1 = pBt->pPage1; /* Local reference to page 1 */ MemPage *pPage; /* Page being freed. May be NULL. */ int rc; /* Return Code */ @@ -42615,7 +42623,7 @@ static int freePage2(BtShared *pBt, MemPage *pMemPage, Pgno iPage){ /* If control flows to this point, then it was not possible to add the ** the page being freed as a leaf page of the first trunk in the free-list. - ** Possibly because the free-list is empty, or possibly because the + ** Possibly because the free-list is empty, or possibly because the ** first trunk in the free-list is full. Either way, the page being freed ** will become the new first trunk page in the free-list. */ @@ -42670,8 +42678,8 @@ static int clearCell(MemPage *pPage, unsigned char *pCell){ Pgno iNext = 0; MemPage *pOvfl = 0; if( ovflPgno<2 || ovflPgno>pagerPagecount(pBt) ){ - /* 0 is not a legal page number and page 1 cannot be an - ** overflow page. Therefore if ovflPgno<2 or past the end of the + /* 0 is not a legal page number and page 1 cannot be an + ** overflow page. Therefore if ovflPgno<2 or past the end of the ** file the database must be corrupt. */ return SQLITE_CORRUPT_BKPT; } @@ -42744,14 +42752,14 @@ static int fillInCell( assert( info.nHeader==nHeader ); assert( info.nKey==nKey ); assert( info.nData==(u32)(nData+nZero) ); - + /* Fill in the payload */ nPayload = nData + nZero; if( pPage->intKey ){ pSrc = pData; nSrc = nData; nData = 0; - }else{ + }else{ if( NEVER(nKey>0x7fffffff || pKey==0) ){ return SQLITE_CORRUPT_BKPT; } @@ -42771,8 +42779,8 @@ static int fillInCell( if( pBt->autoVacuum ){ do{ pgnoOvfl++; - } while( - PTRMAP_ISPAGE(pBt, pgnoOvfl) || pgnoOvfl==PENDING_BYTE_PAGE(pBt) + } while( + PTRMAP_ISPAGE(pBt, pgnoOvfl) || pgnoOvfl==PENDING_BYTE_PAGE(pBt) ); } #endif @@ -42780,9 +42788,9 @@ static int fillInCell( #ifndef SQLITE_OMIT_AUTOVACUUM /* If the database supports auto-vacuum, and the second or subsequent ** overflow page is being allocated, add an entry to the pointer-map - ** for that page now. + ** for that page now. ** - ** If this is the first overflow page, then write a partial entry + ** If this is the first overflow page, then write a partial entry ** to the pointer-map. If we write nothing to this pointer-map slot, ** then the optimistic overflow chain processing in clearCell() ** may misinterpret the uninitialised values and delete the @@ -42905,13 +42913,13 @@ static void dropCell(MemPage *pPage, int idx, int sz, int *pRC){ ** will not fit, then make a copy of the cell content into pTemp if ** pTemp is not null. Regardless of pTemp, allocate a new entry ** in pPage->aOvfl[] and make it point to the cell content (either -** in pTemp or the original pCell) and also record its index. -** Allocating a new entry in pPage->aCell[] implies that +** in pTemp or the original pCell) and also record its index. +** Allocating a new entry in pPage->aCell[] implies that ** pPage->nOverflow is incremented. ** ** If nSkip is non-zero, then do not copy the first nSkip bytes of the ** cell. The caller will overwrite them after this function returns. If -** nSkip is non-zero, then pCell may not point to an invalid memory location +** nSkip is non-zero, then pCell may not point to an invalid memory location ** (but pCell+nSkip is always valid). */ static void insertCell( @@ -42938,8 +42946,13 @@ static void insertCell( assert( i>=0 && i<=pPage->nCell+pPage->nOverflow ); assert( pPage->nCell<=MX_CELL(pPage->pBt) && MX_CELL(pPage->pBt)<=5460 ); assert( pPage->nOverflow<=ArraySize(pPage->aOvfl) ); - assert( sz==cellSizePtr(pPage, pCell) ); assert( sqlite3_mutex_held(pPage->pBt->mutex) ); + /* The cell should normally be sized correctly. However, when moving a + ** malformed cell from a leaf page to an interior page, if the cell size + ** wanted to be less than 4 but got rounded up to 4 on the leaf, then size + ** might be less than 8 (leaf-size + pointer) on the interior node. Hence + ** the term after the || in the following assert(). */ + assert( sz==cellSizePtr(pPage, pCell) || (sz==8 && iChild>0) ); if( pPage->nOverflow || sz+2>pPage->nFree ){ if( pTemp ){ memcpy(pTemp+nSkip, pCell+nSkip, sz-nSkip); @@ -43084,7 +43097,7 @@ static int balance_quick(MemPage *pParent, MemPage *pPage, u8 *pSpace){ if( pPage->nCell<=0 ) return SQLITE_CORRUPT_BKPT; - /* Allocate a new page. This page will become the right-sibling of + /* Allocate a new page. This page will become the right-sibling of ** pPage. Make the parent page writable, so that the new divider cell ** may be inserted. If both these operations are successful, proceed. */ @@ -43103,7 +43116,7 @@ static int balance_quick(MemPage *pParent, MemPage *pPage, u8 *pSpace){ assemblePage(pNew, 1, &pCell, &szCell); /* If this is an auto-vacuum database, update the pointer map - ** with entries for the new page, and any pointer from the + ** with entries for the new page, and any pointer from the ** cell on the page to an overflow page. If either of these ** operations fails, the return code is set, but the contents ** of the parent page are still manipulated by thh code below. @@ -43117,14 +43130,14 @@ static int balance_quick(MemPage *pParent, MemPage *pPage, u8 *pSpace){ ptrmapPutOvflPtr(pNew, pCell, &rc); } } - + /* Create a divider cell to insert into pParent. The divider cell ** consists of a 4-byte page number (the page number of pPage) and ** a variable length key value (which must be the same value as the ** largest key on pPage). ** - ** To find the largest key value on pPage, first find the right-most - ** cell on pPage. The first two fields of this cell are the + ** To find the largest key value on pPage, first find the right-most + ** cell on pPage. The first two fields of this cell are the ** record-length (a variable length integer at most 32-bits in size) ** and the key value (a variable length integer, may have any value). ** The first of the while(...) loops below skips over the record-length @@ -43143,7 +43156,7 @@ static int balance_quick(MemPage *pParent, MemPage *pPage, u8 *pSpace){ /* Set the right-child pointer of pParent to point to the new page. */ put4byte(&pParent->aData[pParent->hdrOffset+8], pgnoNew); - + /* Release the reference to the new page. */ releasePage(pNew); } @@ -43155,7 +43168,7 @@ static int balance_quick(MemPage *pParent, MemPage *pPage, u8 *pSpace){ #if 0 /* ** This function does not contribute anything to the operation of SQLite. -** it is sometimes activated temporarily while debugging code responsible +** it is sometimes activated temporarily while debugging code responsible ** for setting pointer-map entries. */ static int ptrmapCheckPages(MemPage **apPage, int nPage){ @@ -43170,7 +43183,7 @@ static int ptrmapCheckPages(MemPage **apPage, int nPage){ for(j=0; jnCell; j++){ CellInfo info; u8 *z; - + z = findCell(pPage, j); btreeParseCellPtr(pPage, z, &info); if( info.iOverflow ){ @@ -43195,7 +43208,7 @@ static int ptrmapCheckPages(MemPage **apPage, int nPage){ #endif /* -** This function is used to copy the contents of the b-tree node stored +** This function is used to copy the contents of the b-tree node stored ** on page pFrom to page pTo. If page pFrom was not a leaf page, then ** the pointer-map entries for each child page are updated so that the ** parent page stored in the pointer map is page pTo. If pFrom contained @@ -43203,11 +43216,11 @@ static int ptrmapCheckPages(MemPage **apPage, int nPage){ ** map entries are also updated so that the parent page is page pTo. ** ** If pFrom is currently carrying any overflow cells (entries in the -** MemPage.aOvfl[] array), they are not copied to pTo. +** MemPage.aOvfl[] array), they are not copied to pTo. ** ** Before returning, page pTo is reinitialized using btreeInitPage(). ** -** The performance of this function is not critical. It is only used by +** The performance of this function is not critical. It is only used by ** the balance_shallower() and balance_deeper() procedures, neither of ** which are called often under normal circumstances. */ @@ -43218,26 +43231,31 @@ static void copyNodeContent(MemPage *pFrom, MemPage *pTo, int *pRC){ u8 * const aTo = pTo->aData; int const iFromHdr = pFrom->hdrOffset; int const iToHdr = ((pTo->pgno==1) ? 100 : 0); - TESTONLY(int rc;) + int rc; int iData; - - + + assert( pFrom->isInit ); assert( pFrom->nFree>=iToHdr ); assert( get2byte(&aFrom[iFromHdr+5])<=pBt->usableSize ); - + /* Copy the b-tree node content from page pFrom to page pTo. */ iData = get2byte(&aFrom[iFromHdr+5]); memcpy(&aTo[iData], &aFrom[iData], pBt->usableSize-iData); memcpy(&aTo[iToHdr], &aFrom[iFromHdr], pFrom->cellOffset + 2*pFrom->nCell); - + /* Reinitialize page pTo so that the contents of the MemPage structure - ** match the new data. The initialization of pTo "cannot" fail, as the - ** data copied from pFrom is known to be valid. */ + ** match the new data. The initialization of pTo can actually fail under + ** fairly obscure circumstances, even though it is a copy of initialized + ** page pFrom. + */ pTo->isInit = 0; - TESTONLY(rc = ) btreeInitPage(pTo); - assert( rc==SQLITE_OK ); - + rc = btreeInitPage(pTo); + if( rc!=SQLITE_OK ){ + *pRC = rc; + return; + } + /* If this is an auto-vacuum database, update the pointer-map entries ** for any b-tree or overflow pages that pTo now contains the pointers to. */ @@ -43252,13 +43270,13 @@ static void copyNodeContent(MemPage *pFrom, MemPage *pTo, int *pRC){ ** (hereafter "the page") and up to 2 siblings so that all pages have about the ** same amount of free space. Usually a single sibling on either side of the ** page are used in the balancing, though both siblings might come from one -** side if the page is the first or last child of its parent. If the page +** side if the page is the first or last child of its parent. If the page ** has fewer than 2 siblings (something which can only happen if the page ** is a root page or a child of a root page) then all available siblings ** participate in the balancing. ** -** The number of siblings of the page might be increased or decreased by -** one or two in an effort to keep pages nearly full but not over full. +** The number of siblings of the page might be increased or decreased by +** one or two in an effort to keep pages nearly full but not over full. ** ** Note that when this routine is called, some of the cells on the page ** might not actually be stored in MemPage.aData[]. This can happen @@ -43269,7 +43287,7 @@ static void copyNodeContent(MemPage *pFrom, MemPage *pTo, int *pRC){ ** inserted into or removed from the parent page (pParent). Doing so ** may cause the parent page to become overfull or underfull. If this ** happens, it is the responsibility of the caller to invoke the correct -** balancing routine to fix this problem (see the balance() routine). +** balancing routine to fix this problem (see the balance() routine). ** ** If this routine fails for any reason, it might leave the database ** in a corrupted state. So if this routine fails, the database should @@ -43284,7 +43302,7 @@ static void copyNodeContent(MemPage *pFrom, MemPage *pTo, int *pRC){ ** of the page-size, the aOvflSpace[] buffer is guaranteed to be large ** enough for all overflow cells. ** -** If aOvflSpace is set to a null pointer, this function returns +** If aOvflSpace is set to a null pointer, this function returns ** SQLITE_NOMEM. */ static int balance_nonroot( @@ -43330,7 +43348,7 @@ static int balance_nonroot( #endif /* At this point pParent may have at most one overflow cell. And if - ** this overflow cell is present, it must be the cell with + ** this overflow cell is present, it must be the cell with ** index iParentIdx. This scenario comes about when this function ** is called (indirectly) from sqlite3BtreeDelete(). */ @@ -43341,11 +43359,11 @@ static int balance_nonroot( return SQLITE_NOMEM; } - /* Find the sibling pages to balance. Also locate the cells in pParent - ** that divide the siblings. An attempt is made to find NN siblings on - ** either side of pPage. More siblings are taken from one side, however, + /* Find the sibling pages to balance. Also locate the cells in pParent + ** that divide the siblings. An attempt is made to find NN siblings on + ** either side of pPage. More siblings are taken from one side, however, ** if there are fewer than NN siblings on the other side. If pParent - ** has NB or fewer children then all children of pParent are taken. + ** has NB or fewer children then all children of pParent are taken. ** ** This loop also drops the divider cells from the parent page. This ** way, the remainder of the function does not have to deal with any @@ -43358,7 +43376,7 @@ static int balance_nonroot( nOld = i+1; }else{ nOld = 3; - if( iParentIdx==0 ){ + if( iParentIdx==0 ){ nxDiv = 0; }else if( iParentIdx==i ){ nxDiv = i-2; @@ -43397,7 +43415,7 @@ static int balance_nonroot( ** This is safe because dropping a cell only overwrites the first ** four bytes of it, and this function does not need the first ** four bytes of the divider cell. So the pointer is safe to use - ** later on. + ** later on. ** ** Unless SQLite is compiled in secure-delete mode. In this case, ** the dropCell() routine will overwrite the entire cell with zeroes. @@ -43425,7 +43443,7 @@ static int balance_nonroot( + nMaxCells*sizeof(u16) /* szCell */ + pBt->pageSize /* aSpace1 */ + k*nOld; /* Page copies (apCopy) */ - apCell = sqlite3ScratchMalloc( szScratch ); + apCell = sqlite3ScratchMalloc( szScratch ); if( apCell==0 ){ rc = SQLITE_NOMEM; goto balance_cleanup; @@ -43454,7 +43472,7 @@ static int balance_nonroot( leafData = apOld[0]->hasData; for(i=0; iusableSize - 12 + leafCorrection; for(subtotal=k=i=0; i0 || (pParent->pgno==1 && pParent->nCell==0) ); TRACE(("BALANCE: old: %d %d %d ", - apOld[0]->pgno, + apOld[0]->pgno, nOld>=2 ? apOld[1]->pgno : 0, nOld>=3 ? apOld[2]->pgno : 0 )); @@ -43694,9 +43712,9 @@ static int balance_nonroot( if( !pNew->leaf ){ memcpy(&pNew->aData[8], pCell, 4); }else if( leafData ){ - /* If the tree is a leaf-data tree, and the siblings are leaves, - ** then there is no divider cell in apCell[]. Instead, the divider - ** cell consists of the integer key for the right-most cell of + /* If the tree is a leaf-data tree, and the siblings are leaves, + ** then there is no divider cell in apCell[]. Instead, the divider + ** cell consists of the integer key for the right-most cell of ** the sibling-page assembled above only. */ CellInfo info; @@ -43709,9 +43727,9 @@ static int balance_nonroot( pCell -= 4; /* Obscure case for non-leaf-data trees: If the cell at pCell was ** previously stored on a leaf node, and its reported size was 4 - ** bytes, then it may actually be smaller than this + ** bytes, then it may actually be smaller than this ** (see btreeParseCellPtr(), 4 bytes is the minimum size of - ** any cell). But it is important to pass the correct size to + ** any cell). But it is important to pass the correct size to ** insertCell(), so reparse the cell now. ** ** Note that this can never happen in an SQLite data file, as all @@ -43749,8 +43767,8 @@ static int balance_nonroot( ** b-tree structure by one. This is described as the "balance-shallower" ** sub-algorithm in some documentation. ** - ** If this is an auto-vacuum database, the call to copyNodeContent() - ** sets all pointer-map entries corresponding to database image pages + ** If this is an auto-vacuum database, the call to copyNodeContent() + ** sets all pointer-map entries corresponding to database image pages ** for which the pointer is stored within the content being copied. ** ** The second assert below verifies that the child page is defragmented @@ -43758,13 +43776,13 @@ static int balance_nonroot( ** is important if the parent page happens to be page 1 of the database ** image. */ assert( nNew==1 ); - assert( apNew[0]->nFree == - (get2byte(&apNew[0]->aData[5])-apNew[0]->cellOffset-apNew[0]->nCell*2) + assert( apNew[0]->nFree == + (get2byte(&apNew[0]->aData[5])-apNew[0]->cellOffset-apNew[0]->nCell*2) ); copyNodeContent(apNew[0], pParent, &rc); freePage(apNew[0], &rc); }else if( ISAUTOVACUUM ){ - /* Fix the pointer-map entries for all the cells that were shifted around. + /* Fix the pointer-map entries for all the cells that were shifted around. ** There are several different types of pointer-map entries that need to ** be dealt with by this routine. Some of these have been set already, but ** many have not. The following is a summary: @@ -43776,7 +43794,7 @@ static int balance_nonroot( ** of those. ** ** 2) The pointer-map entries associated with the first overflow - ** page in any overflow chains used by new divider cells. These + ** page in any overflow chains used by new divider cells. These ** have also already been taken care of by the insertCell() code. ** ** 3) If the sibling pages are not leaves, then the child pages of @@ -43814,7 +43832,7 @@ static int balance_nonroot( nOverflow = pOld->nOverflow; iOverflow = i + !leafData + pOld->aOvfl[0].idx; } - isDivider = !leafData; + isDivider = !leafData; } assert(nOverflow>0 || iOverflownOverflow>0 ); assert( sqlite3_mutex_held(pBt->mutex) ); - /* Make pRoot, the root page of the b-tree, writable. Allocate a new + /* Make pRoot, the root page of the b-tree, writable. Allocate a new ** page that will become the new right-child of pPage. Copy the contents ** of the node stored on pRoot into the new child page. */ @@ -43954,7 +43972,7 @@ static int balance_deeper(MemPage *pRoot, MemPage **ppChild){ /* ** The page that pCur currently points to has just been modified in ** some way. This function figures out if this modification means the -** tree needs to be balanced, and if so calls the appropriate balancing +** tree needs to be balanced, and if so calls the appropriate balancing ** routine. Balancing routines are: ** ** balance_quick() @@ -43980,7 +43998,7 @@ static int balance(BtCursor *pCur){ ** balance_deeper() function to create a new child for the root-page ** and copy the current contents of the root-page to it. The ** next iteration of the do-loop will balance the child page. - */ + */ assert( (balance_deeper_called++)==0 ); rc = balance_deeper(pPage, &pCur->apPage[1]); if( rc==SQLITE_OK ){ @@ -44010,10 +44028,10 @@ static int balance(BtCursor *pCur){ /* Call balance_quick() to create a new sibling of pPage on which ** to store the overflow cell. balance_quick() inserts a new cell ** into pParent, which may cause pParent overflow. If this - ** happens, the next interation of the do-loop will balance pParent + ** happens, the next interation of the do-loop will balance pParent ** use either balance_nonroot() or balance_deeper(). Until this ** happens, the overflow cell is stored in the aBalanceQuickSpace[] - ** buffer. + ** buffer. ** ** The purpose of the following assert() is to check that only a ** single call to balance_quick() is made for each call to this @@ -44030,24 +44048,24 @@ static int balance(BtCursor *pCur){ ** modifying the contents of pParent, which may cause pParent to ** become overfull or underfull. The next iteration of the do-loop ** will balance the parent page to correct this. - ** + ** ** If the parent page becomes overfull, the overflow cell or cells - ** are stored in the pSpace buffer allocated immediately below. + ** are stored in the pSpace buffer allocated immediately below. ** A subsequent iteration of the do-loop will deal with this by ** calling balance_nonroot() (balance_deeper() may be called first, ** but it doesn't deal with overflow cells - just moves them to a - ** different page). Once this subsequent call to balance_nonroot() + ** different page). Once this subsequent call to balance_nonroot() ** has completed, it is safe to release the pSpace buffer used by - ** the previous call, as the overflow cell data will have been + ** the previous call, as the overflow cell data will have been ** copied either into the body of a database page or into the new ** pSpace buffer passed to the latter call to balance_nonroot(). */ u8 *pSpace = sqlite3PageMalloc(pCur->pBt->pageSize); rc = balance_nonroot(pParent, iIdx, pSpace, iPage==1); if( pFree ){ - /* If pFree is not NULL, it points to the pSpace buffer used + /* If pFree is not NULL, it points to the pSpace buffer used ** by a previous call to balance_nonroot(). Its contents are - ** now stored either on real database pages or within the + ** now stored either on real database pages or within the ** new pSpace buffer, so it may be safely freed here. */ sqlite3PageFree(pFree); } @@ -44087,8 +44105,8 @@ static int balance(BtCursor *pCur){ ** MovetoUnpacked() to seek cursor pCur to (pKey, nKey) has already ** been performed. seekResult is the search result returned (a negative ** number if pCur points at an entry that is smaller than (pKey, nKey), or -** a positive value if pCur points at an etry that is larger than -** (pKey, nKey)). +** a positive value if pCur points at an etry that is larger than +** (pKey, nKey)). ** ** If the seekResult parameter is non-zero, then the caller guarantees that ** cursor pCur is pointing at the existing copy of a row that is to be @@ -44130,7 +44148,7 @@ SQLITE_PRIVATE int sqlite3BtreeInsert( ** blob of associated data. */ assert( (pKey==0)==(pCur->pKeyInfo==0) ); - /* If this is an insert into a table b-tree, invalidate any incrblob + /* If this is an insert into a table b-tree, invalidate any incrblob ** cursors open on the row being replaced (assuming this is a replace ** operation - if it is not, the following is a no-op). */ if( pCur->pKeyInfo==0 ){ @@ -44141,8 +44159,8 @@ SQLITE_PRIVATE int sqlite3BtreeInsert( ** ** In some cases, the call to btreeMoveto() below is a no-op. For ** example, when inserting data into a table with auto-generated integer - ** keys, the VDBE layer invokes sqlite3BtreeLast() to figure out the - ** integer key to use. It then calls this function to actually insert the + ** keys, the VDBE layer invokes sqlite3BtreeLast() to figure out the + ** integer key to use. It then calls this function to actually insert the ** data into the intkey B-Tree. In this case btreeMoveto() recognizes ** that the cursor is already where it needs to be and returns without ** doing any work. To avoid thwarting these optimizations, it is important @@ -44196,7 +44214,7 @@ SQLITE_PRIVATE int sqlite3BtreeInsert( insertCell(pPage, idx, newCell, szNew, 0, 0, &rc); assert( rc!=SQLITE_OK || pPage->nCell>0 || pPage->nOverflow>0 ); - /* If no error has occured and pPage has an overflow cell, call balance() + /* If no error has occured and pPage has an overflow cell, call balance() ** to redistribute the cells within the tree. Since balance() may move ** the cursor, zero the BtCursor.info.nSize and BtCursor.validNKey ** variables. @@ -44222,7 +44240,7 @@ SQLITE_PRIVATE int sqlite3BtreeInsert( rc = balance(pCur); /* Must make sure nOverflow is reset to zero even if the balance() - ** fails. Internal data structure corruption will result otherwise. + ** fails. Internal data structure corruption will result otherwise. ** Also, set the cursor state to invalid. This stops saveCursorPosition() ** from trying to save the current position of the cursor. */ pCur->apPage[pCur->iPage]->nOverflow = 0; @@ -44240,12 +44258,12 @@ end_insert: */ SQLITE_PRIVATE int sqlite3BtreeDelete(BtCursor *pCur){ Btree *p = pCur->pBtree; - BtShared *pBt = p->pBt; + BtShared *pBt = p->pBt; int rc; /* Return code */ MemPage *pPage; /* Page to delete cell from */ unsigned char *pCell; /* Pointer to cell to delete */ int iCellIdx; /* Index of cell to delete */ - int iCellDepth; /* Depth of node containing pCell */ + int iCellDepth; /* Depth of node containing pCell */ assert( cursorHoldsMutex(pCur) ); assert( pBt->inTransaction==TRANS_WRITE ); @@ -44254,7 +44272,7 @@ SQLITE_PRIVATE int sqlite3BtreeDelete(BtCursor *pCur){ assert( hasSharedCacheTableLock(p, pCur->pgnoRoot, pCur->pKeyInfo!=0, 2) ); assert( !hasReadConflicts(p, pCur->pgnoRoot) ); - if( NEVER(pCur->aiIdx[pCur->iPage]>=pCur->apPage[pCur->iPage]->nCell) + if( NEVER(pCur->aiIdx[pCur->iPage]>=pCur->apPage[pCur->iPage]->nCell) || NEVER(pCur->eState!=CURSOR_VALID) ){ return SQLITE_ERROR; /* Something has gone awry. */ @@ -44285,9 +44303,9 @@ SQLITE_PRIVATE int sqlite3BtreeDelete(BtCursor *pCur){ } /* Save the positions of any other cursors open on this table before - ** making any modifications. Make the page containing the entry to be - ** deleted writable. Then free any overflow pages associated with the - ** entry and finally remove the cell itself from within the page. + ** making any modifications. Make the page containing the entry to be + ** deleted writable. Then free any overflow pages associated with the + ** entry and finally remove the cell itself from within the page. */ rc = saveAllCursors(pBt, pCur->pgnoRoot, pCur); if( rc ) return rc; @@ -44334,7 +44352,7 @@ SQLITE_PRIVATE int sqlite3BtreeDelete(BtCursor *pCur){ ** on the leaf node first. If the balance proceeds far enough up the ** tree that we can be sure that any problem in the internal node has ** been corrected, so be it. Otherwise, after balancing the leaf node, - ** walk the cursor up the tree to the internal node and balance it as + ** walk the cursor up the tree to the internal node and balance it as ** well. */ rc = balance(pCur); if( rc==SQLITE_OK && pCur->iPage>iCellDepth ){ @@ -44458,7 +44476,7 @@ static int btreeCreateTable(Btree *p, int *piTable, int flags){ } }else{ pRoot = pPageMove; - } + } /* Update the pointer-map and meta-data with the new root-page number. */ ptrmapPut(pBt, pgnoRoot, PTRMAP_ROOTPAGE, 0, &rc); @@ -44497,9 +44515,9 @@ SQLITE_PRIVATE int sqlite3BtreeCreateTable(Btree *p, int *piTable, int flags){ */ static int clearDatabasePage( BtShared *pBt, /* The BTree that contains the table */ - Pgno pgno, /* Page number to clear */ - int freePageFlag, /* Deallocate page if true */ - int *pnChange + Pgno pgno, /* Page number to clear */ + int freePageFlag, /* Deallocate page if true */ + int *pnChange /* Add number of Cells freed to this counter */ ){ MemPage *pPage; int rc; @@ -44581,12 +44599,12 @@ SQLITE_PRIVATE int sqlite3BtreeClearTable(Btree *p, int iTable, int *pnChange){ ** cursors on the table. ** ** If AUTOVACUUM is enabled and the page at iTable is not the last -** root page in the database file, then the last root page +** root page in the database file, then the last root page ** in the database file is moved into the slot formerly occupied by ** iTable and that last slot formerly occupied by the last root page ** is added to the freelist instead of iTable. In this say, all ** root pages are kept at the beginning of the database file, which -** is necessary for AUTOVACUUM to work right. *piMoved is set to the +** is necessary for AUTOVACUUM to work right. *piMoved is set to the ** page number that used to be the last root page in the file before ** the move. If no page gets moved, *piMoved is set to 0. ** The last root page is recorded in meta[3] and the value of @@ -44603,7 +44621,7 @@ static int btreeDropTable(Btree *p, Pgno iTable, int *piMoved){ /* It is illegal to drop a table if any cursors are open on the ** database. This is because in auto-vacuum mode the backend may ** need to move another root-page to fill a gap left by the deleted - ** root page. If an open cursor was using this page a problem would + ** root page. If an open cursor was using this page a problem would ** occur. ** ** This error is caught long before control reaches this point. @@ -44634,7 +44652,7 @@ static int btreeDropTable(Btree *p, Pgno iTable, int *piMoved){ if( iTable==maxRootPgno ){ /* If the table being dropped is the table with the largest root-page - ** number in the database, put the root page on the free list. + ** number in the database, put the root page on the free list. */ freePage(pPage, &rc); releasePage(pPage); @@ -44643,7 +44661,7 @@ static int btreeDropTable(Btree *p, Pgno iTable, int *piMoved){ } }else{ /* The table being dropped does not have the largest root-page - ** number in the database. So move the page that does into the + ** number in the database. So move the page that does into the ** gap left by the deleted root-page. */ MemPage *pMove; @@ -44688,12 +44706,12 @@ static int btreeDropTable(Btree *p, Pgno iTable, int *piMoved){ }else{ /* If sqlite3BtreeDropTable was called on page 1. ** This really never should happen except in a corrupt - ** database. + ** database. */ zeroPage(pPage, PTF_INTKEY|PTF_LEAF ); releasePage(pPage); } - return rc; + return rc; } SQLITE_PRIVATE int sqlite3BtreeDropTable(Btree *p, int iTable, int *piMoved){ int rc; @@ -44712,7 +44730,7 @@ SQLITE_PRIVATE int sqlite3BtreeDropTable(Btree *p, int iTable, int *piMoved){ ** is the number of free pages currently in the database. Meta[1] ** through meta[15] are available for use by higher layers. Meta[0] ** is read-only, the others are read/write. -** +** ** The schema layer numbers meta values differently. At the schema ** layer (and the SetCookie and ReadCookie opcodes) the number of ** free pages is not visible. So Cookie[0] is the same as Meta[1]. @@ -44770,7 +44788,7 @@ SQLITE_PRIVATE int sqlite3BtreeUpdateMeta(Btree *p, int idx, u32 iMeta){ ** The first argument, pCur, is a cursor opened on some b-tree. Count the ** number of entries in the b-tree and write the result to *pnEntry. ** -** SQLITE_OK is returned if the operation is successfully executed. +** SQLITE_OK is returned if the operation is successfully executed. ** Otherwise, if an error is encountered (i.e. an IO error or database ** corruption) an SQLite error code is returned. */ @@ -44780,13 +44798,13 @@ SQLITE_PRIVATE int sqlite3BtreeCount(BtCursor *pCur, i64 *pnEntry){ rc = moveToRoot(pCur); /* Unless an error occurs, the following loop runs one iteration for each - ** page in the B-Tree structure (not including overflow pages). + ** page in the B-Tree structure (not including overflow pages). */ while( rc==SQLITE_OK ){ int iIdx; /* Index of child node in parent */ MemPage *pPage; /* Current page of the b-tree */ - /* If this is a leaf page or the tree is not an int-key tree, then + /* If this is a leaf page or the tree is not an int-key tree, then ** this page contains countable entries. Increment the entry counter ** accordingly. */ @@ -44795,7 +44813,7 @@ SQLITE_PRIVATE int sqlite3BtreeCount(BtCursor *pCur, i64 *pnEntry){ nEntry += pPage->nCell; } - /* pPage is a leaf node. This loop navigates the cursor so that it + /* pPage is a leaf node. This loop navigates the cursor so that it ** points to the first interior cell that it points to the parent of ** the next page in the tree that has not yet been visited. The ** pCur->aiIdx[pCur->iPage] value is set to the index of the parent cell @@ -44819,7 +44837,7 @@ SQLITE_PRIVATE int sqlite3BtreeCount(BtCursor *pCur, i64 *pnEntry){ pPage = pCur->apPage[pCur->iPage]; } - /* Descend to the child node of the cell that the cursor currently + /* Descend to the child node of the cell that the cursor currently ** points at. This is the right-child if (iIdx==pPage->nCell). */ iIdx = pCur->aiIdx[pCur->iPage]; @@ -44896,7 +44914,7 @@ static int checkRef(IntegrityCk *pCheck, Pgno iPage, char *zContext){ #ifndef SQLITE_OMIT_AUTOVACUUM /* -** Check that the entry in the pointer-map for page iChild maps to +** Check that the entry in the pointer-map for page iChild maps to ** page iParent, pointer type ptrType. If not, append an error message ** to pCheck. */ @@ -44919,8 +44937,8 @@ static void checkPtrmap( } if( ePtrmapType!=eType || iPtrmapParent!=iParent ){ - checkAppendMsg(pCheck, zContext, - "Bad ptr map entry key=%d expected=(%d,%d) got=(%d,%d)", + checkAppendMsg(pCheck, zContext, + "Bad ptr map entry key=%d expected=(%d,%d) got=(%d,%d)", iChild, eType, iParent, ePtrmapType, iPtrmapParent); } } @@ -45002,7 +45020,7 @@ static void checkList( ** Do various sanity checks on a single page of a tree. Return ** the tree depth. Root pages return 0. Parents of root pages ** return 1, and so forth. -** +** ** These checks are done: ** ** 1. Make sure that cells and freeblocks do not overlap @@ -45050,7 +45068,7 @@ static int checkTreePage( pPage->isInit = 0; if( (rc = btreeInitPage(pPage))!=0 ){ assert( rc==SQLITE_CORRUPT ); /* The only possible error from InitPage */ - checkAppendMsg(pCheck, zContext, + checkAppendMsg(pCheck, zContext, "btreeInitPage() returns error code %d", rc); releasePage(pPage); return 0; @@ -45073,7 +45091,7 @@ static int checkTreePage( sz = info.nData; if( !pPage->intKey ) sz += (int)info.nKey; assert( sz==info.nPayload ); - if( (sz>info.nLocal) + if( (sz>info.nLocal) && (&pCell[info.iOverflow]<=&pPage->aData[pBt->usableSize]) ){ int nPage = (sz - info.nLocal + usableSize - 5)/(usableSize - 4); @@ -45104,7 +45122,7 @@ static int checkTreePage( } if( !pPage->leaf ){ pgno = get4byte(&pPage->aData[pPage->hdrOffset+8]); - sqlite3_snprintf(sizeof(zContext), zContext, + sqlite3_snprintf(sizeof(zContext), zContext, "On page %d at right child: ", iPage); #ifndef SQLITE_OMIT_AUTOVACUUM if( pBt->autoVacuum ){ @@ -45113,7 +45131,7 @@ static int checkTreePage( #endif checkTreePage(pCheck, pgno, zContext); } - + /* Check for complete coverage of the page */ data = pPage->aData; @@ -45136,7 +45154,7 @@ static int checkTreePage( size = cellSizePtr(pPage, &data[pc]); } if( (pc+size-1)>=usableSize ){ - checkAppendMsg(pCheck, 0, + checkAppendMsg(pCheck, 0, "Corruption detected in cell %d on page %d",i,iPage,0); }else{ for(j=pc+size-1; j>=pc; j--) hit[j]++; @@ -45164,7 +45182,7 @@ static int checkTreePage( } } if( cnt!=data[hdr+7] ){ - checkAppendMsg(pCheck, 0, + checkAppendMsg(pCheck, 0, "Fragmentation of %d bytes reported as %d on page %d", cnt, data[hdr+7], iPage); } @@ -45257,11 +45275,11 @@ SQLITE_PRIVATE char *sqlite3BtreeIntegrityCheck( /* If the database supports auto-vacuum, make sure no tables contain ** references to pointer-map pages. */ - if( sCheck.anRef[i]==0 && + if( sCheck.anRef[i]==0 && (PTRMAP_PAGENO(pBt, i)!=i || !pBt->autoVacuum) ){ checkAppendMsg(&sCheck, 0, "Page %d is never used", i); } - if( sCheck.anRef[i]!=0 && + if( sCheck.anRef[i]!=0 && (PTRMAP_PAGENO(pBt, i)==i && pBt->autoVacuum) ){ checkAppendMsg(&sCheck, 0, "Pointer map page %d is referenced", i); } @@ -45273,7 +45291,7 @@ SQLITE_PRIVATE char *sqlite3BtreeIntegrityCheck( ** of the integrity check. */ if( NEVER(nRef != sqlite3PagerRefcount(pBt->pPager)) ){ - checkAppendMsg(&sCheck, 0, + checkAppendMsg(&sCheck, 0, "Outstanding page count goes from %d to %d during this analysis", nRef, sqlite3PagerRefcount(pBt->pPager) ); @@ -45344,20 +45362,20 @@ SQLITE_PRIVATE int sqlite3BtreeIsInBackup(Btree *p){ /* ** This function returns a pointer to a blob of memory associated with ** a single shared-btree. The memory is used by client code for its own -** purposes (for example, to store a high-level schema associated with +** purposes (for example, to store a high-level schema associated with ** the shared-btree). The btree layer manages reference counting issues. ** ** The first time this is called on a shared-btree, nBytes bytes of memory -** are allocated, zeroed, and returned to the caller. For each subsequent +** are allocated, zeroed, and returned to the caller. For each subsequent ** call the nBytes parameter is ignored and a pointer to the same blob -** of memory returned. +** of memory returned. ** ** If the nBytes parameter is 0 and the blob of memory has not yet been ** allocated, a null pointer is returned. If the blob has already been ** allocated, it is returned as normal. ** -** Just before the shared-btree is closed, the function passed as the -** xFree argument when the memory allocation was made is invoked on the +** Just before the shared-btree is closed, the function passed as the +** xFree argument when the memory allocation was made is invoked on the ** blob of allocated memory. This function should not call sqlite3_free() ** on the memory, the btree layer does that. */ @@ -45373,8 +45391,8 @@ SQLITE_PRIVATE void *sqlite3BtreeSchema(Btree *p, int nBytes, void(*xFree)(void } /* -** Return SQLITE_LOCKED_SHAREDCACHE if another user of the same shared -** btree as the argument handle holds an exclusive lock on the +** Return SQLITE_LOCKED_SHAREDCACHE if another user of the same shared +** btree as the argument handle holds an exclusive lock on the ** sqlite_master table. Otherwise SQLITE_OK. */ SQLITE_PRIVATE int sqlite3BtreeSchemaLocked(Btree *p){ @@ -45415,11 +45433,11 @@ SQLITE_PRIVATE int sqlite3BtreeLockTable(Btree *p, int iTab, u8 isWriteLock){ #ifndef SQLITE_OMIT_INCRBLOB /* -** Argument pCsr must be a cursor opened for writing on an -** INTKEY table currently pointing at a valid table entry. +** Argument pCsr must be a cursor opened for writing on an +** INTKEY table currently pointing at a valid table entry. ** This function modifies the data stored as part of that entry. ** -** Only the data content may only be modified, it is not possible to +** Only the data content may only be modified, it is not possible to ** change the length of the data stored. If this function is called with ** parameters that attempt to write past the end of the existing data, ** no modifications are made and SQLITE_CORRUPT is returned. @@ -45439,7 +45457,7 @@ SQLITE_PRIVATE int sqlite3BtreePutData(BtCursor *pCsr, u32 offset, u32 amt, void return SQLITE_ABORT; } - /* Check some assumptions: + /* Check some assumptions: ** (a) the cursor is open for writing, ** (b) there is a read/write transaction open, ** (c) the connection holds a write-lock on the table (if required), @@ -45457,8 +45475,8 @@ SQLITE_PRIVATE int sqlite3BtreePutData(BtCursor *pCsr, u32 offset, u32 amt, void return accessPayload(pCsr, offset, amt, (unsigned char *)z, 1); } -/* -** Set a flag on this cursor to cache the locations of pages from the +/* +** Set a flag on this cursor to cache the locations of pages from the ** overflow list for the current row. This is used by cursors opened ** for incremental blob IO only. ** @@ -45489,10 +45507,8 @@ SQLITE_PRIVATE void sqlite3BtreeCacheOverflow(BtCursor *pCur){ ** May you share freely, never taking more than you give. ** ************************************************************************* -** This file contains the implementation of the sqlite3_backup_XXX() +** This file contains the implementation of the sqlite3_backup_XXX() ** API functions and the related features. -** -** $Id: backup.c,v 1.19 2009/07/06 19:03:13 drh Exp $ */ /* Macro to find the minimum of two numeric values. @@ -45532,15 +45548,15 @@ struct sqlite3_backup { ** Once it has been created using backup_init(), a single sqlite3_backup ** structure may be accessed via two groups of thread-safe entry points: ** -** * Via the sqlite3_backup_XXX() API function backup_step() and +** * Via the sqlite3_backup_XXX() API function backup_step() and ** backup_finish(). Both these functions obtain the source database -** handle mutex and the mutex associated with the source BtShared +** handle mutex and the mutex associated with the source BtShared ** structure, in that order. ** ** * Via the BackupUpdate() and BackupRestart() functions, which are ** invoked by the pager layer to report various state changes in ** the page cache associated with the source database. The mutex -** associated with the source database BtShared structure will always +** associated with the source database BtShared structure will always ** be held when either of these functions are invoked. ** ** The other sqlite3_backup_XXX() API functions, backup_remaining() and @@ -45561,8 +45577,8 @@ struct sqlite3_backup { ** in connection handle pDb. If such a database cannot be found, return ** a NULL pointer and write an error message to pErrorDb. ** -** If the "temp" database is requested, it may need to be opened by this -** function. If an error occurs while doing so, return 0 and write an +** If the "temp" database is requested, it may need to be opened by this +** function. If an error occurs while doing so, return 0 and write an ** error message to pErrorDb. */ static Btree *findBtree(sqlite3 *pErrorDb, sqlite3 *pDb, const char *zDb){ @@ -45666,7 +45682,7 @@ SQLITE_API sqlite3_backup *sqlite3_backup_init( } /* -** Argument rc is an SQLite error code. Return true if this error is +** Argument rc is an SQLite error code. Return true if this error is ** considered fatal if encountered during a backup operation. All errors ** are considered fatal except for SQLITE_BUSY and SQLITE_LOCKED. */ @@ -45675,8 +45691,8 @@ static int isFatalError(int rc){ } /* -** Parameter zSrcData points to a buffer containing the data for -** page iSrcPg from the source database. Copy this data into the +** Parameter zSrcData points to a buffer containing the data for +** page iSrcPg from the source database. Copy this data into the ** destination database. */ static int backupOnePage(sqlite3_backup *p, Pgno iSrcPg, const u8 *zSrcData){ @@ -45695,13 +45711,13 @@ static int backupOnePage(sqlite3_backup *p, Pgno iSrcPg, const u8 *zSrcData){ assert( zSrcData ); /* Catch the case where the destination is an in-memory database and the - ** page sizes of the source and destination differ. + ** page sizes of the source and destination differ. */ if( nSrcPgsz!=nDestPgsz && sqlite3PagerIsMemdb(sqlite3BtreePager(p->pDest)) ){ rc = SQLITE_READONLY; } - /* This loop runs once for each destination page spanned by the source + /* This loop runs once for each destination page spanned by the source ** page. For each iteration, variable iOff is set to the byte offset ** of the destination page. */ @@ -45720,7 +45736,7 @@ static int backupOnePage(sqlite3_backup *p, Pgno iSrcPg, const u8 *zSrcData){ ** Then clear the Btree layer MemPage.isInit flag. Both this module ** and the pager code use this trick (clearing the first byte ** of the page 'extra' space to invalidate the Btree layers - ** cached parse of the page). MemPage.isInit is marked + ** cached parse of the page). MemPage.isInit is marked ** "MUST BE FIRST" for this purpose. */ memcpy(zOut, zIn, nCopy); @@ -45737,7 +45753,7 @@ static int backupOnePage(sqlite3_backup *p, Pgno iSrcPg, const u8 *zSrcData){ ** exactly iSize bytes. If pFile is not larger than iSize bytes, then ** this function is a no-op. ** -** Return SQLITE_OK if everything is successful, or an SQLite error +** Return SQLITE_OK if everything is successful, or an SQLite error ** code if an error occurs. */ static int backupTruncateFile(sqlite3_file *pFile, i64 iSize){ @@ -45793,7 +45809,7 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ /* Lock the destination database, if it is not locked already. */ if( SQLITE_OK==rc && p->bDestLocked==0 - && SQLITE_OK==(rc = sqlite3BtreeBeginTrans(p->pDest, 2)) + && SQLITE_OK==(rc = sqlite3BtreeBeginTrans(p->pDest, 2)) ){ p->bDestLocked = 1; sqlite3BtreeGetMeta(p->pDest, BTREE_SCHEMA_VERSION, &p->iDestSchema); @@ -45807,7 +45823,7 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ rc = sqlite3BtreeBeginTrans(p->pSrc, 0); bCloseTrans = 1; } - + /* Now that there is a read-lock on the source database, query the ** source pager for the number of pages in the database. */ @@ -45835,31 +45851,31 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ attachBackupObject(p); } } - + /* Update the schema version field in the destination database. This ** is to make sure that the schema-version really does change in ** the case where the source and destination databases have the ** same schema version. */ - if( rc==SQLITE_DONE + if( rc==SQLITE_DONE && (rc = sqlite3BtreeUpdateMeta(p->pDest,1,p->iDestSchema+1))==SQLITE_OK ){ const int nSrcPagesize = sqlite3BtreeGetPageSize(p->pSrc); const int nDestPagesize = sqlite3BtreeGetPageSize(p->pDest); int nDestTruncate; - + if( p->pDestDb ){ sqlite3ResetInternalSchema(p->pDestDb, 0); } /* Set nDestTruncate to the final number of pages in the destination ** database. The complication here is that the destination page - ** size may be different to the source page size. + ** size may be different to the source page size. ** - ** If the source page size is smaller than the destination page size, + ** If the source page size is smaller than the destination page size, ** round up. In this case the call to sqlite3OsTruncate() below will ** fix the size of the file. However it is important to call - ** sqlite3PagerTruncateImage() here so that any pages in the + ** sqlite3PagerTruncateImage() here so that any pages in the ** destination file that lie beyond the nDestTruncate page mark are ** journalled by PagerCommitPhaseOne() before they are destroyed ** by the file truncation. @@ -45881,7 +45897,7 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ ** ** * The destination may need to be truncated, and ** - ** * Data stored on the pages immediately following the + ** * Data stored on the pages immediately following the ** pending-byte page in the source database may need to be ** copied into the destination database. */ @@ -45900,8 +45916,8 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ i64 iOff; i64 iEnd = MIN(PENDING_BYTE + nDestPagesize, iSize); for( - iOff=PENDING_BYTE+nSrcPagesize; - rc==SQLITE_OK && iOffpDest)) @@ -45925,7 +45941,7 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ rc = SQLITE_DONE; } } - + /* If bCloseTrans is true, then this function opened a read transaction ** on the source database. Close the read transaction here. There is ** no need to check the return values of the btree methods here, as @@ -45937,7 +45953,7 @@ SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){ TESTONLY( rc2 |= ) sqlite3BtreeCommitPhaseTwo(p->pSrc); assert( rc2==SQLITE_OK ); } - + p->rc = rc; } if( p->pDestDb ){ @@ -46005,7 +46021,7 @@ SQLITE_API int sqlite3_backup_remaining(sqlite3_backup *p){ } /* -** Return the total number of pages in the source database as of the most +** Return the total number of pages in the source database as of the most ** recent call to sqlite3_backup_step(). */ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p){ @@ -46014,7 +46030,7 @@ SQLITE_API int sqlite3_backup_pagecount(sqlite3_backup *p){ /* ** This function is called after the contents of page iPage of the -** source database have been modified. If page iPage has already been +** source database have been modified. If page iPage has already been ** copied into the destination database, then the data written to the ** destination is now invalidated. The destination copy of iPage needs ** to be updated with the new data before the backup operation is @@ -46046,7 +46062,7 @@ SQLITE_PRIVATE void sqlite3BackupUpdate(sqlite3_backup *pBackup, Pgno iPage, con ** Restart the backup process. This is called when the pager layer ** detects that the database has been modified by an external database ** connection. In this case there is no way of knowing which of the -** pages that have been copied into the destination database are still +** pages that have been copied into the destination database are still ** valid and which are not, so the entire process needs to be restarted. ** ** It is assumed that the mutex associated with the BtShared object @@ -46066,8 +46082,8 @@ SQLITE_PRIVATE void sqlite3BackupRestart(sqlite3_backup *pBackup){ ** Copy the complete content of pBtFrom into pBtTo. A transaction ** must be active for both files. ** -** The size of file pTo may be reduced by this operation. If anything -** goes wrong, the transaction on pTo is rolled back. If successful, the +** The size of file pTo may be reduced by this operation. If anything +** goes wrong, the transaction on pTo is rolled back. If successful, the ** transaction is committed before returning. */ SQLITE_PRIVATE int sqlite3BtreeCopyFile(Btree *pTo, Btree *pFrom){ @@ -46089,9 +46105,9 @@ SQLITE_PRIVATE int sqlite3BtreeCopyFile(Btree *pTo, Btree *pFrom){ /* 0x7FFFFFFF is the hard limit for the number of pages in a database ** file. By passing this as the number of pages to copy to - ** sqlite3_backup_step(), we can guarantee that the copy finishes + ** sqlite3_backup_step(), we can guarantee that the copy finishes ** within a single call (unless an error occurs). The assert() statement - ** checks this assumption - (p->rc) should be set to either SQLITE_DONE + ** checks this assumption - (p->rc) should be set to either SQLITE_DONE ** or an error code. */ sqlite3_backup_step(&b, 0x7FFFFFFF); @@ -46125,8 +46141,6 @@ SQLITE_PRIVATE int sqlite3BtreeCopyFile(Btree *pTo, Btree *pFrom){ ** stores a single value in the VDBE. Mem is an opaque structure visible ** only within the VDBE. Interface routines refer to a Mem using the ** name sqlite_value -** -** $Id: vdbemem.c,v 1.152 2009/07/22 18:07:41 drh Exp $ */ /* @@ -46173,23 +46187,23 @@ SQLITE_PRIVATE int sqlite3VdbeChangeEncoding(Mem *pMem, int desiredEnc){ } /* -** Make sure pMem->z points to a writable allocation of at least +** Make sure pMem->z points to a writable allocation of at least ** n bytes. ** ** If the memory cell currently contains string or blob data -** and the third argument passed to this function is true, the +** and the third argument passed to this function is true, the ** current content of the cell is preserved. Otherwise, it may -** be discarded. +** be discarded. ** ** This function sets the MEM_Dyn flag and clears any xDel callback. -** It also clears MEM_Ephem and MEM_Static. If the preserve flag is +** It also clears MEM_Ephem and MEM_Static. If the preserve flag is ** not set, Mem.n is zeroed. */ SQLITE_PRIVATE int sqlite3VdbeMemGrow(Mem *pMem, int n, int preserve){ assert( 1 >= ((pMem->zMalloc && pMem->zMalloc==pMem->z) ? 1 : 0) + - (((pMem->flags&MEM_Dyn)&&pMem->xDel) ? 1 : 0) + - ((pMem->flags&MEM_Ephem) ? 1 : 0) + + (((pMem->flags&MEM_Dyn)&&pMem->xDel) ? 1 : 0) + + ((pMem->flags&MEM_Ephem) ? 1 : 0) + ((pMem->flags&MEM_Static) ? 1 : 0) ); assert( (pMem->flags&MEM_RowSet)==0 ); @@ -46328,7 +46342,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemStringify(Mem *pMem, int enc){ /* For a Real or Integer, use sqlite3_mprintf() to produce the UTF-8 ** string representation of the value. Then, if the required encoding ** is UTF-16le or UTF-16be do a translation. - ** + ** ** FIX ME: It would be better if sqlite3_snprintf() could do UTF-16. */ if( fg & MEM_Int ){ @@ -46669,7 +46683,7 @@ SQLITE_PRIVATE void sqlite3VdbeMemSetRowSet(Mem *pMem){ pMem->flags = MEM_Null; }else{ assert( pMem->zMalloc ); - pMem->u.pRowSet = sqlite3RowSetInit(db, pMem->zMalloc, + pMem->u.pRowSet = sqlite3RowSetInit(db, pMem->zMalloc, sqlite3DbMallocSize(db, pMem->zMalloc)); assert( pMem->u.pRowSet!=0 ); pMem->flags = MEM_RowSet; @@ -46689,7 +46703,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemTooBig(Mem *p){ } return n>p->db->aLimit[SQLITE_LIMIT_LENGTH]; } - return 0; + return 0; } /* @@ -46759,8 +46773,8 @@ SQLITE_PRIVATE void sqlite3VdbeMemMove(Mem *pTo, Mem *pFrom){ ** Change the value of a Mem to be a string or a BLOB. ** ** The memory management strategy depends on the value of the xDel -** parameter. If the value passed is SQLITE_TRANSIENT, then the -** string is copied into a (possibly existing) buffer managed by the +** parameter. If the value passed is SQLITE_TRANSIENT, then the +** string is copied into a (possibly existing) buffer managed by the ** Mem structure. Otherwise, any existing buffer is freed and the ** pointer copied. ** @@ -46865,14 +46879,11 @@ SQLITE_PRIVATE int sqlite3MemCompare(const Mem *pMem1, const Mem *pMem2, const C int f1, f2; int combined_flags; - /* Interchange pMem1 and pMem2 if the collating sequence specifies - ** DESC order. - */ f1 = pMem1->flags; f2 = pMem2->flags; combined_flags = f1|f2; assert( (combined_flags & MEM_RowSet)==0 ); - + /* If one value is NULL, it is less than the other. If both values ** are NULL, return 0. */ @@ -46927,7 +46938,7 @@ SQLITE_PRIVATE int sqlite3MemCompare(const Mem *pMem1, const Mem *pMem2, const C } assert( pMem1->enc==pMem2->enc ); - assert( pMem1->enc==SQLITE_UTF8 || + assert( pMem1->enc==SQLITE_UTF8 || pMem1->enc==SQLITE_UTF16LE || pMem1->enc==SQLITE_UTF16BE ); /* The collation sequence must be defined at this point, even if @@ -46963,7 +46974,7 @@ SQLITE_PRIVATE int sqlite3MemCompare(const Mem *pMem1, const Mem *pMem2, const C /* If a NULL pointer was passed as the collate function, fall through ** to the blob case and use memcmp(). */ } - + /* Both values must be blobs. Compare using memcmp(). */ rc = memcmp(pMem1->z, pMem2->z, (pMem1->n>pMem2->n)?pMem2->n:pMem1->n); if( rc==0 ){ @@ -46998,7 +47009,7 @@ SQLITE_PRIVATE int sqlite3VdbeMemFromBtree( assert( sqlite3BtreeCursorIsValid(pCur) ); - /* Note: the calls to BtreeKeyFetch() and DataFetch() below assert() + /* Note: the calls to BtreeKeyFetch() and DataFetch() below assert() ** that both the BtShared and database handle mutexes are held. */ assert( (pMem->flags & MEM_RowSet)==0 ); if( key ){ @@ -47118,7 +47129,7 @@ SQLITE_PRIVATE int sqlite3ValueFromExpr( } op = pExpr->op; if( op==TK_REGISTER ){ - op = pExpr->op2; + op = pExpr->op2; /* This only happens with SQLITE_ENABLE_STAT2 */ } if( op==TK_STRING || op==TK_FLOAT || op==TK_INTEGER ){ @@ -47231,8 +47242,6 @@ SQLITE_PRIVATE int sqlite3ValueBytes(sqlite3_value *pVal, u8 enc){ ** a VDBE (or an "sqlite3_stmt" as it is known to the outside world.) Prior ** to version 2.8.7, all this code was combined into the vdbe.c source file. ** But that file was getting too big so this subroutines were split out. -** -** $Id: vdbeaux.c,v 1.480 2009/08/08 18:01:08 drh Exp $ */ @@ -47318,12 +47327,12 @@ SQLITE_PRIVATE void sqlite3VdbeTrace(Vdbe *p, FILE *trace){ #endif /* -** Resize the Vdbe.aOp array so that it is at least one op larger than +** Resize the Vdbe.aOp array so that it is at least one op larger than ** it was. ** ** If an out-of-memory error occurs while resizing the array, return -** SQLITE_NOMEM. In this case Vdbe.aOp and Vdbe.nOpAlloc remain -** unchanged (this is so that any opcodes already allocated can be +** SQLITE_NOMEM. In this case Vdbe.aOp and Vdbe.nOpAlloc remain +** unchanged (this is so that any opcodes already allocated can be ** correctly deallocated along with the rest of the Vdbe). */ static int growOpArray(Vdbe *p){ @@ -47413,6 +47422,22 @@ SQLITE_PRIVATE int sqlite3VdbeAddOp4( return addr; } +/* +** Add an opcode that includes the p4 value as an integer. +*/ +SQLITE_PRIVATE int sqlite3VdbeAddOp4Int( + Vdbe *p, /* Add the opcode to this VM */ + int op, /* The new opcode */ + int p1, /* The P1 operand */ + int p2, /* The P2 operand */ + int p3, /* The P3 operand */ + int p4 /* The P4 operand as an integer */ +){ + int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3); + sqlite3VdbeChangeP4(p, addr, SQLITE_INT_TO_PTR(p4), P4_INT32); + return addr; +} + /* ** Create a new symbolic label for an instruction that has yet to be ** coded. The symbolic label is really just a negative number. The @@ -47461,19 +47486,19 @@ SQLITE_PRIVATE void sqlite3VdbeResolveLabel(Vdbe *p, int x){ /* ** The following type and function are used to iterate through all opcodes -** in a Vdbe main program and each of the sub-programs (triggers) it may +** in a Vdbe main program and each of the sub-programs (triggers) it may ** invoke directly or indirectly. It should be used as follows: ** ** Op *pOp; ** VdbeOpIter sIter; ** ** memset(&sIter, 0, sizeof(sIter)); -** sIter.v = v; // v is of type Vdbe* +** sIter.v = v; // v is of type Vdbe* ** while( (pOp = opIterNext(&sIter)) ){ ** // Do something with pOp ** } ** sqlite3DbFree(v->db, sIter.apSub); -** +** */ typedef struct VdbeOpIter VdbeOpIter; struct VdbeOpIter { @@ -47506,7 +47531,7 @@ static Op *opIterNext(VdbeOpIter *p){ p->iSub++; p->iAddr = 0; } - + if( pRet->p4type==P4_SUBPROGRAM ){ int nByte = (p->nSub+1)*sizeof(SubProgram*); int j; @@ -47556,11 +47581,11 @@ SQLITE_PRIVATE int sqlite3VdbeAssertMayAbort(Vdbe *v, int mayAbort){ while( (pOp = opIterNext(&sIter))!=0 ){ int opcode = pOp->opcode; - if( opcode==OP_Destroy || opcode==OP_VUpdate || opcode==OP_VRename + if( opcode==OP_Destroy || opcode==OP_VUpdate || opcode==OP_VRename #ifndef SQLITE_OMIT_FOREIGN_KEY - || (opcode==OP_FkCounter && pOp->p1==0 && pOp->p2==1) + || (opcode==OP_FkCounter && pOp->p1==0 && pOp->p2==1) #endif - || ((opcode==OP_Halt || opcode==OP_HaltIfNull) + || ((opcode==OP_Halt || opcode==OP_HaltIfNull) && (pOp->p1==SQLITE_CONSTRAINT && pOp->p2==OE_Abort)) ){ hasAbort = 1; @@ -47585,9 +47610,11 @@ SQLITE_PRIVATE int sqlite3VdbeAssertMayAbort(Vdbe *v, int mayAbort){ ** ** This routine is called once after all opcodes have been inserted. ** -** Variable *pMaxFuncArgs is set to the maximum value of any P2 argument -** to an OP_Function, OP_AggStep or OP_VFilter opcode. This is used by +** Variable *pMaxFuncArgs is set to the maximum value of any P2 argument +** to an OP_Function, OP_AggStep or OP_VFilter opcode. This is used by ** sqlite3VdbeMakeReady() to size the Vdbe.apArg[] array. +** +** The Op.opflags field is set on all opcodes. */ static void resolveP2Values(Vdbe *p, int *pMaxFuncArgs){ int i; @@ -47598,15 +47625,14 @@ static void resolveP2Values(Vdbe *p, int *pMaxFuncArgs){ for(pOp=p->aOp, i=p->nOp-1; i>=0; i--, pOp++){ u8 opcode = pOp->opcode; + pOp->opflags = sqlite3OpcodeProperty[opcode]; if( opcode==OP_Function || opcode==OP_AggStep ){ if( pOp->p5>nMaxArgs ) nMaxArgs = pOp->p5; -#ifndef SQLITE_OMIT_VIRTUALTABLE - }else if( opcode==OP_VUpdate ){ - if( pOp->p2>nMaxArgs ) nMaxArgs = pOp->p2; -#endif }else if( opcode==OP_Transaction && pOp->p2!=0 ){ p->readOnly = 0; #ifndef SQLITE_OMIT_VIRTUALTABLE + }else if( opcode==OP_VUpdate ){ + if( pOp->p2>nMaxArgs ) nMaxArgs = pOp->p2; }else if( opcode==OP_VFilter ){ int n; assert( p->nOp - i >= 3 ); @@ -47616,7 +47642,7 @@ static void resolveP2Values(Vdbe *p, int *pMaxFuncArgs){ #endif } - if( sqlite3VdbeOpcodeHasProperty(opcode, OPFLG_JUMP) && pOp->p2<0 ){ + if( (pOp->opflags & OPFLG_JUMP)!=0 && pOp->p2<0 ){ assert( -1-pOp->p2nLabel ); pOp->p2 = aLabel[-1-pOp->p2]; } @@ -47638,12 +47664,12 @@ SQLITE_PRIVATE int sqlite3VdbeCurrentAddr(Vdbe *p){ /* ** This function returns a pointer to the array of opcodes associated with ** the Vdbe passed as the first argument. It is the callers responsibility -** to arrange for the returned array to be eventually freed using the +** to arrange for the returned array to be eventually freed using the ** vdbeFreeOpArray() function. ** ** Before returning, *pnOp is set to the number of entries in the returned -** array. Also, *pnMaxArg is set to the larger of its current value and -** the number of entries in the Vdbe.apArg[] array required to execute the +** array. Also, *pnMaxArg is set to the larger of its current value and +** the number of entries in the Vdbe.apArg[] array required to execute the ** returned program. */ SQLITE_PRIVATE VdbeOp *sqlite3VdbeTakeOpArray(Vdbe *p, int *pnOp, int *pnMaxArg){ @@ -47678,7 +47704,7 @@ SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe *p, int nOp, VdbeOpList const *aOp) VdbeOp *pOut = &p->aOp[i+addr]; pOut->opcode = pIn->opcode; pOut->p1 = pIn->p1; - if( p2<0 && sqlite3VdbeOpcodeHasProperty(pOut->opcode, OPFLG_JUMP) ){ + if( p2<0 && (sqlite3OpcodeProperty[pOut->opcode] & OPFLG_JUMP)!=0 ){ pOut->p2 = addr + ADDR(p2); }else{ pOut->p2 = p2; @@ -47812,8 +47838,8 @@ static void freeP4(sqlite3 *db, int p4type, void *p4){ /* ** Free the space allocated for aOp and any p4 values allocated for the -** opcodes contained within. If aOp is not NULL it is assumed to contain -** nOp entries. +** opcodes contained within. If aOp is not NULL it is assumed to contain +** nOp entries. */ static void vdbeFreeOpArray(sqlite3 *db, Op *aOp, int nOp){ if( aOp ){ @@ -47822,7 +47848,7 @@ static void vdbeFreeOpArray(sqlite3 *db, Op *aOp, int nOp){ freeP4(db, pOp->p4type, pOp->p4.p); #ifdef SQLITE_DEBUG sqlite3DbFree(db, pOp->zComment); -#endif +#endif } } sqlite3DbFree(db, aOp); @@ -47889,10 +47915,10 @@ SQLITE_PRIVATE void sqlite3VdbeChangeToNoop(Vdbe *p, int addr, int N){ ** A copy is made of the KeyInfo structure into memory obtained from ** sqlite3_malloc, to be freed when the Vdbe is finalized. ** n==P4_KEYINFO_HANDOFF indicates that zP4 points to a KeyInfo structure -** stored in memory that the caller has obtained from sqlite3_malloc. The +** stored in memory that the caller has obtained from sqlite3_malloc. The ** caller should not free the allocation, it will be freed when the Vdbe is ** finalized. -** +** ** Other values of n (P4_STATIC, P4_COLLSEQ etc.) indicate that zP4 points ** to a string or structure that is guaranteed to exist for the lifetime of ** the Vdbe. In these cases we can just copy the pointer. @@ -48167,7 +48193,7 @@ SQLITE_PRIVATE void sqlite3VdbePrintOp(FILE *pOut, int pc, Op *pOp){ static const char *zFormat1 = "%4d %-13s %4d %4d %4d %-4s %.2X %s\n"; if( pOut==0 ) pOut = stdout; zP4 = displayP4(pOp, zPtr, sizeof(zPtr)); - fprintf(pOut, zFormat1, pc, + fprintf(pOut, zFormat1, pc, sqlite3OpcodeName(pOp->opcode), pOp->p1, pOp->p2, pOp->p3, zP4, pOp->p5, #ifdef SQLITE_DEBUG pOp->zComment ? pOp->zComment : "" @@ -48191,15 +48217,15 @@ static void releaseMemArray(Mem *p, int N){ assert( (&p[1])==pEnd || p[0].db==p[1].db ); /* This block is really an inlined version of sqlite3VdbeMemRelease() - ** that takes advantage of the fact that the memory cell value is + ** that takes advantage of the fact that the memory cell value is ** being set to NULL after releasing any dynamic resources. ** - ** The justification for duplicating code is that according to - ** callgrind, this causes a certain test case to hit the CPU 4.7 - ** percent less (x86 linux, gcc version 4.1.2, -O6) than if + ** The justification for duplicating code is that according to + ** callgrind, this causes a certain test case to hit the CPU 4.7 + ** percent less (x86 linux, gcc version 4.1.2, -O6) than if ** sqlite3MemRelease() were called from here. With -O2, this jumps - ** to 6.6 percent. The test case is inserting 1000 rows into a table - ** with no indexes using a single prepared INSERT statement, bind() + ** to 6.6 percent. The test case is inserting 1000 rows into a table + ** with no indexes using a single prepared INSERT statement, bind() ** and reset(). Inserts are grouped into a transaction. */ if( p->flags&(MEM_Agg|MEM_Dyn|MEM_Frame|MEM_RowSet) ){ @@ -48273,7 +48299,7 @@ SQLITE_PRIVATE int sqlite3VdbeList( return SQLITE_ERROR; } - /* Figure out total number of rows that will be returned by this + /* Figure out total number of rows that will be returned by this ** EXPLAIN program. */ nRow = p->nOp; if( p->explain==1 ){ @@ -48315,7 +48341,7 @@ SQLITE_PRIVATE int sqlite3VdbeList( pMem->type = SQLITE_INTEGER; pMem->u.i = i; /* Program counter */ pMem++; - + pMem->flags = MEM_Static|MEM_Str|MEM_Term; pMem->z = (char*)sqlite3OpcodeName(pOp->opcode); /* Opcode */ assert( pMem->z!=0 ); @@ -48383,7 +48409,7 @@ SQLITE_PRIVATE int sqlite3VdbeList( pMem->type = SQLITE_TEXT; pMem->enc = SQLITE_UTF8; pMem++; - + #ifdef SQLITE_DEBUG if( pOp->zComment ){ pMem->flags = MEM_Str|MEM_Term; @@ -48493,7 +48519,7 @@ static void allocSpace( ** Prepare a virtual machine for execution. This involves things such ** as allocating stack space and initializing the program counter. ** After the VDBE has be prepped, it can be executed by one or more -** calls to sqlite3VdbeExec(). +** calls to sqlite3VdbeExec(). ** ** This is the only way to move a VDBE from VDBE_MAGIC_INIT to ** VDBE_MAGIC_RUN. @@ -48501,7 +48527,7 @@ static void allocSpace( ** This function may be called more than once on a single virtual machine. ** The first call is made while compiling the SQL statement. Subsequent ** calls are made as part of the process of resetting a statement to be -** re-executed (from a call to sqlite3_reset()). The nVar, nMem, nCursor +** re-executed (from a call to sqlite3_reset()). The nVar, nMem, nCursor ** and isExplain parameters are only passed correct values the first time ** the function is called. On subsequent calls, from sqlite3_reset(), nVar ** is passed -1 and nMem, nCursor and isExplain are all passed zero. @@ -48531,7 +48557,7 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady( /* For each cursor required, also allocate a memory cell. Memory ** cells (nMem+1-nCursor)..nMem, inclusive, will never be used by ** the vdbe program. Instead they are used to allocate space for - ** VdbeCursor/BtCursor structures. The blob of memory associated with + ** VdbeCursor/BtCursor structures. The blob of memory associated with ** cursor 0 is stored in memory cell nMem. Memory cell (nMem-1) ** stores the blob of memory associated with cursor 1, etc. ** @@ -48539,7 +48565,7 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady( */ nMem += nCursor; - /* Allocate space for memory registers, SQL variables, VDBE cursors and + /* Allocate space for memory registers, SQL variables, VDBE cursors and ** an array to marshal SQL function arguments in. This is only done the ** first time this function is called for a given VDBE, not when it is ** being called from sqlite3_reset() to reset the virtual machine. @@ -48563,7 +48589,7 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady( allocSpace((char*)&p->aVar, nVar*sizeof(Mem), &zCsr, zEnd, &nByte); allocSpace((char*)&p->apArg, nArg*sizeof(Mem*), &zCsr, zEnd, &nByte); allocSpace((char*)&p->azVar, nVar*sizeof(char*), &zCsr, zEnd, &nByte); - allocSpace((char*)&p->apCsr, + allocSpace((char*)&p->apCsr, nCursor*sizeof(VdbeCursor*), &zCsr, zEnd, &nByte ); if( nByte ){ @@ -48617,7 +48643,7 @@ SQLITE_PRIVATE void sqlite3VdbeMakeReady( } /* -** Close a VDBE cursor and release all the resources that cursor +** Close a VDBE cursor and release all the resources that cursor ** happens to hold. */ SQLITE_PRIVATE void sqlite3VdbeFreeCursor(Vdbe *p, VdbeCursor *pCx){ @@ -48665,7 +48691,7 @@ SQLITE_PRIVATE int sqlite3VdbeFrameRestore(VdbeFrame *pFrame){ /* ** Close all cursors. ** -** Also release any dynamic memory held by the VM in the Vdbe.aMem memory +** Also release any dynamic memory held by the VM in the Vdbe.aMem memory ** cell array. This is necessary as the memory cell array may contain ** pointers to VdbeFrame objects, which may in turn contain pointers to ** open cursors. @@ -48705,7 +48731,7 @@ static void Cleanup(Vdbe *p){ sqlite3 *db = p->db; #ifdef SQLITE_DEBUG - /* Execute assert() statements to ensure that the Vdbe.apCsr[] and + /* Execute assert() statements to ensure that the Vdbe.apCsr[] and ** Vdbe.aMem[] arrays have already been cleaned up. */ int i; for(i=0; inCursor; i++) assert( p->apCsr==0 || p->apCsr[i]==0 ); @@ -48786,15 +48812,15 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ int needXcommit = 0; #ifdef SQLITE_OMIT_VIRTUALTABLE - /* With this option, sqlite3VtabSync() is defined to be simply - ** SQLITE_OK so p is not used. + /* With this option, sqlite3VtabSync() is defined to be simply + ** SQLITE_OK so p is not used. */ UNUSED_PARAMETER(p); #endif /* Before doing anything else, call the xSync() callback for any ** virtual module tables written in this transaction. This has to - ** be done before determining whether a master journal file is + ** be done before determining whether a master journal file is ** required, as an xSync() callback may add an attached database ** to the transaction. */ @@ -48804,12 +48830,12 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ } /* This loop determines (a) if the commit hook should be invoked and - ** (b) how many database files have open write transactions, not - ** including the temp database. (b) is important because if more than + ** (b) how many database files have open write transactions, not + ** including the temp database. (b) is important because if more than ** one database file has an open write transaction, a master journal ** file is required for an atomic commit. - */ - for(i=0; inDb; i++){ + */ + for(i=0; inDb; i++){ Btree *pBt = db->aDb[i].pBt; if( sqlite3BtreeIsInTrans(pBt) ){ needXcommit = 1; @@ -48832,8 +48858,8 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ ** master-journal. ** ** If the return value of sqlite3BtreeGetFilename() is a zero length - ** string, it means the main database is :memory: or a temp file. In - ** that case we do not support atomic multi-file commits, so use the + ** string, it means the main database is :memory: or a temp file. In + ** that case we do not support atomic multi-file commits, so use the ** simple case then too. */ if( 0==sqlite3Strlen30(sqlite3BtreeGetFilename(db->aDb[0].pBt)) @@ -48846,7 +48872,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ } } - /* Do the commit only if all databases successfully complete phase 1. + /* Do the commit only if all databases successfully complete phase 1. ** If one of the BtreeCommitPhaseOne() calls fails, this indicates an ** IO error while deleting or truncating a journal file. It is unlikely, ** but could happen. In this case abandon processing and return the error. @@ -48889,7 +48915,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ }while( rc==SQLITE_OK && res ); if( rc==SQLITE_OK ){ /* Open the master journal. */ - rc = sqlite3OsOpenMalloc(pVfs, zMaster, &pMaster, + rc = sqlite3OsOpenMalloc(pVfs, zMaster, &pMaster, SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE| SQLITE_OPEN_EXCLUSIVE|SQLITE_OPEN_MASTER_JOURNAL, 0 ); @@ -48898,7 +48924,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ sqlite3DbFree(db, zMaster); return rc; } - + /* Write the name of each database file in the transaction into the new ** master journal file. If an error occurs at this point close ** and delete the master journal file. All the individual journal files @@ -48928,7 +48954,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ /* Sync the master journal file. If the IOCAP_SEQUENTIAL device ** flag is set this is not required. */ - if( needSync + if( needSync && 0==(sqlite3OsDeviceCharacteristics(pMaster)&SQLITE_IOCAP_SEQUENTIAL) && SQLITE_OK!=(rc = sqlite3OsSync(pMaster, SQLITE_SYNC_NORMAL)) ){ @@ -48948,7 +48974,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ ** in case the master journal file name was written into the journal ** file before the failure occurred. */ - for(i=0; rc==SQLITE_OK && inDb; i++){ + for(i=0; rc==SQLITE_OK && inDb; i++){ Btree *pBt = db->aDb[i].pBt; if( pBt ){ rc = sqlite3BtreeCommitPhaseOne(pBt, zMaster); @@ -48980,7 +49006,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ */ disable_simulated_io_errors(); sqlite3BeginBenignMalloc(); - for(i=0; inDb; i++){ + for(i=0; inDb; i++){ Btree *pBt = db->aDb[i].pBt; if( pBt ){ sqlite3BtreeCommitPhaseTwo(pBt); @@ -48996,7 +49022,7 @@ static int vdbeCommit(sqlite3 *db, Vdbe *p){ return rc; } -/* +/* ** This routine checks that the sqlite3.activeVdbeCnt count variable ** matches the number of vdbe's in the list sqlite3.pVdbe that are ** currently active. An assertion fails if the two counts do not match. @@ -49026,7 +49052,7 @@ static void checkActiveVdbeCnt(sqlite3 *db){ #endif /* -** For every Btree that in database connection db which +** For every Btree that in database connection db which ** has been modified, "trip" or invalidate each cursor in ** that Btree might have been modified so that the cursor ** can never be used again. This happens when a rollback @@ -49055,17 +49081,17 @@ static void invalidateCursorsOnModifiedBtrees(sqlite3 *db){ ** If the Vdbe passed as the first argument opened a statement-transaction, ** close it now. Argument eOp must be either SAVEPOINT_ROLLBACK or ** SAVEPOINT_RELEASE. If it is SAVEPOINT_ROLLBACK, then the statement -** transaction is rolled back. If eOp is SAVEPOINT_RELEASE, then the +** transaction is rolled back. If eOp is SAVEPOINT_RELEASE, then the ** statement transaction is commtted. ** -** If an IO error occurs, an SQLITE_IOERR_XXX error code is returned. +** If an IO error occurs, an SQLITE_IOERR_XXX error code is returned. ** Otherwise SQLITE_OK. */ SQLITE_PRIVATE int sqlite3VdbeCloseStatement(Vdbe *p, int eOp){ sqlite3 *const db = p->db; int rc = SQLITE_OK; - /* If p->iStatement is greater than zero, then this Vdbe opened a + /* If p->iStatement is greater than zero, then this Vdbe opened a ** statement transaction that should be closed here. The only exception ** is that an IO error may have occured, causing an emergency rollback. ** In this case (db->nStatement==0), and there is nothing to do. @@ -49078,7 +49104,7 @@ SQLITE_PRIVATE int sqlite3VdbeCloseStatement(Vdbe *p, int eOp){ assert( db->nStatement>0 ); assert( p->iStatement==(db->nStatement+db->nSavepoint) ); - for(i=0; inDb; i++){ + for(i=0; inDb; i++){ int rc2 = SQLITE_OK; Btree *pBt = db->aDb[i].pBt; if( pBt ){ @@ -49096,8 +49122,8 @@ SQLITE_PRIVATE int sqlite3VdbeCloseStatement(Vdbe *p, int eOp){ db->nStatement--; p->iStatement = 0; - /* If the statement transaction is being rolled back, also restore the - ** database handles deferred constraint counter to the value it had when + /* If the statement transaction is being rolled back, also restore the + ** database handles deferred constraint counter to the value it had when ** the statement transaction was opened. */ if( eOp==SAVEPOINT_ROLLBACK ){ db->nDeferredCons = p->nStmtDefCons; @@ -49115,7 +49141,7 @@ SQLITE_PRIVATE int sqlite3VdbeCloseStatement(Vdbe *p, int eOp){ ** ** If SQLite is not threadsafe but does support shared-cache mode, then ** sqlite3BtreeEnterAll() is invoked to set the BtShared.db variables -** of all of BtShared structures accessible via the database handle +** of all of BtShared structures accessible via the database handle ** associated with the VM. Of course only a subset of these structures ** will be accessed by the VM, and we could use Vdbe.btreeMask to figure ** that subset out, but there is no advantage to doing so. @@ -49134,12 +49160,12 @@ SQLITE_PRIVATE void sqlite3VdbeMutexArrayEnter(Vdbe *p){ #endif /* -** This function is called when a transaction opened by the database -** handle associated with the VM passed as an argument is about to be +** This function is called when a transaction opened by the database +** handle associated with the VM passed as an argument is about to be ** committed. If there are outstanding deferred foreign key constraint ** violations, return SQLITE_ERROR. Otherwise, SQLITE_OK. ** -** If there are outstanding FK violations and this function returns +** If there are outstanding FK violations and this function returns ** SQLITE_ERROR, set the result of the VM to SQLITE_CONSTRAINT and write ** an error message to it. Then return SQLITE_ERROR. */ @@ -49175,7 +49201,7 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){ /* This function contains the logic that determines if a statement or ** transaction will be committed or rolled back as a result of the - ** execution of this virtual machine. + ** execution of this virtual machine. ** ** If any of the following errors occur: ** @@ -49235,25 +49261,25 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){ if( p->rc==SQLITE_OK ){ sqlite3VdbeCheckFk(p, 0); } - - /* If the auto-commit flag is set and this is the only active writer - ** VM, then we do either a commit or rollback of the current transaction. + + /* If the auto-commit flag is set and this is the only active writer + ** VM, then we do either a commit or rollback of the current transaction. ** - ** Note: This block also runs if one of the special errors handled - ** above has occurred. + ** Note: This block also runs if one of the special errors handled + ** above has occurred. */ - if( !sqlite3VtabInSync(db) - && db->autoCommit - && db->writeVdbeCnt==(p->readOnly==0) + if( !sqlite3VtabInSync(db) + && db->autoCommit + && db->writeVdbeCnt==(p->readOnly==0) ){ if( p->rc==SQLITE_OK || (p->errorAction==OE_Fail && !isSpecialError) ){ if( sqlite3VdbeCheckFk(p, 1) ){ sqlite3BtreeMutexArrayLeave(&p->aMutex); return SQLITE_ERROR; } - /* The auto-commit flag is true, the vdbe program was successful + /* The auto-commit flag is true, the vdbe program was successful ** or hit an 'OR FAIL' constraint and there are no deferred foreign - ** key constraints to hold up the transaction. This means a commit + ** key constraints to hold up the transaction. This means a commit ** is required. */ rc = vdbeCommit(db, p); if( rc==SQLITE_BUSY ){ @@ -49282,7 +49308,7 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){ db->autoCommit = 1; } } - + /* If eStatementOp is non-zero, then a statement transaction needs to ** be committed or rolled back. Call sqlite3VdbeCloseStatement() to ** do so. If this operation returns an error, and the current statement @@ -49297,9 +49323,9 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){ p->zErrMsg = 0; } } - + /* If this was an INSERT, UPDATE or DELETE and no statement transaction - ** has been rolled back, update the database connection change-counter. + ** has been rolled back, update the database connection change-counter. */ if( p->changeCntOn ){ if( eStatementOp!=SAVEPOINT_ROLLBACK ){ @@ -49309,7 +49335,7 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){ } p->nChange = 0; } - + /* Rollback or commit any schema changes that occurred. */ if( p->rc!=SQLITE_OK && db->flags&SQLITE_InternChanges ){ sqlite3ResetInternalSchema(db, 0); @@ -49335,7 +49361,7 @@ SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe *p){ } /* If the auto-commit flag is set to true, then any locks that were held - ** by connection db have now been released. Call sqlite3ConnectionUnlocked() + ** by connection db have now been released. Call sqlite3ConnectionUnlocked() ** to invoke any required unlock-notify callbacks. */ if( db->autoCommit ){ @@ -49438,7 +49464,7 @@ SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe *p){ p->magic = VDBE_MAGIC_INIT; return p->rc & db->errMask; } - + /* ** Clean up and delete a VDBE after execution. Return an integer which is ** the result code. Write any error message text into *pzErrMsg. @@ -49638,8 +49664,8 @@ SQLITE_PRIVATE u32 sqlite3VdbeSerialTypeLen(u32 serial_type){ } /* -** If we are on an architecture with mixed-endian floating -** points (ex: ARM7) then swap the lower 4 bytes with the +** If we are on an architecture with mixed-endian floating +** points (ex: ARM7) then swap the lower 4 bytes with the ** upper 4 bytes. Return the result. ** ** For most architectures, this is a no-op. @@ -49661,7 +49687,7 @@ SQLITE_PRIVATE u32 sqlite3VdbeSerialTypeLen(u32 serial_type){ ** (2007-08-30) Frank van Vugt has studied this problem closely ** and has send his findings to the SQLite developers. Frank ** writes that some Linux kernels offer floating point hardware -** emulation that uses only 32-bit mantissas instead of a full +** emulation that uses only 32-bit mantissas instead of a full ** 48-bits as required by the IEEE standard. (This is the ** CONFIG_FPE_FASTFPE option.) On such systems, floating point ** byte swapping becomes very complicated. To avoid problems, @@ -49691,7 +49717,7 @@ static u64 floatSwap(u64 in){ #endif /* -** Write the serialized data blob for the value stored in pMem into +** Write the serialized data blob for the value stored in pMem into ** buf. It is assumed that the caller has allocated sufficient space. ** Return the number of bytes written. ** @@ -49707,7 +49733,7 @@ static u64 floatSwap(u64 in){ ** Return the number of bytes actually written into buf[]. The number ** of bytes in the zero-filled tail is included in the return value only ** if those bytes were zeroed in buf[]. -*/ +*/ SQLITE_PRIVATE u32 sqlite3VdbeSerialPut(u8 *buf, int nBuf, Mem *pMem, int file_format){ u32 serial_type = sqlite3VdbeSerialType(pMem, file_format); u32 len; @@ -49757,7 +49783,7 @@ SQLITE_PRIVATE u32 sqlite3VdbeSerialPut(u8 *buf, int nBuf, Mem *pMem, int file_f /* ** Deserialize the data blob pointed to by buf as serial type serial_type ** and store the result in pMem. Return the number of bytes read. -*/ +*/ SQLITE_PRIVATE u32 sqlite3VdbeSerialGet( const unsigned char *buf, /* Buffer to deserialize from */ u32 serial_type, /* Serial type to deserialize */ @@ -49864,7 +49890,7 @@ SQLITE_PRIVATE u32 sqlite3VdbeSerialGet( ** ** The returned structure should be closed by a call to ** sqlite3VdbeDeleteUnpackedRecord(). -*/ +*/ SQLITE_PRIVATE UnpackedRecord *sqlite3VdbeRecordUnpack( KeyInfo *pKeyInfo, /* Information about the record format */ int nKey, /* Size of the binary record */ @@ -49881,10 +49907,10 @@ SQLITE_PRIVATE UnpackedRecord *sqlite3VdbeRecordUnpack( u32 szHdr; Mem *pMem; int nOff; /* Increase pSpace by this much to 8-byte align it */ - + /* ** We want to shift the pointer pSpace up such that it is 8-byte aligned. - ** Thus, we need to calculate a value, nOff, between 0 and 7, to shift + ** Thus, we need to calculate a value, nOff, between 0 and 7, to shift ** it by. If pSpace is already 8-byte aligned, nOff should be zero. */ nOff = (8 - (SQLITE_PTR_TO_INT(pSpace) & 7)) & 7; @@ -49948,14 +49974,14 @@ SQLITE_PRIVATE void sqlite3VdbeDeleteUnpackedRecord(UnpackedRecord *p){ /* ** This function compares the two table rows or index records ** specified by {nKey1, pKey1} and pPKey2. It returns a negative, zero -** or positive integer if key1 is less than, equal to or +** or positive integer if key1 is less than, equal to or ** greater than key2. The {nKey1, pKey1} key must be a blob ** created by th OP_MakeRecord opcode of the VDBE. The pPKey2 ** key must be a parsed key such as obtained from ** sqlite3VdbeParseRecord. ** ** Key1 and Key2 do not have to contain the same number of fields. -** The key with fewer fields is usually compares less than the +** The key with fewer fields is usually compares less than the ** longer key. However if the UNPACKED_INCRKEY flags in pPKey2 is set ** and the common prefixes are equal, then key1 is less than key2. ** Or if the UNPACKED_MATCH_PREFIX flag is set and the prefixes are @@ -49988,10 +50014,18 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompare( pKeyInfo = pPKey2->pKeyInfo; mem1.enc = pKeyInfo->enc; mem1.db = pKeyInfo->db; - mem1.flags = 0; - mem1.u.i = 0; /* not needed, here to silence compiler warning */ - mem1.zMalloc = 0; - + /* mem1.flags = 0; // Will be initialized by sqlite3VdbeSerialGet() */ + VVA_ONLY( mem1.zMalloc = 0; ) /* Only needed by assert() statements */ + + /* Compilers may complain that mem1.u.i is potentially uninitialized. + ** We could initialize it, as shown here, to silence those complaints. + ** But in fact, mem1.u.i will never actually be used initialized, and doing + ** the unnecessary initialization has a measurable negative performance + ** impact, since this routine is a very high runner. And so, we choose + ** to ignore the compiler warnings and leave this variable uninitialized. + */ + /* mem1.u.i = 0; // not needed, here to silence compiler warning */ + idx1 = getVarint32(aKey1, szHdr1); d1 = szHdr1; if( pPKey2->flags & UNPACKED_IGNORE_ROWID ){ @@ -50014,50 +50048,55 @@ SQLITE_PRIVATE int sqlite3VdbeRecordCompare( rc = sqlite3MemCompare(&mem1, &pPKey2->aMem[i], iaColl[i] : 0); if( rc!=0 ){ - break; + assert( mem1.zMalloc==0 ); /* See comment below */ + + /* Invert the result if we are using DESC sort order. */ + if( pKeyInfo->aSortOrder && iaSortOrder[i] ){ + rc = -rc; + } + + /* If the PREFIX_SEARCH flag is set and all fields except the final + ** rowid field were equal, then clear the PREFIX_SEARCH flag and set + ** pPKey2->rowid to the value of the rowid field in (pKey1, nKey1). + ** This is used by the OP_IsUnique opcode. + */ + if( (pPKey2->flags & UNPACKED_PREFIX_SEARCH) && i==(pPKey2->nField-1) ){ + assert( idx1==szHdr1 && rc ); + assert( mem1.flags & MEM_Int ); + pPKey2->flags &= ~UNPACKED_PREFIX_SEARCH; + pPKey2->rowid = mem1.u.i; + } + + return rc; } i++; } - /* No memory allocation is ever used on mem1. */ - if( NEVER(mem1.zMalloc) ) sqlite3VdbeMemRelease(&mem1); - - /* If the PREFIX_SEARCH flag is set and all fields except the final - ** rowid field were equal, then clear the PREFIX_SEARCH flag and set - ** pPKey2->rowid to the value of the rowid field in (pKey1, nKey1). - ** This is used by the OP_IsUnique opcode. + /* No memory allocation is ever used on mem1. Prove this using + ** the following assert(). If the assert() fails, it indicates a + ** memory leak and a need to call sqlite3VdbeMemRelease(&mem1). */ - if( (pPKey2->flags & UNPACKED_PREFIX_SEARCH) && i==(pPKey2->nField-1) ){ - assert( idx1==szHdr1 && rc ); - assert( mem1.flags & MEM_Int ); - pPKey2->flags &= ~UNPACKED_PREFIX_SEARCH; - pPKey2->rowid = mem1.u.i; - } + assert( mem1.zMalloc==0 ); - if( rc==0 ){ - /* rc==0 here means that one of the keys ran out of fields and - ** all the fields up to that point were equal. If the UNPACKED_INCRKEY - ** flag is set, then break the tie by treating key2 as larger. - ** If the UPACKED_PREFIX_MATCH flag is set, then keys with common prefixes - ** are considered to be equal. Otherwise, the longer key is the - ** larger. As it happens, the pPKey2 will always be the longer - ** if there is a difference. - */ - if( pPKey2->flags & UNPACKED_INCRKEY ){ - rc = -1; - }else if( pPKey2->flags & UNPACKED_PREFIX_MATCH ){ - /* Leave rc==0 */ - }else if( idx1aSortOrder && inField - && pKeyInfo->aSortOrder[i] ){ - rc = -rc; + /* rc==0 here means that one of the keys ran out of fields and + ** all the fields up to that point were equal. If the UNPACKED_INCRKEY + ** flag is set, then break the tie by treating key2 as larger. + ** If the UPACKED_PREFIX_MATCH flag is set, then keys with common prefixes + ** are considered to be equal. Otherwise, the longer key is the + ** larger. As it happens, the pPKey2 will always be the longer + ** if there is a difference. + */ + assert( rc==0 ); + if( pPKey2->flags & UNPACKED_INCRKEY ){ + rc = -1; + }else if( pPKey2->flags & UNPACKED_PREFIX_MATCH ){ + /* Leave rc==0 */ + }else if( idx1mutex) ); @@ -50221,7 +50260,7 @@ SQLITE_PRIVATE sqlite3 *sqlite3VdbeDb(Vdbe *v){ /* ** Return a pointer to an sqlite3_value structure containing the value bound -** parameter iVar of VM v. Except, if the value is an SQL NULL, return +** parameter iVar of VM v. Except, if the value is an SQL NULL, return ** 0 instead. Unless it is NULL, apply affinity aff (one of the SQLITE_AFF_* ** constants) to the value before returning it. ** @@ -50274,8 +50313,6 @@ SQLITE_PRIVATE void sqlite3VdbeSetVarmask(Vdbe *v, int iVar){ ** ** This file contains code use to implement APIs that are part of the ** VDBE. -** -** $Id: vdbeapi.c,v 1.167 2009/06/25 01:47:12 drh Exp $ */ #ifndef SQLITE_OMIT_DEPRECATED @@ -50435,9 +50472,9 @@ static void setResultStrOrError( } } SQLITE_API void sqlite3_result_blob( - sqlite3_context *pCtx, - const void *z, - int n, + sqlite3_context *pCtx, + const void *z, + int n, void (*xDel)(void *) ){ assert( n>=0 ); @@ -50473,8 +50510,8 @@ SQLITE_API void sqlite3_result_null(sqlite3_context *pCtx){ sqlite3VdbeMemSetNull(&pCtx->s); } SQLITE_API void sqlite3_result_text( - sqlite3_context *pCtx, - const char *z, + sqlite3_context *pCtx, + const char *z, int n, void (*xDel)(void *) ){ @@ -50483,27 +50520,27 @@ SQLITE_API void sqlite3_result_text( } #ifndef SQLITE_OMIT_UTF16 SQLITE_API void sqlite3_result_text16( - sqlite3_context *pCtx, - const void *z, - int n, + sqlite3_context *pCtx, + const void *z, + int n, void (*xDel)(void *) ){ assert( sqlite3_mutex_held(pCtx->s.db->mutex) ); setResultStrOrError(pCtx, z, n, SQLITE_UTF16NATIVE, xDel); } SQLITE_API void sqlite3_result_text16be( - sqlite3_context *pCtx, - const void *z, - int n, + sqlite3_context *pCtx, + const void *z, + int n, void (*xDel)(void *) ){ assert( sqlite3_mutex_held(pCtx->s.db->mutex) ); setResultStrOrError(pCtx, z, n, SQLITE_UTF16BE, xDel); } SQLITE_API void sqlite3_result_text16le( - sqlite3_context *pCtx, - const void *z, - int n, + sqlite3_context *pCtx, + const void *z, + int n, void (*xDel)(void *) ){ assert( sqlite3_mutex_held(pCtx->s.db->mutex) ); @@ -50521,7 +50558,7 @@ SQLITE_API void sqlite3_result_zeroblob(sqlite3_context *pCtx, int n){ SQLITE_API void sqlite3_result_error_code(sqlite3_context *pCtx, int errCode){ pCtx->isError = errCode; if( pCtx->s.flags & MEM_Null ){ - sqlite3VdbeMemSetStr(&pCtx->s, sqlite3ErrStr(errCode), -1, + sqlite3VdbeMemSetStr(&pCtx->s, sqlite3ErrStr(errCode), -1, SQLITE_UTF8, SQLITE_STATIC); } } @@ -50530,7 +50567,7 @@ SQLITE_API void sqlite3_result_error_code(sqlite3_context *pCtx, int errCode){ SQLITE_API void sqlite3_result_error_toobig(sqlite3_context *pCtx){ assert( sqlite3_mutex_held(pCtx->s.db->mutex) ); pCtx->isError = SQLITE_TOOBIG; - sqlite3VdbeMemSetStr(&pCtx->s, "string or blob too big", -1, + sqlite3VdbeMemSetStr(&pCtx->s, "string or blob too big", -1, SQLITE_UTF8, SQLITE_STATIC); } @@ -50547,7 +50584,7 @@ SQLITE_API void sqlite3_result_error_nomem(sqlite3_context *pCtx){ ** statement is completely executed or an error occurs. ** ** This routine implements the bulk of the logic behind the sqlite_step() -** API. The only thing omitted is the automatic recompile if a +** API. The only thing omitted is the automatic recompile if a ** schema change has occurred. That detail is handled by the ** outer sqlite3_step() wrapper procedure. */ @@ -50632,14 +50669,14 @@ static int sqlite3Step(Vdbe *p){ p->rc = SQLITE_NOMEM; } end_of_step: - /* At this point local variable rc holds the value that should be - ** returned if this statement was compiled using the legacy + /* At this point local variable rc holds the value that should be + ** returned if this statement was compiled using the legacy ** sqlite3_prepare() interface. According to the docs, this can only - ** be one of the values in the first assert() below. Variable p->rc - ** contains the value that would be returned if sqlite3_finalize() + ** be one of the values in the first assert() below. Variable p->rc + ** contains the value that would be returned if sqlite3_finalize() ** were called on statement p. */ - assert( rc==SQLITE_ROW || rc==SQLITE_DONE || rc==SQLITE_ERROR + assert( rc==SQLITE_ROW || rc==SQLITE_DONE || rc==SQLITE_ERROR || rc==SQLITE_BUSY || rc==SQLITE_MISUSE ); assert( p->rc!=SQLITE_ROW && p->rc!=SQLITE_DONE ); @@ -50647,7 +50684,7 @@ end_of_step: /* If this statement was prepared using sqlite3_prepare_v2(), and an ** error has occured, then return the error code in p->rc to the ** caller. Set the error code in the database handle to the same value. - */ + */ rc = db->errCode = p->rc; } return (rc&db->errMask); @@ -50672,15 +50709,15 @@ SQLITE_API int sqlite3_step(sqlite3_stmt *pStmt){ v->expired = 0; } if( rc==SQLITE_SCHEMA && ALWAYS(v->isPrepareV2) && ALWAYS(db->pErr) ){ - /* This case occurs after failing to recompile an sql statement. - ** The error message from the SQL compiler has already been loaded - ** into the database handle. This block copies the error message + /* This case occurs after failing to recompile an sql statement. + ** The error message from the SQL compiler has already been loaded + ** into the database handle. This block copies the error message ** from the database handle into the statement and sets the statement - ** program counter to 0 to ensure that when the statement is + ** program counter to 0 to ensure that when the statement is ** finalized or reset the parser error message is available via ** sqlite3_errmsg() and sqlite3_errcode(). */ - const char *zErr = (const char *)sqlite3_value_text(db->pErr); + const char *zErr = (const char *)sqlite3_value_text(db->pErr); sqlite3DbFree(db, v->zErrMsg); if( !db->mallocFailed ){ v->zErrMsg = sqlite3DbStrDup(db, zErr); @@ -50783,9 +50820,9 @@ SQLITE_API void *sqlite3_get_auxdata(sqlite3_context *pCtx, int iArg){ ** deleted by calling the delete function specified when it was set. */ SQLITE_API void sqlite3_set_auxdata( - sqlite3_context *pCtx, - int iArg, - void *pAux, + sqlite3_context *pCtx, + int iArg, + void *pAux, void (*xDelete)(void*) ){ struct AuxData *pAuxData; @@ -50823,7 +50860,7 @@ failed: #ifndef SQLITE_OMIT_DEPRECATED /* -** Return the number of times the Step function of a aggregate has been +** Return the number of times the Step function of a aggregate has been ** called. ** ** This function is deprecated. Do not use it for new code. It is @@ -50884,9 +50921,9 @@ static Mem *columnMem(sqlite3_stmt *pStmt, int i){ ** this assert() from failing, when building with SQLITE_DEBUG defined ** using gcc, force nullMem to be 8-byte aligned using the magical ** __attribute__((aligned(8))) macro. */ - static const Mem nullMem + static const Mem nullMem #if defined(SQLITE_DEBUG) && defined(__GNUC__) - __attribute__((aligned(8))) + __attribute__((aligned(8))) #endif = {{0}, (double)0, 0, "", 0, MEM_Null, SQLITE_NULL, 0, 0, 0 }; @@ -50900,9 +50937,9 @@ static Mem *columnMem(sqlite3_stmt *pStmt, int i){ } /* -** This function is called after invoking an sqlite3_value_XXX function on a +** This function is called after invoking an sqlite3_value_XXX function on a ** column value (i.e. a value returned by evaluating an SQL expression in the -** select list of a SELECT statement) that may cause a malloc() failure. If +** select list of a SELECT statement) that may cause a malloc() failure. If ** malloc() has failed, the threads mallocFailed flag is cleared and the result ** code of statement pStmt set to SQLITE_NOMEM. ** @@ -50940,8 +50977,8 @@ SQLITE_API const void *sqlite3_column_blob(sqlite3_stmt *pStmt, int i){ const void *val; val = sqlite3_value_blob( columnMem(pStmt,i) ); /* Even though there is no encoding conversion, value_blob() might - ** need to call malloc() to expand the result of a zeroblob() - ** expression. + ** need to call malloc() to expand the result of a zeroblob() + ** expression. */ columnMallocFailure(pStmt); return val; @@ -51031,7 +51068,7 @@ static const void *columnName( Vdbe *p = (Vdbe *)pStmt; int n; sqlite3 *db = p->db; - + assert( db!=0 ); n = sqlite3_column_count(pStmt); if( N=0 ){ @@ -51144,11 +51181,11 @@ SQLITE_API const void *sqlite3_column_origin_name16(sqlite3_stmt *pStmt, int N){ /******************************* sqlite3_bind_ *************************** -** +** ** Routines used to attach values to wildcards in a compiled SQL statement. */ /* -** Unbind the value bound to variable i in virtual machine p. This is the +** Unbind the value bound to variable i in virtual machine p. This is the ** the same as binding a NULL value to the column. If the "i" parameter is ** out of range, then SQLITE_RANGE is returned. Othewise SQLITE_OK. ** @@ -51178,7 +51215,7 @@ static int vdbeUnbind(Vdbe *p, int i){ pVar->flags = MEM_Null; sqlite3Error(p->db, SQLITE_OK, 0); - /* If the bit corresponding to this variable in Vdbe.expmask is set, then + /* If the bit corresponding to this variable in Vdbe.expmask is set, then ** binding a new value to this variable invalidates the current query plan. */ if( p->isPrepareV2 && @@ -51225,10 +51262,10 @@ static int bindText( ** Bind a blob value to an SQL statement variable. */ SQLITE_API int sqlite3_bind_blob( - sqlite3_stmt *pStmt, - int i, - const void *zData, - int nData, + sqlite3_stmt *pStmt, + int i, + const void *zData, + int nData, void (*xDel)(void*) ){ return bindText(pStmt, i, zData, nData, xDel, 0); @@ -51265,21 +51302,21 @@ SQLITE_API int sqlite3_bind_null(sqlite3_stmt *pStmt, int i){ } return rc; } -SQLITE_API int sqlite3_bind_text( - sqlite3_stmt *pStmt, - int i, - const char *zData, - int nData, +SQLITE_API int sqlite3_bind_text( + sqlite3_stmt *pStmt, + int i, + const char *zData, + int nData, void (*xDel)(void*) ){ return bindText(pStmt, i, zData, nData, xDel, SQLITE_UTF8); } #ifndef SQLITE_OMIT_UTF16 SQLITE_API int sqlite3_bind_text16( - sqlite3_stmt *pStmt, - int i, - const void *zData, - int nData, + sqlite3_stmt *pStmt, + int i, + const void *zData, + int nData, void (*xDel)(void*) ){ return bindText(pStmt, i, zData, nData, xDel, SQLITE_UTF16NATIVE); @@ -51329,7 +51366,7 @@ SQLITE_API int sqlite3_bind_zeroblob(sqlite3_stmt *pStmt, int i, int n){ /* ** Return the number of wildcards that can be potentially bound to. -** This routine is added to support DBD::SQLite. +** This routine is added to support DBD::SQLite. */ SQLITE_API int sqlite3_bind_parameter_count(sqlite3_stmt *pStmt){ Vdbe *p = (Vdbe*)pStmt; @@ -51382,23 +51419,25 @@ SQLITE_API const char *sqlite3_bind_parameter_name(sqlite3_stmt *pStmt, int i){ ** with that name. If there is no variable with the given name, ** return 0. */ -SQLITE_API int sqlite3_bind_parameter_index(sqlite3_stmt *pStmt, const char *zName){ - Vdbe *p = (Vdbe*)pStmt; +SQLITE_PRIVATE int sqlite3VdbeParameterIndex(Vdbe *p, const char *zName, int nName){ int i; if( p==0 ){ return 0; } - createVarMap(p); + createVarMap(p); if( zName ){ for(i=0; inVar; i++){ const char *z = p->azVar[i]; - if( z && strcmp(z,zName)==0 ){ + if( z && memcmp(z,zName,nName)==0 && z[nName]==0 ){ return i+1; } } } return 0; } +SQLITE_API int sqlite3_bind_parameter_index(sqlite3_stmt *pStmt, const char *zName){ + return sqlite3VdbeParameterIndex((Vdbe*)pStmt, zName, sqlite3Strlen30(zName)); +} /* ** Transfer all bindings from the first statement over to the second. @@ -51485,6 +51524,149 @@ SQLITE_API int sqlite3_stmt_status(sqlite3_stmt *pStmt, int op, int resetFlag){ } /************** End of vdbeapi.c *********************************************/ +/************** Begin file vdbetrace.c ***************************************/ +/* +** 2009 November 25 +** +** The author disclaims copyright to this source code. In place of +** a legal notice, here is a blessing: +** +** May you do good and not evil. +** May you find forgiveness for yourself and forgive others. +** May you share freely, never taking more than you give. +** +************************************************************************* +** +** This file contains code used to insert the values of host parameters +** (aka "wildcards") into the SQL text output by sqlite3_trace(). +*/ + +#ifndef SQLITE_OMIT_TRACE + +/* +** zSql is a zero-terminated string of UTF-8 SQL text. Return the number of +** bytes in this text up to but excluding the first character in +** a host parameter. If the text contains no host parameters, return +** the total number of bytes in the text. +*/ +static int findNextHostParameter(const char *zSql, int *pnToken){ + int tokenType; + int nTotal = 0; + int n; + + *pnToken = 0; + while( zSql[0] ){ + n = sqlite3GetToken((u8*)zSql, &tokenType); + assert( n>0 && tokenType!=TK_ILLEGAL ); + if( tokenType==TK_VARIABLE ){ + *pnToken = n; + break; + } + nTotal += n; + zSql += n; + } + return nTotal; +} + +/* +** Return a pointer to a string in memory obtained form sqlite3DbMalloc() which +** holds a copy of zRawSql but with host parameters expanded to their +** current bindings. +** +** The calling function is responsible for making sure the memory returned +** is eventually freed. +** +** ALGORITHM: Scan the input string looking for host parameters in any of +** these forms: ?, ?N, $A, @A, :A. Take care to avoid text within +** string literals, quoted identifier names, and comments. For text forms, +** the host parameter index is found by scanning the perpared +** statement for the corresponding OP_Variable opcode. Once the host +** parameter index is known, locate the value in p->aVar[]. Then render +** the value as a literal in place of the host parameter name. +*/ +SQLITE_PRIVATE char *sqlite3VdbeExpandSql( + Vdbe *p, /* The prepared statement being evaluated */ + const char *zRawSql /* Raw text of the SQL statement */ +){ + sqlite3 *db; /* The database connection */ + int idx = 0; /* Index of a host parameter */ + int nextIndex = 1; /* Index of next ? host parameter */ + int n; /* Length of a token prefix */ + int nToken; /* Length of the parameter token */ + int i; /* Loop counter */ + Mem *pVar; /* Value of a host parameter */ + StrAccum out; /* Accumulate the output here */ + char zBase[100]; /* Initial working space */ + + db = p->db; + sqlite3StrAccumInit(&out, zBase, sizeof(zBase), + db->aLimit[SQLITE_LIMIT_LENGTH]); + out.db = db; + while( zRawSql[0] ){ + n = findNextHostParameter(zRawSql, &nToken); + assert( n>0 ); + sqlite3StrAccumAppend(&out, zRawSql, n); + zRawSql += n; + assert( zRawSql[0] || nToken==0 ); + if( nToken==0 ) break; + if( zRawSql[0]=='?' ){ + if( nToken>1 ){ + assert( sqlite3Isdigit(zRawSql[1]) ); + sqlite3GetInt32(&zRawSql[1], &idx); + }else{ + idx = nextIndex; + } + }else{ + assert( zRawSql[0]==':' || zRawSql[0]=='$' || zRawSql[0]=='@' ); + testcase( zRawSql[0]==':' ); + testcase( zRawSql[0]=='$' ); + testcase( zRawSql[0]=='@' ); + idx = sqlite3VdbeParameterIndex(p, zRawSql, nToken); + assert( idx>0 ); + } + zRawSql += nToken; + nextIndex = idx + 1; + assert( idx>0 && idx<=p->nVar ); + pVar = &p->aVar[idx-1]; + if( pVar->flags & MEM_Null ){ + sqlite3StrAccumAppend(&out, "NULL", 4); + }else if( pVar->flags & MEM_Int ){ + sqlite3XPrintf(&out, "%lld", pVar->u.i); + }else if( pVar->flags & MEM_Real ){ + sqlite3XPrintf(&out, "%!.15g", pVar->r); + }else if( pVar->flags & MEM_Str ){ +#ifndef SQLITE_OMIT_UTF16 + u8 enc = ENC(db); + if( enc!=SQLITE_UTF8 ){ + Mem utf8; + memset(&utf8, 0, sizeof(utf8)); + utf8.db = db; + sqlite3VdbeMemSetStr(&utf8, pVar->z, pVar->n, enc, SQLITE_STATIC); + sqlite3VdbeChangeEncoding(&utf8, SQLITE_UTF8); + sqlite3XPrintf(&out, "'%.*q'", utf8.n, utf8.z); + sqlite3VdbeMemRelease(&utf8); + }else +#endif + { + sqlite3XPrintf(&out, "'%.*q'", pVar->n, pVar->z); + } + }else if( pVar->flags & MEM_Zero ){ + sqlite3XPrintf(&out, "zeroblob(%d)", pVar->u.nZero); + }else{ + assert( pVar->flags & MEM_Blob ); + sqlite3StrAccumAppend(&out, "x'", 2); + for(i=0; in; i++){ + sqlite3XPrintf(&out, "%02x", pVar->z[i]&0xff); + } + sqlite3StrAccumAppend(&out, "'", 1); + } + } + return sqlite3StrAccumFinish(&out); +} + +#endif /* #ifndef SQLITE_OMIT_TRACE */ + +/************** End of vdbetrace.c *******************************************/ /************** Begin file vdbe.c ********************************************/ /* ** 2001 September 15 @@ -51497,7 +51679,7 @@ SQLITE_API int sqlite3_stmt_status(sqlite3_stmt *pStmt, int op, int resetFlag){ ** May you share freely, never taking more than you give. ** ************************************************************************* -** The code in this file implements execution method of the +** The code in this file implements execution method of the ** Virtual Database Engine (VDBE). A separate file ("vdbeaux.c") ** handles housekeeping details such as creating and deleting ** VDBE instances. This file is solely interested in executing @@ -51507,10 +51689,10 @@ SQLITE_API int sqlite3_stmt_status(sqlite3_stmt *pStmt, int op, int resetFlag){ ** to a VDBE. ** ** The SQL parser generates a program which is then executed by -** the VDBE to do the work of the SQL statement. VDBE programs are +** the VDBE to do the work of the SQL statement. VDBE programs are ** similar in form to assembly language. The program consists of -** a linear sequence of operations. Each operation has an opcode -** and 5 operands. Operands P1, P2, and P3 are integers. Operand P4 +** a linear sequence of operations. Each operation has an opcode +** and 5 operands. Operands P1, P2, and P3 are integers. Operand P4 ** is a null-terminated string. Operand P5 is an unsigned character. ** Few opcodes use all 5 operands. ** @@ -51519,7 +51701,7 @@ SQLITE_API int sqlite3_stmt_status(sqlite3_stmt *pStmt, int op, int resetFlag){ ** either an integer, a null-terminated string, a floating point ** number, or the SQL "NULL" value. An implicit conversion from one ** type to the other occurs as necessary. -** +** ** Most of the code in this file is taken up by the sqlite3VdbeExec() ** function which does the work of interpreting a VDBE program. ** But other routines are also provided to help in building up @@ -51530,8 +51712,6 @@ SQLITE_API int sqlite3_stmt_status(sqlite3_stmt *pStmt, int op, int resetFlag){ ** of the code in this file is, therefore, important. See other comments ** in this file for details. If in doubt, do not deviate from existing ** commenting and indentation practices when changing or adding code. -** -** $Id: vdbe.c,v 1.874 2009/07/24 17:58:53 danielk1977 Exp $ */ /* @@ -51637,7 +51817,7 @@ SQLITE_API int sqlite3_found_count = 0; /* ** Argument pMem points at a register that will be passed to a ** user-defined function or returned to the user as the result of a query. -** This routine sets the pMem->type variable used by the sqlite3_value_*() +** This routine sets the pMem->type variable used by the sqlite3_value_*() ** routines. */ SQLITE_PRIVATE void sqlite3VdbeMemStoreType(Mem *pMem){ @@ -51658,23 +51838,6 @@ SQLITE_PRIVATE void sqlite3VdbeMemStoreType(Mem *pMem){ } } -/* -** Properties of opcodes. The OPFLG_INITIALIZER macro is -** created by mkopcodeh.awk during compilation. Data is obtained -** from the comments following the "case OP_xxxx:" statements in -** this file. -*/ -static const unsigned char opcodeProperty[] = OPFLG_INITIALIZER; - -/* -** Return true if an opcode has any of the OPFLG_xxx properties -** specified by mask. -*/ -SQLITE_PRIVATE int sqlite3VdbeOpcodeHasProperty(int opcode, int mask){ - assert( opcode>0 && opcode<(int)sizeof(opcodeProperty) ); - return (opcodeProperty[opcode]&mask)!=0; -} - /* ** Allocate VdbeCursor number iCur. Return a pointer to it. Return NULL ** if we run out of memory. @@ -51687,7 +51850,7 @@ static VdbeCursor *allocateCursor( int isBtreeCursor /* True for B-Tree. False for pseudo-table or vtab */ ){ /* Find the memory cell that will be used to store the blob of memory - ** required for this VdbeCursor structure. It is convenient to use a + ** required for this VdbeCursor structure. It is convenient to use a ** vdbe memory cell to manage the memory allocation required for a ** VdbeCursor structure for the following reasons: ** @@ -51708,9 +51871,9 @@ static VdbeCursor *allocateCursor( int nByte; VdbeCursor *pCx = 0; - nByte = - sizeof(VdbeCursor) + - (isBtreeCursor?sqlite3BtreeCursorSize():0) + + nByte = + ROUND8(sizeof(VdbeCursor)) + + (isBtreeCursor?sqlite3BtreeCursorSize():0) + 2*nField*sizeof(u32); assert( iCurnCursor ); @@ -51720,15 +51883,16 @@ static VdbeCursor *allocateCursor( } if( SQLITE_OK==sqlite3VdbeMemGrow(pMem, nByte, 0) ){ p->apCsr[iCur] = pCx = (VdbeCursor*)pMem->z; - memset(pMem->z, 0, nByte); + memset(pCx, 0, sizeof(VdbeCursor)); pCx->iDb = iDb; pCx->nField = nField; if( nField ){ - pCx->aType = (u32 *)&pMem->z[sizeof(VdbeCursor)]; + pCx->aType = (u32 *)&pMem->z[ROUND8(sizeof(VdbeCursor))]; } if( isBtreeCursor ){ pCx->pCursor = (BtCursor*) - &pMem->z[sizeof(VdbeCursor)+2*nField*sizeof(u32)]; + &pMem->z[ROUND8(sizeof(VdbeCursor))+2*nField*sizeof(u32)]; + sqlite3BtreeCursorZero(pCx->pCursor); } } return pCx; @@ -51764,7 +51928,7 @@ static void applyNumericAffinity(Mem *pRec){ ** SQLITE_AFF_INTEGER: ** SQLITE_AFF_REAL: ** SQLITE_AFF_NUMERIC: -** Try to convert pRec to an integer representation or a +** Try to convert pRec to an integer representation or a ** floating-point representation if an integer representation ** is not possible. Note that the integer representation is ** always preferred, even if the affinity is REAL, because @@ -51816,12 +51980,12 @@ SQLITE_API int sqlite3_value_numeric_type(sqlite3_value *pVal){ } /* -** Exported version of applyAffinity(). This one works on sqlite3_value*, +** Exported version of applyAffinity(). This one works on sqlite3_value*, ** not the internal Mem* type. */ SQLITE_PRIVATE void sqlite3ValueApplyAffinity( - sqlite3_value *pVal, - u8 affinity, + sqlite3_value *pVal, + u8 affinity, u8 enc ){ applyAffinity((Mem *)pVal, affinity, enc); @@ -51950,8 +52114,8 @@ static void registerTrace(FILE *out, int iReg, Mem *p){ #ifdef VDBE_PROFILE -/* -** hwtime.h contains inline assembler code for implementing +/* +** hwtime.h contains inline assembler code for implementing ** high-performance timing routines. */ /************** Include hwtime.h in the middle of vdbe.c *********************/ @@ -51970,8 +52134,6 @@ static void registerTrace(FILE *out, int iReg, Mem *p){ ** ** This file contains inline asm code for retrieving "high-performance" ** counters for x86 class CPUs. -** -** $Id: hwtime.h,v 1.3 2008/08/01 14:33:15 shane Exp $ */ #ifndef _HWTIME_H_ #define _HWTIME_H_ @@ -52011,7 +52173,7 @@ static void registerTrace(FILE *out, int iReg, Mem *p){ __asm__ __volatile__ ("rdtsc" : "=A" (val)); return val; } - + #elif (defined(__GNUC__) && defined(__ppc__)) __inline__ sqlite_uint64 sqlite3Hwtime(void){ @@ -52083,9 +52245,9 @@ static int fileExists(sqlite3 *db, const char *zFile){ /* ** This function is only called from within an assert() expression. It ** checks that the sqlite3.nTransaction variable is correctly set to -** the number of non-transaction savepoints currently in the +** the number of non-transaction savepoints currently in the ** linked list starting at sqlite3.pSavepoint. -** +** ** Usage: ** ** assert( checkSavepointCount(db) ); @@ -52134,23 +52296,26 @@ SQLITE_PRIVATE int sqlite3VdbeExec( Vdbe *p /* The VDBE */ ){ int pc; /* The program counter */ + Op *aOp = p->aOp; /* Copy of p->aOp */ Op *pOp; /* Current operation */ int rc = SQLITE_OK; /* Value to return */ sqlite3 *db = p->db; /* The database */ + u8 resetSchemaOnFault = 0; /* Reset schema after an error if true */ u8 encoding = ENC(db); /* The database encoding */ +#ifndef SQLITE_OMIT_PROGRESS_CALLBACK + int checkProgress; /* True if progress callbacks are enabled */ + int nProgressOps = 0; /* Opcodes executed since progress callback. */ +#endif + Mem *aMem = p->aMem; /* Copy of p->aMem */ Mem *pIn1 = 0; /* 1st input operand */ Mem *pIn2 = 0; /* 2nd input operand */ Mem *pIn3 = 0; /* 3rd input operand */ Mem *pOut = 0; /* Output operand */ - u8 opProperty; int iCompare = 0; /* Result of last OP_Compare operation */ int *aPermute = 0; /* Permutation of columns for OP_Compare */ #ifdef VDBE_PROFILE u64 start; /* CPU clock count at start of opcode */ int origPc; /* Program counter at start of opcode */ -#endif -#ifndef SQLITE_OMIT_PROGRESS_CALLBACK - int nProgressOps = 0; /* Opcodes executed since progress callback. */ #endif /******************************************************************** ** Automatically generated code @@ -52247,10 +52412,8 @@ SQLITE_PRIVATE int sqlite3VdbeExec( Mem *pReg; /* PseudoTable input register */ } am; struct OP_Affinity_stack_vars { - char *zAffinity; /* The affinity to be applied */ - Mem *pData0; /* First register to which to apply affinity */ - Mem *pLast; /* Last register to which to apply affinity */ - Mem *pRec; /* Current register */ + const char *zAffinity; /* The affinity to be applied */ + char cAff; /* A single character of affinity */ } an; struct OP_MakeRecord_stack_vars { u8 *zNewRecord; /* A buffer to hold the data for the new record */ @@ -52335,6 +52498,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec( VdbeCursor *pC; int res; UnpackedRecord *pIdxKey; + UnpackedRecord r; char aTempRec[ROUND8(sizeof(UnpackedRecord)) + sizeof(Mem)*3 + 7]; } bb; struct OP_IsUnique_stack_vars { @@ -52342,7 +52506,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec( VdbeCursor *pCx; BtCursor *pCrsr; u16 nField; - Mem *aMem; + Mem *aMx; UnpackedRecord r; /* B-Tree index search key */ i64 R; /* Rowid stored in register P3 */ } bc; @@ -52455,18 +52619,13 @@ SQLITE_PRIVATE int sqlite3VdbeExec( char *z; /* Text of the error report */ Mem *pnErr; /* Register keeping track of errors remaining */ } bv; - struct OP_RowSetAdd_stack_vars { - Mem *pIdx; - Mem *pVal; - } bw; struct OP_RowSetRead_stack_vars { - Mem *pIdx; i64 val; - } bx; + } bw; struct OP_RowSetTest_stack_vars { int iSet; int exists; - } by; + } bx; struct OP_Program_stack_vars { int nMem; /* Number of memory registers for sub-program */ int nByte; /* Bytes of runtime space required for sub-program */ @@ -52476,15 +52635,15 @@ SQLITE_PRIVATE int sqlite3VdbeExec( VdbeFrame *pFrame; /* New vdbe frame to execute in */ SubProgram *pProgram; /* Sub-program to execute */ void *t; /* Token identifying trigger */ - } bz; + } by; struct OP_Param_stack_vars { VdbeFrame *pFrame; Mem *pIn; - } ca; + } bz; struct OP_MemMax_stack_vars { Mem *pIn1; VdbeFrame *pFrame; - } cb; + } ca; struct OP_AggStep_stack_vars { int n; int i; @@ -52492,22 +52651,22 @@ SQLITE_PRIVATE int sqlite3VdbeExec( Mem *pRec; sqlite3_context ctx; sqlite3_value **apVal; - } cc; + } cb; struct OP_AggFinal_stack_vars { Mem *pMem; - } cd; + } cc; struct OP_IncrVacuum_stack_vars { Btree *pBt; - } ce; + } cd; struct OP_VBegin_stack_vars { VTable *pVTab; - } cf; + } ce; struct OP_VOpen_stack_vars { VdbeCursor *pCur; sqlite3_vtab_cursor *pVtabCursor; sqlite3_vtab *pVtab; sqlite3_module *pModule; - } cg; + } cf; struct OP_VFilter_stack_vars { int nArg; int iQuery; @@ -52520,23 +52679,23 @@ SQLITE_PRIVATE int sqlite3VdbeExec( int res; int i; Mem **apArg; - } ch; + } cg; struct OP_VColumn_stack_vars { sqlite3_vtab *pVtab; const sqlite3_module *pModule; Mem *pDest; sqlite3_context sContext; - } ci; + } ch; struct OP_VNext_stack_vars { sqlite3_vtab *pVtab; const sqlite3_module *pModule; int res; VdbeCursor *pCur; - } cj; + } ci; struct OP_VRename_stack_vars { sqlite3_vtab *pVtab; Mem *pName; - } ck; + } cj; struct OP_VUpdate_stack_vars { sqlite3_vtab *pVtab; sqlite3_module *pModule; @@ -52545,15 +52704,15 @@ SQLITE_PRIVATE int sqlite3VdbeExec( sqlite_int64 rowid; Mem **apArg; Mem *pX; - } cl; + } ck; struct OP_Pagecount_stack_vars { int p1; int nPage; Pager *pPager; - } cm; + } cl; struct OP_Trace_stack_vars { char *zTrace; - } cn; + } cm; } u; /* End automatically generated code ********************************************************************/ @@ -52573,16 +52732,19 @@ SQLITE_PRIVATE int sqlite3VdbeExec( db->busyHandler.nBusy = 0; CHECK_FOR_INTERRUPT; sqlite3VdbeIOTraceSql(p); +#ifndef SQLITE_OMIT_PROGRESS_CALLBACK + checkProgress = db->xProgress!=0; +#endif #ifdef SQLITE_DEBUG sqlite3BeginBenignMalloc(); - if( p->pc==0 + if( p->pc==0 && ((p->db->flags & SQLITE_VdbeListing) || fileExists(db, "vdbe_explain")) ){ int i; printf("VDBE Program Listing:\n"); sqlite3VdbePrintSql(p); for(i=0; inOp; i++){ - sqlite3VdbePrintOp(stdout, i, &p->aOp[i]); + sqlite3VdbePrintOp(stdout, i, &aOp[i]); } } if( fileExists(db, "vdbe_trace") ){ @@ -52597,7 +52759,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec( origPc = pc; start = sqlite3Hwtime(); #endif - pOp = &p->aOp[pc]; + pOp = &aOp[pc]; /* Only allow tracing if SQLITE_DEBUG is defined. */ @@ -52617,7 +52779,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec( sqlite3EndBenignMalloc(); } #endif - + /* Check to see if we need to simulate an interrupt. This only happens ** if we have a special test build. @@ -52638,7 +52800,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec( ** If the progress callback returns non-zero, exit the virtual machine with ** a return code SQLITE_ABORT. */ - if( db->xProgress ){ + if( checkProgress ){ if( db->nProgressOps==nProgressOps ){ int prc; if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; @@ -52654,65 +52816,46 @@ SQLITE_PRIVATE int sqlite3VdbeExec( } #endif - /* Do common setup processing for any opcode that is marked - ** with the "out2-prerelease" tag. Such opcodes have a single - ** output which is specified by the P2 parameter. The P2 register - ** is initialized to a NULL. + /* On any opcode with the "out2-prerelase" tag, free any + ** external allocations out of mem[p2] and set mem[p2] to be + ** an undefined integer. Opcodes will either fill in the integer + ** value or convert mem[p2] to a different type. */ - opProperty = opcodeProperty[pOp->opcode]; - if( (opProperty & OPFLG_OUT2_PRERELEASE)!=0 ){ + assert( pOp->opflags==sqlite3OpcodeProperty[pOp->opcode] ); + if( pOp->opflags & OPFLG_OUT2_PRERELEASE ){ assert( pOp->p2>0 ); assert( pOp->p2<=p->nMem ); - pOut = &p->aMem[pOp->p2]; + pOut = &aMem[pOp->p2]; sqlite3VdbeMemReleaseExternal(pOut); - pOut->flags = MEM_Null; - pOut->n = 0; - }else - - /* Do common setup for opcodes marked with one of the following - ** combinations of properties. - ** - ** in1 - ** in1 in2 - ** in1 in2 out3 - ** in1 in3 - ** - ** Variables pIn1, pIn2, and pIn3 are made to point to appropriate - ** registers for inputs. Variable pOut points to the output register. - */ - if( (opProperty & OPFLG_IN1)!=0 ){ + pOut->flags = MEM_Int; + } + + /* Sanity checking on other operands */ +#ifdef SQLITE_DEBUG + if( (pOp->opflags & OPFLG_IN1)!=0 ){ assert( pOp->p1>0 ); assert( pOp->p1<=p->nMem ); - pIn1 = &p->aMem[pOp->p1]; - REGISTER_TRACE(pOp->p1, pIn1); - if( (opProperty & OPFLG_IN2)!=0 ){ - assert( pOp->p2>0 ); - assert( pOp->p2<=p->nMem ); - pIn2 = &p->aMem[pOp->p2]; - REGISTER_TRACE(pOp->p2, pIn2); - /* As currently implemented, in2 implies out3. There is no reason - ** why this has to be, it just worked out that way. */ - assert( (opProperty & OPFLG_OUT3)!=0 ); - assert( pOp->p3>0 ); - assert( pOp->p3<=p->nMem ); - pOut = &p->aMem[pOp->p3]; - }else if( (opProperty & OPFLG_IN3)!=0 ){ - assert( pOp->p3>0 ); - assert( pOp->p3<=p->nMem ); - pIn3 = &p->aMem[pOp->p3]; - REGISTER_TRACE(pOp->p3, pIn3); - } - }else if( (opProperty & OPFLG_IN2)!=0 ){ + REGISTER_TRACE(pOp->p1, &aMem[pOp->p1]); + } + if( (pOp->opflags & OPFLG_IN2)!=0 ){ assert( pOp->p2>0 ); assert( pOp->p2<=p->nMem ); - pIn2 = &p->aMem[pOp->p2]; - REGISTER_TRACE(pOp->p2, pIn2); - }else if( (opProperty & OPFLG_IN3)!=0 ){ + REGISTER_TRACE(pOp->p2, &aMem[pOp->p2]); + } + if( (pOp->opflags & OPFLG_IN3)!=0 ){ assert( pOp->p3>0 ); assert( pOp->p3<=p->nMem ); - pIn3 = &p->aMem[pOp->p3]; - REGISTER_TRACE(pOp->p3, pIn3); + REGISTER_TRACE(pOp->p3, &aMem[pOp->p3]); } + if( (pOp->opflags & OPFLG_OUT2)!=0 ){ + assert( pOp->p2>0 ); + assert( pOp->p2<=p->nMem ); + } + if( (pOp->opflags & OPFLG_OUT3)!=0 ){ + assert( pOp->p3>0 ); + assert( pOp->p3<=p->nMem ); + } +#endif switch( pOp->opcode ){ @@ -52754,7 +52897,7 @@ SQLITE_PRIVATE int sqlite3VdbeExec( /* Opcode: Goto * P2 * * * ** ** An unconditional jump to address P2. -** The next instruction executed will be +** The next instruction executed will be ** the one at index P2 from the beginning of ** the program. */ @@ -52769,10 +52912,8 @@ case OP_Goto: { /* jump */ ** Write the current address onto register P1 ** and then jump to address P2. */ -case OP_Gosub: { /* jump */ - assert( pOp->p1>0 ); - assert( pOp->p1<=p->nMem ); - pIn1 = &p->aMem[pOp->p1]; +case OP_Gosub: { /* jump, in1 */ + pIn1 = &aMem[pOp->p1]; assert( (pIn1->flags & MEM_Dyn)==0 ); pIn1->flags = MEM_Int; pIn1->u.i = pc; @@ -52786,6 +52927,7 @@ case OP_Gosub: { /* jump */ ** Jump to the next instruction after the address in register P1. */ case OP_Return: { /* in1 */ + pIn1 = &aMem[pOp->p1]; assert( pIn1->flags & MEM_Int ); pc = (int)pIn1->u.i; break; @@ -52799,6 +52941,7 @@ case OP_Yield: { /* in1 */ #if 0 /* local variables moved into u.aa */ int pcDest; #endif /* local variables moved into u.aa */ + pIn1 = &aMem[pOp->p1]; assert( (pIn1->flags & MEM_Dyn)==0 ); pIn1->flags = MEM_Int; u.aa.pcDest = (int)pIn1->u.i; @@ -52815,6 +52958,7 @@ case OP_Yield: { /* in1 */ ** value in register P3 is not NULL, then this routine is a no-op. */ case OP_HaltIfNull: { /* in3 */ + pIn3 = &aMem[pOp->p3]; if( (pIn3->flags & MEM_Null)==0 ) break; /* Fall through into OP_Halt */ } @@ -52830,7 +52974,7 @@ case OP_HaltIfNull: { /* in3 */ ** whether or not to rollback the current transaction. Do not rollback ** if P2==OE_Fail. Do the rollback if P2==OE_Rollback. If P2==OE_Abort, ** then back out all changes that have occurred during this execution of the -** VDBE, but do not rollback the transaction. +** VDBE, but do not rollback the transaction. ** ** If P4 is not null then it is an error message string. ** @@ -52847,13 +52991,15 @@ case OP_Halt: { sqlite3VdbeSetChanges(db, p->nChange); pc = sqlite3VdbeFrameRestore(pFrame); if( pOp->p2==OE_Ignore ){ - /* Instruction pc is the OP_Program that invoked the sub-program + /* Instruction pc is the OP_Program that invoked the sub-program ** currently being halted. If the p2 instruction of this OP_Halt ** instruction is set to OE_Ignore, then the sub-program is throwing ** an IGNORE exception. In this case jump to the address specified ** as the p2 of the calling OP_Program. */ pc = p->aOp[pc].p2-1; } + aOp = p->aOp; + aMem = p->aMem; break; } @@ -52880,7 +53026,6 @@ case OP_Halt: { ** The 32-bit integer value P1 is written into register P2. */ case OP_Integer: { /* out2-prerelease */ - pOut->flags = MEM_Int; pOut->u.i = pOp->p1; break; } @@ -52892,7 +53037,6 @@ case OP_Integer: { /* out2-prerelease */ */ case OP_Int64: { /* out2-prerelease */ assert( pOp->p4.pI64!=0 ); - pOut->flags = MEM_Int; pOut->u.i = *pOp->p4.pI64; break; } @@ -52911,7 +53055,7 @@ case OP_Real: { /* same as TK_FLOAT, out2-prerelease */ /* Opcode: String8 * P2 * P4 * ** -** P4 points to a nul terminated UTF-8 string. This opcode is transformed +** P4 points to a nul terminated UTF-8 string. This opcode is transformed ** into an OP_String before it is executed for the first time. */ case OP_String8: { /* same as TK_STRING, out2-prerelease */ @@ -52942,7 +53086,7 @@ case OP_String8: { /* same as TK_STRING, out2-prerelease */ } /* Fall through to the next case, OP_String */ } - + /* Opcode: String P1 P2 * P4 * ** ** The string value P4 of length P1 (bytes) is stored in register P2. @@ -52962,6 +53106,7 @@ case OP_String: { /* out2-prerelease */ ** Write a NULL into register P2. */ case OP_Null: { /* out2-prerelease */ + pOut->flags = MEM_Null; break; } @@ -53011,7 +53156,7 @@ case OP_Variable: { if( sqlite3VdbeMemTooBig(u.ab.pVar) ){ goto too_big; } - pOut = &p->aMem[u.ab.p2++]; + pOut = &aMem[u.ab.p2++]; sqlite3VdbeMemReleaseExternal(pOut); pOut->flags = MEM_Null; sqlite3VdbeMemShallowCopy(pOut, u.ab.pVar, MEM_Static); @@ -53041,11 +53186,11 @@ case OP_Move: { assert( u.ac.n>0 && u.ac.p1>0 && u.ac.p2>0 ); assert( u.ac.p1+u.ac.n<=u.ac.p2 || u.ac.p2+u.ac.n<=u.ac.p1 ); - pIn1 = &p->aMem[u.ac.p1]; - pOut = &p->aMem[u.ac.p2]; + pIn1 = &aMem[u.ac.p1]; + pOut = &aMem[u.ac.p2]; while( u.ac.n-- ){ - assert( pOut<=&p->aMem[p->nMem] ); - assert( pIn1<=&p->aMem[p->nMem] ); + assert( pOut<=&aMem[p->nMem] ); + assert( pIn1<=&aMem[p->nMem] ); u.ac.zMalloc = pOut->zMalloc; pOut->zMalloc = 0; sqlite3VdbeMemMove(pOut, pIn1); @@ -53064,10 +53209,9 @@ case OP_Move: { ** This instruction makes a deep copy of the value. A duplicate ** is made of any string or blob constant. See also OP_SCopy. */ -case OP_Copy: { /* in1 */ - assert( pOp->p2>0 ); - assert( pOp->p2<=p->nMem ); - pOut = &p->aMem[pOp->p2]; +case OP_Copy: { /* in1, out2 */ + pIn1 = &aMem[pOp->p1]; + pOut = &aMem[pOp->p2]; assert( pOut!=pIn1 ); sqlite3VdbeMemShallowCopy(pOut, pIn1, MEM_Ephem); Deephemeralize(pOut); @@ -53087,11 +53231,9 @@ case OP_Copy: { /* in1 */ ** during the lifetime of the copy. Use OP_Copy to make a complete ** copy. */ -case OP_SCopy: { /* in1 */ - REGISTER_TRACE(pOp->p1, pIn1); - assert( pOp->p2>0 ); - assert( pOp->p2<=p->nMem ); - pOut = &p->aMem[pOp->p2]; +case OP_SCopy: { /* in1, out2 */ + pIn1 = &aMem[pOp->p1]; + pOut = &aMem[pOp->p2]; assert( pOut!=pIn1 ); sqlite3VdbeMemShallowCopy(pOut, pIn1, MEM_Ephem); REGISTER_TRACE(pOp->p2, pOut); @@ -53152,7 +53294,7 @@ case OP_ResultRow: { ** and have an assigned type. The results are de-ephemeralized as ** as side effect. */ - u.ad.pMem = p->pResultSet = &p->aMem[pOp->p1]; + u.ad.pMem = p->pResultSet = &aMem[pOp->p1]; for(u.ad.i=0; u.ad.ip2; u.ad.i++){ sqlite3VdbeMemNulTerminate(&u.ad.pMem[u.ad.i]); sqlite3VdbeMemStoreType(&u.ad.pMem[u.ad.i]); @@ -53184,6 +53326,9 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */ i64 nByte; #endif /* local variables moved into u.ae */ + pIn1 = &aMem[pOp->p1]; + pIn2 = &aMem[pOp->p2]; + pOut = &aMem[pOp->p3]; assert( pIn1!=pOut ); if( (pIn1->flags | pIn2->flags) & MEM_Null ){ sqlite3VdbeMemSetNull(pOut); @@ -53235,14 +53380,14 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */ /* Opcode: Divide P1 P2 P3 * * ** ** Divide the value in register P1 by the value in register P2 -** and store the result in register P3 (P3=P2/P1). If the value in -** register P1 is zero, then the result is NULL. If either input is +** and store the result in register P3 (P3=P2/P1). If the value in +** register P1 is zero, then the result is NULL. If either input is ** NULL, the result is NULL. */ /* Opcode: Remainder P1 P2 P3 * * ** ** Compute the remainder after integer division of the value in -** register P1 by the value in register P2 and store the result in P3. +** register P1 by the value in register P2 and store the result in P3. ** If the value in register P2 is zero the result is NULL. ** If either operand is NULL, the result is NULL. */ @@ -53259,8 +53404,11 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */ double rB; /* Real value of right operand */ #endif /* local variables moved into u.af */ + pIn1 = &aMem[pOp->p1]; applyNumericAffinity(pIn1); + pIn2 = &aMem[pOp->p2]; applyNumericAffinity(pIn2); + pOut = &aMem[pOp->p3]; u.af.flags = pIn1->flags | pIn2->flags; if( (u.af.flags & MEM_Null)!=0 ) goto arithmetic_result_is_null; if( (pIn1->flags & pIn2->flags & MEM_Int)==MEM_Int ){ @@ -53353,7 +53501,7 @@ case OP_CollSeq: { ** successors. The result of the function is stored in register P3. ** Register P3 must not be one of the function inputs. ** -** P1 is a 32-bit bitmask indicating whether or not each argument to the +** P1 is a 32-bit bitmask indicating whether or not each argument to the ** function was determined to be constant at compile time. If the first ** argument was constant then bit 0 of P1 is set. This is used to determine ** whether meta data associated with a user function argument using the @@ -53377,7 +53525,7 @@ case OP_Function: { assert( u.ag.n==0 || (pOp->p2>0 && pOp->p2+u.ag.n<=p->nMem+1) ); assert( pOp->p3p2 || pOp->p3>=pOp->p2+u.ag.n ); - u.ag.pArg = &p->aMem[pOp->p2]; + u.ag.pArg = &aMem[pOp->p2]; for(u.ag.i=0; u.ag.ip3>0 && pOp->p3<=p->nMem ); - pOut = &p->aMem[pOp->p3]; + pOut = &aMem[pOp->p3]; u.ag.ctx.s.flags = MEM_Null; u.ag.ctx.s.db = db; u.ag.ctx.s.xDel = 0; @@ -53409,7 +53557,7 @@ case OP_Function: { u.ag.ctx.isError = 0; if( u.ag.ctx.pFunc->flags & SQLITE_FUNC_NEEDCOLL ){ - assert( pOp>p->aOp ); + assert( pOp>aOp ); assert( pOp[-1].p4type==P4_COLLSEQ ); assert( pOp[-1].opcode==OP_CollSeq ); u.ag.ctx.pColl = pOp[-1].p4.pColl; @@ -53495,6 +53643,9 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */ i64 b; #endif /* local variables moved into u.ah */ + pIn1 = &aMem[pOp->p1]; + pIn2 = &aMem[pOp->p2]; + pOut = &aMem[pOp->p3]; if( (pIn1->flags | pIn2->flags) & MEM_Null ){ sqlite3VdbeMemSetNull(pOut); break; @@ -53514,26 +53665,28 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */ } /* Opcode: AddImm P1 P2 * * * -** +** ** Add the constant P2 to the value in register P1. ** The result is always an integer. ** ** To force any register to be an integer, just add 0. */ case OP_AddImm: { /* in1 */ + pIn1 = &aMem[pOp->p1]; sqlite3VdbeMemIntegerify(pIn1); pIn1->u.i += pOp->p2; break; } /* Opcode: MustBeInt P1 P2 * * * -** +** ** Force the value in register P1 to be an integer. If the value ** in P1 is not an integer and cannot be converted into an integer ** without data loss, then jump immediately to P2, or if P2==0 ** raise an SQLITE_MISMATCH exception. */ case OP_MustBeInt: { /* jump, in1 */ + pIn1 = &aMem[pOp->p1]; applyAffinity(pIn1, SQLITE_AFF_NUMERIC, encoding); if( (pIn1->flags & MEM_Int)==0 ){ if( pOp->p2==0 ){ @@ -53558,6 +53711,7 @@ case OP_MustBeInt: { /* jump, in1 */ ** to have only a real value. */ case OP_RealAffinity: { /* in1 */ + pIn1 = &aMem[pOp->p1]; if( pIn1->flags & MEM_Int ){ sqlite3VdbeMemRealify(pIn1); } @@ -53575,6 +53729,7 @@ case OP_RealAffinity: { /* in1 */ ** A NULL value is not changed by this routine. It remains NULL. */ case OP_ToText: { /* same as TK_TO_TEXT, in1 */ + pIn1 = &aMem[pOp->p1]; if( pIn1->flags & MEM_Null ) break; assert( MEM_Str==(MEM_Blob>>3) ); pIn1->flags |= (pIn1->flags&MEM_Blob)>>3; @@ -53596,6 +53751,7 @@ case OP_ToText: { /* same as TK_TO_TEXT, in1 */ ** A NULL value is not changed by this routine. It remains NULL. */ case OP_ToBlob: { /* same as TK_TO_BLOB, in1 */ + pIn1 = &aMem[pOp->p1]; if( pIn1->flags & MEM_Null ) break; if( (pIn1->flags & MEM_Blob)==0 ){ applyAffinity(pIn1, SQLITE_AFF_TEXT, encoding); @@ -53613,12 +53769,13 @@ case OP_ToBlob: { /* same as TK_TO_BLOB, in1 */ ** Force the value in register P1 to be numeric (either an ** integer or a floating-point number.) ** If the value is text or blob, try to convert it to an using the -** equivalent of atoi() or atof() and store 0 if no such conversion +** equivalent of atoi() or atof() and store 0 if no such conversion ** is possible. ** ** A NULL value is not changed by this routine. It remains NULL. */ case OP_ToNumeric: { /* same as TK_TO_NUMERIC, in1 */ + pIn1 = &aMem[pOp->p1]; if( (pIn1->flags & (MEM_Null|MEM_Int|MEM_Real))==0 ){ sqlite3VdbeMemNumerify(pIn1); } @@ -53636,6 +53793,7 @@ case OP_ToNumeric: { /* same as TK_TO_NUMERIC, in1 */ ** A NULL value is not changed by this routine. It remains NULL. */ case OP_ToInt: { /* same as TK_TO_INT, in1 */ + pIn1 = &aMem[pOp->p1]; if( (pIn1->flags & MEM_Null)==0 ){ sqlite3VdbeMemIntegerify(pIn1); } @@ -53653,6 +53811,7 @@ case OP_ToInt: { /* same as TK_TO_INT, in1 */ ** A NULL value is not changed by this routine. It remains NULL. */ case OP_ToReal: { /* same as TK_TO_REAL, in1 */ + pIn1 = &aMem[pOp->p1]; if( (pIn1->flags & MEM_Null)==0 ){ sqlite3VdbeMemRealify(pIn1); } @@ -53663,21 +53822,21 @@ case OP_ToReal: { /* same as TK_TO_REAL, in1 */ /* Opcode: Lt P1 P2 P3 P4 P5 ** ** Compare the values in register P1 and P3. If reg(P3)p1]; + pIn3 = &aMem[pOp->p3]; if( (pIn1->flags | pIn3->flags)&MEM_Null ){ /* One or both operands are NULL */ if( pOp->p5 & SQLITE_NULLEQ ){ @@ -53758,7 +53919,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */ ** The jump is taken if the SQLITE_JUMPIFNULL bit is set. */ if( pOp->p5 & SQLITE_STOREP2 ){ - pOut = &p->aMem[pOp->p2]; + pOut = &aMem[pOp->p2]; MemSetTypeFlag(pOut, MEM_Null); REGISTER_TRACE(pOp->p2, pOut); }else if( pOp->p5 & SQLITE_JUMPIFNULL ){ @@ -53790,7 +53951,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */ } if( pOp->p5 & SQLITE_STOREP2 ){ - pOut = &p->aMem[pOp->p2]; + pOut = &aMem[pOp->p2]; MemSetTypeFlag(pOut, MEM_Int); pOut->u.i = u.ai.res; REGISTER_TRACE(pOp->p2, pOut); @@ -53861,12 +54022,12 @@ case OP_Compare: { #endif /* SQLITE_DEBUG */ for(u.aj.i=0; u.aj.iaMem[u.aj.p1+u.aj.idx]); - REGISTER_TRACE(u.aj.p2+u.aj.idx, &p->aMem[u.aj.p2+u.aj.idx]); + REGISTER_TRACE(u.aj.p1+u.aj.idx, &aMem[u.aj.p1+u.aj.idx]); + REGISTER_TRACE(u.aj.p2+u.aj.idx, &aMem[u.aj.p2+u.aj.idx]); assert( u.aj.inField ); u.aj.pColl = u.aj.pKeyInfo->aColl[u.aj.i]; u.aj.bRev = u.aj.pKeyInfo->aSortOrder[u.aj.i]; - iCompare = sqlite3MemCompare(&p->aMem[u.aj.p1+u.aj.idx], &p->aMem[u.aj.p2+u.aj.idx], u.aj.pColl); + iCompare = sqlite3MemCompare(&aMem[u.aj.p1+u.aj.idx], &aMem[u.aj.p2+u.aj.idx], u.aj.pColl); if( iCompare ){ if( u.aj.bRev ) iCompare = -iCompare; break; @@ -53918,11 +54079,13 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */ int v2; /* Right operand: 0==FALSE, 1==TRUE, 2==UNKNOWN or NULL */ #endif /* local variables moved into u.ak */ + pIn1 = &aMem[pOp->p1]; if( pIn1->flags & MEM_Null ){ u.ak.v1 = 2; }else{ u.ak.v1 = sqlite3VdbeIntValue(pIn1)!=0; } + pIn2 = &aMem[pOp->p2]; if( pIn2->flags & MEM_Null ){ u.ak.v2 = 2; }else{ @@ -53935,6 +54098,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */ static const unsigned char or_logic[] = { 0, 1, 2, 1, 1, 1, 2, 1, 2 }; u.ak.v1 = or_logic[u.ak.v1*3+u.ak.v2]; } + pOut = &aMem[pOp->p3]; if( u.ak.v1==2 ){ MemSetTypeFlag(pOut, MEM_Null); }else{ @@ -53947,11 +54111,12 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */ /* Opcode: Not P1 P2 * * * ** ** Interpret the value in register P1 as a boolean value. Store the -** boolean complement in register P2. If the value in register P1 is +** boolean complement in register P2. If the value in register P1 is ** NULL, then a NULL is stored in P2. */ -case OP_Not: { /* same as TK_NOT, in1 */ - pOut = &p->aMem[pOp->p2]; +case OP_Not: { /* same as TK_NOT, in1, out2 */ + pIn1 = &aMem[pOp->p1]; + pOut = &aMem[pOp->p2]; if( pIn1->flags & MEM_Null ){ sqlite3VdbeMemSetNull(pOut); }else{ @@ -53966,8 +54131,9 @@ case OP_Not: { /* same as TK_NOT, in1 */ ** ones-complement of the P1 value into register P2. If P1 holds ** a NULL then store a NULL in P2. */ -case OP_BitNot: { /* same as TK_BITNOT, in1 */ - pOut = &p->aMem[pOp->p2]; +case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */ + pIn1 = &aMem[pOp->p1]; + pOut = &aMem[pOp->p2]; if( pIn1->flags & MEM_Null ){ sqlite3VdbeMemSetNull(pOut); }else{ @@ -53993,6 +54159,7 @@ case OP_IfNot: { /* jump, in1 */ #if 0 /* local variables moved into u.al */ int c; #endif /* local variables moved into u.al */ + pIn1 = &aMem[pOp->p1]; if( pIn1->flags & MEM_Null ){ u.al.c = pOp->p3; }else{ @@ -54014,6 +54181,7 @@ case OP_IfNot: { /* jump, in1 */ ** Jump to P2 if the value in register P1 is NULL. */ case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */ + pIn1 = &aMem[pOp->p1]; if( (pIn1->flags & MEM_Null)!=0 ){ pc = pOp->p2 - 1; } @@ -54022,9 +54190,10 @@ case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */ /* Opcode: NotNull P1 P2 * * * ** -** Jump to P2 if the value in register P1 is not NULL. +** Jump to P2 if the value in register P1 is not NULL. */ case OP_NotNull: { /* same as TK_NOTNULL, jump, in1 */ + pIn1 = &aMem[pOp->p1]; if( (pIn1->flags & MEM_Null)==0 ){ pc = pOp->p2 - 1; } @@ -54036,7 +54205,7 @@ case OP_NotNull: { /* same as TK_NOTNULL, jump, in1 */ ** Interpret the data that cursor P1 points to as a structure built using ** the MakeRecord instruction. (See the MakeRecord opcode for additional ** information about the format of the data.) Extract the P2-th column -** from this record. If there are less that (P2+1) +** from this record. If there are less that (P2+1) ** values in the record, extract a NULL. ** ** The value extracted is stored in register P3. @@ -54083,7 +54252,7 @@ case OP_Column: { memset(&u.am.sMem, 0, sizeof(u.am.sMem)); assert( u.am.p1nCursor ); assert( pOp->p3>0 && pOp->p3<=p->nMem ); - u.am.pDest = &p->aMem[pOp->p3]; + u.am.pDest = &aMem[pOp->p3]; MemSetTypeFlag(u.am.pDest, MEM_Null); u.am.zRec = 0; @@ -54129,7 +54298,7 @@ case OP_Column: { assert( rc==SQLITE_OK ); /* DataSize() cannot fail */ } }else if( u.am.pC->pseudoTableReg>0 ){ - u.am.pReg = &p->aMem[u.am.pC->pseudoTableReg]; + u.am.pReg = &aMem[u.am.pC->pseudoTableReg]; assert( u.am.pReg->flags & MEM_Blob ); u.am.payloadSize = u.am.pReg->n; u.am.zRec = u.am.pReg->z; @@ -54341,18 +54510,19 @@ op_column_out: */ case OP_Affinity: { #if 0 /* local variables moved into u.an */ - char *zAffinity; /* The affinity to be applied */ - Mem *pData0; /* First register to which to apply affinity */ - Mem *pLast; /* Last register to which to apply affinity */ - Mem *pRec; /* Current register */ + const char *zAffinity; /* The affinity to be applied */ + char cAff; /* A single character of affinity */ #endif /* local variables moved into u.an */ u.an.zAffinity = pOp->p4.z; - u.an.pData0 = &p->aMem[pOp->p1]; - u.an.pLast = &u.an.pData0[pOp->p2-1]; - for(u.an.pRec=u.an.pData0; u.an.pRec<=u.an.pLast; u.an.pRec++){ - ExpandBlob(u.an.pRec); - applyAffinity(u.an.pRec, u.an.zAffinity[u.an.pRec-u.an.pData0], encoding); + assert( u.an.zAffinity!=0 ); + assert( u.an.zAffinity[pOp->p2]==0 ); + pIn1 = &aMem[pOp->p1]; + while( (u.an.cAff = *(u.an.zAffinity++))!=0 ){ + assert( pIn1 <= &p->aMem[p->nMem] ); + ExpandBlob(pIn1); + applyAffinity(pIn1, u.an.cAff, encoding); + pIn1++; } break; } @@ -54416,7 +54586,7 @@ case OP_MakeRecord: { u.ao.nField = pOp->p1; u.ao.zAffinity = pOp->p4.z; assert( u.ao.nField>0 && pOp->p2>0 && pOp->p2+u.ao.nField<=p->nMem+1 ); - u.ao.pData0 = &p->aMem[u.ao.nField]; + u.ao.pData0 = &aMem[u.ao.nField]; u.ao.nField = pOp->p2; u.ao.pLast = &u.ao.pData0[u.ao.nField-1]; u.ao.file_format = p->minWriteFileFormat; @@ -54460,7 +54630,7 @@ case OP_MakeRecord: { ** sqlite3VdbeMemGrow() could clobber the value before it is used). */ assert( pOp->p3p1 || pOp->p3>=pOp->p1+pOp->p2 ); - pOut = &p->aMem[pOp->p3]; + pOut = &aMem[pOp->p3]; if( sqlite3VdbeMemGrow(pOut, (int)u.ao.nByte, 0) ){ goto no_mem; } @@ -54493,7 +54663,7 @@ case OP_MakeRecord: { /* Opcode: Count P1 P2 * * * ** -** Store the number of entries (an integer value) in the table or index +** Store the number of entries (an integer value) in the table or index ** opened by cursor P1 in register P2 */ #ifndef SQLITE_OMIT_BTREECOUNT @@ -54509,7 +54679,6 @@ case OP_Count: { /* out2-prerelease */ }else{ u.ap.nEntry = 0; } - pOut->flags = MEM_Int; pOut->u.i = u.ap.nEntry; break; } @@ -54838,16 +55007,15 @@ case OP_ReadCookie: { /* out2-prerelease */ sqlite3BtreeGetMeta(db->aDb[u.at.iDb].pBt, u.at.iCookie, (u32 *)&u.at.iMeta); pOut->u.i = u.at.iMeta; - MemSetTypeFlag(pOut, MEM_Int); break; } /* Opcode: SetCookie P1 P2 P3 * * ** ** Write the content of register P3 (interpreted as an integer) -** into cookie number P2 of database P1. P2==1 is the schema version. -** P2==2 is the database format. P2==3 is the recommended pager cache -** size, and so forth. P1==0 is the main database file and P1==1 is the +** into cookie number P2 of database P1. P2==1 is the schema version. +** P2==2 is the database format. P2==3 is the recommended pager cache +** size, and so forth. P1==0 is the main database file and P1==1 is the ** database file used to store temporary tables. ** ** A transaction must be started before executing this opcode. @@ -54861,6 +55029,7 @@ case OP_SetCookie: { /* in3 */ assert( (p->btreeMask & (1<p1))!=0 ); u.au.pDb = &db->aDb[pOp->p1]; assert( u.au.pDb->pBt!=0 ); + pIn3 = &aMem[pOp->p3]; sqlite3VdbeMemIntegerify(pIn3); /* See note about index shifting on OP_ReadCookie */ rc = sqlite3BtreeUpdateMeta(u.au.pDb->pBt, pOp->p2, (int)pIn3->u.i); @@ -54884,7 +55053,7 @@ case OP_SetCookie: { /* in3 */ /* Opcode: VerifyCookie P1 P2 * ** ** Check the value of global database parameter number 0 (the -** schema version) and make sure it is equal to P2. +** schema version) and make sure it is equal to P2. ** P1 is the database number which is 0 for the main database file ** and 1 for the file holding temporary tables and some higher number ** for auxiliary databases. @@ -54939,8 +55108,8 @@ case OP_VerifyCookie: { /* Opcode: OpenRead P1 P2 P3 P4 P5 ** ** Open a read-only cursor for the database table whose root page is -** P2 in a database file. The database file is determined by P3. -** P3==0 means the main database, P3==1 means the database used for +** P2 in a database file. The database file is determined by P3. +** P3==0 means the main database, P3==1 means the database used for ** temporary tables, and P3>1 means used the corresponding attached ** database. Give the new cursor an identifier of P1. The P1 ** values need not be contiguous but all P1 values should be small integers. @@ -54959,9 +55128,9 @@ case OP_VerifyCookie: { ** SQLITE_BUSY error code. ** ** The P4 value may be either an integer (P4_INT32) or a pointer to -** a KeyInfo structure (P4_KEYINFO). If it is a pointer to a KeyInfo -** structure, then said structure defines the content and collating -** sequence of the index being opened. Otherwise, if P4 is an integer +** a KeyInfo structure (P4_KEYINFO). If it is a pointer to a KeyInfo +** structure, then said structure defines the content and collating +** sequence of the index being opened. Otherwise, if P4 is an integer ** value, it is set to the number of columns in the table. ** ** See also OpenWrite. @@ -54973,9 +55142,9 @@ case OP_VerifyCookie: { ** root page. ** ** The P4 value may be either an integer (P4_INT32) or a pointer to -** a KeyInfo structure (P4_KEYINFO). If it is a pointer to a KeyInfo -** structure, then said structure defines the content and collating -** sequence of the index being opened. Otherwise, if P4 is an integer +** a KeyInfo structure (P4_KEYINFO). If it is a pointer to a KeyInfo +** structure, then said structure defines the content and collating +** sequence of the index being opened. Otherwise, if P4 is an integer ** value, it is set to the number of columns in the table, or to the ** largest index of any column of the table that is actually used. ** @@ -55023,7 +55192,7 @@ case OP_OpenWrite: { if( pOp->p5 ){ assert( u.aw.p2>0 ); assert( u.aw.p2<=p->nMem ); - pIn2 = &p->aMem[u.aw.p2]; + pIn2 = &aMem[u.aw.p2]; sqlite3VdbeMemIntegerify(pIn2); u.aw.p2 = (int)pIn2->u.i; /* The u.aw.p2 value always comes from a prior OP_CreateTable opcode and @@ -55071,7 +55240,7 @@ case OP_OpenWrite: { /* Opcode: OpenEphemeral P1 P2 * P4 * ** ** Open a new cursor P1 to a transient table. -** The cursor is always opened read/write even if +** The cursor is always opened read/write even if ** the main database is read-only. The transient or virtual ** table is deleted automatically when the cursor is closed. ** @@ -55137,7 +55306,7 @@ case OP_OpenEphemeral: { ** ** Open a new cursor that points to a fake table that contains a single ** row of data. The content of that one row in the content of memory -** register P2. In other words, cursor P1 becomes an alias for the +** register P2. In other words, cursor P1 becomes an alias for the ** MEM_Blob content contained in register P2. ** ** A pseudo-table created by this opcode is used to hold the a single @@ -55177,52 +55346,52 @@ case OP_Close: { /* Opcode: SeekGe P1 P2 P3 P4 * ** -** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), -** use the value in register P3 as the key. If cursor P1 refers -** to an SQL index, then P3 is the first in an array of P4 registers -** that are used as an unpacked index key. +** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), +** use the value in register P3 as the key. If cursor P1 refers +** to an SQL index, then P3 is the first in an array of P4 registers +** that are used as an unpacked index key. ** -** Reposition cursor P1 so that it points to the smallest entry that -** is greater than or equal to the key value. If there are no records +** Reposition cursor P1 so that it points to the smallest entry that +** is greater than or equal to the key value. If there are no records ** greater than or equal to the key and P2 is not zero, then jump to P2. ** ** See also: Found, NotFound, Distinct, SeekLt, SeekGt, SeekLe */ /* Opcode: SeekGt P1 P2 P3 P4 * ** -** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), -** use the value in register P3 as a key. If cursor P1 refers -** to an SQL index, then P3 is the first in an array of P4 registers -** that are used as an unpacked index key. +** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), +** use the value in register P3 as a key. If cursor P1 refers +** to an SQL index, then P3 is the first in an array of P4 registers +** that are used as an unpacked index key. ** -** Reposition cursor P1 so that it points to the smallest entry that -** is greater than the key value. If there are no records greater than +** Reposition cursor P1 so that it points to the smallest entry that +** is greater than the key value. If there are no records greater than ** the key and P2 is not zero, then jump to P2. ** ** See also: Found, NotFound, Distinct, SeekLt, SeekGe, SeekLe */ -/* Opcode: SeekLt P1 P2 P3 P4 * +/* Opcode: SeekLt P1 P2 P3 P4 * ** -** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), -** use the value in register P3 as a key. If cursor P1 refers -** to an SQL index, then P3 is the first in an array of P4 registers -** that are used as an unpacked index key. +** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), +** use the value in register P3 as a key. If cursor P1 refers +** to an SQL index, then P3 is the first in an array of P4 registers +** that are used as an unpacked index key. ** -** Reposition cursor P1 so that it points to the largest entry that -** is less than the key value. If there are no records less than +** Reposition cursor P1 so that it points to the largest entry that +** is less than the key value. If there are no records less than ** the key and P2 is not zero, then jump to P2. ** ** See also: Found, NotFound, Distinct, SeekGt, SeekGe, SeekLe */ /* Opcode: SeekLe P1 P2 P3 P4 * ** -** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), -** use the value in register P3 as a key. If cursor P1 refers -** to an SQL index, then P3 is the first in an array of P4 registers -** that are used as an unpacked index key. +** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), +** use the value in register P3 as a key. If cursor P1 refers +** to an SQL index, then P3 is the first in an array of P4 registers +** that are used as an unpacked index key. ** -** Reposition cursor P1 so that it points to the largest entry that -** is less than or equal to the key value. If there are no records +** Reposition cursor P1 so that it points to the largest entry that +** is less than or equal to the key value. If there are no records ** less than or equal to the key and P2 is not zero, then jump to P2. ** ** See also: Found, NotFound, Distinct, SeekGt, SeekGe, SeekLt @@ -55245,6 +55414,9 @@ case OP_SeekGt: { /* jump, in3 */ u.az.pC = p->apCsr[pOp->p1]; assert( u.az.pC!=0 ); assert( u.az.pC->pseudoTableReg==0 ); + assert( OP_SeekLe == OP_SeekLt+1 ); + assert( OP_SeekGe == OP_SeekLt+2 ); + assert( OP_SeekGt == OP_SeekLt+3 ); if( u.az.pC->pCursor!=0 ){ u.az.oc = pOp->opcode; u.az.pC->nullRow = 0; @@ -55252,6 +55424,7 @@ case OP_SeekGt: { /* jump, in3 */ /* The input value in P3 might be of any type: integer, real, string, ** blob, or NULL. But it needs to be an integer before we can do ** the seek, so covert it. */ + pIn3 = &aMem[pOp->p3]; applyNumericAffinity(pIn3); u.az.iKey = sqlite3VdbeIntValue(pIn3); u.az.pC->rowidIsValid = 0; @@ -55274,12 +55447,12 @@ case OP_SeekGt: { /* jump, in3 */ ** integer. */ u.az.res = 1; if( pIn3->r<0 ){ - if( u.az.oc==OP_SeekGt || u.az.oc==OP_SeekGe ){ + if( u.az.oc>=OP_SeekGe ){ assert( u.az.oc==OP_SeekGe || u.az.oc==OP_SeekGt ); rc = sqlite3BtreeFirst(u.az.pC->pCursor, &u.az.res); if( rc!=SQLITE_OK ) goto abort_due_to_error; } }else{ - if( u.az.oc==OP_SeekLt || u.az.oc==OP_SeekLe ){ + if( u.az.oc<=OP_SeekLe ){ assert( u.az.oc==OP_SeekLt || u.az.oc==OP_SeekLe ); rc = sqlite3BtreeLast(u.az.pC->pCursor, &u.az.res); if( rc!=SQLITE_OK ) goto abort_due_to_error; } @@ -55311,12 +55484,22 @@ case OP_SeekGt: { /* jump, in3 */ assert( u.az.nField>0 ); u.az.r.pKeyInfo = u.az.pC->pKeyInfo; u.az.r.nField = (u16)u.az.nField; - if( u.az.oc==OP_SeekGt || u.az.oc==OP_SeekLe ){ - u.az.r.flags = UNPACKED_INCRKEY; - }else{ - u.az.r.flags = 0; - } - u.az.r.aMem = &p->aMem[pOp->p3]; + + /* The next line of code computes as follows, only faster: + ** if( u.az.oc==OP_SeekGt || u.az.oc==OP_SeekLe ){ + ** u.az.r.flags = UNPACKED_INCRKEY; + ** }else{ + ** u.az.r.flags = 0; + ** } + */ + u.az.r.flags = (u16)(UNPACKED_INCRKEY * (1 & (u.az.oc - OP_SeekLt))); + assert( u.az.oc!=OP_SeekGt || u.az.r.flags==UNPACKED_INCRKEY ); + assert( u.az.oc!=OP_SeekLe || u.az.r.flags==UNPACKED_INCRKEY ); + assert( u.az.oc!=OP_SeekGe || u.az.r.flags==0 ); + assert( u.az.oc!=OP_SeekLt || u.az.r.flags==0 ); + + u.az.r.aMem = &aMem[pOp->p3]; + ExpandBlob(u.az.r.aMem); rc = sqlite3BtreeMovetoUnpacked(u.az.pC->pCursor, &u.az.r, 0, 0, &u.az.res); if( rc!=SQLITE_OK ){ goto abort_due_to_error; @@ -55328,7 +55511,7 @@ case OP_SeekGt: { /* jump, in3 */ #ifdef SQLITE_TEST sqlite3_search_count++; #endif - if( u.az.oc==OP_SeekGe || u.az.oc==OP_SeekGt ){ + if( u.az.oc>=OP_SeekGe ){ assert( u.az.oc==OP_SeekGe || u.az.oc==OP_SeekGt ); if( u.az.res<0 || (u.az.res==0 && u.az.oc==OP_SeekGt) ){ rc = sqlite3BtreeNext(u.az.pC->pCursor, &u.az.res); if( rc!=SQLITE_OK ) goto abort_due_to_error; @@ -55383,41 +55566,36 @@ case OP_Seek: { /* in2 */ if( ALWAYS(u.ba.pC->pCursor!=0) ){ assert( u.ba.pC->isTable ); u.ba.pC->nullRow = 0; + pIn2 = &aMem[pOp->p2]; u.ba.pC->movetoTarget = sqlite3VdbeIntValue(pIn2); u.ba.pC->rowidIsValid = 0; u.ba.pC->deferredMoveto = 1; } break; } - -/* Opcode: Found P1 P2 P3 * * + +/* Opcode: Found P1 P2 P3 P4 * ** -** Register P3 holds a blob constructed by MakeRecord. P1 is an index. -** If an entry that matches the value in register p3 exists in P1 then -** jump to P2. If the P3 value does not match any entry in P1 -** then fall thru. The P1 cursor is left pointing at the matching entry -** if it exists. +** If P4==0 then register P3 holds a blob constructed by MakeRecord. If +** P4>0 then register P3 is the first of P4 registers that form an unpacked +** record. ** -** This instruction is used to implement the IN operator where the -** left-hand side is a SELECT statement. P1 may be a true index, or it -** may be a temporary index that holds the results of the SELECT -** statement. This instruction is also used to implement the -** DISTINCT keyword in SELECT statements. -** -** This instruction checks if index P1 contains a record for which -** the first N serialized values exactly match the N serialized values -** in the record in register P3, where N is the total number of values in -** the P3 record (the P3 record is a prefix of the P1 record). -** -** See also: NotFound, IsUnique, NotExists +** Cursor P1 is on an index btree. If the record identified by P3 and P4 +** is a prefix of any entry in P1 then a jump is made to P2 and +** P1 is left pointing at the matching entry. */ -/* Opcode: NotFound P1 P2 P3 * * +/* Opcode: NotFound P1 P2 P3 P4 * ** -** Register P3 holds a blob constructed by MakeRecord. P1 is -** an index. If no entry exists in P1 that matches the blob then jump -** to P2. If an entry does existing, fall through. The cursor is left -** pointing to the entry that matches. +** If P4==0 then register P3 holds a blob constructed by MakeRecord. If +** P4>0 then register P3 is the first of P4 registers that form an unpacked +** record. +** +** Cursor P1 is on an index btree. If the record identified by P3 and P4 +** is not the prefix of any entry in P1 then a jump is made to P2. If P1 +** does contain an entry whose prefix matches the P3/P4 record then control +** falls through to the next instruction and P1 is left pointing at the +** matching entry. ** ** See also: Found, NotExists, IsUnique */ @@ -55428,6 +55606,7 @@ case OP_Found: { /* jump, in3 */ VdbeCursor *pC; int res; UnpackedRecord *pIdxKey; + UnpackedRecord r; char aTempRec[ROUND8(sizeof(UnpackedRecord)) + sizeof(Mem)*3 + 7]; #endif /* local variables moved into u.bb */ @@ -55437,23 +55616,33 @@ case OP_Found: { /* jump, in3 */ u.bb.alreadyExists = 0; assert( pOp->p1>=0 && pOp->p1nCursor ); + assert( pOp->p4type==P4_INT32 ); u.bb.pC = p->apCsr[pOp->p1]; assert( u.bb.pC!=0 ); + pIn3 = &aMem[pOp->p3]; if( ALWAYS(u.bb.pC->pCursor!=0) ){ assert( u.bb.pC->isTable==0 ); - assert( pIn3->flags & MEM_Blob ); - ExpandBlob(pIn3); - u.bb.pIdxKey = sqlite3VdbeRecordUnpack(u.bb.pC->pKeyInfo, pIn3->n, pIn3->z, - u.bb.aTempRec, sizeof(u.bb.aTempRec)); - if( u.bb.pIdxKey==0 ){ - goto no_mem; - } - if( pOp->opcode==OP_Found ){ + if( pOp->p4.i>0 ){ + u.bb.r.pKeyInfo = u.bb.pC->pKeyInfo; + u.bb.r.nField = (u16)pOp->p4.i; + u.bb.r.aMem = pIn3; + u.bb.r.flags = UNPACKED_PREFIX_MATCH; + u.bb.pIdxKey = &u.bb.r; + }else{ + assert( pIn3->flags & MEM_Blob ); + ExpandBlob(pIn3); + u.bb.pIdxKey = sqlite3VdbeRecordUnpack(u.bb.pC->pKeyInfo, pIn3->n, pIn3->z, + u.bb.aTempRec, sizeof(u.bb.aTempRec)); + if( u.bb.pIdxKey==0 ){ + goto no_mem; + } u.bb.pIdxKey->flags |= UNPACKED_PREFIX_MATCH; } rc = sqlite3BtreeMovetoUnpacked(u.bb.pC->pCursor, u.bb.pIdxKey, 0, 0, &u.bb.res); - sqlite3VdbeDeleteUnpackedRecord(u.bb.pIdxKey); + if( pOp->p4.i==0 ){ + sqlite3VdbeDeleteUnpackedRecord(u.bb.pIdxKey); + } if( rc!=SQLITE_OK ){ break; } @@ -55471,11 +55660,12 @@ case OP_Found: { /* jump, in3 */ /* Opcode: IsUnique P1 P2 P3 P4 * ** -** Cursor P1 is open on an index. So it has no data and its key consists -** of a record generated by OP_MakeRecord where the last field is the -** rowid of the entry that the index refers to. +** Cursor P1 is open on an index b-tree - that is to say, a btree which +** no data and where the key are records generated by OP_MakeRecord with +** the list field being the integer ROWID of the entry that the index +** entry refers to. ** -** The P3 register contains an integer record number. Call this record +** The P3 register contains an integer record number. Call this record ** number R. Register P4 is the first in a set of N contiguous registers ** that make up an unpacked index key that can be used with cursor P1. ** The value of N can be inferred from the cursor. N includes the rowid @@ -55500,12 +55690,13 @@ case OP_IsUnique: { /* jump, in3 */ VdbeCursor *pCx; BtCursor *pCrsr; u16 nField; - Mem *aMem; + Mem *aMx; UnpackedRecord r; /* B-Tree index search key */ i64 R; /* Rowid stored in register P3 */ #endif /* local variables moved into u.bc */ - u.bc.aMem = &p->aMem[pOp->p4.i]; + pIn3 = &aMem[pOp->p3]; + u.bc.aMx = &aMem[pOp->p4.i]; /* Assert that the values of parameters P1 and P4 are in range. */ assert( pOp->p4type==P4_INT32 ); assert( pOp->p4.i>0 && pOp->p4.i<=p->nMem ); @@ -55521,20 +55712,20 @@ case OP_IsUnique: { /* jump, in3 */ /* If any of the values are NULL, take the jump. */ u.bc.nField = u.bc.pCx->pKeyInfo->nField; for(u.bc.ii=0; u.bc.iip2 - 1; u.bc.pCrsr = 0; break; } } - assert( (u.bc.aMem[u.bc.nField].flags & MEM_Null)==0 ); + assert( (u.bc.aMx[u.bc.nField].flags & MEM_Null)==0 ); if( u.bc.pCrsr!=0 ){ /* Populate the index search key. */ u.bc.r.pKeyInfo = u.bc.pCx->pKeyInfo; u.bc.r.nField = u.bc.nField + 1; u.bc.r.flags = UNPACKED_PREFIX_SEARCH; - u.bc.r.aMem = u.bc.aMem; + u.bc.r.aMem = u.bc.aMx; /* Extract the value of u.bc.R from register P3. */ sqlite3VdbeMemIntegerify(pIn3); @@ -55555,9 +55746,9 @@ case OP_IsUnique: { /* jump, in3 */ /* Opcode: NotExists P1 P2 P3 * * ** -** Use the content of register P3 as a integer key. If a record -** with that key does not exist in table of P1, then jump to P2. -** If the record does exist, then fall thru. The cursor is left +** Use the content of register P3 as a integer key. If a record +** with that key does not exist in table of P1, then jump to P2. +** If the record does exist, then fall thru. The cursor is left ** pointing to the record if it exists. ** ** The difference between this operation and NotFound is that this @@ -55575,6 +55766,7 @@ case OP_NotExists: { /* jump, in3 */ u64 iKey; #endif /* local variables moved into u.bd */ + pIn3 = &aMem[pOp->p3]; assert( pIn3->flags & MEM_Int ); assert( pOp->p1>=0 && pOp->p1nCursor ); u.bd.pC = p->apCsr[pOp->p1]; @@ -55612,13 +55804,12 @@ case OP_NotExists: { /* jump, in3 */ ** Find the next available sequence number for cursor P1. ** Write the sequence number into register P2. ** The sequence number on the cursor is incremented after this -** instruction. +** instruction. */ case OP_Sequence: { /* out2-prerelease */ assert( pOp->p1>=0 && pOp->p1nCursor ); assert( p->apCsr[pOp->p1]!=0 ); pOut->u.i = p->apCsr[pOp->p1]->seqCount++; - MemSetTypeFlag(pOut, MEM_Int); break; } @@ -55630,9 +55821,9 @@ case OP_Sequence: { /* out2-prerelease */ ** table that cursor P1 points to. The new record number is written ** written to register P2. ** -** If P3>0 then P3 is a register in the root frame of this VDBE that holds +** If P3>0 then P3 is a register in the root frame of this VDBE that holds ** the largest previously generated record number. No new record numbers are -** allowed to be less than this value. When this value reaches its maximum, +** allowed to be less than this value. When this value reaches its maximum, ** a SQLITE_FULL error is generated. The P3 register is updated with the ' ** generated record number. This P3 mechanism is used to help implement the ** AUTOINCREMENT feature. @@ -55714,7 +55905,7 @@ case OP_NewRowid: { /* out2-prerelease */ }else{ /* Assert that P3 is a valid memory cell. */ assert( pOp->p3<=p->nMem ); - u.be.pMem = &p->aMem[pOp->p3]; + u.be.pMem = &aMem[pOp->p3]; } REGISTER_TRACE(pOp->p3, u.be.pMem); @@ -55757,7 +55948,6 @@ case OP_NewRowid: { /* out2-prerelease */ u.be.pC->deferredMoveto = 0; u.be.pC->cacheStatus = CACHE_STALE; } - MemSetTypeFlag(pOut, MEM_Int); pOut->u.i = u.be.v; break; } @@ -55789,7 +55979,7 @@ case OP_NewRowid: { /* out2-prerelease */ ** the update hook. ** ** Parameter P4 may point to a string containing the table-name, or -** may be NULL. If it is not NULL, then the update-hook +** may be NULL. If it is not NULL, then the update-hook ** (sqlite3.xUpdateCallback) is invoked following a successful insert. ** ** (WARNING/TODO: If P1 is a pseudo-cursor and P2 is dynamically @@ -55806,7 +55996,7 @@ case OP_NewRowid: { /* out2-prerelease */ ** This works exactly like OP_Insert except that the key is the ** integer value P3, not the value of the integer stored in register P3. */ -case OP_Insert: +case OP_Insert: case OP_InsertInt: { #if 0 /* local variables moved into u.bf */ Mem *pData; /* MEM cell holding data for the record to be inserted */ @@ -55820,7 +56010,7 @@ case OP_InsertInt: { int op; /* Opcode for update hook: SQLITE_UPDATE or SQLITE_INSERT */ #endif /* local variables moved into u.bf */ - u.bf.pData = &p->aMem[pOp->p2]; + u.bf.pData = &aMem[pOp->p2]; assert( pOp->p1>=0 && pOp->p1nCursor ); u.bf.pC = p->apCsr[pOp->p1]; assert( u.bf.pC!=0 ); @@ -55830,7 +56020,7 @@ case OP_InsertInt: { REGISTER_TRACE(pOp->p2, u.bf.pData); if( pOp->opcode==OP_Insert ){ - u.bf.pKey = &p->aMem[pOp->p3]; + u.bf.pKey = &aMem[pOp->p3]; assert( u.bf.pKey->flags & MEM_Int ); REGISTER_TRACE(pOp->p3, u.bf.pKey); u.bf.iKey = u.bf.pKey->u.i; @@ -55956,8 +56146,8 @@ case OP_ResetCount: { /* Opcode: RowData P1 P2 * * * ** ** Write into register P2 the complete row data for cursor P1. -** There is no interpretation of the data. -** It is just copied onto the P2 register exactly as +** There is no interpretation of the data. +** It is just copied onto the P2 register exactly as ** it is found in the database file. ** ** If the P1 cursor must be pointing to a valid row (not a NULL row) @@ -55966,8 +56156,8 @@ case OP_ResetCount: { /* Opcode: RowKey P1 P2 * * * ** ** Write into register P2 the complete row key for cursor P1. -** There is no interpretation of the data. -** The key is copied onto the P3 register exactly as +** There is no interpretation of the data. +** The key is copied onto the P3 register exactly as ** it is found in the database file. ** ** If the P1 cursor must be pointing to a valid row (not a NULL row) @@ -55982,7 +56172,7 @@ case OP_RowData: { i64 n64; #endif /* local variables moved into u.bh */ - pOut = &p->aMem[pOp->p2]; + pOut = &aMem[pOp->p2]; /* Note that RowKey and RowData are really exactly the same instruction */ assert( pOp->p1>=0 && pOp->p1nCursor ); @@ -56057,7 +56247,7 @@ case OP_Rowid: { /* out2-prerelease */ assert( u.bi.pC!=0 ); assert( u.bi.pC->pseudoTableReg==0 ); if( u.bi.pC->nullRow ){ - /* Do nothing so that reg[P2] remains NULL */ + pOut->flags = MEM_Null; break; }else if( u.bi.pC->deferredMoveto ){ u.bi.v = u.bi.pC->movetoTarget; @@ -56085,7 +56275,6 @@ case OP_Rowid: { /* out2-prerelease */ } } pOut->u.i = u.bi.v; - MemSetTypeFlag(pOut, MEM_Int); break; } @@ -56113,7 +56302,7 @@ case OP_NullRow: { /* Opcode: Last P1 P2 * * * ** -** The next use of the Rowid or Column or Next instruction for P1 +** The next use of the Rowid or Column or Next instruction for P1 ** will refer to the last entry in the database table or index. ** If the table or index is empty and P2>0, then jump immediately to P2. ** If P2 is 0 or if the table or index is not empty, fall through @@ -56168,7 +56357,7 @@ case OP_Sort: { /* jump */ } /* Opcode: Rewind P1 P2 * * * ** -** The next use of the Rowid or Column or Next instruction for P1 +** The next use of the Rowid or Column or Next instruction for P1 ** will refer to the first entry in the database table or index. ** If the table or index is empty and P2>0, then jump immediately to P2. ** If P2 is 0 or if the table or index is not empty, fall through @@ -56280,6 +56469,7 @@ case OP_IdxInsert: { /* in2 */ assert( pOp->p1>=0 && pOp->p1nCursor ); u.bn.pC = p->apCsr[pOp->p1]; assert( u.bn.pC!=0 ); + pIn2 = &aMem[pOp->p2]; assert( pIn2->flags & MEM_Blob ); u.bn.pCrsr = u.bn.pC->pCursor; if( ALWAYS(u.bn.pCrsr!=0) ){ @@ -56301,7 +56491,7 @@ case OP_IdxInsert: { /* in2 */ /* Opcode: IdxDelete P1 P2 P3 * * ** ** The content of P3 registers starting at register P2 form -** an unpacked index key. This opcode removes that entry from the +** an unpacked index key. This opcode removes that entry from the ** index opened by cursor P1. */ case OP_IdxDelete: { @@ -56322,7 +56512,7 @@ case OP_IdxDelete: { u.bo.r.pKeyInfo = u.bo.pC->pKeyInfo; u.bo.r.nField = (u16)pOp->p3; u.bo.r.flags = 0; - u.bo.r.aMem = &p->aMem[pOp->p2]; + u.bo.r.aMem = &aMem[pOp->p2]; rc = sqlite3BtreeMovetoUnpacked(u.bo.pCrsr, &u.bo.r, 0, 0, &u.bo.res); if( rc==SQLITE_OK && u.bo.res==0 ){ rc = sqlite3BtreeDelete(u.bo.pCrsr); @@ -56352,6 +56542,7 @@ case OP_IdxRowid: { /* out2-prerelease */ u.bp.pC = p->apCsr[pOp->p1]; assert( u.bp.pC!=0 ); u.bp.pCrsr = u.bp.pC->pCursor; + pOut->flags = MEM_Null; if( ALWAYS(u.bp.pCrsr!=0) ){ rc = sqlite3VdbeCursorMoveto(u.bp.pC); if( NEVER(rc) ) goto abort_due_to_error; @@ -56362,8 +56553,8 @@ case OP_IdxRowid: { /* out2-prerelease */ if( rc!=SQLITE_OK ){ goto abort_due_to_error; } - MemSetTypeFlag(pOut, MEM_Int); pOut->u.i = u.bp.rowid; + pOut->flags = MEM_Int; } } break; @@ -56371,32 +56562,32 @@ case OP_IdxRowid: { /* out2-prerelease */ /* Opcode: IdxGE P1 P2 P3 P4 P5 ** -** The P4 register values beginning with P3 form an unpacked index -** key that omits the ROWID. Compare this key value against the index +** The P4 register values beginning with P3 form an unpacked index +** key that omits the ROWID. Compare this key value against the index ** that P1 is currently pointing to, ignoring the ROWID on the P1 index. ** ** If the P1 index entry is greater than or equal to the key value ** then jump to P2. Otherwise fall through to the next instruction. ** -** If P5 is non-zero then the key value is increased by an epsilon +** If P5 is non-zero then the key value is increased by an epsilon ** prior to the comparison. This make the opcode work like IdxGT except ** that if the key from register P3 is a prefix of the key in the cursor, ** the result is false whereas it would be true with IdxGT. */ /* Opcode: IdxLT P1 P2 P3 * P5 ** -** The P4 register values beginning with P3 form an unpacked index -** key that omits the ROWID. Compare this key value against the index +** The P4 register values beginning with P3 form an unpacked index +** key that omits the ROWID. Compare this key value against the index ** that P1 is currently pointing to, ignoring the ROWID on the P1 index. ** ** If the P1 index entry is less than the key value then jump to P2. ** Otherwise fall through to the next instruction. ** -** If P5 is non-zero then the key value is increased by an epsilon prior +** If P5 is non-zero then the key value is increased by an epsilon prior ** to the comparison. This makes the opcode work like IdxLE. */ -case OP_IdxLT: /* jump, in3 */ -case OP_IdxGE: { /* jump, in3 */ +case OP_IdxLT: /* jump */ +case OP_IdxGE: { /* jump */ #if 0 /* local variables moved into u.bq */ VdbeCursor *pC; int res; @@ -56417,7 +56608,7 @@ case OP_IdxGE: { /* jump, in3 */ }else{ u.bq.r.flags = UNPACKED_IGNORE_ROWID; } - u.bq.r.aMem = &p->aMem[pOp->p3]; + u.bq.r.aMem = &aMem[pOp->p3]; rc = sqlite3VdbeIdxKeyCompare(u.bq.pC, &u.bq.r, &u.bq.res); if( pOp->opcode==OP_IdxLT ){ u.bq.res = -u.bq.res; @@ -56445,8 +56636,8 @@ case OP_IdxGE: { /* jump, in3 */ ** might be moved into the newly deleted root page in order to keep all ** root pages contiguous at the beginning of the database. The former ** value of the root page that moved - its value before the move occurred - -** is stored in register P2. If no page -** movement was required (because the table being dropped was already +** is stored in register P2. If no page +** movement was required (because the table being dropped was already ** the last one in the database) then a zero is stored in register P2. ** If AUTOVACUUM is disabled then a zero is stored in register P2. ** @@ -56469,6 +56660,7 @@ case OP_Destroy: { /* out2-prerelease */ #else u.br.iCnt = db->activeVdbeCnt; #endif + pOut->flags = MEM_Null; if( u.br.iCnt>1 ){ rc = SQLITE_LOCKED; p->errorAction = OE_Abort; @@ -56477,11 +56669,12 @@ case OP_Destroy: { /* out2-prerelease */ assert( u.br.iCnt==1 ); assert( (p->btreeMask & (1<aDb[u.br.iDb].pBt, pOp->p1, &u.br.iMoved); - MemSetTypeFlag(pOut, MEM_Int); + pOut->flags = MEM_Int; pOut->u.i = u.br.iMoved; #ifndef SQLITE_OMIT_AUTOVACUUM if( rc==SQLITE_OK && u.br.iMoved!=0 ){ sqlite3RootPageMoved(&db->aDb[u.br.iDb], u.br.iMoved, pOp->p1); + resetSchemaOnFault = 1; } #endif } @@ -56499,8 +56692,8 @@ case OP_Destroy: { /* out2-prerelease */ ** that is used to store tables create using CREATE TEMPORARY TABLE. ** ** If the P3 value is non-zero, then the table referred to must be an -** intkey table (an SQL table, not an index). In this case the row change -** count is incremented by the number of rows in the table being cleared. +** intkey table (an SQL table, not an index). In this case the row change +** count is incremented by the number of rows in the table being cleared. ** If P3 is greater than zero, then the value stored in register P3 is ** also incremented by the number of rows in the table being cleared. ** @@ -56519,7 +56712,7 @@ case OP_Clear: { if( pOp->p3 ){ p->nChange += u.bs.nChange; if( pOp->p3>0 ){ - p->aMem[pOp->p3].u.i += u.bs.nChange; + aMem[pOp->p3].u.i += u.bs.nChange; } } break; @@ -56568,7 +56761,6 @@ case OP_CreateTable: { /* out2-prerelease */ } rc = sqlite3BtreeCreateTable(u.bt.pDb->pBt, &u.bt.pgno, u.bt.flags); pOut->u.i = u.bt.pgno; - MemSetTypeFlag(pOut, MEM_Int); break; } @@ -56658,7 +56850,7 @@ case OP_ParseSchema: { case OP_LoadAnalysis: { assert( pOp->p1>=0 && pOp->p1nDb ); rc = sqlite3AnalysisLoad(db, pOp->p1); - break; + break; } #endif /* !defined(SQLITE_OMIT_ANALYZE) */ @@ -56708,7 +56900,7 @@ case OP_DropTrigger: { ** ** The register P3 contains the maximum number of allowed errors. ** At most reg(P3) errors will be reported. -** In other words, the analysis stops as soon as reg(P1) errors are +** In other words, the analysis stops as soon as reg(P1) errors are ** seen. Reg(P1) is updated with the number of errors remaining. ** ** The root page numbers of all tables in the database are integer @@ -56735,10 +56927,10 @@ case OP_IntegrityCk: { u.bv.aRoot = sqlite3DbMallocRaw(db, sizeof(int)*(u.bv.nRoot+1) ); if( u.bv.aRoot==0 ) goto no_mem; assert( pOp->p3>0 && pOp->p3<=p->nMem ); - u.bv.pnErr = &p->aMem[pOp->p3]; + u.bv.pnErr = &aMem[pOp->p3]; assert( (u.bv.pnErr->flags & MEM_Int)!=0 ); assert( (u.bv.pnErr->flags & (MEM_Str|MEM_Blob))==0 ); - pIn1 = &p->aMem[pOp->p1]; + pIn1 = &aMem[pOp->p1]; for(u.bv.j=0; u.bv.jp1>0 && pOp->p1<=p->nMem ); - u.bw.pIdx = &p->aMem[pOp->p1]; - assert( pOp->p2>0 && pOp->p2<=p->nMem ); - u.bw.pVal = &p->aMem[pOp->p2]; - assert( (u.bw.pVal->flags & MEM_Int)!=0 ); - if( (u.bw.pIdx->flags & MEM_RowSet)==0 ){ - sqlite3VdbeMemSetRowSet(u.bw.pIdx); - if( (u.bw.pIdx->flags & MEM_RowSet)==0 ) goto no_mem; +case OP_RowSetAdd: { /* in1, in2 */ + pIn1 = &aMem[pOp->p1]; + pIn2 = &aMem[pOp->p2]; + assert( (pIn2->flags & MEM_Int)!=0 ); + if( (pIn1->flags & MEM_RowSet)==0 ){ + sqlite3VdbeMemSetRowSet(pIn1); + if( (pIn1->flags & MEM_RowSet)==0 ) goto no_mem; } - sqlite3RowSetInsert(u.bw.pIdx->u.pRowSet, u.bw.pVal->u.i); + sqlite3RowSetInsert(pIn1->u.pRowSet, pIn2->u.i); break; } @@ -56794,25 +56980,21 @@ case OP_RowSetAdd: { /* in2 */ ** register P3. Or, if boolean index P1 is initially empty, leave P3 ** unchanged and jump to instruction P2. */ -case OP_RowSetRead: { /* jump, out3 */ -#if 0 /* local variables moved into u.bx */ - Mem *pIdx; +case OP_RowSetRead: { /* jump, in1, out3 */ +#if 0 /* local variables moved into u.bw */ i64 val; -#endif /* local variables moved into u.bx */ - assert( pOp->p1>0 && pOp->p1<=p->nMem ); +#endif /* local variables moved into u.bw */ CHECK_FOR_INTERRUPT; - u.bx.pIdx = &p->aMem[pOp->p1]; - pOut = &p->aMem[pOp->p3]; - if( (u.bx.pIdx->flags & MEM_RowSet)==0 - || sqlite3RowSetNext(u.bx.pIdx->u.pRowSet, &u.bx.val)==0 + pIn1 = &aMem[pOp->p1]; + if( (pIn1->flags & MEM_RowSet)==0 + || sqlite3RowSetNext(pIn1->u.pRowSet, &u.bw.val)==0 ){ /* The boolean index is empty */ - sqlite3VdbeMemSetNull(u.bx.pIdx); + sqlite3VdbeMemSetNull(pIn1); pc = pOp->p2 - 1; }else{ /* A value was pulled from the index */ - assert( pOp->p3>0 && pOp->p3<=p->nMem ); - sqlite3VdbeMemSetInt64(pOut, u.bx.val); + sqlite3VdbeMemSetInt64(&aMem[pOp->p3], u.bw.val); } break; } @@ -56841,12 +57023,14 @@ case OP_RowSetRead: { /* jump, out3 */ ** inserted as part of some other set). */ case OP_RowSetTest: { /* jump, in1, in3 */ -#if 0 /* local variables moved into u.by */ +#if 0 /* local variables moved into u.bx */ int iSet; int exists; -#endif /* local variables moved into u.by */ +#endif /* local variables moved into u.bx */ - u.by.iSet = pOp->p4.i; + pIn1 = &aMem[pOp->p1]; + pIn3 = &aMem[pOp->p3]; + u.bx.iSet = pOp->p4.i; assert( pIn3->flags&MEM_Int ); /* If there is anything other than a rowset object in memory cell P1, @@ -56858,17 +57042,17 @@ case OP_RowSetTest: { /* jump, in1, in3 */ } assert( pOp->p4type==P4_INT32 ); - assert( u.by.iSet==-1 || u.by.iSet>=0 ); - if( u.by.iSet ){ - u.by.exists = sqlite3RowSetTest(pIn1->u.pRowSet, - (u8)(u.by.iSet>=0 ? u.by.iSet & 0xf : 0xff), + assert( u.bx.iSet==-1 || u.bx.iSet>=0 ); + if( u.bx.iSet ){ + u.bx.exists = sqlite3RowSetTest(pIn1->u.pRowSet, + (u8)(u.bx.iSet>=0 ? u.bx.iSet & 0xf : 0xff), pIn3->u.i); - if( u.by.exists ){ + if( u.bx.exists ){ pc = pOp->p2 - 1; break; } } - if( u.by.iSet>=0 ){ + if( u.bx.iSet>=0 ){ sqlite3RowSetInsert(pIn1->u.pRowSet, pIn3->u.i); } break; @@ -56879,19 +57063,19 @@ case OP_RowSetTest: { /* jump, in1, in3 */ /* Opcode: Program P1 P2 P3 P4 * ** -** Execute the trigger program passed as P4 (type P4_SUBPROGRAM). +** Execute the trigger program passed as P4 (type P4_SUBPROGRAM). ** -** P1 contains the address of the memory cell that contains the first memory -** cell in an array of values used as arguments to the sub-program. P2 -** contains the address to jump to if the sub-program throws an IGNORE -** exception using the RAISE() function. Register P3 contains the address -** of a memory cell in this (the parent) VM that is used to allocate the +** P1 contains the address of the memory cell that contains the first memory +** cell in an array of values used as arguments to the sub-program. P2 +** contains the address to jump to if the sub-program throws an IGNORE +** exception using the RAISE() function. Register P3 contains the address +** of a memory cell in this (the parent) VM that is used to allocate the ** memory required by the sub-vdbe at runtime. ** ** P4 is a pointer to the VM containing the trigger program. */ case OP_Program: { /* jump */ -#if 0 /* local variables moved into u.bz */ +#if 0 /* local variables moved into u.by */ int nMem; /* Number of memory registers for sub-program */ int nByte; /* Bytes of runtime space required for sub-program */ Mem *pRt; /* Register to allocate runtime space */ @@ -56900,11 +57084,11 @@ case OP_Program: { /* jump */ VdbeFrame *pFrame; /* New vdbe frame to execute in */ SubProgram *pProgram; /* Sub-program to execute */ void *t; /* Token identifying trigger */ -#endif /* local variables moved into u.bz */ +#endif /* local variables moved into u.by */ - u.bz.pProgram = pOp->p4.pProgram; - u.bz.pRt = &p->aMem[pOp->p3]; - assert( u.bz.pProgram->nOp>0 ); + u.by.pProgram = pOp->p4.pProgram; + u.by.pRt = &aMem[pOp->p3]; + assert( u.by.pProgram->nOp>0 ); /* If the p5 flag is clear, then recursive invocation of triggers is ** disabled for backwards compatibility (p5 is set if this sub-program @@ -56918,9 +57102,9 @@ case OP_Program: { /* jump */ ** single trigger all have the same value for the SubProgram.token ** variable. */ if( pOp->p5 ){ - u.bz.t = u.bz.pProgram->token; - for(u.bz.pFrame=p->pFrame; u.bz.pFrame && u.bz.pFrame->token!=u.bz.t; u.bz.pFrame=u.bz.pFrame->pParent); - if( u.bz.pFrame ) break; + u.by.t = u.by.pProgram->token; + for(u.by.pFrame=p->pFrame; u.by.pFrame && u.by.pFrame->token!=u.by.t; u.by.pFrame=u.by.pFrame->pParent); + if( u.by.pFrame ) break; } if( p->nFrame>=db->aLimit[SQLITE_LIMIT_TRIGGER_DEPTH] ){ @@ -56929,64 +57113,64 @@ case OP_Program: { /* jump */ break; } - /* Register u.bz.pRt is used to store the memory required to save the state + /* Register u.by.pRt is used to store the memory required to save the state ** of the current program, and the memory required at runtime to execute - ** the trigger program. If this trigger has been fired before, then u.bz.pRt + ** the trigger program. If this trigger has been fired before, then u.by.pRt ** is already allocated. Otherwise, it must be initialized. */ - if( (u.bz.pRt->flags&MEM_Frame)==0 ){ + if( (u.by.pRt->flags&MEM_Frame)==0 ){ /* SubProgram.nMem is set to the number of memory cells used by the ** program stored in SubProgram.aOp. As well as these, one memory ** cell is required for each cursor used by the program. Set local - ** variable u.bz.nMem (and later, VdbeFrame.nChildMem) to this value. + ** variable u.by.nMem (and later, VdbeFrame.nChildMem) to this value. */ - u.bz.nMem = u.bz.pProgram->nMem + u.bz.pProgram->nCsr; - u.bz.nByte = ROUND8(sizeof(VdbeFrame)) - + u.bz.nMem * sizeof(Mem) - + u.bz.pProgram->nCsr * sizeof(VdbeCursor *); - u.bz.pFrame = sqlite3DbMallocZero(db, u.bz.nByte); - if( !u.bz.pFrame ){ + u.by.nMem = u.by.pProgram->nMem + u.by.pProgram->nCsr; + u.by.nByte = ROUND8(sizeof(VdbeFrame)) + + u.by.nMem * sizeof(Mem) + + u.by.pProgram->nCsr * sizeof(VdbeCursor *); + u.by.pFrame = sqlite3DbMallocZero(db, u.by.nByte); + if( !u.by.pFrame ){ goto no_mem; } - sqlite3VdbeMemRelease(u.bz.pRt); - u.bz.pRt->flags = MEM_Frame; - u.bz.pRt->u.pFrame = u.bz.pFrame; + sqlite3VdbeMemRelease(u.by.pRt); + u.by.pRt->flags = MEM_Frame; + u.by.pRt->u.pFrame = u.by.pFrame; - u.bz.pFrame->v = p; - u.bz.pFrame->nChildMem = u.bz.nMem; - u.bz.pFrame->nChildCsr = u.bz.pProgram->nCsr; - u.bz.pFrame->pc = pc; - u.bz.pFrame->aMem = p->aMem; - u.bz.pFrame->nMem = p->nMem; - u.bz.pFrame->apCsr = p->apCsr; - u.bz.pFrame->nCursor = p->nCursor; - u.bz.pFrame->aOp = p->aOp; - u.bz.pFrame->nOp = p->nOp; - u.bz.pFrame->token = u.bz.pProgram->token; + u.by.pFrame->v = p; + u.by.pFrame->nChildMem = u.by.nMem; + u.by.pFrame->nChildCsr = u.by.pProgram->nCsr; + u.by.pFrame->pc = pc; + u.by.pFrame->aMem = p->aMem; + u.by.pFrame->nMem = p->nMem; + u.by.pFrame->apCsr = p->apCsr; + u.by.pFrame->nCursor = p->nCursor; + u.by.pFrame->aOp = p->aOp; + u.by.pFrame->nOp = p->nOp; + u.by.pFrame->token = u.by.pProgram->token; - u.bz.pEnd = &VdbeFrameMem(u.bz.pFrame)[u.bz.pFrame->nChildMem]; - for(u.bz.pMem=VdbeFrameMem(u.bz.pFrame); u.bz.pMem!=u.bz.pEnd; u.bz.pMem++){ - u.bz.pMem->flags = MEM_Null; - u.bz.pMem->db = db; + u.by.pEnd = &VdbeFrameMem(u.by.pFrame)[u.by.pFrame->nChildMem]; + for(u.by.pMem=VdbeFrameMem(u.by.pFrame); u.by.pMem!=u.by.pEnd; u.by.pMem++){ + u.by.pMem->flags = MEM_Null; + u.by.pMem->db = db; } }else{ - u.bz.pFrame = u.bz.pRt->u.pFrame; - assert( u.bz.pProgram->nMem+u.bz.pProgram->nCsr==u.bz.pFrame->nChildMem ); - assert( u.bz.pProgram->nCsr==u.bz.pFrame->nChildCsr ); - assert( pc==u.bz.pFrame->pc ); + u.by.pFrame = u.by.pRt->u.pFrame; + assert( u.by.pProgram->nMem+u.by.pProgram->nCsr==u.by.pFrame->nChildMem ); + assert( u.by.pProgram->nCsr==u.by.pFrame->nChildCsr ); + assert( pc==u.by.pFrame->pc ); } p->nFrame++; - u.bz.pFrame->pParent = p->pFrame; - u.bz.pFrame->lastRowid = db->lastRowid; - u.bz.pFrame->nChange = p->nChange; + u.by.pFrame->pParent = p->pFrame; + u.by.pFrame->lastRowid = db->lastRowid; + u.by.pFrame->nChange = p->nChange; p->nChange = 0; - p->pFrame = u.bz.pFrame; - p->aMem = &VdbeFrameMem(u.bz.pFrame)[-1]; - p->nMem = u.bz.pFrame->nChildMem; - p->nCursor = (u16)u.bz.pFrame->nChildCsr; - p->apCsr = (VdbeCursor **)&p->aMem[p->nMem+1]; - p->aOp = u.bz.pProgram->aOp; - p->nOp = u.bz.pProgram->nOp; + p->pFrame = u.by.pFrame; + p->aMem = aMem = &VdbeFrameMem(u.by.pFrame)[-1]; + p->nMem = u.by.pFrame->nChildMem; + p->nCursor = (u16)u.by.pFrame->nChildCsr; + p->apCsr = (VdbeCursor **)&aMem[p->nMem+1]; + p->aOp = aOp = u.by.pProgram->aOp; + p->nOp = u.by.pProgram->nOp; pc = -1; break; @@ -56994,10 +57178,10 @@ case OP_Program: { /* jump */ /* Opcode: Param P1 P2 * * * ** -** This opcode is only ever present in sub-programs called via the -** OP_Program instruction. Copy a value currently stored in a memory -** cell of the calling (parent) frame to cell P2 in the current frames -** address space. This is used by trigger programs to access the new.* +** This opcode is only ever present in sub-programs called via the +** OP_Program instruction. Copy a value currently stored in a memory +** cell of the calling (parent) frame to cell P2 in the current frames +** address space. This is used by trigger programs to access the new.* ** and old.* values. ** ** The address of the cell in the parent frame is determined by adding @@ -57005,13 +57189,13 @@ case OP_Program: { /* jump */ ** calling OP_Program instruction. */ case OP_Param: { /* out2-prerelease */ -#if 0 /* local variables moved into u.ca */ +#if 0 /* local variables moved into u.bz */ VdbeFrame *pFrame; Mem *pIn; -#endif /* local variables moved into u.ca */ - u.ca.pFrame = p->pFrame; - u.ca.pIn = &u.ca.pFrame->aMem[pOp->p1 + u.ca.pFrame->aOp[u.ca.pFrame->pc].p1]; - sqlite3VdbeMemShallowCopy(pOut, u.ca.pIn, MEM_Ephem); +#endif /* local variables moved into u.bz */ + u.bz.pFrame = p->pFrame; + u.bz.pIn = &u.bz.pFrame->aMem[pOp->p1 + u.bz.pFrame->aOp[u.bz.pFrame->pc].p1]; + sqlite3VdbeMemShallowCopy(pOut, u.bz.pIn, MEM_Ephem); break; } @@ -57021,8 +57205,8 @@ case OP_Param: { /* out2-prerelease */ /* Opcode: FkCounter P1 P2 * * * ** ** Increment a "constraint counter" by P2 (P2 may be negative or positive). -** If P1 is non-zero, the database constraint counter is incremented -** (deferred foreign key constraints). Otherwise, if P1 is zero, the +** If P1 is non-zero, the database constraint counter is incremented +** (deferred foreign key constraints). Otherwise, if P1 is zero, the ** statement counter is incremented (immediate foreign key constraints). */ case OP_FkCounter: { @@ -57037,7 +57221,7 @@ case OP_FkCounter: { /* Opcode: FkIfZero P1 P2 * * * ** ** This opcode tests if a foreign key constraint-counter is currently zero. -** If so, jump to instruction P2. Otherwise, fall through to the next +** If so, jump to instruction P2. Otherwise, fall through to the next ** instruction. ** ** If P1 is non-zero, then the jump is taken if the database constraint-counter @@ -57060,27 +57244,28 @@ case OP_FkIfZero: { /* jump */ ** ** P1 is a register in the root frame of this VM (the root frame is ** different from the current frame if this instruction is being executed -** within a sub-program). Set the value of register P1 to the maximum of +** within a sub-program). Set the value of register P1 to the maximum of ** its current value and the value in register P2. ** ** This instruction throws an error if the memory cell is not initially ** an integer. */ case OP_MemMax: { /* in2 */ -#if 0 /* local variables moved into u.cb */ +#if 0 /* local variables moved into u.ca */ Mem *pIn1; VdbeFrame *pFrame; -#endif /* local variables moved into u.cb */ +#endif /* local variables moved into u.ca */ if( p->pFrame ){ - for(u.cb.pFrame=p->pFrame; u.cb.pFrame->pParent; u.cb.pFrame=u.cb.pFrame->pParent); - u.cb.pIn1 = &u.cb.pFrame->aMem[pOp->p1]; + for(u.ca.pFrame=p->pFrame; u.ca.pFrame->pParent; u.ca.pFrame=u.ca.pFrame->pParent); + u.ca.pIn1 = &u.ca.pFrame->aMem[pOp->p1]; }else{ - u.cb.pIn1 = &p->aMem[pOp->p1]; + u.ca.pIn1 = &aMem[pOp->p1]; } - sqlite3VdbeMemIntegerify(u.cb.pIn1); + sqlite3VdbeMemIntegerify(u.ca.pIn1); + pIn2 = &aMem[pOp->p2]; sqlite3VdbeMemIntegerify(pIn2); - if( u.cb.pIn1->u.iu.i){ - u.cb.pIn1->u.i = pIn2->u.i; + if( u.ca.pIn1->u.iu.i){ + u.ca.pIn1->u.i = pIn2->u.i; } break; } @@ -57094,6 +57279,7 @@ case OP_MemMax: { /* in2 */ ** not contain an integer. An assertion fault will result if you try. */ case OP_IfPos: { /* jump, in1 */ + pIn1 = &aMem[pOp->p1]; assert( pIn1->flags&MEM_Int ); if( pIn1->u.i>0 ){ pc = pOp->p2 - 1; @@ -57103,12 +57289,13 @@ case OP_IfPos: { /* jump, in1 */ /* Opcode: IfNeg P1 P2 * * * ** -** If the value of register P1 is less than zero, jump to P2. +** If the value of register P1 is less than zero, jump to P2. ** ** It is illegal to use this instruction on a register that does ** not contain an integer. An assertion fault will result if you try. */ case OP_IfNeg: { /* jump, in1 */ + pIn1 = &aMem[pOp->p1]; assert( pIn1->flags&MEM_Int ); if( pIn1->u.i<0 ){ pc = pOp->p2 - 1; @@ -57116,15 +57303,18 @@ case OP_IfNeg: { /* jump, in1 */ break; } -/* Opcode: IfZero P1 P2 * * * +/* Opcode: IfZero P1 P2 P3 * * ** -** If the value of register P1 is exactly 0, jump to P2. +** The register P1 must contain an integer. Add literal P3 to the +** value in register P1. If the result is exactly 0, jump to P2. ** ** It is illegal to use this instruction on a register that does ** not contain an integer. An assertion fault will result if you try. */ case OP_IfZero: { /* jump, in1 */ + pIn1 = &aMem[pOp->p1]; assert( pIn1->flags&MEM_Int ); + pIn1->u.i += pOp->p3; if( pIn1->u.i==0 ){ pc = pOp->p2 - 1; } @@ -57142,47 +57332,47 @@ case OP_IfZero: { /* jump, in1 */ ** successors. */ case OP_AggStep: { -#if 0 /* local variables moved into u.cc */ +#if 0 /* local variables moved into u.cb */ int n; int i; Mem *pMem; Mem *pRec; sqlite3_context ctx; sqlite3_value **apVal; -#endif /* local variables moved into u.cc */ +#endif /* local variables moved into u.cb */ - u.cc.n = pOp->p5; - assert( u.cc.n>=0 ); - u.cc.pRec = &p->aMem[pOp->p2]; - u.cc.apVal = p->apArg; - assert( u.cc.apVal || u.cc.n==0 ); - for(u.cc.i=0; u.cc.ip5; + assert( u.cb.n>=0 ); + u.cb.pRec = &aMem[pOp->p2]; + u.cb.apVal = p->apArg; + assert( u.cb.apVal || u.cb.n==0 ); + for(u.cb.i=0; u.cb.ip4.pFunc; + u.cb.ctx.pFunc = pOp->p4.pFunc; assert( pOp->p3>0 && pOp->p3<=p->nMem ); - u.cc.ctx.pMem = u.cc.pMem = &p->aMem[pOp->p3]; - u.cc.pMem->n++; - u.cc.ctx.s.flags = MEM_Null; - u.cc.ctx.s.z = 0; - u.cc.ctx.s.zMalloc = 0; - u.cc.ctx.s.xDel = 0; - u.cc.ctx.s.db = db; - u.cc.ctx.isError = 0; - u.cc.ctx.pColl = 0; - if( u.cc.ctx.pFunc->flags & SQLITE_FUNC_NEEDCOLL ){ + u.cb.ctx.pMem = u.cb.pMem = &aMem[pOp->p3]; + u.cb.pMem->n++; + u.cb.ctx.s.flags = MEM_Null; + u.cb.ctx.s.z = 0; + u.cb.ctx.s.zMalloc = 0; + u.cb.ctx.s.xDel = 0; + u.cb.ctx.s.db = db; + u.cb.ctx.isError = 0; + u.cb.ctx.pColl = 0; + if( u.cb.ctx.pFunc->flags & SQLITE_FUNC_NEEDCOLL ){ assert( pOp>p->aOp ); assert( pOp[-1].p4type==P4_COLLSEQ ); assert( pOp[-1].opcode==OP_CollSeq ); - u.cc.ctx.pColl = pOp[-1].p4.pColl; + u.cb.ctx.pColl = pOp[-1].p4.pColl; } - (u.cc.ctx.pFunc->xStep)(&u.cc.ctx, u.cc.n, u.cc.apVal); - if( u.cc.ctx.isError ){ - sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.cc.ctx.s)); - rc = u.cc.ctx.isError; + (u.cb.ctx.pFunc->xStep)(&u.cb.ctx, u.cb.n, u.cb.apVal); + if( u.cb.ctx.isError ){ + sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.cb.ctx.s)); + rc = u.cb.ctx.isError; } - sqlite3VdbeMemRelease(&u.cc.ctx.s); + sqlite3VdbeMemRelease(&u.cb.ctx.s); break; } @@ -57199,19 +57389,19 @@ case OP_AggStep: { ** the step function was not previously called. */ case OP_AggFinal: { -#if 0 /* local variables moved into u.cd */ +#if 0 /* local variables moved into u.cc */ Mem *pMem; -#endif /* local variables moved into u.cd */ +#endif /* local variables moved into u.cc */ assert( pOp->p1>0 && pOp->p1<=p->nMem ); - u.cd.pMem = &p->aMem[pOp->p1]; - assert( (u.cd.pMem->flags & ~(MEM_Null|MEM_Agg))==0 ); - rc = sqlite3VdbeMemFinalize(u.cd.pMem, pOp->p4.pFunc); + u.cc.pMem = &aMem[pOp->p1]; + assert( (u.cc.pMem->flags & ~(MEM_Null|MEM_Agg))==0 ); + rc = sqlite3VdbeMemFinalize(u.cc.pMem, pOp->p4.pFunc); if( rc ){ - sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(u.cd.pMem)); + sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(u.cc.pMem)); } - sqlite3VdbeChangeEncoding(u.cd.pMem, encoding); - UPDATE_MAX_BLOBSIZE(u.cd.pMem); - if( sqlite3VdbeMemTooBig(u.cd.pMem) ){ + sqlite3VdbeChangeEncoding(u.cc.pMem, encoding); + UPDATE_MAX_BLOBSIZE(u.cc.pMem); + if( sqlite3VdbeMemTooBig(u.cc.pMem) ){ goto too_big; } break; @@ -57226,7 +57416,7 @@ case OP_AggFinal: { ** a transaction. */ case OP_Vacuum: { - if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; + if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; rc = sqlite3RunVacuum(&p->zErrMsg, db); if( sqlite3SafetyOn(db) ) goto abort_due_to_misuse; break; @@ -57241,14 +57431,14 @@ case OP_Vacuum: { ** P2. Otherwise, fall through to the next instruction. */ case OP_IncrVacuum: { /* jump */ -#if 0 /* local variables moved into u.ce */ +#if 0 /* local variables moved into u.cd */ Btree *pBt; -#endif /* local variables moved into u.ce */ +#endif /* local variables moved into u.cd */ assert( pOp->p1>=0 && pOp->p1nDb ); assert( (p->btreeMask & (1<p1))!=0 ); - u.ce.pBt = db->aDb[pOp->p1].pBt; - rc = sqlite3BtreeIncrVacuum(u.ce.pBt); + u.cd.pBt = db->aDb[pOp->p1].pBt; + rc = sqlite3BtreeIncrVacuum(u.cd.pBt); if( rc==SQLITE_DONE ){ pc = pOp->p2 - 1; rc = SQLITE_OK; @@ -57260,11 +57450,11 @@ case OP_IncrVacuum: { /* jump */ /* Opcode: Expire P1 * * * * ** ** Cause precompiled statements to become expired. An expired statement -** fails with an error code of SQLITE_SCHEMA if it is ever executed +** fails with an error code of SQLITE_SCHEMA if it is ever executed ** (via sqlite3_step()). -** +** ** If P1 is 0, then all SQL statements become expired. If P1 is non-zero, -** then only the currently executing statement is affected. +** then only the currently executing statement is affected. */ case OP_Expire: { if( !pOp->p1 ){ @@ -57279,7 +57469,7 @@ case OP_Expire: { /* Opcode: TableLock P1 P2 P3 P4 * ** ** Obtain a lock on a particular table. This instruction is only used when -** the shared-cache feature is enabled. +** the shared-cache feature is enabled. ** ** P1 is the index of the database in sqlite3.aDb[] of the database ** on which the lock is acquired. A readlock is obtained if P3==0 or @@ -57293,7 +57483,7 @@ case OP_Expire: { case OP_TableLock: { u8 isWriteLock = (u8)pOp->p3; if( isWriteLock || 0==(db->flags&SQLITE_ReadUncommitted) ){ - int p1 = pOp->p1; + int p1 = pOp->p1; assert( p1>=0 && p1nDb ); assert( (p->btreeMask & (1<p4.pVtab; - rc = sqlite3VtabBegin(db, u.cf.pVTab); - if( u.cf.pVTab ){ +#endif /* local variables moved into u.ce */ + u.ce.pVTab = pOp->p4.pVtab; + rc = sqlite3VtabBegin(db, u.ce.pVTab); + if( u.ce.pVTab ){ sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.cf.pVTab->pVtab->zErrMsg; - u.cf.pVTab->pVtab->zErrMsg = 0; + p->zErrMsg = u.ce.pVTab->pVtab->zErrMsg; + u.ce.pVTab->pVtab->zErrMsg = 0; } break; } @@ -57366,36 +57556,36 @@ case OP_VDestroy: { ** table and stores that cursor in P1. */ case OP_VOpen: { -#if 0 /* local variables moved into u.cg */ +#if 0 /* local variables moved into u.cf */ VdbeCursor *pCur; sqlite3_vtab_cursor *pVtabCursor; sqlite3_vtab *pVtab; sqlite3_module *pModule; -#endif /* local variables moved into u.cg */ +#endif /* local variables moved into u.cf */ - u.cg.pCur = 0; - u.cg.pVtabCursor = 0; - u.cg.pVtab = pOp->p4.pVtab->pVtab; - u.cg.pModule = (sqlite3_module *)u.cg.pVtab->pModule; - assert(u.cg.pVtab && u.cg.pModule); + u.cf.pCur = 0; + u.cf.pVtabCursor = 0; + u.cf.pVtab = pOp->p4.pVtab->pVtab; + u.cf.pModule = (sqlite3_module *)u.cf.pVtab->pModule; + assert(u.cf.pVtab && u.cf.pModule); if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; - rc = u.cg.pModule->xOpen(u.cg.pVtab, &u.cg.pVtabCursor); + rc = u.cf.pModule->xOpen(u.cf.pVtab, &u.cf.pVtabCursor); sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.cg.pVtab->zErrMsg; - u.cg.pVtab->zErrMsg = 0; + p->zErrMsg = u.cf.pVtab->zErrMsg; + u.cf.pVtab->zErrMsg = 0; if( sqlite3SafetyOn(db) ) goto abort_due_to_misuse; if( SQLITE_OK==rc ){ /* Initialize sqlite3_vtab_cursor base class */ - u.cg.pVtabCursor->pVtab = u.cg.pVtab; + u.cf.pVtabCursor->pVtab = u.cf.pVtab; /* Initialise vdbe cursor object */ - u.cg.pCur = allocateCursor(p, pOp->p1, 0, -1, 0); - if( u.cg.pCur ){ - u.cg.pCur->pVtabCursor = u.cg.pVtabCursor; - u.cg.pCur->pModule = u.cg.pVtabCursor->pVtab->pModule; + u.cf.pCur = allocateCursor(p, pOp->p1, 0, -1, 0); + if( u.cf.pCur ){ + u.cf.pCur->pVtabCursor = u.cf.pVtabCursor; + u.cf.pCur->pModule = u.cf.pVtabCursor->pVtab->pModule; }else{ db->mallocFailed = 1; - u.cg.pModule->xClose(u.cg.pVtabCursor); + u.cf.pModule->xClose(u.cf.pVtabCursor); } } break; @@ -57422,7 +57612,7 @@ case OP_VOpen: { ** A jump is made to P2 if the result set after filtering would be empty. */ case OP_VFilter: { /* jump */ -#if 0 /* local variables moved into u.ch */ +#if 0 /* local variables moved into u.cg */ int nArg; int iQuery; const sqlite3_module *pModule; @@ -57434,48 +57624,48 @@ case OP_VFilter: { /* jump */ int res; int i; Mem **apArg; -#endif /* local variables moved into u.ch */ +#endif /* local variables moved into u.cg */ - u.ch.pQuery = &p->aMem[pOp->p3]; - u.ch.pArgc = &u.ch.pQuery[1]; - u.ch.pCur = p->apCsr[pOp->p1]; - REGISTER_TRACE(pOp->p3, u.ch.pQuery); - assert( u.ch.pCur->pVtabCursor ); - u.ch.pVtabCursor = u.ch.pCur->pVtabCursor; - u.ch.pVtab = u.ch.pVtabCursor->pVtab; - u.ch.pModule = u.ch.pVtab->pModule; + u.cg.pQuery = &aMem[pOp->p3]; + u.cg.pArgc = &u.cg.pQuery[1]; + u.cg.pCur = p->apCsr[pOp->p1]; + REGISTER_TRACE(pOp->p3, u.cg.pQuery); + assert( u.cg.pCur->pVtabCursor ); + u.cg.pVtabCursor = u.cg.pCur->pVtabCursor; + u.cg.pVtab = u.cg.pVtabCursor->pVtab; + u.cg.pModule = u.cg.pVtab->pModule; /* Grab the index number and argc parameters */ - assert( (u.ch.pQuery->flags&MEM_Int)!=0 && u.ch.pArgc->flags==MEM_Int ); - u.ch.nArg = (int)u.ch.pArgc->u.i; - u.ch.iQuery = (int)u.ch.pQuery->u.i; + assert( (u.cg.pQuery->flags&MEM_Int)!=0 && u.cg.pArgc->flags==MEM_Int ); + u.cg.nArg = (int)u.cg.pArgc->u.i; + u.cg.iQuery = (int)u.cg.pQuery->u.i; /* Invoke the xFilter method */ { - u.ch.res = 0; - u.ch.apArg = p->apArg; - for(u.ch.i = 0; u.ch.iapArg; + for(u.cg.i = 0; u.cg.iinVtabMethod = 1; - rc = u.ch.pModule->xFilter(u.ch.pVtabCursor, u.ch.iQuery, pOp->p4.z, u.ch.nArg, u.ch.apArg); + rc = u.cg.pModule->xFilter(u.cg.pVtabCursor, u.cg.iQuery, pOp->p4.z, u.cg.nArg, u.cg.apArg); p->inVtabMethod = 0; sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.ch.pVtab->zErrMsg; - u.ch.pVtab->zErrMsg = 0; + p->zErrMsg = u.cg.pVtab->zErrMsg; + u.cg.pVtab->zErrMsg = 0; if( rc==SQLITE_OK ){ - u.ch.res = u.ch.pModule->xEof(u.ch.pVtabCursor); + u.cg.res = u.cg.pModule->xEof(u.cg.pVtabCursor); } if( sqlite3SafetyOn(db) ) goto abort_due_to_misuse; - if( u.ch.res ){ + if( u.cg.res ){ pc = pOp->p2 - 1; } } - u.ch.pCur->nullRow = 0; + u.cg.pCur->nullRow = 0; break; } @@ -57485,60 +57675,60 @@ case OP_VFilter: { /* jump */ /* Opcode: VColumn P1 P2 P3 * * ** ** Store the value of the P2-th column of -** the row of the virtual-table that the +** the row of the virtual-table that the ** P1 cursor is pointing to into register P3. */ case OP_VColumn: { -#if 0 /* local variables moved into u.ci */ +#if 0 /* local variables moved into u.ch */ sqlite3_vtab *pVtab; const sqlite3_module *pModule; Mem *pDest; sqlite3_context sContext; -#endif /* local variables moved into u.ci */ +#endif /* local variables moved into u.ch */ VdbeCursor *pCur = p->apCsr[pOp->p1]; assert( pCur->pVtabCursor ); assert( pOp->p3>0 && pOp->p3<=p->nMem ); - u.ci.pDest = &p->aMem[pOp->p3]; + u.ch.pDest = &aMem[pOp->p3]; if( pCur->nullRow ){ - sqlite3VdbeMemSetNull(u.ci.pDest); + sqlite3VdbeMemSetNull(u.ch.pDest); break; } - u.ci.pVtab = pCur->pVtabCursor->pVtab; - u.ci.pModule = u.ci.pVtab->pModule; - assert( u.ci.pModule->xColumn ); - memset(&u.ci.sContext, 0, sizeof(u.ci.sContext)); + u.ch.pVtab = pCur->pVtabCursor->pVtab; + u.ch.pModule = u.ch.pVtab->pModule; + assert( u.ch.pModule->xColumn ); + memset(&u.ch.sContext, 0, sizeof(u.ch.sContext)); /* The output cell may already have a buffer allocated. Move - ** the current contents to u.ci.sContext.s so in case the user-function + ** the current contents to u.ch.sContext.s so in case the user-function ** can use the already allocated buffer instead of allocating a ** new one. */ - sqlite3VdbeMemMove(&u.ci.sContext.s, u.ci.pDest); - MemSetTypeFlag(&u.ci.sContext.s, MEM_Null); + sqlite3VdbeMemMove(&u.ch.sContext.s, u.ch.pDest); + MemSetTypeFlag(&u.ch.sContext.s, MEM_Null); if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; - rc = u.ci.pModule->xColumn(pCur->pVtabCursor, &u.ci.sContext, pOp->p2); + rc = u.ch.pModule->xColumn(pCur->pVtabCursor, &u.ch.sContext, pOp->p2); sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.ci.pVtab->zErrMsg; - u.ci.pVtab->zErrMsg = 0; - if( u.ci.sContext.isError ){ - rc = u.ci.sContext.isError; + p->zErrMsg = u.ch.pVtab->zErrMsg; + u.ch.pVtab->zErrMsg = 0; + if( u.ch.sContext.isError ){ + rc = u.ch.sContext.isError; } /* Copy the result of the function to the P3 register. We ** do this regardless of whether or not an error occurred to ensure any - ** dynamic allocation in u.ci.sContext.s (a Mem struct) is released. + ** dynamic allocation in u.ch.sContext.s (a Mem struct) is released. */ - sqlite3VdbeChangeEncoding(&u.ci.sContext.s, encoding); - REGISTER_TRACE(pOp->p3, u.ci.pDest); - sqlite3VdbeMemMove(u.ci.pDest, &u.ci.sContext.s); - UPDATE_MAX_BLOBSIZE(u.ci.pDest); + sqlite3VdbeChangeEncoding(&u.ch.sContext.s, encoding); + sqlite3VdbeMemMove(u.ch.pDest, &u.ch.sContext.s); + REGISTER_TRACE(pOp->p3, u.ch.pDest); + UPDATE_MAX_BLOBSIZE(u.ch.pDest); if( sqlite3SafetyOn(db) ){ goto abort_due_to_misuse; } - if( sqlite3VdbeMemTooBig(u.ci.pDest) ){ + if( sqlite3VdbeMemTooBig(u.ch.pDest) ){ goto too_big; } break; @@ -57553,22 +57743,22 @@ case OP_VColumn: { ** the end of its result set, then fall through to the next instruction. */ case OP_VNext: { /* jump */ -#if 0 /* local variables moved into u.cj */ +#if 0 /* local variables moved into u.ci */ sqlite3_vtab *pVtab; const sqlite3_module *pModule; int res; VdbeCursor *pCur; -#endif /* local variables moved into u.cj */ +#endif /* local variables moved into u.ci */ - u.cj.res = 0; - u.cj.pCur = p->apCsr[pOp->p1]; - assert( u.cj.pCur->pVtabCursor ); - if( u.cj.pCur->nullRow ){ + u.ci.res = 0; + u.ci.pCur = p->apCsr[pOp->p1]; + assert( u.ci.pCur->pVtabCursor ); + if( u.ci.pCur->nullRow ){ break; } - u.cj.pVtab = u.cj.pCur->pVtabCursor->pVtab; - u.cj.pModule = u.cj.pVtab->pModule; - assert( u.cj.pModule->xNext ); + u.ci.pVtab = u.ci.pCur->pVtabCursor->pVtab; + u.ci.pModule = u.ci.pVtab->pModule; + assert( u.ci.pModule->xNext ); /* Invoke the xNext() method of the module. There is no way for the ** underlying implementation to return an error if one occurs during @@ -57578,17 +57768,17 @@ case OP_VNext: { /* jump */ */ if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; p->inVtabMethod = 1; - rc = u.cj.pModule->xNext(u.cj.pCur->pVtabCursor); + rc = u.ci.pModule->xNext(u.ci.pCur->pVtabCursor); p->inVtabMethod = 0; sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.cj.pVtab->zErrMsg; - u.cj.pVtab->zErrMsg = 0; + p->zErrMsg = u.ci.pVtab->zErrMsg; + u.ci.pVtab->zErrMsg = 0; if( rc==SQLITE_OK ){ - u.cj.res = u.cj.pModule->xEof(u.cj.pCur->pVtabCursor); + u.ci.res = u.ci.pModule->xEof(u.ci.pCur->pVtabCursor); } if( sqlite3SafetyOn(db) ) goto abort_due_to_misuse; - if( !u.cj.res ){ + if( !u.ci.res ){ /* If there is data, jump to P2 */ pc = pOp->p2 - 1; } @@ -57604,21 +57794,21 @@ case OP_VNext: { /* jump */ ** in register P1 is passed as the zName argument to the xRename method. */ case OP_VRename: { -#if 0 /* local variables moved into u.ck */ +#if 0 /* local variables moved into u.cj */ sqlite3_vtab *pVtab; Mem *pName; -#endif /* local variables moved into u.ck */ +#endif /* local variables moved into u.cj */ - u.ck.pVtab = pOp->p4.pVtab->pVtab; - u.ck.pName = &p->aMem[pOp->p1]; - assert( u.ck.pVtab->pModule->xRename ); - REGISTER_TRACE(pOp->p1, u.ck.pName); - assert( u.ck.pName->flags & MEM_Str ); + u.cj.pVtab = pOp->p4.pVtab->pVtab; + u.cj.pName = &aMem[pOp->p1]; + assert( u.cj.pVtab->pModule->xRename ); + REGISTER_TRACE(pOp->p1, u.cj.pName); + assert( u.cj.pName->flags & MEM_Str ); if( sqlite3SafetyOff(db) ) goto abort_due_to_misuse; - rc = u.ck.pVtab->pModule->xRename(u.ck.pVtab, u.ck.pName->z); + rc = u.cj.pVtab->pModule->xRename(u.cj.pVtab, u.cj.pName->z); sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.ck.pVtab->zErrMsg; - u.ck.pVtab->zErrMsg = 0; + p->zErrMsg = u.cj.pVtab->zErrMsg; + u.cj.pVtab->zErrMsg = 0; if( sqlite3SafetyOn(db) ) goto abort_due_to_misuse; break; @@ -57630,27 +57820,27 @@ case OP_VRename: { ** ** P4 is a pointer to a virtual table object, an sqlite3_vtab structure. ** This opcode invokes the corresponding xUpdate method. P2 values -** are contiguous memory cells starting at P3 to pass to the xUpdate -** invocation. The value in register (P3+P2-1) corresponds to the +** are contiguous memory cells starting at P3 to pass to the xUpdate +** invocation. The value in register (P3+P2-1) corresponds to the ** p2th element of the argv array passed to xUpdate. ** ** The xUpdate method will do a DELETE or an INSERT or both. ** The argv[0] element (which corresponds to memory cell P3) -** is the rowid of a row to delete. If argv[0] is NULL then no -** deletion occurs. The argv[1] element is the rowid of the new -** row. This can be NULL to have the virtual table select the new -** rowid for itself. The subsequent elements in the array are +** is the rowid of a row to delete. If argv[0] is NULL then no +** deletion occurs. The argv[1] element is the rowid of the new +** row. This can be NULL to have the virtual table select the new +** rowid for itself. The subsequent elements in the array are ** the values of columns in the new row. ** ** If P2==1 then no insert is performed. argv[0] is the rowid of ** a row to delete. ** ** P1 is a boolean flag. If it is set to true and the xUpdate call -** is successful, then the value returned by sqlite3_last_insert_rowid() +** is successful, then the value returned by sqlite3_last_insert_rowid() ** is set to the value of the rowid for the row just inserted. */ case OP_VUpdate: { -#if 0 /* local variables moved into u.cl */ +#if 0 /* local variables moved into u.ck */ sqlite3_vtab *pVtab; sqlite3_module *pModule; int nArg; @@ -57658,29 +57848,29 @@ case OP_VUpdate: { sqlite_int64 rowid; Mem **apArg; Mem *pX; -#endif /* local variables moved into u.cl */ +#endif /* local variables moved into u.ck */ - u.cl.pVtab = pOp->p4.pVtab->pVtab; - u.cl.pModule = (sqlite3_module *)u.cl.pVtab->pModule; - u.cl.nArg = pOp->p2; + u.ck.pVtab = pOp->p4.pVtab->pVtab; + u.ck.pModule = (sqlite3_module *)u.ck.pVtab->pModule; + u.ck.nArg = pOp->p2; assert( pOp->p4type==P4_VTAB ); - if( ALWAYS(u.cl.pModule->xUpdate) ){ - u.cl.apArg = p->apArg; - u.cl.pX = &p->aMem[pOp->p3]; - for(u.cl.i=0; u.cl.ixUpdate) ){ + u.ck.apArg = p->apArg; + u.ck.pX = &aMem[pOp->p3]; + for(u.ck.i=0; u.ck.ixUpdate(u.cl.pVtab, u.cl.nArg, u.cl.apArg, &u.cl.rowid); + rc = u.ck.pModule->xUpdate(u.ck.pVtab, u.ck.nArg, u.ck.apArg, &u.ck.rowid); sqlite3DbFree(db, p->zErrMsg); - p->zErrMsg = u.cl.pVtab->zErrMsg; - u.cl.pVtab->zErrMsg = 0; + p->zErrMsg = u.ck.pVtab->zErrMsg; + u.ck.pVtab->zErrMsg = 0; if( sqlite3SafetyOn(db) ) goto abort_due_to_misuse; if( rc==SQLITE_OK && pOp->p1 ){ - assert( u.cl.nArg>1 && u.cl.apArg[0] && (u.cl.apArg[0]->flags&MEM_Null) ); - db->lastRowid = u.cl.rowid; + assert( u.ck.nArg>1 && u.ck.apArg[0] && (u.ck.apArg[0]->flags&MEM_Null) ); + db->lastRowid = u.ck.rowid; } p->nChange++; } @@ -57694,21 +57884,20 @@ case OP_VUpdate: { ** Write the current number of pages in database P1 to memory cell P2. */ case OP_Pagecount: { /* out2-prerelease */ -#if 0 /* local variables moved into u.cm */ +#if 0 /* local variables moved into u.cl */ int p1; int nPage; Pager *pPager; -#endif /* local variables moved into u.cm */ +#endif /* local variables moved into u.cl */ - u.cm.p1 = pOp->p1; - u.cm.pPager = sqlite3BtreePager(db->aDb[u.cm.p1].pBt); - rc = sqlite3PagerPagecount(u.cm.pPager, &u.cm.nPage); + u.cl.p1 = pOp->p1; + u.cl.pPager = sqlite3BtreePager(db->aDb[u.cl.p1].pBt); + rc = sqlite3PagerPagecount(u.cl.pPager, &u.cl.nPage); /* OP_Pagecount is always called from within a read transaction. The ** page count has already been successfully read and cached. So the ** sqlite3PagerPagecount() call above cannot fail. */ if( ALWAYS(rc==SQLITE_OK) ){ - pOut->flags = MEM_Int; - pOut->u.i = u.cm.nPage; + pOut->u.i = u.cl.nPage; } break; } @@ -57721,18 +57910,20 @@ case OP_Pagecount: { /* out2-prerelease */ ** the UTF-8 string contained in P4 is emitted on the trace callback. */ case OP_Trace: { -#if 0 /* local variables moved into u.cn */ +#if 0 /* local variables moved into u.cm */ char *zTrace; -#endif /* local variables moved into u.cn */ +#endif /* local variables moved into u.cm */ - u.cn.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql); - if( u.cn.zTrace ){ + u.cm.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql); + if( u.cm.zTrace ){ if( db->xTrace ){ - db->xTrace(db->pTraceArg, u.cn.zTrace); + char *z = sqlite3VdbeExpandSql(p, u.cm.zTrace); + db->xTrace(db->pTraceArg, z); + sqlite3DbFree(db, z); } #ifdef SQLITE_DEBUG if( (db->flags & SQLITE_SqlTrace)!=0 ){ - sqlite3DebugPrintf("SQL-trace: %s\n", u.cn.zTrace); + sqlite3DebugPrintf("SQL-trace: %s\n", u.cm.zTrace); } #endif /* SQLITE_DEBUG */ } @@ -57771,7 +57962,7 @@ default: { /* This is really OP_Noop and OP_Explain */ pOp->cnt++; #if 0 fprintf(stdout, "%10llu ", elapsed); - sqlite3VdbePrintOp(stdout, origPc, &p->aOp[origPc]); + sqlite3VdbePrintOp(stdout, origPc, &aOp[origPc]); #endif } #endif @@ -57787,11 +57978,11 @@ default: { /* This is really OP_Noop and OP_Explain */ #ifdef SQLITE_DEBUG if( p->trace ){ if( rc!=0 ) fprintf(p->trace,"rc=%d\n",rc); - if( opProperty & OPFLG_OUT2_PRERELEASE ){ - registerTrace(p->trace, pOp->p2, pOut); + if( pOp->opflags & (OPFLG_OUT2_PRERELEASE|OPFLG_OUT2) ){ + registerTrace(p->trace, pOp->p2, &aMem[pOp->p2]); } - if( opProperty & OPFLG_OUT3 ){ - registerTrace(p->trace, pOp->p3, pOut); + if( pOp->opflags & OPFLG_OUT3 ){ + registerTrace(p->trace, pOp->p3, &aMem[pOp->p3]); } } #endif /* SQLITE_DEBUG */ @@ -57807,6 +57998,7 @@ vdbe_error_halt: sqlite3VdbeHalt(p); if( rc==SQLITE_IOERR_NOMEM ) db->mallocFailed = 1; rc = SQLITE_ERROR; + if( resetSchemaOnFault ) sqlite3ResetInternalSchema(db, 0); /* This is the only way out of this procedure. We have to ** release the mutexes on btrees that were acquired at the @@ -57874,8 +58066,6 @@ abort_due_to_interrupt: ************************************************************************* ** ** This file contains code used to implement incremental BLOB I/O. -** -** $Id: vdbeblob.c,v 1.35 2009/07/02 07:47:33 danielk1977 Exp $ */ @@ -57909,7 +58099,7 @@ SQLITE_API int sqlite3_blob_open( int nAttempt = 0; int iCol; /* Index of zColumn in row-record */ - /* This VDBE program seeks a btree cursor to the identified + /* This VDBE program seeks a btree cursor to the identified ** db/table/row entry. The reason for using a vdbe program instead ** of writing code to use the b-tree layer directly is that the ** vdbe program will take advantage of the various transaction, @@ -57917,11 +58107,11 @@ SQLITE_API int sqlite3_blob_open( ** ** After seeking the cursor, the vdbe executes an OP_ResultRow. ** Code external to the Vdbe then "borrows" the b-tree cursor and - ** uses it to implement the blob_read(), blob_write() and + ** uses it to implement the blob_read(), blob_write() and ** blob_bytes() functions. ** ** The sqlite3_blob_close() function finalizes the vdbe program, - ** which closes the b-tree cursor and (possibly) commits the + ** which closes the b-tree cursor and (possibly) commits the ** transaction. */ static const VdbeOpList openBlob[] = { @@ -58005,7 +58195,7 @@ SQLITE_API int sqlite3_blob_open( } /* If the value is being opened for writing, check that the - ** column is not indexed, and that it is not part of a foreign key. + ** column is not indexed, and that it is not part of a foreign key. ** It is against the rules to open a column to which either of these ** descriptions applies for writing. */ if( flags ){ @@ -58015,7 +58205,7 @@ SQLITE_API int sqlite3_blob_open( if( db->flags&SQLITE_ForeignKeys ){ /* Check that the column is not part of an FK child key definition. It ** is not necessary to check if it is part of a parent key, as parent - ** key columns must be indexed. The check below will pick up this + ** key columns must be indexed. The check below will pick up this ** case. */ FKey *pFKey; for(pFKey=pTab->pFKey; pFKey; pFKey=pFKey->pNextFrom){ @@ -58061,7 +58251,7 @@ SQLITE_API int sqlite3_blob_open( sqlite3VdbeChangeP2(v, 1, pTab->pSchema->schema_cookie); /* Make sure a mutex is held on the table to be accessed */ - sqlite3VdbeUsesBtree(v, iDb); + sqlite3VdbeUsesBtree(v, iDb); /* Configure the OP_TableLock instruction */ sqlite3VdbeChangeP1(v, 2, iDb); @@ -58069,7 +58259,7 @@ SQLITE_API int sqlite3_blob_open( sqlite3VdbeChangeP3(v, 2, flags); sqlite3VdbeChangeP4(v, 2, pTab->zName, P4_TRANSIENT); - /* Remove either the OP_OpenWrite or OpenRead. Set the P2 + /* Remove either the OP_OpenWrite or OpenRead. Set the P2 ** parameter of the other to pTab->tnum. */ sqlite3VdbeChangeToNoop(v, 4 - flags, 1); sqlite3VdbeChangeP2(v, 3 + flags, pTab->tnum); @@ -58079,7 +58269,7 @@ SQLITE_API int sqlite3_blob_open( ** think that the table has one more column than it really ** does. An OP_Column to retrieve this imaginary column will ** always return an SQL NULL. This is useful because it means - ** we can invoke OP_Column to fill in the vdbe cursors type + ** we can invoke OP_Column to fill in the vdbe cursors type ** and offset cache without causing any IO. */ sqlite3VdbeChangeP4(v, 3+flags, SQLITE_INT_TO_PTR(pTab->nCol+1),P4_INT32); @@ -58088,7 +58278,7 @@ SQLITE_API int sqlite3_blob_open( sqlite3VdbeMakeReady(v, 1, 1, 1, 0, 0, 0); } } - + sqlite3BtreeLeaveAll(db); rc = sqlite3SafetyOff(db); if( NEVER(rc!=SQLITE_OK) || db->mallocFailed ){ @@ -58181,10 +58371,10 @@ SQLITE_API int sqlite3_blob_close(sqlite3_blob *pBlob){ ** Perform a read or write operation on a blob */ static int blobReadWrite( - sqlite3_blob *pBlob, - void *z, - int n, - int iOffset, + sqlite3_blob *pBlob, + void *z, + int n, + int iOffset, int (*xCall)(BtCursor*, u32, u32, void*) ){ int rc; @@ -58268,12 +58458,6 @@ SQLITE_API int sqlite3_blob_bytes(sqlite3_blob *pBlob){ ** ************************************************************************* ** -** @(#) $Id: journal.c,v 1.9 2009/01/20 17:06:27 danielk1977 Exp $ -*/ - -#ifdef SQLITE_ENABLE_ATOMIC_WRITE - -/* ** This file implements a special kind of sqlite3_file object used ** by SQLite to create journal files if the atomic-write optimization ** is enabled. @@ -58284,11 +58468,11 @@ SQLITE_API int sqlite3_blob_bytes(sqlite3_blob *pBlob){ ** be used to service read() and write() requests. The actual file ** on disk is not created or populated until either: ** -** 1) The in-memory representation grows too large for the allocated +** 1) The in-memory representation grows too large for the allocated ** buffer, or ** 2) The sqlite3JournalCreate() function is called. */ - +#ifdef SQLITE_ENABLE_ATOMIC_WRITE /* @@ -58308,7 +58492,7 @@ struct JournalFile { typedef struct JournalFile JournalFile; /* -** If it does not already exists, create and populate the on-disk file +** If it does not already exists, create and populate the on-disk file ** for JournalFile p. */ static int createFile(JournalFile *p){ @@ -58448,7 +58632,7 @@ static struct sqlite3_io_methods JournalFileMethods = { 0 /* xDeviceCharacteristics */ }; -/* +/* ** Open a journal file. */ SQLITE_PRIVATE int sqlite3JournalOpen( @@ -58487,7 +58671,7 @@ SQLITE_PRIVATE int sqlite3JournalCreate(sqlite3_file *p){ return createFile((JournalFile *)p); } -/* +/* ** Return the number of bytes required to store a JournalFile that uses vfs ** pVfs to create the underlying on-disk files. */ @@ -58513,8 +58697,6 @@ SQLITE_PRIVATE int sqlite3JournalSize(sqlite3_vfs *pVfs){ ** This file contains code use to implement an in-memory rollback journal. ** The in-memory rollback journal is used to journal transactions for ** ":memory:" databases and when the journal_mode=MEMORY pragma is used. -** -** @(#) $Id: memjournal.c,v 1.12 2009/05/04 11:42:30 danielk1977 Exp $ */ /* Forward references to internal structures */ @@ -58587,7 +58769,7 @@ static int memjrnlRead( if( p->readpoint.iOffset!=iOfst || iOfst==0 ){ sqlite3_int64 iOff = 0; - for(pChunk=p->pFirst; + for(pChunk=p->pFirst; ALWAYS(pChunk) && (iOff+JOURNAL_CHUNKSIZE)<=iOfst; pChunk=pChunk->pNext ){ @@ -58731,7 +58913,7 @@ static struct sqlite3_io_methods MemJournalMethods = { 0 /* xDeviceCharacteristics */ }; -/* +/* ** Open a journal file. */ SQLITE_PRIVATE void sqlite3MemJournalOpen(sqlite3_file *pJfd){ @@ -58742,14 +58924,14 @@ SQLITE_PRIVATE void sqlite3MemJournalOpen(sqlite3_file *pJfd){ } /* -** Return true if the file-handle passed as an argument is -** an in-memory journal +** Return true if the file-handle passed as an argument is +** an in-memory journal */ SQLITE_PRIVATE int sqlite3IsMemJournal(sqlite3_file *pJfd){ return pJfd->pMethods==&MemJournalMethods; } -/* +/* ** Return the number of bytes required to store a MemJournal that uses vfs ** pVfs to create the underlying on-disk files. */ @@ -58772,8 +58954,6 @@ SQLITE_PRIVATE int sqlite3MemJournalSize(void){ ************************************************************************* ** This file contains routines used for walking the parser tree for ** an SQL statement. -** -** $Id: walker.c,v 1.7 2009/06/15 23:15:59 drh Exp $ */ @@ -58851,7 +59031,7 @@ SQLITE_PRIVATE int sqlite3WalkSelectExpr(Walker *pWalker, Select *p){ ** Walk the parse trees associated with all subqueries in the ** FROM clause of SELECT statement p. Do not invoke the select ** callback on p, but do invoke it on each FROM clause subquery -** and on any subqueries further down in the tree. Return +** and on any subqueries further down in the tree. Return ** WRC_Abort or WRC_Continue; */ SQLITE_PRIVATE int sqlite3WalkSelectFrom(Walker *pWalker, Select *p){ @@ -58868,7 +59048,7 @@ SQLITE_PRIVATE int sqlite3WalkSelectFrom(Walker *pWalker, Select *p){ } } return WRC_Continue; -} +} /* ** Call sqlite3WalkExpr() for every expression in Select statement p. @@ -58912,8 +59092,6 @@ SQLITE_PRIVATE int sqlite3WalkSelect(Walker *pWalker, Select *p){ ** This file contains routines used for walking the parser tree and ** resolve all identifiers by associating them with a particular ** table and column. -** -** $Id: resolve.c,v 1.30 2009/06/15 23:15:59 drh Exp $ */ /* @@ -58928,7 +59106,7 @@ SQLITE_PRIVATE int sqlite3WalkSelect(Walker *pWalker, Select *p){ ** ** The reason for suppressing the TK_AS term when the expression is a simple ** column reference is so that the column reference will be recognized as -** usable by indices within the WHERE clause processing logic. +** usable by indices within the WHERE clause processing logic. ** ** Hack: The TK_AS operator is inhibited if zType[0]=='G'. This means ** that in a GROUP BY clause, the expression is evaluated twice. Hence: @@ -58985,14 +59163,20 @@ static void resolveAlias( pDup->pColl = pExpr->pColl; pDup->flags |= EP_ExpCollate; } - sqlite3ExprClear(db, pExpr); + + /* Before calling sqlite3ExprDelete(), set the EP_Static flag. This + ** prevents ExprDelete() from deleting the Expr structure itself, + ** allowing it to be repopulated by the memcpy() on the following line. + */ + ExprSetProperty(pExpr, EP_Static); + sqlite3ExprDelete(db, pExpr); memcpy(pExpr, pDup, sizeof(*pExpr)); sqlite3DbFree(db, pDup); } /* ** Given the name of a column of the form X.Y.Z or Y.Z or just Z, look up -** that name in the set of source tables in pSrcList and make the pExpr +** that name in the set of source tables in pSrcList and make the pExpr ** expression node refer back to that source column. The following changes ** are made to pExpr: ** @@ -59054,7 +59238,7 @@ static int lookupName( Table *pTab; int iDb; Column *pCol; - + pTab = pItem->pTab; assert( pTab!=0 && pTab->zName!=0 ); iDb = sqlite3SchemaToIndex(db, pTab->pSchema); @@ -59116,7 +59300,7 @@ static int lookupName( } #ifndef SQLITE_OMIT_TRIGGER - /* If we have not already resolved the name, then maybe + /* If we have not already resolved the name, then maybe ** it is a new.* or old.* trigger argument reference */ if( zDb==0 && zTab!=0 && cnt==0 && pParse->pTriggerTab!=0 ){ @@ -59131,7 +59315,7 @@ static int lookupName( pTab = pParse->pTriggerTab; } - if( pTab ){ + if( pTab ){ int iCol; pSchema = pTab->pSchema; cntTab++; @@ -59156,6 +59340,10 @@ static int lookupName( testcase( iCol==31 ); testcase( iCol==32 ); pParse->oldmask |= (iCol>=32 ? 0xffffffff : (((u32)1)<newmask |= (iCol>=32 ? 0xffffffff : (((u32)1)<iColumn = (i16)iCol; pExpr->pTab = pTab; @@ -59205,7 +59393,7 @@ static int lookupName( assert( zTab==0 && zDb==0 ); goto lookupname_end; } - } + } } /* Advance to the next name context. The loop will exit when either @@ -59353,7 +59541,7 @@ static int resolveExprStep(Walker *pWalker, Expr *pExpr){ SrcList *pSrcList = pNC->pSrcList; struct SrcList_item *pItem; assert( pSrcList && pSrcList->nSrc==1 ); - pItem = pSrcList->a; + pItem = pSrcList->a; pExpr->op = TK_COLUMN; pExpr->pTab = pItem->pTab; pExpr->iTable = pItem->iCursor; @@ -59368,7 +59556,7 @@ static int resolveExprStep(Walker *pWalker, Expr *pExpr){ case TK_ID: { return lookupName(pParse, 0, 0, pExpr->u.zToken, pNC, pExpr); } - + /* A table name and column name: ID.ID ** Or a database, table and column: ID.ID.ID */ @@ -59457,7 +59645,7 @@ static int resolveExprStep(Walker *pWalker, Expr *pExpr){ sqlite3WalkExprList(pWalker, pList); if( is_agg ) pNC->allowAgg = 1; /* FIX ME: Compute pExpr->affinity based on the expected return - ** type of the function + ** type of the function */ return WRC_Prune; } @@ -59593,7 +59781,7 @@ static void resolveOutOfRangeError( int i, /* The index (1-based) of the term out of range */ int mx /* Largest permissible value of i */ ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "%r %s BY term out of range - should be " "between 1 and %d", i, zType, mx); } @@ -59814,7 +60002,7 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ ExprList *pGroupBy; /* The GROUP BY clause */ Select *pLeftmost; /* Left-most of SELECT of a compound */ sqlite3 *db; /* Database connection */ - + assert( p!=0 ); if( p->selFlags & SF_Resolved ){ @@ -59854,14 +60042,14 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ sqlite3ResolveExprNames(&sNC, p->pOffset) ){ return WRC_Abort; } - + /* Set up the local name-context to pass to sqlite3ResolveExprNames() to ** resolve the result-set expression list. */ sNC.allowAgg = 1; sNC.pSrcList = p->pSrc; sNC.pNext = pOuterNC; - + /* Resolve names in the result set. */ pEList = p->pEList; assert( pEList!=0 ); @@ -59871,7 +60059,7 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ return WRC_Abort; } } - + /* Recursively resolve names in all subqueries */ for(i=0; ipSrc->nSrc; i++){ @@ -59884,8 +60072,8 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ if( pParse->nErr || db->mallocFailed ) return WRC_Abort; } } - - /* If there are no aggregate functions in the result-set, and no GROUP BY + + /* If there are no aggregate functions in the result-set, and no GROUP BY ** expression, do not allow aggregates in any of the other expressions. */ assert( (p->selFlags & SF_Aggregate)==0 ); @@ -59895,14 +60083,14 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ }else{ sNC.allowAgg = 0; } - + /* If a HAVING clause is present, then there must be a GROUP BY clause. */ if( p->pHaving && !pGroupBy ){ sqlite3ErrorMsg(pParse, "a GROUP BY clause is required before HAVING"); return WRC_Abort; } - + /* Add the expression list to the name-context before parsing the ** other expressions in the SELECT statement. This is so that ** expressions in the WHERE clause (etc.) can refer to expressions by @@ -59919,7 +60107,7 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ } /* The ORDER BY and GROUP BY clauses may not refer to terms in - ** outer queries + ** outer queries */ sNC.pNext = 0; sNC.allowAgg = 1; @@ -59935,13 +60123,13 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ if( db->mallocFailed ){ return WRC_Abort; } - - /* Resolve the GROUP BY clause. At the same time, make sure + + /* Resolve the GROUP BY clause. At the same time, make sure ** the GROUP BY clause does not contain aggregate functions. */ if( pGroupBy ){ struct ExprList_item *pItem; - + if( resolveOrderGroupBy(&sNC, p, pGroupBy, "GROUP") || db->mallocFailed ){ return WRC_Abort; } @@ -59976,7 +60164,7 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ ** checking on function usage and set a flag if any aggregate functions ** are seen. ** -** To resolve table columns references we look for nodes (or subtrees) of the +** To resolve table columns references we look for nodes (or subtrees) of the ** form X.Y.Z or Y.Z or just Z where ** ** X: The name of a database. Ex: "main" or "temp" or @@ -60008,7 +60196,7 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ ** ** SELECT a+b AS x, c+d AS y FROM t1 ORDER BY a+b; ** -** Function calls are checked to make sure that the function is +** Function calls are checked to make sure that the function is ** defined and that the correct number of arguments are specified. ** If the function is an aggregate function, then the pNC->hasAgg is ** set and the opcode is changed from TK_FUNCTION to TK_AGG_FUNCTION. @@ -60018,7 +60206,7 @@ static int resolveSelectStep(Walker *pWalker, Select *p){ ** An error message is left in pParse if anything is amiss. The number ** if errors is returned. */ -SQLITE_PRIVATE int sqlite3ResolveExprNames( +SQLITE_PRIVATE int sqlite3ResolveExprNames( NameContext *pNC, /* Namespace to resolve expressions in. */ Expr *pExpr /* The expression to be analyzed. */ ){ @@ -60105,7 +60293,7 @@ SQLITE_PRIVATE void sqlite3ResolveSelectNames( ** Return the 'affinity' of the expression pExpr if any. ** ** If pExpr is a column, a reference to a column via an 'AS' alias, -** or a sub-select with a column as the return value, then the +** or a sub-select with a column as the return value, then the ** affinity of that column is returned. Otherwise, 0x00 is returned, ** indicating no affinity for the expression. ** @@ -60129,7 +60317,7 @@ SQLITE_PRIVATE char sqlite3ExprAffinity(Expr *pExpr){ return sqlite3AffinityType(pExpr->u.zToken); } #endif - if( (op==TK_AGG_COLUMN || op==TK_COLUMN || op==TK_REGISTER) + if( (op==TK_AGG_COLUMN || op==TK_COLUMN || op==TK_REGISTER) && pExpr->pTab!=0 ){ /* op==TK_REGISTER && pExpr->pTab!=0 happens when pExpr was originally @@ -60197,7 +60385,7 @@ SQLITE_PRIVATE CollSeq *sqlite3ExprCollSeq(Parse *pParse, Expr *pExpr){ } p = p->pLeft; } - if( sqlite3CheckCollSeq(pParse, pColl) ){ + if( sqlite3CheckCollSeq(pParse, pColl) ){ pColl = 0; } return pColl; @@ -60293,8 +60481,8 @@ static u8 binaryCompareP5(Expr *pExpr1, Expr *pExpr2, int jumpIfNull){ ** it is not considered. */ SQLITE_PRIVATE CollSeq *sqlite3BinaryCompareCollSeq( - Parse *pParse, - Expr *pLeft, + Parse *pParse, + Expr *pLeft, Expr *pRight ){ CollSeq *pColl; @@ -60376,7 +60564,7 @@ SQLITE_PRIVATE int sqlite3ExprCheckHeight(Parse *pParse, int nHeight){ int rc = SQLITE_OK; int mxHeight = pParse->db->aLimit[SQLITE_LIMIT_EXPR_DEPTH]; if( nHeight>mxHeight ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "Expression tree is too large (maximum depth %d)", mxHeight ); rc = SQLITE_ERROR; @@ -60422,10 +60610,10 @@ static void heightOfSelect(Select *p, int *pnHeight){ } /* -** Set the Expr.nHeight variable in the structure passed as an -** argument. An expression with no children, Expr.pList or +** Set the Expr.nHeight variable in the structure passed as an +** argument. An expression with no children, Expr.pList or ** Expr.pSelect member has a height of 1. Any other expression -** has a height equal to the maximum height of any other +** has a height equal to the maximum height of any other ** referenced Expr plus one. */ static void exprSetHeight(Expr *p){ @@ -60512,7 +60700,7 @@ SQLITE_PRIVATE Expr *sqlite3ExprAlloc( pNew->u.zToken = (char*)&pNew[1]; memcpy(pNew->u.zToken, pToken->z, pToken->n); pNew->u.zToken[pToken->n] = 0; - if( dequote && nExtra>=3 + if( dequote && nExtra>=3 && ((c = pToken->z[0])=='\'' || c=='"' || c=='[' || c=='`') ){ sqlite3Dequote(pNew->u.zToken); if( c=='"' ) pNew->flags |= EP_DblQuoted; @@ -60521,7 +60709,7 @@ SQLITE_PRIVATE Expr *sqlite3ExprAlloc( } #if SQLITE_MAX_EXPR_DEPTH>0 pNew->nHeight = 1; -#endif +#endif } return pNew; } @@ -60632,7 +60820,7 @@ SQLITE_PRIVATE Expr *sqlite3ExprFunction(Parse *pParse, ExprList *pList, Token * /* ** Assign a variable number to an expression that encodes a wildcard -** in the original SQL statement. +** in the original SQL statement. ** ** Wildcards consisting of a single "?" are assigned the next sequential ** variable number. @@ -60707,18 +60895,17 @@ SQLITE_PRIVATE void sqlite3ExprAssignVarNumber(Parse *pParse, Expr *pExpr){ pParse->apVarExpr[pParse->nVarExpr++] = pExpr; } } - } + } if( !pParse->nErr && pParse->nVar>db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ){ sqlite3ErrorMsg(pParse, "too many SQL variables"); } } /* -** Clear an expression structure without deleting the structure itself. -** Substructure is deleted. +** Recursively delete an expression tree. */ -SQLITE_PRIVATE void sqlite3ExprClear(sqlite3 *db, Expr *p){ - assert( p!=0 ); +SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3 *db, Expr *p){ + if( p==0 ) return; if( !ExprHasAnyProperty(p, EP_TokenOnly) ){ sqlite3ExprDelete(db, p->pLeft); sqlite3ExprDelete(db, p->pRight); @@ -60731,21 +60918,13 @@ SQLITE_PRIVATE void sqlite3ExprClear(sqlite3 *db, Expr *p){ sqlite3ExprListDelete(db, p->x.pList); } } -} - -/* -** Recursively delete an expression tree. -*/ -SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3 *db, Expr *p){ - if( p==0 ) return; - sqlite3ExprClear(db, p); if( !ExprHasProperty(p, EP_Static) ){ sqlite3DbFree(db, p); } } /* -** Return the number of bytes allocated for the expression structure +** Return the number of bytes allocated for the expression structure ** passed as the first argument. This is always one of EXPR_FULLSIZE, ** EXPR_REDUCEDSIZE or EXPR_TOKENONLYSIZE. */ @@ -60760,14 +60939,14 @@ static int exprStructSize(Expr *p){ ** to store a copy of an expression or expression tree. They differ in ** how much of the tree is measured. ** -** dupedExprStructSize() Size of only the Expr structure +** dupedExprStructSize() Size of only the Expr structure ** dupedExprNodeSize() Size of Expr + space for token ** dupedExprSize() Expr + token + subtree components ** *************************************************************************** ** -** The dupedExprStructSize() function returns two values OR-ed together: -** (1) the space required for a copy of the Expr structure only and +** The dupedExprStructSize() function returns two values OR-ed together: +** (1) the space required for a copy of the Expr structure only and ** (2) the EP_xxx flags that indicate what the structure size should be. ** The return values is always one of: ** @@ -60796,7 +60975,7 @@ static int dupedExprStructSize(Expr *p, int flags){ nSize = EXPR_FULLSIZE; }else{ assert( !ExprHasAnyProperty(p, EP_TokenOnly|EP_Reduced) ); - assert( !ExprHasProperty(p, EP_FromJoin) ); + assert( !ExprHasProperty(p, EP_FromJoin) ); assert( (p->flags2 & EP2_MallocedToken)==0 ); assert( (p->flags2 & EP2_Irreducible)==0 ); if( p->pLeft || p->pRight || p->pColl || p->x.pList ){ @@ -60809,7 +60988,7 @@ static int dupedExprStructSize(Expr *p, int flags){ } /* -** This function returns the space in bytes required to store the copy +** This function returns the space in bytes required to store the copy ** of the Expr structure and a copy of the Expr.u.zToken string (if that ** string is defined.) */ @@ -60822,16 +61001,16 @@ static int dupedExprNodeSize(Expr *p, int flags){ } /* -** Return the number of bytes required to create a duplicate of the +** Return the number of bytes required to create a duplicate of the ** expression passed as the first argument. The second argument is a ** mask containing EXPRDUP_XXX flags. ** ** The value returned includes space to create a copy of the Expr struct ** itself and the buffer referred to by Expr.u.zToken, if any. ** -** If the EXPRDUP_REDUCE flag is set, then the return value includes -** space to duplicate all Expr nodes in the tree formed by Expr.pLeft -** and Expr.pRight variables (but not for any structures pointed to or +** If the EXPRDUP_REDUCE flag is set, then the return value includes +** space to duplicate all Expr nodes in the tree formed by Expr.pLeft +** and Expr.pRight variables (but not for any structures pointed to or ** descended from the Expr.x.pList or Expr.x.pSelect variables). */ static int dupedExprSize(Expr *p, int flags){ @@ -60846,8 +61025,8 @@ static int dupedExprSize(Expr *p, int flags){ } /* -** This function is similar to sqlite3ExprDup(), except that if pzBuffer -** is not NULL then *pzBuffer is assumed to point to a buffer large enough +** This function is similar to sqlite3ExprDup(), except that if pzBuffer +** is not NULL then *pzBuffer is assumed to point to a buffer large enough ** to store the copy of expression p, the copies of p->u.zToken ** (if applicable), and the copies of the p->pLeft and p->pRight expressions, ** if any. Before returning, *pzBuffer is set to the first byte passed the @@ -60944,7 +61123,7 @@ static Expr *exprDup(sqlite3 *db, Expr *p, int flags, u8 **pzBuffer){ ** without effecting the originals. ** ** The expression list, ID, and source lists return by sqlite3ExprListDup(), -** sqlite3IdListDup(), and sqlite3SrcListDup() can not be further expanded +** sqlite3IdListDup(), and sqlite3SrcListDup() can not be further expanded ** by subsequent calls to sqlite*ListAppend() routines. ** ** Any tables that the SrcList might point to are not duplicated. @@ -60970,7 +61149,7 @@ SQLITE_PRIVATE ExprList *sqlite3ExprListDup(sqlite3 *db, ExprList *p, int flags) if( pItem==0 ){ sqlite3DbFree(db, pNew); return 0; - } + } pOldItem = p->a; for(i=0; inExpr; i++, pItem++, pOldItem++){ Expr *pOldExpr = pOldItem->pExpr; @@ -60987,7 +61166,7 @@ SQLITE_PRIVATE ExprList *sqlite3ExprListDup(sqlite3 *db, ExprList *p, int flags) /* ** If cursors, triggers, views and subqueries are all omitted from -** the build, then none of the following routines, except for +** the build, then none of the following routines, except for ** sqlite3SelectDup(), can be called. sqlite3SelectDup() is sometimes ** called with a NULL argument. */ @@ -61117,7 +61296,7 @@ SQLITE_PRIVATE ExprList *sqlite3ExprListAppend( } return pList; -no_mem: +no_mem: /* Avoid leaking memory if malloc has failed. */ sqlite3ExprDelete(db, pExpr); sqlite3ExprListDelete(db, pList); @@ -61345,6 +61524,94 @@ SQLITE_PRIVATE int sqlite3ExprIsInteger(Expr *p, int *pValue){ return rc; } +/* +** Return FALSE if there is no chance that the expression can be NULL. +** +** If the expression might be NULL or if the expression is too complex +** to tell return TRUE. +** +** This routine is used as an optimization, to skip OP_IsNull opcodes +** when we know that a value cannot be NULL. Hence, a false positive +** (returning TRUE when in fact the expression can never be NULL) might +** be a small performance hit but is otherwise harmless. On the other +** hand, a false negative (returning FALSE when the result could be NULL) +** will likely result in an incorrect answer. So when in doubt, return +** TRUE. +*/ +SQLITE_PRIVATE int sqlite3ExprCanBeNull(const Expr *p){ + u8 op; + while( p->op==TK_UPLUS || p->op==TK_UMINUS ){ p = p->pLeft; } + op = p->op; + if( op==TK_REGISTER ) op = p->op2; + switch( op ){ + case TK_INTEGER: + case TK_STRING: + case TK_FLOAT: + case TK_BLOB: + return 0; + default: + return 1; + } +} + +/* +** Generate an OP_IsNull instruction that tests register iReg and jumps +** to location iDest if the value in iReg is NULL. The value in iReg +** was computed by pExpr. If we can look at pExpr at compile-time and +** determine that it can never generate a NULL, then the OP_IsNull operation +** can be omitted. +*/ +SQLITE_PRIVATE void sqlite3ExprCodeIsNullJump( + Vdbe *v, /* The VDBE under construction */ + const Expr *pExpr, /* Only generate OP_IsNull if this expr can be NULL */ + int iReg, /* Test the value in this register for NULL */ + int iDest /* Jump here if the value is null */ +){ + if( sqlite3ExprCanBeNull(pExpr) ){ + sqlite3VdbeAddOp2(v, OP_IsNull, iReg, iDest); + } +} + +/* +** Return TRUE if the given expression is a constant which would be +** unchanged by OP_Affinity with the affinity given in the second +** argument. +** +** This routine is used to determine if the OP_Affinity operation +** can be omitted. When in doubt return FALSE. A false negative +** is harmless. A false positive, however, can result in the wrong +** answer. +*/ +SQLITE_PRIVATE int sqlite3ExprNeedsNoAffinityChange(const Expr *p, char aff){ + u8 op; + if( aff==SQLITE_AFF_NONE ) return 1; + while( p->op==TK_UPLUS || p->op==TK_UMINUS ){ p = p->pLeft; } + op = p->op; + if( op==TK_REGISTER ) op = p->op2; + switch( op ){ + case TK_INTEGER: { + return aff==SQLITE_AFF_INTEGER || aff==SQLITE_AFF_NUMERIC; + } + case TK_FLOAT: { + return aff==SQLITE_AFF_REAL || aff==SQLITE_AFF_NUMERIC; + } + case TK_STRING: { + return aff==SQLITE_AFF_TEXT; + } + case TK_BLOB: { + return 1; + } + case TK_COLUMN: { + assert( p->iTable>=0 ); /* p cannot be part of a CHECK constraint */ + return p->iColumn<0 + && (aff==SQLITE_AFF_INTEGER || aff==SQLITE_AFF_NUMERIC); + } + default: { + return 0; + } + } +} + /* ** Return TRUE if the given string is a row-id column name. */ @@ -61404,7 +61671,7 @@ static int isCandidateForInOpt(Select *p){ ** either to test for membership of the (...) set or to iterate through ** its members, skipping duplicates. ** -** The index of the cursor opened on the b-tree (database table, database index +** The index of the cursor opened on the b-tree (database table, database index ** or ephermal table) is stored in pX->iTable before this function returns. ** The returned value of this function indicates the b-tree type, as follows: ** @@ -61424,24 +61691,24 @@ static int isCandidateForInOpt(Select *p){ ** to be unique - either because it is an INTEGER PRIMARY KEY or it ** has a UNIQUE constraint or UNIQUE index. ** -** If the prNotFound parameter is not 0, then the b-tree will be used -** for fast set membership tests. In this case an epheremal table must -** be used unless is an INTEGER PRIMARY KEY or an index can +** If the prNotFound parameter is not 0, then the b-tree will be used +** for fast set membership tests. In this case an epheremal table must +** be used unless is an INTEGER PRIMARY KEY or an index can ** be found with as its left-most column. ** ** When the b-tree is being used for membership tests, the calling function -** needs to know whether or not the structure contains an SQL NULL +** needs to know whether or not the structure contains an SQL NULL ** value in order to correctly evaluate expressions like "X IN (Y, Z)". -** If there is a chance that the b-tree might contain a NULL value at +** If there is any chance that the (...) might contain a NULL value at ** runtime, then a register is allocated and the register number written -** to *prNotFound. If there is no chance that the b-tree contains a +** to *prNotFound. If there is no chance that the (...) contains a ** NULL value, then *prNotFound is left unchanged. ** ** If a register is allocated and its location stored in *prNotFound, then -** its initial value is NULL. If the b-tree does not remain constant -** for the duration of the query (i.e. the SELECT that generates the b-tree +** its initial value is NULL. If the (...) does not remain constant +** for the duration of the query (i.e. the SELECT within the (...) ** is a correlated subquery) then the value of the allocated register is -** reset to NULL each time the b-tree is repopulated. This allows the +** reset to NULL each time the subquery is rerun. This allows the ** caller to use vdbe code equivalent to the following: ** ** if( register==NULL ){ @@ -61462,7 +61729,7 @@ SQLITE_PRIVATE int sqlite3FindInIndex(Parse *pParse, Expr *pX, int *prNotFound){ assert( pX->op==TK_IN ); /* Check to see if an existing table or index can be used to - ** satisfy the query. This is preferable to generating a new + ** satisfy the query. This is preferable to generating a new ** ephemeral table. */ p = (ExprHasProperty(pX, EP_xIsSelect) ? pX->x.pSelect : 0); @@ -61473,7 +61740,7 @@ SQLITE_PRIVATE int sqlite3FindInIndex(Parse *pParse, Expr *pX, int *prNotFound){ Vdbe *v = sqlite3GetVdbe(pParse); /* Virtual machine being coded */ Table *pTab = p->pSrc->a[0].pTab; /* Table . */ int iDb; /* Database idx for pTab */ - + /* Code an OP_VerifyCookie and OP_TableLock for
    . */ iDb = sqlite3SchemaToIndex(db, pTab->pSchema); sqlite3CodeVerifySchema(pParse, iDb); @@ -61503,7 +61770,7 @@ SQLITE_PRIVATE int sqlite3FindInIndex(Parse *pParse, Expr *pX, int *prNotFound){ ** to this collation sequence. */ CollSeq *pReq = sqlite3BinaryCompareCollSeq(pParse, pX->pLeft, pExpr); - /* Check that the affinity that will be used to perform the + /* Check that the affinity that will be used to perform the ** comparison is the same as the affinity of the column. If ** it is not, it is not possible to use any index. */ @@ -61518,11 +61785,11 @@ SQLITE_PRIVATE int sqlite3FindInIndex(Parse *pParse, Expr *pX, int *prNotFound){ int iMem = ++pParse->nMem; int iAddr; char *pKey; - + pKey = (char *)sqlite3IndexKeyinfo(pParse, pIdx); iAddr = sqlite3VdbeAddOp1(v, OP_If, iMem); sqlite3VdbeAddOp2(v, OP_Integer, 1, iMem); - + sqlite3VdbeAddOp4(v, OP_OpenRead, iTab, pIdx->tnum, iDb, pKey,P4_KEYINFO_HANDOFF); VdbeComment((v, "%s", pIdx->zName)); @@ -61633,7 +61900,7 @@ SQLITE_PRIVATE int sqlite3CodeSubselect( affinity = sqlite3ExprAffinity(pLeft); /* Whether this is an 'x IN(SELECT...)' or an 'x IN()' - ** expression it is handled the same way. A virtual table is + ** expression it is handled the same way. An ephemeral table is ** filled with single-field index keys representing the results ** from the SELECT or the . ** @@ -61667,7 +61934,7 @@ SQLITE_PRIVATE int sqlite3CodeSubselect( return 0; } pEList = pExpr->x.pSelect->pEList; - if( ALWAYS(pEList!=0 && pEList->nExpr>0) ){ + if( ALWAYS(pEList!=0 && pEList->nExpr>0) ){ keyInfo.aColl[0] = sqlite3BinaryCompareCollSeq(pParse, pExpr->pLeft, pEList->a[0].pExpr); } @@ -61781,6 +62048,128 @@ SQLITE_PRIVATE int sqlite3CodeSubselect( } #endif /* SQLITE_OMIT_SUBQUERY */ +#ifndef SQLITE_OMIT_SUBQUERY +/* +** Generate code for an IN expression. +** +** x IN (SELECT ...) +** x IN (value, value, ...) +** +** The left-hand side (LHS) is a scalar expression. The right-hand side (RHS) +** is an array of zero or more values. The expression is true if the LHS is +** contained within the RHS. The value of the expression is unknown (NULL) +** if the LHS is NULL or if the LHS is not contained within the RHS and the +** RHS contains one or more NULL values. +** +** This routine generates code will jump to destIfFalse if the LHS is not +** contained within the RHS. If due to NULLs we cannot determine if the LHS +** is contained in the RHS then jump to destIfNull. If the LHS is contained +** within the RHS then fall through. +*/ +static void sqlite3ExprCodeIN( + Parse *pParse, /* Parsing and code generating context */ + Expr *pExpr, /* The IN expression */ + int destIfFalse, /* Jump here if LHS is not contained in the RHS */ + int destIfNull /* Jump here if the results are unknown due to NULLs */ +){ + int rRhsHasNull = 0; /* Register that is true if RHS contains NULL values */ + char affinity; /* Comparison affinity to use */ + int eType; /* Type of the RHS */ + int r1; /* Temporary use register */ + Vdbe *v; /* Statement under construction */ + + /* Compute the RHS. After this step, the table with cursor + ** pExpr->iTable will contains the values that make up the RHS. + */ + v = pParse->pVdbe; + assert( v!=0 ); /* OOM detected prior to this routine */ + VdbeNoopComment((v, "begin IN expr")); + eType = sqlite3FindInIndex(pParse, pExpr, &rRhsHasNull); + + /* Figure out the affinity to use to create a key from the results + ** of the expression. affinityStr stores a static string suitable for + ** P4 of OP_MakeRecord. + */ + affinity = comparisonAffinity(pExpr); + + /* Code the LHS, the from " IN (...)". + */ + sqlite3ExprCachePush(pParse); + r1 = sqlite3GetTempReg(pParse); + sqlite3ExprCode(pParse, pExpr->pLeft, r1); + sqlite3VdbeAddOp2(v, OP_IsNull, r1, destIfNull); + + + if( eType==IN_INDEX_ROWID ){ + /* In this case, the RHS is the ROWID of table b-tree + */ + sqlite3VdbeAddOp2(v, OP_MustBeInt, r1, destIfFalse); + sqlite3VdbeAddOp3(v, OP_NotExists, pExpr->iTable, destIfFalse, r1); + }else{ + /* In this case, the RHS is an index b-tree. + */ + sqlite3VdbeAddOp4(v, OP_Affinity, r1, 1, 0, &affinity, 1); + + /* If the set membership test fails, then the result of the + ** "x IN (...)" expression must be either 0 or NULL. If the set + ** contains no NULL values, then the result is 0. If the set + ** contains one or more NULL values, then the result of the + ** expression is also NULL. + */ + if( rRhsHasNull==0 || destIfFalse==destIfNull ){ + /* This branch runs if it is known at compile time that the RHS + ** cannot contain NULL values. This happens as the result + ** of a "NOT NULL" constraint in the database schema. + ** + ** Also run this branch if NULL is equivalent to FALSE + ** for this particular IN operator. + */ + sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable, destIfFalse, r1, 1); + + }else{ + /* In this branch, the RHS of the IN might contain a NULL and + ** the presence of a NULL on the RHS makes a difference in the + ** outcome. + */ + int j1, j2, j3; + + /* First check to see if the LHS is contained in the RHS. If so, + ** then the presence of NULLs in the RHS does not matter, so jump + ** over all of the code that follows. + */ + j1 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, r1, 1); + + /* Here we begin generating code that runs if the LHS is not + ** contained within the RHS. Generate additional code that + ** tests the RHS for NULLs. If the RHS contains a NULL then + ** jump to destIfNull. If there are no NULLs in the RHS then + ** jump to destIfFalse. + */ + j2 = sqlite3VdbeAddOp1(v, OP_NotNull, rRhsHasNull); + j3 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, rRhsHasNull, 1); + sqlite3VdbeAddOp2(v, OP_Integer, -1, rRhsHasNull); + sqlite3VdbeJumpHere(v, j3); + sqlite3VdbeAddOp2(v, OP_AddImm, rRhsHasNull, 1); + sqlite3VdbeJumpHere(v, j2); + + /* Jump to the appropriate target depending on whether or not + ** the RHS contains a NULL + */ + sqlite3VdbeAddOp2(v, OP_If, rRhsHasNull, destIfNull); + sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfFalse); + + /* The OP_Found at the top of this branch jumps here when true, + ** causing the overall IN expression evaluation to fall through. + */ + sqlite3VdbeJumpHere(v, j1); + } + } + sqlite3ReleaseTempReg(pParse, r1); + sqlite3ExprCachePop(pParse, 1); + VdbeComment((v, "end IN expr")); +} +#endif /* SQLITE_OMIT_SUBQUERY */ + /* ** Duplicate an 8-byte value */ @@ -61796,7 +62185,7 @@ static char *dup8bytes(Vdbe *v, const char *in){ ** Generate an instruction that will put the floating point ** value described by z[0..n-1] into register iMem. ** -** The z[] string will probably not be zero-terminated. But the +** The z[] string will probably not be zero-terminated. But the ** z[n] character is guaranteed to be something that does not look ** like the continuation of the number. */ @@ -61817,7 +62206,7 @@ static void codeReal(Vdbe *v, const char *z, int negateFlag, int iMem){ ** Generate an instruction that will put the integer describe by ** text z[0..n-1] into register iMem. ** -** The z[] string will probably not be zero-terminated. But the +** The z[] string will probably not be zero-terminated. But the ** z[n] character is guaranteed to be something that does not look ** like the continuation of the number. */ @@ -62009,7 +62398,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeGetColumn( sqlite3ExprCachePinRegister(pParse, p->iReg); return p->iReg; } - } + } assert( v!=0 ); if( iColumn<0 ){ sqlite3VdbeAddOp2(v, OP_Rowid, iTable, iReg); @@ -62126,7 +62515,7 @@ SQLITE_PRIVATE void sqlite3ExprHardCopy(Parse *pParse, int iReg, int nReg){ ** same order that it is executed. ** ** Aliases are numbered starting with 1. So iAlias is in the range -** of 1 to pParse->nAlias inclusive. +** of 1 to pParse->nAlias inclusive. ** ** pParse->aAlias[iAlias-1] records the register number where the value ** of the iAlias-th alias is stored. If zero, that means that the @@ -62366,7 +62755,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) case TK_BITOR: case TK_SLASH: case TK_LSHIFT: - case TK_RSHIFT: + case TK_RSHIFT: case TK_CONCAT: { assert( TK_AND==OP_And ); assert( TK_OR==OP_Or ); @@ -62481,6 +62870,27 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) sqlite3ErrorMsg(pParse, "unknown function: %.*s()", nId, zId); break; } + + /* Attempt a direct implementation of the built-in COALESCE() and + ** IFNULL() functions. This avoids unnecessary evalation of + ** arguments past the first non-NULL argument. + */ + if( pDef->flags & SQLITE_FUNC_COALESCE ){ + int endCoalesce = sqlite3VdbeMakeLabel(v); + assert( nFarg>=2 ); + sqlite3ExprCode(pParse, pFarg->a[0].pExpr, target); + for(i=1; ia[i].pExpr, target); + sqlite3ExprCachePop(pParse, 1); + } + sqlite3VdbeResolveLabel(v, endCoalesce); + break; + } + + if( pFarg ){ r1 = sqlite3GetTempRange(pParse, nFarg); sqlite3ExprCachePush(pParse); /* Ticket 2ea2425d34be */ @@ -62498,7 +62908,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) ** see if it is a column in a virtual table. This is done because ** the left operand of infix functions (the operand we want to ** control overloading) ends up as the second argument to the - ** function. The expression "A glob B" is equivalent to + ** function. The expression "A glob B" is equivalent to ** "glob(B,A). We want to use the A in "A glob B" to test ** for function overloading. But we use the B term in "glob(B,A)". */ @@ -62517,7 +62927,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) } } if( pDef->flags & SQLITE_FUNC_NEEDCOLL ){ - if( !pColl ) pColl = db->pDfltColl; + if( !pColl ) pColl = db->pDfltColl; sqlite3VdbeAddOp4(v, OP_CollSeq, 0, 0, 0, (char *)pColl, P4_COLLSEQ); } sqlite3VdbeAddOp4(v, OP_Function, constMask, r1, target, @@ -62538,95 +62948,19 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) break; } case TK_IN: { - int rNotFound = 0; - int rMayHaveNull = 0; - int j2, j3, j4, j5; - char affinity; - int eType; - - VdbeNoopComment((v, "begin IN expr r%d", target)); - eType = sqlite3FindInIndex(pParse, pExpr, &rMayHaveNull); - if( rMayHaveNull ){ - rNotFound = ++pParse->nMem; - } - - /* Figure out the affinity to use to create a key from the results - ** of the expression. affinityStr stores a static string suitable for - ** P4 of OP_MakeRecord. - */ - affinity = comparisonAffinity(pExpr); - - - /* Code the from " IN (...)". The temporary table - ** pExpr->iTable contains the values that make up the (...) set. - */ - sqlite3ExprCachePush(pParse); - sqlite3ExprCode(pParse, pExpr->pLeft, target); - j2 = sqlite3VdbeAddOp1(v, OP_IsNull, target); - if( eType==IN_INDEX_ROWID ){ - j3 = sqlite3VdbeAddOp1(v, OP_MustBeInt, target); - j4 = sqlite3VdbeAddOp3(v, OP_NotExists, pExpr->iTable, 0, target); - sqlite3VdbeAddOp2(v, OP_Integer, 1, target); - j5 = sqlite3VdbeAddOp0(v, OP_Goto); - sqlite3VdbeJumpHere(v, j3); - sqlite3VdbeJumpHere(v, j4); - sqlite3VdbeAddOp2(v, OP_Integer, 0, target); - }else{ - r2 = regFree2 = sqlite3GetTempReg(pParse); - - /* Create a record and test for set membership. If the set contains - ** the value, then jump to the end of the test code. The target - ** register still contains the true (1) value written to it earlier. - */ - sqlite3VdbeAddOp4(v, OP_MakeRecord, target, 1, r2, &affinity, 1); - sqlite3VdbeAddOp2(v, OP_Integer, 1, target); - j5 = sqlite3VdbeAddOp3(v, OP_Found, pExpr->iTable, 0, r2); - - /* If the set membership test fails, then the result of the - ** "x IN (...)" expression must be either 0 or NULL. If the set - ** contains no NULL values, then the result is 0. If the set - ** contains one or more NULL values, then the result of the - ** expression is also NULL. - */ - if( rNotFound==0 ){ - /* This branch runs if it is known at compile time (now) that - ** the set contains no NULL values. This happens as the result - ** of a "NOT NULL" constraint in the database schema. No need - ** to test the data structure at runtime in this case. - */ - sqlite3VdbeAddOp2(v, OP_Integer, 0, target); - }else{ - /* This block populates the rNotFound register with either NULL - ** or 0 (an integer value). If the data structure contains one - ** or more NULLs, then set rNotFound to NULL. Otherwise, set it - ** to 0. If register rMayHaveNull is already set to some value - ** other than NULL, then the test has already been run and - ** rNotFound is already populated. - */ - static const char nullRecord[] = { 0x02, 0x00 }; - j3 = sqlite3VdbeAddOp1(v, OP_NotNull, rMayHaveNull); - sqlite3VdbeAddOp2(v, OP_Null, 0, rNotFound); - sqlite3VdbeAddOp4(v, OP_Blob, 2, rMayHaveNull, 0, - nullRecord, P4_STATIC); - j4 = sqlite3VdbeAddOp3(v, OP_Found, pExpr->iTable, 0, rMayHaveNull); - sqlite3VdbeAddOp2(v, OP_Integer, 0, rNotFound); - sqlite3VdbeJumpHere(v, j4); - sqlite3VdbeJumpHere(v, j3); - - /* Copy the value of register rNotFound (which is either NULL or 0) - ** into the target register. This will be the result of the - ** expression. - */ - sqlite3VdbeAddOp2(v, OP_Copy, rNotFound, target); - } - } - sqlite3VdbeJumpHere(v, j2); - sqlite3VdbeJumpHere(v, j5); - sqlite3ExprCachePop(pParse, 1); - VdbeComment((v, "end IN expr r%d", target)); + int destIfFalse = sqlite3VdbeMakeLabel(v); + int destIfNull = sqlite3VdbeMakeLabel(v); + sqlite3VdbeAddOp2(v, OP_Null, 0, target); + sqlite3ExprCodeIN(pParse, pExpr, destIfFalse, destIfNull); + sqlite3VdbeAddOp2(v, OP_Integer, 1, target); + sqlite3VdbeResolveLabel(v, destIfFalse); + sqlite3VdbeAddOp2(v, OP_AddImm, target, 0); + sqlite3VdbeResolveLabel(v, destIfNull); break; } -#endif +#endif /* SQLITE_OMIT_SUBQUERY */ + + /* ** x BETWEEN y AND z ** @@ -62677,7 +63011,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) ** ** The expression is implemented using an OP_Param opcode. The p1 ** parameter is set to 0 for an old.rowid reference, or to (i+1) - ** to reference another column of the old.* pseudo-table, where + ** to reference another column of the old.* pseudo-table, where ** i is the index of the column. For a new.rowid reference, p1 is ** set to (n+1), where n is the number of columns in each pseudo-table. ** For a reference to any other column in the new.* pseudo-table, p1 @@ -62691,7 +63025,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) ** ** p1==0 -> old.rowid p1==3 -> new.rowid ** p1==1 -> old.a p1==4 -> new.a - ** p1==2 -> old.b p1==5 -> new.b + ** p1==2 -> old.b p1==5 -> new.b */ Table *pTab = pExpr->pTab; int p1 = pExpr->iTable * (pTab->nCol+1) + 1 + pExpr->iColumn; @@ -62710,7 +63044,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) /* If the column has REAL affinity, it may currently be stored as an ** integer. Use OP_RealAffinity to make sure it is really real. */ - if( pExpr->iColumn>=0 + if( pExpr->iColumn>=0 && pTab->aCol[pExpr->iColumn].affinity==SQLITE_AFF_REAL ){ sqlite3VdbeAddOp1(v, OP_RealAffinity, target); @@ -62796,14 +63130,14 @@ SQLITE_PRIVATE int sqlite3ExprCodeTarget(Parse *pParse, Expr *pExpr, int target) }else{ sqlite3VdbeAddOp2(v, OP_Null, 0, target); } - assert( db->mallocFailed || pParse->nErr>0 + assert( db->mallocFailed || pParse->nErr>0 || pParse->iCacheLevel==iCacheLevel ); sqlite3VdbeResolveLabel(v, endLabel); break; } #ifndef SQLITE_OMIT_TRIGGER case TK_RAISE: { - assert( pExpr->affinity==OE_Rollback + assert( pExpr->affinity==OE_Rollback || pExpr->affinity==OE_Abort || pExpr->affinity==OE_Fail || pExpr->affinity==OE_Ignore @@ -62879,7 +63213,7 @@ SQLITE_PRIVATE int sqlite3ExprCode(Parse *pParse, Expr *pExpr, int target){ ** and modify the expression so that the next time it is evaluated, ** the result is a copy of the cache register. ** -** This routine is used for expressions that are used multiple +** This routine is used for expressions that are used multiple ** times. They are evaluated once and the results of the expression ** are reused. */ @@ -62894,7 +63228,7 @@ SQLITE_PRIVATE int sqlite3ExprCodeAndCache(Parse *pParse, Expr *pExpr, int targe ** no way for a TK_REGISTER to exist here. But it seems prudent to ** keep the ALWAYS() in case the conditions above change with future ** modifications or enhancements. */ - if( ALWAYS(pExpr->op!=TK_REGISTER) ){ + if( ALWAYS(pExpr->op!=TK_REGISTER) ){ int iMem; iMem = ++pParse->nMem; sqlite3VdbeAddOp2(v, OP_Copy, inReg, iMem); @@ -62911,12 +63245,12 @@ SQLITE_PRIVATE int sqlite3ExprCodeAndCache(Parse *pParse, Expr *pExpr, int targe ** ** * Any expression that evaluates to two or more opcodes. ** -** * Any OP_Integer, OP_Real, OP_String, OP_Blob, OP_Null, -** or OP_Variable that does not need to be placed in a +** * Any OP_Integer, OP_Real, OP_String, OP_Blob, OP_Null, +** or OP_Variable that does not need to be placed in a ** specific register. ** ** There is no point in factoring out single-instruction constant -** expressions that need to be placed in a particular register. +** expressions that need to be placed in a particular register. ** We could factor them out, but then we would end up adding an ** OP_SCopy instruction to move the value into the correct register ** later. We might as well just use the original instruction and @@ -62982,7 +63316,7 @@ static int evalConstExpr(Walker *pWalker, Expr *pExpr){ case TK_CONST_FUNC: { /* The arguments to a function have a fixed destination. ** Mark them this way to avoid generated unneeded OP_SCopy - ** instructions. + ** instructions. */ ExprList *pList = pExpr->x.pList; assert( !ExprHasProperty(pExpr, EP_xIsSelect) ); @@ -63057,6 +63391,62 @@ SQLITE_PRIVATE int sqlite3ExprCodeExprList( return n; } +/* +** Generate code for a BETWEEN operator. +** +** x BETWEEN y AND z +** +** The above is equivalent to +** +** x>=y AND x<=z +** +** Code it as such, taking care to do the common subexpression +** elementation of x. +*/ +static void exprCodeBetween( + Parse *pParse, /* Parsing and code generating context */ + Expr *pExpr, /* The BETWEEN expression */ + int dest, /* Jump here if the jump is taken */ + int jumpIfTrue, /* Take the jump if the BETWEEN is true */ + int jumpIfNull /* Take the jump if the BETWEEN is NULL */ +){ + Expr exprAnd; /* The AND operator in x>=y AND x<=z */ + Expr compLeft; /* The x>=y term */ + Expr compRight; /* The x<=z term */ + Expr exprX; /* The x subexpression */ + int regFree1 = 0; /* Temporary use register */ + + assert( !ExprHasProperty(pExpr, EP_xIsSelect) ); + exprX = *pExpr->pLeft; + exprAnd.op = TK_AND; + exprAnd.pLeft = &compLeft; + exprAnd.pRight = &compRight; + compLeft.op = TK_GE; + compLeft.pLeft = &exprX; + compLeft.pRight = pExpr->x.pList->a[0].pExpr; + compRight.op = TK_LE; + compRight.pLeft = &exprX; + compRight.pRight = pExpr->x.pList->a[1].pExpr; + exprX.iTable = sqlite3ExprCodeTemp(pParse, &exprX, ®Free1); + exprX.op = TK_REGISTER; + if( jumpIfTrue ){ + sqlite3ExprIfTrue(pParse, &exprAnd, dest, jumpIfNull); + }else{ + sqlite3ExprIfFalse(pParse, &exprAnd, dest, jumpIfNull); + } + sqlite3ReleaseTempReg(pParse, regFree1); + + /* Ensure adequate test coverage */ + testcase( jumpIfTrue==0 && jumpIfNull==0 && regFree1==0 ); + testcase( jumpIfTrue==0 && jumpIfNull==0 && regFree1!=0 ); + testcase( jumpIfTrue==0 && jumpIfNull!=0 && regFree1==0 ); + testcase( jumpIfTrue==0 && jumpIfNull!=0 && regFree1!=0 ); + testcase( jumpIfTrue!=0 && jumpIfNull==0 && regFree1==0 ); + testcase( jumpIfTrue!=0 && jumpIfNull==0 && regFree1!=0 ); + testcase( jumpIfTrue!=0 && jumpIfNull!=0 && regFree1==0 ); + testcase( jumpIfTrue!=0 && jumpIfNull!=0 && regFree1!=0 ); +} + /* ** Generate code for a boolean expression such that a jump is made ** to the label "dest" if the expression is true but execution @@ -63156,36 +63546,16 @@ SQLITE_PRIVATE void sqlite3ExprIfTrue(Parse *pParse, Expr *pExpr, int dest, int break; } case TK_BETWEEN: { - /* x BETWEEN y AND z - ** - ** Is equivalent to - ** - ** x>=y AND x<=z - ** - ** Code it as such, taking care to do the common subexpression - ** elementation of x. - */ - Expr exprAnd; - Expr compLeft; - Expr compRight; - Expr exprX; - - assert( !ExprHasProperty(pExpr, EP_xIsSelect) ); - exprX = *pExpr->pLeft; - exprAnd.op = TK_AND; - exprAnd.pLeft = &compLeft; - exprAnd.pRight = &compRight; - compLeft.op = TK_GE; - compLeft.pLeft = &exprX; - compLeft.pRight = pExpr->x.pList->a[0].pExpr; - compRight.op = TK_LE; - compRight.pLeft = &exprX; - compRight.pRight = pExpr->x.pList->a[1].pExpr; - exprX.iTable = sqlite3ExprCodeTemp(pParse, &exprX, ®Free1); - testcase( regFree1==0 ); - exprX.op = TK_REGISTER; testcase( jumpIfNull==0 ); - sqlite3ExprIfTrue(pParse, &exprAnd, dest, jumpIfNull); + exprCodeBetween(pParse, pExpr, dest, 1, jumpIfNull); + break; + } + case TK_IN: { + int destIfFalse = sqlite3VdbeMakeLabel(v); + int destIfNull = jumpIfNull ? dest : destIfFalse; + sqlite3ExprCodeIN(pParse, pExpr, destIfFalse, destIfNull); + sqlite3VdbeAddOp2(v, OP_Goto, 0, dest); + sqlite3VdbeResolveLabel(v, destIfFalse); break; } default: { @@ -63197,7 +63567,7 @@ SQLITE_PRIVATE void sqlite3ExprIfTrue(Parse *pParse, Expr *pExpr, int dest, int } } sqlite3ReleaseTempReg(pParse, regFree1); - sqlite3ReleaseTempReg(pParse, regFree2); + sqlite3ReleaseTempReg(pParse, regFree2); } /* @@ -63269,6 +63639,7 @@ SQLITE_PRIVATE void sqlite3ExprIfFalse(Parse *pParse, Expr *pExpr, int dest, int break; } case TK_NOT: { + testcase( jumpIfNull==0 ); sqlite3ExprIfTrue(pParse, pExpr->pLeft, dest, jumpIfNull); break; } @@ -63316,36 +63687,18 @@ SQLITE_PRIVATE void sqlite3ExprIfFalse(Parse *pParse, Expr *pExpr, int dest, int break; } case TK_BETWEEN: { - /* x BETWEEN y AND z - ** - ** Is equivalent to - ** - ** x>=y AND x<=z - ** - ** Code it as such, taking care to do the common subexpression - ** elementation of x. - */ - Expr exprAnd; - Expr compLeft; - Expr compRight; - Expr exprX; - - assert( !ExprHasProperty(pExpr, EP_xIsSelect) ); - exprX = *pExpr->pLeft; - exprAnd.op = TK_AND; - exprAnd.pLeft = &compLeft; - exprAnd.pRight = &compRight; - compLeft.op = TK_GE; - compLeft.pLeft = &exprX; - compLeft.pRight = pExpr->x.pList->a[0].pExpr; - compRight.op = TK_LE; - compRight.pLeft = &exprX; - compRight.pRight = pExpr->x.pList->a[1].pExpr; - exprX.iTable = sqlite3ExprCodeTemp(pParse, &exprX, ®Free1); - testcase( regFree1==0 ); - exprX.op = TK_REGISTER; testcase( jumpIfNull==0 ); - sqlite3ExprIfFalse(pParse, &exprAnd, dest, jumpIfNull); + exprCodeBetween(pParse, pExpr, dest, 0, jumpIfNull); + break; + } + case TK_IN: { + if( jumpIfNull ){ + sqlite3ExprCodeIN(pParse, pExpr, dest, dest); + }else{ + int destIfNull = sqlite3VdbeMakeLabel(v); + sqlite3ExprCodeIN(pParse, pExpr, dest, destIfNull); + sqlite3VdbeResolveLabel(v, destIfNull); + } break; } default: { @@ -63431,7 +63784,7 @@ static int addAggInfoColumn(sqlite3 *db, AggInfo *pInfo){ &i ); return i; -} +} /* ** Add a new element to the pAggInfo->aFunc[] array. Return the index of @@ -63440,7 +63793,7 @@ static int addAggInfoColumn(sqlite3 *db, AggInfo *pInfo){ static int addAggInfoFunc(sqlite3 *db, AggInfo *pInfo){ int i; pInfo->aFunc = sqlite3ArrayAllocate( - db, + db, pInfo->aFunc, sizeof(pInfo->aFunc[0]), 3, @@ -63449,7 +63802,7 @@ static int addAggInfoFunc(sqlite3 *db, AggInfo *pInfo){ &i ); return i; -} +} /* ** This is the xExprCallback for a tree walker. It is used to @@ -63477,7 +63830,7 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){ assert( !ExprHasAnyProperty(pExpr, EP_TokenOnly|EP_Reduced) ); if( pExpr->iTable==pItem->iCursor ){ /* If we reach this point, it means that pExpr refers to a table - ** that is in the FROM clause of the aggregate query. + ** that is in the FROM clause of the aggregate query. ** ** Make an entry for the column in pAggInfo->aCol[] if there ** is not an entry there already. @@ -63491,7 +63844,7 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){ } } if( (k>=pAggInfo->nColumn) - && (k = addAggInfoColumn(pParse->db, pAggInfo))>=0 + && (k = addAggInfoColumn(pParse->db, pAggInfo))>=0 ){ pCol = &pAggInfo->aCol[k]; pCol->pTab = pExpr->pTab; @@ -63537,7 +63890,7 @@ static int analyzeAggregate(Walker *pWalker, Expr *pExpr){ /* The pNC->nDepth==0 test causes aggregate functions in subqueries ** to be ignored */ if( pNC->nDepth==0 ){ - /* Check to see if pExpr is a duplicate of another aggregate + /* Check to see if pExpr is a duplicate of another aggregate ** function that is already in the pAggInfo structure */ struct AggInfo_func *pItem = pAggInfo->aFunc; @@ -63694,8 +64047,6 @@ SQLITE_PRIVATE void sqlite3ReleaseTempRange(Parse *pParse, int iReg, int nReg){ ************************************************************************* ** This file contains C code routines that used to generate VDBE code ** that implements the ALTER TABLE command. -** -** $Id: alter.c,v 1.62 2009/07/24 17:58:53 danielk1977 Exp $ */ /* @@ -63706,9 +64057,9 @@ SQLITE_PRIVATE void sqlite3ReleaseTempRange(Parse *pParse, int iReg, int nReg){ /* -** This function is used by SQL generated to implement the +** This function is used by SQL generated to implement the ** ALTER TABLE command. The first argument is the text of a CREATE TABLE or -** CREATE INDEX command. The second is a table name. The table name in +** CREATE INDEX command. The second is a table name. The table name in ** the CREATE TABLE or CREATE INDEX statement is replaced with the third ** argument and the result returned. Examples: ** @@ -63736,7 +64087,7 @@ static void renameTableFunc( UNUSED_PARAMETER(NotUsed); - /* The principle used to locate the table name in the CREATE TABLE + /* The principle used to locate the table name in the CREATE TABLE ** statement is that the table name is the first non-space token that ** is immediately followed by a TK_LP or TK_USING token. */ @@ -63761,7 +64112,7 @@ static void renameTableFunc( assert( len>0 ); } while( token!=TK_LP && token!=TK_USING ); - zRet = sqlite3MPrintf(db, "%.*s\"%w\"%s", ((u8*)tname.z) - zSql, zSql, + zRet = sqlite3MPrintf(db, "%.*s\"%w\"%s", ((u8*)tname.z) - zSql, zSql, zTableName, tname.z+tname.n); sqlite3_result_text(context, zRet, -1, SQLITE_DYNAMIC); } @@ -63770,7 +64121,7 @@ static void renameTableFunc( /* ** This C function implements an SQL user function that is used by SQL code ** generated by the ALTER TABLE ... RENAME command to modify the definition -** of any foreign key constraints that use the table being renamed as the +** of any foreign key constraints that use the table being renamed as the ** parent table. It is passed three arguments: ** ** 1) The complete text of the CREATE TABLE statement being modified, @@ -63813,7 +64164,7 @@ static void renameParentFunc( if( zParent==0 ) break; sqlite3Dequote(zParent); if( 0==sqlite3StrICmp((const char *)zOld, zParent) ){ - char *zOut = sqlite3MPrintf(db, "%s%.*s\"%w\"", + char *zOut = sqlite3MPrintf(db, "%s%.*s\"%w\"", (zOutput?zOutput:""), z-zInput, zInput, (const char *)zNew ); sqlite3DbFree(db, zOutput); @@ -63824,7 +64175,7 @@ static void renameParentFunc( } } - zResult = sqlite3MPrintf(db, "%s%s", (zOutput?zOutput:""), zInput), + zResult = sqlite3MPrintf(db, "%s%s", (zOutput?zOutput:""), zInput), sqlite3_result_text(context, zResult, -1, SQLITE_DYNAMIC); sqlite3DbFree(db, zOutput); } @@ -63832,9 +64183,9 @@ static void renameParentFunc( #ifndef SQLITE_OMIT_TRIGGER /* This function is used by SQL generated to implement the -** ALTER TABLE command. The first argument is the text of a CREATE TRIGGER -** statement. The second is a table name. The table name in the CREATE -** TRIGGER statement is replaced with the third argument and the result +** ALTER TABLE command. The first argument is the text of a CREATE TRIGGER +** statement. The second is a table name. The table name in the CREATE +** TRIGGER statement is replaced with the third argument and the result ** returned. This is analagous to renameTableFunc() above, except for CREATE ** TRIGGER, not CREATE INDEX and CREATE TABLE. */ @@ -63856,7 +64207,7 @@ static void renameTriggerFunc( UNUSED_PARAMETER(NotUsed); - /* The principle used to locate the table name in the CREATE TRIGGER + /* The principle used to locate the table name in the CREATE TRIGGER ** statement is that the table name is the first token that is immediatedly ** preceded by either TK_ON or TK_DOT and immediatedly followed by one ** of TK_WHEN, TK_BEGIN or TK_FOR. @@ -63883,12 +64234,12 @@ static void renameTriggerFunc( assert( len>0 ); /* Variable 'dist' stores the number of tokens read since the most - ** recent TK_DOT or TK_ON. This means that when a WHEN, FOR or BEGIN + ** recent TK_DOT or TK_ON. This means that when a WHEN, FOR or BEGIN ** token is read and 'dist' equals 2, the condition stated above ** to be met. ** ** Note that ON cannot be a database, table or column name, so - ** there is no need to worry about syntax like + ** there is no need to worry about syntax like ** "CREATE TRIGGER ... ON ON.ON BEGIN ..." etc. */ dist++; @@ -63900,7 +64251,7 @@ static void renameTriggerFunc( /* Variable tname now contains the token that is the old table-name ** in the CREATE TRIGGER statement. */ - zRet = sqlite3MPrintf(db, "%.*s\"%w\"%s", ((u8*)tname.z) - zSql, zSql, + zRet = sqlite3MPrintf(db, "%.*s\"%w\"%s", ((u8*)tname.z) - zSql, zSql, zTableName, tname.z+tname.n); sqlite3_result_text(context, zRet, -1, SQLITE_DYNAMIC); } @@ -63928,16 +64279,16 @@ SQLITE_PRIVATE void sqlite3AlterFunctions(sqlite3 *db){ ** ** name= OR name= OR ... ** -** If argument zWhere is NULL, then a pointer string containing the text +** If argument zWhere is NULL, then a pointer string containing the text ** "name=" is returned, where is the quoted version ** of the string passed as argument zConstant. The returned buffer is ** allocated using sqlite3DbMalloc(). It is the responsibility of the ** caller to ensure that it is eventually freed. ** -** If argument zWhere is not NULL, then the string returned is +** If argument zWhere is not NULL, then the string returned is ** " OR name=", where is the contents of zWhere. ** In this case zWhere is passed to sqlite3DbFree() before returning. -** +** */ static char *whereOrName(sqlite3 *db, char *zWhere, char *zConstant){ char *zNew; @@ -63970,7 +64321,7 @@ static char *whereForeignKeys(Parse *pParse, Table *pTab){ /* ** Generate the text of a WHERE expression which can be used to select all ** temporary triggers on table pTab from the sqlite_temp_master table. If -** table pTab has no temporary triggers, or is itself stored in the +** table pTab has no temporary triggers, or is itself stored in the ** temporary database, NULL is returned. */ static char *whereTempTriggers(Parse *pParse, Table *pTab){ @@ -63978,9 +64329,9 @@ static char *whereTempTriggers(Parse *pParse, Table *pTab){ char *zWhere = 0; const Schema *pTempSchema = pParse->db->aDb[1].pSchema; /* Temp db schema */ - /* If the table is not located in the temp-db (in which case NULL is + /* If the table is not located in the temp-db (in which case NULL is ** returned, loop through the tables list of triggers. For each trigger - ** that is not part of the temp-db schema, add a clause to the WHERE + ** that is not part of the temp-db schema, add a clause to the WHERE ** expression being built up in zWhere. */ if( pTab->pSchema!=pTempSchema ){ @@ -63999,7 +64350,7 @@ static char *whereTempTriggers(Parse *pParse, Table *pTab){ ** pTab from the database, including triggers and temporary triggers. ** Argument zName is the name of the table in the database schema at ** the time the generated code is executed. This can be different from -** pTab->zName if this function is being called to code part of an +** pTab->zName if this function is being called to code part of an ** "ALTER TABLE RENAME TO" statement. */ static void reloadTableSchema(Parse *pParse, Table *pTab, const char *zName){ @@ -64034,8 +64385,8 @@ static void reloadTableSchema(Parse *pParse, Table *pTab, const char *zName){ sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC); #ifndef SQLITE_OMIT_TRIGGER - /* Now, if the table is not stored in the temp database, reload any temp - ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined. + /* Now, if the table is not stored in the temp database, reload any temp + ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined. */ if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){ sqlite3VdbeAddOp4(v, OP_ParseSchema, 1, 0, 0, zWhere, P4_DYNAMIC); @@ -64044,8 +64395,8 @@ static void reloadTableSchema(Parse *pParse, Table *pTab, const char *zName){ } /* -** Generate code to implement the "ALTER TABLE xxx RENAME TO yyy" -** command. +** Generate code to implement the "ALTER TABLE xxx RENAME TO yyy" +** command. */ SQLITE_PRIVATE void sqlite3AlterRenameTable( Parse *pParse, /* Parser context. */ @@ -64055,7 +64406,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( int iDb; /* Database that contains the table */ char *zDb; /* Name of database iDb */ Table *pTab; /* Table being renamed */ - char *zName = 0; /* NULL-terminated version of pName */ + char *zName = 0; /* NULL-terminated version of pName */ sqlite3 *db = pParse->db; /* Database connection */ int nTabName; /* Number of UTF-8 characters in zTabName */ const char *zTabName; /* Original name of the table */ @@ -64064,7 +64415,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( char *zWhere = 0; /* Where clause to locate temp triggers */ #endif VTable *pVTab = 0; /* Non-zero if this is a v-tab with an xRename() */ - + if( NEVER(db->mallocFailed) ) goto exit_rename_table; assert( pSrc->nSrc==1 ); assert( sqlite3BtreeHoldsAllMutexes(pParse->db) ); @@ -64082,7 +64433,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( ** in database iDb. If so, this is an error. */ if( sqlite3FindTable(db, zName, zDb) || sqlite3FindIndex(db, zName, zDb) ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "there is already another table or index with this name: %s", zName); goto exit_rename_table; } @@ -64090,7 +64441,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( /* Make sure it is not a system table being altered, or a reserved name ** that the table is being renamed to. */ - if( sqlite3Strlen30(pTab->zName)>6 + if( sqlite3Strlen30(pTab->zName)>6 && 0==sqlite3StrNICmp(pTab->zName, "sqlite_", 7) ){ sqlite3ErrorMsg(pParse, "table %s may not be altered", pTab->zName); @@ -64126,7 +64477,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( } #endif - /* Begin a transaction and code the VerifyCookie for database iDb. + /* Begin a transaction and code the VerifyCookie for database iDb. ** Then modify the schema cookie (since the ALTER TABLE modifies the ** schema). Open a statement transaction if the table is a virtual ** table. @@ -64158,11 +64509,11 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( #if !defined(SQLITE_OMIT_FOREIGN_KEY) && !defined(SQLITE_OMIT_TRIGGER) if( db->flags&SQLITE_ForeignKeys ){ - /* If foreign-key support is enabled, rewrite the CREATE TABLE + /* If foreign-key support is enabled, rewrite the CREATE TABLE ** statements corresponding to all child tables of foreign key constraints ** for which the renamed table is the parent table. */ if( (zWhere=whereForeignKeys(pParse, pTab))!=0 ){ - sqlite3NestedParse(pParse, + sqlite3NestedParse(pParse, "UPDATE sqlite_master SET " "sql = sqlite_rename_parent(sql, %Q, %Q) " "WHERE %s;", zTabName, zName, zWhere); @@ -64188,8 +64539,8 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( "'sqlite_autoindex_' || %Q || substr(name,%d+18) " "ELSE name END " "WHERE tbl_name=%Q AND " - "(type='table' OR type='index' OR type='trigger');", - zDb, SCHEMA_TABLE(iDb), zName, zName, zName, + "(type='table' OR type='index' OR type='trigger');", + zDb, SCHEMA_TABLE(iDb), zName, zName, zName, #ifndef SQLITE_OMIT_TRIGGER zName, #endif @@ -64197,7 +64548,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( ); #ifndef SQLITE_OMIT_AUTOINCREMENT - /* If the sqlite_sequence table exists in this database, then update + /* If the sqlite_sequence table exists in this database, then update ** it with the new table name. */ if( sqlite3FindTable(db, "sqlite_sequence", zDb) ){ @@ -64213,7 +64564,7 @@ SQLITE_PRIVATE void sqlite3AlterRenameTable( ** the temp database. */ if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){ - sqlite3NestedParse(pParse, + sqlite3NestedParse(pParse, "UPDATE sqlite_temp_master SET " "sql = sqlite_rename_trigger(sql, %Q), " "tbl_name = %Q " @@ -64308,7 +64659,7 @@ SQLITE_PRIVATE void sqlite3AlterFinishAddColumn(Parse *pParse, Token *pColDef){ } #endif - /* If the default value for the new column was specified with a + /* If the default value for the new column was specified with a ** literal NULL, then set pDflt to 0. This simplifies checking ** for an SQL NULL default below. */ @@ -64329,12 +64680,12 @@ SQLITE_PRIVATE void sqlite3AlterFinishAddColumn(Parse *pParse, Token *pColDef){ return; } if( (db->flags&SQLITE_ForeignKeys) && pNew->pFKey && pDflt ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "Cannot add a REFERENCES column with non-NULL default value"); return; } if( pCol->notNull && !pDflt ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "Cannot add a NOT NULL column with default value NULL"); return; } @@ -64362,10 +64713,10 @@ SQLITE_PRIVATE void sqlite3AlterFinishAddColumn(Parse *pParse, Token *pColDef){ while( zEnd>zCol && (*zEnd==';' || sqlite3Isspace(*zEnd)) ){ *zEnd-- = '\0'; } - sqlite3NestedParse(pParse, + sqlite3NestedParse(pParse, "UPDATE \"%w\".%s SET " "sql = substr(sql,1,%d) || ', ' || %Q || substr(sql,%d) " - "WHERE type = 'table' AND name = %Q", + "WHERE type = 'table' AND name = %Q", zDb, SCHEMA_TABLE(iDb), pNew->addColOffset, zCol, pNew->addColOffset+1, zTab ); @@ -64384,14 +64735,14 @@ SQLITE_PRIVATE void sqlite3AlterFinishAddColumn(Parse *pParse, Token *pColDef){ /* ** This function is called by the parser after the table-name in -** an "ALTER TABLE ADD" statement is parsed. Argument +** an "ALTER TABLE ADD" statement is parsed. Argument ** pSrc is the full-name of the table being altered. ** ** This routine makes a (partial) copy of the Table structure ** for the table being altered and sets Parse.pNewTable to point ** to it. Routines called by the parser as the column definition -** is parsed (i.e. sqlite3AddColumn()) add the new Column data to -** the copy. The copy of the Table structure is deleted by tokenize.c +** is parsed (i.e. sqlite3AddColumn()) add the new Column data to +** the copy. The copy of the Table structure is deleted by tokenize.c ** after parsing is finished. ** ** Routine sqlite3AlterFinishAddColumn() will be called to complete @@ -64490,8 +64841,6 @@ exit_begin_add_column: ** ************************************************************************* ** This file contains code associated with the ANALYZE command. -** -** @(#) $Id: analyze.c,v 1.52 2009/04/16 17:45:48 drh Exp $ */ #ifndef SQLITE_OMIT_ANALYZE @@ -64503,7 +64852,7 @@ exit_begin_add_column: ** ** If the sqlite_stat1 tables does not previously exist, it is created. ** Similarly, if the sqlite_stat2 table does not exist and the library -** is compiled with SQLITE_ENABLE_STAT2 defined, it is created. +** is compiled with SQLITE_ENABLE_STAT2 defined, it is created. ** ** Argument zWhere may be a pointer to a buffer containing a table name, ** or it may be a NULL pointer. If it is not NULL, then all entries in @@ -64543,9 +64892,9 @@ static void openStatTable( const char *zTab = aTable[i].zName; Table *pStat; if( (pStat = sqlite3FindTable(db, zTab, pDb->zName))==0 ){ - /* The sqlite_stat[12] table does not exist. Create it. Note that a - ** side-effect of the CREATE TABLE statement is to leave the rootpage - ** of the new table in register pParse->regRoot. This is important + /* The sqlite_stat[12] table does not exist. Create it. Note that a + ** side-effect of the CREATE TABLE statement is to leave the rootpage + ** of the new table in register pParse->regRoot. This is important ** because the OpenWrite opcode below will be needing it. */ sqlite3NestedParse(pParse, "CREATE TABLE %Q.%s(%s)", pDb->zName, zTab, aTable[i].zCols @@ -64553,7 +64902,7 @@ static void openStatTable( aRoot[i] = pParse->regRoot; aCreateTbl[i] = 1; }else{ - /* The table already exists. If zWhere is not NULL, delete all entries + /* The table already exists. If zWhere is not NULL, delete all entries ** associated with the table zWhere. If zWhere is NULL, delete the ** entire contents of the table. */ aRoot[i] = pStat->tnum; @@ -64679,18 +65028,18 @@ static void analyzeOneTable( /* The block of memory cells initialized here is used as follows. ** - ** iMem: + ** iMem: ** The total number of rows in the table. ** - ** iMem+1 .. iMem+nCol: - ** Number of distinct entries in index considering the - ** left-most N columns only, where N is between 1 and nCol, + ** iMem+1 .. iMem+nCol: + ** Number of distinct entries in index considering the + ** left-most N columns only, where N is between 1 and nCol, ** inclusive. ** - ** iMem+nCol+1 .. Mem+2*nCol: + ** iMem+nCol+1 .. Mem+2*nCol: ** Previous value of indexed columns, from left to right. ** - ** Cells iMem through iMem+nCol are initialized to 0. The others are + ** Cells iMem through iMem+nCol are initialized to 0. The others are ** initialized to contain an SQL NULL. */ for(i=0; i<=nCol; i++){ @@ -64715,7 +65064,7 @@ static void analyzeOneTable( ** value that should be stored in the sqlite_stat2 table. If so, ** store it. */ int ne = sqlite3VdbeAddOp3(v, OP_Ne, regRecno, 0, regSamplerecno); - assert( regTabname+1==regIdxname + assert( regTabname+1==regIdxname && regTabname+2==regSampleno && regTabname+3==regCol ); @@ -64747,8 +65096,8 @@ static void analyzeOneTable( sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL); } if( db->mallocFailed ){ - /* If a malloc failure has occurred, then the result of the expression - ** passed as the second argument to the call to sqlite3VdbeJumpHere() + /* If a malloc failure has occurred, then the result of the expression + ** passed as the second argument to the call to sqlite3VdbeJumpHere() ** below may be negative. Which causes an assert() to fail (or an ** out-of-bounds write if SQLITE_DEBUG is not defined). */ return; @@ -64779,7 +65128,7 @@ static void analyzeOneTable( ** ** I = (K+D-1)/D ** - ** If K==0 then no entry is made into the sqlite_stat1 table. + ** If K==0 then no entry is made into the sqlite_stat1 table. ** If K>0 then it is always the case the D>0 so division by zero ** is never possible. */ @@ -64916,7 +65265,7 @@ SQLITE_PRIVATE void sqlite3Analyze(Parse *pParse, Token *pName1, Token *pName2){ analyzeTable(pParse, pTab); } } - } + } } } @@ -64932,7 +65281,7 @@ struct analysisInfo { /* ** This callback is invoked once for each index when reading the -** sqlite_stat1 table. +** sqlite_stat1 table. ** ** argv[0] = name of the index ** argv[1] = results of analysis - on integer for each column @@ -64997,11 +65346,11 @@ SQLITE_PRIVATE void sqlite3DeleteIndexSamples(Index *pIdx){ ** Index.aSample[] arrays. ** ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR -** is returned. In this case, even if SQLITE_ENABLE_STAT2 was defined -** during compilation and the sqlite_stat2 table is present, no data is +** is returned. In this case, even if SQLITE_ENABLE_STAT2 was defined +** during compilation and the sqlite_stat2 table is present, no data is ** read from it. ** -** If SQLITE_ENABLE_STAT2 was defined during compilation and the +** If SQLITE_ENABLE_STAT2 was defined during compilation and the ** sqlite_stat2 table is not present in the database, SQLITE_ERROR is ** returned. However, in this case, data is read from the sqlite_stat1 ** table (if it is present) before returning. @@ -65035,7 +65384,7 @@ SQLITE_PRIVATE int sqlite3AnalysisLoad(sqlite3 *db, int iDb){ } /* Load new statistics out of the sqlite_stat1 table */ - zSql = sqlite3MPrintf(db, + zSql = sqlite3MPrintf(db, "SELECT idx, stat FROM %Q.sqlite_stat1", sInfo.zDatabase); if( zSql==0 ){ rc = SQLITE_NOMEM; @@ -65055,7 +65404,7 @@ SQLITE_PRIVATE int sqlite3AnalysisLoad(sqlite3 *db, int iDb){ if( rc==SQLITE_OK ){ sqlite3_stmt *pStmt = 0; - zSql = sqlite3MPrintf(db, + zSql = sqlite3MPrintf(db, "SELECT idx,sampleno,sample FROM %Q.sqlite_stat2", sInfo.zDatabase); if( !zSql ){ rc = SQLITE_NOMEM; @@ -65145,8 +65494,6 @@ SQLITE_PRIVATE int sqlite3AnalysisLoad(sqlite3 *db, int iDb){ ** ************************************************************************* ** This file contains code used to implement the ATTACH and DETACH commands. -** -** $Id: attach.c,v 1.93 2009/05/31 21:21:41 drh Exp $ */ #ifndef SQLITE_OMIT_ATTACH @@ -65223,7 +65570,7 @@ static void attachFunc( ** * Specified database name already being used. */ if( db->nDb>=db->aLimit[SQLITE_LIMIT_ATTACHED]+2 ){ - zErrDyn = sqlite3MPrintf(db, "too many attached databases - max %d", + zErrDyn = sqlite3MPrintf(db, "too many attached databases - max %d", db->aLimit[SQLITE_LIMIT_ATTACHED] ); goto attach_error; @@ -65273,7 +65620,7 @@ static void attachFunc( if( !aNew->pSchema ){ rc = SQLITE_NOMEM; }else if( aNew->pSchema->file_format && aNew->pSchema->enc!=ENC(db) ){ - zErrDyn = sqlite3MPrintf(db, + zErrDyn = sqlite3MPrintf(db, "attached databases must use the same text encoding as main database"); rc = SQLITE_ERROR; } @@ -65297,7 +65644,7 @@ static void attachFunc( zErrDyn = sqlite3DbStrDup(db, "Invalid key value"); rc = SQLITE_ERROR; break; - + case SQLITE_TEXT: case SQLITE_BLOB: nKey = sqlite3_value_bytes(argv[2]); @@ -65315,7 +65662,7 @@ static void attachFunc( #endif /* If the file was opened successfully, read the schema for the new database. - ** If this fails, or if opening the file failed, then close the file and + ** If this fails, or if opening the file failed, then close the file and ** remove the entry from the db->aDb[] array. i.e. put everything back the way ** we found it. */ @@ -65345,7 +65692,7 @@ static void attachFunc( } goto attach_error; } - + return; attach_error: @@ -65435,7 +65782,7 @@ static void codeAttach( memset(&sName, 0, sizeof(NameContext)); sName.pParse = pParse; - if( + if( SQLITE_OK!=(rc = resolveAttachExpr(&sName, pFilename)) || SQLITE_OK!=(rc = resolveAttachExpr(&sName, pDbname)) || SQLITE_OK!=(rc = resolveAttachExpr(&sName, pKey)) @@ -65477,7 +65824,7 @@ static void codeAttach( */ sqlite3VdbeAddOp1(v, OP_Expire, (type==SQLITE_ATTACH)); } - + attach_end: sqlite3ExprDelete(db, pFilename); sqlite3ExprDelete(db, pDbname); @@ -65687,8 +66034,6 @@ SQLITE_PRIVATE int sqlite3FixTriggerStep( ** API. This facility is an optional feature of the library. Embedded ** systems that do not need this facility may omit it by recompiling ** the library with -DSQLITE_OMIT_AUTHORIZATION=1 -** -** $Id: auth.c,v 1.32 2009/07/02 18:40:35 danielk1977 Exp $ */ /* @@ -65799,10 +66144,10 @@ SQLITE_PRIVATE int sqlite3AuthReadCol( /* ** The pExpr should be a TK_COLUMN expression. The table referred to -** is in pTabList or else it is the NEW or OLD table of a trigger. +** is in pTabList or else it is the NEW or OLD table of a trigger. ** Check to see if it is OK to read this particular column. ** -** If the auth function returns SQLITE_IGNORE, change the TK_COLUMN +** If the auth function returns SQLITE_IGNORE, change the TK_COLUMN ** instruction into a TK_NULL. If the auth function returns SQLITE_DENY, ** then generate an error. */ @@ -65901,7 +66246,7 @@ SQLITE_PRIVATE int sqlite3AuthCheck( */ SQLITE_PRIVATE void sqlite3AuthContextPush( Parse *pParse, - AuthContext *pContext, + AuthContext *pContext, const char *zContext ){ assert( pParse ); @@ -65948,8 +66293,6 @@ SQLITE_PRIVATE void sqlite3AuthContextPop(AuthContext *pContext){ ** BEGIN TRANSACTION ** COMMIT ** ROLLBACK -** -** $Id: build.c,v 1.557 2009/07/24 17:58:53 danielk1977 Exp $ */ /* @@ -65974,7 +66317,7 @@ struct TableLock { }; /* -** Record the fact that we want to lock a table at run-time. +** Record the fact that we want to lock a table at run-time. ** ** The table to be locked has root page iTab and is found in database iDb. ** A read or a write lock can be taken depending on isWritelock. @@ -66025,7 +66368,7 @@ SQLITE_PRIVATE void sqlite3TableLock( */ static void codeTableLocks(Parse *pParse){ int i; - Vdbe *pVdbe; + Vdbe *pVdbe; pVdbe = sqlite3GetVdbe(pParse); assert( pVdbe!=0 ); /* sqlite3GetVdbe cannot fail: VDBE already allocated */ @@ -66064,7 +66407,7 @@ SQLITE_PRIVATE void sqlite3FinishCoding(Parse *pParse){ ** vdbe program */ v = sqlite3GetVdbe(pParse); - assert( !pParse->isMultiWrite + assert( !pParse->isMultiWrite || sqlite3VdbeAssertMayAbort(v, pParse->mayAbort)); if( v ){ sqlite3VdbeAddOp0(v, OP_Halt); @@ -66098,8 +66441,8 @@ SQLITE_PRIVATE void sqlite3FinishCoding(Parse *pParse){ } #endif - /* Once all the cookies have been verified and transactions opened, - ** obtain the required table-locks. This is a no-op unless the + /* Once all the cookies have been verified and transactions opened, + ** obtain the required table-locks. This is a no-op unless the ** shared-cache feature is enabled. */ codeTableLocks(pParse); @@ -66244,7 +66587,7 @@ SQLITE_PRIVATE Table *sqlite3LocateTable( } /* -** Locate the in-memory structure that describes +** Locate the in-memory structure that describes ** a particular index given the name of that index ** and the name of the database that contains the index. ** Return NULL if not found. @@ -66425,7 +66768,7 @@ static void sqliteResetColumnNames(Table *pTable){ ** ** This routine just deletes the data structure. It does not unlink ** the table data structure from the hash table. But it does destroy -** memory structures of the indices and foreign keys associated with +** memory structures of the indices and foreign keys associated with ** the table. */ SQLITE_PRIVATE void sqlite3DeleteTable(Table *pTable){ @@ -66535,7 +66878,7 @@ SQLITE_PRIVATE int sqlite3FindDbName(sqlite3 *db, const char *zName){ Db *pDb; int n = sqlite3Strlen30(zName); for(i=(db->nDb-1), pDb=&db->aDb[i]; i>=0; i--, pDb--){ - if( (!OMIT_TEMPDB || i!=1 ) && n==sqlite3Strlen30(pDb->zName) && + if( (!OMIT_TEMPDB || i!=1 ) && n==sqlite3Strlen30(pDb->zName) && 0==sqlite3StrICmp(pDb->zName, zName) ){ break; } @@ -66547,7 +66890,7 @@ SQLITE_PRIVATE int sqlite3FindDbName(sqlite3 *db, const char *zName){ /* ** The token *pName contains the name of a database (either "main" or ** "temp" or the name of an attached db). This routine returns the -** index of the named database in db->aDb[], or -1 if the named db +** index of the named database in db->aDb[], or -1 if the named db ** does not exist. */ SQLITE_PRIVATE int sqlite3FindDb(sqlite3 *db, Token *pName){ @@ -66563,7 +66906,7 @@ SQLITE_PRIVATE int sqlite3FindDb(sqlite3 *db, Token *pName){ ** pName1 and pName2. If the table name was fully qualified, for example: ** ** CREATE TABLE xxx.yyy (...); -** +** ** Then pName1 is set to "xxx" and pName2 "yyy". On the other hand if ** the table name is not fully qualified, i.e.: ** @@ -66613,7 +66956,7 @@ SQLITE_PRIVATE int sqlite3TwoPartName( ** is reserved for internal use. */ SQLITE_PRIVATE int sqlite3CheckObjectName(Parse *pParse, const char *zName){ - if( !pParse->db->init.busy && pParse->nested==0 + if( !pParse->db->init.busy && pParse->nested==0 && (pParse->db->flags & SQLITE_WriteSchema)==0 && 0==sqlite3StrNICmp(zName, "sqlite_", 7) ){ sqlite3ErrorMsg(pParse, "object name reserved for internal use: %s", zName); @@ -66658,7 +67001,7 @@ SQLITE_PRIVATE void sqlite3StartTable( ** pName1 and pName2. If the table name was fully qualified, for example: ** ** CREATE TABLE xxx.yyy (...); - ** + ** ** Then pName1 is set to "xxx" and pName2 "yyy". On the other hand if ** the table name is not fully qualified, i.e.: ** @@ -66767,7 +67110,7 @@ SQLITE_PRIVATE void sqlite3StartTable( ** the SQLITE_MASTER table. Note in particular that we must go ahead ** and allocate the record number for the table entry now. Before any ** PRIMARY KEY or UNIQUE keywords are parsed. Those keywords will cause - ** indices to be created and the table record must come before the + ** indices to be created and the table record must come before the ** indices. Hence, the record number for the table must be allocated ** now. */ @@ -66783,7 +67126,7 @@ SQLITE_PRIVATE void sqlite3StartTable( } #endif - /* If the file format and encoding in the database have not been set, + /* If the file format and encoding in the database have not been set, ** set them now. */ reg1 = pParse->regRowid = ++pParse->nMem; @@ -66889,7 +67232,7 @@ SQLITE_PRIVATE void sqlite3AddColumn(Parse *pParse, Token *pName){ pCol = &p->aCol[p->nCol]; memset(pCol, 0, sizeof(p->aCol[0])); pCol->zName = z; - + /* If there is no type specified, columns have the default affinity ** 'NONE'. If there is a type specified, then sqlite3AddColumnType() will ** be called next to set pCol->affinity correctly. @@ -66915,11 +67258,11 @@ SQLITE_PRIVATE void sqlite3AddNotNull(Parse *pParse, int onError){ ** Scan the column type name zType (length nType) and return the ** associated affinity type. ** -** This routine does a case-independent search of zType for the +** This routine does a case-independent search of zType for the ** substrings in the following table. If one of the substrings is ** found, the corresponding affinity is returned. If zType contains -** more than one of the substrings, entries toward the top of -** the table take priority. For example, if zType is 'BLOBINT', +** more than one of the substrings, entries toward the top of +** the table take priority. For example, if zType is 'BLOBINT', ** SQLITE_AFF_INTEGER is returned. ** ** Substring | Affinity @@ -66944,7 +67287,7 @@ SQLITE_PRIVATE char sqlite3AffinityType(const char *zIn){ h = (h<<8) + sqlite3UpperToLower[(*zIn)&0xff]; zIn++; if( h==(('c'<<24)+('h'<<16)+('a'<<8)+'r') ){ /* CHAR */ - aff = SQLITE_AFF_TEXT; + aff = SQLITE_AFF_TEXT; }else if( h==(('c'<<24)+('l'<<16)+('o'<<8)+'b') ){ /* CLOB */ aff = SQLITE_AFF_TEXT; }else if( h==(('t'<<24)+('e'<<16)+('x'<<8)+'t') ){ /* TEXT */ @@ -66980,7 +67323,7 @@ SQLITE_PRIVATE char sqlite3AffinityType(const char *zIn){ ** in the sequence. Use this information to construct a string ** that contains the typename of the column and store that string ** in zType. -*/ +*/ SQLITE_PRIVATE void sqlite3AddColumnType(Parse *pParse, Token *pType){ Table *p; Column *pCol; @@ -67029,7 +67372,7 @@ SQLITE_PRIVATE void sqlite3AddDefaultValue(Parse *pParse, ExprSpan *pSpan){ } /* -** Designate the PRIMARY KEY for the table. pList is a list of names +** Designate the PRIMARY KEY for the table. pList is a list of names ** of columns that form the primary key. If pList is NULL, then the ** most recently added column of the table is the primary key. ** @@ -67058,7 +67401,7 @@ SQLITE_PRIVATE void sqlite3AddPrimaryKey( int iCol = -1, i; if( pTab==0 || IN_DECLARE_VTAB ) goto primary_key_exit; if( pTab->tabFlags & TF_HasPrimaryKey ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "table \"%s\" has more than one primary key", pTab->zName); goto primary_key_exit; } @@ -67145,7 +67488,7 @@ SQLITE_PRIVATE void sqlite3AddCollateType(Parse *pParse, Token *pToken){ if( sqlite3LocateCollSeq(pParse, zColl) ){ Index *pIdx; p->aCol[i].zColl = zColl; - + /* If the column is declared as " PRIMARY KEY COLLATE ", ** then an index may have been created on this column before the ** collation type was added. Correct this if it is the case. @@ -67241,13 +67584,13 @@ static int identLength(const char *z){ } /* -** The first parameter is a pointer to an output buffer. The second +** The first parameter is a pointer to an output buffer. The second ** parameter is a pointer to an integer that contains the offset at ** which to write into the output buffer. This function copies the ** nul-terminated string pointed to by the third parameter, zSignedIdent, ** to the specified offset in the buffer and updates *pIdx to refer ** to the first byte after the last byte written before returning. -** +** ** If the string zSignedIdent consists entirely of alpha-numeric ** characters, does not begin with a digit and is not an SQL keyword, ** then it is copied to the output buffer exactly as it is. Otherwise, @@ -67291,7 +67634,7 @@ static char *createTableStmt(sqlite3 *db, Table *p){ n += identLength(pCol->zName) + 5; } n += identLength(p->zName); - if( n<50 ){ + if( n<50 ){ zSep = ""; zSep2 = ","; zEnd = ")"; @@ -67332,10 +67675,10 @@ static char *createTableStmt(sqlite3 *db, Table *p){ testcase( pCol->affinity==SQLITE_AFF_NUMERIC ); testcase( pCol->affinity==SQLITE_AFF_INTEGER ); testcase( pCol->affinity==SQLITE_AFF_REAL ); - + zType = azType[pCol->affinity - SQLITE_AFF_TEXT]; len = sqlite3Strlen30(zType); - assert( pCol->affinity==SQLITE_AFF_NONE + assert( pCol->affinity==SQLITE_AFF_NONE || pCol->affinity==sqlite3AffinityType(zType) ); memcpy(&zStmt[k], zType, len); k += len; @@ -67361,7 +67704,7 @@ static char *createTableStmt(sqlite3 *db, Table *p){ ** the sqlite_master table. We do not want to create it again. ** ** If the pSelect argument is not NULL, it means that this routine -** was called to create a table generated from a +** was called to create a table generated from a ** "CREATE TABLE ... AS SELECT ..." statement. The column names of ** the new table will match the result set of the SELECT. */ @@ -67435,7 +67778,7 @@ SQLITE_PRIVATE void sqlite3EndTable( sqlite3VdbeAddOp1(v, OP_Close, 0); - /* + /* ** Initialize zType for the new view or table. */ if( p->pSelect==0 ){ @@ -67491,12 +67834,12 @@ SQLITE_PRIVATE void sqlite3EndTable( zStmt = createTableStmt(db, p); }else{ n = (int)(pEnd->z - pParse->sNameToken.z) + 1; - zStmt = sqlite3MPrintf(db, + zStmt = sqlite3MPrintf(db, "CREATE %s %.*s", zType2, n, pParse->sNameToken.z ); } - /* A slot for the record has already been allocated in the + /* A slot for the record has already been allocated in the ** SQLITE_MASTER table. We just need to update that slot with all ** the information we've collected. */ @@ -67684,7 +68027,7 @@ SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){ ** Actually, the error above is now caught prior to reaching this point. ** But the following test is still important as it does come up ** in the following: - ** + ** ** CREATE TABLE main.ex1(a); ** CREATE TEMP VIEW ex1 AS SELECT a FROM ex1; ** SELECT * FROM temp.ex1; @@ -67737,7 +68080,7 @@ SQLITE_PRIVATE int sqlite3ViewGetColumnNames(Parse *pParse, Table *pTable){ nErr++; } #endif /* SQLITE_OMIT_VIEW */ - return nErr; + return nErr; } #endif /* !defined(SQLITE_OMIT_VIEW) || !defined(SQLITE_OMIT_VIRTUALTABLE) */ @@ -67770,7 +68113,7 @@ static void sqliteViewResetAll(sqlite3 *db, int idx){ ** on tables and/or indices that are the process of being deleted. ** If you are unlucky, one of those deleted indices or tables might ** have the same rootpage number as the real table or index that is -** being moved. So we cannot stop searching after the first match +** being moved. So we cannot stop searching after the first match ** because the first match might be for one of the deleted indices ** or tables and not the table/index that is actually being moved. ** We must continue looping until all tables and indices with @@ -67804,7 +68147,7 @@ SQLITE_PRIVATE void sqlite3RootPageMoved(Db *pDb, int iFrom, int iTo){ ** Also write code to modify the sqlite_master table and internal schema ** if a root-page of another table is moved by the btree-layer whilst ** erasing iTable (this can happen with an auto-vacuum database). -*/ +*/ static void destroyRootPage(Parse *pParse, int iTable, int iDb){ Vdbe *v = sqlite3GetVdbe(pParse); int r1 = sqlite3GetTempReg(pParse); @@ -67820,7 +68163,7 @@ static void destroyRootPage(Parse *pParse, int iTable, int iDb){ ** is in register NNN. See grammar rules associated with the TK_REGISTER ** token for additional information. */ - sqlite3NestedParse(pParse, + sqlite3NestedParse(pParse, "UPDATE %Q.%s SET rootpage=%d WHERE #%d AND rootpage=#%d", pParse->db->aDb[iDb].zName, SCHEMA_TABLE(iDb), iTable, r1, r1); #endif @@ -67844,7 +68187,7 @@ static void destroyTable(Parse *pParse, Table *pTab){ #else /* If the database may be auto-vacuum capable (if SQLITE_OMIT_AUTOVACUUM ** is not defined), then it is important to call OP_Destroy on the - ** table and index root-pages in order, starting with the numerically + ** table and index root-pages in order, starting with the numerically ** largest root-page number. This guarantees that none of the root-pages ** to be destroyed is relocated by an earlier OP_Destroy. i.e. if the ** following were coded: @@ -67854,7 +68197,7 @@ static void destroyTable(Parse *pParse, Table *pTab){ ** OP_Destroy 5 0 ** ** and root page 5 happened to be the largest root-page number in the - ** database, then root page 5 would be moved to page 4 by the + ** database, then root page 5 would be moved to page 4 by the ** "OP_Destroy 4 0" opcode. The subsequent "OP_Destroy 5 0" would hit ** a free-list page. */ @@ -67901,7 +68244,7 @@ SQLITE_PRIVATE void sqlite3DropTable(Parse *pParse, SrcList *pName, int isView, } assert( pParse->nErr==0 ); assert( pName->nSrc==1 ); - pTab = sqlite3LocateTable(pParse, isView, + pTab = sqlite3LocateTable(pParse, isView, pName->a[0].zName, pName->a[0].zDatabase); if( pTab==0 ){ @@ -67995,7 +68338,7 @@ SQLITE_PRIVATE void sqlite3DropTable(Parse *pParse, SrcList *pName, int isView, */ pTrigger = sqlite3TriggerList(pParse, pTab); while( pTrigger ){ - assert( pTrigger->pSchema==pTab->pSchema || + assert( pTrigger->pSchema==pTab->pSchema || pTrigger->pSchema==db->aDb[1].pSchema ); sqlite3DropTriggerPtr(pParse, pTrigger); pTrigger = pTrigger->pNext; @@ -68022,7 +68365,7 @@ SQLITE_PRIVATE void sqlite3DropTable(Parse *pParse, SrcList *pName, int isView, ** created in the temp database that refers to a table in another ** database. */ - sqlite3NestedParse(pParse, + sqlite3NestedParse(pParse, "DELETE FROM %Q.%s WHERE tbl_name=%Q and type!='trigger'", pDb->zName, SCHEMA_TABLE(iDb), pTab->zName); @@ -68136,8 +68479,8 @@ SQLITE_PRIVATE void sqlite3CreateForeignKey( } } if( j>=p->nCol ){ - sqlite3ErrorMsg(pParse, - "unknown column \"%s\" in foreign key definition", + sqlite3ErrorMsg(pParse, + "unknown column \"%s\" in foreign key definition", pFromCol->a[i].zName); goto fk_end; } @@ -68156,7 +68499,7 @@ SQLITE_PRIVATE void sqlite3CreateForeignKey( pFKey->aAction[0] = (u8)(flags & 0xff); /* ON DELETE action */ pFKey->aAction[1] = (u8)((flags >> 8 ) & 0xff); /* ON UPDATE action */ - pNextTo = (FKey *)sqlite3HashInsert(&p->pSchema->fkeyHash, + pNextTo = (FKey *)sqlite3HashInsert(&p->pSchema->fkeyHash, pFKey->zTo, sqlite3Strlen30(pFKey->zTo), (void *)pFKey ); if( pNextTo==pFKey ){ @@ -68241,7 +68584,7 @@ static void sqlite3RefillIndex(Parse *pParse, Index *pIndex, int memRootPage){ sqlite3VdbeAddOp2(v, OP_Clear, tnum, iDb); } pKey = sqlite3IndexKeyinfo(pParse, pIndex); - sqlite3VdbeAddOp4(v, OP_OpenWrite, iIdx, tnum, iDb, + sqlite3VdbeAddOp4(v, OP_OpenWrite, iIdx, tnum, iDb, (char *)pKey, P4_KEYINFO_HANDOFF); if( memRootPage>=0 ){ sqlite3VdbeChangeP5(v, 1); @@ -68258,7 +68601,7 @@ static void sqlite3RefillIndex(Parse *pParse, Index *pIndex, int memRootPage){ /* The registers accessed by the OP_IsUnique opcode were allocated ** using sqlite3GetTempRange() inside of the sqlite3GenerateIndexKey() ** call above. Just before that function was freed they were released - ** (made available to the compiler for reuse) using + ** (made available to the compiler for reuse) using ** sqlite3ReleaseTempRange(). So in some ways having the OP_IsUnique ** opcode use the values stored within seems dangerous. However, since ** we can be sure that no other temp registers have been allocated @@ -68278,8 +68621,8 @@ static void sqlite3RefillIndex(Parse *pParse, Index *pIndex, int memRootPage){ } /* -** Create a new index for an SQL table. pName1.pName2 is the name of the index -** and pTblList is the name of the table that is to be indexed. Both will +** Create a new index for an SQL table. pName1.pName2 is the name of the index +** and pTblList is the name of the table that is to be indexed. Both will ** be NULL for a primary key or an index that is created to satisfy a ** UNIQUE constraint. If pTable and pIndex are NULL, use pParse->pNewTable ** as the table to be indexed. pParse->pNewTable is a table that is @@ -68287,7 +68630,7 @@ static void sqlite3RefillIndex(Parse *pParse, Index *pIndex, int memRootPage){ ** ** pList is a list of columns to be indexed. pList will be NULL if this ** is a primary key or unique-constraint on the most recent column added -** to the table currently under construction. +** to the table currently under construction. ** ** If the index is created successfully, return a pointer to the new Index ** structure. This is used by sqlite3AddPrimaryKey() to mark the index @@ -68337,7 +68680,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( */ if( pTblName!=0 ){ - /* Use the two-part index name to determine the database + /* Use the two-part index name to determine the database ** to search for the table. 'Fix' the table name to this db ** before looking up the table. */ @@ -68365,7 +68708,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( ** sqlite3FixSrcList can never fail. */ assert(0); } - pTab = sqlite3LocateTable(pParse, 0, pTblName->a[0].zName, + pTab = sqlite3LocateTable(pParse, 0, pTblName->a[0].zName, pTblName->a[0].zDatabase); if( !pTab || db->mallocFailed ) goto exit_create_index; assert( db->aDb[iDb].pSchema==pTab->pSchema ); @@ -68379,7 +68722,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( assert( pTab!=0 ); assert( pParse->nErr==0 ); - if( sqlite3StrNICmp(pTab->zName, "sqlite_", 7)==0 + if( sqlite3StrNICmp(pTab->zName, "sqlite_", 7)==0 && memcmp(&pTab->zName[7],"altertab_",9)!=0 ){ sqlite3ErrorMsg(pParse, "table %s may not be indexed", pTab->zName); goto exit_create_index; @@ -68399,7 +68742,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( /* ** Find the name of the index. Make sure there is not already another - ** index or table with the same name. + ** index or table with the same name. ** ** Exception: If we are reading the names of permanent indices from the ** sqlite_master table (because some other process changed the schema) and @@ -68482,12 +68825,12 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( } } - /* - ** Allocate the index structure. + /* + ** Allocate the index structure. */ nName = sqlite3Strlen30(zName); nCol = pList->nExpr; - pIndex = sqlite3DbMallocZero(db, + pIndex = sqlite3DbMallocZero(db, sizeof(Index) + /* Index structure */ sizeof(int)*nCol + /* Index.aiColumn */ sizeof(int)*(nCol+1) + /* Index.aiRowEst */ @@ -68527,7 +68870,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( ** TODO: Add a test to make sure that the same column is not named ** more than once within the same index. Only the first instance of ** the column will ever be used by the optimizer. Note that using the - ** same column more than once cannot be an error because that would + ** same column more than once cannot be an error because that would ** break backwards compatibility - it needs to be a warning. */ for(i=0, pListItem=pList->a; inExpr; i++, pListItem++){ @@ -68616,13 +68959,13 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( if( pIdx->onError!=pIndex->onError ){ /* This constraint creates the same index as a previous ** constraint specified somewhere in the CREATE TABLE statement. - ** However the ON CONFLICT clauses are different. If both this + ** However the ON CONFLICT clauses are different. If both this ** constraint and the previous equivalent constraint have explicit ** ON CONFLICT clauses this is an error. Otherwise, use the ** explicitly specified behaviour for the index. */ if( !(pIdx->onError==OE_Default || pIndex->onError==OE_Default) ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "conflicting ON CONFLICT clauses specified", 0); } if( pIdx->onError==OE_Default ){ @@ -68635,11 +68978,11 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( } /* Link the new Index structure to its table and to the other - ** in-memory database structures. + ** in-memory database structures. */ if( db->init.busy ){ Index *p; - p = sqlite3HashInsert(&pIndex->pSchema->idxHash, + p = sqlite3HashInsert(&pIndex->pSchema->idxHash, pIndex->zName, sqlite3Strlen30(pIndex->zName), pIndex); if( p ){ @@ -68657,8 +69000,8 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( ** involves writing the index into the master table and filling in the ** index with the current table contents. ** - ** The db->init.busy is 0 when the user first enters a CREATE INDEX - ** command. db->init.busy is 1 when a database is opened and + ** The db->init.busy is 0 when the user first enters a CREATE INDEX + ** command. db->init.busy is 1 when a database is opened and ** CREATE INDEX statements are read out of the master table. In ** the latter case the index already exists on disk, which is why ** we don't want to recreate it. @@ -68700,7 +69043,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( /* Add an entry in sqlite_master for this index */ - sqlite3NestedParse(pParse, + sqlite3NestedParse(pParse, "INSERT INTO %Q.%s VALUES('index',%Q,%Q,#%d,%Q);", db->aDb[iDb].zName, SCHEMA_TABLE(iDb), pIndex->zName, @@ -68726,7 +69069,7 @@ SQLITE_PRIVATE Index *sqlite3CreateIndex( ** sure all indices labeled OE_Replace come after all those labeled ** OE_Ignore. This is necessary for the correct constraint check ** processing (in sqlite3GenerateConstraintChecks()) as part of - ** UPDATE and INSERT statements. + ** UPDATE and INSERT statements. */ if( db->init.busy || pTblName==0 ){ if( onError!=OE_Replace || pTab->pIndex==0 @@ -69044,7 +69387,7 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListEnlarge( ** database name prefix. Like this: "database.table". The pDatabase ** points to the table name and the pTable points to the database name. ** The SrcList.a[].zName field is filled with the table name which might -** come from pTable (if pDatabase is NULL) or from pDatabase. +** come from pTable (if pDatabase is NULL) or from pDatabase. ** SrcList.a[].zDatabase is filled with the database name from pTable, ** or with NULL if no database is specified. ** @@ -69164,7 +69507,7 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListAppendFromTerm( struct SrcList_item *pItem; sqlite3 *db = pParse->db; if( !p && (pOn || pUsing) ){ - sqlite3ErrorMsg(pParse, "a JOIN clause is required before %s", + sqlite3ErrorMsg(pParse, "a JOIN clause is required before %s", (pOn ? "ON" : "USING") ); goto append_from_error; @@ -69192,7 +69535,7 @@ SQLITE_PRIVATE SrcList *sqlite3SrcListAppendFromTerm( } /* -** Add an INDEXED BY or NOT INDEXED clause to the most recently added +** Add an INDEXED BY or NOT INDEXED clause to the most recently added ** element of the source-list passed as the second argument. */ SQLITE_PRIVATE void sqlite3SrcListIndexedBy(Parse *pParse, SrcList *p, Token *pIndexedBy){ @@ -69201,7 +69544,7 @@ SQLITE_PRIVATE void sqlite3SrcListIndexedBy(Parse *pParse, SrcList *p, Token *pI struct SrcList_item *pItem = &p->a[p->nSrc-1]; assert( pItem->notIndexed==0 && pItem->zIndex==0 ); if( pIndexedBy->n==1 && !pIndexedBy->z ){ - /* A "NOT INDEXED" clause was supplied. See parse.y + /* A "NOT INDEXED" clause was supplied. See parse.y ** construct "indexed_opt" for details. */ pItem->notIndexed = 1; }else{ @@ -69303,7 +69646,7 @@ SQLITE_PRIVATE void sqlite3RollbackTransaction(Parse *pParse){ /* ** This function is called by the parser when it parses a command to create, -** release or rollback an SQL savepoint. +** release or rollback an SQL savepoint. */ SQLITE_PRIVATE void sqlite3Savepoint(Parse *pParse, int op, Token *pName){ char *zName = sqlite3NameFromToken(pParse->db, pName); @@ -69329,7 +69672,7 @@ SQLITE_PRIVATE int sqlite3OpenTempDatabase(Parse *pParse){ sqlite3 *db = pParse->db; if( db->aDb[1].pBt==0 && !pParse->explain ){ int rc; - static const int flags = + static const int flags = SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_EXCLUSIVE | @@ -69432,9 +69775,9 @@ SQLITE_PRIVATE void sqlite3MultiWrite(Parse *pParse){ pToplevel->isMultiWrite = 1; } -/* +/* ** The code generator calls this routine if is discovers that it is -** possible to abort a statement prior to completion. In order to +** possible to abort a statement prior to completion. In order to ** perform this abort without corrupting the database, we need to make ** sure that the statement is protected by a statement transaction. ** @@ -69443,7 +69786,7 @@ SQLITE_PRIVATE void sqlite3MultiWrite(Parse *pParse){ ** such that the abort must occur after the multiwrite. This makes ** some statements involving the REPLACE conflict resolution algorithm ** go a little faster. But taking advantage of this time dependency -** makes it more difficult to prove that the code is correct (in +** makes it more difficult to prove that the code is correct (in ** particular, it prevents us from writing an effective ** implementation of sqlite3AssertMayAbort()) and so we have chosen ** to take the safe route and skip the optimization. @@ -69599,8 +69942,8 @@ SQLITE_PRIVATE void sqlite3Reindex(Parse *pParse, Token *pName1, Token *pName2){ ** with OP_OpenRead or OP_OpenWrite to access database index pIdx. ** ** If successful, a pointer to the new structure is returned. In this case -** the caller is responsible for calling sqlite3DbFree(db, ) on the returned -** pointer. If an error occurs (out of memory or missing collation +** the caller is responsible for calling sqlite3DbFree(db, ) on the returned +** pointer. If an error occurs (out of memory or missing collation ** sequence), NULL is returned and the state of pParse updated to reflect ** the error. */ @@ -69634,7 +69977,7 @@ SQLITE_PRIVATE KeyInfo *sqlite3IndexKeyinfo(Parse *pParse, Index *pIdx){ /************** End of build.c ***********************************************/ /************** Begin file callback.c ****************************************/ /* -** 2005 May 23 +** 2005 May 23 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: @@ -69647,8 +69990,6 @@ SQLITE_PRIVATE KeyInfo *sqlite3IndexKeyinfo(Parse *pParse, Index *pIdx){ ** ** This file contains functions used to access the internal hash tables ** of user defined functions and collation sequences. -** -** $Id: callback.c,v 1.42 2009/06/17 00:35:31 drh Exp $ */ @@ -69705,8 +70046,8 @@ static int synthCollSeq(sqlite3 *db, CollSeq *pColl){ ** This function is responsible for invoking the collation factory callback ** or substituting a collation sequence of a different encoding when the ** requested collation sequence is not available in the desired encoding. -** -** If it is not NULL, then pColl must point to the database native encoding +** +** If it is not NULL, then pColl must point to the database native encoding ** collation sequence with name zName, length nName. ** ** The return value is either the collation sequence to be used in database @@ -69748,7 +70089,7 @@ SQLITE_PRIVATE CollSeq *sqlite3GetCollSeq( ** that have not been defined by sqlite3_create_collation() etc. ** ** If required, this routine calls the 'collation needed' callback to -** request a definition of the collating sequence. If this doesn't work, +** request a definition of the collating sequence. If this doesn't work, ** an equivalent collating sequence that uses a text encoding different ** from the main database is substituted, if one is available. */ @@ -69805,7 +70146,7 @@ static CollSeq *findCollSeqEntry( pColl[0].zName[nName] = 0; pDel = sqlite3HashInsert(&db->aCollSeq, pColl[0].zName, nName, pColl); - /* If a malloc() failure occurred in sqlite3HashInsert(), it will + /* If a malloc() failure occurred in sqlite3HashInsert(), it will ** return the pColl pointer to be deleted (because it wasn't added ** to the hash table). */ @@ -69876,7 +70217,7 @@ SQLITE_PRIVATE CollSeq *sqlite3FindCollSeq( */ static int matchQuality(FuncDef *p, int nArg, u8 enc){ int match = 0; - if( p->nArg==-1 || p->nArg==nArg + if( p->nArg==-1 || p->nArg==nArg || (nArg==-1 && (p->xFunc!=0 || p->xStep!=0)) ){ match = 1; @@ -69935,8 +70276,8 @@ SQLITE_PRIVATE void sqlite3FuncDefInsert( pHash->a[h] = pDef; } } - - + + /* ** Locate a user function given a name, a number of arguments and a flag @@ -69994,7 +70335,7 @@ SQLITE_PRIVATE FuncDef *sqlite3FindFunction( ** have fields overwritten with new information appropriate for the ** new function. But the FuncDefs for built-in functions are read-only. ** So we must not search for built-ins when creating a new function. - */ + */ if( !createFlag && !pBest ){ FuncDefHash *pHash = &GLOBAL(FuncDefHash, sqlite3GlobalFunctions); p = functionSearch(pHash, h, zName, nName); @@ -70012,7 +70353,7 @@ SQLITE_PRIVATE FuncDef *sqlite3FindFunction( ** exact match for the name, number of arguments and encoding, then add a ** new entry to the hash table and return it. */ - if( createFlag && (bestScore<6 || pBest->nArg!=nArg) && + if( createFlag && (bestScore<6 || pBest->nArg!=nArg) && (pBest = sqlite3DbMallocZero(db, sizeof(*pBest)+nName+1))!=0 ){ pBest->zName = (char *)&pBest[1]; pBest->nArg = (u16)nArg; @@ -70030,7 +70371,7 @@ SQLITE_PRIVATE FuncDef *sqlite3FindFunction( /* ** Free all resources held by the schema structure. The void* argument points -** at a Schema struct. This function does not call sqlite3DbFree(db, ) on the +** at a Schema struct. This function does not call sqlite3DbFree(db, ) on the ** pointer itself, it just cleans up subsiduary resources (i.e. the contents ** of the schema hash tables). ** @@ -70100,8 +70441,6 @@ SQLITE_PRIVATE Schema *sqlite3SchemaGet(sqlite3 *db, Btree *pBt){ ************************************************************************* ** This file contains C code routines that are called by the parser ** in order to generate code for DELETE FROM statements. -** -** $Id: delete.c,v 1.207 2009/08/08 18:01:08 drh Exp $ */ /* @@ -70136,12 +70475,12 @@ SQLITE_PRIVATE int sqlite3IsReadOnly(Parse *pParse, Table *pTab, int viewOk){ ** 1) It is a virtual table and no implementation of the xUpdate method ** has been provided, or ** 2) It is a system table (i.e. sqlite_master), this call is not - ** part of a nested parse and writable_schema pragma has not + ** part of a nested parse and writable_schema pragma has not ** been specified. ** ** In either case leave an error message in pParse and return non-zero. */ - if( ( IsVirtual(pTab) + if( ( IsVirtual(pTab) && sqlite3GetVTable(pParse->db, pTab)->pMod->pModule->xUpdate==0 ) || ( (pTab->tabFlags & TF_Readonly)!=0 && (pParse->db->flags & SQLITE_WriteSchema)==0 @@ -70180,7 +70519,7 @@ SQLITE_PRIVATE void sqlite3MaterializeView( pDup = sqlite3SelectDup(db, pView->pSelect, 0); if( pWhere ){ SrcList *pFrom; - + pWhere = sqlite3ExprDup(db, pWhere, 0); pFrom = sqlite3SrcListAppend(db, 0, 0, 0); if( pFrom ){ @@ -70242,11 +70581,11 @@ SQLITE_PRIVATE Expr *sqlite3LimitWhere( return pWhere; } - /* Generate a select expression tree to enforce the limit/offset + /* Generate a select expression tree to enforce the limit/offset ** term for the DELETE or UPDATE statement. For example: ** DELETE FROM table_a WHERE col1=1 ORDER BY col2 LIMIT 1 OFFSET 1 ** becomes: - ** DELETE FROM table_a WHERE rowid IN ( + ** DELETE FROM table_a WHERE rowid IN ( ** SELECT rowid FROM table_a WHERE col1=1 ORDER BY col2 LIMIT 1 OFFSET 1 ** ); */ @@ -70427,9 +70766,9 @@ SQLITE_PRIVATE void sqlite3DeleteFrom( #ifndef SQLITE_OMIT_TRUNCATE_OPTIMIZATION /* Special case: A DELETE without a WHERE clause deletes everything. ** It is easier just to erase the whole table. Prior to version 3.6.5, - ** this optimization caused the row change count (the value returned by + ** this optimization caused the row change count (the value returned by ** API function sqlite3_count_changes) to be set incorrectly. */ - if( rcauth==SQLITE_OK && pWhere==0 && !pTrigger && !IsVirtual(pTab) + if( rcauth==SQLITE_OK && pWhere==0 && !pTrigger && !IsVirtual(pTab) && 0==sqlite3FkRequired(pParse, pTab, 0, 0) ){ assert( !isView ); @@ -70466,9 +70805,9 @@ SQLITE_PRIVATE void sqlite3DeleteFrom( ** because deleting an item can change the scan order. */ end = sqlite3VdbeMakeLabel(v); - /* Unless this is a view, open cursors for the table we are + /* Unless this is a view, open cursors for the table we are ** deleting from and all its indices. If this is a view, then the - ** only effect this statement has is to fire the INSTEAD OF + ** only effect this statement has is to fire the INSTEAD OF ** triggers. */ if( !isView ){ sqlite3OpenTableAndIndices(pParse, pTab, iCur, OP_OpenWrite); @@ -70511,7 +70850,7 @@ SQLITE_PRIVATE void sqlite3DeleteFrom( sqlite3AutoincrementEnd(pParse); } - /* Return the number of rows that were deleted. If this routine is + /* Return the number of rows that were deleted. If this routine is ** generating code because of a call to sqlite3NestedParse(), do not ** invoke the callback function. */ @@ -70553,7 +70892,7 @@ delete_from_cleanup: ** 3. The record number of the row to be deleted must be stored in ** memory cell iRowid. ** -** This routine generates code to remove both the table record and all +** This routine generates code to remove both the table record and all ** index entries that point to that record. */ SQLITE_PRIVATE void sqlite3GenerateRowDelete( @@ -70572,12 +70911,12 @@ SQLITE_PRIVATE void sqlite3GenerateRowDelete( /* Vdbe is guaranteed to have been allocated by this stage. */ assert( v ); - /* Seek cursor iCur to the row to delete. If this row no longer exists + /* Seek cursor iCur to the row to delete. If this row no longer exists ** (this can happen if a trigger program has already deleted it), do ** not attempt to delete it or fire any DELETE triggers. */ iLabel = sqlite3VdbeMakeLabel(v); sqlite3VdbeAddOp3(v, OP_NotExists, iCur, iLabel, iRowid); - + /* If there are any triggers to fire, allocate a range of registers to ** use for the old.* references in the triggers. */ if( sqlite3FkRequired(pParse, pTab, 0, 0) || pTrigger ){ @@ -70586,12 +70925,14 @@ SQLITE_PRIVATE void sqlite3GenerateRowDelete( /* TODO: Could use temporary registers here. Also could attempt to ** avoid copying the contents of the rowid register. */ - mask = sqlite3TriggerOldmask(pParse, pTrigger, 0, pTab, onconf); + mask = sqlite3TriggerColmask( + pParse, pTrigger, 0, 0, TRIGGER_BEFORE|TRIGGER_AFTER, pTab, onconf + ); mask |= sqlite3FkOldmask(pParse, pTab); iOld = pParse->nMem+1; pParse->nMem += (1 + pTab->nCol); - /* Populate the OLD.* pseudo-table register array. These values will be + /* Populate the OLD.* pseudo-table register array. These values will be ** used by any BEFORE and AFTER triggers that exist. */ sqlite3VdbeAddOp2(v, OP_Copy, iRowid, iOld); for(iCol=0; iColnCol; iCol++){ @@ -70603,25 +70944,25 @@ SQLITE_PRIVATE void sqlite3GenerateRowDelete( } /* Invoke BEFORE DELETE trigger programs. */ - sqlite3CodeRowTrigger(pParse, pTrigger, + sqlite3CodeRowTrigger(pParse, pTrigger, TK_DELETE, 0, TRIGGER_BEFORE, pTab, iOld, onconf, iLabel ); /* Seek the cursor to the row to be deleted again. It may be that ** the BEFORE triggers coded above have already removed the row - ** being deleted. Do not attempt to delete the row a second time, and + ** being deleted. Do not attempt to delete the row a second time, and ** do not fire AFTER triggers. */ sqlite3VdbeAddOp3(v, OP_NotExists, iCur, iLabel, iRowid); /* Do FK processing. This call checks that any FK constraints that - ** refer to this table (i.e. constraints attached to other tables) + ** refer to this table (i.e. constraints attached to other tables) ** are not violated by deleting this row. */ sqlite3FkCheck(pParse, pTab, iOld, 0); } /* Delete the index and table entries. Skip this step if pTab is really ** a view (in which case the only effect of the DELETE statement is to - ** fire the INSTEAD OF triggers). */ + ** fire the INSTEAD OF triggers). */ if( pTab->pSelect==0 ){ sqlite3GenerateRowIndexDelete(pParse, pTab, iCur, 0); sqlite3VdbeAddOp2(v, OP_Delete, iCur, (count?OPFLAG_NCHANGE:0)); @@ -70632,16 +70973,16 @@ SQLITE_PRIVATE void sqlite3GenerateRowDelete( /* Do any ON CASCADE, SET NULL or SET DEFAULT operations required to ** handle rows (possibly in other tables) that refer via a foreign key - ** to the row just deleted. */ + ** to the row just deleted. */ sqlite3FkActions(pParse, pTab, 0, iOld); /* Invoke AFTER DELETE trigger programs. */ - sqlite3CodeRowTrigger(pParse, pTrigger, + sqlite3CodeRowTrigger(pParse, pTrigger, TK_DELETE, 0, TRIGGER_AFTER, pTab, iOld, onconf, iLabel ); /* Jump here if the row had already been deleted before any BEFORE - ** trigger programs were invoked. Or if a trigger program throws a + ** trigger programs were invoked. Or if a trigger program throws a ** RAISE(IGNORE) exception. */ sqlite3VdbeResolveLabel(v, iLabel); } @@ -70724,7 +71065,6 @@ SQLITE_PRIVATE int sqlite3GenerateIndexKey( return regBase; } - /************** End of delete.c **********************************************/ /************** Begin file func.c ********************************************/ /* @@ -70739,7 +71079,7 @@ SQLITE_PRIVATE int sqlite3GenerateIndexKey( ** ************************************************************************* ** This file contains the C functions that implement various SQL -** functions of SQLite. +** functions of SQLite. ** ** There is only one exported symbol in this file - the function ** sqliteRegisterBuildinFunctions() found at the bottom of the file. @@ -70856,7 +71196,7 @@ static void absFunc(sqlite3_context *context, int argc, sqlite3_value **argv){ return; } iVal = -iVal; - } + } sqlite3_result_int64(context, iVal); break; } @@ -70882,6 +71222,8 @@ static void absFunc(sqlite3_context *context, int argc, sqlite3_value **argv){ ** If x is a blob, then we count bytes. ** ** If p1 is negative, then we begin abs(p1) from the end of x[]. +** +** If p2 is negative, return the p2 characters preceeding p1. */ static void substrFunc( sqlite3_context *context, @@ -70902,6 +71244,7 @@ static void substrFunc( return; } p0type = sqlite3_value_type(argv[0]); + p1 = sqlite3_value_int(argv[1]); if( p0type==SQLITE_BLOB ){ len = sqlite3_value_bytes(argv[0]); z = sqlite3_value_blob(argv[0]); @@ -70911,11 +71254,12 @@ static void substrFunc( z = sqlite3_value_text(argv[0]); if( z==0 ) return; len = 0; - for(z2=z; *z2; len++){ - SQLITE_SKIP_UTF8(z2); + if( p1<0 ){ + for(z2=z; *z2; len++){ + SQLITE_SKIP_UTF8(z2); + } } } - p1 = sqlite3_value_int(argv[1]); if( argc==3 ){ p2 = sqlite3_value_int(argv[2]); if( p2<0 ){ @@ -70945,10 +71289,6 @@ static void substrFunc( } } assert( p1>=0 && p2>=0 ); - if( p1+p2>len ){ - p2 = len-p1; - if( p2<0 ) p2 = 0; - } if( p0type!=SQLITE_BLOB ){ while( *z && p1 ){ SQLITE_SKIP_UTF8(z); @@ -70959,6 +71299,10 @@ static void substrFunc( } sqlite3_result_text(context, (char*)z, (int)(z2-z), SQLITE_TRANSIENT); }else{ + if( p1+p2>len ){ + p2 = len-p1; + if( p2<0 ) p2 = 0; + } sqlite3_result_blob(context, (char*)&z[p1], (int)p2, SQLITE_TRANSIENT); } } @@ -71060,8 +71404,16 @@ static void lowerFunc(sqlite3_context *context, int argc, sqlite3_value **argv){ } } + +#if 0 /* This function is never used. */ /* -** Implementation of the IFNULL(), NVL(), and COALESCE() functions. +** The COALESCE() and IFNULL() functions used to be implemented as shown +** here. But now they are implemented as VDBE code so that unused arguments +** do not have to be computed. This legacy implementation is retained as +** comment. +*/ +/* +** Implementation of the IFNULL(), NVL(), and COALESCE() functions. ** All three do the same thing. They return the first non-NULL ** argument. */ @@ -71078,9 +71430,11 @@ static void ifnullFunc( } } } +#endif /* NOT USED */ +#define ifnullFunc versionFunc /* Substitute function - never called */ /* -** Implementation of random(). Return a random integer. +** Implementation of random(). Return a random integer. */ static void randomFunc( sqlite3_context *context, @@ -71091,11 +71445,11 @@ static void randomFunc( UNUSED_PARAMETER2(NotUsed, NotUsed2); sqlite3_randomness(sizeof(r), &r); if( r<0 ){ - /* We need to prevent a random number of 0x8000000000000000 + /* We need to prevent a random number of 0x8000000000000000 ** (or -9223372036854775808) since when you do abs() of that ** number of you get the same value back again. To do this ** in a way that is testable, mask the sign bit off of negative - ** values, resulting in a positive value. Then take the + ** values, resulting in a positive value. Then take the ** 2s complement of that positive value. The end result can ** therefore be no less than -9223372036854775807. */ @@ -71133,8 +71487,8 @@ static void randomBlob( ** value is the same as the sqlite3_last_insert_rowid() API function. */ static void last_insert_rowid( - sqlite3_context *context, - int NotUsed, + sqlite3_context *context, + int NotUsed, sqlite3_value **NotUsed2 ){ sqlite3 *db = sqlite3_context_db_handle(context); @@ -71241,7 +71595,7 @@ static int patternCompare( u8 matchOne = pInfo->matchOne; u8 matchAll = pInfo->matchAll; u8 matchSet = pInfo->matchSet; - u8 noCase = pInfo->noCase; + u8 noCase = pInfo->noCase; int prevEscape = 0; /* True if the previous character was 'escape' */ while( (c = sqlite3Utf8Read(zPattern,&zPattern))!=0 ){ @@ -71360,8 +71714,8 @@ SQLITE_API int sqlite3_like_count = 0; ** the GLOB operator. */ static void likeFunc( - sqlite3_context *context, - int argc, + sqlite3_context *context, + int argc, sqlite3_value **argv ){ const unsigned char *zA, *zB; @@ -71391,7 +71745,7 @@ static void likeFunc( const unsigned char *zEsc = sqlite3_value_text(argv[2]); if( zEsc==0 ) return; if( sqlite3Utf8CharLen((char*)zEsc, -1)!=1 ){ - sqlite3_result_error(context, + sqlite3_result_error(context, "ESCAPE expression must be a single character", -1); return; } @@ -71402,7 +71756,7 @@ static void likeFunc( #ifdef SQLITE_TEST sqlite3_like_count++; #endif - + sqlite3_result_int(context, patternCompare(zB, zA, pInfo, escape)); } } @@ -71455,7 +71809,7 @@ static void sourceidFunc( ** digits. */ static const char hexdigits[] = { '0', '1', '2', '3', '4', '5', '6', '7', - '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' + '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' }; /* @@ -71483,7 +71837,7 @@ static void quoteFunc(sqlite3_context *context, int argc, sqlite3_value **argv){ char const *zBlob = sqlite3_value_blob(argv[0]); int nBlob = sqlite3_value_bytes(argv[0]); assert( zBlob==sqlite3_value_blob(argv[0]) ); /* No encoding change */ - zText = (char *)contextMalloc(context, (2*(i64)nBlob)+4); + zText = (char *)contextMalloc(context, (2*(i64)nBlob)+4); if( zText ){ int i; for(i=0; in>0x7fffffff || p->n==sqlite3_aggregate_count(context) ); #endif -} +} static void countFinalize(sqlite3_context *context){ CountCtx *p; p = sqlite3_aggregate_context(context, 0); @@ -71940,8 +72294,8 @@ static void countFinalize(sqlite3_context *context){ ** Routines to implement min() and max() aggregate functions. */ static void minmaxStep( - sqlite3_context *context, - int NotUsed, + sqlite3_context *context, + int NotUsed, sqlite3_value **argv ){ Mem *pArg = (Mem *)argv[0]; @@ -72028,8 +72382,8 @@ static void groupConcatFinalize(sqlite3_context *context){ sqlite3_result_error_toobig(context); }else if( pAccum->mallocFailed ){ sqlite3_result_error_nomem(context); - }else{ - sqlite3_result_text(context, sqlite3StrAccumFinish(pAccum), -1, + }else{ + sqlite3_result_text(context, sqlite3StrAccumFinish(pAccum), -1, sqlite3_free); } } @@ -72079,10 +72433,10 @@ SQLITE_PRIVATE void sqlite3RegisterLikeFunctions(sqlite3 *db, int caseSensitive) } sqlite3CreateFunc(db, "like", 2, SQLITE_ANY, pInfo, likeFunc, 0, 0); sqlite3CreateFunc(db, "like", 3, SQLITE_ANY, pInfo, likeFunc, 0, 0); - sqlite3CreateFunc(db, "glob", 2, SQLITE_ANY, + sqlite3CreateFunc(db, "glob", 2, SQLITE_ANY, (struct compareInfo*)&globInfo, likeFunc, 0,0); setLikeOptFlag(db, "glob", SQLITE_FUNC_LIKE | SQLITE_FUNC_CASE); - setLikeOptFlag(db, "like", + setLikeOptFlag(db, "like", caseSensitive ? (SQLITE_FUNC_LIKE | SQLITE_FUNC_CASE) : SQLITE_FUNC_LIKE); } @@ -72095,14 +72449,14 @@ SQLITE_PRIVATE void sqlite3RegisterLikeFunctions(sqlite3 *db, int caseSensitive) */ SQLITE_PRIVATE int sqlite3IsLikeFunction(sqlite3 *db, Expr *pExpr, int *pIsNocase, char *aWc){ FuncDef *pDef; - if( pExpr->op!=TK_FUNCTION - || !pExpr->x.pList + if( pExpr->op!=TK_FUNCTION + || !pExpr->x.pList || pExpr->x.pList->nExpr!=2 ){ return 0; } assert( !ExprHasProperty(pExpr, EP_xIsSelect) ); - pDef = sqlite3FindFunction(db, pExpr->u.zToken, + pDef = sqlite3FindFunction(db, pExpr->u.zToken, sqlite3Strlen30(pExpr->u.zToken), 2, SQLITE_UTF8, 0); if( NEVER(pDef==0) || (pDef->flags & SQLITE_FUNC_LIKE)==0 ){ @@ -72162,10 +72516,12 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ FUNCTION(upper, 1, 0, 0, upperFunc ), FUNCTION(lower, 1, 0, 0, lowerFunc ), FUNCTION(coalesce, 1, 0, 0, 0 ), - FUNCTION(coalesce, -1, 0, 0, ifnullFunc ), FUNCTION(coalesce, 0, 0, 0, 0 ), +/* FUNCTION(coalesce, -1, 0, 0, ifnullFunc ), */ + {-1,SQLITE_UTF8,SQLITE_FUNC_COALESCE,0,0,ifnullFunc,0,0,"coalesce",0}, FUNCTION(hex, 1, 0, 0, hexFunc ), - FUNCTION(ifnull, 2, 0, 1, ifnullFunc ), +/* FUNCTION(ifnull, 2, 0, 0, ifnullFunc ), */ + {2,SQLITE_UTF8,SQLITE_FUNC_COALESCE,0,0,ifnullFunc,0,0,"ifnull",0}, FUNCTION(random, 0, 0, 0, randomFunc ), FUNCTION(randomblob, 1, 0, 0, randomBlob ), FUNCTION(nullif, 2, 0, 1, nullifFunc ), @@ -72192,7 +72548,7 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ AGGREGATE(count, 1, 0, 0, countStep, countFinalize ), AGGREGATE(group_concat, 1, 0, 0, groupConcatStep, groupConcatFinalize), AGGREGATE(group_concat, 2, 0, 0, groupConcatStep, groupConcatFinalize), - + LIKEFUNC(glob, 2, &globInfo, SQLITE_FUNC_LIKE|SQLITE_FUNC_CASE), #ifdef SQLITE_CASE_SENSITIVE_LIKE LIKEFUNC(like, 2, &likeInfoAlt, SQLITE_FUNC_LIKE|SQLITE_FUNC_CASE), @@ -72238,25 +72594,25 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ ** ** Foreign keys in SQLite come in two flavours: deferred and immediate. ** If an immediate foreign key constraint is violated, SQLITE_CONSTRAINT -** is returned and the current statement transaction rolled back. If a -** deferred foreign key constraint is violated, no action is taken -** immediately. However if the application attempts to commit the +** is returned and the current statement transaction rolled back. If a +** deferred foreign key constraint is violated, no action is taken +** immediately. However if the application attempts to commit the ** transaction before fixing the constraint violation, the attempt fails. ** ** Deferred constraints are implemented using a simple counter associated -** with the database handle. The counter is set to zero each time a -** database transaction is opened. Each time a statement is executed +** with the database handle. The counter is set to zero each time a +** database transaction is opened. Each time a statement is executed ** that causes a foreign key violation, the counter is incremented. Each ** time a statement is executed that removes an existing violation from ** the database, the counter is decremented. When the transaction is ** committed, the commit fails if the current value of the counter is ** greater than zero. This scheme has two big drawbacks: ** -** * When a commit fails due to a deferred foreign key constraint, +** * When a commit fails due to a deferred foreign key constraint, ** there is no way to tell which foreign constraint is not satisfied, ** or which row it is not satisfied for. ** -** * If the database contains foreign key violations when the +** * If the database contains foreign key violations when the ** transaction is opened, this may cause the mechanism to malfunction. ** ** Despite these problems, this approach is adopted as it seems simpler @@ -72268,26 +72624,26 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ ** the parent table for a match. If none is found increment the ** constraint counter. ** -** I.2) For each FK for which the table is the parent table, +** I.2) For each FK for which the table is the parent table, ** search the child table for rows that correspond to the new ** row in the parent table. Decrement the counter for each row ** found (as the constraint is now satisfied). ** ** DELETE operations: ** -** D.1) For each FK for which the table is the child table, -** search the parent table for a row that corresponds to the -** deleted row in the child table. If such a row is not found, +** D.1) For each FK for which the table is the child table, +** search the parent table for a row that corresponds to the +** deleted row in the child table. If such a row is not found, ** decrement the counter. ** -** D.2) For each FK for which the table is the parent table, search -** the child table for rows that correspond to the deleted row +** D.2) For each FK for which the table is the parent table, search +** the child table for rows that correspond to the deleted row ** in the parent table. For each found increment the counter. ** ** UPDATE operations: ** ** An UPDATE command requires that all 4 steps above are taken, but only -** for FK constraints for which the affected columns are actually +** for FK constraints for which the affected columns are actually ** modified (values must be compared at runtime). ** ** Note that I.1 and D.1 are very similar operations, as are I.2 and D.2. @@ -72296,10 +72652,10 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ ** For the purposes of immediate FK constraints, the OR REPLACE conflict ** resolution is considered to delete rows before the new row is inserted. ** If a delete caused by OR REPLACE violates an FK constraint, an exception -** is thrown, even if the FK constraint would be satisfied after the new +** is thrown, even if the FK constraint would be satisfied after the new ** row is inserted. ** -** Immediate constraints are usually handled similarly. The only difference +** Immediate constraints are usually handled similarly. The only difference ** is that the counter used is stored as part of each individual statement ** object (struct Vdbe). If, after the statement has run, its immediate ** constraint counter is greater than zero, it returns SQLITE_CONSTRAINT @@ -72309,7 +72665,7 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ ** INSERT violates a foreign key constraint. This is necessary as such ** an INSERT does not open a statement transaction. ** -** TODO: How should dropping a table be handled? How should renaming a +** TODO: How should dropping a table be handled? How should renaming a ** table be handled? ** ** @@ -72320,7 +72676,7 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ ** for those two operations needs to know whether or not the operation ** requires any FK processing and, if so, which columns of the original ** row are required by the FK processing VDBE code (i.e. if FKs were -** implemented using triggers, which of the old.* columns would be +** implemented using triggers, which of the old.* columns would be ** accessed). No information is required by the code-generator before ** coding an INSERT operation. The functions used by the UPDATE/DELETE ** generation code to query for this information are: @@ -72357,13 +72713,13 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ /* ** A foreign key constraint requires that the key columns in the parent ** table are collectively subject to a UNIQUE or PRIMARY KEY constraint. -** Given that pParent is the parent table for foreign key constraint pFKey, -** search the schema a unique index on the parent key columns. +** Given that pParent is the parent table for foreign key constraint pFKey, +** search the schema a unique index on the parent key columns. +** +** If successful, zero is returned. If the parent key is an INTEGER PRIMARY +** KEY column, then output variable *ppIdx is set to NULL. Otherwise, *ppIdx +** is set to point to the unique index. ** -** If successful, zero is returned. If the parent key is an INTEGER PRIMARY -** KEY column, then output variable *ppIdx is set to NULL. Otherwise, *ppIdx -** is set to point to the unique index. -** ** If the parent key consists of a single column (the foreign key constraint ** is not a composite foreign key), output variable *paiCol is set to NULL. ** Otherwise, it is set to point to an allocated array of size N, where @@ -72386,8 +72742,8 @@ SQLITE_PRIVATE void sqlite3RegisterGlobalFunctions(void){ ** PRIMARY KEY, or ** ** 4) No parent key columns were provided explicitly as part of the -** foreign key definition, and the PRIMARY KEY of the parent table -** consists of a a different number of columns to the child key in +** foreign key definition, and the PRIMARY KEY of the parent table +** consists of a a different number of columns to the child key in ** the child table. ** ** then non-zero is returned, and a "foreign key mismatch" error loaded @@ -72411,9 +72767,9 @@ static int locateFkeyIndex( assert( !paiCol || *paiCol==0 ); assert( pParse ); - /* If this is a non-composite (single column) foreign key, check if it - ** maps to the INTEGER PRIMARY KEY of table pParent. If so, leave *ppIdx - ** and *paiCol set to zero and return early. + /* If this is a non-composite (single column) foreign key, check if it + ** maps to the INTEGER PRIMARY KEY of table pParent. If so, leave *ppIdx + ** and *paiCol set to zero and return early. ** ** Otherwise, for a composite foreign key (more than one column), allocate ** space for the aiCol array (returned via output parameter *paiCol). @@ -72422,7 +72778,7 @@ static int locateFkeyIndex( if( nCol==1 ){ /* The FK maps to the IPK if any of the following are true: ** - ** 1) There is an INTEGER PRIMARY KEY column and the FK is implicitly + ** 1) There is an INTEGER PRIMARY KEY column and the FK is implicitly ** mapped to the primary key of table pParent, or ** 2) The FK is explicitly mapped to a column declared as INTEGER ** PRIMARY KEY. @@ -72439,14 +72795,14 @@ static int locateFkeyIndex( } for(pIdx=pParent->pIndex; pIdx; pIdx=pIdx->pNext){ - if( pIdx->nColumn==nCol && pIdx->onError!=OE_None ){ + if( pIdx->nColumn==nCol && pIdx->onError!=OE_None ){ /* pIdx is a UNIQUE index (or a PRIMARY KEY) and has the right number ** of columns. If each indexed column corresponds to a foreign key ** column of pFKey, then this index is a winner. */ if( zKey==0 ){ - /* If zKey is NULL, then this foreign key is implicitly mapped to - ** the PRIMARY KEY of table pParent. The PRIMARY KEY index may be + /* If zKey is NULL, then this foreign key is implicitly mapped to + ** the PRIMARY KEY of table pParent. The PRIMARY KEY index may be ** identified by the test (Index.autoIndex==2). */ if( pIdx->autoIndex==2 ){ if( aiCol ){ @@ -72502,15 +72858,15 @@ static int locateFkeyIndex( } /* -** This function is called when a row is inserted into or deleted from the -** child table of foreign key constraint pFKey. If an SQL UPDATE is executed +** This function is called when a row is inserted into or deleted from the +** child table of foreign key constraint pFKey. If an SQL UPDATE is executed ** on the child table of pFKey, this function is invoked twice for each row ** affected - once to "delete" the old row, and then again to "insert" the ** new row. ** ** Each time it is called, this function generates VDBE code to locate the -** row in the parent table that corresponds to the row being inserted into -** or deleted from the child table. If the parent row can be found, no +** row in the parent table that corresponds to the row being inserted into +** or deleted from the child table. If the parent row can be found, no ** special action is taken. Otherwise, if the parent row can *not* be ** found in the parent table: ** @@ -72524,7 +72880,7 @@ static int locateFkeyIndex( ** ** DELETE deferred Decrement the "deferred constraint counter". ** -** These operations are identified in the comment at the top of this file +** These operations are identified in the comment at the top of this file ** (fkey.c) as "I.1" and "D.1". */ static void fkLookupParent( @@ -72547,8 +72903,8 @@ static void fkLookupParent( ** outstanding constraints to resolve. If there are not, there is no need ** to check if deleting this row resolves any outstanding violations. ** - ** Check if any of the key columns in the child table row are NULL. If - ** any are, then the constraint is considered satisfied. No need to + ** Check if any of the key columns in the child table row are NULL. If + ** any are, then the constraint is considered satisfied. No need to ** search for a matching row in the parent table. */ if( nIncr<0 ){ sqlite3VdbeAddOp2(v, OP_FkIfZero, pFKey->isDeferred, iOk); @@ -72564,15 +72920,15 @@ static void fkLookupParent( ** column of the parent table (table pTab). */ int iMustBeInt; /* Address of MustBeInt instruction */ int regTemp = sqlite3GetTempReg(pParse); - - /* Invoke MustBeInt to coerce the child key value to an integer (i.e. + + /* Invoke MustBeInt to coerce the child key value to an integer (i.e. ** apply the affinity of the parent key). If this fails, then there ** is no matching parent key. Before using MustBeInt, make a copy of ** the value. Otherwise, the value inserted into the child key column ** will have INTEGER affinity applied to it, which may not be correct. */ sqlite3VdbeAddOp2(v, OP_SCopy, aiCol[0]+1+regData, regTemp); iMustBeInt = sqlite3VdbeAddOp2(v, OP_MustBeInt, regTemp, 0); - + /* If the parent table is the same as the child table, and we are about ** to increment the constraint-counter (i.e. this is an INSERT operation), ** then check if the row being inserted matches itself. If so, do not @@ -72580,7 +72936,7 @@ static void fkLookupParent( if( pTab==pFKey->pFrom && nIncr==1 ){ sqlite3VdbeAddOp3(v, OP_Eq, regData, iOk, regTemp); } - + sqlite3OpenTable(pParse, iCur, iDb, pTab, OP_OpenRead); sqlite3VdbeAddOp3(v, OP_NotExists, iCur, 0, regTemp); sqlite3VdbeAddOp2(v, OP_Goto, 0, iOk); @@ -72592,13 +72948,13 @@ static void fkLookupParent( int regTemp = sqlite3GetTempRange(pParse, nCol); int regRec = sqlite3GetTempReg(pParse); KeyInfo *pKey = sqlite3IndexKeyinfo(pParse, pIdx); - + sqlite3VdbeAddOp3(v, OP_OpenRead, iCur, pIdx->tnum, iDb); sqlite3VdbeChangeP4(v, -1, (char*)pKey, P4_KEYINFO_HANDOFF); for(i=0; ipFrom && nIncr>0 ){ Expr *pEq; /* Expression (pLeft = pRight) */ @@ -72799,7 +73155,7 @@ SQLITE_PRIVATE FKey *sqlite3FkReferences(Table *pTab){ } /* -** The second argument is a Trigger structure allocated by the +** The second argument is a Trigger structure allocated by the ** fkActionTrigger() routine. This function deletes the Trigger structure ** and all of its sub-components. ** @@ -72827,7 +73183,7 @@ static void fkTriggerDelete(sqlite3 *dbMem, Trigger *p){ ** ** (a) The table is the parent table of a FK constraint, or ** (b) The table is the child table of a deferred FK constraint and it is -** determined at runtime that there are outstanding deferred FK +** determined at runtime that there are outstanding deferred FK ** constraint violations in the database, ** ** then the equivalent of "DELETE FROM " is executed before dropping @@ -72843,7 +73199,7 @@ SQLITE_PRIVATE void sqlite3FkDropTable(Parse *pParse, SrcList *pName, Table *pTa assert( v ); /* VDBE has already been allocated */ if( sqlite3FkReferences(pTab)==0 ){ /* Search for a deferred foreign key constraint for which this table - ** is the child table. If one cannot be found, return without + ** is the child table. If one cannot be found, return without ** generating any VDBE code. If one can be found, then jump over ** the entire DELETE if there are no outstanding deferred constraints ** when this statement is run. */ @@ -72860,7 +73216,7 @@ SQLITE_PRIVATE void sqlite3FkDropTable(Parse *pParse, SrcList *pName, Table *pTa sqlite3DeleteFrom(pParse, sqlite3SrcListDup(db, pName, 0), 0); pParse->disableTriggers = 0; - /* If the DELETE has generated immediate foreign key constraint + /* If the DELETE has generated immediate foreign key constraint ** violations, halt the VDBE and return an error at this point, before ** any modifications to the schema are made. This is because statement ** transactions are not able to rollback schema changes. */ @@ -72877,7 +73233,7 @@ SQLITE_PRIVATE void sqlite3FkDropTable(Parse *pParse, SrcList *pName, Table *pTa /* ** This function is called when inserting, deleting or updating a row of -** table pTab to generate VDBE code to perform foreign key constraint +** table pTab to generate VDBE code to perform foreign key constraint ** processing for the operation. ** ** For a DELETE operation, parameter regOld is passed the index of the @@ -72893,11 +73249,11 @@ SQLITE_PRIVATE void sqlite3FkDropTable(Parse *pParse, SrcList *pName, Table *pTa ** For an UPDATE operation, this function is called twice. Once before ** the original record is deleted from the table using the calling convention ** described for DELETE. Then again after the original record is deleted -** but before the new record is inserted using the INSERT convention. +** but before the new record is inserted using the INSERT convention. */ SQLITE_PRIVATE void sqlite3FkCheck( Parse *pParse, /* Parse context */ - Table *pTab, /* Row is being deleted from this table */ + Table *pTab, /* Row is being deleted from this table */ int regOld, /* Previous row data is stored here */ int regNew /* New row data is stored here */ ){ @@ -72929,9 +73285,9 @@ SQLITE_PRIVATE void sqlite3FkCheck( int i; int isIgnore = 0; - /* Find the parent table of this foreign key. Also find a unique index - ** on the parent key columns in the parent table. If either of these - ** schema items cannot be located, set an error in pParse and return + /* Find the parent table of this foreign key. Also find a unique index + ** on the parent key columns in the parent table. If either of these + ** schema items cannot be located, set an error in pParse and return ** early. */ if( pParse->disableTriggers ){ pTo = sqlite3FindTable(db, pFKey->zTo, zDb); @@ -72955,7 +73311,7 @@ SQLITE_PRIVATE void sqlite3FkCheck( aiCol[i] = -1; } #ifndef SQLITE_OMIT_AUTHORIZATION - /* Request permission to read the parent key columns. If the + /* Request permission to read the parent key columns. If the ** authorization callback returns SQLITE_IGNORE, behave as if any ** values read from the parent table are NULL. */ if( db->xAuth ){ @@ -72967,21 +73323,21 @@ SQLITE_PRIVATE void sqlite3FkCheck( #endif } - /* Take a shared-cache advisory read-lock on the parent table. Allocate - ** a cursor to use to search the unique index on the parent key columns + /* Take a shared-cache advisory read-lock on the parent table. Allocate + ** a cursor to use to search the unique index on the parent key columns ** in the parent table. */ sqlite3TableLock(pParse, iDb, pTo->tnum, 0, pTo->zName); pParse->nTab++; if( regOld!=0 ){ /* A row is being removed from the child table. Search for the parent. - ** If the parent does not exist, removing the child row resolves an + ** If the parent does not exist, removing the child row resolves an ** outstanding foreign key constraint violation. */ fkLookupParent(pParse, iDb, pTo, pIdx, pFKey, aiCol, regOld, -1,isIgnore); } if( regNew!=0 ){ /* A row is being added to the child table. If a parent row cannot - ** be found, adding the child row has violated the FK constraint. */ + ** be found, adding the child row has violated the FK constraint. */ fkLookupParent(pParse, iDb, pTo, pIdx, pFKey, aiCol, regNew, +1,isIgnore); } @@ -73007,7 +73363,7 @@ SQLITE_PRIVATE void sqlite3FkCheck( } assert( aiCol || pFKey->nCol==1 ); - /* Create a SrcList structure containing a single table (the table + /* Create a SrcList structure containing a single table (the table ** the foreign key that refers to this table is attached to). This ** is required for the sqlite3WhereXXX() interface. */ pSrc = sqlite3SrcListAppend(db, 0, 0, 0); @@ -73017,13 +73373,13 @@ SQLITE_PRIVATE void sqlite3FkCheck( pItem->zName = pFKey->pFrom->zName; pItem->pTab->nRef++; pItem->iCursor = pParse->nTab++; - + if( regNew!=0 ){ fkScanChildren(pParse, pSrc, pTab, pIdx, pFKey, aiCol, regNew, -1); } if( regOld!=0 ){ /* If there is a RESTRICT action configured for the current operation - ** on the parent table of this FK, then throw an exception + ** on the parent table of this FK, then throw an exception ** immediately if the FK constraint is violated, even if this is a ** deferred trigger. That's what RESTRICT means. To defer checking ** the constraint, the FK should specify NO ACTION (represented @@ -73040,7 +73396,7 @@ SQLITE_PRIVATE void sqlite3FkCheck( #define COLUMN_MASK(x) (((x)>31) ? 0xffffffff : ((u32)1<<(x))) /* -** This function is called before generating code to update or delete a +** This function is called before generating code to update or delete a ** row contained in table pTab. */ SQLITE_PRIVATE u32 sqlite3FkOldmask( @@ -73066,17 +73422,17 @@ SQLITE_PRIVATE u32 sqlite3FkOldmask( } /* -** This function is called before generating code to update or delete a +** This function is called before generating code to update or delete a ** row contained in table pTab. If the operation is a DELETE, then ** parameter aChange is passed a NULL value. For an UPDATE, aChange points ** to an array of size N, where N is the number of columns in table pTab. -** If the i'th column is not modified by the UPDATE, then the corresponding +** If the i'th column is not modified by the UPDATE, then the corresponding ** entry in the aChange[] array is set to -1. If the column is modified, ** the value is 0 or greater. Parameter chngRowid is set to true if the ** UPDATE statement modifies the rowid fields of the table. ** ** If any foreign key processing will be required, this function returns -** true. If there is no foreign key related processing, this function +** true. If there is no foreign key related processing, this function ** returns false. */ SQLITE_PRIVATE int sqlite3FkRequired( @@ -73087,8 +73443,8 @@ SQLITE_PRIVATE int sqlite3FkRequired( ){ if( pParse->db->flags&SQLITE_ForeignKeys ){ if( !aChange ){ - /* A DELETE operation. Foreign key processing is required if the - ** table in question is either the child or parent table for any + /* A DELETE operation. Foreign key processing is required if the + ** table in question is either the child or parent table for any ** foreign key constraint. */ return (sqlite3FkReferences(pTab) || pTab->pFKey); }else{ @@ -73126,7 +73482,7 @@ SQLITE_PRIVATE int sqlite3FkRequired( } /* -** This function is called when an UPDATE or DELETE operation is being +** This function is called when an UPDATE or DELETE operation is being ** compiled on table pTab, which is the parent table of foreign-key pFKey. ** If the current operation is an UPDATE, then the pChanges parameter is ** passed a pointer to the list of columns being modified. If it is a @@ -73138,7 +73494,7 @@ SQLITE_PRIVATE int sqlite3FkRequired( ** returned (these actions require no special handling by the triggers ** sub-system, code for them is created by fkScanChildren()). ** -** For example, if pFKey is the foreign key and pTab is table "p" in +** For example, if pFKey is the foreign key and pTab is table "p" in ** the following schema: ** ** CREATE TABLE p(pk PRIMARY KEY); @@ -73151,7 +73507,7 @@ SQLITE_PRIVATE int sqlite3FkRequired( ** END; ** ** The returned pointer is cached as part of the foreign key object. It -** is eventually freed along with the rest of the foreign key object by +** is eventually freed along with the rest of the foreign key object by ** sqlite3FkDelete(). */ static Trigger *fkActionTrigger( @@ -73205,7 +73561,7 @@ static Trigger *fkActionTrigger( ** that the affinity and collation sequence associated with the ** parent table are used for the comparison. */ pEq = sqlite3PExpr(pParse, TK_EQ, - sqlite3PExpr(pParse, TK_DOT, + sqlite3PExpr(pParse, TK_DOT, sqlite3PExpr(pParse, TK_ID, 0, 0, &tOld), sqlite3PExpr(pParse, TK_ID, 0, 0, &tToCol) , 0), @@ -73220,22 +73576,22 @@ static Trigger *fkActionTrigger( */ if( pChanges ){ pEq = sqlite3PExpr(pParse, TK_IS, - sqlite3PExpr(pParse, TK_DOT, + sqlite3PExpr(pParse, TK_DOT, sqlite3PExpr(pParse, TK_ID, 0, 0, &tOld), sqlite3PExpr(pParse, TK_ID, 0, 0, &tToCol), 0), - sqlite3PExpr(pParse, TK_DOT, + sqlite3PExpr(pParse, TK_DOT, sqlite3PExpr(pParse, TK_ID, 0, 0, &tNew), sqlite3PExpr(pParse, TK_ID, 0, 0, &tToCol), 0), 0); pWhen = sqlite3ExprAnd(db, pWhen, pEq); } - + if( action!=OE_Restrict && (action!=OE_Cascade || pChanges) ){ Expr *pNew; if( action==OE_Cascade ){ - pNew = sqlite3PExpr(pParse, TK_DOT, + pNew = sqlite3PExpr(pParse, TK_DOT, sqlite3PExpr(pParse, TK_ID, 0, 0, &tNew), sqlite3PExpr(pParse, TK_ID, 0, 0, &tToCol) , 0); @@ -73260,7 +73616,7 @@ static Trigger *fkActionTrigger( if( action==OE_Restrict ){ Token tFrom; - Expr *pRaise; + Expr *pRaise; tFrom.z = zFrom; tFrom.n = nFrom; @@ -73268,7 +73624,7 @@ static Trigger *fkActionTrigger( if( pRaise ){ pRaise->affinity = OE_Abort; } - pSelect = sqlite3SelectNew(pParse, + pSelect = sqlite3SelectNew(pParse, sqlite3ExprListAppend(pParse, 0, pRaise), sqlite3SrcListAppend(db, 0, &tFrom, 0), pWhere, @@ -73285,7 +73641,7 @@ static Trigger *fkActionTrigger( enableLookaside = db->lookaside.bEnabled; db->lookaside.bEnabled = 0; - pTrigger = (Trigger *)sqlite3DbMallocZero(db, + pTrigger = (Trigger *)sqlite3DbMallocZero(db, sizeof(Trigger) + /* struct Trigger */ sizeof(TriggerStep) + /* Single step in trigger program */ nFrom + 1 /* Space for pStep->target.z */ @@ -73295,7 +73651,7 @@ static Trigger *fkActionTrigger( pStep->target.z = (char *)&pStep[1]; pStep->target.n = nFrom; memcpy((char *)pStep->target.z, zFrom, nFrom); - + pStep->pWhere = sqlite3ExprDup(db, pWhere, EXPRDUP_REDUCE); pStep->pExprList = sqlite3ExprListDup(db, pList, EXPRDUP_REDUCE); pStep->pSelect = sqlite3SelectDup(db, pSelect, EXPRDUP_REDUCE); @@ -73319,12 +73675,12 @@ static Trigger *fkActionTrigger( switch( action ){ case OE_Restrict: - pStep->op = TK_SELECT; + pStep->op = TK_SELECT; break; - case OE_Cascade: - if( !pChanges ){ - pStep->op = TK_DELETE; - break; + case OE_Cascade: + if( !pChanges ){ + pStep->op = TK_DELETE; + break; } default: pStep->op = TK_UPDATE; @@ -73349,9 +73705,9 @@ SQLITE_PRIVATE void sqlite3FkActions( ExprList *pChanges, /* Change-list for UPDATE, NULL for DELETE */ int regOld /* Address of array containing old row */ ){ - /* If foreign-key support is enabled, iterate through all FKs that - ** refer to table pTab. If there is an action associated with the FK - ** for this operation (either update or delete), invoke the associated + /* If foreign-key support is enabled, iterate through all FKs that + ** refer to table pTab. If there is an action associated with the FK + ** for this operation (either update or delete), invoke the associated ** trigger sub-program. */ if( pParse->db->flags&SQLITE_ForeignKeys ){ FKey *pFKey; /* Iterator variable */ @@ -73421,8 +73777,6 @@ SQLITE_PRIVATE void sqlite3FkDelete(Table *pTab){ ************************************************************************* ** This file contains C code routines that are called by the parser ** to handle INSERT statements in SQLite. -** -** $Id: insert.c,v 1.270 2009/07/24 17:58:53 danielk1977 Exp $ */ /* @@ -73447,7 +73801,7 @@ SQLITE_PRIVATE void sqlite3OpenTable( /* ** Return a pointer to the column affinity string associated with index -** pIdx. A column affinity string has one character for each column in +** pIdx. A column affinity string has one character for each column in ** the table, according to the affinity of the column: ** ** Character Column affinity @@ -73489,7 +73843,7 @@ SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(Vdbe *v, Index *pIdx){ pIdx->zColAff[n++] = SQLITE_AFF_NONE; pIdx->zColAff[n] = 0; } - + return pIdx->zColAff; } @@ -73509,7 +73863,7 @@ SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(Vdbe *v, Index *pIdx){ */ SQLITE_PRIVATE void sqlite3TableAffinityStr(Vdbe *v, Table *pTab){ /* The first time a column affinity string for a particular table - ** is required, it is allocated and populated here. It is then + ** is required, it is allocated and populated here. It is then ** stored as a member of the Table structure for subsequent use. ** ** The column affinity string will eventually be deleted by @@ -73540,9 +73894,9 @@ SQLITE_PRIVATE void sqlite3TableAffinityStr(Vdbe *v, Table *pTab){ /* ** Return non-zero if the table pTab in database iDb or any of its indices ** have been opened at any point in the VDBE program beginning at location -** iStartAddr throught the end of the program. This is used to see if -** a statement of the form "INSERT INTO SELECT ..." can -** run without using temporary table for the results of the SELECT. +** iStartAddr throught the end of the program. This is used to see if +** a statement of the form "INSERT INTO SELECT ..." can +** run without using temporary table for the results of the SELECT. */ static int readsTable(Parse *p, int iStartAddr, int iDb, Table *pTab){ Vdbe *v = sqlite3GetVdbe(p); @@ -73629,7 +73983,7 @@ static int autoIncBegin( /* ** This routine generates code that will initialize all of the -** register used by the autoincrement tracker. +** register used by the autoincrement tracker. */ SQLITE_PRIVATE void sqlite3AutoincrementBegin(Parse *pParse){ AutoincInfo *p; /* Information about an AUTOINCREMENT */ @@ -73927,7 +74281,7 @@ SQLITE_PRIVATE void sqlite3Insert( assert( (pTrigger && tmask) || (pTrigger==0 && tmask==0) ); /* If pTab is really a view, make sure it has been initialized. - ** ViewGetColumnNames() is a no-op if pTab is not a view (or virtual + ** ViewGetColumnNames() is a no-op if pTab is not a view (or virtual ** module table). */ if( sqlite3ViewGetColumnNames(pParse, pTab) ){ @@ -73935,7 +74289,7 @@ SQLITE_PRIVATE void sqlite3Insert( } /* Ensure that: - * (a) the table is not read-only, + * (a) the table is not read-only, * (b) that if it is a view then ON INSERT triggers exist */ if( sqlite3IsReadOnly(pParse, pTab, tmask) ){ @@ -74033,7 +74387,7 @@ SQLITE_PRIVATE void sqlite3Insert( ** the destination table (template 3). ** ** A temp table must be used if the table being updated is also one - ** of the tables being read by the SELECT statement. Also use a + ** of the tables being read by the SELECT statement. Also use a ** temp table in the case of row triggers. */ if( pTrigger || readsTable(pParse, addrSelect, iDb, pTab) ){ @@ -74097,7 +74451,7 @@ SQLITE_PRIVATE void sqlite3Insert( } } if( pColumn==0 && nColumn && nColumn!=(pTab->nCol-nHidden) ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "table %S has %d columns but %d values were supplied", pTabList, 0, pTab->nCol-nHidden, nColumn); goto insert_cleanup; @@ -74108,7 +74462,7 @@ SQLITE_PRIVATE void sqlite3Insert( } /* If the INSERT statement included an IDLIST term, then make sure - ** all elements of the IDLIST really are columns of the table and + ** all elements of the IDLIST really are columns of the table and ** remember the column indices. ** ** If the table has an INTEGER PRIMARY KEY column and that column @@ -74152,7 +74506,7 @@ SQLITE_PRIVATE void sqlite3Insert( if( pColumn==0 && nColumn>0 ){ keyColumn = pTab->iPKey; } - + /* Initialize the count of rows to be inserted */ if( db->flags & SQLITE_CountRows ){ @@ -74260,7 +74614,7 @@ SQLITE_PRIVATE void sqlite3Insert( if( pColumn && j>=pColumn->nId ){ sqlite3ExprCode(pParse, pTab->aCol[i].pDflt, regCols+i+1); }else if( useTempTable ){ - sqlite3VdbeAddOp3(v, OP_Column, srcTab, j, regCols+i+1); + sqlite3VdbeAddOp3(v, OP_Column, srcTab, j, regCols+i+1); }else{ assert( pSelect==0 ); /* Otherwise useTempTable is true */ sqlite3ExprCodeAndCache(pParse, pList->a[j].pExpr, regCols+i+1); @@ -74278,7 +74632,7 @@ SQLITE_PRIVATE void sqlite3Insert( } /* Fire BEFORE or INSTEAD OF triggers */ - sqlite3CodeRowTrigger(pParse, pTrigger, TK_INSERT, 0, TRIGGER_BEFORE, + sqlite3CodeRowTrigger(pParse, pTrigger, TK_INSERT, 0, TRIGGER_BEFORE, pTab, regCols-pTab->nCol-1, onError, endOfLoop); sqlite3ReleaseTempRange(pParse, regCols, pTab->nCol+1); @@ -74287,7 +74641,7 @@ SQLITE_PRIVATE void sqlite3Insert( /* Push the record number for the new entry onto the stack. The ** record number is a randomly generate integer created by NewRowid ** except when the table has an INTEGER PRIMARY KEY column, in which - ** case the record number is the same as that column. + ** case the record number is the same as that column. */ if( !isView ){ if( IsVirtual(pTab) ){ @@ -74364,7 +74718,7 @@ SQLITE_PRIVATE void sqlite3Insert( if( j<0 || nColumn==0 || (pColumn && j>=pColumn->nId) ){ sqlite3ExprCode(pParse, pTab->aCol[i].pDflt, iRegStore); }else if( useTempTable ){ - sqlite3VdbeAddOp3(v, OP_Column, srcTab, j, iRegStore); + sqlite3VdbeAddOp3(v, OP_Column, srcTab, j, iRegStore); }else if( pSelect ){ sqlite3VdbeAddOp2(v, OP_SCopy, regFromSelect+j, iRegStore); }else{ @@ -74403,7 +74757,7 @@ SQLITE_PRIVATE void sqlite3Insert( if( pTrigger ){ /* Code AFTER triggers */ - sqlite3CodeRowTrigger(pParse, pTrigger, TK_INSERT, 0, TRIGGER_AFTER, + sqlite3CodeRowTrigger(pParse, pTrigger, TK_INSERT, 0, TRIGGER_AFTER, pTab, regData-2-pTab->nCol, onError, endOfLoop); } @@ -74438,7 +74792,7 @@ insert_end: } /* - ** Return the number of rows inserted. If this routine is + ** Return the number of rows inserted. If this routine is ** generating code because of a call to sqlite3NestedParse(), do not ** invoke the callback function. */ @@ -74488,10 +74842,10 @@ insert_cleanup: ** If isUpdate is true and rowidChng is non-zero, then rowidChng contains ** the address of a register containing the rowid before the update takes ** place. isUpdate is true for UPDATEs and false for INSERTs. If isUpdate -** is false, indicating an INSERT statement, then a non-zero rowidChng +** is false, indicating an INSERT statement, then a non-zero rowidChng ** indicates that the rowid was explicitly specified as part of the ** INSERT statement. If rowidChng is false, it means that the rowid is -** computed automatically in an insert or that the rowid value is not +** computed automatically in an insert or that the rowid value is not ** modified by an update. ** ** The code generated by this routine store new index entries into @@ -74648,7 +75002,7 @@ SQLITE_PRIVATE void sqlite3GenerateConstraintChecks( }else if( onError==OE_Default ){ onError = OE_Abort; } - + if( isUpdate ){ j2 = sqlite3VdbeAddOp3(v, OP_Eq, regRowid, 0, rowidChng); } @@ -74673,7 +75027,7 @@ SQLITE_PRIVATE void sqlite3GenerateConstraintChecks( ** ** Otherwise, if there are no triggers or the recursive-triggers ** flag is not set, call GenerateRowIndexDelete(). This removes - ** the index b-tree entries only. The table b-tree entry will be + ** the index b-tree entries only. The table b-tree entry will be ** replaced by the new entry when it is inserted. */ Trigger *pTrigger = 0; if( pParse->db->flags&SQLITE_RecTriggers ){ @@ -74729,7 +75083,7 @@ SQLITE_PRIVATE void sqlite3GenerateConstraintChecks( /* Find out what action to take in case there is an indexing conflict */ onError = pIdx->onError; - if( onError==OE_None ){ + if( onError==OE_None ){ sqlite3ReleaseTempRange(pParse, regIdx, pIdx->nColumn+1); continue; /* pIdx is not a UNIQUE index */ } @@ -74742,7 +75096,7 @@ SQLITE_PRIVATE void sqlite3GenerateConstraintChecks( if( onError==OE_Ignore ) onError = OE_Replace; else if( onError==OE_Fail ) onError = OE_Abort; } - + /* Check to see if the new index entry will be unique */ regR = sqlite3GetTempReg(pParse); sqlite3VdbeAddOp2(v, OP_SCopy, regOldRowid, regR); @@ -74801,7 +75155,7 @@ SQLITE_PRIVATE void sqlite3GenerateConstraintChecks( sqlite3VdbeJumpHere(v, j3); sqlite3ReleaseTempReg(pParse, regR); } - + if( pbMayReplace ){ *pbMayReplace = seenReplace; } @@ -75242,8 +75596,6 @@ static int xferOptimization( ** implement the programmer interface to the library. Routines in ** other files are for internal use by SQLite and should not be ** accessed by users of the library. -** -** $Id: legacy.c,v 1.35 2009/08/07 16:56:00 danielk1977 Exp $ */ @@ -75299,7 +75651,7 @@ SQLITE_API int sqlite3_exec( rc = sqlite3_step(pStmt); /* Invoke the callback function if required */ - if( xCallback && (SQLITE_ROW==rc || + if( xCallback && (SQLITE_ROW==rc || (SQLITE_DONE==rc && !callbackIsInit && db->flags&SQLITE_NullCallback)) ){ if( !callbackIsInit ){ @@ -75388,8 +75740,6 @@ exec_out: ************************************************************************* ** This file contains code used to dynamically load extensions into ** the SQLite library. -** -** $Id: loadext.c,v 1.60 2009/06/03 01:24:54 drh Exp $ */ #ifndef SQLITE_CORE @@ -75411,10 +75761,8 @@ exec_out: ** This header file defines the SQLite interface for use by ** shared libraries that want to be imported as extensions into ** an SQLite instance. Shared libraries that intend to be loaded -** as extensions by SQLite should #include this file instead of +** as extensions by SQLite should #include this file instead of ** sqlite3.h. -** -** @(#) $Id: sqlite3ext.h,v 1.25 2008/10/12 00:27:54 shane Exp $ */ #ifndef _SQLITE3EXT_H_ #define _SQLITE3EXT_H_ @@ -76047,8 +76395,8 @@ static const sqlite3_api_routines sqlite3Apis = { sqlite3_memory_highwater, sqlite3_memory_used, #ifdef SQLITE_MUTEX_OMIT - 0, - 0, + 0, + 0, 0, 0, 0, @@ -76097,7 +76445,7 @@ static const sqlite3_api_routines sqlite3Apis = { ** ** Return SQLITE_OK on success and SQLITE_ERROR if something goes wrong. ** -** If an error occurs and pzErrMsg is not 0, then fill *pzErrMsg with +** If an error occurs and pzErrMsg is not 0, then fill *pzErrMsg with ** error message text. The calling function should free this memory ** by calling sqlite3DbFree(db, ). */ @@ -76138,7 +76486,7 @@ static int sqlite3LoadExtension( if( pzErrMsg ){ zErrmsg = sqlite3StackAllocZero(db, nMsg); if( zErrmsg ){ - sqlite3_snprintf(nMsg, zErrmsg, + sqlite3_snprintf(nMsg, zErrmsg, "unable to open shared library [%s]", zFile); sqlite3OsDlError(pVfs, nMsg-1, zErrmsg); *pzErrMsg = sqlite3DbStrDup(0, zErrmsg); @@ -76249,7 +76597,7 @@ static const sqlite3_api_routines sqlite3Apis = { 0 }; */ typedef struct sqlite3AutoExtList sqlite3AutoExtList; static SQLITE_WSD struct sqlite3AutoExtList { - int nExt; /* Number of entries in aExt[] */ + int nExt; /* Number of entries in aExt[] */ void (**aExt)(void); /* Pointers to the extension init functions */ } sqlite3Autoext = { 0, 0 }; @@ -76382,8 +76730,6 @@ SQLITE_PRIVATE void sqlite3AutoLoadExtensions(sqlite3 *db){ ** ************************************************************************* ** This file contains code used to implement the PRAGMA command. -** -** $Id: pragma.c,v 1.214 2009/07/02 07:47:33 danielk1977 Exp $ */ /* Ignore this whole file if pragmas are disabled @@ -76392,7 +76738,7 @@ SQLITE_PRIVATE void sqlite3AutoLoadExtensions(sqlite3 *db){ /* ** Interpret the given string as a safety level. Return 0 for OFF, -** 1 for ON or NORMAL and 2 for FULL. Return 1 for an empty or +** 1 for ON or NORMAL and 2 for FULL. Return 1 for an empty or ** unrecognized string argument. ** ** Note that the values returned are one less that the values that @@ -76441,7 +76787,7 @@ static int getLockingMode(const char *z){ /* ** Interpret the given string as an auto-vacuum mode value. ** -** The following strings, "none", "full" and "incremental" are +** The following strings, "none", "full" and "incremental" are ** acceptable, as are their numeric equivalents: 0, 1 and 2 respectively. */ static int getAutoVacuum(const char *z){ @@ -76594,7 +76940,7 @@ static int flagPragma(Parse *pParse, const char *zLeft, const char *zRight){ db->flags &= ~mask; } - /* Many of the flag-pragmas modify the code generated by the SQL + /* Many of the flag-pragmas modify the code generated by the SQL ** compiler (eg. count_changes). So add an opcode to expire all ** compiled SQL statements after modifying a pragma value. */ @@ -76620,7 +76966,7 @@ static const char *actionName(u8 action){ case OE_SetDflt: zName = "SET DEFAULT"; break; case OE_Cascade: zName = "CASCADE"; break; case OE_Restrict: zName = "RESTRICT"; break; - default: zName = "NO ACTION"; + default: zName = "NO ACTION"; assert( action==OE_None ); break; } return zName; @@ -76628,7 +76974,7 @@ static const char *actionName(u8 action){ #endif /* -** Process a pragma statement. +** Process a pragma statement. ** ** Pragmas are of this form: ** @@ -76643,7 +76989,7 @@ static const char *actionName(u8 action){ ** id and pId2 is any empty string. */ SQLITE_PRIVATE void sqlite3Pragma( - Parse *pParse, + Parse *pParse, Token *pId1, /* First part of [database.]id field */ Token *pId2, /* Second part of [database.]id field, or NULL */ Token *pValue, /* Token for , or NULL */ @@ -76666,8 +77012,8 @@ SQLITE_PRIVATE void sqlite3Pragma( if( iDb<0 ) return; pDb = &db->aDb[iDb]; - /* If the temp database has been explicitly named as part of the - ** pragma, make sure it is open. + /* If the temp database has been explicitly named as part of the + ** pragma, make sure it is open. */ if( iDb==1 && sqlite3OpenTempDatabase(pParse) ){ return; @@ -76686,7 +77032,7 @@ SQLITE_PRIVATE void sqlite3Pragma( if( sqlite3AuthCheck(pParse, SQLITE_PRAGMA, zLeft, zRight, zDb) ){ goto pragma_out; } - + #ifndef SQLITE_OMIT_PAGER_PRAGMAS /* ** PRAGMA [database.]default_cache_size @@ -76772,7 +77118,7 @@ SQLITE_PRIVATE void sqlite3Pragma( ** PRAGMA [database.]max_page_count=N ** ** The first form reports the current setting for the - ** maximum number of pages in the database file. The + ** maximum number of pages in the database file. The ** second form attempts to change this setting. Both ** forms return the current setting. */ @@ -76908,7 +77254,7 @@ SQLITE_PRIVATE void sqlite3Pragma( || eMode==PAGER_JOURNALMODE_MEMORY ); sqlite3VdbeSetNumCols(v, 1); sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "journal_mode", SQLITE_STATIC); - sqlite3VdbeAddOp4(v, OP_String8, 0, 1, 0, + sqlite3VdbeAddOp4(v, OP_String8, 0, 1, 0, azModeName[eMode], P4_STATIC); sqlite3VdbeAddOp2(v, OP_ResultRow, 1, 1); }else @@ -76966,7 +77312,7 @@ SQLITE_PRIVATE void sqlite3Pragma( */ int rc = sqlite3BtreeSetAutoVacuum(pBt, eAuto); if( rc==SQLITE_OK && (eAuto==1 || eAuto==2) ){ - /* When setting the auto_vacuum mode to either "full" or + /* When setting the auto_vacuum mode to either "full" or ** "incremental", write the value of meta[6] in the database ** file. Before writing to meta[6], check that meta[3] indicates ** that this really is an auto-vacuum capable database. @@ -77077,7 +77423,7 @@ SQLITE_PRIVATE void sqlite3Pragma( if( !zRight ){ if( sqlite3_temp_directory ){ sqlite3VdbeSetNumCols(v, 1); - sqlite3VdbeSetColName(v, 0, COLNAME_NAME, + sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "temp_store_directory", SQLITE_STATIC); sqlite3VdbeAddOp4(v, OP_String8, 0, 1, 0, sqlite3_temp_directory, 0); sqlite3VdbeAddOp2(v, OP_ResultRow, 1, 1); @@ -77130,12 +77476,12 @@ SQLITE_PRIVATE void sqlite3Pragma( Pager *pPager = sqlite3BtreePager(pDb->pBt); char *proxy_file_path = NULL; sqlite3_file *pFile = sqlite3PagerFile(pPager); - sqlite3OsFileControl(pFile, SQLITE_GET_LOCKPROXYFILE, + sqlite3OsFileControl(pFile, SQLITE_GET_LOCKPROXYFILE, &proxy_file_path); - + if( proxy_file_path ){ sqlite3VdbeSetNumCols(v, 1); - sqlite3VdbeSetColName(v, 0, COLNAME_NAME, + sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "lock_proxy_file", SQLITE_STATIC); sqlite3VdbeAddOp4(v, OP_String8, 0, 1, 0, proxy_file_path, 0); sqlite3VdbeAddOp2(v, OP_ResultRow, 1, 1); @@ -77145,10 +77491,10 @@ SQLITE_PRIVATE void sqlite3Pragma( sqlite3_file *pFile = sqlite3PagerFile(pPager); int res; if( zRight[0] ){ - res=sqlite3OsFileControl(pFile, SQLITE_SET_LOCKPROXYFILE, + res=sqlite3OsFileControl(pFile, SQLITE_SET_LOCKPROXYFILE, zRight); } else { - res=sqlite3OsFileControl(pFile, SQLITE_SET_LOCKPROXYFILE, + res=sqlite3OsFileControl(pFile, SQLITE_SET_LOCKPROXYFILE, NULL); } if( res!=SQLITE_OK ){ @@ -77157,8 +77503,8 @@ SQLITE_PRIVATE void sqlite3Pragma( } } }else -#endif /* SQLITE_ENABLE_LOCKING_STYLE */ - +#endif /* SQLITE_ENABLE_LOCKING_STYLE */ + /* ** PRAGMA [database.]synchronous ** PRAGMA [database.]synchronous=OFF|ON|NORMAL|FULL @@ -77174,7 +77520,7 @@ SQLITE_PRIVATE void sqlite3Pragma( returnSingleInt(pParse, "synchronous", pDb->safety_level-1); }else{ if( !db->autoCommit ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "Safety level may not be changed inside a transaction"); }else{ pDb->safety_level = getSafetyLevel(zRight)+1; @@ -77274,7 +77620,7 @@ SQLITE_PRIVATE void sqlite3Pragma( v = sqlite3GetVdbe(pParse); pIdx = pTab->pIndex; if( pIdx ){ - int i = 0; + int i = 0; sqlite3VdbeSetNumCols(v, 3); pParse->nMem = 3; sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "seq", SQLITE_STATIC); @@ -77337,7 +77683,7 @@ SQLITE_PRIVATE void sqlite3Pragma( v = sqlite3GetVdbe(pParse); pFK = pTab->pFKey; if( pFK ){ - int i = 0; + int i = 0; sqlite3VdbeSetNumCols(v, 8); pParse->nMem = 8; sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "id", SQLITE_STATIC); @@ -77399,12 +77745,12 @@ SQLITE_PRIVATE void sqlite3Pragma( #endif #ifndef SQLITE_OMIT_INTEGRITY_CHECK - /* Pragma "quick_check" is an experimental reduced version of + /* Pragma "quick_check" is an experimental reduced version of ** integrity_check designed to detect most database corruption ** without most of the overhead of a full integrity-check. */ if( sqlite3StrICmp(zLeft, "integrity_check")==0 - || sqlite3StrICmp(zLeft, "quick_check")==0 + || sqlite3StrICmp(zLeft, "quick_check")==0 ){ int i, j, addr, mxErr; @@ -77501,6 +77847,7 @@ SQLITE_PRIVATE void sqlite3Pragma( sqlite3VdbeAddOp2(v, OP_AddImm, 2, 1); /* increment entry count */ for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){ int jmp2; + int r1; static const VdbeOpList idxErr[] = { { OP_AddImm, 1, -1, 0}, { OP_String8, 0, 3, 0}, /* 1 */ @@ -77514,8 +77861,8 @@ SQLITE_PRIVATE void sqlite3Pragma( { OP_IfPos, 1, 0, 0}, /* 9 */ { OP_Halt, 0, 0, 0}, }; - sqlite3GenerateIndexKey(pParse, pIdx, 1, 3, 1); - jmp2 = sqlite3VdbeAddOp3(v, OP_Found, j+2, 0, 3); + r1 = sqlite3GenerateIndexKey(pParse, pIdx, 1, 3, 0); + jmp2 = sqlite3VdbeAddOp4Int(v, OP_Found, j+2, 0, r1, pIdx->nColumn+1); addr = sqlite3VdbeAddOpList(v, ArraySize(idxErr), idxErr); sqlite3VdbeChangeP4(v, addr+1, "rowid ", P4_STATIC); sqlite3VdbeChangeP4(v, addr+3, " missing from index ", P4_STATIC); @@ -77547,11 +77894,11 @@ SQLITE_PRIVATE void sqlite3Pragma( sqlite3VdbeChangeP1(v, addr+3, j+2); sqlite3VdbeChangeP2(v, addr+3, addr+2); sqlite3VdbeJumpHere(v, addr+4); - sqlite3VdbeChangeP4(v, addr+6, + sqlite3VdbeChangeP4(v, addr+6, "wrong # of entries in index ", P4_STATIC); sqlite3VdbeChangeP4(v, addr+7, pIdx->zName, P4_STATIC); } - } + } } addr = sqlite3VdbeAddOpList(v, ArraySize(endCode), endCode); sqlite3VdbeChangeP2(v, addr, -mxErr); @@ -77573,7 +77920,7 @@ SQLITE_PRIVATE void sqlite3Pragma( ** encoding that will be used for the main database file if a new file ** is created. If an existing main database file is opened, then the ** default text encoding for the existing database is used. - ** + ** ** In all cases new databases created using the ATTACH command are ** created to use the same default text encoding as the main database. If ** the main database has not been initialized and/or created when ATTACH @@ -77615,9 +77962,9 @@ SQLITE_PRIVATE void sqlite3Pragma( ** will be overwritten when the schema is next loaded. If it does not ** already exists, it will be created to use the new encoding value. */ - if( - !(DbHasProperty(db, 0, DB_SchemaLoaded)) || - DbHasProperty(db, 0, DB_Empty) + if( + !(DbHasProperty(db, 0, DB_SchemaLoaded)) || + DbHasProperty(db, 0, DB_Empty) ){ for(pEnc=&encnames[0]; pEnc->zName; pEnc++){ if( 0==sqlite3StrICmp(zRight, pEnc->zName) ){ @@ -77659,9 +78006,9 @@ SQLITE_PRIVATE void sqlite3Pragma( ** The user-version is not used internally by SQLite. It may be used by ** applications for any purpose. */ - if( sqlite3StrICmp(zLeft, "schema_version")==0 - || sqlite3StrICmp(zLeft, "user_version")==0 - || sqlite3StrICmp(zLeft, "freelist_count")==0 + if( sqlite3StrICmp(zLeft, "schema_version")==0 + || sqlite3StrICmp(zLeft, "user_version")==0 + || sqlite3StrICmp(zLeft, "freelist_count")==0 ){ int iCookie; /* Cookie index. 1 for schema-cookie, 6 for user-cookie. */ sqlite3VdbeUsesBtree(v, iDb); @@ -77729,7 +78076,7 @@ SQLITE_PRIVATE void sqlite3Pragma( pBt = db->aDb[i].pBt; if( pBt==0 || (pPager = sqlite3BtreePager(pBt))==0 ){ zState = "closed"; - }else if( sqlite3_file_control(db, i ? db->aDb[i].zName : 0, + }else if( sqlite3_file_control(db, i ? db->aDb[i].zName : 0, SQLITE_FCNTL_LOCKSTATE, &j)==SQLITE_OK ){ zState = azLockName[j]; } @@ -77780,7 +78127,7 @@ SQLITE_PRIVATE void sqlite3Pragma( }else #endif - + {/* Empty ELSE clause */} /* Code an OP_Expire at the end of each PRAGMA program to cause @@ -77822,8 +78169,6 @@ pragma_out: ** This file contains the implementation of the sqlite3_prepare() ** interface, and routines that contribute to loading the database schema ** from disk. -** -** $Id: prepare.c,v 1.131 2009/08/06 17:43:31 drh Exp $ */ /* @@ -77841,7 +78186,7 @@ static void corruptSchema( sqlite3SetString(pData->pzErrMsg, db, "malformed database schema (%s)", zObj); if( zExtra ){ - *pData->pzErrMsg = sqlite3MAppendf(db, *pData->pzErrMsg, + *pData->pzErrMsg = sqlite3MAppendf(db, *pData->pzErrMsg, "%s - %s", *pData->pzErrMsg, zExtra); } } @@ -77955,7 +78300,7 @@ static int sqlite3InitOne(sqlite3 *db, int iDb, char **pzErrMsg){ /* ** The master database table has a structure like this */ - static const char master_schema[] = + static const char master_schema[] = "CREATE TABLE sqlite_master(\n" " type text,\n" " name text,\n" @@ -77965,7 +78310,7 @@ static int sqlite3InitOne(sqlite3 *db, int iDb, char **pzErrMsg){ ")" ; #ifndef SQLITE_OMIT_TEMPDB - static const char temp_master_schema[] = + static const char temp_master_schema[] = "CREATE TEMP TABLE sqlite_temp_master(\n" " type text,\n" " name text,\n" @@ -78026,7 +78371,7 @@ static int sqlite3InitOne(sqlite3 *db, int iDb, char **pzErrMsg){ } /* If there is not already a read-only (or read-write) transaction opened - ** on the b-tree database, open one now. If a transaction is opened, it + ** on the b-tree database, open one now. If a transaction is opened, it ** will be closed before this function returns. */ sqlite3BtreeEnter(pDb->pBt); if( !sqlite3BtreeIsInReadTrans(pDb->pBt) ){ @@ -78125,7 +78470,7 @@ static int sqlite3InitOne(sqlite3 *db, int iDb, char **pzErrMsg){ assert( db->init.busy ); { char *zSql; - zSql = sqlite3MPrintf(db, + zSql = sqlite3MPrintf(db, "SELECT name, rootpage, sql FROM '%q'.%s", db->aDb[iDb].zName, zMasterName); (void)sqlite3SafetyOff(db); @@ -78155,7 +78500,7 @@ static int sqlite3InitOne(sqlite3 *db, int iDb, char **pzErrMsg){ } if( rc==SQLITE_OK || (db->flags&SQLITE_RecoveryMode)){ /* Black magic: If the SQLITE_RecoveryMode flag is set, then consider - ** the schema loaded, even if errors occurred. In this situation the + ** the schema loaded, even if errors occurred. In this situation the ** current sqlite3_prepare() operation will fail, but the following one ** will attempt to compile the supplied statement against whatever subset ** of the schema was loaded before the error occurred. The primary @@ -78196,7 +78541,7 @@ error_out: SQLITE_PRIVATE int sqlite3Init(sqlite3 *db, char **pzErrMsg){ int i, rc; int commit_internal = !(db->flags&SQLITE_InternChanges); - + assert( sqlite3_mutex_held(db->mutex) ); rc = SQLITE_OK; db->init.busy = 1; @@ -78227,7 +78572,7 @@ SQLITE_PRIVATE int sqlite3Init(sqlite3 *db, char **pzErrMsg){ sqlite3CommitInternalChanges(db); } - return rc; + return rc; } /* @@ -78268,7 +78613,7 @@ static void schemaIsValid(Parse *pParse){ if( pBt==0 ) continue; /* If there is not already a read-only (or read-write) transaction opened - ** on the b-tree database, open one now. If a transaction is opened, it + ** on the b-tree database, open one now. If a transaction is opened, it ** will be closed immediately after reading the meta-value. */ if( !sqlite3BtreeIsInReadTrans(pBt) ){ rc = sqlite3BtreeBeginTrans(pBt, 0); @@ -78279,7 +78624,7 @@ static void schemaIsValid(Parse *pParse){ openedTransaction = 1; } - /* Read the schema cookie from the database. If it does not match the + /* Read the schema cookie from the database. If it does not match the ** value stored as part of the in the in-memory schema representation, ** set Parse.rc to SQLITE_SCHEMA. */ sqlite3BtreeGetMeta(pBt, BTREE_SCHEMA_VERSION, (u32 *)&cookie); @@ -78304,13 +78649,13 @@ static void schemaIsValid(Parse *pParse){ SQLITE_PRIVATE int sqlite3SchemaToIndex(sqlite3 *db, Schema *pSchema){ int i = -1000000; - /* If pSchema is NULL, then return -1000000. This happens when code in + /* If pSchema is NULL, then return -1000000. This happens when code in ** expr.c is trying to resolve a reference to a transient table (i.e. one - ** created by a sub-select). In this case the return value of this + ** created by a sub-select). In this case the return value of this ** function should never be used. ** ** We return -1000000 instead of the more usual -1 simply because using - ** -1000000 as the incorrect index into db->aDb[] is much + ** -1000000 as the incorrect index into db->aDb[] is much ** more likely to cause a segfault than -1 (of course there are assert() ** statements too, but it never hurts to play the odds). */ @@ -78374,8 +78719,8 @@ static int sqlite3Prepare( ** This thread is currently holding mutexes on all Btrees (because ** of the sqlite3BtreeEnterAll() in sqlite3LockAndPrepare()) so it ** is not possible for another thread to start a new schema change - ** while this routine is running. Hence, we do not need to hold - ** locks on the schema, we just need to make sure nobody else is + ** while this routine is running. Hence, we do not need to hold + ** locks on the schema, we just need to make sure nobody else is ** holding them. ** ** Note that setting READ_UNCOMMITTED overrides most lock detection, @@ -78605,7 +78950,7 @@ SQLITE_API int sqlite3_prepare_v2( ** Compile the UTF-16 encoded SQL statement zSql into a statement handle. */ static int sqlite3Prepare16( - sqlite3 *db, /* Database handle. */ + sqlite3 *db, /* Database handle. */ const void *zSql, /* UTF-8 encoded SQL statement. */ int nBytes, /* Length of zSql in bytes. */ int saveSqlFlag, /* True to save SQL text into the sqlite3_stmt */ @@ -78640,7 +78985,7 @@ static int sqlite3Prepare16( int chars_parsed = sqlite3Utf8CharLen(zSql8, (int)(zTail8-zSql8)); *pzTail = (u8 *)zSql + sqlite3Utf16ByteLen(zSql, chars_parsed); } - sqlite3DbFree(db, zSql8); + sqlite3DbFree(db, zSql8); rc = sqlite3ApiExit(db, rc); sqlite3_mutex_leave(db->mutex); return rc; @@ -78655,7 +79000,7 @@ static int sqlite3Prepare16( ** occurs. */ SQLITE_API int sqlite3_prepare16( - sqlite3 *db, /* Database handle. */ + sqlite3 *db, /* Database handle. */ const void *zSql, /* UTF-8 encoded SQL statement. */ int nBytes, /* Length of zSql in bytes. */ sqlite3_stmt **ppStmt, /* OUT: A pointer to the prepared statement */ @@ -78667,7 +79012,7 @@ SQLITE_API int sqlite3_prepare16( return rc; } SQLITE_API int sqlite3_prepare16_v2( - sqlite3 *db, /* Database handle. */ + sqlite3 *db, /* Database handle. */ const void *zSql, /* UTF-8 encoded SQL statement. */ int nBytes, /* Length of zSql in bytes. */ sqlite3_stmt **ppStmt, /* OUT: A pointer to the prepared statement */ @@ -78696,8 +79041,6 @@ SQLITE_API int sqlite3_prepare16_v2( ************************************************************************* ** This file contains C code routines that are called by the parser ** to handle SELECT statements in SQLite. -** -** $Id: select.c,v 1.526 2009/08/01 15:09:58 drh Exp $ */ @@ -78832,7 +79175,7 @@ SQLITE_PRIVATE int sqlite3JoinType(Parse *pParse, Token *pA, Token *pB, Token *p for(i=0; i<3 && apAll[i]; i++){ p = apAll[i]; for(j=0; jn==aKeyword[j].nChar + if( p->n==aKeyword[j].nChar && sqlite3StrNICmp((char*)p->z, &zKeyText[aKeyword[j].i], p->n)==0 ){ jointype |= aKeyword[j].code; break; @@ -78854,9 +79197,9 @@ SQLITE_PRIVATE int sqlite3JoinType(Parse *pParse, Token *pA, Token *pB, Token *p sqlite3ErrorMsg(pParse, "unknown or unsupported join type: " "%T %T%s%T", pA, pB, zSp, pC); jointype = JT_INNER; - }else if( (jointype & JT_OUTER)!=0 + }else if( (jointype & JT_OUTER)!=0 && (jointype & (JT_LEFT|JT_RIGHT))!=JT_LEFT ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "RIGHT and FULL OUTER JOINs are not currently supported"); jointype = JT_INNER; } @@ -78882,7 +79225,7 @@ static int columnIndex(Table *pTab, const char *zCol){ ** ** (tab1.col1 = tab2.col2) ** -** where tab1 is the iSrc'th table in SrcList pSrc and tab2 is the +** where tab1 is the iSrc'th table in SrcList pSrc and tab2 is the ** (iSrc+1)'th. Column col1 is column iColLeft of tab1, and col2 is ** column iColRight of tab2. */ @@ -78951,7 +79294,7 @@ static void setJoinExpr(Expr *p, int iTable){ p->iRightJoinTable = (i16)iTable; setJoinExpr(p->pLeft, iTable); p = p->pRight; - } + } } /* @@ -79021,7 +79364,7 @@ static int sqliteProcessJoin(Parse *pParse, Select *p){ } /* Create extra terms on the WHERE clause for each column named - ** in the USING clause. Example: If the two tables to be joined are + ** in the USING clause. Example: If the two tables to be joined are ** A and B and the USING clause names X, Y, and Z, then add this ** to the WHERE clause: A.X=B.X AND A.Y=B.Y AND A.Z=B.Z ** Report an error if any column mentioned in the USING clause is @@ -79125,8 +79468,8 @@ static void codeDistinct( v = pParse->pVdbe; r1 = sqlite3GetTempReg(pParse); + sqlite3VdbeAddOp4Int(v, OP_Found, iTab, addrRepeat, iMem, N); sqlite3VdbeAddOp3(v, OP_MakeRecord, iMem, N, r1); - sqlite3VdbeAddOp3(v, OP_Found, iTab, addrRepeat, r1); sqlite3VdbeAddOp2(v, OP_IdxInsert, iTab, r1); sqlite3ReleaseTempReg(pParse, r1); } @@ -79200,7 +79543,7 @@ static void selectInnerLoop( pDest->iMem = pParse->nMem+1; pDest->nMem = nResultCol; pParse->nMem += nResultCol; - }else{ + }else{ assert( pDest->nMem==nResultCol ); } regResult = pDest->iMem; @@ -79367,8 +79710,7 @@ static void selectInnerLoop( if( p->iLimit ){ assert( pOrderBy==0 ); /* If there is an ORDER BY, the call to ** pushOntoSorter() would have cleared p->iLimit */ - sqlite3VdbeAddOp2(v, OP_AddImm, p->iLimit, -1); - sqlite3VdbeAddOp2(v, OP_IfZero, p->iLimit, iBreak); + sqlite3VdbeAddOp3(v, OP_IfZero, p->iLimit, iBreak, -1); } } @@ -79480,7 +79822,7 @@ static void generateSortTail( #endif default: { int i; - assert( eDest==SRT_Output || eDest==SRT_Coroutine ); + assert( eDest==SRT_Output || eDest==SRT_Coroutine ); testcase( eDest==SRT_Output ); testcase( eDest==SRT_Coroutine ); for(i=0; i=0 && iColpEList->nExpr) ){ /* If iCol is less than zero, then the expression requests the - ** rowid of the sub-select or view. This expression is legal (see + ** rowid of the sub-select or view. This expression is legal (see ** test case misc2.2.2) - it always evaluates to NULL. */ NameContext sNC; @@ -79608,7 +79950,7 @@ static const char *columnType( sNC.pSrcList = pS->pSrc; sNC.pNext = pNC; sNC.pParse = pNC->pParse; - zType = columnType(&sNC, p, &zOriginDb, &zOriginTab, &zOriginCol); + zType = columnType(&sNC, p, &zOriginDb, &zOriginTab, &zOriginCol); } }else if( ALWAYS(pTab->pSchema) ){ /* A real table */ @@ -79643,12 +79985,12 @@ static const char *columnType( sNC.pSrcList = pS->pSrc; sNC.pNext = pNC; sNC.pParse = pNC->pParse; - zType = columnType(&sNC, p, &zOriginDb, &zOriginTab, &zOriginCol); + zType = columnType(&sNC, p, &zOriginDb, &zOriginTab, &zOriginCol); break; } #endif } - + if( pzOriginDb ){ assert( pzOriginTab && pzOriginCol ); *pzOriginDb = zOriginDb; @@ -79682,7 +80024,7 @@ static void generateColumnTypes( const char *zOrigCol = 0; zType = columnType(&sNC, p, &zOrigDb, &zOrigTab, &zOrigCol); - /* The vdbe must make its own copy of the column-type and other + /* The vdbe must make its own copy of the column-type and other ** column specific strings, in case the schema is reset before this ** virtual machine is deleted. */ @@ -79748,7 +80090,7 @@ static void generateColumnNames( zCol = pTab->aCol[iCol].zName; } if( !shortNames && !fullNames ){ - sqlite3VdbeSetColName(v, i, COLNAME_NAME, + sqlite3VdbeSetColName(v, i, COLNAME_NAME, sqlite3DbStrDup(db, pEList->a[i].zSpan), SQLITE_DYNAMIC); }else if( fullNames ){ char *zName = 0; @@ -79758,7 +80100,7 @@ static void generateColumnNames( sqlite3VdbeSetColName(v, i, COLNAME_NAME, zCol, SQLITE_TRANSIENT); } }else{ - sqlite3VdbeSetColName(v, i, COLNAME_NAME, + sqlite3VdbeSetColName(v, i, COLNAME_NAME, sqlite3DbStrDup(db, pEList->a[i].zSpan), SQLITE_DYNAMIC); } } @@ -79877,7 +80219,7 @@ static int selectColumnsFromExprList( /* ** Add type and collation information to a column list based on ** a SELECT statement. -** +** ** The column list presumably came from selectColumnNamesFromExprList(). ** The column list has only names, not types or collations. This ** routine goes through and adds the types and collations. @@ -79976,9 +80318,9 @@ SQLITE_PRIVATE Vdbe *sqlite3GetVdbe(Parse *pParse){ ** Compute the iLimit and iOffset fields of the SELECT based on the ** pLimit and pOffset expressions. pLimit and pOffset hold the expressions ** that appear in the original SQL statement after the LIMIT and OFFSET -** keywords. Or NULL if those keywords are omitted. iLimit and iOffset -** are the integer memory register numbers for counters used to compute -** the limit and offset. If there is no limit and/or offset, then +** keywords. Or NULL if those keywords are omitted. iLimit and iOffset +** are the integer memory register numbers for counters used to compute +** the limit and offset. If there is no limit and/or offset, then ** iLimit and iOffset are negative. ** ** This routine changes the values of iLimit and iOffset only if @@ -79994,10 +80336,10 @@ static void computeLimitRegisters(Parse *pParse, Select *p, int iBreak){ Vdbe *v = 0; int iLimit = 0; int iOffset; - int addr1; + int addr1, n; if( p->iLimit ) return; - /* + /* ** "LIMIT -1" always shows all rows. There is some ** contraversy about what the correct behavior should be. ** The current implementation interprets "LIMIT 0" to mean @@ -80009,10 +80351,18 @@ static void computeLimitRegisters(Parse *pParse, Select *p, int iBreak){ p->iLimit = iLimit = ++pParse->nMem; v = sqlite3GetVdbe(pParse); if( NEVER(v==0) ) return; /* VDBE should have already been allocated */ - sqlite3ExprCode(pParse, p->pLimit, iLimit); - sqlite3VdbeAddOp1(v, OP_MustBeInt, iLimit); - VdbeComment((v, "LIMIT counter")); - sqlite3VdbeAddOp2(v, OP_IfZero, iLimit, iBreak); + if( sqlite3ExprIsInteger(p->pLimit, &n) ){ + sqlite3VdbeAddOp2(v, OP_Integer, n, iLimit); + VdbeComment((v, "LIMIT counter")); + if( n==0 ){ + sqlite3VdbeAddOp2(v, OP_Goto, 0, iBreak); + } + }else{ + sqlite3ExprCode(pParse, p->pLimit, iLimit); + sqlite3VdbeAddOp1(v, OP_MustBeInt, iLimit); + VdbeComment((v, "LIMIT counter")); + sqlite3VdbeAddOp2(v, OP_IfZero, iLimit, iBreak); + } if( p->pOffset ){ p->iOffset = iOffset = ++pParse->nMem; pParse->nMem++; /* Allocate an extra register for limit+offset */ @@ -80071,7 +80421,7 @@ static int multiSelectOrderBy( ** ** "p" points to the right-most of the two queries. the query on the ** left is p->pPrior. The left query could also be a compound query -** in which case this routine will be called recursively. +** in which case this routine will be called recursively. ** ** The results of the total query are to be written into a destination ** of type eDest with parameter iParm. @@ -80348,7 +80698,7 @@ static int multiSelect( sqlite3VdbeAddOp2(v, OP_Rewind, tab1, iBreak); r1 = sqlite3GetTempReg(pParse); iStart = sqlite3VdbeAddOp2(v, OP_RowKey, tab1, r1); - sqlite3VdbeAddOp3(v, OP_NotFound, tab2, iCont, r1); + sqlite3VdbeAddOp4Int(v, OP_NotFound, tab2, iCont, r1, 0); sqlite3ReleaseTempReg(pParse, r1); selectInnerLoop(pParse, p, p->pEList, tab1, p->pEList->nExpr, 0, -1, &dest, iCont, iBreak); @@ -80361,7 +80711,7 @@ static int multiSelect( } } - /* Compute collating sequences used by + /* Compute collating sequences used by ** temporary tables needed to implement the compound select. ** Attach the KeyInfo structure to all temporary tables. ** @@ -80459,7 +80809,7 @@ static int generateOutputSubroutine( addr = sqlite3VdbeCurrentAddr(v); iContinue = sqlite3VdbeMakeLabel(v); - /* Suppress duplicates for UNION, EXCEPT, and INTERSECT + /* Suppress duplicates for UNION, EXCEPT, and INTERSECT */ if( regPrev ){ int j1, j2; @@ -80503,7 +80853,7 @@ static int generateOutputSubroutine( case SRT_Set: { int r1; assert( pIn->nMem==1 ); - p->affinity = + p->affinity = sqlite3CompareAffinity(p->pEList->a[0].pExpr, pDest->affinity); r1 = sqlite3GetTempReg(pParse); sqlite3VdbeAddOp4(v, OP_MakeRecord, pIn->iMem, 1, r1, &p->affinity, 1); @@ -80552,7 +80902,7 @@ static int generateOutputSubroutine( ** SRT_Output. This routine is never called with any other ** destination other than the ones handled above or SRT_Output. ** - ** For SRT_Output, results are stored in a sequence of registers. + ** For SRT_Output, results are stored in a sequence of registers. ** Then the OP_ResultRow opcode is used to cause sqlite3_step() to ** return the next row of result. */ @@ -80567,8 +80917,7 @@ static int generateOutputSubroutine( /* Jump to the end of the loop if the LIMIT is reached. */ if( p->iLimit ){ - sqlite3VdbeAddOp2(v, OP_AddImm, p->iLimit, -1); - sqlite3VdbeAddOp2(v, OP_IfZero, p->iLimit, iBreak); + sqlite3VdbeAddOp3(v, OP_IfZero, p->iLimit, iBreak, -1); } /* Generate the subroutine return @@ -80611,7 +80960,7 @@ static int generateOutputSubroutine( ** ** EofB: Called when data is exhausted from selectB. ** -** The implementation of the latter five subroutines depend on which +** The implementation of the latter five subroutines depend on which ** is used: ** ** @@ -80717,7 +81066,7 @@ static int multiSelectOrderBy( /* Patch up the ORDER BY clause */ - op = p->op; + op = p->op; pPrior = p->pPrior; assert( pPrior->pOrderBy==0 ); pOrderBy = p->pOrderBy; @@ -80812,7 +81161,7 @@ static int multiSelectOrderBy( } } } - + /* Separate the left and the right query from one another */ p->pPrior = 0; @@ -80864,7 +81213,7 @@ static int multiSelectOrderBy( sqlite3VdbeAddOp1(v, OP_Yield, regAddrA); VdbeNoopComment((v, "End coroutine for left SELECT")); - /* Generate a coroutine to evaluate the SELECT statement on + /* Generate a coroutine to evaluate the SELECT statement on ** the right - the "B" select */ addrSelectB = sqlite3VdbeCurrentAddr(v); @@ -80872,7 +81221,7 @@ static int multiSelectOrderBy( savedLimit = p->iLimit; savedOffset = p->iOffset; p->iLimit = regLimitB; - p->iOffset = 0; + p->iOffset = 0; sqlite3Select(pParse, p, &destB); p->iLimit = savedLimit; p->iOffset = savedOffset; @@ -80887,7 +81236,7 @@ static int multiSelectOrderBy( addrOutA = generateOutputSubroutine(pParse, p, &destA, pDest, regOutA, regPrev, pKeyDup, P4_KEYINFO_HANDOFF, labelEnd); - + /* Generate a subroutine that outputs the current row of the B ** select as the next output row of the compound select. */ @@ -80904,7 +81253,7 @@ static int multiSelectOrderBy( VdbeNoopComment((v, "eof-A subroutine")); if( op==TK_EXCEPT || op==TK_INTERSECT ){ addrEofA = sqlite3VdbeAddOp2(v, OP_Goto, 0, labelEnd); - }else{ + }else{ addrEofA = sqlite3VdbeAddOp2(v, OP_If, regEofB, labelEnd); sqlite3VdbeAddOp2(v, OP_Gosub, regOutB, addrOutB); sqlite3VdbeAddOp1(v, OP_Yield, regAddrB); @@ -80916,7 +81265,7 @@ static int multiSelectOrderBy( */ if( op==TK_INTERSECT ){ addrEofB = addrEofA; - }else{ + }else{ VdbeNoopComment((v, "eof-B subroutine")); addrEofB = sqlite3VdbeAddOp2(v, OP_If, regEofA, labelEnd); sqlite3VdbeAddOp2(v, OP_Gosub, regOutA, addrOutA); @@ -81015,13 +81364,13 @@ static void substSelect(sqlite3*, Select *, int, ExprList *); /* ** Scan through the expression pExpr. Replace every reference to ** a column in table number iTable with a copy of the iColumn-th -** entry in pEList. (But leave references to the ROWID column +** entry in pEList. (But leave references to the ROWID column ** unchanged.) ** ** This routine is part of the flattening procedure. A subquery ** whose result set is defined by pEList appears as entry in the ** FROM clause of a SELECT such that the VDBE cursor assigned to that -** FORM clause entry is iTable. This routine make the necessary +** FORM clause entry is iTable. This routine make the necessary ** changes to pExpr so that it refers directly to the source table ** of the subquery rather the result set of the subquery. */ @@ -81119,7 +81468,7 @@ static void substSelect( ** SELECT x+y AS a FROM t1 WHERE z<100 AND a>5 ** ** The code generated for this simpification gives the same result -** but only has to scan the data once. And because indices might +** but only has to scan the data once. And because indices might ** exist on the table t1, a complete scan of the data might be ** avoided. ** @@ -81165,10 +81514,10 @@ static void substSelect( ** ** (16) The outer query is not an aggregate or the subquery does ** not contain ORDER BY. (Ticket #2942) This used to not matter -** until we introduced the group_concat() function. +** until we introduced the group_concat() function. ** -** (17) The sub-query is not a compound select, or it is a UNION ALL -** compound clause made up entirely of non-aggregate queries, and +** (17) The sub-query is not a compound select, or it is a UNION ALL +** compound clause made up entirely of non-aggregate queries, and ** the parent query: ** ** * is not itself part of a compound select, @@ -81180,7 +81529,7 @@ static void substSelect( ** LIMIT and OFFSET clauses. ** ** (18) If the sub-query is a compound select, then all terms of the -** ORDER by clause of the parent must be simple references to +** ORDER by clause of the parent must be simple references to ** columns of the sub-query. ** ** (19) The subquery does not use LIMIT or the outer query does not @@ -81247,9 +81596,9 @@ static int flattenSubquery( return 0; /* Restriction (15) */ } if( pSubSrc->nSrc==0 ) return 0; /* Restriction (7) */ - if( ((pSub->selFlags & SF_Distinct)!=0 || pSub->pLimit) + if( ((pSub->selFlags & SF_Distinct)!=0 || pSub->pLimit) && (pSrc->nSrc>1 || isAgg) ){ /* Restrictions (4)(5)(8)(9) */ - return 0; + return 0; } if( (p->selFlags & SF_Distinct)!=0 && subqueryIsAgg ){ return 0; /* Restriction (6) */ @@ -81261,7 +81610,7 @@ static int flattenSubquery( if( pSub->pLimit && p->pWhere ) return 0; /* Restriction (19) */ /* OBSOLETE COMMENT 1: - ** Restriction 3: If the subquery is a join, make sure the subquery is + ** Restriction 3: If the subquery is a join, make sure the subquery is ** not used as the right operand of an outer join. Examples of why this ** is not allowed: ** @@ -81312,7 +81661,7 @@ static int flattenSubquery( testcase( (pSub1->selFlags & (SF_Distinct|SF_Aggregate))==SF_Distinct ); testcase( (pSub1->selFlags & (SF_Distinct|SF_Aggregate))==SF_Aggregate ); if( (pSub1->selFlags & (SF_Distinct|SF_Aggregate))!=0 - || (pSub1->pPrior && pSub1->op!=TK_ALL) + || (pSub1->pPrior && pSub1->op!=TK_ALL) || NEVER(pSub1->pSrc==0) || pSub1->pSrc->nSrc!=1 ){ return 0; @@ -81336,13 +81685,13 @@ static int flattenSubquery( pParse->zAuthContext = zSavedAuthContext; /* If the sub-query is a compound SELECT statement, then (by restrictions - ** 17 and 18 above) it must be a UNION ALL and the parent query must + ** 17 and 18 above) it must be a UNION ALL and the parent query must ** be of the form: ** - ** SELECT FROM () + ** SELECT FROM () ** ** followed by any ORDER BY, LIMIT and/or OFFSET clauses. This block - ** creates N-1 copies of the parent query without any ORDER BY, LIMIT or + ** creates N-1 copies of the parent query without any ORDER BY, LIMIT or ** OFFSET clauses and joins them to the left-hand-side of the original ** using UNION ALL operators. In this case N is the number of simple ** select statements in the compound sub-query. @@ -81393,7 +81742,7 @@ static int flattenSubquery( if( db->mallocFailed ) return 1; } - /* Begin flattening the iFrom-th entry of the FROM clause + /* Begin flattening the iFrom-th entry of the FROM clause ** in the outer query. */ pSub = pSub1 = pSubitem->pSelect; @@ -81491,10 +81840,10 @@ static int flattenSubquery( memset(&pSubSrc->a[i], 0, sizeof(pSubSrc->a[i])); } pSrc->a[iFrom].jointype = jointype; - - /* Now begin substituting subquery result set expressions for + + /* Now begin substituting subquery result set expressions for ** references to the iParent in the outer query. - ** + ** ** Example: ** ** SELECT a+5, b*10 FROM (SELECT x*3 AS a, y+10 AS b FROM t1) WHERE a>b; @@ -81535,7 +81884,7 @@ static int flattenSubquery( pParent->pHaving = pParent->pWhere; pParent->pWhere = pWhere; pParent->pHaving = substExpr(db, pParent->pHaving, iParent, pSub->pEList); - pParent->pHaving = sqlite3ExprAnd(db, pParent->pHaving, + pParent->pHaving = sqlite3ExprAnd(db, pParent->pHaving, sqlite3ExprDup(db, pSub->pHaving, 0)); assert( pParent->pGroupBy==0 ); pParent->pGroupBy = sqlite3ExprListDup(db, pSub->pGroupBy, 0); @@ -81543,12 +81892,12 @@ static int flattenSubquery( pParent->pWhere = substExpr(db, pParent->pWhere, iParent, pSub->pEList); pParent->pWhere = sqlite3ExprAnd(db, pParent->pWhere, pWhere); } - + /* The flattened query is distinct if either the inner or the - ** outer query is distinct. + ** outer query is distinct. */ pParent->selFlags |= pSub->selFlags & SF_Distinct; - + /* ** SELECT ... FROM (SELECT ... LIMIT a OFFSET b) LIMIT x OFFSET y; ** @@ -81572,7 +81921,7 @@ static int flattenSubquery( /* ** Analyze the SELECT statement passed as an argument to see if it -** is a min() or max() query. Return WHERE_ORDERBY_MIN or WHERE_ORDERBY_MAX if +** is a min() or max() query. Return WHERE_ORDERBY_MIN or WHERE_ORDERBY_MAX if ** it is, or 0 otherwise. At present, a query is considered to be ** a min()/max() query if: ** @@ -81603,7 +81952,7 @@ static u8 minMaxQuery(Select *p){ /* ** The select statement passed as the first argument is an aggregate query. -** The second argment is the associated aggregate-info object. This +** The second argment is the associated aggregate-info object. This ** function tests if the SELECT is of the form: ** ** SELECT count(*) FROM @@ -81618,7 +81967,7 @@ static Table *isSimpleCount(Select *p, AggInfo *pAggInfo){ assert( !p->pGroupBy ); - if( p->pWhere || p->pEList->nExpr!=1 + if( p->pWhere || p->pEList->nExpr!=1 || p->pSrc->nSrc!=1 || p->pSrc->a[0].pSelect ){ return 0; @@ -81638,8 +81987,8 @@ static Table *isSimpleCount(Select *p, AggInfo *pAggInfo){ /* ** If the source-list item passed as an argument was augmented with an ** INDEXED BY clause, then try to locate the specified index. If there -** was such a clause and the named index cannot be found, return -** SQLITE_ERROR and leave an error in pParse. Otherwise, populate +** was such a clause and the named index cannot be found, return +** SQLITE_ERROR and leave an error in pParse. Otherwise, populate ** pFrom->pIndex and return SQLITE_OK. */ SQLITE_PRIVATE int sqlite3IndexedByLookup(Parse *pParse, struct SrcList_item *pFrom){ @@ -81647,8 +81996,8 @@ SQLITE_PRIVATE int sqlite3IndexedByLookup(Parse *pParse, struct SrcList_item *pF Table *pTab = pFrom->pTab; char *zIndex = pFrom->zIndex; Index *pIdx; - for(pIdx=pTab->pIndex; - pIdx && sqlite3StrICmp(pIdx->zName, zIndex); + for(pIdx=pTab->pIndex; + pIdx && sqlite3StrICmp(pIdx->zName, zIndex); pIdx=pIdx->pNext ); if( !pIdx ){ @@ -81667,7 +82016,7 @@ SQLITE_PRIVATE int sqlite3IndexedByLookup(Parse *pParse, struct SrcList_item *pF ** (1) Make sure VDBE cursor numbers have been assigned to every ** element of the FROM clause. ** -** (2) Fill in the pTabList->a[].pTab fields in the SrcList that +** (2) Fill in the pTabList->a[].pTab fields in the SrcList that ** defines FROM clause. When views appear in the FROM clause, ** fill pTabList->a[].pSelect with a copy of the SELECT statement ** that implements the view. A copy is made of the view's SELECT @@ -81739,7 +82088,7 @@ static int selectExpander(Walker *pWalker, Select *p){ }else{ /* An ordinary table or view name in the FROM clause */ assert( pFrom->pTab==0 ); - pFrom->pTab = pTab = + pFrom->pTab = pTab = sqlite3LocateTable(pParse,0,pFrom->zName,pFrom->zDatabase); if( pTab==0 ) return WRC_Abort; pTab->nRef++; @@ -81852,7 +82201,7 @@ static int selectExpander(Walker *pWalker, Select *p){ struct SrcList_item *pLeft = &pTabList->a[i-1]; if( (pLeft[1].jointype & JT_NATURAL)!=0 && columnIndex(pLeft->pTab, zName)>=0 ){ - /* In a NATURAL join, omit the join columns from the + /* In a NATURAL join, omit the join columns from the ** table on the right */ continue; } @@ -81908,8 +82257,8 @@ static int selectExpander(Walker *pWalker, Select *p){ ** ** When this routine is the Walker.xExprCallback then expression trees ** are walked without any actions being taken at each node. Presumably, -** when this routine is used for Walker.xExprCallback then -** Walker.xSelectCallback is set to do something useful for every +** when this routine is used for Walker.xExprCallback then +** Walker.xSelectCallback is set to do something useful for every ** subquery in the parser tree. */ static int exprWalkNoop(Walker *NotUsed, Expr *NotUsed2){ @@ -82138,7 +82487,7 @@ static void updateAccumulator(Parse *pParse, AggInfo *pAggInfo){ } /* -** Generate code for the SELECT statement given in the p argument. +** Generate code for the SELECT statement given in the p argument. ** ** The results are distributed in various ways depending on the ** contents of the SelectDest structure pointed to by argument pDest @@ -82155,7 +82504,7 @@ static void updateAccumulator(Parse *pParse, AggInfo *pAggInfo){ ** of the query. This destination implies "LIMIT 1". ** ** SRT_Set The result must be a single column. Store each -** row of result as the key in table pDest->iParm. +** row of result as the key in table pDest->iParm. ** Apply the affinity pDest->affinity before storing ** results. Used to implement "IN (SELECT ...)". ** @@ -82222,7 +82571,7 @@ SQLITE_PRIVATE int sqlite3Select( memset(&sAggInfo, 0, sizeof(sAggInfo)); if( IgnorableOrderby(pDest) ){ - assert(pDest->eDest==SRT_Exists || pDest->eDest==SRT_Union || + assert(pDest->eDest==SRT_Exists || pDest->eDest==SRT_Union || pDest->eDest==SRT_Except || pDest->eDest==SRT_Discard); /* If ORDER BY makes no difference in the output then neither does ** DISTINCT so it can be removed too. */ @@ -82339,7 +82688,7 @@ SQLITE_PRIVATE int sqlite3Select( } /* If there is an ORDER BY clause, then this sorting - ** index might end up being unused if the data can be + ** index might end up being unused if the data can be ** extracted in pre-sorted order. If that is the case, then the ** OP_OpenEphemeral instruction will be changed to an OP_Noop once ** we figure out that the sorting index is not needed. The addrSortIndex @@ -82389,7 +82738,7 @@ SQLITE_PRIVATE int sqlite3Select( pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pOrderBy, 0); if( pWInfo==0 ) goto select_end; - /* If sorting index that was created by a prior OP_OpenEphemeral + /* If sorting index that was created by a prior OP_OpenEphemeral ** instruction ended up not being needed, then change the OP_OpenEphemeral ** into an OP_Noop. */ @@ -82434,7 +82783,7 @@ SQLITE_PRIVATE int sqlite3Select( } } - + /* Create a label to jump to when we want to abort the query */ addrEnd = sqlite3VdbeMakeLabel(v); @@ -82477,12 +82826,12 @@ SQLITE_PRIVATE int sqlite3Select( /* If there is a GROUP BY clause we might need a sorting index to ** implement it. Allocate that sorting index now. If it turns out ** that we do not need it after all, the OpenEphemeral instruction - ** will be converted into a Noop. + ** will be converted into a Noop. */ sAggInfo.sortingIdx = pParse->nTab++; pKeyInfo = keyInfoFromExprList(pParse, pGroupBy); - addrSortingIdx = sqlite3VdbeAddOp4(v, OP_OpenEphemeral, - sAggInfo.sortingIdx, sAggInfo.nSortingColumn, + addrSortingIdx = sqlite3VdbeAddOp4(v, OP_OpenEphemeral, + sAggInfo.sortingIdx, sAggInfo.nSortingColumn, 0, (char*)pKeyInfo, P4_KEYINFO_HANDOFF); /* Initialize memory locations used by GROUP BY aggregate processing @@ -82549,7 +82898,7 @@ SQLITE_PRIVATE int sqlite3Select( int r1 = j + regBase; int r2; - r2 = sqlite3ExprCodeGetColumn(pParse, + r2 = sqlite3ExprCodeGetColumn(pParse, pCol->pTab, pCol->iColumn, pCol->iTable, r1, 0); if( r1!=r2 ){ sqlite3VdbeAddOp2(v, OP_SCopy, r2, r1); @@ -82661,7 +83010,7 @@ SQLITE_PRIVATE int sqlite3Select( sqlite3VdbeResolveLabel(v, addrReset); resetAccumulator(pParse, &sAggInfo); sqlite3VdbeAddOp1(v, OP_Return, regReset); - + } /* endif pGroupBy. Begin aggregate queries without GROUP BY: */ else { ExprList *pDel = 0; @@ -82699,7 +83048,7 @@ SQLITE_PRIVATE int sqlite3Select( ** and pKeyInfo to the KeyInfo structure required to navigate the ** index. ** - ** In practice the KeyInfo structure will not be used. It is only + ** In practice the KeyInfo structure will not be used. It is only ** passed to keep OP_OpenRead happy. */ for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){ @@ -82728,11 +83077,11 @@ SQLITE_PRIVATE int sqlite3Select( ** SELECT max(x) FROM ... ** ** If it is, then ask the code in where.c to attempt to sort results - ** as if there was an "ORDER ON x" or "ORDER ON x DESC" clause. + ** as if there was an "ORDER ON x" or "ORDER ON x DESC" clause. ** If where.c is able to produce results sorted in this order, then - ** add vdbe code to break out of the processing loop after the - ** first iteration (since the first iteration of the loop is - ** guaranteed to operate on the row with the minimum or maximum + ** add vdbe code to break out of the processing loop after the + ** first iteration (since the first iteration of the loop is + ** guaranteed to operate on the row with the minimum or maximum ** value of x, the only row required). ** ** A special flag must be passed to sqlite3WhereBegin() to slightly @@ -82743,7 +83092,7 @@ SQLITE_PRIVATE int sqlite3Select( ** for x. ** ** + The optimizer code in where.c (the thing that decides which - ** index or indices to use) should place a different priority on + ** index or indices to use) should place a different priority on ** satisfying the 'ORDER BY' clause than it does in other cases. ** Refer to code and comments in where.c for details. */ @@ -82758,7 +83107,7 @@ SQLITE_PRIVATE int sqlite3Select( pMinMax->a[0].pExpr->op = TK_COLUMN; } } - + /* This case runs if the aggregate has no GROUP BY clause. The ** processing is much simpler since there is only a single row ** of output. @@ -82781,12 +83130,12 @@ SQLITE_PRIVATE int sqlite3Select( pOrderBy = 0; sqlite3ExprIfFalse(pParse, pHaving, addrEnd, SQLITE_JUMPIFNULL); - selectInnerLoop(pParse, p, p->pEList, 0, 0, 0, -1, + selectInnerLoop(pParse, p, p->pEList, 0, 0, 0, -1, pDest, addrEnd, addrEnd); sqlite3ExprListDelete(db, pDel); } sqlite3VdbeResolveLabel(v, addrEnd); - + } /* endif aggregate query */ /* If there is an ORDER BY clause, then we need to sort the results @@ -82827,7 +83176,7 @@ select_end: ** The following code is used for testing and debugging only. The code ** that follows does not appear in normal builds. ** -** These routines are used to print out the content of all or part of a +** These routines are used to print out the content of all or part of a ** parse structures such as Select or Expr. Such printouts are useful ** for helping to understand what is happening inside the code generator ** during the execution of complex SELECT statements. @@ -82936,8 +83285,6 @@ SQLITE_PRIVATE void sqlite3PrintSelect(Select *p, int indent){ ** ** These routines are in a separate files so that they will not be linked ** if they are not used. -** -** $Id: table.c,v 1.40 2009/04/10 14:28:00 drh Exp $ */ #ifndef SQLITE_OMIT_GET_TABLE @@ -83031,7 +83378,7 @@ malloc_failed: ** at the conclusion of the call. ** ** The result that is written to ***pazResult is held in memory obtained -** from malloc(). But the caller cannot free this memory directly. +** from malloc(). But the caller cannot free this memory directly. ** Instead, the entire table should be passed to sqlite3_free_table() when ** the calling procedure is finished using it. */ @@ -83128,9 +83475,7 @@ SQLITE_API void sqlite3_free_table( ** May you share freely, never taking more than you give. ** ************************************************************************* -** -** -** $Id: trigger.c,v 1.143 2009/08/10 03:57:58 shane Exp $ +** This file contains the implementation for TRIGGERs */ #ifndef SQLITE_OMIT_TRIGGER @@ -83152,7 +83497,7 @@ SQLITE_PRIVATE void sqlite3DeleteTriggerStep(sqlite3 *db, TriggerStep *pTriggerS } /* -** Given table pTab, return a list of all the triggers attached to +** Given table pTab, return a list of all the triggers attached to ** the table. The list is connected by Trigger.pNext pointers. ** ** All of the triggers on pTab that are in the same database as pTab @@ -83178,7 +83523,7 @@ SQLITE_PRIVATE Trigger *sqlite3TriggerList(Parse *pParse, Table *pTab){ for(p=sqliteHashFirst(&pTmpSchema->trigHash); p; p=sqliteHashNext(p)){ Trigger *pTrig = (Trigger *)sqliteHashData(p); if( pTrig->pTabSchema==pTab->pSchema - && 0==sqlite3StrICmp(pTrig->table, pTab->zName) + && 0==sqlite3StrICmp(pTrig->table, pTab->zName) ){ pTrig->pNext = (pList ? pList : pTab->pTrigger); pList = pTrig; @@ -83254,7 +83599,7 @@ SQLITE_PRIVATE void sqlite3BeginTrigger( /* Ensure the table name matches database name and that the table exists */ if( db->mallocFailed ) goto trigger_cleanup; assert( pTableName->nSrc==1 ); - if( sqlite3FixInit(&sFix, pParse, iDb, "trigger", pName) && + if( sqlite3FixInit(&sFix, pParse, iDb, "trigger", pName) && sqlite3FixSrcList(&sFix, pTableName) ){ goto trigger_cleanup; } @@ -83304,7 +83649,7 @@ SQLITE_PRIVATE void sqlite3BeginTrigger( ** of triggers. */ if( pTab->pSelect && tr_tm!=TK_INSTEAD ){ - sqlite3ErrorMsg(pParse, "cannot create %s trigger on view: %S", + sqlite3ErrorMsg(pParse, "cannot create %s trigger on view: %S", (tr_tm == TK_BEFORE)?"BEFORE":"AFTER", pTableName, 0); goto trigger_cleanup; } @@ -83394,12 +83739,12 @@ SQLITE_PRIVATE void sqlite3FinishTrigger( } nameToken.z = pTrig->zName; nameToken.n = sqlite3Strlen30(nameToken.z); - if( sqlite3FixInit(&sFix, pParse, iDb, "trigger", &nameToken) + if( sqlite3FixInit(&sFix, pParse, iDb, "trigger", &nameToken) && sqlite3FixTriggerStep(&sFix, pTrig->step_list) ){ goto triggerfinish_cleanup; } - /* if we are not initializing, and this trigger is not on a TEMP table, + /* if we are not initializing, and this trigger is not on a TEMP table, ** build the sqlite_master entry */ if( !db->init.busy ){ @@ -83449,7 +83794,7 @@ triggerfinish_cleanup: ** a trigger step. Return a pointer to a TriggerStep structure. ** ** The parser calls this routine when it finds a SELECT statement in -** body of a TRIGGER. +** body of a TRIGGER. */ SQLITE_PRIVATE TriggerStep *sqlite3TriggerSelectStep(sqlite3 *db, Select *pSelect){ TriggerStep *pTriggerStep = sqlite3DbMallocZero(db, sizeof(TriggerStep)); @@ -83568,7 +83913,7 @@ SQLITE_PRIVATE TriggerStep *sqlite3TriggerDeleteStep( return pTriggerStep; } -/* +/* ** Recursively delete a Trigger structure */ SQLITE_PRIVATE void sqlite3DeleteTrigger(sqlite3 *db, Trigger *pTrigger){ @@ -83582,7 +83927,7 @@ SQLITE_PRIVATE void sqlite3DeleteTrigger(sqlite3 *db, Trigger *pTrigger){ } /* -** This function is called to drop a trigger from the database schema. +** This function is called to drop a trigger from the database schema. ** ** This may be called directly from the parser and therefore identifies ** the trigger by name. The sqlite3DropTriggerPtr() routine does the @@ -83635,7 +83980,7 @@ static Table *tableOfTrigger(Trigger *pTrigger){ /* -** Drop a trigger given a pointer to that trigger. +** Drop a trigger given a pointer to that trigger. */ SQLITE_PRIVATE void sqlite3DropTriggerPtr(Parse *pParse, Trigger *pTrigger){ Table *pTable; @@ -83726,12 +84071,12 @@ static int checkColumnOverlap(IdList *pIdList, ExprList *pEList){ for(e=0; enExpr; e++){ if( sqlite3IdListIndex(pIdList, pEList->a[e].zName)>=0 ) return 1; } - return 0; + return 0; } /* ** Return a list of all triggers on table pTab if there exists at least -** one trigger that must be fired when an operation of type 'op' is +** one trigger that must be fired when an operation of type 'op' is ** performed on the table, and, if that operation is an UPDATE, if at ** least one of the columns in pChanges is being modified. */ @@ -83789,13 +84134,13 @@ static SrcList *targetSrcList( } /* -** Generate VDBE code for the statements inside the body of a single +** Generate VDBE code for the statements inside the body of a single ** trigger. */ static int codeTriggerProgram( Parse *pParse, /* The parser context */ TriggerStep *pStepList, /* List of statements inside the trigger body */ - int orconf /* Conflict algorithm. (OE_Abort, etc) */ + int orconf /* Conflict algorithm. (OE_Abort, etc) */ ){ TriggerStep *pStep; Vdbe *v = pParse->pVdbe; @@ -83822,26 +84167,26 @@ static int codeTriggerProgram( switch( pStep->op ){ case TK_UPDATE: { - sqlite3Update(pParse, + sqlite3Update(pParse, targetSrcList(pParse, pStep), - sqlite3ExprListDup(db, pStep->pExprList, 0), - sqlite3ExprDup(db, pStep->pWhere, 0), + sqlite3ExprListDup(db, pStep->pExprList, 0), + sqlite3ExprDup(db, pStep->pWhere, 0), pParse->eOrconf ); break; } case TK_INSERT: { - sqlite3Insert(pParse, + sqlite3Insert(pParse, targetSrcList(pParse, pStep), - sqlite3ExprListDup(db, pStep->pExprList, 0), - sqlite3SelectDup(db, pStep->pSelect, 0), - sqlite3IdListDup(db, pStep->pIdList), + sqlite3ExprListDup(db, pStep->pExprList, 0), + sqlite3SelectDup(db, pStep->pSelect, 0), + sqlite3IdListDup(db, pStep->pIdList), pParse->eOrconf ); break; } case TK_DELETE: { - sqlite3DeleteFrom(pParse, + sqlite3DeleteFrom(pParse, targetSrcList(pParse, pStep), sqlite3ExprDup(db, pStep->pWhere, 0) ); @@ -83855,7 +84200,7 @@ static int codeTriggerProgram( sqlite3SelectDelete(db, pSelect); break; } - } + } if( pStep->op!=TK_SELECT ){ sqlite3VdbeAddOp0(v, OP_ResetCount); } @@ -83899,7 +84244,7 @@ static void transferParseError(Parse *pTo, Parse *pFrom){ } /* -** Create and populate a new TriggerPrg object with a sub-program +** Create and populate a new TriggerPrg object with a sub-program ** implementing trigger pTrigger with ON CONFLICT policy orconf. */ static TriggerPrg *codeRowTrigger( @@ -83921,7 +84266,7 @@ static TriggerPrg *codeRowTrigger( assert( pTrigger->zName==0 || pTab==tableOfTrigger(pTrigger) ); /* Allocate the TriggerPrg and SubProgram objects. To ensure that they - ** are freed if an error occurs, link them into the Parse.pTriggerPrg + ** are freed if an error occurs, link them into the Parse.pTriggerPrg ** list of the top-level Parse object sooner rather than later. */ pPrg = sqlite3DbMallocZero(db, sizeof(TriggerPrg)); if( !pPrg ) return 0; @@ -83932,9 +84277,10 @@ static TriggerPrg *codeRowTrigger( pProgram->nRef = 1; pPrg->pTrigger = pTrigger; pPrg->orconf = orconf; - pPrg->oldmask = 0xffffffff; + pPrg->aColmask[0] = 0xffffffff; + pPrg->aColmask[1] = 0xffffffff; - /* Allocate and populate a new Parse context to use for coding the + /* Allocate and populate a new Parse context to use for coding the ** trigger sub-program. */ pSubParse = sqlite3StackAllocZero(db, sizeof(Parse)); if( !pSubParse ) return 0; @@ -83948,7 +84294,7 @@ static TriggerPrg *codeRowTrigger( v = sqlite3GetVdbe(pSubParse); if( v ){ - VdbeComment((v, "Start: %s.%s (%s %s%s%s ON %s)", + VdbeComment((v, "Start: %s.%s (%s %s%s%s ON %s)", pTrigger->zName, onErrorText(orconf), (pTrigger->tr_tm==TRIGGER_BEFORE ? "BEFORE" : "AFTER"), (pTrigger->op==TK_UPDATE ? "UPDATE" : ""), @@ -83957,18 +84303,18 @@ static TriggerPrg *codeRowTrigger( pTab->zName )); #ifndef SQLITE_OMIT_TRACE - sqlite3VdbeChangeP4(v, -1, + sqlite3VdbeChangeP4(v, -1, sqlite3MPrintf(db, "-- TRIGGER %s", pTrigger->zName), P4_DYNAMIC ); #endif /* If one was specified, code the WHEN clause. If it evaluates to false - ** (or NULL) the sub-vdbe is immediately halted by jumping to the + ** (or NULL) the sub-vdbe is immediately halted by jumping to the ** OP_Halt inserted at the end of the program. */ if( pTrigger->pWhen ){ pWhen = sqlite3ExprDup(db, pTrigger->pWhen, 0); - if( SQLITE_OK==sqlite3ResolveExprNames(&sNC, pWhen) - && db->mallocFailed==0 + if( SQLITE_OK==sqlite3ResolveExprNames(&sNC, pWhen) + && db->mallocFailed==0 ){ iEndTrigger = sqlite3VdbeMakeLabel(v); sqlite3ExprIfFalse(pSubParse, pWhen, iEndTrigger, SQLITE_JUMPIFNULL); @@ -83993,7 +84339,8 @@ static TriggerPrg *codeRowTrigger( pProgram->nMem = pSubParse->nMem; pProgram->nCsr = pSubParse->nTab; pProgram->token = (void *)pTrigger; - pPrg->oldmask = pSubParse->oldmask; + pPrg->aColmask[0] = pSubParse->oldmask; + pPrg->aColmask[1] = pSubParse->newmask; sqlite3VdbeDelete(v); } @@ -84003,7 +84350,7 @@ static TriggerPrg *codeRowTrigger( return pPrg; } - + /* ** Return a pointer to a TriggerPrg object containing the sub-program for ** trigger pTrigger with default ON CONFLICT algorithm orconf. If no such @@ -84025,8 +84372,8 @@ static TriggerPrg *getRowTrigger( ** process of being coded). If this is the case, then an entry with ** a matching TriggerPrg.pTrigger field will be present somewhere ** in the Parse.pTriggerPrg list. Search for such an entry. */ - for(pPrg=pRoot->pTriggerPrg; - pPrg && (pPrg->pTrigger!=pTrigger || pPrg->orconf!=orconf); + for(pPrg=pRoot->pTriggerPrg; + pPrg && (pPrg->pTrigger!=pTrigger || pPrg->orconf!=orconf); pPrg=pPrg->pNext ); @@ -84039,7 +84386,7 @@ static TriggerPrg *getRowTrigger( } /* -** Generate code for the trigger program associated with trigger p on +** Generate code for the trigger program associated with trigger p on ** table pTab. The reg, orconf and ignoreJump parameters passed to this ** function are the same as those described in the header function for ** sqlite3CodeRowTrigger() @@ -84057,7 +84404,7 @@ SQLITE_PRIVATE void sqlite3CodeRowTriggerDirect( pPrg = getRowTrigger(pParse, p, pTab, orconf); assert( pPrg || pParse->nErr || pParse->db->mallocFailed ); - /* Code the OP_Program opcode in the parent VDBE. P4 of the OP_Program + /* Code the OP_Program opcode in the parent VDBE. P4 of the OP_Program ** is a pointer to the sub-vdbe containing the trigger program. */ if( pPrg ){ sqlite3VdbeAddOp3(v, OP_Program, reg, ignoreJump, ++pParse->nMem); @@ -84085,7 +84432,7 @@ SQLITE_PRIVATE void sqlite3CodeRowTriggerDirect( ** If there are no triggers that fire at the specified time for the specified ** operation on pTab, this function is a no-op. ** -** The reg argument is the address of the first in an array of registers +** The reg argument is the address of the first in an array of registers ** that contain the values substituted for the new.* and old.* references ** in the trigger program. If N is the number of columns in table pTab ** (a copy of pTab->nCol), then registers are populated as follows: @@ -84102,12 +84449,12 @@ SQLITE_PRIVATE void sqlite3CodeRowTriggerDirect( ** reg+N+N+1 NEW.* value of right-most column of pTab ** ** For ON DELETE triggers, the registers containing the NEW.* values will -** never be accessed by the trigger program, so they are not allocated or -** populated by the caller (there is no data to populate them with anyway). +** never be accessed by the trigger program, so they are not allocated or +** populated by the caller (there is no data to populate them with anyway). ** Similarly, for ON INSERT triggers the values stored in the OLD.* registers ** are never accessed, and so are not allocated by the caller. So, for an ** ON INSERT trigger, the value passed to this function as parameter reg -** is not a readable register, although registers (reg+N) through +** is not a readable register, although registers (reg+N) through ** (reg+N+N+1) are. ** ** Parameter orconf is the default conflict resolution algorithm for the @@ -84139,12 +84486,12 @@ SQLITE_PRIVATE void sqlite3CodeRowTrigger( ** or else it must be a TEMP trigger. */ assert( p->pSchema!=0 ); assert( p->pTabSchema!=0 ); - assert( p->pSchema==p->pTabSchema + assert( p->pSchema==p->pTabSchema || p->pSchema==pParse->db->aDb[1].pSchema ); /* Determine whether we should code this trigger */ - if( p->op==op - && p->tr_tm==tr_tm + if( p->op==op + && p->tr_tm==tr_tm && checkColumnOverlap(p->pColumns, pChanges) ){ sqlite3CodeRowTriggerDirect(pParse, p, pTab, reg, orconf, ignoreJump); @@ -84153,28 +84500,36 @@ SQLITE_PRIVATE void sqlite3CodeRowTrigger( } /* -** Triggers fired by UPDATE or DELETE statements may access values stored -** in the old.* pseudo-table. This function returns a 32-bit bitmask -** indicating which columns of the old.* table actually are used by -** triggers. This information may be used by the caller to avoid having -** to load the entire old.* record into memory when executing an UPDATE -** or DELETE command. +** Triggers may access values stored in the old.* or new.* pseudo-table. +** This function returns a 32-bit bitmask indicating which columns of the +** old.* or new.* tables actually are used by triggers. This information +** may be used by the caller, for example, to avoid having to load the entire +** old.* record into memory when executing an UPDATE or DELETE command. ** ** Bit 0 of the returned mask is set if the left-most column of the -** table may be accessed using an old.reference. Bit 1 is set if +** table may be accessed using an [old|new].reference. Bit 1 is set if ** the second leftmost column value is required, and so on. If there ** are more than 32 columns in the table, and at least one of the columns ** with an index greater than 32 may be accessed, 0xffffffff is returned. ** -** It is not possible to determine if the old.rowid column is accessed -** by triggers. The caller must always assume that it is. +** It is not possible to determine if the old.rowid or new.rowid column is +** accessed by triggers. The caller must always assume that it is. ** -** There is no equivalent function for new.* references. +** Parameter isNew must be either 1 or 0. If it is 0, then the mask returned +** applies to the old.* table. If 1, the new.* table. +** +** Parameter tr_tm must be a mask with one or both of the TRIGGER_BEFORE +** and TRIGGER_AFTER bits set. Values accessed by BEFORE triggers are only +** included in the returned mask if the TRIGGER_BEFORE bit is set in the +** tr_tm parameter. Similarly, values accessed by AFTER triggers are only +** included in the returned mask if the TRIGGER_AFTER bit is set in tr_tm. */ -SQLITE_PRIVATE u32 sqlite3TriggerOldmask( +SQLITE_PRIVATE u32 sqlite3TriggerColmask( Parse *pParse, /* Parse context */ Trigger *pTrigger, /* List of triggers on table pTab */ ExprList *pChanges, /* Changes list for any UPDATE OF triggers */ + int isNew, /* 1 for new.* ref mask, 0 for old.* ref mask */ + int tr_tm, /* Mask of TRIGGER_BEFORE|TRIGGER_AFTER */ Table *pTab, /* The table to code triggers from */ int orconf /* Default ON CONFLICT policy for trigger steps */ ){ @@ -84182,12 +84537,15 @@ SQLITE_PRIVATE u32 sqlite3TriggerOldmask( u32 mask = 0; Trigger *p; + assert( isNew==1 || isNew==0 ); for(p=pTrigger; p; p=p->pNext){ - if( p->op==op && checkColumnOverlap(p->pColumns,pChanges) ){ + if( p->op==op && (tr_tm&p->tr_tm) + && checkColumnOverlap(p->pColumns,pChanges) + ){ TriggerPrg *pPrg; pPrg = getRowTrigger(pParse, p, pTab, orconf); if( pPrg ){ - mask |= pPrg->oldmask; + mask |= pPrg->aColmask[isNew]; } } } @@ -84212,8 +84570,6 @@ SQLITE_PRIVATE u32 sqlite3TriggerOldmask( ************************************************************************* ** This file contains C code routines that are called by the parser ** to handle UPDATE statements. -** -** $Id: update.c,v 1.207 2009/08/08 18:01:08 drh Exp $ */ #ifndef SQLITE_OMIT_VIRTUALTABLE @@ -84231,10 +84587,10 @@ static void updateVirtualTable( /* ** The most recently coded instruction was an OP_Column to retrieve the -** i-th column of table pTab. This routine sets the P4 parameter of the +** i-th column of table pTab. This routine sets the P4 parameter of the ** OP_Column to the default value, if any. ** -** The default value of a column is specified by a DEFAULT clause in the +** The default value of a column is specified by a DEFAULT clause in the ** column definition. This was either supplied by the user when the table ** was created, or added later to the table definition by an ALTER TABLE ** command. If the latter, then the row-records in the table btree on disk @@ -84243,9 +84599,9 @@ static void updateVirtualTable( ** If the former, then all row-records are guaranteed to include a value ** for the column and the P4 value is not required. ** -** Column definitions created by an ALTER TABLE command may only have +** Column definitions created by an ALTER TABLE command may only have ** literal default values specified: a number, null or a string. (If a more -** complicated default expression value was provided, it is evaluated +** complicated default expression value was provided, it is evaluated ** when the ALTER TABLE is executed and one of the literal values written ** into the sqlite_master table.) ** @@ -84255,8 +84611,8 @@ static void updateVirtualTable( ** sqlite3_value objects. ** ** If parameter iReg is not negative, code an OP_RealAffinity instruction -** on register iReg. This is used when an equivalent integer value is -** stored in place of an 8-byte floating point value in order to save +** on register iReg. This is used when an equivalent integer value is +** stored in place of an 8-byte floating point value in order to save ** space. */ SQLITE_PRIVATE void sqlite3ColumnDefault(Vdbe *v, Table *pTab, int i, int iReg){ @@ -84267,7 +84623,7 @@ SQLITE_PRIVATE void sqlite3ColumnDefault(Vdbe *v, Table *pTab, int i, int iReg){ Column *pCol = &pTab->aCol[i]; VdbeComment((v, "%s.%s", pTab->zName, pCol->zName)); assert( inCol ); - sqlite3ValueFromExpr(sqlite3VdbeDb(v), pCol->pDflt, enc, + sqlite3ValueFromExpr(sqlite3VdbeDb(v), pCol->pDflt, enc, pCol->affinity, &pValue); if( pValue ){ sqlite3VdbeChangeP4(v, -1, (const char *)pValue, P4_MEM); @@ -84313,14 +84669,15 @@ SQLITE_PRIVATE void sqlite3Update( AuthContext sContext; /* The authorization context */ NameContext sNC; /* The name-context to resolve expressions in */ int iDb; /* Database containing the table being updated */ - int j1; /* Addresses of jump instructions */ int okOnePass; /* True for one-pass algorithm without the FIFO */ int hasFK; /* True if foreign key processing is required */ #ifndef SQLITE_OMIT_TRIGGER - int isView; /* Trying to update a view */ - Trigger *pTrigger; /* List of triggers on pTab, if required */ + int isView; /* True when updating a view (INSTEAD OF trigger) */ + Trigger *pTrigger; /* List of triggers on pTab, if required */ + int tmask; /* Mask of TRIGGER_BEFORE|TRIGGER_AFTER */ #endif + int newmask; /* Mask of NEW.* columns accessed by BEFORE triggers */ /* Register Allocations */ int regRowCount = 0; /* A count of rows changed */ @@ -84338,7 +84695,7 @@ SQLITE_PRIVATE void sqlite3Update( } assert( pTabList->nSrc==1 ); - /* Locate the table which we want to update. + /* Locate the table which we want to update. */ pTab = sqlite3SrcListLookup(pParse, pTabList); if( pTab==0 ) goto update_cleanup; @@ -84348,11 +84705,13 @@ SQLITE_PRIVATE void sqlite3Update( ** updated is a view. */ #ifndef SQLITE_OMIT_TRIGGER - pTrigger = sqlite3TriggersExist(pParse, pTab, TK_UPDATE, pChanges, 0); + pTrigger = sqlite3TriggersExist(pParse, pTab, TK_UPDATE, pChanges, &tmask); isView = pTab->pSelect!=0; + assert( pTrigger || tmask==0 ); #else # define pTrigger 0 # define isView 0 +# define tmask 0 #endif #ifdef SQLITE_OMIT_VIEW # undef isView @@ -84362,7 +84721,7 @@ SQLITE_PRIVATE void sqlite3Update( if( sqlite3ViewGetColumnNames(pParse, pTab) ){ goto update_cleanup; } - if( sqlite3IsReadOnly(pParse, pTab, (pTrigger?1:0)) ){ + if( sqlite3IsReadOnly(pParse, pTab, tmask) ){ goto update_cleanup; } aXRef = sqlite3DbMallocRaw(db, sizeof(int) * pTab->nCol ); @@ -84534,13 +84893,13 @@ SQLITE_PRIVATE void sqlite3Update( } if( !isView ){ - /* + /* ** Open every index that needs updating. Note that if any - ** index could potentially invoke a REPLACE conflict resolution + ** index could potentially invoke a REPLACE conflict resolution ** action, then we need to open all indices because we might need ** to be deleting some records. */ - if( !okOnePass ) sqlite3OpenTable(pParse, iCur, iDb, pTab, OP_OpenWrite); + if( !okOnePass ) sqlite3OpenTable(pParse, iCur, iDb, pTab, OP_OpenWrite); if( onError==OE_Replace ){ openAll = 1; }else{ @@ -84586,11 +84945,13 @@ SQLITE_PRIVATE void sqlite3Update( sqlite3VdbeAddOp1(v, OP_MustBeInt, regNewRowid); } - /* If there are triggers on this table, populate an array of registers + /* If there are triggers on this table, populate an array of registers ** with the required old.* column data. */ if( hasFK || pTrigger ){ u32 oldmask = (hasFK ? sqlite3FkOldmask(pParse, pTab) : 0); - oldmask |= sqlite3TriggerOldmask(pParse, pTrigger, pChanges, pTab, onError); + oldmask |= sqlite3TriggerColmask(pParse, + pTrigger, pChanges, 0, TRIGGER_BEFORE|TRIGGER_AFTER, pTab, onError + ); for(i=0; inCol; i++){ if( aXRef[i]<0 || oldmask==0xffffffff || (oldmask & (1<nCol; i++){ if( i==pTab->iPKey ){ sqlite3VdbeAddOp2(v, OP_Null, 0, regNew+i); }else{ j = aXRef[i]; - if( j<0 ){ + if( j>=0 ){ + sqlite3ExprCode(pParse, pChanges->a[j].pExpr, regNew+i); + }else if( 0==(tmask&TRIGGER_BEFORE) || i>31 || (newmask&(1<a[j].pExpr, regNew+i); } } } /* Fire any BEFORE UPDATE triggers. This happens before constraints are - ** verified. One could argue that this is wrong. */ - if( pTrigger ){ + ** verified. One could argue that this is wrong. + */ + if( tmask&TRIGGER_BEFORE ){ sqlite3VdbeAddOp2(v, OP_Affinity, regNew, pTab->nCol); sqlite3TableAffinityStr(v, pTab); - sqlite3CodeRowTrigger(pParse, pTrigger, TK_UPDATE, pChanges, + sqlite3CodeRowTrigger(pParse, pTrigger, TK_UPDATE, pChanges, TRIGGER_BEFORE, pTab, regOldRowid, onError, addr); /* The row-trigger may have deleted the row being updated. In this - ** case, jump to the next row. No updates or AFTER triggers are + ** case, jump to the next row. No updates or AFTER triggers are ** required. This behaviour - what happens when the row being updated ** is deleted or renamed by a BEFORE trigger - is left undefined in the - ** documentation. */ + ** documentation. + */ sqlite3VdbeAddOp3(v, OP_NotExists, iCur, addr, regOldRowid); + + /* If it did not delete it, the row-trigger may still have modified + ** some of the columns of the row being updated. Load the values for + ** all columns not modified by the update statement into their + ** registers in case this has happened. + */ + for(i=0; inCol; i++){ + if( aXRef[i]<0 && i!=pTab->iPKey ){ + sqlite3VdbeAddOp3(v, OP_Column, iCur, i, regNew+i); + sqlite3ColumnDefault(v, pTab, i, regNew+i); + } + } } if( !isView ){ + int j1; /* Address of jump instruction */ /* Do constraint checks. */ sqlite3GenerateConstraintChecks(pParse, pTab, iCur, regNewRowid, @@ -84652,7 +85047,7 @@ SQLITE_PRIVATE void sqlite3Update( /* Delete the index entries associated with the current record. */ j1 = sqlite3VdbeAddOp3(v, OP_NotExists, iCur, 0, regOldRowid); sqlite3GenerateRowIndexDelete(pParse, pTab, iCur, aRegIdx); - + /* If changing the record number, delete the old record. */ if( hasFK || chngRowid ){ sqlite3VdbeAddOp2(v, OP_Delete, iCur, 0); @@ -84662,25 +85057,25 @@ SQLITE_PRIVATE void sqlite3Update( if( hasFK ){ sqlite3FkCheck(pParse, pTab, 0, regNewRowid); } - + /* Insert the new index entries and the new record. */ sqlite3CompleteInsertion(pParse, pTab, iCur, regNewRowid, aRegIdx, 1, 0, 0); /* Do any ON CASCADE, SET NULL or SET DEFAULT operations required to ** handle rows (possibly in other tables) that refer via a foreign key - ** to the row just updated. */ + ** to the row just updated. */ if( hasFK ){ sqlite3FkActions(pParse, pTab, pChanges, regOldRowid); } } - /* Increment the row counter + /* Increment the row counter */ if( (db->flags & SQLITE_CountRows) && !pParse->pTriggerTab){ sqlite3VdbeAddOp2(v, OP_AddImm, regRowCount, 1); } - sqlite3CodeRowTrigger(pParse, pTrigger, TK_UPDATE, pChanges, + sqlite3CodeRowTrigger(pParse, pTrigger, TK_UPDATE, pChanges, TRIGGER_AFTER, pTab, regOldRowid, onError, addr); /* Repeat the above with the next record to be updated, until @@ -84706,7 +85101,7 @@ SQLITE_PRIVATE void sqlite3Update( } /* - ** Return the number of rows that were changed. If this routine is + ** Return the number of rows that were changed. If this routine is ** generating code because of a call to sqlite3NestedParse(), do not ** invoke the callback function. */ @@ -84777,7 +85172,7 @@ static void updateVirtualTable( SelectDest dest; /* Construct the SELECT statement that will find the new values for - ** all updated rows. + ** all updated rows. */ pEList = sqlite3ExprListAppend(pParse, 0, sqlite3Expr(db, TK_ID, "_rowid_")); if( pRowid ){ @@ -84794,7 +85189,7 @@ static void updateVirtualTable( pEList = sqlite3ExprListAppend(pParse, pEList, pExpr); } pSelect = sqlite3SelectNew(pParse, pEList, pSrc, pWhere, 0, 0, 0, 0, 0, 0); - + /* Create the ephemeral table into which the update results will ** be stored. */ @@ -84802,7 +85197,7 @@ static void updateVirtualTable( ephemTab = pParse->nTab++; sqlite3VdbeAddOp2(v, OP_OpenEphemeral, ephemTab, pTab->nCol+1+(pRowid!=0)); - /* fill the ephemeral table + /* fill the ephemeral table */ sqlite3SelectDestInit(&dest, SRT_Table, ephemTab); sqlite3Select(pParse, pSelect, &dest); @@ -84824,7 +85219,7 @@ static void updateVirtualTable( sqlite3VdbeAddOp2(v, OP_Close, ephemTab, 0); /* Cleanup */ - sqlite3SelectDelete(db, pSelect); + sqlite3SelectDelete(db, pSelect); } #endif /* SQLITE_OMIT_VIRTUALTABLE */ @@ -84845,8 +85240,6 @@ static void updateVirtualTable( ** ** Most of the code in this file may be omitted by defining the ** SQLITE_OMIT_VACUUM macro. -** -** $Id: vacuum.c,v 1.91 2009/07/02 07:47:33 danielk1977 Exp $ */ #if !defined(SQLITE_OMIT_VACUUM) && !defined(SQLITE_OMIT_ATTACH) @@ -84927,7 +85320,7 @@ SQLITE_PRIVATE int sqlite3RunVacuum(char **pzErrMsg, sqlite3 *db){ return SQLITE_ERROR; } - /* Save the current value of the database flags so that it can be + /* Save the current value of the database flags so that it can be ** restored before returning. Then set the writable-schema flag, and ** disable CHECK and foreign key constraints. */ saved_flags = db->flags; @@ -85003,17 +85396,17 @@ SQLITE_PRIVATE int sqlite3RunVacuum(char **pzErrMsg, sqlite3 *db){ /* Query the schema of the main database. Create a mirror schema ** in the temporary database. */ - rc = execExecSql(db, + rc = execExecSql(db, "SELECT 'CREATE TABLE vacuum_db.' || substr(sql,14) " " FROM sqlite_master WHERE type='table' AND name!='sqlite_sequence'" " AND rootpage>0" ); if( rc!=SQLITE_OK ) goto end_of_vacuum; - rc = execExecSql(db, + rc = execExecSql(db, "SELECT 'CREATE INDEX vacuum_db.' || substr(sql,14)" " FROM sqlite_master WHERE sql LIKE 'CREATE INDEX %' "); if( rc!=SQLITE_OK ) goto end_of_vacuum; - rc = execExecSql(db, + rc = execExecSql(db, "SELECT 'CREATE UNIQUE INDEX vacuum_db.' || substr(sql,21) " " FROM sqlite_master WHERE sql LIKE 'CREATE UNIQUE INDEX %'"); if( rc!=SQLITE_OK ) goto end_of_vacuum; @@ -85022,7 +85415,7 @@ SQLITE_PRIVATE int sqlite3RunVacuum(char **pzErrMsg, sqlite3 *db){ ** an "INSERT INTO vacuum_db.xxx SELECT * FROM main.xxx;" to copy ** the contents to the temporary database. */ - rc = execExecSql(db, + rc = execExecSql(db, "SELECT 'INSERT INTO vacuum_db.' || quote(name) " "|| ' SELECT * FROM main.' || quote(name) || ';'" "FROM main.sqlite_master " @@ -85034,12 +85427,12 @@ SQLITE_PRIVATE int sqlite3RunVacuum(char **pzErrMsg, sqlite3 *db){ /* Copy over the sequence table */ - rc = execExecSql(db, + rc = execExecSql(db, "SELECT 'DELETE FROM vacuum_db.' || quote(name) || ';' " "FROM vacuum_db.sqlite_master WHERE name='sqlite_sequence' " ); if( rc!=SQLITE_OK ) goto end_of_vacuum; - rc = execExecSql(db, + rc = execExecSql(db, "SELECT 'INSERT INTO vacuum_db.' || quote(name) " "|| ' SELECT * FROM main.' || quote(name) || ';' " "FROM vacuum_db.sqlite_master WHERE name=='sqlite_sequence';" @@ -85151,8 +85544,6 @@ end_of_vacuum: ** ************************************************************************* ** This file contains code used to help implement virtual tables. -** -** $Id: vtab.c,v 1.94 2009/08/08 18:01:08 drh Exp $ */ #ifndef SQLITE_OMIT_VIRTUALTABLE @@ -85228,7 +85619,7 @@ SQLITE_API int sqlite3_create_module_v2( /* ** Lock the virtual table so that it cannot be disconnected. ** Locks nest. Every lock should have a corresponding unlock. -** If an unlock is omitted, resources leaks will occur. +** If an unlock is omitted, resources leaks will occur. ** ** If a disconnect is attempted while a virtual table is locked, ** the disconnect is deferred until all locks have been removed. @@ -85240,7 +85631,7 @@ SQLITE_PRIVATE void sqlite3VtabLock(VTable *pVTab){ /* ** pTab is a pointer to a Table structure representing a virtual-table. -** Return a pointer to the VTable object used by connection db to access +** Return a pointer to the VTable object used by connection db to access ** this virtual-table, if one has been created, or NULL otherwise. */ SQLITE_PRIVATE VTable *sqlite3GetVTable(sqlite3 *db, Table *pTab){ @@ -85283,7 +85674,7 @@ SQLITE_PRIVATE void sqlite3VtabUnlock(VTable *pVTab){ /* ** Table p is a virtual table. This function moves all elements in the ** p->pVTable list to the sqlite3.pDisconnect lists of their associated -** database connections to be disconnected at the next opportunity. +** database connections to be disconnected at the next opportunity. ** Except, if argument db is not NULL, then the entry associated with ** connection db is left in the p->pVTable list. */ @@ -85292,13 +85683,13 @@ static VTable *vtabDisconnectAll(sqlite3 *db, Table *p){ VTable *pVTable = p->pVTable; p->pVTable = 0; - /* Assert that the mutex (if any) associated with the BtShared database - ** that contains table p is held by the caller. See header comments + /* Assert that the mutex (if any) associated with the BtShared database + ** that contains table p is held by the caller. See header comments ** above function sqlite3VtabUnlockList() for an explanation of why ** this makes it safe to access the sqlite3.pDisconnect list of any ** database connection that may have an entry in the p->pVTable list. */ assert( db==0 || - sqlite3BtreeHoldsMutex(db->aDb[sqlite3SchemaToIndex(db, p->pSchema)].pBt) + sqlite3BtreeHoldsMutex(db->aDb[sqlite3SchemaToIndex(db, p->pSchema)].pBt) ); while( pVTable ){ @@ -85325,7 +85716,7 @@ static VTable *vtabDisconnectAll(sqlite3 *db, Table *p){ ** Disconnect all the virtual table objects in the sqlite3.pDisconnect list. ** ** This function may only be called when the mutexes associated with all -** shared b-tree databases opened using connection db are held by the +** shared b-tree databases opened using connection db are held by the ** caller. This is done to protect the sqlite3.pDisconnect list. The ** sqlite3.pDisconnect list is accessed only as follows: ** @@ -85338,7 +85729,7 @@ static VTable *vtabDisconnectAll(sqlite3 *db, Table *p){ ** or, if the virtual table is stored in a non-sharable database, then ** the database handle mutex is held. ** -** As a result, a sqlite3.pDisconnect cannot be accessed simultaneously +** As a result, a sqlite3.pDisconnect cannot be accessed simultaneously ** by multiple threads. It is thread-safe. */ SQLITE_PRIVATE void sqlite3VtabUnlockList(sqlite3 *db){ @@ -85364,12 +85755,12 @@ SQLITE_PRIVATE void sqlite3VtabUnlockList(sqlite3 *db){ ** record. ** ** Since it is a virtual-table, the Table structure contains a pointer -** to the head of a linked list of VTable structures. Each VTable +** to the head of a linked list of VTable structures. Each VTable ** structure is associated with a single sqlite3* user of the schema. -** The reference count of the VTable structure associated with database -** connection db is decremented immediately (which may lead to the +** The reference count of the VTable structure associated with database +** connection db is decremented immediately (which may lead to the ** structure being xDisconnected and free). Any other VTable structures -** in the list are moved to the sqlite3.pDisconnect list of the associated +** in the list are moved to the sqlite3.pDisconnect list of the associated ** database connection. */ SQLITE_PRIVATE void sqlite3VtabClear(Table *p){ @@ -85447,7 +85838,7 @@ SQLITE_PRIVATE void sqlite3VtabBeginParse( ** The second call, to obtain permission to create the table, is made now. */ if( pTable->azModuleArg ){ - sqlite3AuthCheck(pParse, SQLITE_CREATE_VTABLE, pTable->zName, + sqlite3AuthCheck(pParse, SQLITE_CREATE_VTABLE, pTable->zName, pTable->azModuleArg[0], pParse->db->aDb[iDb].zName); } #endif @@ -85479,7 +85870,7 @@ SQLITE_PRIVATE void sqlite3VtabFinishParse(Parse *pParse, Token *pEnd){ addArgumentToVtab(pParse); pParse->sArg.z = 0; if( pTab->nModuleArg<1 ) return; - + /* If the CREATE VIRTUAL TABLE statement is being entered for the ** first time (in other words if the virtual table is actually being ** created now instead of just being read out of sqlite_master) then @@ -85498,9 +85889,9 @@ SQLITE_PRIVATE void sqlite3VtabFinishParse(Parse *pParse, Token *pEnd){ } zStmt = sqlite3MPrintf(db, "CREATE VIRTUAL TABLE %T", &pParse->sNameToken); - /* A slot for the record has already been allocated in the + /* A slot for the record has already been allocated in the ** SQLITE_MASTER table. We just need to update that slot with all - ** the information we've collected. + ** the information we've collected. ** ** The VM register number pParse->regRowid holds the rowid of an ** entry in the sqlite_master table tht was created for this vtab @@ -85524,7 +85915,7 @@ SQLITE_PRIVATE void sqlite3VtabFinishParse(Parse *pParse, Token *pEnd){ sqlite3VdbeAddOp2(v, OP_Expire, 0, 0); zWhere = sqlite3MPrintf(db, "name='%q'", pTab->zName); sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 1, 0, zWhere, P4_DYNAMIC); - sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0, + sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0, pTab->zName, sqlite3Strlen30(pTab->zName) + 1); } @@ -85580,7 +85971,7 @@ SQLITE_PRIVATE void sqlite3VtabArgExtend(Parse *pParse, Token *p){ ** to this procedure. */ static int vtabCallConstructor( - sqlite3 *db, + sqlite3 *db, Table *pTab, Module *pMod, int (*xConstruct)(sqlite3*,void*,int,const char*const*,sqlite3_vtab**,char**), @@ -85636,7 +86027,7 @@ static int vtabCallConstructor( }else{ int iCol; /* If everything went according to plan, link the new VTable structure - ** into the linked list headed by pTab->pVTable. Then loop through the + ** into the linked list headed by pTab->pVTable. Then loop through the ** columns of the table to see if any of them contain the token "hidden". ** If so, set the Column.isHidden flag and remove the token from ** the type string. */ @@ -85682,7 +86073,7 @@ static int vtabCallConstructor( /* ** This function is invoked by the parser to call the xConnect() method -** of the virtual table pTab. If an error occurs, an error code is returned +** of the virtual table pTab. If an error occurs, an error code is returned ** and an error left in pParse. ** ** This call is a no-op if table pTab is not a virtual table. @@ -85744,7 +86135,7 @@ static int addToVTrans(sqlite3 *db, VTable *pVTab){ /* ** This function is invoked by the vdbe to call the xCreate method -** of the virtual table named zTab in database iDb. +** of the virtual table named zTab in database iDb. ** ** If an error occurs, *pzErr is set to point an an English language ** description of the error and an SQLITE_XXX error code is returned. @@ -85763,8 +86154,8 @@ SQLITE_PRIVATE int sqlite3VtabCallCreate(sqlite3 *db, int iDb, const char *zTab, zMod = pTab->azModuleArg[0]; pMod = (Module*)sqlite3HashFind(&db->aModule, zMod, sqlite3Strlen30(zMod)); - /* If the module has been registered and includes a Create method, - ** invoke it now. If the module has not been registered, return an + /* If the module has been registered and includes a Create method, + ** invoke it now. If the module has not been registered, return an ** error. Otherwise, do nothing. */ if( !pMod ){ @@ -85810,11 +86201,11 @@ SQLITE_API int sqlite3_declare_vtab(sqlite3 *db, const char *zCreateTable){ }else{ pParse->declareVtab = 1; pParse->db = db; - - if( - SQLITE_OK == sqlite3RunParser(pParse, zCreateTable, &zErr) && - pParse->pNewTable && - !pParse->pNewTable->pSelect && + + if( + SQLITE_OK == sqlite3RunParser(pParse, zCreateTable, &zErr) && + pParse->pNewTable && + !pParse->pNewTable->pSelect && (pParse->pNewTable->tabFlags & TF_Virtual)==0 ){ if( !pTab->aCol ){ @@ -85830,7 +86221,7 @@ SQLITE_API int sqlite3_declare_vtab(sqlite3 *db, const char *zCreateTable){ rc = SQLITE_ERROR; } pParse->declareVtab = 0; - + if( pParse->pVdbe ){ sqlite3VdbeFinalize(pParse->pVdbe); } @@ -85882,7 +86273,7 @@ SQLITE_PRIVATE int sqlite3VtabCallDestroy(sqlite3 *db, int iDb, const char *zTab ** called is identified by the second argument, "offset", which is ** the offset of the method to call in the sqlite3_module structure. ** -** The array is cleared after invoking the callbacks. +** The array is cleared after invoking the callbacks. */ static void callFinaliser(sqlite3 *db, int offset){ int i; @@ -85908,7 +86299,7 @@ static void callFinaliser(sqlite3 *db, int offset){ ** array. Return the error code for the first error that occurs, or ** SQLITE_OK if all xSync operations are successful. ** -** Set *pzErrmsg to point to a buffer that should be released using +** Set *pzErrmsg to point to a buffer that should be released using ** sqlite3DbFree() containing an error message, if one is available. */ SQLITE_PRIVATE int sqlite3VtabSync(sqlite3 *db, char **pzErrmsg){ @@ -85939,7 +86330,7 @@ SQLITE_PRIVATE int sqlite3VtabSync(sqlite3 *db, char **pzErrmsg){ } /* -** Invoke the xRollback method of all virtual tables in the +** Invoke the xRollback method of all virtual tables in the ** sqlite3.aVTrans array. Then clear the array itself. */ SQLITE_PRIVATE int sqlite3VtabRollback(sqlite3 *db){ @@ -85948,7 +86339,7 @@ SQLITE_PRIVATE int sqlite3VtabRollback(sqlite3 *db){ } /* -** Invoke the xCommit method of all virtual tables in the +** Invoke the xCommit method of all virtual tables in the ** sqlite3.aVTrans array. Then clear the array itself. */ SQLITE_PRIVATE int sqlite3VtabCommit(sqlite3 *db){ @@ -85970,7 +86361,7 @@ SQLITE_PRIVATE int sqlite3VtabBegin(sqlite3 *db, VTable *pVTab){ /* Special case: If db->aVTrans is NULL and db->nVTrans is greater ** than zero, then this function is being called from within a - ** virtual module xSync() callback. It is illegal to write to + ** virtual module xSync() callback. It is illegal to write to ** virtual module tables in this case, so return SQLITE_LOCKED. */ if( sqlite3VtabInSync(db) ){ @@ -85978,7 +86369,7 @@ SQLITE_PRIVATE int sqlite3VtabBegin(sqlite3 *db, VTable *pVTab){ } if( !pVTab ){ return SQLITE_OK; - } + } pModule = pVTab->pVtab->pModule; if( pModule->xBegin ){ @@ -86010,7 +86401,7 @@ SQLITE_PRIVATE int sqlite3VtabBegin(sqlite3 *db, VTable *pVTab){ ** This routine is used to allow virtual table implementations to ** overload MATCH, LIKE, GLOB, and REGEXP operators. ** -** Return either the pDef argument (indicating no change) or a +** Return either the pDef argument (indicating no change) or a ** new FuncDef structure that is marked as ephemeral using the ** SQLITE_FUNC_EPHEM flag. */ @@ -86042,9 +86433,9 @@ SQLITE_PRIVATE FuncDef *sqlite3VtabOverloadFunction( assert( pVtab->pModule!=0 ); pMod = (sqlite3_module *)pVtab->pModule; if( pMod->xFindFunction==0 ) return pDef; - + /* Call the xFindFunction method on the virtual table implementation - ** to see if the implementation wants to overload this function + ** to see if the implementation wants to overload this function */ zLowerName = sqlite3DbStrDup(db, pDef->zName); if( zLowerName ){ @@ -86120,8 +86511,6 @@ SQLITE_PRIVATE void sqlite3VtabMakeWritable(Parse *pParse, Table *pTab){ ** rows. Indices are selected and used to speed the search when doing ** so is applicable. Because this module is responsible for selecting ** indices, you might also think of this module as the "query optimizer". -** -** $Id: where.c,v 1.411 2009/07/31 06:14:52 danielk1977 Exp $ */ /* @@ -86150,7 +86539,7 @@ typedef struct WhereCost WhereCost; ** clause subexpression is separated from the others by AND operators, ** usually, or sometimes subexpressions separated by OR. ** -** All WhereTerms are collected into a single WhereClause structure. +** All WhereTerms are collected into a single WhereClause structure. ** The following identity holds: ** ** WhereTerm.pWC->a[WhereTerm.idx] == WhereTerm @@ -86264,8 +86653,8 @@ struct WhereAndInfo { ** An instance of the following structure keeps track of a mapping ** between VDBE cursor numbers and bits of the bitmasks in WhereTerm. ** -** The VDBE cursor numbers are small integers contained in -** SrcList_item.iCursor and Expr.iTable fields. For any given WHERE +** The VDBE cursor numbers are small integers contained in +** SrcList_item.iCursor and Expr.iTable fields. For any given WHERE ** clause, the cursor numbers might not begin with 0 and they might ** contain gaps in the numbering sequence. But we want to make maximum ** use of the bits in our bitmasks. This structure provides a mapping @@ -86596,9 +86985,9 @@ static int allowedOp(int op){ ** ** If a collation sequence is associated with either the left or right ** side of the comparison, it remains associated with the same side after -** the commutation. So "Y collate NOCASE op X" becomes +** the commutation. So "Y collate NOCASE op X" becomes ** "X collate NOCASE op Y". This is because any collation sequence on -** the left hand side of a comparison overrides any collation sequence +** the left hand side of a comparison overrides any collation sequence ** attached to the right. For the same reason the EP_ExpCollate flag ** is not commuted. */ @@ -86703,7 +87092,7 @@ static WhereTerm *findTerm( static void exprAnalyze(SrcList*, WhereClause*, int); /* -** Call exprAnalyze on all terms in a WHERE clause. +** Call exprAnalyze on all terms in a WHERE clause. ** ** */ @@ -86724,7 +87113,7 @@ static void exprAnalyzeAll( ** so and false if not. ** ** In order for the operator to be optimizible, the RHS must be a string -** literal that does not begin with a wildcard. +** literal that does not begin with a wildcard. */ static int isLikeOrGlob( Parse *pParse, /* Parsing and code generating context */ @@ -86808,7 +87197,7 @@ static int isLikeOrGlob( ** function, then no OP_Variable will be added to the program. ** This causes problems for the sqlite3_bind_parameter_name() ** API. To workaround them, add a dummy OP_Variable here. - */ + */ int r1 = sqlite3GetTempReg(pParse); sqlite3ExprCodeTarget(pParse, pRight, r1); sqlite3VdbeChangeP3(v, sqlite3VdbeCurrentAddr(v)-1, 0); @@ -86911,10 +87300,10 @@ static void transferJoinMarkings(Expr *pDerived, Expr *pBase){ ** WhereTerm.u.pOrInfo->indexable |= the cursor number for table T ** ** A subterm is "indexable" if it is of the form -** "T.C " where C is any column of table T and +** "T.C " where C is any column of table T and ** is one of "=", "<", "<=", ">", ">=", "IS NULL", or "IN". ** A subterm is also indexable if it is an AND of two or more -** subsubterms at least one of which is indexable. Indexable AND +** subsubterms at least one of which is indexable. Indexable AND ** subterms have their eOperator set to WO_AND and they have ** u.pAndInfo set to a dynamically allocated WhereAndTerm object. ** @@ -87084,7 +87473,7 @@ static void exprAnalyzeOrTerm( if( (chngToIN & getMask(pMaskSet, pOrTerm->leftCursor))==0 ){ /* This term must be of the form t1.a==t2.b where t2 is in the ** chngToIN set but t1 is not. This term will be either preceeded - ** or follwed by an inverted copy (t2.b==t1.a). Skip this term + ** or follwed by an inverted copy (t2.b==t1.a). Skip this term ** and use its inversion. */ testcase( pOrTerm->wtFlags & TERM_COPIED ); testcase( pOrTerm->wtFlags & TERM_VIRTUAL ); @@ -87132,7 +87521,7 @@ static void exprAnalyzeOrTerm( } /* At this point, okToChngToIN is true if original pTerm satisfies - ** case 1. In that case, construct a new virtual term that is + ** case 1. In that case, construct a new virtual term that is ** pTerm converted into an IN operator. */ if( okToChngToIN ){ @@ -87305,7 +87694,7 @@ static void exprAnalyze( for(i=0; i<2; i++){ Expr *pNewExpr; int idxNew; - pNewExpr = sqlite3PExpr(pParse, ops[i], + pNewExpr = sqlite3PExpr(pParse, ops[i], sqlite3ExprDup(db, pExpr->pLeft, 0), sqlite3ExprDup(db, pList->a[i].pExpr, 0), 0); idxNew = whereClauseInsert(pWC, pNewExpr, TERM_VIRTUAL|TERM_DYNAMIC); @@ -87340,7 +87729,7 @@ static void exprAnalyze( ** The last character of the prefix "abc" is incremented to form the ** termination condition "abd". */ - if( pWC->op==TK_AND + if( pWC->op==TK_AND && isLikeOrGlob(pParse, pExpr, &pStr1, &isComplete, &noCase) ){ Expr *pLeft; /* LHS of LIKE/GLOB operator */ @@ -87359,7 +87748,7 @@ static void exprAnalyze( if( noCase ){ /* The point is to increment the last character before the first ** wildcard. But if we increment '@', that will push it into the - ** alphabetic range where case conversions will mess up the + ** alphabetic range where case conversions will mess up the ** inequality. To avoid this, make sure to also run the full ** LIKE on all candidate expressions by clearing the isComplete flag */ @@ -87405,7 +87794,7 @@ static void exprAnalyze( prereqColumn = exprTableUsage(pMaskSet, pLeft); if( (prereqExpr & prereqColumn)==0 ){ Expr *pNewExpr; - pNewExpr = sqlite3PExpr(pParse, TK_MATCH, + pNewExpr = sqlite3PExpr(pParse, TK_MATCH, 0, sqlite3ExprDup(db, pRight, 0), 0); idxNew = whereClauseInsert(pWC, pNewExpr, TERM_VIRTUAL|TERM_DYNAMIC); testcase( idxNew==0 ); @@ -87487,7 +87876,7 @@ static int isSortingIndex( nTerm = pOrderBy->nExpr; assert( nTerm>0 ); - /* Argument pIdx must either point to a 'real' named index structure, + /* Argument pIdx must either point to a 'real' named index structure, ** or an index structure allocated on the stack by bestBtreeIndex() to ** represent the rowid index that is part of every table. */ assert( pIdx->zName || (pIdx->nColumn==1 && pIdx->aiColumn[0]==-1) ); @@ -87565,7 +87954,7 @@ static int isSortingIndex( if( iColumn<0 && !referencesOtherTables(pOrderBy, pMaskSet, j, base) ){ /* If the indexed column is the primary key and everything matches ** so far and none of the ORDER BY terms to the right reference other - ** tables in the join, then we are assured that the index can be used + ** tables in the join, then we are assured that the index can be used ** to sort because the primary key is unique and so none of the other ** columns will make any difference */ @@ -87651,15 +88040,15 @@ static void TRACE_IDX_OUTPUTS(sqlite3_index_info *p){ #define TRACE_IDX_OUTPUTS(A) #endif -/* -** Required because bestIndex() is called by bestOrClauseIndex() +/* +** Required because bestIndex() is called by bestOrClauseIndex() */ static void bestIndex( Parse*, WhereClause*, struct SrcList_item*, Bitmask, ExprList*, WhereCost*); /* -** This routine attempts to find an scanning strategy that can be used -** to optimize an 'OR' expression that is part of a WHERE clause. +** This routine attempts to find an scanning strategy that can be used +** to optimize an 'OR' expression that is part of a WHERE clause. ** ** The table associated with FROM clause term pSrc may be either a ** regular B-Tree table or a virtual table. @@ -87680,9 +88069,9 @@ static void bestOrClauseIndex( /* Search the WHERE clause terms for a usable WO_OR term. */ for(pTerm=pWC->a; pTermeOperator==WO_OR + if( pTerm->eOperator==WO_OR && ((pTerm->prereqAll & ~maskSrc) & notReady)==0 - && (pTerm->u.pOrInfo->indexable & maskSrc)!=0 + && (pTerm->u.pOrInfo->indexable & maskSrc)!=0 ){ WhereClause * const pOrWC = &pTerm->u.pOrInfo->wc; WhereTerm * const pOrWCEnd = &pOrWC->a[pOrWC->nTerm]; @@ -87694,7 +88083,7 @@ static void bestOrClauseIndex( for(pOrTerm=pOrWC->a; pOrTerma), (pTerm - pWC->a) )); if( pOrTerm->eOperator==WO_AND ){ @@ -87717,7 +88106,7 @@ static void bestOrClauseIndex( if( rTotal>=pCost->rCost ) break; } - /* If there is an ORDER BY clause, increase the scan cost to account + /* If there is an ORDER BY clause, increase the scan cost to account ** for the cost of the sort. */ if( pOrderBy!=0 ){ rTotal += nRow*estLog(nRow); @@ -87742,12 +88131,12 @@ static void bestOrClauseIndex( #ifndef SQLITE_OMIT_VIRTUALTABLE /* -** Allocate and populate an sqlite3_index_info structure. It is the +** Allocate and populate an sqlite3_index_info structure. It is the ** responsibility of the caller to eventually release the structure ** by passing the pointer returned by this function to sqlite3_free(). */ static sqlite3_index_info *allocateIndexInfo( - Parse *pParse, + Parse *pParse, WhereClause *pWC, struct SrcList_item *pSrc, ExprList *pOrderBy @@ -87774,7 +88163,7 @@ static sqlite3_index_info *allocateIndexInfo( nTerm++; } - /* If the ORDER BY clause contains only columns in the current + /* If the ORDER BY clause contains only columns in the current ** virtual table then allocate space for the aOrderBy part of ** the sqlite3_index_info structure. */ @@ -87885,7 +88274,7 @@ static int vtabBestIndex(Parse *pParse, Table *pTab, sqlite3_index_info *p){ for(i=0; inConstraint; i++){ if( !p->aConstraint[i].usable && p->aConstraintUsage[i].argvIndex>0 ){ - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "table %s: xBestIndex returned an invalid plan", pTab->zName); } } @@ -87906,7 +88295,7 @@ static int vtabBestIndex(Parse *pParse, Table *pTab, sqlite3_index_info *p){ ** same virtual table. The sqlite3_index_info structure is created ** and initialized on the first invocation and reused on all subsequent ** invocations. The sqlite3_index_info structure is also used when -** code is generated to access the virtual table. The whereInfoDelete() +** code is generated to access the virtual table. The whereInfoDelete() ** routine takes care of freeing the sqlite3_index_info structure after ** everybody has finished with it. */ @@ -87927,7 +88316,7 @@ static void bestVirtualIndex( int i, j; int nOrderBy; - /* Make sure wsFlags is initialized to some sane value. Otherwise, if the + /* Make sure wsFlags is initialized to some sane value. Otherwise, if the ** malloc in allocateIndexInfo() fails and this function returns leaving ** wsFlags in an uninitialized state, the caller may behave unpredictably. */ @@ -87954,12 +88343,12 @@ static void bestVirtualIndex( /* The module name must be defined. Also, by this point there must ** be a pointer to an sqlite3_vtab structure. Otherwise - ** sqlite3ViewGetColumnNames() would have picked up the error. + ** sqlite3ViewGetColumnNames() would have picked up the error. */ assert( pTab->azModuleArg && pTab->azModuleArg[0] ); assert( sqlite3GetVTable(pParse->db, pTab) ); - /* Set the aConstraint[].usable fields and initialize all + /* Set the aConstraint[].usable fields and initialize all ** output variables to zero. ** ** aConstraint[].usable is true for constraints where the right-hand @@ -87968,7 +88357,7 @@ static void bestVirtualIndex( ** ** column = expr ** - ** and we are evaluating a join, then the constraint on column is + ** and we are evaluating a join, then the constraint on column is ** only valid if all tables referenced in expr occur to the left ** of the table containing column. ** @@ -88015,8 +88404,8 @@ static void bestVirtualIndex( /* The cost is not allowed to be larger than SQLITE_BIG_DBL (the ** inital value of lowestCost in this loop. If it is, then the ** (costestimatedCost ){ @@ -88032,7 +88421,7 @@ static void bestVirtualIndex( pIdxInfo->nOrderBy = nOrderBy; /* Try to find a more efficient access pattern by using multiple indexes - ** to optimize an OR expression within the WHERE clause. + ** to optimize an OR expression within the WHERE clause. */ bestOrClauseIndex(pParse, pWC, pSrc, notReady, pOrderBy, pCost); } @@ -88047,7 +88436,7 @@ static void bestVirtualIndex( ** 1 contains values larger than or equal to the value of the first sample, ** but smaller than the value of the second. And so on. ** -** If successful, this function determines which of the regions value +** If successful, this function determines which of the regions value ** pVal lies in, sets *piRegion to the region index (a value between 0 ** and SQLITE_INDEX_SAMPLES+1, inclusive) and returns SQLITE_OK. ** Or, if an OOM occurs while converting text values between encodings, @@ -88071,7 +88460,7 @@ static int whereRangeRegion( if( aSample[i].eType==SQLITE_NULL ) continue; if( aSample[i].eType>=SQLITE_TEXT || aSample[i].u.r>r ) break; } - }else{ + }else{ sqlite3 *db = pParse->db; CollSeq *pColl; const u8 *z; @@ -88135,8 +88524,8 @@ static int whereRangeRegion( /* ** If expression pExpr represents a literal value, set *pp to point to ** an sqlite3_value structure containing the same value, with affinity -** aff applied to it, before returning. It is the responsibility of the -** caller to eventually release this structure by passing it to +** aff applied to it, before returning. It is the responsibility of the +** caller to eventually release this structure by passing it to ** sqlite3ValueFree(). ** ** If the current parse is a recompile (sqlite3Reprepare()) and pExpr @@ -88150,9 +88539,9 @@ static int whereRangeRegion( */ #ifdef SQLITE_ENABLE_STAT2 static int valueFromExpr( - Parse *pParse, - Expr *pExpr, - u8 aff, + Parse *pParse, + Expr *pExpr, + u8 aff, sqlite3_value **pp ){ /* The evalConstExpr() function will have already converted any TK_VARIABLE @@ -88301,13 +88690,13 @@ range_est_fallback: ** index and in the main table. ** ** If there was an INDEXED BY clause (pSrc->pIndex) attached to the table in -** the SQL statement, then this function only considers plans using the +** the SQL statement, then this function only considers plans using the ** named index. If no such plan is found, then the returned cost is -** SQLITE_BIG_DBL. If a plan is found that uses the named index, +** SQLITE_BIG_DBL. If a plan is found that uses the named index, ** then the cost is calculated in the usual way. ** -** If a NOT INDEXED clause (pSrc->notIndexed!=0) was attached to the table -** in the SELECT statement, then no indexes are considered. However, the +** If a NOT INDEXED clause (pSrc->notIndexed!=0) was attached to the table +** in the SELECT statement, then no indexes are considered. However, the ** selected plan may still take advantage of the tables built-in rowid ** index. */ @@ -88396,18 +88785,18 @@ static void bestBtreeIndex( ** scan being evaluated. They are then used to determine the expected ** cost and number of rows returned. ** - ** nEq: + ** nEq: ** Number of equality terms that can be implemented using the index. ** - ** nInMul: - ** The "in-multiplier". This is an estimate of how many seek operations - ** SQLite must perform on the index in question. For example, if the + ** nInMul: + ** The "in-multiplier". This is an estimate of how many seek operations + ** SQLite must perform on the index in question. For example, if the ** WHERE clause is: ** ** WHERE a IN (1, 2, 3) AND b IN (4, 5, 6) ** - ** SQLite must perform 9 lookups on an index on (a, b), so nInMul is - ** set to 9. Given the same schema and either of the following WHERE + ** SQLite must perform 9 lookups on an index on (a, b), so nInMul is + ** set to 9. Given the same schema and either of the following WHERE ** clauses: ** ** WHERE a = 1 @@ -88415,12 +88804,12 @@ static void bestBtreeIndex( ** ** nInMul is set to 1. ** - ** If there exists a WHERE term of the form "x IN (SELECT ...)", then - ** the sub-select is assumed to return 25 rows for the purposes of + ** If there exists a WHERE term of the form "x IN (SELECT ...)", then + ** the sub-select is assumed to return 25 rows for the purposes of ** determining nInMul. ** - ** bInEst: - ** Set to true if there was at least one "x IN (SELECT ...)" term used + ** bInEst: + ** Set to true if there was at least one "x IN (SELECT ...)" term used ** in determining the value of nInMul. ** ** nBound: @@ -88432,18 +88821,18 @@ static void bestBtreeIndex( ** space to 1/3rd its original size. So an x>? constraint reduces ** nBound to 33. Two constraints (x>? AND xpTab->zName, (pIdx ? pIdx->zName : "ipk"), + pSrc->pTab->zName, (pIdx ? pIdx->zName : "ipk"), nEq, nInMul, nBound, bSort, bLookup, wsFlags, nRow, cost )); @@ -88616,22 +89005,22 @@ static void bestBtreeIndex( assert( pOrderBy || (pCost->plan.wsFlags&WHERE_ORDERBY)==0 ); assert( pCost->plan.u.pIdx==0 || (pCost->plan.wsFlags&WHERE_ROWID_EQ)==0 ); - assert( pSrc->pIndex==0 - || pCost->plan.u.pIdx==0 - || pCost->plan.u.pIdx==pSrc->pIndex + assert( pSrc->pIndex==0 + || pCost->plan.u.pIdx==0 + || pCost->plan.u.pIdx==pSrc->pIndex ); - WHERETRACE(("best index is: %s\n", + WHERETRACE(("best index is: %s\n", (pCost->plan.u.pIdx ? pCost->plan.u.pIdx->zName : "ipk") )); - + bestOrClauseIndex(pParse, pWC, pSrc, notReady, pOrderBy, pCost); pCost->plan.wsFlags |= eqTermMask; } /* ** Find the query plan for accessing table pSrc->pTab. Write the -** best query plan and its cost into the WhereCost object supplied +** best query plan and its cost into the WhereCost object supplied ** as the last parameter. This function may calculate the cost of ** both real and virtual table scans. */ @@ -88698,24 +89087,47 @@ static void disableTerm(WhereLevel *pLevel, WhereTerm *pTerm){ /* ** Code an OP_Affinity opcode to apply the column affinity string zAff -** to the n registers starting at base. +** to the n registers starting at base. ** -** Buffer zAff was allocated using sqlite3DbMalloc(). It is the -** responsibility of this function to arrange for it to be eventually -** freed using sqlite3DbFree(). +** As an optimization, SQLITE_AFF_NONE entries (which are no-ops) at the +** beginning and end of zAff are ignored. If all entries in zAff are +** SQLITE_AFF_NONE, then no code gets generated. +** +** This routine makes its own copy of zAff so that the caller is free +** to modify zAff after this routine returns. */ static void codeApplyAffinity(Parse *pParse, int base, int n, char *zAff){ Vdbe *v = pParse->pVdbe; + if( zAff==0 ){ + assert( pParse->db->mallocFailed ); + return; + } assert( v!=0 ); - sqlite3VdbeAddOp2(v, OP_Affinity, base, n); - sqlite3VdbeChangeP4(v, -1, zAff, P4_DYNAMIC); - sqlite3ExprCacheAffinityChange(pParse, base, n); + + /* Adjust base and n to skip over SQLITE_AFF_NONE entries at the beginning + ** and end of the affinity string. + */ + while( n>0 && zAff[0]==SQLITE_AFF_NONE ){ + n--; + base++; + zAff++; + } + while( n>1 && zAff[n-1]==SQLITE_AFF_NONE ){ + n--; + } + + /* Code the OP_Affinity opcode if there is anything left to do. */ + if( n>0 ){ + sqlite3VdbeAddOp2(v, OP_Affinity, base, n); + sqlite3VdbeChangeP4(v, -1, zAff, n); + sqlite3ExprCacheAffinityChange(pParse, base, n); + } } /* ** Generate code for a single equality term of the WHERE clause. An equality -** term can be either X=expr or X IN (...). pTerm is the term to be +** term can be either X=expr or X IN (...). pTerm is the term to be ** coded. ** ** The current value for the constraint is left in register iReg. @@ -88780,19 +89192,20 @@ static int codeEqualityTerm( /* ** Generate code that will evaluate all == and IN constraints for an -** index. The values for all constraints are left on the stack. +** index. ** ** For example, consider table t1(a,b,c,d,e,f) with index i1(a,b,c). ** Suppose the WHERE clause is this: a==5 AND b IN (1,2,3) AND c>5 AND c<10 ** The index has as many as three equality constraints, but in this -** example, the third "c" value is an inequality. So only two +** example, the third "c" value is an inequality. So only two ** constraints are coded. This routine will generate code to evaluate ** a==5 and b IN (1,2,3). The current values for a and b will be stored ** in consecutive registers and the index of the first register is returned. ** ** In the example above nEq==2. But this subroutine works for any value ** of nEq including 0. If nEq==0, this routine is nearly a no-op. -** The only thing it does is allocate the pLevel->iMem memory cell. +** The only thing it does is allocate the pLevel->iMem memory cell and +** compute the affinity string. ** ** This routine always allocates at least one memory cell and returns ** the index of that memory cell. The code that @@ -88870,11 +89283,15 @@ static int codeAllEqualityTerms( testcase( pTerm->eOperator & WO_ISNULL ); testcase( pTerm->eOperator & WO_IN ); if( (pTerm->eOperator & (WO_ISNULL|WO_IN))==0 ){ - sqlite3VdbeAddOp2(v, OP_IsNull, regBase+j, pLevel->addrBrk); - if( zAff - && sqlite3CompareAffinity(pTerm->pExpr->pRight, zAff[j])==SQLITE_AFF_NONE - ){ - zAff[j] = SQLITE_AFF_NONE; + Expr *pRight = pTerm->pExpr->pRight; + sqlite3ExprCodeIsNullJump(v, pRight, regBase+j, pLevel->addrBrk); + if( zAff ){ + if( sqlite3CompareAffinity(pRight, zAff[j])==SQLITE_AFF_NONE ){ + zAff[j] = SQLITE_AFF_NONE; + } + if( sqlite3ExprNeedsNoAffinityChange(pRight, zAff[j]) ){ + zAff[j] = SQLITE_AFF_NONE; + } } } } @@ -88915,7 +89332,7 @@ static Bitmask codeOneLoopStart( pTabItem = &pWInfo->pTabList->a[pLevel->iFrom]; iCur = pTabItem->iCursor; bRev = (pLevel->plan.wsFlags & WHERE_REVERSE)!=0; - omitTable = (pLevel->plan.wsFlags & WHERE_IDX_ONLY)!=0 + omitTable = (pLevel->plan.wsFlags & WHERE_IDX_ONLY)!=0 && (wctrlFlags & WHERE_FORCE_TABLE)==0; /* Create labels for the "break" and "continue" instructions @@ -88954,6 +89371,7 @@ static Bitmask codeOneLoopStart( const struct sqlite3_index_constraint *aConstraint = pVtabIdx->aConstraint; + sqlite3ExprCachePush(pParse); iReg = sqlite3GetTempRange(pParse, nConstraint+2); for(j=1; j<=nConstraint; j++){ for(k=0; kp1 = iCur; pLevel->p2 = sqlite3VdbeCurrentAddr(v); sqlite3ReleaseTempRange(pParse, iReg, nConstraint+2); + sqlite3ExprCachePop(pParse, 1); }else #endif /* SQLITE_OMIT_VIRTUALTABLE */ @@ -89022,7 +89441,7 @@ static Bitmask codeOneLoopStart( Expr *pX; /* The expression that defines the start bound */ int r1, rTemp; /* Registers for holding the start boundary */ - /* The following constant maps TK_xx codes into corresponding + /* The following constant maps TK_xx codes into corresponding ** seek opcodes. It depends on a particular ordering of TK_xx */ const u8 aMoveOp[] = { @@ -89076,14 +89495,14 @@ static Bitmask codeOneLoopStart( }else if( pLevel->plan.wsFlags & (WHERE_COLUMN_RANGE|WHERE_COLUMN_EQ) ){ /* Case 3: A scan using an index. ** - ** The WHERE clause may contain zero or more equality + ** The WHERE clause may contain zero or more equality ** terms ("==" or "IN" operators) that refer to the N ** left-most columns of the index. It may also contain ** inequality constraints (>, <, >= or <=) on the indexed - ** column that immediately follows the N equalities. Only + ** column that immediately follows the N equalities. Only ** the right-most column can be an inequality - the rest must - ** use the "==" and "IN" operators. For example, if the - ** index is on (x,y,z), then the following clauses are all + ** use the "==" and "IN" operators. For example, if the + ** index is on (x,y,z), then the following clauses are all ** optimized: ** ** x=5 @@ -89104,7 +89523,7 @@ static Bitmask codeOneLoopStart( ** This case is also used when there are no WHERE clause ** constraints but an index is selected anyway, in order ** to force the output order to conform to an ORDER BY. - */ + */ int aStartOp[] = { 0, 0, @@ -89140,8 +89559,8 @@ static Bitmask codeOneLoopStart( iIdxCur = pLevel->iIdxCur; k = pIdx->aiColumn[nEq]; /* Column for inequality constraints */ - /* If this loop satisfies a sort order (pOrderBy) request that - ** was passed to this function to implement a "SELECT min(x) ..." + /* If this loop satisfies a sort order (pOrderBy) request that + ** was passed to this function to implement a "SELECT min(x) ..." ** query, then the caller will only allow the loop to run for ** a single iteration. This means that the first row returned ** should not have a NULL value stored in 'x'. If column 'x' is @@ -89158,8 +89577,8 @@ static Bitmask codeOneLoopStart( nExtraReg = 1; } - /* Find any inequality constraint terms for the start and end - ** of the range. + /* Find any inequality constraint terms for the start and end + ** of the range. */ if( pLevel->plan.wsFlags & WHERE_TOP_LIMIT ){ pRangeEnd = findTerm(pWC, iCur, k, notReady, (WO_LT|WO_LE), pIdx); @@ -89180,7 +89599,7 @@ static Bitmask codeOneLoopStart( addrNxt = pLevel->addrNxt; /* If we are doing a reverse order scan on an ascending index, or - ** a forward order scan on a descending index, interchange the + ** a forward order scan on a descending index, interchange the ** start and end terms (pRangeStart and pRangeEnd). */ if( bRev==(pIdx->aSortOrder[nEq]==SQLITE_SO_ASC) ){ @@ -89200,14 +89619,17 @@ static Bitmask codeOneLoopStart( if( pRangeStart ){ Expr *pRight = pRangeStart->pExpr->pRight; sqlite3ExprCode(pParse, pRight, regBase+nEq); - sqlite3VdbeAddOp2(v, OP_IsNull, regBase+nEq, addrNxt); - if( zAff - && sqlite3CompareAffinity(pRight, zAff[nConstraint])==SQLITE_AFF_NONE - ){ - /* Since the comparison is to be performed with no conversions applied - ** to the operands, set the affinity to apply to pRight to - ** SQLITE_AFF_NONE. */ - zAff[nConstraint] = SQLITE_AFF_NONE; + sqlite3ExprCodeIsNullJump(v, pRight, regBase+nEq, addrNxt); + if( zAff ){ + if( sqlite3CompareAffinity(pRight, zAff[nConstraint])==SQLITE_AFF_NONE){ + /* Since the comparison is to be performed with no conversions + ** applied to the operands, set the affinity to apply to pRight to + ** SQLITE_AFF_NONE. */ + zAff[nConstraint] = SQLITE_AFF_NONE; + } + if( sqlite3ExprNeedsNoAffinityChange(pRight, zAff[nConstraint]) ){ + zAff[nConstraint] = SQLITE_AFF_NONE; + } } nConstraint++; }else if( isMinQuery ){ @@ -89225,8 +89647,7 @@ static Bitmask codeOneLoopStart( testcase( op==OP_SeekGe ); testcase( op==OP_SeekLe ); testcase( op==OP_SeekLt ); - sqlite3VdbeAddOp4(v, op, iIdxCur, addrNxt, regBase, - SQLITE_INT_TO_PTR(nConstraint), P4_INT32); + sqlite3VdbeAddOp4Int(v, op, iIdxCur, addrNxt, regBase, nConstraint); /* Load the value for the inequality constraint at the end of the ** range (if any). @@ -89236,19 +89657,22 @@ static Bitmask codeOneLoopStart( Expr *pRight = pRangeEnd->pExpr->pRight; sqlite3ExprCacheRemove(pParse, regBase+nEq); sqlite3ExprCode(pParse, pRight, regBase+nEq); - sqlite3VdbeAddOp2(v, OP_IsNull, regBase+nEq, addrNxt); - zAff = sqlite3DbStrDup(pParse->db, zAff); - if( zAff - && sqlite3CompareAffinity(pRight, zAff[nConstraint])==SQLITE_AFF_NONE - ){ - /* Since the comparison is to be performed with no conversions applied - ** to the operands, set the affinity to apply to pRight to - ** SQLITE_AFF_NONE. */ - zAff[nConstraint] = SQLITE_AFF_NONE; + sqlite3ExprCodeIsNullJump(v, pRight, regBase+nEq, addrNxt); + if( zAff ){ + if( sqlite3CompareAffinity(pRight, zAff[nConstraint])==SQLITE_AFF_NONE){ + /* Since the comparison is to be performed with no conversions + ** applied to the operands, set the affinity to apply to pRight to + ** SQLITE_AFF_NONE. */ + zAff[nConstraint] = SQLITE_AFF_NONE; + } + if( sqlite3ExprNeedsNoAffinityChange(pRight, zAff[nConstraint]) ){ + zAff[nConstraint] = SQLITE_AFF_NONE; + } } codeApplyAffinity(pParse, regBase, nEq+1, zAff); nConstraint++; } + sqlite3DbFree(pParse->db, zAff); /* Top of the loop body */ pLevel->p2 = sqlite3VdbeCurrentAddr(v); @@ -89259,8 +89683,7 @@ static Bitmask codeOneLoopStart( testcase( op==OP_IdxGE ); testcase( op==OP_IdxLT ); if( op!=OP_Noop ){ - sqlite3VdbeAddOp4(v, op, iIdxCur, addrNxt, regBase, - SQLITE_INT_TO_PTR(nConstraint), P4_INT32); + sqlite3VdbeAddOp4Int(v, op, iIdxCur, addrNxt, regBase, nConstraint); sqlite3VdbeChangeP5(v, endEq!=bRev ?1:0); } @@ -89287,7 +89710,7 @@ static Bitmask codeOneLoopStart( sqlite3VdbeAddOp2(v, OP_Seek, iCur, iRowidReg); /* Deferred seek */ } - /* Record the instruction used to terminate the loop. Disable + /* Record the instruction used to terminate the loop. Disable ** WHERE clause terms made redundant by the index range scan. */ pLevel->op = bRev ? OP_Prev : OP_Next; @@ -89345,7 +89768,7 @@ static Bitmask codeOneLoopStart( int iLoopBody = sqlite3VdbeMakeLabel(v); /* Start of loop body */ int iRetInit; /* Address of regReturn init */ int ii; - + pTerm = pLevel->plan.u.pTerm; assert( pTerm!=0 ); assert( pTerm->eOperator==WO_OR ); @@ -89358,14 +89781,14 @@ static Bitmask codeOneLoopStart( oneTab.nAlloc = 1; oneTab.a[0] = *pTabItem; - /* Initialize the rowset register to contain NULL. An SQL NULL is + /* Initialize the rowset register to contain NULL. An SQL NULL is ** equivalent to an empty rowset. ** - ** Also initialize regReturn to contain the address of the instruction + ** Also initialize regReturn to contain the address of the instruction ** immediately following the OP_Return at the bottom of the loop. This ** is required in a few obscure LEFT JOIN cases where control jumps - ** over the top of the loop into the body of it. In this case the - ** correct response for the end-of-loop code (the OP_Return) is to + ** over the top of the loop into the body of it. In this case the + ** correct response for the end-of-loop code (the OP_Return) is to ** fall through to the next instruction, just as an OP_Next does if ** called on an uninitialized cursor. */ @@ -89387,11 +89810,10 @@ static Bitmask codeOneLoopStart( if( (wctrlFlags & WHERE_DUPLICATES_OK)==0 ){ int iSet = ((ii==pOrWc->nTerm-1)?-1:ii); int r; - r = sqlite3ExprCodeGetColumn(pParse, pTabItem->pTab, -1, iCur, + r = sqlite3ExprCodeGetColumn(pParse, pTabItem->pTab, -1, iCur, regRowid, 0); - sqlite3VdbeAddOp4(v, OP_RowSetTest, regRowset, - sqlite3VdbeCurrentAddr(v)+2, - r, SQLITE_INT_TO_PTR(iSet), P4_INT32); + sqlite3VdbeAddOp4Int(v, OP_RowSetTest, regRowset, + sqlite3VdbeCurrentAddr(v)+2, r, iSet); } sqlite3VdbeAddOp2(v, OP_Gosub, regReturn, iLoopBody); @@ -89447,7 +89869,7 @@ static Bitmask codeOneLoopStart( } /* For a LEFT OUTER JOIN, generate code that will record the fact that - ** at least one row of the right table has matched the left table. + ** at least one row of the right table has matched the left table. */ if( pLevel->iLeftJoin ){ pLevel->addrFirst = sqlite3VdbeCurrentAddr(v); @@ -89613,7 +90035,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( sqlite3 *db; /* Database connection */ /* The number of tables in the FROM clause is limited by the number of - ** bits in a Bitmask + ** bits in a Bitmask */ if( pTabList->nSrc>BMS ){ sqlite3ErrorMsg(pParse, "at most %d tables in a join", BMS); @@ -89629,8 +90051,8 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( */ db = pParse->db; nByteWInfo = ROUND8(sizeof(WhereInfo)+(pTabList->nSrc-1)*sizeof(WhereLevel)); - pWInfo = sqlite3DbMallocZero(db, - nByteWInfo + + pWInfo = sqlite3DbMallocZero(db, + nByteWInfo + sizeof(WhereClause) + sizeof(WhereMaskSet) ); @@ -89652,7 +90074,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( whereClauseInit(pWC, pParse, pMaskSet); sqlite3ExprCodeConstants(pParse, pWhere); whereSplit(pWC, pWhere, TK_AND); - + /* Special case: a WHERE clause that is constant. Evaluate the ** expression and either jump over all of the code or fall thru. */ @@ -89673,8 +90095,8 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( ** for all tables to the left of a left join is important. Ticket #3015. ** ** Configure the WhereClause.vmask variable so that bits that correspond - ** to virtual table cursors are set. This is used to selectively disable - ** the OR-to-IN transformation in exprAnalyzeOrTerm(). It is not helpful + ** to virtual table cursors are set. This is used to selectively disable + ** the OR-to-IN transformation in exprAnalyzeOrTerm(). It is not helpful ** with virtual tables. */ assert( pWC->vmask==0 && pMaskSet->n==0 ); @@ -89740,7 +90162,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( /* Loop through the remaining entries in the FROM clause to find the ** next nested loop. The FROM clause entries may be iterated through - ** either once or twice. + ** either once or twice. ** ** The first iteration, which is always performed, searches for the ** FROM clause entry that permits the lowest-cost, "optimal" scan. In @@ -89758,15 +90180,15 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( ** the next outermost loop was always that with the lowest overall ** cost. However, this meant that SQLite could select the wrong plan ** for scripts such as the following: - ** - ** CREATE TABLE t1(a, b); + ** + ** CREATE TABLE t1(a, b); ** CREATE TABLE t2(c, d); ** SELECT * FROM t2, t1 WHERE t2.rowid = t1.a; ** ** The best strategy is to iterate through table t1 first. However it ** is not possible to determine this with a simple greedy algorithm. - ** However, since the cost of a linear scan through table t2 is the same - ** as the cost of a linear scan through table t1, a simple greedy + ** However, since the cost of a linear scan through table t2 is the same + ** as the cost of a linear scan through table t1, a simple greedy ** algorithm may choose to use t2 for the outer loop, which is a much ** costlier approach. */ @@ -89777,7 +90199,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( int doNotReorder; /* True if this table should not be reordered */ WhereCost sCost; /* Cost information from best[Virtual]Index() */ ExprList *pOrderBy; /* ORDER BY clause for index to optimize */ - + doNotReorder = (pTabItem->jointype & (JT_LEFT|JT_CROSS))!=0; if( j!=iFrom && doNotReorder ) break; m = getMask(pMaskSet, pTabItem->iCursor); @@ -89786,13 +90208,13 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( continue; } pOrderBy = ((i==0 && ppOrderBy )?*ppOrderBy:0); - + assert( pTabItem->pTab ); #ifndef SQLITE_OMIT_VIRTUALTABLE if( IsVirtual(pTabItem->pTab) ){ sqlite3_index_info **pp = &pWInfo->a[j].pIdxInfo; bestVirtualIndex(pParse, pWC, pTabItem, mask, pOrderBy, &sCost, pp); - }else + }else #endif { bestBtreeIndex(pParse, pWC, pTabItem, mask, pOrderBy, &sCost); @@ -89800,7 +90222,7 @@ SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin( assert( isOptimal || (sCost.used¬Ready)==0 ); if( (sCost.used¬Ready)==0 - && (j==iFrom || sCost.rCostcolUsed; int n = 0; for(; b; b=b>>1, n++){} - sqlite3VdbeChangeP4(v, sqlite3VdbeCurrentAddr(v)-1, SQLITE_INT_TO_PTR(n), P4_INT32); + sqlite3VdbeChangeP4(v, sqlite3VdbeCurrentAddr(v)-1, + SQLITE_INT_TO_PTR(n), P4_INT32); assert( n<=pTab->nCol ); } }else{ @@ -90014,7 +90437,7 @@ whereBeginError: } /* -** Generate the end of the WHERE loop. See comments on +** Generate the end of the WHERE loop. See comments on ** sqlite3WhereBegin() for additional information. */ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){ @@ -90050,7 +90473,11 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){ if( pLevel->iLeftJoin ){ int addr; addr = sqlite3VdbeAddOp1(v, OP_IfPos, pLevel->iLeftJoin); - sqlite3VdbeAddOp1(v, OP_NullRow, pTabList->a[i].iCursor); + assert( (pLevel->plan.wsFlags & WHERE_IDX_ONLY)==0 + || (pLevel->plan.wsFlags & WHERE_INDEXED)!=0 ); + if( (pLevel->plan.wsFlags & WHERE_IDX_ONLY)==0 ){ + sqlite3VdbeAddOp1(v, OP_NullRow, pTabList->a[i].iCursor); + } if( pLevel->iIdxCur>=0 ){ sqlite3VdbeAddOp1(v, OP_NullRow, pLevel->iIdxCur); } @@ -90090,7 +90517,7 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){ ** as the vdbe level waits until the table is read before actually ** seeking the table cursor to the record corresponding to the current ** position in the index. - ** + ** ** Calls to the code generator in between sqlite3WhereBegin and ** sqlite3WhereEnd will have created code that references the table ** directly. This loop scans all that code looking for opcodes @@ -90101,7 +90528,6 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){ int k, j, last; VdbeOp *pOp; Index *pIdx = pLevel->plan.u.pIdx; - int useIndexOnly = pLevel->plan.wsFlags & WHERE_IDX_ONLY; assert( pIdx!=0 ); pOp = sqlite3VdbeGetOp(v, pWInfo->iTop); @@ -90116,12 +90542,11 @@ SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo *pWInfo){ break; } } - assert(!useIndexOnly || jnColumn); + assert( (pLevel->plan.wsFlags & WHERE_IDX_ONLY)==0 + || jnColumn ); }else if( pOp->opcode==OP_Rowid ){ pOp->p1 = pLevel->iIdxCur; pOp->opcode = OP_IdxRowid; - }else if( pOp->opcode==OP_NullRow && useIndexOnly ){ - pOp->opcode = OP_Noop; } } } @@ -90240,7 +90665,18 @@ struct AttachKey { int type; Token key; }; pOut->pExpr = sqlite3PExpr(pParse, op, pOperand->pExpr, 0, 0); pOut->zStart = pOperand->zStart; pOut->zEnd = &pPostOp->z[pPostOp->n]; - } + } + + /* A routine to convert a binary TK_IS or TK_ISNOT expression into a + ** unary TK_ISNULL or TK_NOTNULL expression. */ + static void binaryToUnaryIfNull(Parse *pParse, Expr *pY, Expr *pA, int op){ + sqlite3 *db = pParse->db; + if( db->mallocFailed==0 && pY->op==TK_NULL ){ + pA->op = (u8)op; + sqlite3ExprDelete(db, pA->pRight); + pA->pRight = 0; + } + } /* Construct an expression node for a unary prefix operator */ @@ -90258,10 +90694,10 @@ struct AttachKey { int type; Token key; }; /* Next is all token values, in a form suitable for use by makeheaders. ** This section will be null unless lemon is run with the -m switch. */ -/* +/* ** These constants (all generated automatically by the parser generator) ** specify the various kinds of tokens (terminals) that the parser -** understands. +** understands. ** ** Each symbol here is a terminal symbol in the grammar. */ @@ -90278,7 +90714,7 @@ struct AttachKey { int type; Token key; }; ** and nonterminals. "int" is used otherwise. ** YYNOCODE is a number of type YYCODETYPE which corresponds ** to no legal terminal or nonterminal number. This -** number is used to fill in empty slots of the hash +** number is used to fill in empty slots of the hash ** table. ** YYFALLBACK If defined, this indicates that one or more tokens ** have fall-back values which should be used if the @@ -90287,7 +90723,7 @@ struct AttachKey { int type; Token key; }; ** and nonterminal numbers. "unsigned char" is ** used if there are fewer than 250 rules and ** states combined. "int" is used otherwise. -** sqlite3ParserTOKENTYPE is the data type used for minor tokens given +** sqlite3ParserTOKENTYPE is the data type used for minor tokens given ** directly to the parser from the tokenizer. ** YYMINORTYPE is the data type used for all minor tokens. ** This is typically a union of many types, one of @@ -90360,7 +90796,7 @@ static const YYMINORTYPE yyzerominor = { 0 }; /* Next are the tables used to determine what action to take based on the ** current state and lookahead token. These tables are used to implement ** functions that take a state number and lookahead value and return an -** action integer. +** action integer. ** ** Suppose the action integer is N. Then the action is determined as ** follows @@ -90385,7 +90821,7 @@ static const YYMINORTYPE yyzerominor = { 0 }; ** If the index value yy_shift_ofst[S]+X is out of range or if the value ** yy_lookahead[yy_shift_ofst[S]+X] is not equal to X or if yy_shift_ofst[S] ** is equal to YY_SHIFT_USE_DFLT, it means that the action is not in the table -** and that yy_default[S] should be used instead. +** and that yy_default[S] should be used instead. ** ** The formula above is for computing the action when the lookahead is ** a terminal symbol. If the lookahead is a non-terminal (as occurs after @@ -90873,7 +91309,7 @@ static const YYACTIONTYPE yy_default[] = { /* The next table maps tokens into fallback tokens. If a construct ** like the following: -** +** ** %fallback ID X Y Z. ** ** appears in the grammar, then ID becomes a fallback token for X, Y, @@ -90998,10 +91434,10 @@ static char *yyTracePrompt = 0; #endif /* NDEBUG */ #ifndef NDEBUG -/* +/* ** Turn parser tracing on by giving a stream to which to write the trace ** and a prompt to preface each trace message. Tracing is turned off -** by making either argument NULL +** by making either argument NULL ** ** Inputs: **
      @@ -91026,71 +91462,71 @@ SQLITE_PRIVATE void sqlite3ParserTrace(FILE *TraceFILE, char *zTracePrompt){ #ifndef NDEBUG /* For tracing shifts, the names of all terminals and nonterminals ** are required. The following table supplies these names */ -static const char *const yyTokenName[] = { - "$", "SEMI", "EXPLAIN", "QUERY", - "PLAN", "BEGIN", "TRANSACTION", "DEFERRED", - "IMMEDIATE", "EXCLUSIVE", "COMMIT", "END", - "ROLLBACK", "SAVEPOINT", "RELEASE", "TO", - "TABLE", "CREATE", "IF", "NOT", - "EXISTS", "TEMP", "LP", "RP", - "AS", "COMMA", "ID", "INDEXED", - "ABORT", "ACTION", "AFTER", "ANALYZE", - "ASC", "ATTACH", "BEFORE", "BY", - "CASCADE", "CAST", "COLUMNKW", "CONFLICT", - "DATABASE", "DESC", "DETACH", "EACH", - "FAIL", "FOR", "IGNORE", "INITIALLY", - "INSTEAD", "LIKE_KW", "MATCH", "NO", - "KEY", "OF", "OFFSET", "PRAGMA", - "RAISE", "REPLACE", "RESTRICT", "ROW", - "TRIGGER", "VACUUM", "VIEW", "VIRTUAL", - "REINDEX", "RENAME", "CTIME_KW", "ANY", - "OR", "AND", "IS", "BETWEEN", - "IN", "ISNULL", "NOTNULL", "NE", - "EQ", "GT", "LE", "LT", - "GE", "ESCAPE", "BITAND", "BITOR", - "LSHIFT", "RSHIFT", "PLUS", "MINUS", - "STAR", "SLASH", "REM", "CONCAT", - "COLLATE", "BITNOT", "STRING", "JOIN_KW", - "CONSTRAINT", "DEFAULT", "NULL", "PRIMARY", - "UNIQUE", "CHECK", "REFERENCES", "AUTOINCR", - "ON", "DELETE", "UPDATE", "SET", - "DEFERRABLE", "FOREIGN", "DROP", "UNION", - "ALL", "EXCEPT", "INTERSECT", "SELECT", - "DISTINCT", "DOT", "FROM", "JOIN", - "USING", "ORDER", "GROUP", "HAVING", - "LIMIT", "WHERE", "INTO", "VALUES", - "INSERT", "INTEGER", "FLOAT", "BLOB", - "REGISTER", "VARIABLE", "CASE", "WHEN", - "THEN", "ELSE", "INDEX", "ALTER", - "ADD", "error", "input", "cmdlist", - "ecmd", "explain", "cmdx", "cmd", +static const char *const yyTokenName[] = { + "$", "SEMI", "EXPLAIN", "QUERY", + "PLAN", "BEGIN", "TRANSACTION", "DEFERRED", + "IMMEDIATE", "EXCLUSIVE", "COMMIT", "END", + "ROLLBACK", "SAVEPOINT", "RELEASE", "TO", + "TABLE", "CREATE", "IF", "NOT", + "EXISTS", "TEMP", "LP", "RP", + "AS", "COMMA", "ID", "INDEXED", + "ABORT", "ACTION", "AFTER", "ANALYZE", + "ASC", "ATTACH", "BEFORE", "BY", + "CASCADE", "CAST", "COLUMNKW", "CONFLICT", + "DATABASE", "DESC", "DETACH", "EACH", + "FAIL", "FOR", "IGNORE", "INITIALLY", + "INSTEAD", "LIKE_KW", "MATCH", "NO", + "KEY", "OF", "OFFSET", "PRAGMA", + "RAISE", "REPLACE", "RESTRICT", "ROW", + "TRIGGER", "VACUUM", "VIEW", "VIRTUAL", + "REINDEX", "RENAME", "CTIME_KW", "ANY", + "OR", "AND", "IS", "BETWEEN", + "IN", "ISNULL", "NOTNULL", "NE", + "EQ", "GT", "LE", "LT", + "GE", "ESCAPE", "BITAND", "BITOR", + "LSHIFT", "RSHIFT", "PLUS", "MINUS", + "STAR", "SLASH", "REM", "CONCAT", + "COLLATE", "BITNOT", "STRING", "JOIN_KW", + "CONSTRAINT", "DEFAULT", "NULL", "PRIMARY", + "UNIQUE", "CHECK", "REFERENCES", "AUTOINCR", + "ON", "DELETE", "UPDATE", "SET", + "DEFERRABLE", "FOREIGN", "DROP", "UNION", + "ALL", "EXCEPT", "INTERSECT", "SELECT", + "DISTINCT", "DOT", "FROM", "JOIN", + "USING", "ORDER", "GROUP", "HAVING", + "LIMIT", "WHERE", "INTO", "VALUES", + "INSERT", "INTEGER", "FLOAT", "BLOB", + "REGISTER", "VARIABLE", "CASE", "WHEN", + "THEN", "ELSE", "INDEX", "ALTER", + "ADD", "error", "input", "cmdlist", + "ecmd", "explain", "cmdx", "cmd", "transtype", "trans_opt", "nm", "savepoint_opt", - "create_table", "create_table_args", "createkw", "temp", + "create_table", "create_table_args", "createkw", "temp", "ifnotexists", "dbnm", "columnlist", "conslist_opt", - "select", "column", "columnid", "type", - "carglist", "id", "ids", "typetoken", - "typename", "signed", "plus_num", "minus_num", - "carg", "ccons", "term", "expr", - "onconf", "sortorder", "autoinc", "idxlist_opt", - "refargs", "defer_subclause", "refarg", "refact", - "init_deferred_pred_opt", "conslist", "tcons", "idxlist", - "defer_subclause_opt", "orconf", "resolvetype", "raisetype", + "select", "column", "columnid", "type", + "carglist", "id", "ids", "typetoken", + "typename", "signed", "plus_num", "minus_num", + "carg", "ccons", "term", "expr", + "onconf", "sortorder", "autoinc", "idxlist_opt", + "refargs", "defer_subclause", "refarg", "refact", + "init_deferred_pred_opt", "conslist", "tcons", "idxlist", + "defer_subclause_opt", "orconf", "resolvetype", "raisetype", "ifexists", "fullname", "oneselect", "multiselect_op", - "distinct", "selcollist", "from", "where_opt", - "groupby_opt", "having_opt", "orderby_opt", "limit_opt", - "sclp", "as", "seltablist", "stl_prefix", - "joinop", "indexed_opt", "on_opt", "using_opt", - "joinop2", "inscollist", "sortlist", "sortitem", + "distinct", "selcollist", "from", "where_opt", + "groupby_opt", "having_opt", "orderby_opt", "limit_opt", + "sclp", "as", "seltablist", "stl_prefix", + "joinop", "indexed_opt", "on_opt", "using_opt", + "joinop2", "inscollist", "sortlist", "sortitem", "nexprlist", "setlist", "insert_cmd", "inscollist_opt", - "itemlist", "exprlist", "likeop", "escape", + "itemlist", "exprlist", "likeop", "escape", "between_op", "in_op", "case_operand", "case_exprlist", - "case_else", "uniqueflag", "collate", "nmnum", + "case_else", "uniqueflag", "collate", "nmnum", "plus_opt", "number", "trigger_decl", "trigger_cmd_list", - "trigger_time", "trigger_event", "foreach_clause", "when_clause", + "trigger_time", "trigger_event", "foreach_clause", "when_clause", "trigger_cmd", "trnm", "tridxby", "database_kw_opt", - "key_opt", "add_column_fullname", "kwcolumn_opt", "create_vtab", - "vtabarglist", "vtabarg", "vtabargtoken", "lp", - "anylist", + "key_opt", "add_column_fullname", "kwcolumn_opt", "create_vtab", + "vtabarglist", "vtabarg", "vtabargtoken", "lp", + "anylist", }; #endif /* NDEBUG */ @@ -91454,7 +91890,7 @@ static void yyGrowStack(yyParser *p){ } #endif -/* +/* ** This function allocates a new parser. ** The only argument is a pointer to a function which works like ** malloc. @@ -91498,7 +91934,7 @@ static void yy_destructor( /* Here is inserted the actions which take place when a ** terminal or non-terminal is destroyed. This can happen ** when the symbol is popped from the stack during a - ** reduce or during error processing or when a parser is + ** reduce or during error processing or when a parser is ** being destroyed before it is finished parsing. ** ** Note: during a reduce, the only symbols destroyed are those @@ -91604,7 +92040,7 @@ static int yy_pop_parser_stack(yyParser *pParser){ return yymajor; } -/* +/* ** Deallocate and destroy a parser. Destructors are all called for ** all stack elements before shutting the parser down. ** @@ -91655,7 +92091,7 @@ static int yy_find_shift_action( ){ int i; int stateno = pParser->yystack[pParser->yyidx].stateno; - + if( stateno>YY_SHIFT_COUNT || (i = yy_shift_ofst[stateno])==YY_SHIFT_USE_DFLT ){ return yy_default[stateno]; @@ -91680,7 +92116,7 @@ static int yy_find_shift_action( #ifdef YYWILDCARD { int j = i - iLookAhead + YYWILDCARD; - if( + if( #if YY_SHIFT_MIN+YYWILDCARD<0 j>=0 && #endif @@ -91778,7 +92214,7 @@ static void yy_shift( yypParser->yyidxMax = yypParser->yyidx; } #endif -#if YYSTACKDEPTH>0 +#if YYSTACKDEPTH>0 if( yypParser->yyidx>=YYSTACKDEPTH ){ yyStackOverflow(yypParser, yypMinor); return; @@ -92164,7 +92600,7 @@ static void yy_reduce( sqlite3ParserARG_FETCH; yymsp = &yypParser->yystack[yypParser->yyidx]; #ifndef NDEBUG - if( yyTraceFILE && yyruleno>=0 + if( yyTraceFILE && yyruleno>=0 && yyruleno<(int)(sizeof(yyRuleName)/sizeof(yyRuleName[0])) ){ fprintf(yyTraceFILE, "%sReduce [%s].\n", yyTracePrompt, yyRuleName[yyruleno]); @@ -92177,7 +92613,7 @@ static void yy_reduce( ** not set the value of its left-hand side nonterminal. Leaving the ** value of the nonterminal uninitialized is utterly harmless as long ** as the value is never used. So really the only thing this code - ** accomplishes is to quieten purify. + ** accomplishes is to quieten purify. ** ** 2007-01-16: The wireshark project (www.wireshark.org) reports that ** without this code, their parser segfaults. I'm not sure what there @@ -92671,7 +93107,7 @@ static void yy_reduce( case 169: /* cmd ::= UPDATE orconf fullname indexed_opt SET setlist where_opt */ { sqlite3SrcListIndexedBy(pParse, yymsp[-4].minor.yy65, &yymsp[-3].minor.yy0); - sqlite3ExprListCheckLength(pParse,yymsp[-1].minor.yy14,"set list"); + sqlite3ExprListCheckLength(pParse,yymsp[-1].minor.yy14,"set list"); sqlite3Update(pParse,yymsp[-4].minor.yy65,yymsp[-1].minor.yy14,yymsp[0].minor.yy132,yymsp[-5].minor.yy186); } break; @@ -92809,7 +93245,7 @@ static void yy_reduce( ** treated as functions that return constants */ yygotominor.yy346.pExpr = sqlite3ExprFunction(pParse, 0,&yymsp[0].minor.yy0); if( yygotominor.yy346.pExpr ){ - yygotominor.yy346.pExpr->op = TK_CONST_FUNC; + yygotominor.yy346.pExpr->op = TK_CONST_FUNC; } spanSet(&yygotominor.yy346, &yymsp[0].minor.yy0, &yymsp[0].minor.yy0); } @@ -92859,17 +93295,13 @@ static void yy_reduce( case 216: /* expr ::= expr IS expr */ { spanBinaryExpr(&yygotominor.yy346,pParse,TK_IS,&yymsp[-2].minor.yy346,&yymsp[0].minor.yy346); - if( pParse->db->mallocFailed==0 && yymsp[0].minor.yy346.pExpr->op==TK_NULL ){ - yygotominor.yy346.pExpr->op = TK_ISNULL; - } + binaryToUnaryIfNull(pParse, yymsp[0].minor.yy346.pExpr, yygotominor.yy346.pExpr, TK_ISNULL); } break; case 217: /* expr ::= expr IS NOT expr */ { spanBinaryExpr(&yygotominor.yy346,pParse,TK_ISNOT,&yymsp[-3].minor.yy346,&yymsp[0].minor.yy346); - if( pParse->db->mallocFailed==0 && yymsp[0].minor.yy346.pExpr->op==TK_NULL ){ - yygotominor.yy346.pExpr->op = TK_NOTNULL; - } + binaryToUnaryIfNull(pParse, yymsp[0].minor.yy346.pExpr, yygotominor.yy346.pExpr, TK_NOTNULL); } break; case 218: /* expr ::= NOT expr */ @@ -92891,7 +93323,7 @@ static void yy_reduce( yygotominor.yy346.pExpr->x.pList = pList; }else{ sqlite3ExprListDelete(pParse->db, pList); - } + } if( yymsp[-3].minor.yy328 ) yygotominor.yy346.pExpr = sqlite3PExpr(pParse, TK_NOT, yygotominor.yy346.pExpr, 0, 0); yygotominor.yy346.zStart = yymsp[-4].minor.yy346.zStart; yygotominor.yy346.zEnd = yymsp[0].minor.yy346.zEnd; @@ -92997,7 +93429,7 @@ static void yy_reduce( break; case 243: /* cmd ::= createkw uniqueflag INDEX ifnotexists nm dbnm ON nm LP idxlist RP */ { - sqlite3CreateIndex(pParse, &yymsp[-6].minor.yy0, &yymsp[-5].minor.yy0, + sqlite3CreateIndex(pParse, &yymsp[-6].minor.yy0, &yymsp[-5].minor.yy0, sqlite3SrcListAppend(pParse->db,0,&yymsp[-3].minor.yy0,0), yymsp[-1].minor.yy14, yymsp[-9].minor.yy328, &yymsp[-10].minor.yy0, &yymsp[0].minor.yy0, SQLITE_SO_ASC, yymsp[-7].minor.yy328); } @@ -93108,7 +93540,7 @@ static void yy_reduce( } break; case 284: /* trigger_cmd_list ::= trigger_cmd SEMI */ -{ +{ assert( yymsp[-1].minor.yy473!=0 ); yymsp[-1].minor.yy473->pLast = yymsp[-1].minor.yy473; yygotominor.yy473 = yymsp[-1].minor.yy473; @@ -93117,7 +93549,7 @@ static void yy_reduce( case 286: /* trnm ::= nm DOT nm */ { yygotominor.yy0 = yymsp[0].minor.yy0; - sqlite3ErrorMsg(pParse, + sqlite3ErrorMsg(pParse, "qualified table names are not allowed on INSERT, UPDATE, and DELETE " "statements within triggers"); } @@ -93153,7 +93585,7 @@ static void yy_reduce( break; case 295: /* expr ::= RAISE LP IGNORE RP */ { - yygotominor.yy346.pExpr = sqlite3PExpr(pParse, TK_RAISE, 0, 0, 0); + yygotominor.yy346.pExpr = sqlite3PExpr(pParse, TK_RAISE, 0, 0, 0); if( yygotominor.yy346.pExpr ){ yygotominor.yy346.pExpr->affinity = OE_Ignore; } @@ -93163,7 +93595,7 @@ static void yy_reduce( break; case 296: /* expr ::= RAISE LP raisetype COMMA nm RP */ { - yygotominor.yy346.pExpr = sqlite3PExpr(pParse, TK_RAISE, 0, 0, &yymsp[-1].minor.yy0); + yygotominor.yy346.pExpr = sqlite3PExpr(pParse, TK_RAISE, 0, 0, &yymsp[-1].minor.yy0); if( yygotominor.yy346.pExpr ) { yygotominor.yy346.pExpr->affinity = (char)yymsp[-3].minor.yy328; } @@ -93446,7 +93878,7 @@ SQLITE_PRIVATE void sqlite3Parser( #ifdef YYERRORSYMBOL /* A syntax error has occurred. ** The response to an error depends upon whether or not the - ** grammar defines an error token "ERROR". + ** grammar defines an error token "ERROR". ** ** This is what we do if the grammar does define ERROR: ** @@ -93509,7 +93941,7 @@ SQLITE_PRIVATE void sqlite3Parser( yy_syntax_error(yypParser,yymajor,yyminorunion); yy_destructor(yypParser,(YYCODETYPE)yymajor,&yyminorunion); yymajor = YYNOCODE; - + #else /* YYERRORSYMBOL is not defined */ /* This is what we do if the grammar does not define ERROR: ** @@ -93553,8 +93985,6 @@ SQLITE_PRIVATE void sqlite3Parser( ** This file contains C code that splits an SQL input string up into ** individual tokens and sends those tokens one-by-one over to the ** parser for analysis. -** -** $Id: tokenize.c,v 1.163 2009/07/03 22:54:37 drh Exp $ */ /* @@ -93592,7 +94022,7 @@ const unsigned char ebcdicToAscii[] = { /* ** The sqlite3KeywordCode function looks up an identifier to determine if -** it is a keyword. If it is a keyword, the token code of that keyword is +** it is a keyword. If it is a keyword, the token code of that keyword is ** returned. If the input is not a keyword, TK_ID is returned. ** ** The implementation of this routine was generated by a program, @@ -93710,31 +94140,31 @@ static int keywordCode(const char *z, int n){ 521, 527, 531, 536, }; static const unsigned char aCode[121] = { - TK_REINDEX, TK_INDEXED, TK_INDEX, TK_DESC, TK_ESCAPE, - TK_EACH, TK_CHECK, TK_KEY, TK_BEFORE, TK_FOREIGN, - TK_FOR, TK_IGNORE, TK_LIKE_KW, TK_EXPLAIN, TK_INSTEAD, - TK_ADD, TK_DATABASE, TK_AS, TK_SELECT, TK_TABLE, - TK_JOIN_KW, TK_THEN, TK_END, TK_DEFERRABLE, TK_ELSE, - TK_EXCEPT, TK_TRANSACTION,TK_ACTION, TK_ON, TK_JOIN_KW, - TK_ALTER, TK_RAISE, TK_EXCLUSIVE, TK_EXISTS, TK_SAVEPOINT, - TK_INTERSECT, TK_TRIGGER, TK_REFERENCES, TK_CONSTRAINT, TK_INTO, - TK_OFFSET, TK_OF, TK_SET, TK_TEMP, TK_TEMP, - TK_OR, TK_UNIQUE, TK_QUERY, TK_ATTACH, TK_HAVING, - TK_GROUP, TK_UPDATE, TK_BEGIN, TK_JOIN_KW, TK_RELEASE, - TK_BETWEEN, TK_NOTNULL, TK_NOT, TK_NO, TK_NULL, - TK_LIKE_KW, TK_CASCADE, TK_ASC, TK_DELETE, TK_CASE, - TK_COLLATE, TK_CREATE, TK_CTIME_KW, TK_DETACH, TK_IMMEDIATE, - TK_JOIN, TK_INSERT, TK_MATCH, TK_PLAN, TK_ANALYZE, - TK_PRAGMA, TK_ABORT, TK_VALUES, TK_VIRTUAL, TK_LIMIT, - TK_WHEN, TK_WHERE, TK_RENAME, TK_AFTER, TK_REPLACE, - TK_AND, TK_DEFAULT, TK_AUTOINCR, TK_TO, TK_IN, - TK_CAST, TK_COLUMNKW, TK_COMMIT, TK_CONFLICT, TK_JOIN_KW, - TK_CTIME_KW, TK_CTIME_KW, TK_PRIMARY, TK_DEFERRED, TK_DISTINCT, - TK_IS, TK_DROP, TK_FAIL, TK_FROM, TK_JOIN_KW, - TK_LIKE_KW, TK_BY, TK_IF, TK_ISNULL, TK_ORDER, - TK_RESTRICT, TK_JOIN_KW, TK_JOIN_KW, TK_ROLLBACK, TK_ROW, - TK_UNION, TK_USING, TK_VACUUM, TK_VIEW, TK_INITIALLY, - TK_ALL, + TK_REINDEX, TK_INDEXED, TK_INDEX, TK_DESC, TK_ESCAPE, + TK_EACH, TK_CHECK, TK_KEY, TK_BEFORE, TK_FOREIGN, + TK_FOR, TK_IGNORE, TK_LIKE_KW, TK_EXPLAIN, TK_INSTEAD, + TK_ADD, TK_DATABASE, TK_AS, TK_SELECT, TK_TABLE, + TK_JOIN_KW, TK_THEN, TK_END, TK_DEFERRABLE, TK_ELSE, + TK_EXCEPT, TK_TRANSACTION,TK_ACTION, TK_ON, TK_JOIN_KW, + TK_ALTER, TK_RAISE, TK_EXCLUSIVE, TK_EXISTS, TK_SAVEPOINT, + TK_INTERSECT, TK_TRIGGER, TK_REFERENCES, TK_CONSTRAINT, TK_INTO, + TK_OFFSET, TK_OF, TK_SET, TK_TEMP, TK_TEMP, + TK_OR, TK_UNIQUE, TK_QUERY, TK_ATTACH, TK_HAVING, + TK_GROUP, TK_UPDATE, TK_BEGIN, TK_JOIN_KW, TK_RELEASE, + TK_BETWEEN, TK_NOTNULL, TK_NOT, TK_NO, TK_NULL, + TK_LIKE_KW, TK_CASCADE, TK_ASC, TK_DELETE, TK_CASE, + TK_COLLATE, TK_CREATE, TK_CTIME_KW, TK_DETACH, TK_IMMEDIATE, + TK_JOIN, TK_INSERT, TK_MATCH, TK_PLAN, TK_ANALYZE, + TK_PRAGMA, TK_ABORT, TK_VALUES, TK_VIRTUAL, TK_LIMIT, + TK_WHEN, TK_WHERE, TK_RENAME, TK_AFTER, TK_REPLACE, + TK_AND, TK_DEFAULT, TK_AUTOINCR, TK_TO, TK_IN, + TK_CAST, TK_COLUMNKW, TK_COMMIT, TK_CONFLICT, TK_JOIN_KW, + TK_CTIME_KW, TK_CTIME_KW, TK_PRIMARY, TK_DEFERRED, TK_DISTINCT, + TK_IS, TK_DROP, TK_FAIL, TK_FROM, TK_JOIN_KW, + TK_LIKE_KW, TK_BY, TK_IF, TK_ISNULL, TK_ORDER, + TK_RESTRICT, TK_JOIN_KW, TK_JOIN_KW, TK_ROLLBACK, TK_ROW, + TK_UNION, TK_USING, TK_VACUUM, TK_VIEW, TK_INITIALLY, + TK_ALL, }; int h, i; if( n<2 ) return TK_ID; @@ -93882,28 +94312,19 @@ SQLITE_PRIVATE int sqlite3KeywordCode(const unsigned char *z, int n){ ** IdChar(X) will be true. Otherwise it is false. ** ** For ASCII, any character with the high-order bit set is -** allowed in an identifier. For 7-bit characters, +** allowed in an identifier. For 7-bit characters, ** sqlite3IsIdChar[X] must be 1. ** ** For EBCDIC, the rules are more complex but have the same ** end result. ** ** Ticket #1066. the SQL standard does not allow '$' in the -** middle of identfiers. But many SQL implementations do. +** middle of identfiers. But many SQL implementations do. ** SQLite will allow '$' in identifiers for compatibility. ** But the feature is undocumented. */ #ifdef SQLITE_ASCII -SQLITE_PRIVATE const char sqlite3IsAsciiIdChar[] = { -/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */ - 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 2x */ - 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, /* 3x */ - 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 4x */ - 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, /* 5x */ - 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 6x */ - 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, /* 7x */ -}; -#define IdChar(C) (((c=C)&0x80)!=0 || (c>0x1f && sqlite3IsAsciiIdChar[c-0x20])) +#define IdChar(C) ((sqlite3CtypeMap[(unsigned char)C]&0x46)!=0) #endif #ifdef SQLITE_EBCDIC SQLITE_PRIVATE const char sqlite3IsEbcdicIdChar[] = { @@ -93926,7 +94347,7 @@ SQLITE_PRIVATE const char sqlite3IsEbcdicIdChar[] = { /* -** Return the length of the token that begins at z[0]. +** Return the length of the token that begins at z[0]. ** Store the token type in *tokenType before returning. */ SQLITE_PRIVATE int sqlite3GetToken(const unsigned char *z, int *tokenType){ @@ -94099,7 +94520,7 @@ SQLITE_PRIVATE int sqlite3GetToken(const unsigned char *z, int *tokenType){ *tokenType = TK_FLOAT; } if( (z[i]=='e' || z[i]=='E') && - ( sqlite3Isdigit(z[i+1]) + ( sqlite3Isdigit(z[i+1]) || ((z[i+1]=='+' || z[i+1]=='-') && sqlite3Isdigit(z[i+2])) ) ){ @@ -94200,7 +94621,7 @@ SQLITE_PRIVATE int sqlite3GetToken(const unsigned char *z, int *tokenType){ /* ** Run the parser on the given SQL string. The parser structure is ** passed in. An SQLITE_ status code is returned. If an error occurs -** then an and attempt is made to write an error message into +** then an and attempt is made to write an error message into ** memory obtained from sqlite3_malloc() and to make *pzErrMsg point to that ** error message. */ @@ -94318,7 +94739,7 @@ abort_parse: #endif if( !IN_DECLARE_VTAB ){ - /* If the pParse->declareVtab flag is set, do not delete any table + /* If the pParse->declareVtab flag is set, do not delete any table ** structure built up in pParse->pNewTable. The calling code (see vtab.c) ** will take responsibility for freeing the Table structure. */ @@ -94363,8 +94784,6 @@ abort_parse: ** This code used to be part of the tokenizer.c source file. But by ** separating it out, the code will be automatically omitted from ** static links that do not use it. -** -** $Id: complete.c,v 1.8 2009/04/28 04:46:42 drh Exp $ */ #ifndef SQLITE_OMIT_COMPLETE @@ -94373,8 +94792,7 @@ abort_parse: */ #ifndef SQLITE_AMALGAMATION #ifdef SQLITE_ASCII -SQLITE_PRIVATE const char sqlite3IsAsciiIdChar[]; -#define IdChar(C) (((c=C)&0x80)!=0 || (c>0x1f && sqlite3IsAsciiIdChar[c-0x20])) +#define IdChar(C) ((sqlite3CtypeMap[(unsigned char)C]&0x46)!=0) #endif #ifdef SQLITE_EBCDIC SQLITE_PRIVATE const char sqlite3IsEbcdicIdChar[]; @@ -94412,7 +94830,7 @@ SQLITE_PRIVATE const char sqlite3IsEbcdicIdChar[]; ** (1) NORMAL We are in the middle of statement which ends with a single ** semicolon. ** -** (2) EXPLAIN The keyword EXPLAIN has been seen at the beginning of +** (2) EXPLAIN The keyword EXPLAIN has been seen at the beginning of ** a statement. ** ** (3) CREATE The keyword CREATE has been seen at the beginning of a @@ -94531,7 +94949,9 @@ SQLITE_API int sqlite3_complete(const char *zSql){ break; } default: { - int c; +#ifdef SQLITE_EBCDIC + unsigned char c; +#endif if( IdChar((u8)*zSql) ){ /* Keywords and unquoted identifiers */ int nId; @@ -94771,13 +95191,13 @@ SQLITE_PRIVATE void (*sqlite3IoTrace)(const char*, ...) = 0; SQLITE_API char *sqlite3_temp_directory = 0; /* -** Initialize SQLite. +** Initialize SQLite. ** ** This routine must be called to initialize the memory allocation, ** VFS, and mutex subsystems prior to doing any serious work with ** SQLite. But as long as you do not compile with SQLITE_OMIT_AUTOINIT ** this routine will be called automatically by key routines such as -** sqlite3_open(). +** sqlite3_open(). ** ** This routine is a no-op except on its very first call for the process, ** or for the first call after a call to sqlite3_shutdown. @@ -94819,7 +95239,7 @@ SQLITE_API int sqlite3_initialize(void){ */ if( sqlite3GlobalConfig.isInit ) return SQLITE_OK; - /* Make sure the mutex subsystem is initialized. If unable to + /* Make sure the mutex subsystem is initialized. If unable to ** initialize the mutex subsystem, return early with the error. ** If the system is so sick that we are unable to allocate a mutex, ** there is not much SQLite is going to be able to do. @@ -94884,7 +95304,7 @@ SQLITE_API int sqlite3_initialize(void){ rc = sqlite3OsInit(); } if( rc==SQLITE_OK ){ - sqlite3PCacheBufferSetup( sqlite3GlobalConfig.pPage, + sqlite3PCacheBufferSetup( sqlite3GlobalConfig.pPage, sqlite3GlobalConfig.szPage, sqlite3GlobalConfig.nPage); sqlite3GlobalConfig.isInit = 1; } @@ -94977,7 +95397,7 @@ SQLITE_API int sqlite3_config(int op, ...){ switch( op ){ /* Mutex configuration options are only available in a threadsafe - ** compile. + ** compile. */ #if SQLITE_THREADSAFE case SQLITE_CONFIG_SINGLETHREAD: { @@ -95104,7 +95524,7 @@ SQLITE_API int sqlite3_config(int op, ...){ /* ** Set up the lookaside buffers for a database connection. -** Return SQLITE_OK on success. +** Return SQLITE_OK on success. ** If lookaside is already active, return SQLITE_BUSY. ** ** The sz parameter is the number of bytes in each lookaside slot. @@ -95119,7 +95539,7 @@ static int setupLookaside(sqlite3 *db, void *pBuf, int sz, int cnt){ return SQLITE_BUSY; } /* Free any existing lookaside buffer for this handle before - ** allocating a new one so we don't have to have space for + ** allocating a new one so we don't have to have space for ** both at the same time. */ if( db->lookaside.bMalloced ){ @@ -95235,7 +95655,7 @@ static int binCollFunc( } /* -** Another built-in collating sequence: NOCASE. +** Another built-in collating sequence: NOCASE. ** ** This collating sequence is intended to be used for "case independant ** comparison". SQLite's knowledge of upper and lower case equivalents @@ -95322,7 +95742,7 @@ SQLITE_API int sqlite3_close(sqlite3 *db){ /* If there are any outstanding VMs, return SQLITE_BUSY. */ if( db->pVdbe ){ - sqlite3Error(db, SQLITE_BUSY, + sqlite3Error(db, SQLITE_BUSY, "unable to close due to unfinalised statements"); sqlite3_mutex_leave(db->mutex); return SQLITE_BUSY; @@ -95332,7 +95752,7 @@ SQLITE_API int sqlite3_close(sqlite3 *db){ for(j=0; jnDb; j++){ Btree *pBt = db->aDb[j].pBt; if( pBt && sqlite3BtreeIsInBackup(pBt) ){ - sqlite3Error(db, SQLITE_BUSY, + sqlite3Error(db, SQLITE_BUSY, "unable to close due to unfinished backup operation"); sqlite3_mutex_leave(db->mutex); return SQLITE_BUSY; @@ -95405,7 +95825,7 @@ SQLITE_API int sqlite3_close(sqlite3 *db){ /* The temp-database schema is allocated differently from the other schema ** objects (using sqliteMalloc() directly, instead of sqlite3BtreeSchema()). ** So it needs to be freed here. Todo: Why not roll the temp schema into - ** the same sqliteMalloc() as the one that allocates the database + ** the same sqliteMalloc() as the one that allocates the database ** structure? */ sqlite3DbFree(db, db->aDb[1].pSchema); @@ -95557,7 +95977,7 @@ SQLITE_PRIVATE int sqlite3InvokeBusyHandler(BusyHandler *p){ }else{ p->nBusy++; } - return rc; + return rc; } /* @@ -95584,9 +96004,9 @@ SQLITE_API int sqlite3_busy_handler( ** be invoked every nOps opcodes. */ SQLITE_API void sqlite3_progress_handler( - sqlite3 *db, + sqlite3 *db, int nOps, - int (*xProgress)(void*), + int (*xProgress)(void*), void *pArg ){ sqlite3_mutex_enter(db->mutex); @@ -95630,7 +96050,7 @@ SQLITE_API void sqlite3_interrupt(sqlite3 *db){ ** This function is exactly the same as sqlite3_create_function(), except ** that it is designed to be called by internal code. The difference is ** that if a malloc() fails in sqlite3_create_function(), an error code -** is returned and the mallocFailed flag cleared. +** is returned and the mallocFailed flag cleared. */ SQLITE_PRIVATE int sqlite3CreateFunc( sqlite3 *db, @@ -95647,14 +96067,14 @@ SQLITE_PRIVATE int sqlite3CreateFunc( assert( sqlite3_mutex_held(db->mutex) ); if( zFunctionName==0 || - (xFunc && (xFinal || xStep)) || + (xFunc && (xFinal || xStep)) || (!xFunc && (xFinal && !xStep)) || (!xFunc && (!xFinal && xStep)) || (nArg<-1 || nArg>SQLITE_MAX_FUNCTION_ARG) || (255<(nName = sqlite3Strlen30( zFunctionName))) ){ return SQLITE_MISUSE; } - + #ifndef SQLITE_OMIT_UTF16 /* If SQLITE_UTF16 is specified as the encoding type, transform this ** to one of SQLITE_UTF16LE or SQLITE_UTF16BE using the @@ -95681,7 +96101,7 @@ SQLITE_PRIVATE int sqlite3CreateFunc( #else enc = SQLITE_UTF8; #endif - + /* Check if an existing function is being overridden or deleted. If so, ** and there are active VMs, then return SQLITE_BUSY. If a function ** is being overridden/deleted but there are no active VMs, allow the @@ -95690,7 +96110,7 @@ SQLITE_PRIVATE int sqlite3CreateFunc( p = sqlite3FindFunction(db, zFunctionName, nName, nArg, (u8)enc, 0); if( p && p->iPrefEnc==enc && p->nArg==nArg ){ if( db->activeVdbeCnt ){ - sqlite3Error(db, SQLITE_BUSY, + sqlite3Error(db, SQLITE_BUSY, "unable to delete/modify user-function due to active statements"); assert( !db->mallocFailed ); return SQLITE_BUSY; @@ -95764,7 +96184,7 @@ SQLITE_API int sqlite3_create_function16( ** ** If the function already exists as a regular global function, then ** this routine is a no-op. If the function does not exist, then create -** a new one that always throws a run-time error. +** a new one that always throws a run-time error. ** ** When virtual tables intend to provide an overloaded function, they ** should call this routine to make sure the global function exists. @@ -95791,7 +96211,7 @@ SQLITE_API int sqlite3_overload_function( #ifndef SQLITE_OMIT_TRACE /* ** Register a trace function. The pArg from the previously registered trace -** is returned. +** is returned. ** ** A NULL trace function means that no tracing is executes. A non-NULL ** trace is a pointer to a function that is invoked at the start of each @@ -95807,8 +96227,8 @@ SQLITE_API void *sqlite3_trace(sqlite3 *db, void (*xTrace)(void*,const char*), v return pOld; } /* -** Register a profile function. The pArg from the previously registered -** profile function is returned. +** Register a profile function. The pArg from the previously registered +** profile function is returned. ** ** A NULL profile function means that no profiling is executes. A non-NULL ** profile is a pointer to a function that is invoked at the conclusion of @@ -95942,7 +96362,7 @@ SQLITE_PRIVATE int sqlite3BtreeFactory( ){ int btFlags = 0; int rc; - + assert( sqlite3_mutex_held(db->mutex) ); assert( ppBtree != 0); if( omitJournal ){ @@ -95964,7 +96384,7 @@ SQLITE_PRIVATE int sqlite3BtreeFactory( /* If the B-Tree was successfully opened, set the pager-cache size to the ** default value. Except, if the call to BtreeOpen() returned a handle - ** open on an existing shared pager-cache, do not change the pager-cache + ** open on an existing shared pager-cache, do not change the pager-cache ** size. */ if( rc==SQLITE_OK && 0==sqlite3BtreeSchema(*ppBtree, 0, 0) ){ @@ -96009,11 +96429,11 @@ SQLITE_API const void *sqlite3_errmsg16(sqlite3 *db){ 'o', 'u', 't', ' ', 'o', 'f', ' ', 'm', 'e', 'm', 'o', 'r', 'y', 0 }; static const u16 misuse[] = { - 'l', 'i', 'b', 'r', 'a', 'r', 'y', ' ', - 'r', 'o', 'u', 't', 'i', 'n', 'e', ' ', - 'c', 'a', 'l', 'l', 'e', 'd', ' ', - 'o', 'u', 't', ' ', - 'o', 'f', ' ', + 'l', 'i', 'b', 'r', 'a', 'r', 'y', ' ', + 'r', 'o', 'u', 't', 'i', 'n', 'e', ' ', + 'c', 'a', 'l', 'l', 'e', 'd', ' ', + 'o', 'u', 't', ' ', + 'o', 'f', ' ', 's', 'e', 'q', 'u', 'e', 'n', 'c', 'e', 0 }; @@ -96075,7 +96495,7 @@ SQLITE_API int sqlite3_extended_errcode(sqlite3 *db){ */ static int createCollation( sqlite3* db, - const char *zName, + const char *zName, u8 enc, u8 collType, void* pCtx, @@ -96085,7 +96505,7 @@ static int createCollation( CollSeq *pColl; int enc2; int nName = sqlite3Strlen30(zName); - + assert( sqlite3_mutex_held(db->mutex) ); /* If SQLITE_UTF16 is specified as the encoding type, transform this @@ -96102,14 +96522,14 @@ static int createCollation( return SQLITE_MISUSE; } - /* Check if this call is removing or replacing an existing collation + /* Check if this call is removing or replacing an existing collation ** sequence. If so, and there are active VMs, return busy. If there ** are no active VMs, invalidate any pre-compiled statements. */ pColl = sqlite3FindCollSeq(db, (u8)enc2, zName, 0); if( pColl && pColl->xCmp ){ if( db->activeVdbeCnt ){ - sqlite3Error(db, SQLITE_BUSY, + sqlite3Error(db, SQLITE_BUSY, "unable to delete/modify collation sequence due to active statements"); return SQLITE_BUSY; } @@ -96120,7 +96540,7 @@ static int createCollation( ** then any copies made by synthCollSeq() need to be invalidated. ** Also, collation destructor - CollSeq.xDel() - function may need ** to be called. - */ + */ if( (pColl->enc & ~SQLITE_UTF16_ALIGNED)==enc2 ){ CollSeq *aColl = sqlite3HashFind(&db->aCollSeq, zName, nName); int j; @@ -96230,7 +96650,7 @@ SQLITE_API int sqlite3_limit(sqlite3 *db, int limitId, int newLimit){ /* ** This routine does the work of opening a database on behalf of -** sqlite3_open() and sqlite3_open16(). The database filename "zFilename" +** sqlite3_open() and sqlite3_open16(). The database filename "zFilename" ** is UTF-8 encoded. */ static int openDatabase( @@ -96275,11 +96695,11 @@ static int openDatabase( flags &= ~( SQLITE_OPEN_DELETEONCLOSE | SQLITE_OPEN_EXCLUSIVE | SQLITE_OPEN_MAIN_DB | - SQLITE_OPEN_TEMP_DB | - SQLITE_OPEN_TRANSIENT_DB | - SQLITE_OPEN_MAIN_JOURNAL | - SQLITE_OPEN_TEMP_JOURNAL | - SQLITE_OPEN_SUBJOURNAL | + SQLITE_OPEN_TEMP_DB | + SQLITE_OPEN_TRANSIENT_DB | + SQLITE_OPEN_MAIN_JOURNAL | + SQLITE_OPEN_TEMP_JOURNAL | + SQLITE_OPEN_SUBJOURNAL | SQLITE_OPEN_MASTER_JOURNAL | SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_FULLMUTEX @@ -96354,7 +96774,7 @@ static int openDatabase( /* Open the backend database driver */ db->openFlags = flags; - rc = sqlite3BtreeFactory(db, zFilename, 0, SQLITE_DEFAULT_CACHE_SIZE, + rc = sqlite3BtreeFactory(db, zFilename, 0, SQLITE_DEFAULT_CACHE_SIZE, flags | SQLITE_OPEN_MAIN_DB, &db->aDb[0].pBt); if( rc!=SQLITE_OK ){ @@ -96369,7 +96789,7 @@ static int openDatabase( /* The default safety_level for the main database is 'full'; for the temp - ** database it is 'NONE'. This matches the pager layer defaults. + ** database it is 'NONE'. This matches the pager layer defaults. */ db->aDb[0].zName = "main"; db->aDb[0].safety_level = 3; @@ -96465,8 +96885,8 @@ opendb_out: ** Open a new database handle. */ SQLITE_API int sqlite3_open( - const char *zFilename, - sqlite3 **ppDb + const char *zFilename, + sqlite3 **ppDb ){ return openDatabase(zFilename, ppDb, SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, 0); @@ -96485,7 +96905,7 @@ SQLITE_API int sqlite3_open_v2( ** Open a new database handle. */ SQLITE_API int sqlite3_open16( - const void *zFilename, + const void *zFilename, sqlite3 **ppDb ){ char const *zFilename8; /* zFilename encoded in UTF-8 instead of UTF-16 */ @@ -96522,9 +96942,9 @@ SQLITE_API int sqlite3_open16( ** Register a new collation sequence with the database handle db. */ SQLITE_API int sqlite3_create_collation( - sqlite3* db, - const char *zName, - int enc, + sqlite3* db, + const char *zName, + int enc, void* pCtx, int(*xCompare)(void*,int,const void*,int,const void*) ){ @@ -96541,9 +96961,9 @@ SQLITE_API int sqlite3_create_collation( ** Register a new collation sequence with the database handle db. */ SQLITE_API int sqlite3_create_collation_v2( - sqlite3* db, - const char *zName, - int enc, + sqlite3* db, + const char *zName, + int enc, void* pCtx, int(*xCompare)(void*,int,const void*,int,const void*), void(*xDel)(void*) @@ -96562,9 +96982,9 @@ SQLITE_API int sqlite3_create_collation_v2( ** Register a new collation sequence with the database handle db. */ SQLITE_API int sqlite3_create_collation16( - sqlite3* db, + sqlite3* db, const void *zName, - int enc, + int enc, void* pCtx, int(*xCompare)(void*,int,const void*,int,const void*) ){ @@ -96588,8 +97008,8 @@ SQLITE_API int sqlite3_create_collation16( ** db. Replace any previously installed collation sequence factory. */ SQLITE_API int sqlite3_collation_needed( - sqlite3 *db, - void *pCollNeededArg, + sqlite3 *db, + void *pCollNeededArg, void(*xCollNeeded)(void*,sqlite3*,int eTextRep,const char*) ){ sqlite3_mutex_enter(db->mutex); @@ -96606,8 +97026,8 @@ SQLITE_API int sqlite3_collation_needed( ** db. Replace any previously installed collation sequence factory. */ SQLITE_API int sqlite3_collation_needed16( - sqlite3 *db, - void *pCollNeededArg, + sqlite3 *db, + void *pCollNeededArg, void(*xCollNeeded16)(void*,sqlite3*,int eTextRep,const void*) ){ sqlite3_mutex_enter(db->mutex); @@ -96732,13 +97152,13 @@ SQLITE_API int sqlite3_table_column_metadata( /* The following block stores the meta information that will be returned ** to the caller in local variables zDataType, zCollSeq, notnull, primarykey ** and autoinc. At this point there are two possibilities: - ** - ** 1. The specified column name was rowid", "oid" or "_rowid_" - ** and there is no explicitly declared IPK column. ** - ** 2. The table is not a view and the column name identified an + ** 1. The specified column name was rowid", "oid" or "_rowid_" + ** and there is no explicitly declared IPK column. + ** + ** 2. The table is not a view and the column name identified an ** explicitly declared column. Copy meta information from *pCol. - */ + */ if( pCol ){ zDataType = pCol->zType; zCollSeq = pCol->zColl; @@ -96790,7 +97210,7 @@ SQLITE_API int sqlite3_sleep(int ms){ pVfs = sqlite3_vfs_find(0); if( pVfs==0 ) return 0; - /* This function works in milliseconds, but the underlying OsSleep() + /* This function works in milliseconds, but the underlying OsSleep() ** API uses microseconds. Hence the 1000's. */ rc = (sqlite3OsSleep(pVfs, 1000*ms)/1000); @@ -96838,7 +97258,7 @@ SQLITE_API int sqlite3_file_control(sqlite3 *db, const char *zDbName, int op, vo } } sqlite3_mutex_leave(db->mutex); - return rc; + return rc; } /* @@ -96897,7 +97317,7 @@ SQLITE_API int sqlite3_test_control(int op, ...){ /* ** sqlite3_test_control(BENIGN_MALLOC_HOOKS, xBegin, xEnd) ** - ** Register hooks to call to indicate which malloc() failures + ** Register hooks to call to indicate which malloc() failures ** are benign. */ case SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS: { @@ -96954,7 +97374,7 @@ SQLITE_API int sqlite3_test_control(int op, ...){ ** This action provides a run-time test to see how the ALWAYS and ** NEVER macros were defined at compile-time. ** - ** The return value is ALWAYS(X). + ** The return value is ALWAYS(X). ** ** The recommended test is X==2. If the return value is 2, that means ** ALWAYS() and NEVER() are both no-op pass-through macros, which is the @@ -97017,8 +97437,6 @@ SQLITE_API int sqlite3_test_control(int op, ...){ ** ** This file contains the implementation of the sqlite3_unlock_notify() ** API method and its associated functionality. -** -** $Id: notify.c,v 1.4 2009/04/07 22:06:57 drh Exp $ */ /* Omit this entire file if SQLITE_ENABLE_UNLOCK_NOTIFY is not defined. */ @@ -97046,13 +97464,13 @@ static sqlite3 *SQLITE_WSD sqlite3BlockedList = 0; #ifndef NDEBUG /* -** This function is a complex assert() that verifies the following +** This function is a complex assert() that verifies the following ** properties of the blocked connections list: ** -** 1) Each entry in the list has a non-NULL value for either +** 1) Each entry in the list has a non-NULL value for either ** pUnlockConnection or pBlockingConnection, or both. ** -** 2) All entries in the list that share a common value for +** 2) All entries in the list that share a common value for ** xUnlockNotify are grouped together. ** ** 3) If the argument db is not NULL, then none of the entries in the @@ -97104,8 +97522,8 @@ static void addToBlockedList(sqlite3 *db){ sqlite3 **pp; assertMutexHeld(); for( - pp=&sqlite3BlockedList; - *pp && (*pp)->xUnlockNotify!=db->xUnlockNotify; + pp=&sqlite3BlockedList; + *pp && (*pp)->xUnlockNotify!=db->xUnlockNotify; pp=&(*pp)->pNextBlocked ); db->pNextBlocked = *pp; @@ -97166,9 +97584,9 @@ SQLITE_API int sqlite3_unlock_notify( db->xUnlockNotify = 0; db->pUnlockArg = 0; }else if( 0==db->pBlockingConnection ){ - /* The blocking transaction has been concluded. Or there never was a + /* The blocking transaction has been concluded. Or there never was a ** blocking transaction. In either case, invoke the notify callback - ** immediately. + ** immediately. */ xNotify(&pArg, 1); }else{ @@ -97194,7 +97612,7 @@ SQLITE_API int sqlite3_unlock_notify( } /* -** This function is called while stepping or preparing a statement +** This function is called while stepping or preparing a statement ** associated with connection db. The operation will return SQLITE_LOCKED ** to the user because it requires a lock that will not be available ** until connection pBlocker concludes its current transaction. @@ -97210,7 +97628,7 @@ SQLITE_PRIVATE void sqlite3ConnectionBlocked(sqlite3 *db, sqlite3 *pBlocker){ /* ** This function is called when -** the transaction opened by database db has just finished. Locks held +** the transaction opened by database db has just finished. Locks held ** by database connection db have been released. ** ** This function loops through each entry in the blocked connections @@ -97270,7 +97688,7 @@ SQLITE_PRIVATE void sqlite3ConnectionUnlocked(sqlite3 *db){ }else{ /* This occurs when the array of context pointers that need to ** be passed to the unlock-notify callback is larger than the - ** aStatic[] array allocated on the stack and the attempt to + ** aStatic[] array allocated on the stack and the attempt to ** allocate a larger array from the heap has failed. ** ** This is a difficult situation to handle. Returning an error @@ -97278,17 +97696,17 @@ SQLITE_PRIVATE void sqlite3ConnectionUnlocked(sqlite3 *db){ ** is returned the transaction on connection db will still be ** closed and the unlock-notify callbacks on blocked connections ** will go unissued. This might cause the application to wait - ** indefinitely for an unlock-notify callback that will never + ** indefinitely for an unlock-notify callback that will never ** arrive. ** ** Instead, invoke the unlock-notify callback with the context ** array already accumulated. We can then clear the array and - ** begin accumulating any further context pointers without + ** begin accumulating any further context pointers without ** requiring any dynamic allocation. This is sub-optimal because ** it means that instead of one callback with a large array of ** context pointers the application will receive two or more ** callbacks with smaller arrays of context pointers, which will - ** reduce the applications ability to prioritize multiple + ** reduce the applications ability to prioritize multiple ** connections. But it is the best that can be done under the ** circumstances. */ @@ -97323,7 +97741,7 @@ SQLITE_PRIVATE void sqlite3ConnectionUnlocked(sqlite3 *db){ } /* -** This is called when the database connection passed as an argument is +** This is called when the database connection passed as an argument is ** being closed. The connection is removed from the blocked list. */ SQLITE_PRIVATE void sqlite3ConnectionClosed(sqlite3 *db){ @@ -97617,11 +98035,10 @@ SQLITE_PRIVATE void sqlite3ConnectionClosed(sqlite3 *db){ # define SQLITE_CORE 1 #endif - -/************** Include fts3_expr.h in the middle of fts3.c ******************/ -/************** Begin file fts3_expr.h ***************************************/ +/************** Include fts3Int.h in the middle of fts3.c ********************/ +/************** Begin file fts3Int.h *****************************************/ /* -** 2008 Nov 28 +** 2009 Nov 12 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: @@ -97634,7 +98051,14 @@ SQLITE_PRIVATE void sqlite3ConnectionClosed(sqlite3 *db){ ** */ -/************** Include fts3_tokenizer.h in the middle of fts3_expr.h ********/ +#ifndef _FTSINT_H +#define _FTSINT_H + +#if !defined(NDEBUG) && !defined(SQLITE_DEBUG) +# define NDEBUG 1 +#endif + +/************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/ /************** Begin file fts3_tokenizer.h **********************************/ /* ** 2006 July 10 @@ -97672,7 +98096,7 @@ SQLITE_PRIVATE void sqlite3ConnectionClosed(sqlite3 *db){ ** When an fts3 table is created, it passes any arguments passed to ** the tokenizer clause of the CREATE VIRTUAL TABLE statement to the ** sqlite3_tokenizer_module.xCreate() function of the requested tokenizer -** implementation. The xCreate() function in turn returns an +** implementation. The xCreate() function in turn returns an ** sqlite3_tokenizer structure representing the specific tokenizer to ** be used for the fts3 table (customized by the tokenizer clause arguments). ** @@ -97704,7 +98128,7 @@ struct sqlite3_tokenizer_module { ** then argc is set to 2, and the argv[] array contains pointers ** to the strings "arg1" and "arg2". ** - ** This method should return either SQLITE_OK (0), or an SQLite error + ** This method should return either SQLITE_OK (0), or an SQLite error ** code. If SQLITE_OK is returned, then *ppTokenizer should be set ** to point at the newly created tokenizer structure. The generic ** sqlite3_tokenizer.pModule variable should not be initialised by @@ -97725,7 +98149,7 @@ struct sqlite3_tokenizer_module { /* ** Create a tokenizer cursor to tokenize an input buffer. The caller ** is responsible for ensuring that the input buffer remains valid - ** until the cursor is closed (using the xClose() method). + ** until the cursor is closed (using the xClose() method). */ int (*xOpen)( sqlite3_tokenizer *pTokenizer, /* Tokenizer object */ @@ -97734,7 +98158,7 @@ struct sqlite3_tokenizer_module { ); /* - ** Destroy an existing tokenizer cursor. The fts3 module calls this + ** Destroy an existing tokenizer cursor. The fts3 module calls this ** method exactly once for each successful call to xOpen(). */ int (*xClose)(sqlite3_tokenizer_cursor *pCursor); @@ -97745,7 +98169,7 @@ struct sqlite3_tokenizer_module { ** "OUT" variables identified below, or SQLITE_DONE to indicate that ** the end of the buffer has been reached, or an SQLite error code. ** - ** *ppToken should be set to point at a buffer containing the + ** *ppToken should be set to point at a buffer containing the ** normalized version of the token (i.e. after any case-folding and/or ** stemming has been performed). *pnBytes should be set to the length ** of this buffer in bytes. The input text that generated the token is @@ -97757,7 +98181,7 @@ struct sqlite3_tokenizer_module { ** ** The buffer *ppToken is set to point at is managed by the tokenizer ** implementation. It is only required to be valid until the next call - ** to xNext() or xClose(). + ** to xNext() or xClose(). */ /* TODO(shess) current implementation requires pInput to be ** nul-terminated. This should either be fixed, or pInput/nBytes @@ -97785,91 +98209,8 @@ struct sqlite3_tokenizer_cursor { #endif /* _FTS3_TOKENIZER_H_ */ /************** End of fts3_tokenizer.h **************************************/ -/************** Continuing where we left off in fts3_expr.h ******************/ - -/* -** The following describes the syntax supported by the fts3 MATCH -** operator in a similar format to that used by the lemon parser -** generator. This module does not use actually lemon, it uses a -** custom parser. -** -** query ::= andexpr (OR andexpr)*. -** -** andexpr ::= notexpr (AND? notexpr)*. -** -** notexpr ::= nearexpr (NOT nearexpr|-TOKEN)*. -** notexpr ::= LP query RP. -** -** nearexpr ::= phrase (NEAR distance_opt nearexpr)*. -** -** distance_opt ::= . -** distance_opt ::= / INTEGER. -** -** phrase ::= TOKEN. -** phrase ::= COLUMN:TOKEN. -** phrase ::= "TOKEN TOKEN TOKEN...". -*/ - -typedef struct Fts3Expr Fts3Expr; -typedef struct Fts3Phrase Fts3Phrase; - -/* -** A "phrase" is a sequence of one or more tokens that must match in -** sequence. A single token is the base case and the most common case. -** For a sequence of tokens contained in "...", nToken will be the number -** of tokens in the string. -*/ -struct Fts3Phrase { - int nToken; /* Number of tokens in the phrase */ - int iColumn; /* Index of column this phrase must match */ - int isNot; /* Phrase prefixed by unary not (-) operator */ - struct PhraseToken { - char *z; /* Text of the token */ - int n; /* Number of bytes in buffer pointed to by z */ - int isPrefix; /* True if token ends in with a "*" character */ - } aToken[1]; /* One entry for each token in the phrase */ -}; - -/* -** A tree of these objects forms the RHS of a MATCH operator. -*/ -struct Fts3Expr { - int eType; /* One of the FTSQUERY_XXX values defined below */ - int nNear; /* Valid if eType==FTSQUERY_NEAR */ - Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */ - Fts3Expr *pLeft; /* Left operand */ - Fts3Expr *pRight; /* Right operand */ - Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */ -}; - -SQLITE_PRIVATE int sqlite3Fts3ExprParse(sqlite3_tokenizer *, char **, int, int, - const char *, int, Fts3Expr **); -SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *); - -/* -** Candidate values for Fts3Query.eType. Note that the order of the first -** four values is in order of precedence when parsing expressions. For -** example, the following: -** -** "a OR b AND c NOT d NEAR e" -** -** is equivalent to: -** -** "a OR (b AND (c NOT (d NEAR e)))" -*/ -#define FTSQUERY_NEAR 1 -#define FTSQUERY_NOT 2 -#define FTSQUERY_AND 3 -#define FTSQUERY_OR 4 -#define FTSQUERY_PHRASE 5 - -#ifdef SQLITE_TEST -SQLITE_PRIVATE void sqlite3Fts3ExprInitTestInterface(sqlite3 *db); -#endif - -/************** End of fts3_expr.h *******************************************/ -/************** Continuing where we left off in fts3.c ***********************/ -/************** Include fts3_hash.h in the middle of fts3.c ******************/ +/************** Continuing where we left off in fts3Int.h ********************/ +/************** Include fts3_hash.h in the middle of fts3Int.h ***************/ /************** Begin file fts3_hash.h ***************************************/ /* ** 2001 September 22 @@ -97891,8 +98232,8 @@ SQLITE_PRIVATE void sqlite3Fts3ExprInitTestInterface(sqlite3 *db); #define _FTS3_HASH_H_ /* Forward declarations of structures. */ -typedef struct fts3Hash fts3Hash; -typedef struct fts3HashElem fts3HashElem; +typedef struct Fts3Hash Fts3Hash; +typedef struct Fts3HashElem Fts3HashElem; /* A complete hash table is an instance of the following structure. ** The internals of this structure are intended to be opaque -- client @@ -97902,26 +98243,26 @@ typedef struct fts3HashElem fts3HashElem; ** accessing this structure are really macros, so we can't really make ** this structure opaque. */ -struct fts3Hash { +struct Fts3Hash { char keyClass; /* HASH_INT, _POINTER, _STRING, _BINARY */ char copyKey; /* True if copy of key made on insert */ int count; /* Number of entries in this table */ - fts3HashElem *first; /* The first element of the array */ + Fts3HashElem *first; /* The first element of the array */ int htsize; /* Number of buckets in the hash table */ struct _fts3ht { /* the hash table */ int count; /* Number of entries with this hash */ - fts3HashElem *chain; /* Pointer to first entry with this hash */ + Fts3HashElem *chain; /* Pointer to first entry with this hash */ } *ht; }; -/* Each element in the hash table is an instance of the following +/* Each element in the hash table is an instance of the following ** structure. All elements are stored on a single doubly-linked list. ** ** Again, this structure is intended to be opaque, but it can't really ** be opaque because it is used by macros. */ -struct fts3HashElem { - fts3HashElem *next, *prev; /* Next and previous elements in the table */ +struct Fts3HashElem { + Fts3HashElem *next, *prev; /* Next and previous elements in the table */ void *data; /* Data associated with this element */ void *pKey; int nKey; /* Key associated with this element */ }; @@ -97933,10 +98274,10 @@ struct fts3HashElem { ** (including the null-terminator, if any). Case ** is respected in comparisons. ** -** FTS3_HASH_BINARY pKey points to binary data nKey bytes long. +** FTS3_HASH_BINARY pKey points to binary data nKey bytes long. ** memcmp() is used to compare keys. ** -** A copy of the key is made if the copyKey parameter to fts3HashInit is 1. +** A copy of the key is made if the copyKey parameter to fts3HashInit is 1. */ #define FTS3_HASH_STRING 1 #define FTS3_HASH_BINARY 2 @@ -97944,10 +98285,10 @@ struct fts3HashElem { /* ** Access routines. To delete, insert a NULL pointer. */ -SQLITE_PRIVATE void sqlite3Fts3HashInit(fts3Hash*, int keytype, int copyKey); -SQLITE_PRIVATE void *sqlite3Fts3HashInsert(fts3Hash*, const void *pKey, int nKey, void *pData); -SQLITE_PRIVATE void *sqlite3Fts3HashFind(const fts3Hash*, const void *pKey, int nKey); -SQLITE_PRIVATE void sqlite3Fts3HashClear(fts3Hash*); +SQLITE_PRIVATE void sqlite3Fts3HashInit(Fts3Hash *pNew, char keyClass, char copyKey); +SQLITE_PRIVATE void *sqlite3Fts3HashInsert(Fts3Hash*, const void *pKey, int nKey, void *pData); +SQLITE_PRIVATE void *sqlite3Fts3HashFind(const Fts3Hash*, const void *pKey, int nKey); +SQLITE_PRIVATE void sqlite3Fts3HashClear(Fts3Hash*); /* ** Shorthand for the functions above @@ -97961,8 +98302,8 @@ SQLITE_PRIVATE void sqlite3Fts3HashClear(fts3Hash*); ** Macros for looping over all elements of a hash table. The idiom is ** like this: ** -** fts3Hash h; -** fts3HashElem *p; +** Fts3Hash h; +** Fts3HashElem *p; ** ... ** for(p=fts3HashFirst(&h); p; p=fts3HashNext(p)){ ** SomeStructure *pData = fts3HashData(p); @@ -97983,12 +98324,273 @@ SQLITE_PRIVATE void sqlite3Fts3HashClear(fts3Hash*); #endif /* _FTS3_HASH_H_ */ /************** End of fts3_hash.h *******************************************/ +/************** Continuing where we left off in fts3Int.h ********************/ + +/* +** This constant controls how often segments are merged. Once there are +** FTS3_MERGE_COUNT segments of level N, they are merged into a single +** segment of level N+1. +*/ +#define FTS3_MERGE_COUNT 16 + +/* +** This is the maximum amount of data (in bytes) to store in the +** Fts3Table.pendingTerms hash table. Normally, the hash table is +** populated as documents are inserted/updated/deleted in a transaction +** and used to create a new segment when the transaction is committed. +** However if this limit is reached midway through a transaction, a new +** segment is created and the hash table cleared immediately. +*/ +#define FTS3_MAX_PENDING_DATA (1*1024*1024) + +/* +** Macro to return the number of elements in an array. SQLite has a +** similar macro called ArraySize(). Use a different name to avoid +** a collision when building an amalgamation with built-in FTS3. +*/ +#define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0]))) + +/* +** Maximum length of a varint encoded integer. The varint format is different +** from that used by SQLite, so the maximum length is 10, not 9. +*/ +#define FTS3_VARINT_MAX 10 + +/* +** This section provides definitions to allow the +** FTS3 extension to be compiled outside of the +** amalgamation. +*/ +#ifndef SQLITE_AMALGAMATION +/* +** Macros indicating that conditional expressions are always true or +** false. +*/ +# define ALWAYS(x) (x) +# define NEVER(X) (x) +/* +** Internal types used by SQLite. +*/ +typedef unsigned char u8; /* 1-byte (or larger) unsigned integer */ +typedef short int i16; /* 2-byte (or larger) signed integer */ +/* +** Macro used to suppress compiler warnings for unused parameters. +*/ +#define UNUSED_PARAMETER(x) (void)(x) +#endif + +typedef struct Fts3Table Fts3Table; +typedef struct Fts3Cursor Fts3Cursor; +typedef struct Fts3Expr Fts3Expr; +typedef struct Fts3Phrase Fts3Phrase; +typedef struct Fts3SegReader Fts3SegReader; +typedef struct Fts3SegFilter Fts3SegFilter; + +/* +** A connection to a fulltext index is an instance of the following +** structure. The xCreate and xConnect methods create an instance +** of this structure and xDestroy and xDisconnect free that instance. +** All other methods receive a pointer to the structure as one of their +** arguments. +*/ +struct Fts3Table { + sqlite3_vtab base; /* Base class used by SQLite core */ + sqlite3 *db; /* The database connection */ + const char *zDb; /* logical database name */ + const char *zName; /* virtual table name */ + int nColumn; /* number of named columns in virtual table */ + char **azColumn; /* column names. malloced */ + sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */ + + /* Precompiled statements used by the implementation. Each of these + ** statements is run and reset within a single virtual table API call. + */ + sqlite3_stmt *aStmt[18]; + + /* Pointer to string containing the SQL: + ** + ** "SELECT block FROM %_segments WHERE blockid BETWEEN ? AND ? + ** ORDER BY blockid" + */ + char *zSelectLeaves; + int nLeavesStmt; /* Valid statements in aLeavesStmt */ + int nLeavesTotal; /* Total number of prepared leaves stmts */ + int nLeavesAlloc; /* Allocated size of aLeavesStmt */ + sqlite3_stmt **aLeavesStmt; /* Array of prepared zSelectLeaves stmts */ + + int nNodeSize; /* Soft limit for node size */ + + /* The following hash table is used to buffer pending index updates during + ** transactions. Variable nPendingData estimates the memory size of the + ** pending data, including hash table overhead, but not malloc overhead. + ** When nPendingData exceeds FTS3_MAX_PENDING_DATA, the buffer is flushed + ** automatically. Variable iPrevDocid is the docid of the most recently + ** inserted record. + */ + int nPendingData; + sqlite_int64 iPrevDocid; + Fts3Hash pendingTerms; +}; + +/* +** When the core wants to read from the virtual table, it creates a +** virtual table cursor (an instance of the following structure) using +** the xOpen method. Cursors are destroyed using the xClose method. +*/ +struct Fts3Cursor { + sqlite3_vtab_cursor base; /* Base class used by SQLite core */ + i16 eSearch; /* Search strategy (see below) */ + u8 isEof; /* True if at End Of Results */ + u8 isRequireSeek; /* True if must seek pStmt to %_content row */ + sqlite3_stmt *pStmt; /* Prepared statement in use by the cursor */ + Fts3Expr *pExpr; /* Parsed MATCH query string */ + sqlite3_int64 iPrevId; /* Previous id read from aDoclist */ + char *pNextId; /* Pointer into the body of aDoclist */ + char *aDoclist; /* List of docids for full-text queries */ + int nDoclist; /* Size of buffer at aDoclist */ +}; + +/* +** The Fts3Cursor.eSearch member is always set to one of the following. +** Actualy, Fts3Cursor.eSearch can be greater than or equal to +** FTS3_FULLTEXT_SEARCH. If so, then Fts3Cursor.eSearch - 2 is the index +** of the column to be searched. For example, in +** +** CREATE VIRTUAL TABLE ex1 USING fts3(a,b,c,d); +** SELECT docid FROM ex1 WHERE b MATCH 'one two three'; +** +** Because the LHS of the MATCH operator is 2nd column "b", +** Fts3Cursor.eSearch will be set to FTS3_FULLTEXT_SEARCH+1. (+0 for a, +** +1 for b, +2 for c, +3 for d.) If the LHS of MATCH were "ex1" +** indicating that all columns should be searched, +** then eSearch would be set to FTS3_FULLTEXT_SEARCH+4. +*/ +#define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */ +#define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */ +#define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */ + +/* +** A "phrase" is a sequence of one or more tokens that must match in +** sequence. A single token is the base case and the most common case. +** For a sequence of tokens contained in "...", nToken will be the number +** of tokens in the string. +*/ +struct Fts3Phrase { + int nToken; /* Number of tokens in the phrase */ + int iColumn; /* Index of column this phrase must match */ + int isNot; /* Phrase prefixed by unary not (-) operator */ + struct PhraseToken { + char *z; /* Text of the token */ + int n; /* Number of bytes in buffer pointed to by z */ + int isPrefix; /* True if token ends in with a "*" character */ + } aToken[1]; /* One entry for each token in the phrase */ +}; + +/* +** A tree of these objects forms the RHS of a MATCH operator. +*/ +struct Fts3Expr { + int eType; /* One of the FTSQUERY_XXX values defined below */ + int nNear; /* Valid if eType==FTSQUERY_NEAR */ + Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */ + Fts3Expr *pLeft; /* Left operand */ + Fts3Expr *pRight; /* Right operand */ + Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */ +}; + +/* +** Candidate values for Fts3Query.eType. Note that the order of the first +** four values is in order of precedence when parsing expressions. For +** example, the following: +** +** "a OR b AND c NOT d NEAR e" +** +** is equivalent to: +** +** "a OR (b AND (c NOT (d NEAR e)))" +*/ +#define FTSQUERY_NEAR 1 +#define FTSQUERY_NOT 2 +#define FTSQUERY_AND 3 +#define FTSQUERY_OR 4 +#define FTSQUERY_PHRASE 5 + + +/* fts3_init.c */ +SQLITE_PRIVATE int sqlite3Fts3DeleteVtab(int, sqlite3_vtab *); +SQLITE_PRIVATE int sqlite3Fts3InitVtab(int, sqlite3*, void*, int, const char*const*, + sqlite3_vtab **, char **); + +/* fts3_write.c */ +SQLITE_PRIVATE int sqlite3Fts3UpdateMethod(sqlite3_vtab*,int,sqlite3_value**,sqlite3_int64*); +SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *); +SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *); +SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *); +SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(Fts3Table *,int, sqlite3_int64, + sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**); +SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3Table *, Fts3SegReader *); +SQLITE_PRIVATE int sqlite3Fts3SegReaderIterate( + Fts3Table *, Fts3SegReader **, int, Fts3SegFilter *, + int (*)(Fts3Table *, void *, char *, int, char *, int), void * +); +SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char const**, int*); +SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, sqlite3_stmt **); + +/* Flags allowed as part of the 4th argument to SegmentReaderIterate() */ +#define FTS3_SEGMENT_REQUIRE_POS 0x00000001 +#define FTS3_SEGMENT_IGNORE_EMPTY 0x00000002 +#define FTS3_SEGMENT_COLUMN_FILTER 0x00000004 +#define FTS3_SEGMENT_PREFIX 0x00000008 + +/* Type passed as 4th argument to SegmentReaderIterate() */ +struct Fts3SegFilter { + const char *zTerm; + int nTerm; + int iCol; + int flags; +}; + +/* fts3.c */ +SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *, sqlite3_int64); +SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *, sqlite_int64 *); +SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *, int *); +SQLITE_PRIVATE int sqlite3Fts3VarintLen(sqlite3_uint64); +SQLITE_PRIVATE void sqlite3Fts3Dequote(char *); + +/* fts3_tokenizer.c */ +SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *, int *); +SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *); +SQLITE_PRIVATE int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, + const char *, sqlite3_tokenizer **, const char **, char ** +); + +/* fts3_snippet.c */ +SQLITE_PRIVATE void sqlite3Fts3Offsets(sqlite3_context*, Fts3Cursor*); +SQLITE_PRIVATE void sqlite3Fts3Snippet(sqlite3_context*, Fts3Cursor*, + const char *, const char *, const char * +); + +/* fts3_expr.c */ +SQLITE_PRIVATE int sqlite3Fts3ExprParse(sqlite3_tokenizer *, + char **, int, int, const char *, int, Fts3Expr ** +); +SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *); +#ifdef SQLITE_TEST +SQLITE_PRIVATE void sqlite3Fts3ExprInitTestInterface(sqlite3 *db); +#endif + +#endif /* _FTSINT_H */ + +/************** End of fts3Int.h *********************************************/ /************** Continuing where we left off in fts3.c ***********************/ -#ifndef SQLITE_CORE + + +#ifndef SQLITE_CORE SQLITE_EXTENSION_INIT1 #endif + /* TODO(shess) MAN, this thing needs some refactoring. At minimum, it ** would be nice to order the file better, perhaps something along the ** lines of: @@ -98008,25 +98610,6 @@ SQLITE_PRIVATE void sqlite3Fts3HashClear(fts3Hash*); # define FTSTRACE(A) #endif -/* It is not safe to call isspace(), tolower(), or isalnum() on -** hi-bit-set characters. This is the same solution used in the -** tokenizer. -*/ -/* TODO(shess) The snippet-generation code should be using the -** tokenizer-generated tokens rather than doing its own local -** tokenization. -*/ -/* TODO(shess) Is __isascii() a portable version of (c&0x80)==0? */ -static int safe_isspace(char c){ - return (c&0x80)==0 ? isspace(c) : 0; -} -static int safe_tolower(char c){ - return (c&0x80)==0 ? tolower(c) : c; -} -static int safe_isalnum(char c){ - return (c&0x80)==0 ? isalnum(c) : 0; -} - typedef enum DocListType { DL_DOCIDS, /* docids only */ DL_POSITIONS, /* docids + positions */ @@ -98052,11 +98635,6 @@ enum { POS_BASE }; -/* MERGE_COUNT controls how often we merge segments (see comment at -** top of file). -*/ -#define MERGE_COUNT 16 - /* utility functions */ /* CLEAR() and SCRAMBLE() abstract memset() on a pointer to a single @@ -98075,13 +98653,12 @@ enum { # define SCRAMBLE(b) #endif -/* We may need up to VARINT_MAX bytes to store an encoded 64-bit integer. */ -#define VARINT_MAX 10 - -/* Write a 64-bit variable-length integer to memory starting at p[0]. - * The length of data written will be between 1 and VARINT_MAX bytes. - * The number of bytes written is returned. */ -static int fts3PutVarint(char *p, sqlite_int64 v){ +/* +** Write a 64-bit variable-length integer to memory starting at p[0]. +** The length of data written will be between 1 and FTS3_VARINT_MAX bytes. +** The number of bytes written is returned. +*/ +SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *p, sqlite_int64 v){ unsigned char *q = (unsigned char *) p; sqlite_uint64 vu = v; do{ @@ -98089,20 +98666,22 @@ static int fts3PutVarint(char *p, sqlite_int64 v){ vu >>= 7; }while( vu!=0 ); q[-1] &= 0x7f; /* turn off high bit in final byte */ - assert( q - (unsigned char *)p <= VARINT_MAX ); + assert( q - (unsigned char *)p <= FTS3_VARINT_MAX ); return (int) (q - (unsigned char *)p); } -/* Read a 64-bit variable-length integer from memory starting at p[0]. - * Return the number of bytes read, or 0 on error. - * The value is stored in *v. */ -static int fts3GetVarint(const char *p, sqlite_int64 *v){ +/* +** Read a 64-bit variable-length integer from memory starting at p[0]. +** Return the number of bytes read, or 0 on error. +** The value is stored in *v. +*/ +SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *p, sqlite_int64 *v){ const unsigned char *q = (const unsigned char *) p; sqlite_uint64 x = 0, y = 1; while( (*q & 0x80) == 0x80 ){ x += y * (*q++ & 0x7f); y <<= 7; - if( q - (unsigned char *)p >= VARINT_MAX ){ /* bad data */ + if( q - (unsigned char *)p >= FTS3_VARINT_MAX ){ /* bad data */ assert( 0 ); return 0; } @@ -98112,2184 +98691,29 @@ static int fts3GetVarint(const char *p, sqlite_int64 *v){ return (int) (q - (unsigned char *)p); } -static int fts3GetVarint32(const char *p, int *pi){ +/* +** Similar to sqlite3Fts3GetVarint(), except that the output is truncated to a +** 32-bit integer before it is returned. +*/ +SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *p, int *pi){ sqlite_int64 i; - int ret = fts3GetVarint(p, &i); + int ret = sqlite3Fts3GetVarint(p, &i); *pi = (int) i; assert( *pi==i ); return ret; } -/*******************************************************************/ -/* DataBuffer is used to collect data into a buffer in piecemeal -** fashion. It implements the usual distinction between amount of -** data currently stored (nData) and buffer capacity (nCapacity). -** -** dataBufferInit - create a buffer with given initial capacity. -** dataBufferReset - forget buffer's data, retaining capacity. -** dataBufferDestroy - free buffer's data. -** dataBufferSwap - swap contents of two buffers. -** dataBufferExpand - expand capacity without adding data. -** dataBufferAppend - append data. -** dataBufferAppend2 - append two pieces of data at once. -** dataBufferReplace - replace buffer's data. -*/ -typedef struct DataBuffer { - char *pData; /* Pointer to malloc'ed buffer. */ - int nCapacity; /* Size of pData buffer. */ - int nData; /* End of data loaded into pData. */ -} DataBuffer; - -static void dataBufferInit(DataBuffer *pBuffer, int nCapacity){ - assert( nCapacity>=0 ); - pBuffer->nData = 0; - pBuffer->nCapacity = nCapacity; - pBuffer->pData = nCapacity==0 ? NULL : sqlite3_malloc(nCapacity); -} -static void dataBufferReset(DataBuffer *pBuffer){ - pBuffer->nData = 0; -} -static void dataBufferDestroy(DataBuffer *pBuffer){ - if( pBuffer->pData!=NULL ) sqlite3_free(pBuffer->pData); - SCRAMBLE(pBuffer); -} -static void dataBufferSwap(DataBuffer *pBuffer1, DataBuffer *pBuffer2){ - DataBuffer tmp = *pBuffer1; - *pBuffer1 = *pBuffer2; - *pBuffer2 = tmp; -} -static void dataBufferExpand(DataBuffer *pBuffer, int nAddCapacity){ - assert( nAddCapacity>0 ); - /* TODO(shess) Consider expanding more aggressively. Note that the - ** underlying malloc implementation may take care of such things for - ** us already. - */ - if( pBuffer->nData+nAddCapacity>pBuffer->nCapacity ){ - pBuffer->nCapacity = pBuffer->nData+nAddCapacity; - pBuffer->pData = sqlite3_realloc(pBuffer->pData, pBuffer->nCapacity); - } -} -static void dataBufferAppend(DataBuffer *pBuffer, - const char *pSource, int nSource){ - assert( nSource>0 && pSource!=NULL ); - dataBufferExpand(pBuffer, nSource); - memcpy(pBuffer->pData+pBuffer->nData, pSource, nSource); - pBuffer->nData += nSource; -} -static void dataBufferAppend2(DataBuffer *pBuffer, - const char *pSource1, int nSource1, - const char *pSource2, int nSource2){ - assert( nSource1>0 && pSource1!=NULL ); - assert( nSource2>0 && pSource2!=NULL ); - dataBufferExpand(pBuffer, nSource1+nSource2); - memcpy(pBuffer->pData+pBuffer->nData, pSource1, nSource1); - memcpy(pBuffer->pData+pBuffer->nData+nSource1, pSource2, nSource2); - pBuffer->nData += nSource1+nSource2; -} -static void dataBufferReplace(DataBuffer *pBuffer, - const char *pSource, int nSource){ - dataBufferReset(pBuffer); - dataBufferAppend(pBuffer, pSource, nSource); -} - -/* StringBuffer is a null-terminated version of DataBuffer. */ -typedef struct StringBuffer { - DataBuffer b; /* Includes null terminator. */ -} StringBuffer; - -static void initStringBuffer(StringBuffer *sb){ - dataBufferInit(&sb->b, 100); - dataBufferReplace(&sb->b, "", 1); -} -static int stringBufferLength(StringBuffer *sb){ - return sb->b.nData-1; -} -static char *stringBufferData(StringBuffer *sb){ - return sb->b.pData; -} -static void stringBufferDestroy(StringBuffer *sb){ - dataBufferDestroy(&sb->b); -} - -static void nappend(StringBuffer *sb, const char *zFrom, int nFrom){ - assert( sb->b.nData>0 ); - if( nFrom>0 ){ - sb->b.nData--; - dataBufferAppend2(&sb->b, zFrom, nFrom, "", 1); - } -} -static void append(StringBuffer *sb, const char *zFrom){ - nappend(sb, zFrom, strlen(zFrom)); -} - -/* Append a list of strings separated by commas. */ -static void appendList(StringBuffer *sb, int nString, char **azString){ - int i; - for(i=0; i0 ) append(sb, ", "); - append(sb, azString[i]); - } -} - -static int endsInWhiteSpace(StringBuffer *p){ - return stringBufferLength(p)>0 && - safe_isspace(stringBufferData(p)[stringBufferLength(p)-1]); -} - -/* If the StringBuffer ends in something other than white space, add a -** single space character to the end. -*/ -static void appendWhiteSpace(StringBuffer *p){ - if( stringBufferLength(p)==0 ) return; - if( !endsInWhiteSpace(p) ) append(p, " "); -} - -/* Remove white space from the end of the StringBuffer */ -static void trimWhiteSpace(StringBuffer *p){ - while( endsInWhiteSpace(p) ){ - p->b.pData[--p->b.nData-1] = '\0'; - } -} - -/*******************************************************************/ -/* DLReader is used to read document elements from a doclist. The -** current docid is cached, so dlrDocid() is fast. DLReader does not -** own the doclist buffer. -** -** dlrAtEnd - true if there's no more data to read. -** dlrDocid - docid of current document. -** dlrDocData - doclist data for current document (including docid). -** dlrDocDataBytes - length of same. -** dlrAllDataBytes - length of all remaining data. -** dlrPosData - position data for current document. -** dlrPosDataLen - length of pos data for current document (incl POS_END). -** dlrStep - step to current document. -** dlrInit - initial for doclist of given type against given data. -** dlrDestroy - clean up. -** -** Expected usage is something like: -** -** DLReader reader; -** dlrInit(&reader, pData, nData); -** while( !dlrAtEnd(&reader) ){ -** // calls to dlrDocid() and kin. -** dlrStep(&reader); -** } -** dlrDestroy(&reader); -*/ -typedef struct DLReader { - DocListType iType; - const char *pData; - int nData; - - sqlite_int64 iDocid; - int nElement; -} DLReader; - -static int dlrAtEnd(DLReader *pReader){ - assert( pReader->nData>=0 ); - return pReader->nData==0; -} -static sqlite_int64 dlrDocid(DLReader *pReader){ - assert( !dlrAtEnd(pReader) ); - return pReader->iDocid; -} -static const char *dlrDocData(DLReader *pReader){ - assert( !dlrAtEnd(pReader) ); - return pReader->pData; -} -static int dlrDocDataBytes(DLReader *pReader){ - assert( !dlrAtEnd(pReader) ); - return pReader->nElement; -} -static int dlrAllDataBytes(DLReader *pReader){ - assert( !dlrAtEnd(pReader) ); - return pReader->nData; -} -/* TODO(shess) Consider adding a field to track iDocid varint length -** to make these two functions faster. This might matter (a tiny bit) -** for queries. -*/ -static const char *dlrPosData(DLReader *pReader){ - sqlite_int64 iDummy; - int n = fts3GetVarint(pReader->pData, &iDummy); - assert( !dlrAtEnd(pReader) ); - return pReader->pData+n; -} -static int dlrPosDataLen(DLReader *pReader){ - sqlite_int64 iDummy; - int n = fts3GetVarint(pReader->pData, &iDummy); - assert( !dlrAtEnd(pReader) ); - return pReader->nElement-n; -} -static void dlrStep(DLReader *pReader){ - assert( !dlrAtEnd(pReader) ); - - /* Skip past current doclist element. */ - assert( pReader->nElement<=pReader->nData ); - pReader->pData += pReader->nElement; - pReader->nData -= pReader->nElement; - - /* If there is more data, read the next doclist element. */ - if( pReader->nData!=0 ){ - sqlite_int64 iDocidDelta; - int iDummy, n = fts3GetVarint(pReader->pData, &iDocidDelta); - pReader->iDocid += iDocidDelta; - if( pReader->iType>=DL_POSITIONS ){ - assert( nnData ); - while( 1 ){ - n += fts3GetVarint32(pReader->pData+n, &iDummy); - assert( n<=pReader->nData ); - if( iDummy==POS_END ) break; - if( iDummy==POS_COLUMN ){ - n += fts3GetVarint32(pReader->pData+n, &iDummy); - assert( nnData ); - }else if( pReader->iType==DL_POSITIONS_OFFSETS ){ - n += fts3GetVarint32(pReader->pData+n, &iDummy); - n += fts3GetVarint32(pReader->pData+n, &iDummy); - assert( nnData ); - } - } - } - pReader->nElement = n; - assert( pReader->nElement<=pReader->nData ); - } -} -static void dlrInit(DLReader *pReader, DocListType iType, - const char *pData, int nData){ - assert( pData!=NULL && nData!=0 ); - pReader->iType = iType; - pReader->pData = pData; - pReader->nData = nData; - pReader->nElement = 0; - pReader->iDocid = 0; - - /* Load the first element's data. There must be a first element. */ - dlrStep(pReader); -} -static void dlrDestroy(DLReader *pReader){ - SCRAMBLE(pReader); -} - -#ifndef NDEBUG -/* Verify that the doclist can be validly decoded. Also returns the -** last docid found because it is convenient in other assertions for -** DLWriter. -*/ -static void docListValidate(DocListType iType, const char *pData, int nData, - sqlite_int64 *pLastDocid){ - sqlite_int64 iPrevDocid = 0; - assert( nData>0 ); - assert( pData!=0 ); - assert( pData+nData>pData ); - while( nData!=0 ){ - sqlite_int64 iDocidDelta; - int n = fts3GetVarint(pData, &iDocidDelta); - iPrevDocid += iDocidDelta; - if( iType>DL_DOCIDS ){ - int iDummy; - while( 1 ){ - n += fts3GetVarint32(pData+n, &iDummy); - if( iDummy==POS_END ) break; - if( iDummy==POS_COLUMN ){ - n += fts3GetVarint32(pData+n, &iDummy); - }else if( iType>DL_POSITIONS ){ - n += fts3GetVarint32(pData+n, &iDummy); - n += fts3GetVarint32(pData+n, &iDummy); - } - assert( n<=nData ); - } - } - assert( n<=nData ); - pData += n; - nData -= n; - } - if( pLastDocid ) *pLastDocid = iPrevDocid; -} -#define ASSERT_VALID_DOCLIST(i, p, n, o) docListValidate(i, p, n, o) -#else -#define ASSERT_VALID_DOCLIST(i, p, n, o) assert( 1 ) -#endif - -/*******************************************************************/ -/* DLWriter is used to write doclist data to a DataBuffer. DLWriter -** always appends to the buffer and does not own it. -** -** dlwInit - initialize to write a given type doclistto a buffer. -** dlwDestroy - clear the writer's memory. Does not free buffer. -** dlwAppend - append raw doclist data to buffer. -** dlwCopy - copy next doclist from reader to writer. -** dlwAdd - construct doclist element and append to buffer. -** Only apply dlwAdd() to DL_DOCIDS doclists (else use PLWriter). -*/ -typedef struct DLWriter { - DocListType iType; - DataBuffer *b; - sqlite_int64 iPrevDocid; -#ifndef NDEBUG - int has_iPrevDocid; -#endif -} DLWriter; - -static void dlwInit(DLWriter *pWriter, DocListType iType, DataBuffer *b){ - pWriter->b = b; - pWriter->iType = iType; - pWriter->iPrevDocid = 0; -#ifndef NDEBUG - pWriter->has_iPrevDocid = 0; -#endif -} -static void dlwDestroy(DLWriter *pWriter){ - SCRAMBLE(pWriter); -} -/* iFirstDocid is the first docid in the doclist in pData. It is -** needed because pData may point within a larger doclist, in which -** case the first item would be delta-encoded. -** -** iLastDocid is the final docid in the doclist in pData. It is -** needed to create the new iPrevDocid for future delta-encoding. The -** code could decode the passed doclist to recreate iLastDocid, but -** the only current user (docListMerge) already has decoded this -** information. -*/ -/* TODO(shess) This has become just a helper for docListMerge. -** Consider a refactor to make this cleaner. -*/ -static void dlwAppend(DLWriter *pWriter, - const char *pData, int nData, - sqlite_int64 iFirstDocid, sqlite_int64 iLastDocid){ - sqlite_int64 iDocid = 0; - char c[VARINT_MAX]; - int nFirstOld, nFirstNew; /* Old and new varint len of first docid. */ -#ifndef NDEBUG - sqlite_int64 iLastDocidDelta; -#endif - - /* Recode the initial docid as delta from iPrevDocid. */ - nFirstOld = fts3GetVarint(pData, &iDocid); - assert( nFirstOldiType==DL_DOCIDS) ); - nFirstNew = fts3PutVarint(c, iFirstDocid-pWriter->iPrevDocid); - - /* Verify that the incoming doclist is valid AND that it ends with - ** the expected docid. This is essential because we'll trust this - ** docid in future delta-encoding. - */ - ASSERT_VALID_DOCLIST(pWriter->iType, pData, nData, &iLastDocidDelta); - assert( iLastDocid==iFirstDocid-iDocid+iLastDocidDelta ); - - /* Append recoded initial docid and everything else. Rest of docids - ** should have been delta-encoded from previous initial docid. - */ - if( nFirstOldb, c, nFirstNew, - pData+nFirstOld, nData-nFirstOld); - }else{ - dataBufferAppend(pWriter->b, c, nFirstNew); - } - pWriter->iPrevDocid = iLastDocid; -} -static void dlwCopy(DLWriter *pWriter, DLReader *pReader){ - dlwAppend(pWriter, dlrDocData(pReader), dlrDocDataBytes(pReader), - dlrDocid(pReader), dlrDocid(pReader)); -} -static void dlwAdd(DLWriter *pWriter, sqlite_int64 iDocid){ - char c[VARINT_MAX]; - int n = fts3PutVarint(c, iDocid-pWriter->iPrevDocid); - - /* Docids must ascend. */ - assert( !pWriter->has_iPrevDocid || iDocid>pWriter->iPrevDocid ); - assert( pWriter->iType==DL_DOCIDS ); - - dataBufferAppend(pWriter->b, c, n); - pWriter->iPrevDocid = iDocid; -#ifndef NDEBUG - pWriter->has_iPrevDocid = 1; -#endif -} - -/*******************************************************************/ -/* PLReader is used to read data from a document's position list. As -** the caller steps through the list, data is cached so that varints -** only need to be decoded once. -** -** plrInit, plrDestroy - create/destroy a reader. -** plrColumn, plrPosition, plrStartOffset, plrEndOffset - accessors -** plrAtEnd - at end of stream, only call plrDestroy once true. -** plrStep - step to the next element. -*/ -typedef struct PLReader { - /* These refer to the next position's data. nData will reach 0 when - ** reading the last position, so plrStep() signals EOF by setting - ** pData to NULL. - */ - const char *pData; - int nData; - - DocListType iType; - int iColumn; /* the last column read */ - int iPosition; /* the last position read */ - int iStartOffset; /* the last start offset read */ - int iEndOffset; /* the last end offset read */ -} PLReader; - -static int plrAtEnd(PLReader *pReader){ - return pReader->pData==NULL; -} -static int plrColumn(PLReader *pReader){ - assert( !plrAtEnd(pReader) ); - return pReader->iColumn; -} -static int plrPosition(PLReader *pReader){ - assert( !plrAtEnd(pReader) ); - return pReader->iPosition; -} -static int plrStartOffset(PLReader *pReader){ - assert( !plrAtEnd(pReader) ); - return pReader->iStartOffset; -} -static int plrEndOffset(PLReader *pReader){ - assert( !plrAtEnd(pReader) ); - return pReader->iEndOffset; -} -static void plrStep(PLReader *pReader){ - int i, n; - - assert( !plrAtEnd(pReader) ); - - if( pReader->nData==0 ){ - pReader->pData = NULL; - return; - } - - n = fts3GetVarint32(pReader->pData, &i); - if( i==POS_COLUMN ){ - n += fts3GetVarint32(pReader->pData+n, &pReader->iColumn); - pReader->iPosition = 0; - pReader->iStartOffset = 0; - n += fts3GetVarint32(pReader->pData+n, &i); - } - /* Should never see adjacent column changes. */ - assert( i!=POS_COLUMN ); - - if( i==POS_END ){ - pReader->nData = 0; - pReader->pData = NULL; - return; - } - - pReader->iPosition += i-POS_BASE; - if( pReader->iType==DL_POSITIONS_OFFSETS ){ - n += fts3GetVarint32(pReader->pData+n, &i); - pReader->iStartOffset += i; - n += fts3GetVarint32(pReader->pData+n, &i); - pReader->iEndOffset = pReader->iStartOffset+i; - } - assert( n<=pReader->nData ); - pReader->pData += n; - pReader->nData -= n; -} - -static void plrInit(PLReader *pReader, DLReader *pDLReader){ - pReader->pData = dlrPosData(pDLReader); - pReader->nData = dlrPosDataLen(pDLReader); - pReader->iType = pDLReader->iType; - pReader->iColumn = 0; - pReader->iPosition = 0; - pReader->iStartOffset = 0; - pReader->iEndOffset = 0; - plrStep(pReader); -} -static void plrDestroy(PLReader *pReader){ - SCRAMBLE(pReader); -} - -/*******************************************************************/ -/* PLWriter is used in constructing a document's position list. As a -** convenience, if iType is DL_DOCIDS, PLWriter becomes a no-op. -** PLWriter writes to the associated DLWriter's buffer. -** -** plwInit - init for writing a document's poslist. -** plwDestroy - clear a writer. -** plwAdd - append position and offset information. -** plwCopy - copy next position's data from reader to writer. -** plwTerminate - add any necessary doclist terminator. -** -** Calling plwAdd() after plwTerminate() may result in a corrupt -** doclist. -*/ -/* TODO(shess) Until we've written the second item, we can cache the -** first item's information. Then we'd have three states: -** -** - initialized with docid, no positions. -** - docid and one position. -** - docid and multiple positions. -** -** Only the last state needs to actually write to dlw->b, which would -** be an improvement in the DLCollector case. -*/ -typedef struct PLWriter { - DLWriter *dlw; - - int iColumn; /* the last column written */ - int iPos; /* the last position written */ - int iOffset; /* the last start offset written */ -} PLWriter; - -/* TODO(shess) In the case where the parent is reading these values -** from a PLReader, we could optimize to a copy if that PLReader has -** the same type as pWriter. -*/ -static void plwAdd(PLWriter *pWriter, int iColumn, int iPos, - int iStartOffset, int iEndOffset){ - /* Worst-case space for POS_COLUMN, iColumn, iPosDelta, - ** iStartOffsetDelta, and iEndOffsetDelta. - */ - char c[5*VARINT_MAX]; - int n = 0; - - /* Ban plwAdd() after plwTerminate(). */ - assert( pWriter->iPos!=-1 ); - - if( pWriter->dlw->iType==DL_DOCIDS ) return; - - if( iColumn!=pWriter->iColumn ){ - n += fts3PutVarint(c+n, POS_COLUMN); - n += fts3PutVarint(c+n, iColumn); - pWriter->iColumn = iColumn; - pWriter->iPos = 0; - pWriter->iOffset = 0; - } - assert( iPos>=pWriter->iPos ); - n += fts3PutVarint(c+n, POS_BASE+(iPos-pWriter->iPos)); - pWriter->iPos = iPos; - if( pWriter->dlw->iType==DL_POSITIONS_OFFSETS ){ - assert( iStartOffset>=pWriter->iOffset ); - n += fts3PutVarint(c+n, iStartOffset-pWriter->iOffset); - pWriter->iOffset = iStartOffset; - assert( iEndOffset>=iStartOffset ); - n += fts3PutVarint(c+n, iEndOffset-iStartOffset); - } - dataBufferAppend(pWriter->dlw->b, c, n); -} -static void plwCopy(PLWriter *pWriter, PLReader *pReader){ - plwAdd(pWriter, plrColumn(pReader), plrPosition(pReader), - plrStartOffset(pReader), plrEndOffset(pReader)); -} -static void plwInit(PLWriter *pWriter, DLWriter *dlw, sqlite_int64 iDocid){ - char c[VARINT_MAX]; - int n; - - pWriter->dlw = dlw; - - /* Docids must ascend. */ - assert( !pWriter->dlw->has_iPrevDocid || iDocid>pWriter->dlw->iPrevDocid ); - n = fts3PutVarint(c, iDocid-pWriter->dlw->iPrevDocid); - dataBufferAppend(pWriter->dlw->b, c, n); - pWriter->dlw->iPrevDocid = iDocid; -#ifndef NDEBUG - pWriter->dlw->has_iPrevDocid = 1; -#endif - - pWriter->iColumn = 0; - pWriter->iPos = 0; - pWriter->iOffset = 0; -} -/* TODO(shess) Should plwDestroy() also terminate the doclist? But -** then plwDestroy() would no longer be just a destructor, it would -** also be doing work, which isn't consistent with the overall idiom. -** Another option would be for plwAdd() to always append any necessary -** terminator, so that the output is always correct. But that would -** add incremental work to the common case with the only benefit being -** API elegance. Punt for now. -*/ -static void plwTerminate(PLWriter *pWriter){ - if( pWriter->dlw->iType>DL_DOCIDS ){ - char c[VARINT_MAX]; - int n = fts3PutVarint(c, POS_END); - dataBufferAppend(pWriter->dlw->b, c, n); - } -#ifndef NDEBUG - /* Mark as terminated for assert in plwAdd(). */ - pWriter->iPos = -1; -#endif -} -static void plwDestroy(PLWriter *pWriter){ - SCRAMBLE(pWriter); -} - -/*******************************************************************/ -/* DLCollector wraps PLWriter and DLWriter to provide a -** dynamically-allocated doclist area to use during tokenization. -** -** dlcNew - malloc up and initialize a collector. -** dlcDelete - destroy a collector and all contained items. -** dlcAddPos - append position and offset information. -** dlcAddDoclist - add the collected doclist to the given buffer. -** dlcNext - terminate the current document and open another. -*/ -typedef struct DLCollector { - DataBuffer b; - DLWriter dlw; - PLWriter plw; -} DLCollector; - -/* TODO(shess) This could also be done by calling plwTerminate() and -** dataBufferAppend(). I tried that, expecting nominal performance -** differences, but it seemed to pretty reliably be worth 1% to code -** it this way. I suspect it is the incremental malloc overhead (some -** percentage of the plwTerminate() calls will cause a realloc), so -** this might be worth revisiting if the DataBuffer implementation -** changes. -*/ -static void dlcAddDoclist(DLCollector *pCollector, DataBuffer *b){ - if( pCollector->dlw.iType>DL_DOCIDS ){ - char c[VARINT_MAX]; - int n = fts3PutVarint(c, POS_END); - dataBufferAppend2(b, pCollector->b.pData, pCollector->b.nData, c, n); - }else{ - dataBufferAppend(b, pCollector->b.pData, pCollector->b.nData); - } -} -static void dlcNext(DLCollector *pCollector, sqlite_int64 iDocid){ - plwTerminate(&pCollector->plw); - plwDestroy(&pCollector->plw); - plwInit(&pCollector->plw, &pCollector->dlw, iDocid); -} -static void dlcAddPos(DLCollector *pCollector, int iColumn, int iPos, - int iStartOffset, int iEndOffset){ - plwAdd(&pCollector->plw, iColumn, iPos, iStartOffset, iEndOffset); -} - -static DLCollector *dlcNew(sqlite_int64 iDocid, DocListType iType){ - DLCollector *pCollector = sqlite3_malloc(sizeof(DLCollector)); - dataBufferInit(&pCollector->b, 0); - dlwInit(&pCollector->dlw, iType, &pCollector->b); - plwInit(&pCollector->plw, &pCollector->dlw, iDocid); - return pCollector; -} -static void dlcDelete(DLCollector *pCollector){ - plwDestroy(&pCollector->plw); - dlwDestroy(&pCollector->dlw); - dataBufferDestroy(&pCollector->b); - SCRAMBLE(pCollector); - sqlite3_free(pCollector); -} - - -/* Copy the doclist data of iType in pData/nData into *out, trimming -** unnecessary data as we go. Only columns matching iColumn are -** copied, all columns copied if iColumn is -1. Elements with no -** matching columns are dropped. The output is an iOutType doclist. -*/ -/* NOTE(shess) This code is only valid after all doclists are merged. -** If this is run before merges, then doclist items which represent -** deletion will be trimmed, and will thus not effect a deletion -** during the merge. -*/ -static void docListTrim(DocListType iType, const char *pData, int nData, - int iColumn, DocListType iOutType, DataBuffer *out){ - DLReader dlReader; - DLWriter dlWriter; - - assert( iOutType<=iType ); - - dlrInit(&dlReader, iType, pData, nData); - dlwInit(&dlWriter, iOutType, out); - - while( !dlrAtEnd(&dlReader) ){ - PLReader plReader; - PLWriter plWriter; - int match = 0; - - plrInit(&plReader, &dlReader); - - while( !plrAtEnd(&plReader) ){ - if( iColumn==-1 || plrColumn(&plReader)==iColumn ){ - if( !match ){ - plwInit(&plWriter, &dlWriter, dlrDocid(&dlReader)); - match = 1; - } - plwAdd(&plWriter, plrColumn(&plReader), plrPosition(&plReader), - plrStartOffset(&plReader), plrEndOffset(&plReader)); - } - plrStep(&plReader); - } - if( match ){ - plwTerminate(&plWriter); - plwDestroy(&plWriter); - } - - plrDestroy(&plReader); - dlrStep(&dlReader); - } - dlwDestroy(&dlWriter); - dlrDestroy(&dlReader); -} - -/* Used by docListMerge() to keep doclists in the ascending order by -** docid, then ascending order by age (so the newest comes first). -*/ -typedef struct OrderedDLReader { - DLReader *pReader; - - /* TODO(shess) If we assume that docListMerge pReaders is ordered by - ** age (which we do), then we could use pReader comparisons to break - ** ties. - */ - int idx; -} OrderedDLReader; - -/* Order eof to end, then by docid asc, idx desc. */ -static int orderedDLReaderCmp(OrderedDLReader *r1, OrderedDLReader *r2){ - if( dlrAtEnd(r1->pReader) ){ - if( dlrAtEnd(r2->pReader) ) return 0; /* Both atEnd(). */ - return 1; /* Only r1 atEnd(). */ - } - if( dlrAtEnd(r2->pReader) ) return -1; /* Only r2 atEnd(). */ - - if( dlrDocid(r1->pReader)pReader) ) return -1; - if( dlrDocid(r1->pReader)>dlrDocid(r2->pReader) ) return 1; - - /* Descending on idx. */ - return r2->idx-r1->idx; -} - -/* Bubble p[0] to appropriate place in p[1..n-1]. Assumes that -** p[1..n-1] is already sorted. -*/ -/* TODO(shess) Is this frequent enough to warrant a binary search? -** Before implementing that, instrument the code to check. In most -** current usage, I expect that p[0] will be less than p[1] a very -** high proportion of the time. -*/ -static void orderedDLReaderReorder(OrderedDLReader *p, int n){ - while( n>1 && orderedDLReaderCmp(p, p+1)>0 ){ - OrderedDLReader tmp = p[0]; - p[0] = p[1]; - p[1] = tmp; - n--; - p++; - } -} - -/* Given an array of doclist readers, merge their doclist elements -** into out in sorted order (by docid), dropping elements from older -** readers when there is a duplicate docid. pReaders is assumed to be -** ordered by age, oldest first. -*/ -/* TODO(shess) nReaders must be <= MERGE_COUNT. This should probably -** be fixed. -*/ -static void docListMerge(DataBuffer *out, - DLReader *pReaders, int nReaders){ - OrderedDLReader readers[MERGE_COUNT]; - DLWriter writer; - int i, n; - const char *pStart = 0; - int nStart = 0; - sqlite_int64 iFirstDocid = 0, iLastDocid = 0; - - assert( nReaders>0 ); - if( nReaders==1 ){ - dataBufferAppend(out, dlrDocData(pReaders), dlrAllDataBytes(pReaders)); - return; - } - - assert( nReaders<=MERGE_COUNT ); - n = 0; - for(i=0; i0 ){ - orderedDLReaderReorder(readers+i, nReaders-i); - } - - dlwInit(&writer, pReaders[0].iType, out); - while( !dlrAtEnd(readers[0].pReader) ){ - sqlite_int64 iDocid = dlrDocid(readers[0].pReader); - - /* If this is a continuation of the current buffer to copy, extend - ** that buffer. memcpy() seems to be more efficient if it has a - ** lots of data to copy. - */ - if( dlrDocData(readers[0].pReader)==pStart+nStart ){ - nStart += dlrDocDataBytes(readers[0].pReader); - }else{ - if( pStart!=0 ){ - dlwAppend(&writer, pStart, nStart, iFirstDocid, iLastDocid); - } - pStart = dlrDocData(readers[0].pReader); - nStart = dlrDocDataBytes(readers[0].pReader); - iFirstDocid = iDocid; - } - iLastDocid = iDocid; - dlrStep(readers[0].pReader); - - /* Drop all of the older elements with the same docid. */ - for(i=1; i0 ){ - orderedDLReaderReorder(readers+i, nReaders-i); - } - } - - /* Copy over any remaining elements. */ - if( nStart>0 ) dlwAppend(&writer, pStart, nStart, iFirstDocid, iLastDocid); - dlwDestroy(&writer); -} - -/* Helper function for posListUnion(). Compares the current position -** between left and right, returning as standard C idiom of <0 if -** left0 if left>right, and 0 if left==right. "End" always -** compares greater. -*/ -static int posListCmp(PLReader *pLeft, PLReader *pRight){ - assert( pLeft->iType==pRight->iType ); - if( pLeft->iType==DL_DOCIDS ) return 0; - - if( plrAtEnd(pLeft) ) return plrAtEnd(pRight) ? 0 : 1; - if( plrAtEnd(pRight) ) return -1; - - if( plrColumn(pLeft)plrColumn(pRight) ) return 1; - - if( plrPosition(pLeft)plrPosition(pRight) ) return 1; - if( pLeft->iType==DL_POSITIONS ) return 0; - - if( plrStartOffset(pLeft)plrStartOffset(pRight) ) return 1; - - if( plrEndOffset(pLeft)plrEndOffset(pRight) ) return 1; - - return 0; -} - -/* Write the union of position lists in pLeft and pRight to pOut. -** "Union" in this case meaning "All unique position tuples". Should -** work with any doclist type, though both inputs and the output -** should be the same type. -*/ -static void posListUnion(DLReader *pLeft, DLReader *pRight, DLWriter *pOut){ - PLReader left, right; - PLWriter writer; - - assert( dlrDocid(pLeft)==dlrDocid(pRight) ); - assert( pLeft->iType==pRight->iType ); - assert( pLeft->iType==pOut->iType ); - - plrInit(&left, pLeft); - plrInit(&right, pRight); - plwInit(&writer, pOut, dlrDocid(pLeft)); - - while( !plrAtEnd(&left) || !plrAtEnd(&right) ){ - int c = posListCmp(&left, &right); - if( c<0 ){ - plwCopy(&writer, &left); - plrStep(&left); - }else if( c>0 ){ - plwCopy(&writer, &right); - plrStep(&right); - }else{ - plwCopy(&writer, &left); - plrStep(&left); - plrStep(&right); - } - } - - plwTerminate(&writer); - plwDestroy(&writer); - plrDestroy(&left); - plrDestroy(&right); -} - -/* Write the union of doclists in pLeft and pRight to pOut. For -** docids in common between the inputs, the union of the position -** lists is written. Inputs and outputs are always type DL_DEFAULT. -*/ -static void docListUnion( - const char *pLeft, int nLeft, - const char *pRight, int nRight, - DataBuffer *pOut /* Write the combined doclist here */ -){ - DLReader left, right; - DLWriter writer; - - if( nLeft==0 ){ - if( nRight!=0) dataBufferAppend(pOut, pRight, nRight); - return; - } - if( nRight==0 ){ - dataBufferAppend(pOut, pLeft, nLeft); - return; - } - - dlrInit(&left, DL_DEFAULT, pLeft, nLeft); - dlrInit(&right, DL_DEFAULT, pRight, nRight); - dlwInit(&writer, DL_DEFAULT, pOut); - - while( !dlrAtEnd(&left) || !dlrAtEnd(&right) ){ - if( dlrAtEnd(&right) ){ - dlwCopy(&writer, &left); - dlrStep(&left); - }else if( dlrAtEnd(&left) ){ - dlwCopy(&writer, &right); - dlrStep(&right); - }else if( dlrDocid(&left)dlrDocid(&right) ){ - dlwCopy(&writer, &right); - dlrStep(&right); - }else{ - posListUnion(&left, &right, &writer); - dlrStep(&left); - dlrStep(&right); - } - } - - dlrDestroy(&left); - dlrDestroy(&right); - dlwDestroy(&writer); -} - -/* -** This function is used as part of the implementation of phrase and -** NEAR matching. -** -** pLeft and pRight are DLReaders positioned to the same docid in -** lists of type DL_POSITION. This function writes an entry to the -** DLWriter pOut for each position in pRight that is less than -** (nNear+1) greater (but not equal to or smaller) than a position -** in pLeft. For example, if nNear is 0, and the positions contained -** by pLeft and pRight are: -** -** pLeft: 5 10 15 20 -** pRight: 6 9 17 21 -** -** then the docid is added to pOut. If pOut is of type DL_POSITIONS, -** then a positionids "6" and "21" are also added to pOut. -** -** If boolean argument isSaveLeft is true, then positionids are copied -** from pLeft instead of pRight. In the example above, the positions "5" -** and "20" would be added instead of "6" and "21". -*/ -static void posListPhraseMerge( - DLReader *pLeft, - DLReader *pRight, - int nNear, - int isSaveLeft, - DLWriter *pOut -){ - PLReader left, right; - PLWriter writer; - int match = 0; - - assert( dlrDocid(pLeft)==dlrDocid(pRight) ); - assert( pOut->iType!=DL_POSITIONS_OFFSETS ); - - plrInit(&left, pLeft); - plrInit(&right, pRight); - - while( !plrAtEnd(&left) && !plrAtEnd(&right) ){ - if( plrColumn(&left)plrColumn(&right) ){ - plrStep(&right); - }else if( plrPosition(&left)>=plrPosition(&right) ){ - plrStep(&right); - }else{ - if( (plrPosition(&right)-plrPosition(&left))<=(nNear+1) ){ - if( !match ){ - plwInit(&writer, pOut, dlrDocid(pLeft)); - match = 1; - } - if( !isSaveLeft ){ - plwAdd(&writer, plrColumn(&right), plrPosition(&right), 0, 0); - }else{ - plwAdd(&writer, plrColumn(&left), plrPosition(&left), 0, 0); - } - plrStep(&right); - }else{ - plrStep(&left); - } - } - } - - if( match ){ - plwTerminate(&writer); - plwDestroy(&writer); - } - - plrDestroy(&left); - plrDestroy(&right); -} - /* -** Compare the values pointed to by the PLReaders passed as arguments. -** Return -1 if the value pointed to by pLeft is considered less than -** the value pointed to by pRight, +1 if it is considered greater -** than it, or 0 if it is equal. i.e. -** -** (*pLeft - *pRight) -** -** A PLReader that is in the EOF condition is considered greater than -** any other. If neither argument is in EOF state, the return value of -** plrColumn() is used. If the plrColumn() values are equal, the -** comparison is on the basis of plrPosition(). +** Return the number of bytes required to store the value passed as the +** first argument in varint form. */ -static int plrCompare(PLReader *pLeft, PLReader *pRight){ - assert(!plrAtEnd(pLeft) || !plrAtEnd(pRight)); - - if( plrAtEnd(pRight) || plrAtEnd(pLeft) ){ - return (plrAtEnd(pRight) ? -1 : 1); - } - if( plrColumn(pLeft)!=plrColumn(pRight) ){ - return ((plrColumn(pLeft)0) -** and write the results into pOut. -** -** A phrase intersection means that two documents only match -** if pLeft.iPos+1==pRight.iPos. -** -** A NEAR intersection means that two documents only match if -** (abs(pLeft.iPos-pRight.iPos) 0", - /* SEGDIR_DELETE */ "delete from %_segdir where level = ?", - - /* NOTE(shess): The first three results of the following two - ** statements must match. - */ - /* SEGDIR_SELECT_SEGMENT */ - "select start_block, leaves_end_block, root from %_segdir " - " where level = ? and idx = ?", - /* SEGDIR_SELECT_ALL */ - "select start_block, leaves_end_block, root from %_segdir " - " order by level desc, idx asc", - /* SEGDIR_DELETE_ALL */ "delete from %_segdir", - /* SEGDIR_COUNT */ "select count(*), ifnull(max(level),0) from %_segdir", -}; - -/* -** A connection to a fulltext index is an instance of the following -** structure. The xCreate and xConnect methods create an instance -** of this structure and xDestroy and xDisconnect free that instance. -** All other methods receive a pointer to the structure as one of their -** arguments. -*/ -struct fulltext_vtab { - sqlite3_vtab base; /* Base class used by SQLite core */ - sqlite3 *db; /* The database connection */ - const char *zDb; /* logical database name */ - const char *zName; /* virtual table name */ - int nColumn; /* number of columns in virtual table */ - char **azColumn; /* column names. malloced */ - char **azContentColumn; /* column names in content table; malloced */ - sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */ - - /* Precompiled statements which we keep as long as the table is - ** open. - */ - sqlite3_stmt *pFulltextStatements[MAX_STMT]; - - /* Precompiled statements used for segment merges. We run a - ** separate select across the leaf level of each tree being merged. - */ - sqlite3_stmt *pLeafSelectStmts[MERGE_COUNT]; - /* The statement used to prepare pLeafSelectStmts. */ -#define LEAF_SELECT \ - "select block from %_segments where blockid between ? and ? order by blockid" - - /* These buffer pending index updates during transactions. - ** nPendingData estimates the memory size of the pending data. It - ** doesn't include the hash-bucket overhead, nor any malloc - ** overhead. When nPendingData exceeds kPendingThreshold, the - ** buffer is flushed even before the transaction closes. - ** pendingTerms stores the data, and is only valid when nPendingData - ** is >=0 (nPendingData<0 means pendingTerms has not been - ** initialized). iPrevDocid is the last docid written, used to make - ** certain we're inserting in sorted order. - */ - int nPendingData; -#define kPendingThreshold (1*1024*1024) - sqlite_int64 iPrevDocid; - fts3Hash pendingTerms; -}; - -/* -** When the core wants to do a query, it create a cursor using a -** call to xOpen. This structure is an instance of a cursor. It -** is destroyed by xClose. -*/ -typedef struct fulltext_cursor { - sqlite3_vtab_cursor base; /* Base class used by SQLite core */ - QueryType iCursorType; /* Copy of sqlite3_index_info.idxNum */ - sqlite3_stmt *pStmt; /* Prepared statement in use by the cursor */ - int eof; /* True if at End Of Results */ - Fts3Expr *pExpr; /* Parsed MATCH query string */ - Snippet snippet; /* Cached snippet for the current row */ - int iColumn; /* Column being searched */ - DataBuffer result; /* Doclist results from fulltextQuery */ - DLReader reader; /* Result reader if result not empty */ -} fulltext_cursor; - -static fulltext_vtab *cursor_vtab(fulltext_cursor *c){ - return (fulltext_vtab *) c->base.pVtab; -} - -static const sqlite3_module fts3Module; /* forward declaration */ - -/* Return a dynamically generated statement of the form - * insert into %_content (docid, ...) values (?, ...) - */ -static const char *contentInsertStatement(fulltext_vtab *v){ - StringBuffer sb; - int i; - - initStringBuffer(&sb); - append(&sb, "insert into %_content (docid, "); - appendList(&sb, v->nColumn, v->azContentColumn); - append(&sb, ") values (?"); - for(i=0; inColumn; ++i) - append(&sb, ", ?"); - append(&sb, ")"); - return stringBufferData(&sb); -} - -/* Return a dynamically generated statement of the form - * select from %_content where docid = ? - */ -static const char *contentSelectStatement(fulltext_vtab *v){ - StringBuffer sb; - initStringBuffer(&sb); - append(&sb, "SELECT "); - appendList(&sb, v->nColumn, v->azContentColumn); - append(&sb, " FROM %_content WHERE docid = ?"); - return stringBufferData(&sb); -} - -/* Return a dynamically generated statement of the form - * update %_content set [col_0] = ?, [col_1] = ?, ... - * where docid = ? - */ -static const char *contentUpdateStatement(fulltext_vtab *v){ - StringBuffer sb; - int i; - - initStringBuffer(&sb); - append(&sb, "update %_content set "); - for(i=0; inColumn; ++i) { - if( i>0 ){ - append(&sb, ", "); - } - append(&sb, v->azContentColumn[i]); - append(&sb, " = ?"); - } - append(&sb, " where docid = ?"); - return stringBufferData(&sb); -} - -/* Puts a freshly-prepared statement determined by iStmt in *ppStmt. -** If the indicated statement has never been prepared, it is prepared -** and cached, otherwise the cached version is reset. -*/ -static int sql_get_statement(fulltext_vtab *v, fulltext_statement iStmt, - sqlite3_stmt **ppStmt){ - assert( iStmtpFulltextStatements[iStmt]==NULL ){ - const char *zStmt; - int rc; - switch( iStmt ){ - case CONTENT_INSERT_STMT: - zStmt = contentInsertStatement(v); break; - case CONTENT_SELECT_STMT: - zStmt = contentSelectStatement(v); break; - case CONTENT_UPDATE_STMT: - zStmt = contentUpdateStatement(v); break; - default: - zStmt = fulltext_zStatement[iStmt]; - } - rc = sql_prepare(v->db, v->zDb, v->zName, &v->pFulltextStatements[iStmt], - zStmt); - if( zStmt != fulltext_zStatement[iStmt]) sqlite3_free((void *) zStmt); - if( rc!=SQLITE_OK ) return rc; - } else { - int rc = sqlite3_reset(v->pFulltextStatements[iStmt]); - if( rc!=SQLITE_OK ) return rc; - } - - *ppStmt = v->pFulltextStatements[iStmt]; - return SQLITE_OK; -} - -/* Like sqlite3_step(), but convert SQLITE_DONE to SQLITE_OK and -** SQLITE_ROW to SQLITE_ERROR. Useful for statements like UPDATE, -** where we expect no results. -*/ -static int sql_single_step(sqlite3_stmt *s){ - int rc = sqlite3_step(s); - return (rc==SQLITE_DONE) ? SQLITE_OK : rc; -} - -/* Like sql_get_statement(), but for special replicated LEAF_SELECT -** statements. idx -1 is a special case for an uncached version of -** the statement (used in the optimize implementation). -*/ -/* TODO(shess) Write version for generic statements and then share -** that between the cached-statement functions. -*/ -static int sql_get_leaf_statement(fulltext_vtab *v, int idx, - sqlite3_stmt **ppStmt){ - assert( idx>=-1 && idxdb, v->zDb, v->zName, ppStmt, LEAF_SELECT); - }else if( v->pLeafSelectStmts[idx]==NULL ){ - int rc = sql_prepare(v->db, v->zDb, v->zName, &v->pLeafSelectStmts[idx], - LEAF_SELECT); - if( rc!=SQLITE_OK ) return rc; - }else{ - int rc = sqlite3_reset(v->pLeafSelectStmts[idx]); - if( rc!=SQLITE_OK ) return rc; - } - - *ppStmt = v->pLeafSelectStmts[idx]; - return SQLITE_OK; -} - -/* insert into %_content (docid, ...) values ([docid], [pValues]) -** If the docid contains SQL NULL, then a unique docid will be -** generated. -*/ -static int content_insert(fulltext_vtab *v, sqlite3_value *docid, - sqlite3_value **pValues){ - sqlite3_stmt *s; - int i; - int rc = sql_get_statement(v, CONTENT_INSERT_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_value(s, 1, docid); - if( rc!=SQLITE_OK ) return rc; - - for(i=0; inColumn; ++i){ - rc = sqlite3_bind_value(s, 2+i, pValues[i]); - if( rc!=SQLITE_OK ) return rc; - } - - return sql_single_step(s); -} - -/* update %_content set col0 = pValues[0], col1 = pValues[1], ... - * where docid = [iDocid] */ -static int content_update(fulltext_vtab *v, sqlite3_value **pValues, - sqlite_int64 iDocid){ - sqlite3_stmt *s; - int i; - int rc = sql_get_statement(v, CONTENT_UPDATE_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - for(i=0; inColumn; ++i){ - rc = sqlite3_bind_value(s, 1+i, pValues[i]); - if( rc!=SQLITE_OK ) return rc; - } - - rc = sqlite3_bind_int64(s, 1+v->nColumn, iDocid); - if( rc!=SQLITE_OK ) return rc; - - return sql_single_step(s); -} - -static void freeStringArray(int nString, const char **pString){ - int i; - - for (i=0 ; i < nString ; ++i) { - if( pString[i]!=NULL ) sqlite3_free((void *) pString[i]); - } - sqlite3_free((void *) pString); -} - -/* select * from %_content where docid = [iDocid] - * The caller must delete the returned array and all strings in it. - * null fields will be NULL in the returned array. - * - * TODO: Perhaps we should return pointer/length strings here for consistency - * with other code which uses pointer/length. */ -static int content_select(fulltext_vtab *v, sqlite_int64 iDocid, - const char ***pValues){ - sqlite3_stmt *s; - const char **values; - int i; - int rc; - - *pValues = NULL; - - rc = sql_get_statement(v, CONTENT_SELECT_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 1, iDocid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - if( rc!=SQLITE_ROW ) return rc; - - values = (const char **) sqlite3_malloc(v->nColumn * sizeof(const char *)); - for(i=0; inColumn; ++i){ - if( sqlite3_column_type(s, i)==SQLITE_NULL ){ - values[i] = NULL; - }else{ - values[i] = string_dup((char*)sqlite3_column_text(s, i)); - } - } - - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain locked. */ - rc = sqlite3_step(s); - if( rc==SQLITE_DONE ){ - *pValues = values; - return SQLITE_OK; - } - - freeStringArray(v->nColumn, values); - return rc; -} - -/* delete from %_content where docid = [iDocid ] */ -static int content_delete(fulltext_vtab *v, sqlite_int64 iDocid){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, CONTENT_DELETE_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 1, iDocid); - if( rc!=SQLITE_OK ) return rc; - - return sql_single_step(s); -} - -/* Returns SQLITE_ROW if any rows exist in %_content, SQLITE_DONE if -** no rows exist, and any error in case of failure. -*/ -static int content_exists(fulltext_vtab *v){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, CONTENT_EXISTS_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - if( rc!=SQLITE_ROW ) return rc; - - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain locked. */ - rc = sqlite3_step(s); - if( rc==SQLITE_DONE ) return SQLITE_ROW; - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - return rc; -} - -/* insert into %_segments values ([pData]) -** returns assigned blockid in *piBlockid -*/ -static int block_insert(fulltext_vtab *v, const char *pData, int nData, - sqlite_int64 *piBlockid){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, BLOCK_INSERT_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_blob(s, 1, pData, nData, SQLITE_STATIC); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - if( rc!=SQLITE_DONE ) return rc; - - /* blockid column is an alias for rowid. */ - *piBlockid = sqlite3_last_insert_rowid(v->db); - return SQLITE_OK; -} - -/* delete from %_segments -** where blockid between [iStartBlockid] and [iEndBlockid] -** -** Deletes the range of blocks, inclusive, used to delete the blocks -** which form a segment. -*/ -static int block_delete(fulltext_vtab *v, - sqlite_int64 iStartBlockid, sqlite_int64 iEndBlockid){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, BLOCK_DELETE_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 1, iStartBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 2, iEndBlockid); - if( rc!=SQLITE_OK ) return rc; - - return sql_single_step(s); -} - -/* Returns SQLITE_ROW with *pidx set to the maximum segment idx found -** at iLevel. Returns SQLITE_DONE if there are no segments at -** iLevel. Otherwise returns an error. -*/ -static int segdir_max_index(fulltext_vtab *v, int iLevel, int *pidx){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, SEGDIR_MAX_INDEX_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int(s, 1, iLevel); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - /* Should always get at least one row due to how max() works. */ - if( rc==SQLITE_DONE ) return SQLITE_DONE; - if( rc!=SQLITE_ROW ) return rc; - - /* NULL means that there were no inputs to max(). */ - if( SQLITE_NULL==sqlite3_column_type(s, 0) ){ - rc = sqlite3_step(s); - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - return rc; - } - - *pidx = sqlite3_column_int(s, 0); - - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain locked. */ - rc = sqlite3_step(s); - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - if( rc!=SQLITE_DONE ) return rc; - return SQLITE_ROW; -} - -/* insert into %_segdir values ( -** [iLevel], [idx], -** [iStartBlockid], [iLeavesEndBlockid], [iEndBlockid], -** [pRootData] -** ) -*/ -static int segdir_set(fulltext_vtab *v, int iLevel, int idx, - sqlite_int64 iStartBlockid, - sqlite_int64 iLeavesEndBlockid, - sqlite_int64 iEndBlockid, - const char *pRootData, int nRootData){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, SEGDIR_SET_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int(s, 1, iLevel); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int(s, 2, idx); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 3, iStartBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 4, iLeavesEndBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 5, iEndBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_blob(s, 6, pRootData, nRootData, SQLITE_STATIC); - if( rc!=SQLITE_OK ) return rc; - - return sql_single_step(s); -} - -/* Queries %_segdir for the block span of the segments in level -** iLevel. Returns SQLITE_DONE if there are no blocks for iLevel, -** SQLITE_ROW if there are blocks, else an error. -*/ -static int segdir_span(fulltext_vtab *v, int iLevel, - sqlite_int64 *piStartBlockid, - sqlite_int64 *piEndBlockid){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, SEGDIR_SPAN_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int(s, 1, iLevel); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - if( rc==SQLITE_DONE ) return SQLITE_DONE; /* Should never happen */ - if( rc!=SQLITE_ROW ) return rc; - - /* This happens if all segments at this level are entirely inline. */ - if( SQLITE_NULL==sqlite3_column_type(s, 0) ){ - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain locked. */ - int rc2 = sqlite3_step(s); - if( rc2==SQLITE_ROW ) return SQLITE_ERROR; - return rc2; - } - - *piStartBlockid = sqlite3_column_int64(s, 0); - *piEndBlockid = sqlite3_column_int64(s, 1); - - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain locked. */ - rc = sqlite3_step(s); - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - if( rc!=SQLITE_DONE ) return rc; - return SQLITE_ROW; -} - -/* Delete the segment blocks and segment directory records for all -** segments at iLevel. -*/ -static int segdir_delete(fulltext_vtab *v, int iLevel){ - sqlite3_stmt *s; - sqlite_int64 iStartBlockid, iEndBlockid; - int rc = segdir_span(v, iLevel, &iStartBlockid, &iEndBlockid); - if( rc!=SQLITE_ROW && rc!=SQLITE_DONE ) return rc; - - if( rc==SQLITE_ROW ){ - rc = block_delete(v, iStartBlockid, iEndBlockid); - if( rc!=SQLITE_OK ) return rc; - } - - /* Delete the segment directory itself. */ - rc = sql_get_statement(v, SEGDIR_DELETE_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 1, iLevel); - if( rc!=SQLITE_OK ) return rc; - - return sql_single_step(s); -} - -/* Delete entire fts index, SQLITE_OK on success, relevant error on -** failure. -*/ -static int segdir_delete_all(fulltext_vtab *v){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, SEGDIR_DELETE_ALL_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sql_single_step(s); - if( rc!=SQLITE_OK ) return rc; - - rc = sql_get_statement(v, BLOCK_DELETE_ALL_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - return sql_single_step(s); -} - -/* Returns SQLITE_OK with *pnSegments set to the number of entries in -** %_segdir and *piMaxLevel set to the highest level which has a -** segment. Otherwise returns the SQLite error which caused failure. -*/ -static int segdir_count(fulltext_vtab *v, int *pnSegments, int *piMaxLevel){ - sqlite3_stmt *s; - int rc = sql_get_statement(v, SEGDIR_COUNT_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - /* TODO(shess): This case should not be possible? Should stronger - ** measures be taken if it happens? - */ - if( rc==SQLITE_DONE ){ - *pnSegments = 0; - *piMaxLevel = 0; - return SQLITE_OK; - } - if( rc!=SQLITE_ROW ) return rc; - - *pnSegments = sqlite3_column_int(s, 0); - *piMaxLevel = sqlite3_column_int(s, 1); - - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain locked. */ - rc = sqlite3_step(s); - if( rc==SQLITE_DONE ) return SQLITE_OK; - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - return rc; -} - -/* TODO(shess) clearPendingTerms() is far down the file because -** writeZeroSegment() is far down the file because LeafWriter is far -** down the file. Consider refactoring the code to move the non-vtab -** code above the vtab code so that we don't need this forward -** reference. -*/ -static int clearPendingTerms(fulltext_vtab *v); - -/* -** Free the memory used to contain a fulltext_vtab structure. -*/ -static void fulltext_vtab_destroy(fulltext_vtab *v){ - int iStmt, i; - - FTSTRACE(("FTS3 Destroy %p\n", v)); - for( iStmt=0; iStmtpFulltextStatements[iStmt]!=NULL ){ - sqlite3_finalize(v->pFulltextStatements[iStmt]); - v->pFulltextStatements[iStmt] = NULL; - } - } - - for( i=0; ipLeafSelectStmts[i]!=NULL ){ - sqlite3_finalize(v->pLeafSelectStmts[i]); - v->pLeafSelectStmts[i] = NULL; - } - } - - if( v->pTokenizer!=NULL ){ - v->pTokenizer->pModule->xDestroy(v->pTokenizer); - v->pTokenizer = NULL; - } - - clearPendingTerms(v); - - sqlite3_free(v->azColumn); - for(i = 0; i < v->nColumn; ++i) { - sqlite3_free(v->azContentColumn[i]); - } - sqlite3_free(v->azContentColumn); - sqlite3_free(v); -} - -/* -** Token types for parsing the arguments to xConnect or xCreate. -*/ -#define TOKEN_EOF 0 /* End of file */ -#define TOKEN_SPACE 1 /* Any kind of whitespace */ -#define TOKEN_ID 2 /* An identifier */ -#define TOKEN_STRING 3 /* A string literal */ -#define TOKEN_PUNCT 4 /* A single punctuation character */ - -/* -** If X is a character that can be used in an identifier then -** ftsIdChar(X) will be true. Otherwise it is false. -** -** For ASCII, any character with the high-order bit set is -** allowed in an identifier. For 7-bit characters, -** isFtsIdChar[X] must be 1. -** -** Ticket #1066. the SQL standard does not allow '$' in the -** middle of identfiers. But many SQL implementations do. -** SQLite will allow '$' in identifiers for compatibility. -** But the feature is undocumented. -*/ -static const char isFtsIdChar[] = { -/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */ - 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 2x */ - 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, /* 3x */ - 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 4x */ - 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, /* 5x */ - 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 6x */ - 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, /* 7x */ -}; -#define ftsIdChar(C) (((c=C)&0x80)!=0 || (c>0x1f && isFtsIdChar[c-0x20])) - - -/* -** Return the length of the token that begins at z[0]. -** Store the token type in *tokenType before returning. -*/ -static int ftsGetToken(const char *z, int *tokenType){ - int i, c; - switch( *z ){ - case 0: { - *tokenType = TOKEN_EOF; - return 0; - } - case ' ': case '\t': case '\n': case '\f': case '\r': { - for(i=1; safe_isspace(z[i]); i++){} - *tokenType = TOKEN_SPACE; - return i; - } - case '`': - case '\'': - case '"': { - int delim = z[0]; - for(i=1; (c=z[i])!=0; i++){ - if( c==delim ){ - if( z[i+1]==delim ){ - i++; - }else{ - break; - } - } - } - *tokenType = TOKEN_STRING; - return i + (c!=0); - } - case '[': { - for(i=1, c=z[0]; c!=']' && (c=z[i])!=0; i++){} - *tokenType = TOKEN_ID; - return i; - } - default: { - if( !ftsIdChar(*z) ){ - break; - } - for(i=1; ftsIdChar(z[i]); i++){} - *tokenType = TOKEN_ID; - return i; - } - } - *tokenType = TOKEN_PUNCT; - return 1; -} - -/* -** A token extracted from a string is an instance of the following -** structure. -*/ -typedef struct FtsToken { - const char *z; /* Pointer to token text. Not '\000' terminated */ - short int n; /* Length of the token text in bytes. */ -} FtsToken; - -/* -** Given a input string (which is really one of the argv[] parameters -** passed into xConnect or xCreate) split the string up into tokens. -** Return an array of pointers to '\000' terminated strings, one string -** for each non-whitespace token. -** -** The returned array is terminated by a single NULL pointer. -** -** Space to hold the returned array is obtained from a single -** malloc and should be freed by passing the return value to free(). -** The individual strings within the token list are all a part of -** the single memory allocation and will all be freed at once. -*/ -static char **tokenizeString(const char *z, int *pnToken){ - int nToken = 0; - FtsToken *aToken = sqlite3_malloc( strlen(z) * sizeof(aToken[0]) ); - int n = 1; - int e, i; - int totalSize = 0; - char **azToken; - char *zCopy; - while( n>0 ){ - n = ftsGetToken(z, &e); - if( e!=TOKEN_SPACE ){ - aToken[nToken].z = z; - aToken[nToken].n = n; - nToken++; - totalSize += n+1; - } - z += n; - } - azToken = (char**)sqlite3_malloc( nToken*sizeof(char*) + totalSize ); - zCopy = (char*)&azToken[nToken]; - nToken--; - for(i=0; i>= 7; + }while( v!=0 ); + return i; } /* @@ -100305,10 +98729,10 @@ static char **tokenizeString(const char *z, int *pnToken){ ** [pqr] becomes pqr ** `mno` becomes mno */ -static void dequoteString(char *z){ +SQLITE_PRIVATE void sqlite3Fts3Dequote(char *z){ int quote; int i, j; - if( z==0 ) return; + quote = z[0]; switch( quote ){ case '\'': break; @@ -100320,7 +98744,7 @@ static void dequoteString(char *z){ for(i=1, j=0; z[i]; i++){ if( z[i]==quote ){ if( z[i+1]==quote ){ - z[j++] = quote; + z[j++] = (char)quote; i++; }else{ z[j++] = 0; @@ -100332,4255 +98756,1785 @@ static void dequoteString(char *z){ } } -/* -** The input azIn is a NULL-terminated list of tokens. Remove the first -** token and all punctuation tokens. Remove the quotes from -** around string literal tokens. -** -** Example: -** -** input: tokenize chinese ( 'simplifed' , 'mixed' ) -** output: chinese simplifed mixed -** -** Another example: -** -** input: delimiters ( '[' , ']' , '...' ) -** output: [ ] ... -*/ -static void tokenListToIdList(char **azIn){ - int i, j; - if( azIn ){ - for(i=0, j=-1; azIn[i]; i++){ - if( safe_isalnum(azIn[i][0]) || azIn[i][1] ){ - dequoteString(azIn[i]); - if( j>=0 ){ - azIn[j] = azIn[i]; - } - j++; - } - } - azIn[j] = 0; +static void fts3GetDeltaVarint(char **pp, sqlite3_int64 *pVal){ + sqlite3_int64 iVal; + *pp += sqlite3Fts3GetVarint(*pp, &iVal); + *pVal += iVal; +} + +static void fts3GetDeltaVarint2(char **pp, char *pEnd, sqlite3_int64 *pVal){ + if( *pp>=pEnd ){ + *pp = 0; + }else{ + fts3GetDeltaVarint(pp, pVal); } } - /* -** Find the first alphanumeric token in the string zIn. Null-terminate -** this token. Remove any quotation marks. And return a pointer to -** the result. +** The xDisconnect() virtual table method. */ -static char *firstToken(char *zIn, char **pzTail){ - int n, ttype; - while(1){ - n = ftsGetToken(zIn, &ttype); - if( ttype==TOKEN_SPACE ){ - zIn += n; - }else if( ttype==TOKEN_EOF ){ - *pzTail = zIn; - return 0; - }else{ - zIn[n] = 0; - *pzTail = &zIn[1]; - dequoteString(zIn); - return zIn; - } +static int fts3DisconnectMethod(sqlite3_vtab *pVtab){ + Fts3Table *p = (Fts3Table *)pVtab; + int i; + + assert( p->nPendingData==0 ); + + /* Free any prepared statements held */ + for(i=0; iaStmt); i++){ + sqlite3_finalize(p->aStmt[i]); } - /*NOTREACHED*/ -} - -/* Return true if... -** -** * s begins with the string t, ignoring case -** * s is longer than t -** * The first character of s beyond t is not a alphanumeric -** -** Ignore leading space in *s. -** -** To put it another way, return true if the first token of -** s[] is t[]. -*/ -static int startsWith(const char *s, const char *t){ - while( safe_isspace(*s) ){ s++; } - while( *t ){ - if( safe_tolower(*s++)!=safe_tolower(*t++) ) return 0; + for(i=0; inLeavesStmt; i++){ + sqlite3_finalize(p->aLeavesStmt[i]); } - return *s!='_' && !safe_isalnum(*s); -} + sqlite3_free(p->zSelectLeaves); + sqlite3_free(p->aLeavesStmt); -/* -** An instance of this structure defines the "spec" of a -** full text index. This structure is populated by parseSpec -** and use by fulltextConnect and fulltextCreate. -*/ -typedef struct TableSpec { - const char *zDb; /* Logical database name */ - const char *zName; /* Name of the full-text index */ - int nColumn; /* Number of columns to be indexed */ - char **azColumn; /* Original names of columns to be indexed */ - char **azContentColumn; /* Column names for %_content */ - char **azTokenizer; /* Name of tokenizer and its arguments */ -} TableSpec; - -/* -** Reclaim all of the memory used by a TableSpec -*/ -static void clearTableSpec(TableSpec *p) { - sqlite3_free(p->azColumn); - sqlite3_free(p->azContentColumn); - sqlite3_free(p->azTokenizer); -} - -/* Parse a CREATE VIRTUAL TABLE statement, which looks like this: - * - * CREATE VIRTUAL TABLE email - * USING fts3(subject, body, tokenize mytokenizer(myarg)) - * - * We return parsed information in a TableSpec structure. - * - */ -static int parseSpec(TableSpec *pSpec, int argc, const char *const*argv, - char**pzErr){ - int i, n; - char *z, *zDummy; - char **azArg; - const char *zTokenizer = 0; /* argv[] entry describing the tokenizer */ - - assert( argc>=3 ); - /* Current interface: - ** argv[0] - module name - ** argv[1] - database name - ** argv[2] - table name - ** argv[3..] - columns, optionally followed by tokenizer specification - ** and snippet delimiters specification. - */ - - /* Make a copy of the complete argv[][] array in a single allocation. - ** The argv[][] array is read-only and transient. We can write to the - ** copy in order to modify things and the copy is persistent. - */ - CLEAR(pSpec); - for(i=n=0; izDb = azArg[1]; - pSpec->zName = azArg[2]; - pSpec->nColumn = 0; - pSpec->azColumn = azArg; - zTokenizer = "tokenize simple"; - for(i=3; inColumn] = firstToken(azArg[i], &zDummy); - pSpec->nColumn++; - } - } - if( pSpec->nColumn==0 ){ - azArg[0] = "content"; - pSpec->nColumn = 1; - } - - /* - ** Construct the list of content column names. - ** - ** Each content column name will be of the form cNNAAAA - ** where NN is the column number and AAAA is the sanitized - ** column name. "sanitized" means that special characters are - ** converted to "_". The cNN prefix guarantees that all column - ** names are unique. - ** - ** The AAAA suffix is not strictly necessary. It is included - ** for the convenience of people who might examine the generated - ** %_content table and wonder what the columns are used for. - */ - pSpec->azContentColumn = sqlite3_malloc( pSpec->nColumn * sizeof(char *) ); - if( pSpec->azContentColumn==0 ){ - clearTableSpec(pSpec); - return SQLITE_NOMEM; - } - for(i=0; inColumn; i++){ - char *p; - pSpec->azContentColumn[i] = sqlite3_mprintf("c%d%s", i, azArg[i]); - for (p = pSpec->azContentColumn[i]; *p ; ++p) { - if( !safe_isalnum(*p) ) *p = '_'; - } - } - - /* - ** Parse the tokenizer specification string. - */ - pSpec->azTokenizer = tokenizeString(zTokenizer, &n); - tokenListToIdList(pSpec->azTokenizer); + /* Invoke the tokenizer destructor to free the tokenizer. */ + p->pTokenizer->pModule->xDestroy(p->pTokenizer); + sqlite3_free(p); return SQLITE_OK; } /* -** Generate a CREATE TABLE statement that describes the schema of -** the virtual table. Return a pointer to this schema string. -** -** Space is obtained from sqlite3_mprintf() and should be freed -** using sqlite3_free(). +** The xDestroy() virtual table method. */ -static char *fulltextSchema( - int nColumn, /* Number of columns */ - const char *const* azColumn, /* List of columns */ - const char *zTableName /* Name of the table */ -){ - int i; - char *zSchema, *zNext; - const char *zSep = "("; - zSchema = sqlite3_mprintf("CREATE TABLE x"); - for(i=0; izDb, p->zName, p->zDb, p->zName, p->zDb, p->zName + ); + + /* If malloc has failed, set rc to SQLITE_NOMEM. Otherwise, try to + ** execute the SQL script created above. + */ + if( zSql ){ + rc = sqlite3_exec(p->db, zSql, 0, 0, 0); + sqlite3_free(zSql); + }else{ + rc = SQLITE_NOMEM; } - zNext = sqlite3_mprintf("%s,%Q HIDDEN", zSchema, zTableName); - sqlite3_free(zSchema); - zSchema = zNext; - zNext = sqlite3_mprintf("%s,docid HIDDEN)", zSchema); - sqlite3_free(zSchema); - return zNext; + + /* If everything has worked, invoke fts3DisconnectMethod() to free the + ** memory associated with the Fts3Table structure and return SQLITE_OK. + ** Otherwise, return an SQLite error code. + */ + return (rc==SQLITE_OK ? fts3DisconnectMethod(pVtab) : rc); } + /* -** Build a new sqlite3_vtab structure that will describe the -** fulltext index defined by spec. +** Invoke sqlite3_declare_vtab() to declare the schema for the FTS3 table +** passed as the first argument. This is done as part of the xConnect() +** and xCreate() methods. */ -static int constructVtab( - sqlite3 *db, /* The SQLite database connection */ - fts3Hash *pHash, /* Hash table containing tokenizers */ - TableSpec *spec, /* Parsed spec information from parseSpec() */ - sqlite3_vtab **ppVTab, /* Write the resulting vtab structure here */ - char **pzErr /* Write any error message here */ -){ - int rc; - int n; - fulltext_vtab *v = 0; - const sqlite3_tokenizer_module *m = NULL; - char *schema; +static int fts3DeclareVtab(Fts3Table *p){ + int i; /* Iterator variable */ + int rc; /* Return code */ + char *zSql; /* SQL statement passed to declare_vtab() */ + char *zCols; /* List of user defined columns */ - char const *zTok; /* Name of tokenizer to use for this fts table */ - int nTok; /* Length of zTok, including nul terminator */ - - v = (fulltext_vtab *) sqlite3_malloc(sizeof(fulltext_vtab)); - if( v==0 ) return SQLITE_NOMEM; - CLEAR(v); - /* sqlite will initialize v->base */ - v->db = db; - v->zDb = spec->zDb; /* Freed when azColumn is freed */ - v->zName = spec->zName; /* Freed when azColumn is freed */ - v->nColumn = spec->nColumn; - v->azContentColumn = spec->azContentColumn; - spec->azContentColumn = 0; - v->azColumn = spec->azColumn; - spec->azColumn = 0; - - if( spec->azTokenizer==0 ){ - return SQLITE_NOMEM; + /* Create a list of user columns for the virtual table */ + zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]); + for(i=1; zCols && inColumn; i++){ + zCols = sqlite3_mprintf("%z%Q, ", zCols, p->azColumn[i]); } - zTok = spec->azTokenizer[0]; - if( !zTok ){ - zTok = "simple"; - } - nTok = strlen(zTok)+1; + /* Create the whole "CREATE TABLE" statement to pass to SQLite */ + zSql = sqlite3_mprintf( + "CREATE TABLE x(%s %Q HIDDEN, docid HIDDEN)", zCols, p->zName + ); - m = (sqlite3_tokenizer_module *)sqlite3Fts3HashFind(pHash, zTok, nTok); - if( !m ){ - *pzErr = sqlite3_mprintf("unknown tokenizer: %s", spec->azTokenizer[0]); - rc = SQLITE_ERROR; - goto err; - } - - for(n=0; spec->azTokenizer[n]; n++){} - if( n ){ - rc = m->xCreate(n-1, (const char*const*)&spec->azTokenizer[1], - &v->pTokenizer); + if( !zCols || !zSql ){ + rc = SQLITE_NOMEM; }else{ - rc = m->xCreate(0, 0, &v->pTokenizer); + rc = sqlite3_declare_vtab(p->db, zSql); } - if( rc!=SQLITE_OK ) goto err; - v->pTokenizer->pModule = m; - /* TODO: verify the existence of backing tables foo_content, foo_term */ - - schema = fulltextSchema(v->nColumn, (const char*const*)v->azColumn, - spec->zName); - rc = sqlite3_declare_vtab(db, schema); - sqlite3_free(schema); - if( rc!=SQLITE_OK ) goto err; - - memset(v->pFulltextStatements, 0, sizeof(v->pFulltextStatements)); - - /* Indicate that the buffer is not live. */ - v->nPendingData = -1; - - *ppVTab = &v->base; - FTSTRACE(("FTS3 Connect %p\n", v)); - - return rc; - -err: - fulltext_vtab_destroy(v); + sqlite3_free(zSql); + sqlite3_free(zCols); return rc; } -static int fulltextConnect( - sqlite3 *db, - void *pAux, - int argc, const char *const*argv, - sqlite3_vtab **ppVTab, - char **pzErr -){ - TableSpec spec; - int rc = parseSpec(&spec, argc, argv, pzErr); - if( rc!=SQLITE_OK ) return rc; +/* +** Create the backing store tables (%_content, %_segments and %_segdir) +** required by the FTS3 table passed as the only argument. This is done +** as part of the vtab xCreate() method. +*/ +static int fts3CreateTables(Fts3Table *p){ + int rc; /* Return code */ + int i; /* Iterator variable */ + char *zContentCols; /* Columns of %_content table */ + char *zSql; /* SQL script to create required tables */ - rc = constructVtab(db, (fts3Hash *)pAux, &spec, ppVTab, pzErr); - clearTableSpec(&spec); + /* Create a list of user columns for the content table */ + zContentCols = sqlite3_mprintf("docid INTEGER PRIMARY KEY"); + for(i=0; zContentCols && inColumn; i++){ + char *z = p->azColumn[i]; + zContentCols = sqlite3_mprintf("%z, 'c%d%q'", zContentCols, i, z); + } + + /* Create the whole SQL script */ + zSql = sqlite3_mprintf( + "CREATE TABLE %Q.'%q_content'(%s);" + "CREATE TABLE %Q.'%q_segments'(blockid INTEGER PRIMARY KEY, block BLOB);" + "CREATE TABLE %Q.'%q_segdir'(" + "level INTEGER," + "idx INTEGER," + "start_block INTEGER," + "leaves_end_block INTEGER," + "end_block INTEGER," + "root BLOB," + "PRIMARY KEY(level, idx)" + ");", + p->zDb, p->zName, zContentCols, p->zDb, p->zName, p->zDb, p->zName + ); + + /* Unless a malloc() failure has occurred, execute the SQL script to + ** create the tables used to store data for this FTS3 virtual table. + */ + if( zContentCols==0 || zSql==0 ){ + rc = SQLITE_NOMEM; + }else{ + rc = sqlite3_exec(p->db, zSql, 0, 0, 0); + } + + sqlite3_free(zSql); + sqlite3_free(zContentCols); return rc; } -/* The %_content table holds the text of each document, with -** the docid column exposed as the SQLite rowid for the table. -*/ -/* TODO(shess) This comment needs elaboration to match the updated -** code. Work it into the top-of-file comment at that time. -*/ -static int fulltextCreate(sqlite3 *db, void *pAux, - int argc, const char * const *argv, - sqlite3_vtab **ppVTab, char **pzErr){ - int rc; - TableSpec spec; - StringBuffer schema; - FTSTRACE(("FTS3 Create\n")); - - rc = parseSpec(&spec, argc, argv, pzErr); - if( rc!=SQLITE_OK ) return rc; - - initStringBuffer(&schema); - append(&schema, "CREATE TABLE %_content("); - append(&schema, " docid INTEGER PRIMARY KEY,"); - appendList(&schema, spec.nColumn, spec.azContentColumn); - append(&schema, ")"); - rc = sql_exec(db, spec.zDb, spec.zName, stringBufferData(&schema)); - stringBufferDestroy(&schema); - if( rc!=SQLITE_OK ) goto out; - - rc = sql_exec(db, spec.zDb, spec.zName, - "create table %_segments(" - " blockid INTEGER PRIMARY KEY," - " block blob" - ");" - ); - if( rc!=SQLITE_OK ) goto out; - - rc = sql_exec(db, spec.zDb, spec.zName, - "create table %_segdir(" - " level integer," - " idx integer," - " start_block integer," - " leaves_end_block integer," - " end_block integer," - " root blob," - " primary key(level, idx)" - ");"); - if( rc!=SQLITE_OK ) goto out; - - rc = constructVtab(db, (fts3Hash *)pAux, &spec, ppVTab, pzErr); - -out: - clearTableSpec(&spec); - return rc; -} - -/* Decide how to handle an SQL query. */ -static int fulltextBestIndex(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){ - fulltext_vtab *v = (fulltext_vtab *)pVTab; - int i; - FTSTRACE(("FTS3 BestIndex\n")); - - for(i=0; inConstraint; ++i){ - const struct sqlite3_index_constraint *pConstraint; - pConstraint = &pInfo->aConstraint[i]; - if( pConstraint->usable ) { - if( (pConstraint->iColumn==-1 || pConstraint->iColumn==v->nColumn+1) && - pConstraint->op==SQLITE_INDEX_CONSTRAINT_EQ ){ - pInfo->idxNum = QUERY_DOCID; /* lookup by docid */ - FTSTRACE(("FTS3 QUERY_DOCID\n")); - } else if( pConstraint->iColumn>=0 && pConstraint->iColumn<=v->nColumn && - pConstraint->op==SQLITE_INDEX_CONSTRAINT_MATCH ){ - /* full-text search */ - pInfo->idxNum = QUERY_FULLTEXT + pConstraint->iColumn; - FTSTRACE(("FTS3 QUERY_FULLTEXT %d\n", pConstraint->iColumn)); - } else continue; - - pInfo->aConstraintUsage[i].argvIndex = 1; - pInfo->aConstraintUsage[i].omit = 1; - - /* An arbitrary value for now. - * TODO: Perhaps docid matches should be considered cheaper than - * full-text searches. */ - pInfo->estimatedCost = 1.0; - - return SQLITE_OK; - } - } - pInfo->idxNum = QUERY_GENERIC; - return SQLITE_OK; -} - -static int fulltextDisconnect(sqlite3_vtab *pVTab){ - FTSTRACE(("FTS3 Disconnect %p\n", pVTab)); - fulltext_vtab_destroy((fulltext_vtab *)pVTab); - return SQLITE_OK; -} - -static int fulltextDestroy(sqlite3_vtab *pVTab){ - fulltext_vtab *v = (fulltext_vtab *)pVTab; - int rc; - - FTSTRACE(("FTS3 Destroy %p\n", pVTab)); - rc = sql_exec(v->db, v->zDb, v->zName, - "drop table if exists %_content;" - "drop table if exists %_segments;" - "drop table if exists %_segdir;" - ); - if( rc!=SQLITE_OK ) return rc; - - fulltext_vtab_destroy((fulltext_vtab *)pVTab); - return SQLITE_OK; -} - -static int fulltextOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){ - fulltext_cursor *c; - - c = (fulltext_cursor *) sqlite3_malloc(sizeof(fulltext_cursor)); - if( c ){ - memset(c, 0, sizeof(fulltext_cursor)); - /* sqlite will initialize c->base */ - *ppCursor = &c->base; - FTSTRACE(("FTS3 Open %p: %p\n", pVTab, c)); - return SQLITE_OK; - }else{ - return SQLITE_NOMEM; - } -} - -/* Free all of the dynamically allocated memory held by the -** Snippet -*/ -static void snippetClear(Snippet *p){ - sqlite3_free(p->aMatch); - sqlite3_free(p->zOffset); - sqlite3_free(p->zSnippet); - CLEAR(p); -} - /* -** Append a single entry to the p->aMatch[] log. +** This function is the implementation of both the xConnect and xCreate +** methods of the FTS3 virtual table. +** +** The argv[] array contains the following: +** +** argv[0] -> module name +** argv[1] -> database name +** argv[2] -> table name +** argv[...] -> "column name" and other module argument fields. */ -static void snippetAppendMatch( - Snippet *p, /* Append the entry to this snippet */ - int iCol, int iTerm, /* The column and query term */ - int iToken, /* Matching token in document */ - int iStart, int nByte /* Offset and size of the match */ +int fts3InitVtab( + int isCreate, /* True for xCreate, false for xConnect */ + sqlite3 *db, /* The SQLite database connection */ + void *pAux, /* Hash table containing tokenizers */ + int argc, /* Number of elements in argv array */ + const char * const *argv, /* xCreate/xConnect argument array */ + sqlite3_vtab **ppVTab, /* Write the resulting vtab structure here */ + char **pzErr /* Write any error message here */ ){ - int i; - struct snippetMatch *pMatch; - if( p->nMatch+1>=p->nAlloc ){ - p->nAlloc = p->nAlloc*2 + 10; - p->aMatch = sqlite3_realloc(p->aMatch, p->nAlloc*sizeof(p->aMatch[0]) ); - if( p->aMatch==0 ){ - p->nMatch = 0; - p->nAlloc = 0; - return; - } - } - i = p->nMatch++; - pMatch = &p->aMatch[i]; - pMatch->iCol = iCol; - pMatch->iTerm = iTerm; - pMatch->iToken = iToken; - pMatch->iStart = iStart; - pMatch->nByte = nByte; -} - -/* -** Sizing information for the circular buffer used in snippetOffsetsOfColumn() -*/ -#define FTS3_ROTOR_SZ (32) -#define FTS3_ROTOR_MASK (FTS3_ROTOR_SZ-1) - -/* -** Function to iterate through the tokens of a compiled expression. -** -** Except, skip all tokens on the right-hand side of a NOT operator. -** This function is used to find tokens as part of snippet and offset -** generation and we do nt want snippets and offsets to report matches -** for tokens on the RHS of a NOT. -*/ -static int fts3NextExprToken(Fts3Expr **ppExpr, int *piToken){ - Fts3Expr *p = *ppExpr; - int iToken = *piToken; - if( iToken<0 ){ - /* In this case the expression p is the root of an expression tree. - ** Move to the first token in the expression tree. - */ - while( p->pLeft ){ - p = p->pLeft; - } - iToken = 0; - }else{ - assert(p && p->eType==FTSQUERY_PHRASE ); - if( iToken<(p->pPhrase->nToken-1) ){ - iToken++; - }else{ - iToken = 0; - while( p->pParent && p->pParent->pLeft!=p ){ - assert( p->pParent->pRight==p ); - p = p->pParent; - } - p = p->pParent; - if( p ){ - assert( p->pRight!=0 ); - p = p->pRight; - while( p->pLeft ){ - p = p->pLeft; - } - } - } - } - - *ppExpr = p; - *piToken = iToken; - return p?1:0; -} - -/* -** Return TRUE if the expression node pExpr is located beneath the -** RHS of a NOT operator. -*/ -static int fts3ExprBeneathNot(Fts3Expr *p){ - Fts3Expr *pParent; - while( p ){ - pParent = p->pParent; - if( pParent && pParent->eType==FTSQUERY_NOT && pParent->pRight==p ){ - return 1; - } - p = pParent; - } - return 0; -} - -/* -** Add entries to pSnippet->aMatch[] for every match that occurs against -** document zDoc[0..nDoc-1] which is stored in column iColumn. -*/ -static void snippetOffsetsOfColumn( - fulltext_cursor *pCur, /* The fulltest search cursor */ - Snippet *pSnippet, /* The Snippet object to be filled in */ - int iColumn, /* Index of fulltext table column */ - const char *zDoc, /* Text of the fulltext table column */ - int nDoc /* Length of zDoc in bytes */ -){ - const sqlite3_tokenizer_module *pTModule; /* The tokenizer module */ - sqlite3_tokenizer *pTokenizer; /* The specific tokenizer */ - sqlite3_tokenizer_cursor *pTCursor; /* Tokenizer cursor */ - fulltext_vtab *pVtab; /* The full text index */ - int nColumn; /* Number of columns in the index */ - int i, j; /* Loop counters */ - int rc; /* Return code */ - unsigned int match, prevMatch; /* Phrase search bitmasks */ - const char *zToken; /* Next token from the tokenizer */ - int nToken; /* Size of zToken */ - int iBegin, iEnd, iPos; /* Offsets of beginning and end */ - - /* The following variables keep a circular buffer of the last - ** few tokens */ - unsigned int iRotor = 0; /* Index of current token */ - int iRotorBegin[FTS3_ROTOR_SZ]; /* Beginning offset of token */ - int iRotorLen[FTS3_ROTOR_SZ]; /* Length of token */ - - pVtab = cursor_vtab(pCur); - nColumn = pVtab->nColumn; - pTokenizer = pVtab->pTokenizer; - pTModule = pTokenizer->pModule; - rc = pTModule->xOpen(pTokenizer, zDoc, nDoc, &pTCursor); - if( rc ) return; - pTCursor->pTokenizer = pTokenizer; - - prevMatch = 0; - while( !pTModule->xNext(pTCursor, &zToken, &nToken, &iBegin, &iEnd, &iPos) ){ - Fts3Expr *pIter = pCur->pExpr; - int iIter = -1; - iRotorBegin[iRotor&FTS3_ROTOR_MASK] = iBegin; - iRotorLen[iRotor&FTS3_ROTOR_MASK] = iEnd-iBegin; - match = 0; - for(i=0; i<(FTS3_ROTOR_SZ-1) && fts3NextExprToken(&pIter, &iIter); i++){ - int nPhrase; /* Number of tokens in current phrase */ - struct PhraseToken *pToken; /* Current token */ - int iCol; /* Column index */ - - if( fts3ExprBeneathNot(pIter) ) continue; - nPhrase = pIter->pPhrase->nToken; - pToken = &pIter->pPhrase->aToken[iIter]; - iCol = pIter->pPhrase->iColumn; - if( iCol>=0 && iColn>nToken ) continue; - if( !pToken->isPrefix && pToken->nn<=nToken ); - if( memcmp(pToken->z, zToken, pToken->n) ) continue; - if( iIter>0 && (prevMatch & (1<=0; j--){ - int k = (iRotor-j) & FTS3_ROTOR_MASK; - snippetAppendMatch(pSnippet, iColumn, i-j, iPos-j, - iRotorBegin[k], iRotorLen[k]); - } - } - } - prevMatch = match<<1; - iRotor++; - } - pTModule->xClose(pTCursor); -} - -/* -** Remove entries from the pSnippet structure to account for the NEAR -** operator. When this is called, pSnippet contains the list of token -** offsets produced by treating all NEAR operators as AND operators. -** This function removes any entries that should not be present after -** accounting for the NEAR restriction. For example, if the queried -** document is: -** -** "A B C D E A" -** -** and the query is: -** -** A NEAR/0 E -** -** then when this function is called the Snippet contains token offsets -** 0, 4 and 5. This function removes the "0" entry (because the first A -** is not near enough to an E). -** -** When this function is called, the value pointed to by parameter piLeft is -** the integer id of the left-most token in the expression tree headed by -** pExpr. This function increments *piLeft by the total number of tokens -** in the expression tree headed by pExpr. -** -** Return 1 if any trimming occurs. Return 0 if no trimming is required. -*/ -static int trimSnippetOffsets( - Fts3Expr *pExpr, /* The search expression */ - Snippet *pSnippet, /* The set of snippet offsets to be trimmed */ - int *piLeft /* Index of left-most token in pExpr */ -){ - if( pExpr ){ - if( trimSnippetOffsets(pExpr->pLeft, pSnippet, piLeft) ){ - return 1; - } - - switch( pExpr->eType ){ - case FTSQUERY_PHRASE: - *piLeft += pExpr->pPhrase->nToken; - break; - case FTSQUERY_NEAR: { - /* The right-hand-side of a NEAR operator is always a phrase. The - ** left-hand-side is either a phrase or an expression tree that is - ** itself headed by a NEAR operator. The following initializations - ** set local variable iLeft to the token number of the left-most - ** token in the right-hand phrase, and iRight to the right most - ** token in the same phrase. For example, if we had: - ** - **
    MATCH '"abc def" NEAR/2 "ghi jkl"' - ** - ** then iLeft will be set to 2 (token number of ghi) and nToken will - ** be set to 4. - */ - Fts3Expr *pLeft = pExpr->pLeft; - Fts3Expr *pRight = pExpr->pRight; - int iLeft = *piLeft; - int nNear = pExpr->nNear; - int nToken = pRight->pPhrase->nToken; - int jj, ii; - if( pLeft->eType==FTSQUERY_NEAR ){ - pLeft = pLeft->pRight; - } - assert( pRight->eType==FTSQUERY_PHRASE ); - assert( pLeft->eType==FTSQUERY_PHRASE ); - nToken += pLeft->pPhrase->nToken; - - for(ii=0; iinMatch; ii++){ - struct snippetMatch *p = &pSnippet->aMatch[ii]; - if( p->iTerm==iLeft ){ - int isOk = 0; - /* Snippet ii is an occurence of query term iLeft in the document. - ** It occurs at position (p->iToken) of the document. We now - ** search for an instance of token (iLeft-1) somewhere in the - ** range (p->iToken - nNear)...(p->iToken + nNear + nToken) within - ** the set of snippetMatch structures. If one is found, proceed. - ** If one cannot be found, then remove snippets ii..(ii+N-1) - ** from the matching snippets, where N is the number of tokens - ** in phrase pRight->pPhrase. - */ - for(jj=0; isOk==0 && jjnMatch; jj++){ - struct snippetMatch *p2 = &pSnippet->aMatch[jj]; - if( p2->iTerm==(iLeft-1) ){ - if( p2->iToken>=(p->iToken-nNear-1) - && p2->iToken<(p->iToken+nNear+nToken) - ){ - isOk = 1; - } - } - } - if( !isOk ){ - int kk; - for(kk=0; kkpPhrase->nToken; kk++){ - pSnippet->aMatch[kk+ii].iTerm = -2; - } - return 1; - } - } - if( p->iTerm==(iLeft-1) ){ - int isOk = 0; - for(jj=0; isOk==0 && jjnMatch; jj++){ - struct snippetMatch *p2 = &pSnippet->aMatch[jj]; - if( p2->iTerm==iLeft ){ - if( p2->iToken<=(p->iToken+nNear+1) - && p2->iToken>(p->iToken-nNear-nToken) - ){ - isOk = 1; - } - } - } - if( !isOk ){ - int kk; - for(kk=0; kkpPhrase->nToken; kk++){ - pSnippet->aMatch[ii-kk].iTerm = -2; - } - return 1; - } - } - } - break; - } - } - - if( trimSnippetOffsets(pExpr->pRight, pSnippet, piLeft) ){ - return 1; - } - } - return 0; -} - -/* -** Compute all offsets for the current row of the query. -** If the offsets have already been computed, this routine is a no-op. -*/ -static void snippetAllOffsets(fulltext_cursor *p){ - int nColumn; - int iColumn, i; - int iFirst, iLast; - int iTerm = 0; - fulltext_vtab *pFts = cursor_vtab(p); - - if( p->snippet.nMatch || p->pExpr==0 ){ - return; - } - nColumn = pFts->nColumn; - iColumn = (p->iCursorType - QUERY_FULLTEXT); - if( iColumn<0 || iColumn>=nColumn ){ - /* Look for matches over all columns of the full-text index */ - iFirst = 0; - iLast = nColumn-1; - }else{ - /* Look for matches in the iColumn-th column of the index only */ - iFirst = iColumn; - iLast = iColumn; - } - for(i=iFirst; i<=iLast; i++){ - const char *zDoc; - int nDoc; - zDoc = (const char*)sqlite3_column_text(p->pStmt, i+1); - nDoc = sqlite3_column_bytes(p->pStmt, i+1); - snippetOffsetsOfColumn(p, &p->snippet, i, zDoc, nDoc); - } - - while( trimSnippetOffsets(p->pExpr, &p->snippet, &iTerm) ){ - iTerm = 0; - } -} - -/* -** Convert the information in the aMatch[] array of the snippet -** into the string zOffset[0..nOffset-1]. This string is used as -** the return of the SQL offsets() function. -*/ -static void snippetOffsetText(Snippet *p){ - int i; - int cnt = 0; - StringBuffer sb; - char zBuf[200]; - if( p->zOffset ) return; - initStringBuffer(&sb); - for(i=0; inMatch; i++){ - struct snippetMatch *pMatch = &p->aMatch[i]; - if( pMatch->iTerm>=0 ){ - /* If snippetMatch.iTerm is less than 0, then the match was - ** discarded as part of processing the NEAR operator (see the - ** trimSnippetOffsetsForNear() function for details). Ignore - ** it in this case - */ - zBuf[0] = ' '; - sqlite3_snprintf(sizeof(zBuf)-1, &zBuf[cnt>0], "%d %d %d %d", - pMatch->iCol, pMatch->iTerm, pMatch->iStart, pMatch->nByte); - append(&sb, zBuf); - cnt++; - } - } - p->zOffset = stringBufferData(&sb); - p->nOffset = stringBufferLength(&sb); -} - -/* -** zDoc[0..nDoc-1] is phrase of text. aMatch[0..nMatch-1] are a set -** of matching words some of which might be in zDoc. zDoc is column -** number iCol. -** -** iBreak is suggested spot in zDoc where we could begin or end an -** excerpt. Return a value similar to iBreak but possibly adjusted -** to be a little left or right so that the break point is better. -*/ -static int wordBoundary( - int iBreak, /* The suggested break point */ - const char *zDoc, /* Document text */ - int nDoc, /* Number of bytes in zDoc[] */ - struct snippetMatch *aMatch, /* Matching words */ - int nMatch, /* Number of entries in aMatch[] */ - int iCol /* The column number for zDoc[] */ -){ - int i; - if( iBreak<=10 ){ - return 0; - } - if( iBreak>=nDoc-10 ){ - return nDoc; - } - for(i=0; i0 && aMatch[i-1].iStart+aMatch[i-1].nByte>=iBreak ){ - return aMatch[i-1].iStart; - } - } - for(i=1; i<=10; i++){ - if( safe_isspace(zDoc[iBreak-i]) ){ - return iBreak - i + 1; - } - if( safe_isspace(zDoc[iBreak+i]) ){ - return iBreak + i + 1; - } - } - return iBreak; -} - - - -/* -** Allowed values for Snippet.aMatch[].snStatus -*/ -#define SNIPPET_IGNORE 0 /* It is ok to omit this match from the snippet */ -#define SNIPPET_DESIRED 1 /* We want to include this match in the snippet */ - -/* -** Generate the text of a snippet. -*/ -static void snippetText( - fulltext_cursor *pCursor, /* The cursor we need the snippet for */ - const char *zStartMark, /* Markup to appear before each match */ - const char *zEndMark, /* Markup to appear after each match */ - const char *zEllipsis /* Ellipsis mark */ -){ - int i, j; - struct snippetMatch *aMatch; - int nMatch; - int nDesired; - StringBuffer sb; - int tailCol; - int tailOffset; + Fts3Hash *pHash = (Fts3Hash *)pAux; + Fts3Table *p; /* Pointer to allocated vtab */ + int rc; /* Return code */ + int i; /* Iterator variable */ + int nByte; /* Size of allocation used for *p */ int iCol; - int nDoc; - const char *zDoc; - int iStart, iEnd; - int tailEllipsis = 0; - int iMatch; - + int nString = 0; + int nCol = 0; + char *zCsr; + int nDb; + int nName; - sqlite3_free(pCursor->snippet.zSnippet); - pCursor->snippet.zSnippet = 0; - aMatch = pCursor->snippet.aMatch; - nMatch = pCursor->snippet.nMatch; - initStringBuffer(&sb); + const char *zTokenizer = 0; /* Name of tokenizer to use */ + sqlite3_tokenizer *pTokenizer = 0; /* Tokenizer for this table */ - for(i=0; i0; i++){ - if( aMatch[i].snStatus!=SNIPPET_DESIRED ) continue; - nDesired--; - iCol = aMatch[i].iCol; - zDoc = (const char*)sqlite3_column_text(pCursor->pStmt, iCol+1); - nDoc = sqlite3_column_bytes(pCursor->pStmt, iCol+1); - iStart = aMatch[i].iStart - 40; - iStart = wordBoundary(iStart, zDoc, nDoc, aMatch, nMatch, iCol); - if( iStart<=10 ){ - iStart = 0; + if( nCol==0 ){ + nCol = 1; + } + + /* Allocate and populate the Fts3Table structure. */ + nByte = sizeof(Fts3Table) + /* Fts3Table */ + nCol * sizeof(char *) + /* azColumn */ + nName + /* zName */ + nDb + /* zDb */ + nString; /* Space for azColumn strings */ + p = (Fts3Table*)sqlite3_malloc(nByte); + if( p==0 ){ + rc = SQLITE_NOMEM; + goto fts3_init_out; + } + memset(p, 0, nByte); + + p->db = db; + p->nColumn = nCol; + p->nPendingData = 0; + p->azColumn = (char **)&p[1]; + p->pTokenizer = pTokenizer; + p->nNodeSize = 1000; + zCsr = (char *)&p->azColumn[nCol]; + + fts3HashInit(&p->pendingTerms, FTS3_HASH_STRING, 1); + + /* Fill in the zName and zDb fields of the vtab structure. */ + p->zName = zCsr; + memcpy(zCsr, argv[2], nName); + zCsr += nName; + p->zDb = zCsr; + memcpy(zCsr, argv[1], nDb); + zCsr += nDb; + + /* Fill in the azColumn array */ + iCol = 0; + for(i=3; iazColumn[iCol++] = zCsr; + zCsr += n+1; + assert( zCsr <= &((char *)p)[nByte] ); } - if( iCol==tailCol && iStart<=tailOffset+20 ){ - iStart = tailOffset; - } - if( (iCol!=tailCol && tailCol>=0) || iStart!=tailOffset ){ - trimWhiteSpace(&sb); - appendWhiteSpace(&sb); - append(&sb, zEllipsis); - appendWhiteSpace(&sb); - } - iEnd = aMatch[i].iStart + aMatch[i].nByte + 40; - iEnd = wordBoundary(iEnd, zDoc, nDoc, aMatch, nMatch, iCol); - if( iEnd>=nDoc-10 ){ - iEnd = nDoc; - tailEllipsis = 0; + } + if( iCol==0 ){ + assert( nCol==1 ); + p->azColumn[0] = "content"; + } + + /* If this is an xCreate call, create the underlying tables in the + ** database. TODO: For xConnect(), it could verify that said tables exist. + */ + if( isCreate ){ + rc = fts3CreateTables(p); + if( rc!=SQLITE_OK ) goto fts3_init_out; + } + + rc = fts3DeclareVtab(p); + if( rc!=SQLITE_OK ) goto fts3_init_out; + +#ifdef SQLITE_TEST + if( zTestParam ){ + p->nNodeSize = atoi(&zTestParam[5]); + } +#endif + *ppVTab = &p->base; + +fts3_init_out: + assert( p || (pTokenizer && rc!=SQLITE_OK) ); + if( rc!=SQLITE_OK ){ + if( p ){ + fts3DisconnectMethod((sqlite3_vtab *)p); }else{ - tailEllipsis = 1; + pTokenizer->pModule->xDestroy(pTokenizer); } - while( iMatchsnippet.zSnippet = stringBufferData(&sb); - pCursor->snippet.nSnippet = stringBufferLength(&sb); + return rc; } +/* +** The xConnect() and xCreate() methods for the virtual table. All the +** work is done in function fts3InitVtab(). +*/ +static int fts3ConnectMethod( + sqlite3 *db, /* Database connection */ + void *pAux, /* Pointer to tokenizer hash table */ + int argc, /* Number of elements in argv array */ + const char * const *argv, /* xCreate/xConnect argument array */ + sqlite3_vtab **ppVtab, /* OUT: New sqlite3_vtab object */ + char **pzErr /* OUT: sqlite3_malloc'd error message */ +){ + return fts3InitVtab(0, db, pAux, argc, argv, ppVtab, pzErr); +} +static int fts3CreateMethod( + sqlite3 *db, /* Database connection */ + void *pAux, /* Pointer to tokenizer hash table */ + int argc, /* Number of elements in argv array */ + const char * const *argv, /* xCreate/xConnect argument array */ + sqlite3_vtab **ppVtab, /* OUT: New sqlite3_vtab object */ + char **pzErr /* OUT: sqlite3_malloc'd error message */ +){ + return fts3InitVtab(1, db, pAux, argc, argv, ppVtab, pzErr); +} + +/* +** Implementation of the xBestIndex method for FTS3 tables. There +** are three possible strategies, in order of preference: +** +** 1. Direct lookup by rowid or docid. +** 2. Full-text search using a MATCH operator on a non-docid column. +** 3. Linear scan of %_content table. +*/ +static int fts3BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){ + Fts3Table *p = (Fts3Table *)pVTab; + int i; /* Iterator variable */ + int iCons = -1; /* Index of constraint to use */ + + /* By default use a full table scan. This is an expensive option, + ** so search through the constraints to see if a more efficient + ** strategy is possible. + */ + pInfo->idxNum = FTS3_FULLSCAN_SEARCH; + pInfo->estimatedCost = 500000; + for(i=0; inConstraint; i++){ + struct sqlite3_index_constraint *pCons = &pInfo->aConstraint[i]; + if( pCons->usable==0 ) continue; + + /* A direct lookup on the rowid or docid column. This is the best + ** strategy in all cases. Assign a cost of 1.0 and return early. + */ + if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ + && (pCons->iColumn<0 || pCons->iColumn==p->nColumn+1 ) + ){ + pInfo->idxNum = FTS3_DOCID_SEARCH; + pInfo->estimatedCost = 1.0; + iCons = i; + break; + } + + /* A MATCH constraint. Use a full-text search. + ** + ** If there is more than one MATCH constraint available, use the first + ** one encountered. If there is both a MATCH constraint and a direct + ** rowid/docid lookup, prefer the rowid/docid strategy. + */ + if( iCons<0 + && pCons->op==SQLITE_INDEX_CONSTRAINT_MATCH + && pCons->iColumn>=0 && pCons->iColumn<=p->nColumn + ){ + pInfo->idxNum = FTS3_FULLTEXT_SEARCH + pCons->iColumn; + pInfo->estimatedCost = 2.0; + iCons = i; + } + } + + if( iCons>=0 ){ + pInfo->aConstraintUsage[iCons].argvIndex = 1; + pInfo->aConstraintUsage[iCons].omit = 1; + } + return SQLITE_OK; +} + +/* +** Implementation of xOpen method. +*/ +static int fts3OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){ + sqlite3_vtab_cursor *pCsr; /* Allocated cursor */ + + UNUSED_PARAMETER(pVTab); + + /* Allocate a buffer large enough for an Fts3Cursor structure. If the + ** allocation succeeds, zero it and return SQLITE_OK. Otherwise, + ** if the allocation fails, return SQLITE_NOMEM. + */ + *ppCsr = pCsr = (sqlite3_vtab_cursor *)sqlite3_malloc(sizeof(Fts3Cursor)); + if( !pCsr ){ + return SQLITE_NOMEM; + } + memset(pCsr, 0, sizeof(Fts3Cursor)); + return SQLITE_OK; +} + +/****************************************************************/ +/****************************************************************/ +/****************************************************************/ +/****************************************************************/ + /* ** Close the cursor. For additional information see the documentation ** on the xClose method of the virtual table interface. */ static int fulltextClose(sqlite3_vtab_cursor *pCursor){ - fulltext_cursor *c = (fulltext_cursor *) pCursor; - FTSTRACE(("FTS3 Close %p\n", c)); - sqlite3_finalize(c->pStmt); - sqlite3Fts3ExprFree(c->pExpr); - snippetClear(&c->snippet); - if( c->result.nData!=0 ){ - dlrDestroy(&c->reader); - } - dataBufferDestroy(&c->result); - sqlite3_free(c); + Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; + sqlite3_finalize(pCsr->pStmt); + sqlite3Fts3ExprFree(pCsr->pExpr); + sqlite3_free(pCsr->aDoclist); + sqlite3_free(pCsr); return SQLITE_OK; } -static int fulltextNext(sqlite3_vtab_cursor *pCursor){ - fulltext_cursor *c = (fulltext_cursor *) pCursor; - int rc; - - FTSTRACE(("FTS3 Next %p\n", pCursor)); - snippetClear(&c->snippet); - if( c->iCursorType < QUERY_FULLTEXT ){ - /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */ - rc = sqlite3_step(c->pStmt); - switch( rc ){ - case SQLITE_ROW: - c->eof = 0; - return SQLITE_OK; - case SQLITE_DONE: - c->eof = 1; - return SQLITE_OK; - default: - c->eof = 1; - return rc; - } - } else { /* full-text query */ - rc = sqlite3_reset(c->pStmt); - if( rc!=SQLITE_OK ) return rc; - - if( c->result.nData==0 || dlrAtEnd(&c->reader) ){ - c->eof = 1; +static int fts3CursorSeek(Fts3Cursor *pCsr){ + if( pCsr->isRequireSeek ){ + pCsr->isRequireSeek = 0; + sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId); + if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){ return SQLITE_OK; + }else{ + int rc; + pCsr->isEof = 1; + if( SQLITE_OK==(rc = sqlite3_reset(pCsr->pStmt)) ){ + rc = SQLITE_ERROR; + } + return rc; } - rc = sqlite3_bind_int64(c->pStmt, 1, dlrDocid(&c->reader)); - dlrStep(&c->reader); - if( rc!=SQLITE_OK ) return rc; - /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */ - rc = sqlite3_step(c->pStmt); - if( rc==SQLITE_ROW ){ /* the case we expect */ - c->eof = 0; - return SQLITE_OK; - } - /* an error occurred; abort */ - return rc==SQLITE_DONE ? SQLITE_ERROR : rc; + }else{ + return SQLITE_OK; } } +static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){ + int rc = SQLITE_OK; /* Return code */ + Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; -/* TODO(shess) If we pushed LeafReader to the top of the file, or to -** another file, term_select() could be pushed above -** docListOfTerm(). -*/ -static int termSelect(fulltext_vtab *v, int iColumn, - const char *pTerm, int nTerm, int isPrefix, - DocListType iType, DataBuffer *out); - -/* -** Return a DocList corresponding to the phrase *pPhrase. -** -** The resulting DL_DOCIDS doclist is stored in pResult, which is -** overwritten. -*/ -static int docListOfPhrase( - fulltext_vtab *pTab, /* The full text index */ - Fts3Phrase *pPhrase, /* Phrase to return a doclist corresponding to */ - DocListType eListType, /* Either DL_DOCIDS or DL_POSITIONS */ - DataBuffer *pResult /* Write the result here */ -){ - int ii; - int rc = SQLITE_OK; - int iCol = pPhrase->iColumn; - DocListType eType = eListType; - assert( eType==DL_POSITIONS || eType==DL_DOCIDS ); - if( pPhrase->nToken>1 ){ - eType = DL_POSITIONS; - } - - /* This code should never be called with buffered updates. */ - assert( pTab->nPendingData<0 ); - - for(ii=0; rc==SQLITE_OK && iinToken; ii++){ - DataBuffer tmp; - struct PhraseToken *p = &pPhrase->aToken[ii]; - rc = termSelect(pTab, iCol, p->z, p->n, p->isPrefix, eType, &tmp); - if( rc==SQLITE_OK ){ - if( ii==0 ){ - *pResult = tmp; - }else{ - DataBuffer res = *pResult; - dataBufferInit(pResult, 0); - if( ii==(pPhrase->nToken-1) ){ - eType = eListType; - } - docListPhraseMerge( - res.pData, res.nData, tmp.pData, tmp.nData, 0, 0, eType, pResult - ); - dataBufferDestroy(&res); - dataBufferDestroy(&tmp); - } + if( pCsr->aDoclist==0 ){ + if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){ + pCsr->isEof = 1; + rc = sqlite3_reset(pCsr->pStmt); } + }else if( pCsr->pNextId>=&pCsr->aDoclist[pCsr->nDoclist] ){ + pCsr->isEof = 1; + }else{ + sqlite3_reset(pCsr->pStmt); + fts3GetDeltaVarint(&pCsr->pNextId, &pCsr->iPrevId); + pCsr->isRequireSeek = 1; } + return rc; +} + +/* +** The buffer pointed to by argument zNode (size nNode bytes) contains the +** root node of a b-tree segment. The segment is guaranteed to be at least +** one level high (i.e. the root node is not also a leaf). If successful, +** this function locates the leaf node of the segment that may contain the +** term specified by arguments zTerm and nTerm and writes its block number +** to *piLeaf. +** +** It is possible that the returned leaf node does not contain the specified +** term. However, if the segment does contain said term, it is stored on +** the identified leaf node. Because this function only inspects interior +** segment nodes (and never loads leaf nodes into memory), it is not possible +** to be sure. +** +** If an error occurs, an error code other than SQLITE_OK is returned. +*/ +static int fts3SelectLeaf( + Fts3Table *p, /* Virtual table handle */ + const char *zTerm, /* Term to select leaves for */ + int nTerm, /* Size of term zTerm in bytes */ + const char *zNode, /* Buffer containing segment interior node */ + int nNode, /* Size of buffer at zNode */ + sqlite3_int64 *piLeaf /* Selected leaf node */ +){ + int rc = SQLITE_OK; /* Return code */ + const char *zCsr = zNode; /* Cursor to iterate through node */ + const char *zEnd = &zCsr[nNode];/* End of interior node buffer */ + char *zBuffer = 0; /* Buffer to load terms into */ + int nAlloc = 0; /* Size of allocated buffer */ + + while( 1 ){ + int isFirstTerm = 1; /* True when processing first term on page */ + int iHeight; /* Height of this node in tree */ + sqlite3_int64 iChild; /* Block id of child node to descend to */ + int nBlock; /* Size of child node in bytes */ + + zCsr += sqlite3Fts3GetVarint32(zCsr, &iHeight); + zCsr += sqlite3Fts3GetVarint(zCsr, &iChild); + + while( zCsrnAlloc ){ + char *zNew; + nAlloc = (nPrefix+nSuffix) * 2; + zNew = (char *)sqlite3_realloc(zBuffer, nAlloc); + if( !zNew ){ + sqlite3_free(zBuffer); + return SQLITE_NOMEM; + } + zBuffer = zNew; + } + memcpy(&zBuffer[nPrefix], zCsr, nSuffix); + nBuffer = nPrefix + nSuffix; + zCsr += nSuffix; + + /* Compare the term we are searching for with the term just loaded from + ** the interior node. If the specified term is greater than or equal + ** to the term from the interior node, then all terms on the sub-tree + ** headed by node iChild are smaller than zTerm. No need to search + ** iChild. + ** + ** If the interior node term is larger than the specified term, then + ** the tree headed by iChild may contain the specified term. + */ + cmp = memcmp(zTerm, zBuffer, (nBuffer>nTerm ? nTerm : nBuffer)); + if( cmp<0 || (cmp==0 && nBuffer>nTerm) ) break; + iChild++; + }; + + /* If (iHeight==1), the children of this interior node are leaves. The + ** specified term may be present on leaf node iChild. + */ + if( iHeight==1 ){ + *piLeaf = iChild; + break; + } + + /* Descend to interior node iChild. */ + rc = sqlite3Fts3ReadBlock(p, iChild, &zCsr, &nBlock); + if( rc!=SQLITE_OK ) break; + zEnd = &zCsr[nBlock]; + } + sqlite3_free(zBuffer); return rc; } /* -** Evaluate the full-text expression pExpr against fts3 table pTab. Write -** the results into pRes. +** This function is used to create delta-encoded serialized lists of FTS3 +** varints. Each call to this function appends a single varint to a list. +*/ +static void fts3PutDeltaVarint( + char **pp, /* IN/OUT: Output pointer */ + sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */ + sqlite3_int64 iVal /* Write this value to the list */ +){ + assert( iVal-*piPrev > 0 || (*piPrev==0 && iVal==0) ); + *pp += sqlite3Fts3PutVarint(*pp, iVal-*piPrev); + *piPrev = iVal; +} + +static void fts3PoslistCopy(char **pp, char **ppPoslist){ + char *pEnd = *ppPoslist; + char c = 0; + while( *pEnd | c ) c = *pEnd++ & 0x80; + pEnd++; + if( pp ){ + int n = (int)(pEnd - *ppPoslist); + char *p = *pp; + memcpy(p, *ppPoslist, n); + p += n; + *pp = p; + } + *ppPoslist = pEnd; +} + +static void fts3ColumnlistCopy(char **pp, char **ppPoslist){ + char *pEnd = *ppPoslist; + char c = 0; + + /* A column-list is terminated by either a 0x01 or 0x00. */ + while( 0xFE & (*pEnd | c) ) c = *pEnd++ & 0x80; + if( pp ){ + int n = (int)(pEnd - *ppPoslist); + char *p = *pp; + memcpy(p, *ppPoslist, n); + p += n; + *pp = p; + } + *ppPoslist = pEnd; +} + +/* +** Value used to signify the end of an offset-list. This is safe because +** it is not possible to have a document with 2^31 terms. +*/ +#define OFFSET_LIST_END 0x7fffffff + +/* +** This function is used to help parse offset-lists. When this function is +** called, *pp may point to the start of the next varint in the offset-list +** being parsed, or it may point to 1 byte past the end of the offset-list +** (in which case **pp will be 0x00 or 0x01). +** +** If *pp points past the end of the current offset list, set *pi to +** OFFSET_LIST_END and return. Otherwise, read the next varint from *pp, +** increment the current value of *pi by the value read, and set *pp to +** point to the next value before returning. +*/ +static void fts3ReadNextPos( + char **pp, /* IN/OUT: Pointer into offset-list buffer */ + sqlite3_int64 *pi /* IN/OUT: Value read from offset-list */ +){ + if( **pp&0xFE ){ + fts3GetDeltaVarint(pp, pi); + *pi -= 2; + }else{ + *pi = OFFSET_LIST_END; + } +} + +/* +** If parameter iCol is not 0, write an 0x01 byte followed by the value of +** iCol encoded as a varint to *pp. +** +** Set *pp to point to the byte just after the last byte written before +** returning (do not modify it if iCol==0). Return the total number of bytes +** written (0 if iCol==0). +*/ +static int fts3PutColNumber(char **pp, int iCol){ + int n = 0; /* Number of bytes written */ + if( iCol ){ + char *p = *pp; /* Output pointer */ + n = 1 + sqlite3Fts3PutVarint(&p[1], iCol); + *p = 0x01; + *pp = &p[n]; + } + return n; +} + +/* +** +*/ +static void fts3PoslistMerge( + char **pp, /* Output buffer */ + char **pp1, /* Left input list */ + char **pp2 /* Right input list */ +){ + char *p = *pp; + char *p1 = *pp1; + char *p2 = *pp2; + + while( *p1 || *p2 ){ + int iCol1; + int iCol2; + + if( *p1==0x01 ) sqlite3Fts3GetVarint32(&p1[1], &iCol1); + else if( *p1==0x00 ) iCol1 = OFFSET_LIST_END; + else iCol1 = 0; + + if( *p2==0x01 ) sqlite3Fts3GetVarint32(&p2[1], &iCol2); + else if( *p2==0x00 ) iCol2 = OFFSET_LIST_END; + else iCol2 = 0; + + if( iCol1==iCol2 ){ + sqlite3_int64 i1 = 0; + sqlite3_int64 i2 = 0; + sqlite3_int64 iPrev = 0; + int n = fts3PutColNumber(&p, iCol1); + p1 += n; + p2 += n; + + /* At this point, both p1 and p2 point to the start of offset-lists. + ** An offset-list is a list of non-negative delta-encoded varints, each + ** incremented by 2 before being stored. Each list is terminated by a 0 + ** or 1 value (0x00 or 0x01). The following block merges the two lists + ** and writes the results to buffer p. p is left pointing to the byte + ** after the list written. No terminator (0x00 or 0x01) is written to + ** the output. + */ + fts3GetDeltaVarint(&p1, &i1); + fts3GetDeltaVarint(&p2, &i2); + do { + fts3PutDeltaVarint(&p, &iPrev, (i1iPos1 && iPos2<=iPos1+nToken ){ + sqlite3_int64 iSave; + if( !pp ){ + fts3PoslistCopy(0, &p2); + fts3PoslistCopy(0, &p1); + *pp1 = p1; + *pp2 = p2; + return 1; + } + iSave = isSaveLeft ? iPos1 : iPos2; + fts3PutDeltaVarint(&p, &iPrev, iSave+2); iPrev -= 2; + pSave = 0; + } + if( (!isSaveLeft && iPos2<=(iPos1+nToken)) || iPos2<=iPos1 ){ + if( (*p2&0xFE)==0 ) break; + fts3GetDeltaVarint(&p2, &iPos2); iPos2 -= 2; + }else{ + if( (*p1&0xFE)==0 ) break; + fts3GetDeltaVarint(&p1, &iPos1); iPos1 -= 2; + } + + } + if( pSave && pp ){ + p = pSave; + } + + fts3ColumnlistCopy(0, &p1); + fts3ColumnlistCopy(0, &p2); + assert( (*p1&0xFE)==0 && (*p2&0xFE)==0 ); + if( 0==*p1 || 0==*p2 ) break; + + p1++; + p1 += sqlite3Fts3GetVarint32(p1, &iCol1); + p2++; + p2 += sqlite3Fts3GetVarint32(p2, &iCol2); + } + + /* Advance pointer p1 or p2 (whichever corresponds to the smaller of + ** iCol1 and iCol2) so that it points to either the 0x00 that marks the + ** end of the position list, or the 0x01 that precedes the next + ** column-number in the position list. + */ + else if( iCol1 D */ +#define MERGE_AND 3 /* D + D -> D */ +#define MERGE_OR 4 /* D + D -> D */ +#define MERGE_POS_OR 5 /* P + P -> P */ +#define MERGE_PHRASE 6 /* P + P -> D */ +#define MERGE_POS_PHRASE 7 /* P + P -> P */ +#define MERGE_NEAR 8 /* P + P -> D */ +#define MERGE_POS_NEAR 9 /* P + P -> P */ + +/* +** Merge the two doclists passed in buffer a1 (size n1 bytes) and a2 +** (size n2 bytes). The output is written to pre-allocated buffer aBuffer, +** which is guaranteed to be large enough to hold the results. The number +** of bytes written to aBuffer is stored in *pnBuffer before returning. +** +** If successful, SQLITE_OK is returned. Otherwise, if a malloc error +** occurs while allocating a temporary buffer as part of the merge operation, +** SQLITE_NOMEM is returned. +*/ +static int fts3DoclistMerge( + int mergetype, /* One of the MERGE_XXX constants */ + int nParam1, /* Used by MERGE_NEAR and MERGE_POS_NEAR */ + int nParam2, /* Used by MERGE_NEAR and MERGE_POS_NEAR */ + char *aBuffer, /* Pre-allocated output buffer */ + int *pnBuffer, /* OUT: Bytes written to aBuffer */ + char *a1, /* Buffer containing first doclist */ + int n1, /* Size of buffer a1 */ + char *a2, /* Buffer containing second doclist */ + int n2 /* Size of buffer a2 */ +){ + sqlite3_int64 i1 = 0; + sqlite3_int64 i2 = 0; + sqlite3_int64 iPrev = 0; + + char *p = aBuffer; + char *p1 = a1; + char *p2 = a2; + char *pEnd1 = &a1[n1]; + char *pEnd2 = &a2[n2]; + + assert( mergetype==MERGE_OR || mergetype==MERGE_POS_OR + || mergetype==MERGE_AND || mergetype==MERGE_NOT + || mergetype==MERGE_PHRASE || mergetype==MERGE_POS_PHRASE + || mergetype==MERGE_NEAR || mergetype==MERGE_POS_NEAR + ); + + if( !aBuffer ){ + return SQLITE_NOMEM; + } + if( n1==0 && n2==0 ){ + *pnBuffer = 0; + return SQLITE_OK; + } + + /* Read the first docid from each doclist */ + fts3GetDeltaVarint2(&p1, pEnd1, &i1); + fts3GetDeltaVarint2(&p2, pEnd2, &i2); + + switch( mergetype ){ + case MERGE_OR: + case MERGE_POS_OR: + while( p1 || p2 ){ + if( p2 && p1 && i1==i2 ){ + fts3PutDeltaVarint(&p, &iPrev, i1); + if( mergetype==MERGE_POS_OR ) fts3PoslistMerge(&p, &p1, &p2); + fts3GetDeltaVarint2(&p1, pEnd1, &i1); + fts3GetDeltaVarint2(&p2, pEnd2, &i2); + }else if( !p2 || (p1 && i1nOutput + nDoclist; + char *aNew = sqlite3_malloc(nNew); + + UNUSED_PARAMETER(p); + UNUSED_PARAMETER(zTerm); + UNUSED_PARAMETER(nTerm); + + if( !aNew ){ + return SQLITE_NOMEM; + } + + if( pTS->nOutput==0 ){ + /* If this is the first term selected, copy the doclist to the output + ** buffer using memcpy(). TODO: Add a way to transfer control of the + ** aDoclist buffer from the caller so as to avoid the memcpy(). + */ + memcpy(aNew, aDoclist, nDoclist); + }else{ + /* The output buffer is not empty. Merge doclist aDoclist with the + ** existing output. This can only happen with prefix-searches (as + ** searches for exact terms return exactly one doclist). + */ + int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR); + fts3DoclistMerge(mergetype, 0, 0, + aNew, &nNew, pTS->aOutput, pTS->nOutput, aDoclist, nDoclist + ); + } + + sqlite3_free(pTS->aOutput); + pTS->aOutput = aNew; + pTS->nOutput = nNew; + + return SQLITE_OK; +} + +/* +** This function retreives the doclist for the specified term (or term +** prefix) from the database. +** +** The returned doclist may be in one of two formats, depending on the +** value of parameter isReqPos. If isReqPos is zero, then the doclist is +** a sorted list of delta-compressed docids. If isReqPos is non-zero, +** then the returned list is in the same format as is stored in the +** database without the found length specifier at the start of on-disk +** doclists. +*/ +static int fts3TermSelect( + Fts3Table *p, /* Virtual table handle */ + int iColumn, /* Column to query (or -ve for all columns) */ + const char *zTerm, /* Term to query for */ + int nTerm, /* Size of zTerm in bytes */ + int isPrefix, /* True for a prefix search */ + int isReqPos, /* True to include position lists in output */ + int *pnOut, /* OUT: Size of buffer at *ppOut */ + char **ppOut /* OUT: Malloced result buffer */ +){ + int i; + TermSelect tsc; + Fts3SegFilter filter; /* Segment term filter configuration */ + Fts3SegReader **apSegment = 0; /* Array of segments to read data from */ + int nSegment = 0; /* Size of apSegment array */ + int nAlloc = 0; /* Allocated size of segment array */ + int rc; /* Return code */ + sqlite3_stmt *pStmt; /* SQL statement to scan %_segdir table */ + int iAge = 0; /* Used to assign ages to segments */ + + /* Loop through the entire %_segdir table. For each segment, create a + ** Fts3SegReader to iterate through the subset of the segment leaves + ** that may contain a term that matches zTerm/nTerm. For non-prefix + ** searches, this is always a single leaf. For prefix searches, this + ** may be a contiguous block of leaves. + ** + ** The code in this loop does not actually load any leaves into memory + ** (unless the root node happens to be a leaf). It simply examines the + ** b-tree structure to determine which leaves need to be inspected. + */ + rc = sqlite3Fts3AllSegdirs(p, &pStmt); + while( rc==SQLITE_OK && SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){ + Fts3SegReader *pNew = 0; + int nRoot = sqlite3_column_bytes(pStmt, 4); + char const *zRoot = sqlite3_column_blob(pStmt, 4); + if( sqlite3_column_int64(pStmt, 1)==0 ){ + /* The entire segment is stored on the root node (which must be a + ** leaf). Do not bother inspecting any data in this case, just + ** create a Fts3SegReader to scan the single leaf. + */ + rc = sqlite3Fts3SegReaderNew(p, iAge, 0, 0, 0, zRoot, nRoot, &pNew); + }else{ + int rc2; /* Return value of sqlite3Fts3ReadBlock() */ + sqlite3_int64 i1; /* Blockid of leaf that may contain zTerm */ + rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &i1); + if( rc==SQLITE_OK ){ + sqlite3_int64 i2 = sqlite3_column_int64(pStmt, 2); + rc = sqlite3Fts3SegReaderNew(p, iAge, i1, i2, 0, 0, 0, &pNew); + } + + /* The following call to ReadBlock() serves to reset the SQL statement + ** used to retrieve blocks of data from the %_segments table. If it is + ** not reset here, then it may remain classified as an active statement + ** by SQLite, which may lead to "DROP TABLE" or "DETACH" commands + ** failing. + */ + rc2 = sqlite3Fts3ReadBlock(p, 0, 0, 0); + if( rc==SQLITE_OK ){ + rc = rc2; + } + } + iAge++; + + /* If a new Fts3SegReader was allocated, add it to the apSegment array. */ + assert( pNew!=0 || rc!=SQLITE_OK ); + if( pNew ){ + if( nSegment==nAlloc ){ + Fts3SegReader **pArray; + nAlloc += 16; + pArray = (Fts3SegReader **)sqlite3_realloc( + apSegment, nAlloc*sizeof(Fts3SegReader *) + ); + if( !pArray ){ + sqlite3Fts3SegReaderFree(p, pNew); + rc = SQLITE_NOMEM; + goto finished; + } + apSegment = pArray; + } + apSegment[nSegment++] = pNew; + } + } + if( rc!=SQLITE_DONE ){ + assert( rc!=SQLITE_OK ); + goto finished; + } + + memset(&tsc, 0, sizeof(TermSelect)); + tsc.isReqPos = isReqPos; + + filter.flags = FTS3_SEGMENT_IGNORE_EMPTY + | (isPrefix ? FTS3_SEGMENT_PREFIX : 0) + | (isReqPos ? FTS3_SEGMENT_REQUIRE_POS : 0) + | (iColumnnColumn ? FTS3_SEGMENT_COLUMN_FILTER : 0); + filter.iCol = iColumn; + filter.zTerm = zTerm; + filter.nTerm = nTerm; + + rc = sqlite3Fts3SegReaderIterate(p, apSegment, nSegment, &filter, + fts3TermSelectCb, (void *)&tsc + ); + + if( rc==SQLITE_OK ){ + *ppOut = tsc.aOutput; + *pnOut = tsc.nOutput; + }else{ + sqlite3_free(tsc.aOutput); + } + +finished: + sqlite3_reset(pStmt); + for(i=0; iiColumn; + int isTermPos = (pPhrase->nToken>1 || isReqPos); + + assert( p->nPendingData==0 ); + + for(ii=0; iinToken; ii++){ + struct PhraseToken *pTok = &pPhrase->aToken[ii]; + char *z = pTok->z; /* Next token of the phrase */ + int n = pTok->n; /* Size of z in bytes */ + int isPrefix = pTok->isPrefix;/* True if token is a prefix */ + char *pList; /* Pointer to token doclist */ + int nList; /* Size of buffer at pList */ + + rc = fts3TermSelect(p, iCol, z, n, isPrefix, isTermPos, &nList, &pList); + if( rc!=SQLITE_OK ) break; + + if( ii==0 ){ + pOut = pList; + nOut = nList; + }else{ + /* Merge the new term list and the current output. If this is the + ** last term in the phrase, and positions are not required in the + ** output of this function, the positions can be dropped as part + ** of this merge. Either way, the result of this merge will be + ** smaller than nList bytes. The code in fts3DoclistMerge() is written + ** so that it is safe to use pList as the output as well as an input + ** in this case. + */ + int mergetype = MERGE_POS_PHRASE; + if( ii==pPhrase->nToken-1 && !isReqPos ){ + mergetype = MERGE_PHRASE; + } + fts3DoclistMerge(mergetype, 0, 0, pList, &nOut, pOut, nOut, pList, nList); + sqlite3_free(pOut); + pOut = pList; + } + } + + if( rc==SQLITE_OK ){ + *paOut = pOut; + *pnOut = nOut; + }else{ + sqlite3_free(pOut); + } + return rc; +} + +/* +** Evaluate the full-text expression pExpr against fts3 table pTab. Store +** the resulting doclist in *paOut and *pnOut. */ static int evalFts3Expr( - fulltext_vtab *pTab, /* Fts3 Virtual table object */ - Fts3Expr *pExpr, /* Parsed fts3 expression */ - DataBuffer *pRes /* OUT: Write results of the expression here */ + Fts3Table *p, /* Virtual table handle */ + Fts3Expr *pExpr, /* Parsed fts3 expression */ + char **paOut, /* OUT: Pointer to malloc'd result buffer */ + int *pnOut /* OUT: Size of buffer at *paOut */ ){ - int rc = SQLITE_OK; + int rc = SQLITE_OK; /* Return code */ - /* Initialize the output buffer. If this is an empty query (pExpr==0), - ** this is all that needs to be done. Empty queries produce empty - ** result sets. - */ - dataBufferInit(pRes, 0); + /* Zero the output parameters. */ + *paOut = 0; + *pnOut = 0; if( pExpr ){ if( pExpr->eType==FTSQUERY_PHRASE ){ - DocListType eType = DL_DOCIDS; - if( pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR ){ - eType = DL_POSITIONS; - } - rc = docListOfPhrase(pTab, pExpr->pPhrase, eType, pRes); + int isReqPos = (pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR); + rc = fts3PhraseSelect(p, pExpr->pPhrase, isReqPos, paOut, pnOut); }else{ - DataBuffer lhs; - DataBuffer rhs; + char *aLeft; + char *aRight; + int nLeft; + int nRight; - dataBufferInit(&rhs, 0); - if( SQLITE_OK==(rc = evalFts3Expr(pTab, pExpr->pLeft, &lhs)) - && SQLITE_OK==(rc = evalFts3Expr(pTab, pExpr->pRight, &rhs)) + if( SQLITE_OK==(rc = evalFts3Expr(p, pExpr->pRight, &aRight, &nRight)) + && SQLITE_OK==(rc = evalFts3Expr(p, pExpr->pLeft, &aLeft, &nLeft)) ){ + assert( pExpr->eType==FTSQUERY_NEAR || pExpr->eType==FTSQUERY_OR + || pExpr->eType==FTSQUERY_AND || pExpr->eType==FTSQUERY_NOT + ); switch( pExpr->eType ){ case FTSQUERY_NEAR: { - int nToken; Fts3Expr *pLeft; - DocListType eType = DL_DOCIDS; + Fts3Expr *pRight; + int mergetype = MERGE_NEAR; + int nParam1; + int nParam2; + char *aBuffer; + if( pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR ){ - eType = DL_POSITIONS; + mergetype = MERGE_POS_NEAR; } pLeft = pExpr->pLeft; - while( pLeft->eType==FTSQUERY_NEAR ){ + while( pLeft->eType==FTSQUERY_NEAR ){ pLeft=pLeft->pRight; } - assert( pExpr->pRight->eType==FTSQUERY_PHRASE ); + pRight = pExpr->pRight; + assert( pRight->eType==FTSQUERY_PHRASE ); assert( pLeft->eType==FTSQUERY_PHRASE ); - nToken = pLeft->pPhrase->nToken + pExpr->pRight->pPhrase->nToken; - docListPhraseMerge(lhs.pData, lhs.nData, rhs.pData, rhs.nData, - pExpr->nNear+1, nToken, eType, pRes + + nParam1 = pExpr->nNear+1; + nParam2 = nParam1+pLeft->pPhrase->nToken+pRight->pPhrase->nToken-2; + aBuffer = sqlite3_malloc(nLeft+nRight+1); + rc = fts3DoclistMerge(mergetype, nParam1, nParam2, aBuffer, + pnOut, aLeft, nLeft, aRight, nRight ); + if( rc!=SQLITE_OK ){ + sqlite3_free(aBuffer); + }else{ + *paOut = aBuffer; + } + sqlite3_free(aLeft); break; } - case FTSQUERY_NOT: { - docListExceptMerge(lhs.pData, lhs.nData, rhs.pData, rhs.nData,pRes); - break; - } - case FTSQUERY_AND: { - docListAndMerge(lhs.pData, lhs.nData, rhs.pData, rhs.nData, pRes); - break; - } + case FTSQUERY_OR: { - docListOrMerge(lhs.pData, lhs.nData, rhs.pData, rhs.nData, pRes); + /* Allocate a buffer for the output. The maximum size is the + ** sum of the sizes of the two input buffers. The +1 term is + ** so that a buffer of zero bytes is never allocated - this can + ** cause fts3DoclistMerge() to incorrectly return SQLITE_NOMEM. + */ + char *aBuffer = sqlite3_malloc(nRight+nLeft+1); + rc = fts3DoclistMerge(MERGE_OR, 0, 0, aBuffer, pnOut, + aLeft, nLeft, aRight, nRight + ); + *paOut = aBuffer; + sqlite3_free(aLeft); + break; + } + + default: { + assert( FTSQUERY_NOT==MERGE_NOT && FTSQUERY_AND==MERGE_AND ); + fts3DoclistMerge(pExpr->eType, 0, 0, aLeft, pnOut, + aLeft, nLeft, aRight, nRight + ); + *paOut = aLeft; break; } } } - dataBufferDestroy(&lhs); - dataBufferDestroy(&rhs); + sqlite3_free(aRight); } } return rc; } -/* TODO(shess) Refactor the code to remove this forward decl. */ -static int flushPendingTerms(fulltext_vtab *v); - -/* Perform a full-text query using the search expression in -** zInput[0..nInput-1]. Return a list of matching documents -** in pResult. -** -** Queries must match column iColumn. Or if iColumn>=nColumn -** they are allowed to match against any column. -*/ -static int fulltextQuery( - fulltext_vtab *v, /* The full text index */ - int iColumn, /* Match against this column by default */ - const char *zInput, /* The query string */ - int nInput, /* Number of bytes in zInput[] */ - DataBuffer *pResult, /* Write the result doclist here */ - Fts3Expr **ppExpr /* Put parsed query string here */ -){ - int rc; - - /* TODO(shess) Instead of flushing pendingTerms, we could query for - ** the relevant term and merge the doclist into what we receive from - ** the database. Wait and see if this is a common issue, first. - ** - ** A good reason not to flush is to not generate update-related - ** error codes from here. - */ - - /* Flush any buffered updates before executing the query. */ - rc = flushPendingTerms(v); - if( rc!=SQLITE_OK ){ - return rc; - } - - /* Parse the query passed to the MATCH operator. */ - rc = sqlite3Fts3ExprParse(v->pTokenizer, - v->azColumn, v->nColumn, iColumn, zInput, nInput, ppExpr - ); - if( rc!=SQLITE_OK ){ - assert( 0==(*ppExpr) ); - return rc; - } - - return evalFts3Expr(v, *ppExpr, pResult); -} - /* ** This is the xFilter interface for the virtual table. See ** the virtual table xFilter method documentation for additional ** information. ** -** If idxNum==QUERY_GENERIC then do a full table scan against +** If idxNum==FTS3_FULLSCAN_SEARCH then do a full table scan against ** the %_content table. ** -** If idxNum==QUERY_DOCID then do a docid lookup for a single entry +** If idxNum==FTS3_DOCID_SEARCH then do a docid lookup for a single entry ** in the %_content table. ** -** If idxNum>=QUERY_FULLTEXT then use the full text index. The +** If idxNum>=FTS3_FULLTEXT_SEARCH then use the full text index. The ** column on the left-hand side of the MATCH operator is column -** number idxNum-QUERY_FULLTEXT, 0 indexed. argv[0] is the right-hand +** number idxNum-FTS3_FULLTEXT_SEARCH, 0 indexed. argv[0] is the right-hand ** side of the MATCH operator. */ /* TODO(shess) Upgrade the cursor initialization and destruction to -** account for fulltextFilter() being called multiple times on the -** same cursor. The current solution is very fragile. Apply fix to +** account for fts3FilterMethod() being called multiple times on the +** same cursor. The current solution is very fragile. Apply fix to ** fts3 as appropriate. */ -static int fulltextFilter( - sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ - int idxNum, const char *idxStr, /* Which indexing scheme to use */ - int argc, sqlite3_value **argv /* Arguments for the indexing scheme */ +static int fts3FilterMethod( + sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ + int idxNum, /* Strategy index */ + const char *idxStr, /* Unused */ + int nVal, /* Number of elements in apVal */ + sqlite3_value **apVal /* Arguments for the indexing scheme */ ){ - fulltext_cursor *c = (fulltext_cursor *) pCursor; - fulltext_vtab *v = cursor_vtab(c); - int rc; + const char *azSql[] = { + "SELECT * FROM %Q.'%q_content' WHERE docid = ?", /* non-full-table-scan */ + "SELECT * FROM %Q.'%q_content'", /* full-table-scan */ + }; + int rc; /* Return code */ + char *zSql; /* SQL statement used to access %_content */ + Fts3Table *p = (Fts3Table *)pCursor->pVtab; + Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; - FTSTRACE(("FTS3 Filter %p\n",pCursor)); + UNUSED_PARAMETER(idxStr); + UNUSED_PARAMETER(nVal); - /* If the cursor has a statement that was not prepared according to - ** idxNum, clear it. I believe all calls to fulltextFilter with a - ** given cursor will have the same idxNum , but in this case it's - ** easy to be safe. + assert( idxNum>=0 && idxNum<=(FTS3_FULLTEXT_SEARCH+p->nColumn) ); + assert( nVal==0 || nVal==1 ); + assert( (nVal==0)==(idxNum==FTS3_FULLSCAN_SEARCH) ); + + /* In case the cursor has been used before, clear it now. */ + sqlite3_finalize(pCsr->pStmt); + sqlite3_free(pCsr->aDoclist); + memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor)); + + /* Compile a SELECT statement for this cursor. For a full-table-scan, the + ** statement loops through all rows of the %_content table. For a + ** full-text query or docid lookup, the statement retrieves a single + ** row by docid. */ - if( c->pStmt && c->iCursorType!=idxNum ){ - sqlite3_finalize(c->pStmt); - c->pStmt = NULL; - } - - /* Get a fresh statement appropriate to idxNum. */ - /* TODO(shess): Add a prepared-statement cache in the vt structure. - ** The cache must handle multiple open cursors. Easier to cache the - ** statement variants at the vt to reduce malloc/realloc/free here. - ** Or we could have a StringBuffer variant which allowed stack - ** construction for small values. - */ - if( !c->pStmt ){ - StringBuffer sb; - initStringBuffer(&sb); - append(&sb, "SELECT docid, "); - appendList(&sb, v->nColumn, v->azContentColumn); - append(&sb, " FROM %_content"); - if( idxNum!=QUERY_GENERIC ) append(&sb, " WHERE docid = ?"); - rc = sql_prepare(v->db, v->zDb, v->zName, &c->pStmt, - stringBufferData(&sb)); - stringBufferDestroy(&sb); - if( rc!=SQLITE_OK ) return rc; - c->iCursorType = idxNum; + zSql = sqlite3_mprintf(azSql[idxNum==FTS3_FULLSCAN_SEARCH], p->zDb, p->zName); + if( !zSql ){ + rc = SQLITE_NOMEM; }else{ - sqlite3_reset(c->pStmt); - assert( c->iCursorType==idxNum ); + rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0); + sqlite3_free(zSql); } + if( rc!=SQLITE_OK ) return rc; + pCsr->eSearch = (i16)idxNum; - switch( idxNum ){ - case QUERY_GENERIC: - break; + if( idxNum==FTS3_DOCID_SEARCH ){ + rc = sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]); + }else if( idxNum!=FTS3_FULLSCAN_SEARCH ){ + int iCol = idxNum-FTS3_FULLTEXT_SEARCH; + const char *zQuery = (const char *)sqlite3_value_text(apVal[0]); - case QUERY_DOCID: - rc = sqlite3_bind_int64(c->pStmt, 1, sqlite3_value_int64(argv[0])); - if( rc!=SQLITE_OK ) return rc; - break; - - default: /* full-text search */ - { - int iCol = idxNum-QUERY_FULLTEXT; - const char *zQuery = (const char *)sqlite3_value_text(argv[0]); - assert( idxNum<=QUERY_FULLTEXT+v->nColumn); - assert( argc==1 ); - if( c->result.nData!=0 ){ - /* This case happens if the same cursor is used repeatedly. */ - dlrDestroy(&c->reader); - dataBufferReset(&c->result); - }else{ - dataBufferInit(&c->result, 0); - } - rc = fulltextQuery(v, iCol, zQuery, -1, &c->result, &c->pExpr); - if( rc!=SQLITE_OK ) return rc; - if( c->result.nData!=0 ){ - dlrInit(&c->reader, DL_DOCIDS, c->result.pData, c->result.nData); - } - break; + if( zQuery==0 && sqlite3_value_type(apVal[0])!=SQLITE_NULL ){ + return SQLITE_NOMEM; } + rc = sqlite3Fts3PendingTermsFlush(p); + if( rc!=SQLITE_OK ) return rc; + + rc = sqlite3Fts3ExprParse(p->pTokenizer, p->azColumn, p->nColumn, + iCol, zQuery, -1, &pCsr->pExpr + ); + if( rc!=SQLITE_OK ) return rc; + + rc = evalFts3Expr(p, pCsr->pExpr, &pCsr->aDoclist, &pCsr->nDoclist); + pCsr->pNextId = pCsr->aDoclist; + pCsr->iPrevId = 0; } - return fulltextNext(pCursor); + if( rc!=SQLITE_OK ) return rc; + return fts3NextMethod(pCursor); } -/* This is the xEof method of the virtual table. The SQLite core -** calls this routine to find out if it has reached the end of -** a query's results set. +/* +** This is the xEof method of the virtual table. SQLite calls this +** routine to find out if it has reached the end of a result set. */ -static int fulltextEof(sqlite3_vtab_cursor *pCursor){ - fulltext_cursor *c = (fulltext_cursor *) pCursor; - return c->eof; +static int fts3EofMethod(sqlite3_vtab_cursor *pCursor){ + return ((Fts3Cursor *)pCursor)->isEof; } -/* This is the xColumn method of the virtual table. The SQLite -** core calls this method during a query when it needs the value -** of a column from the virtual table. This method needs to use -** one of the sqlite3_result_*() routines to store the requested -** value back in the pContext. -*/ -static int fulltextColumn(sqlite3_vtab_cursor *pCursor, - sqlite3_context *pContext, int idxCol){ - fulltext_cursor *c = (fulltext_cursor *) pCursor; - fulltext_vtab *v = cursor_vtab(c); - - if( idxColnColumn ){ - sqlite3_value *pVal = sqlite3_column_value(c->pStmt, idxCol+1); - sqlite3_result_value(pContext, pVal); - }else if( idxCol==v->nColumn ){ - /* The extra column whose name is the same as the table. - ** Return a blob which is a pointer to the cursor - */ - sqlite3_result_blob(pContext, &c, sizeof(c), SQLITE_TRANSIENT); - }else if( idxCol==v->nColumn+1 ){ - /* The docid column, which is an alias for rowid. */ - sqlite3_value *pVal = sqlite3_column_value(c->pStmt, 0); - sqlite3_result_value(pContext, pVal); - } - return SQLITE_OK; -} - -/* This is the xRowid method. The SQLite core calls this routine to -** retrieve the rowid for the current row of the result set. fts3 -** exposes %_content.docid as the rowid for the virtual table. The +/* +** This is the xRowid method. The SQLite core calls this routine to +** retrieve the rowid for the current row of the result set. fts3 +** exposes %_content.docid as the rowid for the virtual table. The ** rowid should be written to *pRowid. */ -static int fulltextRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ - fulltext_cursor *c = (fulltext_cursor *) pCursor; - - *pRowid = sqlite3_column_int64(c->pStmt, 0); - return SQLITE_OK; -} - -/* Add all terms in [zText] to pendingTerms table. If [iColumn] > 0, -** we also store positions and offsets in the hash table using that -** column number. -*/ -static int buildTerms(fulltext_vtab *v, sqlite_int64 iDocid, - const char *zText, int iColumn){ - sqlite3_tokenizer *pTokenizer = v->pTokenizer; - sqlite3_tokenizer_cursor *pCursor; - const char *pToken; - int nTokenBytes; - int iStartOffset, iEndOffset, iPosition; - int rc; - - rc = pTokenizer->pModule->xOpen(pTokenizer, zText, -1, &pCursor); - if( rc!=SQLITE_OK ) return rc; - - pCursor->pTokenizer = pTokenizer; - while( SQLITE_OK==(rc=pTokenizer->pModule->xNext(pCursor, - &pToken, &nTokenBytes, - &iStartOffset, &iEndOffset, - &iPosition)) ){ - DLCollector *p; - int nData; /* Size of doclist before our update. */ - - /* Positions can't be negative; we use -1 as a terminator - * internally. Token can't be NULL or empty. */ - if( iPosition<0 || pToken == NULL || nTokenBytes == 0 ){ - rc = SQLITE_ERROR; - break; - } - - p = fts3HashFind(&v->pendingTerms, pToken, nTokenBytes); - if( p==NULL ){ - nData = 0; - p = dlcNew(iDocid, DL_DEFAULT); - fts3HashInsert(&v->pendingTerms, pToken, nTokenBytes, p); - - /* Overhead for our hash table entry, the key, and the value. */ - v->nPendingData += sizeof(struct fts3HashElem)+sizeof(*p)+nTokenBytes; - }else{ - nData = p->b.nData; - if( p->dlw.iPrevDocid!=iDocid ) dlcNext(p, iDocid); - } - if( iColumn>=0 ){ - dlcAddPos(p, iColumn, iPosition, iStartOffset, iEndOffset); - } - - /* Accumulate data added by dlcNew or dlcNext, and dlcAddPos. */ - v->nPendingData += p->b.nData-nData; - } - - /* TODO(shess) Check return? Should this be able to cause errors at - ** this point? Actually, same question about sqlite3_finalize(), - ** though one could argue that failure there means that the data is - ** not durable. *ponder* - */ - pTokenizer->pModule->xClose(pCursor); - if( SQLITE_DONE == rc ) return SQLITE_OK; - return rc; -} - -/* Add doclists for all terms in [pValues] to pendingTerms table. */ -static int insertTerms(fulltext_vtab *v, sqlite_int64 iDocid, - sqlite3_value **pValues){ - int i; - for(i = 0; i < v->nColumn ; ++i){ - char *zText = (char*)sqlite3_value_text(pValues[i]); - int rc = buildTerms(v, iDocid, zText, i); - if( rc!=SQLITE_OK ) return rc; - } - return SQLITE_OK; -} - -/* Add empty doclists for all terms in the given row's content to -** pendingTerms. -*/ -static int deleteTerms(fulltext_vtab *v, sqlite_int64 iDocid){ - const char **pValues; - int i, rc; - - /* TODO(shess) Should we allow such tables at all? */ - if( DL_DEFAULT==DL_DOCIDS ) return SQLITE_ERROR; - - rc = content_select(v, iDocid, &pValues); - if( rc!=SQLITE_OK ) return rc; - - for(i = 0 ; i < v->nColumn; ++i) { - rc = buildTerms(v, iDocid, pValues[i], -1); - if( rc!=SQLITE_OK ) break; - } - - freeStringArray(v->nColumn, pValues); - return SQLITE_OK; -} - -/* TODO(shess) Refactor the code to remove this forward decl. */ -static int initPendingTerms(fulltext_vtab *v, sqlite_int64 iDocid); - -/* Insert a row into the %_content table; set *piDocid to be the ID of the -** new row. Add doclists for terms to pendingTerms. -*/ -static int index_insert(fulltext_vtab *v, sqlite3_value *pRequestDocid, - sqlite3_value **pValues, sqlite_int64 *piDocid){ - int rc; - - rc = content_insert(v, pRequestDocid, pValues); /* execute an SQL INSERT */ - if( rc!=SQLITE_OK ) return rc; - - /* docid column is an alias for rowid. */ - *piDocid = sqlite3_last_insert_rowid(v->db); - rc = initPendingTerms(v, *piDocid); - if( rc!=SQLITE_OK ) return rc; - - return insertTerms(v, *piDocid, pValues); -} - -/* Delete a row from the %_content table; add empty doclists for terms -** to pendingTerms. -*/ -static int index_delete(fulltext_vtab *v, sqlite_int64 iRow){ - int rc = initPendingTerms(v, iRow); - if( rc!=SQLITE_OK ) return rc; - - rc = deleteTerms(v, iRow); - if( rc!=SQLITE_OK ) return rc; - - return content_delete(v, iRow); /* execute an SQL DELETE */ -} - -/* Update a row in the %_content table; add delete doclists to -** pendingTerms for old terms not in the new data, add insert doclists -** to pendingTerms for terms in the new data. -*/ -static int index_update(fulltext_vtab *v, sqlite_int64 iRow, - sqlite3_value **pValues){ - int rc = initPendingTerms(v, iRow); - if( rc!=SQLITE_OK ) return rc; - - /* Generate an empty doclist for each term that previously appeared in this - * row. */ - rc = deleteTerms(v, iRow); - if( rc!=SQLITE_OK ) return rc; - - rc = content_update(v, pValues, iRow); /* execute an SQL UPDATE */ - if( rc!=SQLITE_OK ) return rc; - - /* Now add positions for terms which appear in the updated row. */ - return insertTerms(v, iRow, pValues); -} - -/*******************************************************************/ -/* InteriorWriter is used to collect terms and block references into -** interior nodes in %_segments. See commentary at top of file for -** format. -*/ - -/* How large interior nodes can grow. */ -#define INTERIOR_MAX 2048 - -/* Minimum number of terms per interior node (except the root). This -** prevents large terms from making the tree too skinny - must be >0 -** so that the tree always makes progress. Note that the min tree -** fanout will be INTERIOR_MIN_TERMS+1. -*/ -#define INTERIOR_MIN_TERMS 7 -#if INTERIOR_MIN_TERMS<1 -# error INTERIOR_MIN_TERMS must be greater than 0. -#endif - -/* ROOT_MAX controls how much data is stored inline in the segment -** directory. -*/ -/* TODO(shess) Push ROOT_MAX down to whoever is writing things. It's -** only here so that interiorWriterRootInfo() and leafWriterRootInfo() -** can both see it, but if the caller passed it in, we wouldn't even -** need a define. -*/ -#define ROOT_MAX 1024 -#if ROOT_MAXterm, 0); - dataBufferReplace(&block->term, pTerm, nTerm); - - n = fts3PutVarint(c, iHeight); - n += fts3PutVarint(c+n, iChildBlock); - dataBufferInit(&block->data, INTERIOR_MAX); - dataBufferReplace(&block->data, c, n); - } - return block; -} - -#ifndef NDEBUG -/* Verify that the data is readable as an interior node. */ -static void interiorBlockValidate(InteriorBlock *pBlock){ - const char *pData = pBlock->data.pData; - int nData = pBlock->data.nData; - int n, iDummy; - sqlite_int64 iBlockid; - - assert( nData>0 ); - assert( pData!=0 ); - assert( pData+nData>pData ); - - /* Must lead with height of node as a varint(n), n>0 */ - n = fts3GetVarint32(pData, &iDummy); - assert( n>0 ); - assert( iDummy>0 ); - assert( n0 ); - assert( n<=nData ); - pData += n; - nData -= n; - - /* Zero or more terms of positive length */ - if( nData!=0 ){ - /* First term is not delta-encoded. */ - n = fts3GetVarint32(pData, &iDummy); - assert( n>0 ); - assert( iDummy>0 ); - assert( n+iDummy>0); - assert( n+iDummy<=nData ); - pData += n+iDummy; - nData -= n+iDummy; - - /* Following terms delta-encoded. */ - while( nData!=0 ){ - /* Length of shared prefix. */ - n = fts3GetVarint32(pData, &iDummy); - assert( n>0 ); - assert( iDummy>=0 ); - assert( n0 ); - assert( iDummy>0 ); - assert( n+iDummy>0); - assert( n+iDummy<=nData ); - pData += n+iDummy; - nData -= n+iDummy; - } - } -} -#define ASSERT_VALID_INTERIOR_BLOCK(x) interiorBlockValidate(x) -#else -#define ASSERT_VALID_INTERIOR_BLOCK(x) assert( 1 ) -#endif - -typedef struct InteriorWriter { - int iHeight; /* from 0 at leaves. */ - InteriorBlock *first, *last; - struct InteriorWriter *parentWriter; - - DataBuffer term; /* Last term written to block "last". */ - sqlite_int64 iOpeningChildBlock; /* First child block in block "last". */ -#ifndef NDEBUG - sqlite_int64 iLastChildBlock; /* for consistency checks. */ -#endif -} InteriorWriter; - -/* Initialize an interior node where pTerm[nTerm] marks the leftmost -** term in the tree. iChildBlock is the leftmost child block at the -** next level down the tree. -*/ -static void interiorWriterInit(int iHeight, const char *pTerm, int nTerm, - sqlite_int64 iChildBlock, - InteriorWriter *pWriter){ - InteriorBlock *block; - assert( iHeight>0 ); - CLEAR(pWriter); - - pWriter->iHeight = iHeight; - pWriter->iOpeningChildBlock = iChildBlock; -#ifndef NDEBUG - pWriter->iLastChildBlock = iChildBlock; -#endif - block = interiorBlockNew(iHeight, iChildBlock, pTerm, nTerm); - pWriter->last = pWriter->first = block; - ASSERT_VALID_INTERIOR_BLOCK(pWriter->last); - dataBufferInit(&pWriter->term, 0); -} - -/* Append the child node rooted at iChildBlock to the interior node, -** with pTerm[nTerm] as the leftmost term in iChildBlock's subtree. -*/ -static void interiorWriterAppend(InteriorWriter *pWriter, - const char *pTerm, int nTerm, - sqlite_int64 iChildBlock){ - char c[VARINT_MAX+VARINT_MAX]; - int n, nPrefix = 0; - - ASSERT_VALID_INTERIOR_BLOCK(pWriter->last); - - /* The first term written into an interior node is actually - ** associated with the second child added (the first child was added - ** in interiorWriterInit, or in the if clause at the bottom of this - ** function). That term gets encoded straight up, with nPrefix left - ** at 0. - */ - if( pWriter->term.nData==0 ){ - n = fts3PutVarint(c, nTerm); +static int fts3RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ + Fts3Cursor *pCsr = (Fts3Cursor *) pCursor; + if( pCsr->aDoclist ){ + *pRowid = pCsr->iPrevId; }else{ - while( nPrefixterm.nData && - pTerm[nPrefix]==pWriter->term.pData[nPrefix] ){ - nPrefix++; - } - - n = fts3PutVarint(c, nPrefix); - n += fts3PutVarint(c+n, nTerm-nPrefix); - } - -#ifndef NDEBUG - pWriter->iLastChildBlock++; -#endif - assert( pWriter->iLastChildBlock==iChildBlock ); - - /* Overflow to a new block if the new term makes the current block - ** too big, and the current block already has enough terms. - */ - if( pWriter->last->data.nData+n+nTerm-nPrefix>INTERIOR_MAX && - iChildBlock-pWriter->iOpeningChildBlock>INTERIOR_MIN_TERMS ){ - pWriter->last->next = interiorBlockNew(pWriter->iHeight, iChildBlock, - pTerm, nTerm); - pWriter->last = pWriter->last->next; - pWriter->iOpeningChildBlock = iChildBlock; - dataBufferReset(&pWriter->term); - }else{ - dataBufferAppend2(&pWriter->last->data, c, n, - pTerm+nPrefix, nTerm-nPrefix); - dataBufferReplace(&pWriter->term, pTerm, nTerm); - } - ASSERT_VALID_INTERIOR_BLOCK(pWriter->last); -} - -/* Free the space used by pWriter, including the linked-list of -** InteriorBlocks, and parentWriter, if present. -*/ -static int interiorWriterDestroy(InteriorWriter *pWriter){ - InteriorBlock *block = pWriter->first; - - while( block!=NULL ){ - InteriorBlock *b = block; - block = block->next; - dataBufferDestroy(&b->term); - dataBufferDestroy(&b->data); - sqlite3_free(b); - } - if( pWriter->parentWriter!=NULL ){ - interiorWriterDestroy(pWriter->parentWriter); - sqlite3_free(pWriter->parentWriter); - } - dataBufferDestroy(&pWriter->term); - SCRAMBLE(pWriter); - return SQLITE_OK; -} - -/* If pWriter can fit entirely in ROOT_MAX, return it as the root info -** directly, leaving *piEndBlockid unchanged. Otherwise, flush -** pWriter to %_segments, building a new layer of interior nodes, and -** recursively ask for their root into. -*/ -static int interiorWriterRootInfo(fulltext_vtab *v, InteriorWriter *pWriter, - char **ppRootInfo, int *pnRootInfo, - sqlite_int64 *piEndBlockid){ - InteriorBlock *block = pWriter->first; - sqlite_int64 iBlockid = 0; - int rc; - - /* If we can fit the segment inline */ - if( block==pWriter->last && block->data.nDatadata.pData; - *pnRootInfo = block->data.nData; - return SQLITE_OK; - } - - /* Flush the first block to %_segments, and create a new level of - ** interior node. - */ - ASSERT_VALID_INTERIOR_BLOCK(block); - rc = block_insert(v, block->data.pData, block->data.nData, &iBlockid); - if( rc!=SQLITE_OK ) return rc; - *piEndBlockid = iBlockid; - - pWriter->parentWriter = sqlite3_malloc(sizeof(*pWriter->parentWriter)); - interiorWriterInit(pWriter->iHeight+1, - block->term.pData, block->term.nData, - iBlockid, pWriter->parentWriter); - - /* Flush additional blocks and append to the higher interior - ** node. - */ - for(block=block->next; block!=NULL; block=block->next){ - ASSERT_VALID_INTERIOR_BLOCK(block); - rc = block_insert(v, block->data.pData, block->data.nData, &iBlockid); - if( rc!=SQLITE_OK ) return rc; - *piEndBlockid = iBlockid; - - interiorWriterAppend(pWriter->parentWriter, - block->term.pData, block->term.nData, iBlockid); - } - - /* Parent node gets the chance to be the root. */ - return interiorWriterRootInfo(v, pWriter->parentWriter, - ppRootInfo, pnRootInfo, piEndBlockid); -} - -/****************************************************************/ -/* InteriorReader is used to read off the data from an interior node -** (see comment at top of file for the format). -*/ -typedef struct InteriorReader { - const char *pData; - int nData; - - DataBuffer term; /* previous term, for decoding term delta. */ - - sqlite_int64 iBlockid; -} InteriorReader; - -static void interiorReaderDestroy(InteriorReader *pReader){ - dataBufferDestroy(&pReader->term); - SCRAMBLE(pReader); -} - -/* TODO(shess) The assertions are great, but what if we're in NDEBUG -** and the blob is empty or otherwise contains suspect data? -*/ -static void interiorReaderInit(const char *pData, int nData, - InteriorReader *pReader){ - int n, nTerm; - - /* Require at least the leading flag byte */ - assert( nData>0 ); - assert( pData[0]!='\0' ); - - CLEAR(pReader); - - /* Decode the base blockid, and set the cursor to the first term. */ - n = fts3GetVarint(pData+1, &pReader->iBlockid); - assert( 1+n<=nData ); - pReader->pData = pData+1+n; - pReader->nData = nData-(1+n); - - /* A single-child interior node (such as when a leaf node was too - ** large for the segment directory) won't have any terms. - ** Otherwise, decode the first term. - */ - if( pReader->nData==0 ){ - dataBufferInit(&pReader->term, 0); - }else{ - n = fts3GetVarint32(pReader->pData, &nTerm); - dataBufferInit(&pReader->term, nTerm); - dataBufferReplace(&pReader->term, pReader->pData+n, nTerm); - assert( n+nTerm<=pReader->nData ); - pReader->pData += n+nTerm; - pReader->nData -= n+nTerm; - } -} - -static int interiorReaderAtEnd(InteriorReader *pReader){ - return pReader->term.nData==0; -} - -static sqlite_int64 interiorReaderCurrentBlockid(InteriorReader *pReader){ - return pReader->iBlockid; -} - -static int interiorReaderTermBytes(InteriorReader *pReader){ - assert( !interiorReaderAtEnd(pReader) ); - return pReader->term.nData; -} -static const char *interiorReaderTerm(InteriorReader *pReader){ - assert( !interiorReaderAtEnd(pReader) ); - return pReader->term.pData; -} - -/* Step forward to the next term in the node. */ -static void interiorReaderStep(InteriorReader *pReader){ - assert( !interiorReaderAtEnd(pReader) ); - - /* If the last term has been read, signal eof, else construct the - ** next term. - */ - if( pReader->nData==0 ){ - dataBufferReset(&pReader->term); - }else{ - int n, nPrefix, nSuffix; - - n = fts3GetVarint32(pReader->pData, &nPrefix); - n += fts3GetVarint32(pReader->pData+n, &nSuffix); - - /* Truncate the current term and append suffix data. */ - pReader->term.nData = nPrefix; - dataBufferAppend(&pReader->term, pReader->pData+n, nSuffix); - - assert( n+nSuffix<=pReader->nData ); - pReader->pData += n+nSuffix; - pReader->nData -= n+nSuffix; - } - pReader->iBlockid++; -} - -/* Compare the current term to pTerm[nTerm], returning strcmp-style -** results. If isPrefix, equality means equal through nTerm bytes. -*/ -static int interiorReaderTermCmp(InteriorReader *pReader, - const char *pTerm, int nTerm, int isPrefix){ - const char *pReaderTerm = interiorReaderTerm(pReader); - int nReaderTerm = interiorReaderTermBytes(pReader); - int c, n = nReaderTerm0 ) return -1; - if( nTerm>0 ) return 1; - return 0; - } - - c = memcmp(pReaderTerm, pTerm, n); - if( c!=0 ) return c; - if( isPrefix && n==nTerm ) return 0; - return nReaderTerm - nTerm; -} - -/****************************************************************/ -/* LeafWriter is used to collect terms and associated doclist data -** into leaf blocks in %_segments (see top of file for format info). -** Expected usage is: -** -** LeafWriter writer; -** leafWriterInit(0, 0, &writer); -** while( sorted_terms_left_to_process ){ -** // data is doclist data for that term. -** rc = leafWriterStep(v, &writer, pTerm, nTerm, pData, nData); -** if( rc!=SQLITE_OK ) goto err; -** } -** rc = leafWriterFinalize(v, &writer); -**err: -** leafWriterDestroy(&writer); -** return rc; -** -** leafWriterStep() may write a collected leaf out to %_segments. -** leafWriterFinalize() finishes writing any buffered data and stores -** a root node in %_segdir. leafWriterDestroy() frees all buffers and -** InteriorWriters allocated as part of writing this segment. -** -** TODO(shess) Document leafWriterStepMerge(). -*/ - -/* Put terms with data this big in their own block. */ -#define STANDALONE_MIN 1024 - -/* Keep leaf blocks below this size. */ -#define LEAF_MAX 2048 - -typedef struct LeafWriter { - int iLevel; - int idx; - sqlite_int64 iStartBlockid; /* needed to create the root info */ - sqlite_int64 iEndBlockid; /* when we're done writing. */ - - DataBuffer term; /* previous encoded term */ - DataBuffer data; /* encoding buffer */ - - /* bytes of first term in the current node which distinguishes that - ** term from the last term of the previous node. - */ - int nTermDistinct; - - InteriorWriter parentWriter; /* if we overflow */ - int has_parent; -} LeafWriter; - -static void leafWriterInit(int iLevel, int idx, LeafWriter *pWriter){ - CLEAR(pWriter); - pWriter->iLevel = iLevel; - pWriter->idx = idx; - - dataBufferInit(&pWriter->term, 32); - - /* Start out with a reasonably sized block, though it can grow. */ - dataBufferInit(&pWriter->data, LEAF_MAX); -} - -#ifndef NDEBUG -/* Verify that the data is readable as a leaf node. */ -static void leafNodeValidate(const char *pData, int nData){ - int n, iDummy; - - if( nData==0 ) return; - assert( nData>0 ); - assert( pData!=0 ); - assert( pData+nData>pData ); - - /* Must lead with a varint(0) */ - n = fts3GetVarint32(pData, &iDummy); - assert( iDummy==0 ); - assert( n>0 ); - assert( n0 ); - assert( iDummy>0 ); - assert( n+iDummy>0 ); - assert( n+iDummy0 ); - assert( iDummy>0 ); - assert( n+iDummy>0 ); - assert( n+iDummy<=nData ); - ASSERT_VALID_DOCLIST(DL_DEFAULT, pData+n, iDummy, NULL); - pData += n+iDummy; - nData -= n+iDummy; - - /* Verify that trailing terms and doclists also are readable. */ - while( nData!=0 ){ - n = fts3GetVarint32(pData, &iDummy); - assert( n>0 ); - assert( iDummy>=0 ); - assert( n0 ); - assert( iDummy>0 ); - assert( n+iDummy>0 ); - assert( n+iDummy0 ); - assert( iDummy>0 ); - assert( n+iDummy>0 ); - assert( n+iDummy<=nData ); - ASSERT_VALID_DOCLIST(DL_DEFAULT, pData+n, iDummy, NULL); - pData += n+iDummy; - nData -= n+iDummy; - } -} -#define ASSERT_VALID_LEAF_NODE(p, n) leafNodeValidate(p, n) -#else -#define ASSERT_VALID_LEAF_NODE(p, n) assert( 1 ) -#endif - -/* Flush the current leaf node to %_segments, and adding the resulting -** blockid and the starting term to the interior node which will -** contain it. -*/ -static int leafWriterInternalFlush(fulltext_vtab *v, LeafWriter *pWriter, - int iData, int nData){ - sqlite_int64 iBlockid = 0; - const char *pStartingTerm; - int nStartingTerm, rc, n; - - /* Must have the leading varint(0) flag, plus at least some - ** valid-looking data. - */ - assert( nData>2 ); - assert( iData>=0 ); - assert( iData+nData<=pWriter->data.nData ); - ASSERT_VALID_LEAF_NODE(pWriter->data.pData+iData, nData); - - rc = block_insert(v, pWriter->data.pData+iData, nData, &iBlockid); - if( rc!=SQLITE_OK ) return rc; - assert( iBlockid!=0 ); - - /* Reconstruct the first term in the leaf for purposes of building - ** the interior node. - */ - n = fts3GetVarint32(pWriter->data.pData+iData+1, &nStartingTerm); - pStartingTerm = pWriter->data.pData+iData+1+n; - assert( pWriter->data.nData>iData+1+n+nStartingTerm ); - assert( pWriter->nTermDistinct>0 ); - assert( pWriter->nTermDistinct<=nStartingTerm ); - nStartingTerm = pWriter->nTermDistinct; - - if( pWriter->has_parent ){ - interiorWriterAppend(&pWriter->parentWriter, - pStartingTerm, nStartingTerm, iBlockid); - }else{ - interiorWriterInit(1, pStartingTerm, nStartingTerm, iBlockid, - &pWriter->parentWriter); - pWriter->has_parent = 1; - } - - /* Track the span of this segment's leaf nodes. */ - if( pWriter->iEndBlockid==0 ){ - pWriter->iEndBlockid = pWriter->iStartBlockid = iBlockid; - }else{ - pWriter->iEndBlockid++; - assert( iBlockid==pWriter->iEndBlockid ); - } - - return SQLITE_OK; -} -static int leafWriterFlush(fulltext_vtab *v, LeafWriter *pWriter){ - int rc = leafWriterInternalFlush(v, pWriter, 0, pWriter->data.nData); - if( rc!=SQLITE_OK ) return rc; - - /* Re-initialize the output buffer. */ - dataBufferReset(&pWriter->data); - - return SQLITE_OK; -} - -/* Fetch the root info for the segment. If the entire leaf fits -** within ROOT_MAX, then it will be returned directly, otherwise it -** will be flushed and the root info will be returned from the -** interior node. *piEndBlockid is set to the blockid of the last -** interior or leaf node written to disk (0 if none are written at -** all). -*/ -static int leafWriterRootInfo(fulltext_vtab *v, LeafWriter *pWriter, - char **ppRootInfo, int *pnRootInfo, - sqlite_int64 *piEndBlockid){ - /* we can fit the segment entirely inline */ - if( !pWriter->has_parent && pWriter->data.nDatadata.pData; - *pnRootInfo = pWriter->data.nData; - *piEndBlockid = 0; - return SQLITE_OK; - } - - /* Flush remaining leaf data. */ - if( pWriter->data.nData>0 ){ - int rc = leafWriterFlush(v, pWriter); - if( rc!=SQLITE_OK ) return rc; - } - - /* We must have flushed a leaf at some point. */ - assert( pWriter->has_parent ); - - /* Tenatively set the end leaf blockid as the end blockid. If the - ** interior node can be returned inline, this will be the final - ** blockid, otherwise it will be overwritten by - ** interiorWriterRootInfo(). - */ - *piEndBlockid = pWriter->iEndBlockid; - - return interiorWriterRootInfo(v, &pWriter->parentWriter, - ppRootInfo, pnRootInfo, piEndBlockid); -} - -/* Collect the rootInfo data and store it into the segment directory. -** This has the effect of flushing the segment's leaf data to -** %_segments, and also flushing any interior nodes to %_segments. -*/ -static int leafWriterFinalize(fulltext_vtab *v, LeafWriter *pWriter){ - sqlite_int64 iEndBlockid; - char *pRootInfo; - int rc, nRootInfo; - - rc = leafWriterRootInfo(v, pWriter, &pRootInfo, &nRootInfo, &iEndBlockid); - if( rc!=SQLITE_OK ) return rc; - - /* Don't bother storing an entirely empty segment. */ - if( iEndBlockid==0 && nRootInfo==0 ) return SQLITE_OK; - - return segdir_set(v, pWriter->iLevel, pWriter->idx, - pWriter->iStartBlockid, pWriter->iEndBlockid, - iEndBlockid, pRootInfo, nRootInfo); -} - -static void leafWriterDestroy(LeafWriter *pWriter){ - if( pWriter->has_parent ) interiorWriterDestroy(&pWriter->parentWriter); - dataBufferDestroy(&pWriter->term); - dataBufferDestroy(&pWriter->data); -} - -/* Encode a term into the leafWriter, delta-encoding as appropriate. -** Returns the length of the new term which distinguishes it from the -** previous term, which can be used to set nTermDistinct when a node -** boundary is crossed. -*/ -static int leafWriterEncodeTerm(LeafWriter *pWriter, - const char *pTerm, int nTerm){ - char c[VARINT_MAX+VARINT_MAX]; - int n, nPrefix = 0; - - assert( nTerm>0 ); - while( nPrefixterm.nData && - pTerm[nPrefix]==pWriter->term.pData[nPrefix] ){ - nPrefix++; - /* Failing this implies that the terms weren't in order. */ - assert( nPrefixdata.nData==0 ){ - /* Encode the node header and leading term as: - ** varint(0) - ** varint(nTerm) - ** char pTerm[nTerm] - */ - n = fts3PutVarint(c, '\0'); - n += fts3PutVarint(c+n, nTerm); - dataBufferAppend2(&pWriter->data, c, n, pTerm, nTerm); - }else{ - /* Delta-encode the term as: - ** varint(nPrefix) - ** varint(nSuffix) - ** char pTermSuffix[nSuffix] - */ - n = fts3PutVarint(c, nPrefix); - n += fts3PutVarint(c+n, nTerm-nPrefix); - dataBufferAppend2(&pWriter->data, c, n, pTerm+nPrefix, nTerm-nPrefix); - } - dataBufferReplace(&pWriter->term, pTerm, nTerm); - - return nPrefix+1; -} - -/* Used to avoid a memmove when a large amount of doclist data is in -** the buffer. This constructs a node and term header before -** iDoclistData and flushes the resulting complete node using -** leafWriterInternalFlush(). -*/ -static int leafWriterInlineFlush(fulltext_vtab *v, LeafWriter *pWriter, - const char *pTerm, int nTerm, - int iDoclistData){ - char c[VARINT_MAX+VARINT_MAX]; - int iData, n = fts3PutVarint(c, 0); - n += fts3PutVarint(c+n, nTerm); - - /* There should always be room for the header. Even if pTerm shared - ** a substantial prefix with the previous term, the entire prefix - ** could be constructed from earlier data in the doclist, so there - ** should be room. - */ - assert( iDoclistData>=n+nTerm ); - - iData = iDoclistData-(n+nTerm); - memcpy(pWriter->data.pData+iData, c, n); - memcpy(pWriter->data.pData+iData+n, pTerm, nTerm); - - return leafWriterInternalFlush(v, pWriter, iData, pWriter->data.nData-iData); -} - -/* Push pTerm[nTerm] along with the doclist data to the leaf layer of -** %_segments. -*/ -static int leafWriterStepMerge(fulltext_vtab *v, LeafWriter *pWriter, - const char *pTerm, int nTerm, - DLReader *pReaders, int nReaders){ - char c[VARINT_MAX+VARINT_MAX]; - int iTermData = pWriter->data.nData, iDoclistData; - int i, nData, n, nActualData, nActual, rc, nTermDistinct; - - ASSERT_VALID_LEAF_NODE(pWriter->data.pData, pWriter->data.nData); - nTermDistinct = leafWriterEncodeTerm(pWriter, pTerm, nTerm); - - /* Remember nTermDistinct if opening a new node. */ - if( iTermData==0 ) pWriter->nTermDistinct = nTermDistinct; - - iDoclistData = pWriter->data.nData; - - /* Estimate the length of the merged doclist so we can leave space - ** to encode it. - */ - for(i=0, nData=0; idata, c, n); - - docListMerge(&pWriter->data, pReaders, nReaders); - ASSERT_VALID_DOCLIST(DL_DEFAULT, - pWriter->data.pData+iDoclistData+n, - pWriter->data.nData-iDoclistData-n, NULL); - - /* The actual amount of doclist data at this point could be smaller - ** than the length we encoded. Additionally, the space required to - ** encode this length could be smaller. For small doclists, this is - ** not a big deal, we can just use memmove() to adjust things. - */ - nActualData = pWriter->data.nData-(iDoclistData+n); - nActual = fts3PutVarint(c, nActualData); - assert( nActualData<=nData ); - assert( nActual<=n ); - - /* If the new doclist is big enough for force a standalone leaf - ** node, we can immediately flush it inline without doing the - ** memmove(). - */ - /* TODO(shess) This test matches leafWriterStep(), which does this - ** test before it knows the cost to varint-encode the term and - ** doclist lengths. At some point, change to - ** pWriter->data.nData-iTermData>STANDALONE_MIN. - */ - if( nTerm+nActualData>STANDALONE_MIN ){ - /* Push leaf node from before this term. */ - if( iTermData>0 ){ - rc = leafWriterInternalFlush(v, pWriter, 0, iTermData); - if( rc!=SQLITE_OK ) return rc; - - pWriter->nTermDistinct = nTermDistinct; - } - - /* Fix the encoded doclist length. */ - iDoclistData += n - nActual; - memcpy(pWriter->data.pData+iDoclistData, c, nActual); - - /* Push the standalone leaf node. */ - rc = leafWriterInlineFlush(v, pWriter, pTerm, nTerm, iDoclistData); - if( rc!=SQLITE_OK ) return rc; - - /* Leave the node empty. */ - dataBufferReset(&pWriter->data); - - return rc; - } - - /* At this point, we know that the doclist was small, so do the - ** memmove if indicated. - */ - if( nActualdata.pData+iDoclistData+nActual, - pWriter->data.pData+iDoclistData+n, - pWriter->data.nData-(iDoclistData+n)); - pWriter->data.nData -= n-nActual; - } - - /* Replace written length with actual length. */ - memcpy(pWriter->data.pData+iDoclistData, c, nActual); - - /* If the node is too large, break things up. */ - /* TODO(shess) This test matches leafWriterStep(), which does this - ** test before it knows the cost to varint-encode the term and - ** doclist lengths. At some point, change to - ** pWriter->data.nData>LEAF_MAX. - */ - if( iTermData+nTerm+nActualData>LEAF_MAX ){ - /* Flush out the leading data as a node */ - rc = leafWriterInternalFlush(v, pWriter, 0, iTermData); - if( rc!=SQLITE_OK ) return rc; - - pWriter->nTermDistinct = nTermDistinct; - - /* Rebuild header using the current term */ - n = fts3PutVarint(pWriter->data.pData, 0); - n += fts3PutVarint(pWriter->data.pData+n, nTerm); - memcpy(pWriter->data.pData+n, pTerm, nTerm); - n += nTerm; - - /* There should always be room, because the previous encoding - ** included all data necessary to construct the term. - */ - assert( ndata.nData-iDoclistDatadata.pData+n, - pWriter->data.pData+iDoclistData, - pWriter->data.nData-iDoclistData); - pWriter->data.nData -= iDoclistData-n; - } - ASSERT_VALID_LEAF_NODE(pWriter->data.pData, pWriter->data.nData); - - return SQLITE_OK; -} - -/* Push pTerm[nTerm] along with the doclist data to the leaf layer of -** %_segments. -*/ -/* TODO(shess) Revise writeZeroSegment() so that doclists are -** constructed directly in pWriter->data. -*/ -static int leafWriterStep(fulltext_vtab *v, LeafWriter *pWriter, - const char *pTerm, int nTerm, - const char *pData, int nData){ - int rc; - DLReader reader; - - dlrInit(&reader, DL_DEFAULT, pData, nData); - rc = leafWriterStepMerge(v, pWriter, pTerm, nTerm, &reader, 1); - dlrDestroy(&reader); - - return rc; -} - - -/****************************************************************/ -/* LeafReader is used to iterate over an individual leaf node. */ -typedef struct LeafReader { - DataBuffer term; /* copy of current term. */ - - const char *pData; /* data for current term. */ - int nData; -} LeafReader; - -static void leafReaderDestroy(LeafReader *pReader){ - dataBufferDestroy(&pReader->term); - SCRAMBLE(pReader); -} - -static int leafReaderAtEnd(LeafReader *pReader){ - return pReader->nData<=0; -} - -/* Access the current term. */ -static int leafReaderTermBytes(LeafReader *pReader){ - return pReader->term.nData; -} -static const char *leafReaderTerm(LeafReader *pReader){ - assert( pReader->term.nData>0 ); - return pReader->term.pData; -} - -/* Access the doclist data for the current term. */ -static int leafReaderDataBytes(LeafReader *pReader){ - int nData; - assert( pReader->term.nData>0 ); - fts3GetVarint32(pReader->pData, &nData); - return nData; -} -static const char *leafReaderData(LeafReader *pReader){ - int n, nData; - assert( pReader->term.nData>0 ); - n = fts3GetVarint32(pReader->pData, &nData); - return pReader->pData+n; -} - -static void leafReaderInit(const char *pData, int nData, - LeafReader *pReader){ - int nTerm, n; - - assert( nData>0 ); - assert( pData[0]=='\0' ); - - CLEAR(pReader); - - /* Read the first term, skipping the header byte. */ - n = fts3GetVarint32(pData+1, &nTerm); - dataBufferInit(&pReader->term, nTerm); - dataBufferReplace(&pReader->term, pData+1+n, nTerm); - - /* Position after the first term. */ - assert( 1+n+nTermpData = pData+1+n+nTerm; - pReader->nData = nData-1-n-nTerm; -} - -/* Step the reader forward to the next term. */ -static void leafReaderStep(LeafReader *pReader){ - int n, nData, nPrefix, nSuffix; - assert( !leafReaderAtEnd(pReader) ); - - /* Skip previous entry's data block. */ - n = fts3GetVarint32(pReader->pData, &nData); - assert( n+nData<=pReader->nData ); - pReader->pData += n+nData; - pReader->nData -= n+nData; - - if( !leafReaderAtEnd(pReader) ){ - /* Construct the new term using a prefix from the old term plus a - ** suffix from the leaf data. - */ - n = fts3GetVarint32(pReader->pData, &nPrefix); - n += fts3GetVarint32(pReader->pData+n, &nSuffix); - assert( n+nSuffixnData ); - pReader->term.nData = nPrefix; - dataBufferAppend(&pReader->term, pReader->pData+n, nSuffix); - - pReader->pData += n+nSuffix; - pReader->nData -= n+nSuffix; - } -} - -/* strcmp-style comparison of pReader's current term against pTerm. -** If isPrefix, equality means equal through nTerm bytes. -*/ -static int leafReaderTermCmp(LeafReader *pReader, - const char *pTerm, int nTerm, int isPrefix){ - int c, n = pReader->term.nDataterm.nData : nTerm; - if( n==0 ){ - if( pReader->term.nData>0 ) return -1; - if(nTerm>0 ) return 1; - return 0; - } - - c = memcmp(pReader->term.pData, pTerm, n); - if( c!=0 ) return c; - if( isPrefix && n==nTerm ) return 0; - return pReader->term.nData - nTerm; -} - - -/****************************************************************/ -/* LeavesReader wraps LeafReader to allow iterating over the entire -** leaf layer of the tree. -*/ -typedef struct LeavesReader { - int idx; /* Index within the segment. */ - - sqlite3_stmt *pStmt; /* Statement we're streaming leaves from. */ - int eof; /* we've seen SQLITE_DONE from pStmt. */ - - LeafReader leafReader; /* reader for the current leaf. */ - DataBuffer rootData; /* root data for inline. */ -} LeavesReader; - -/* Access the current term. */ -static int leavesReaderTermBytes(LeavesReader *pReader){ - assert( !pReader->eof ); - return leafReaderTermBytes(&pReader->leafReader); -} -static const char *leavesReaderTerm(LeavesReader *pReader){ - assert( !pReader->eof ); - return leafReaderTerm(&pReader->leafReader); -} - -/* Access the doclist data for the current term. */ -static int leavesReaderDataBytes(LeavesReader *pReader){ - assert( !pReader->eof ); - return leafReaderDataBytes(&pReader->leafReader); -} -static const char *leavesReaderData(LeavesReader *pReader){ - assert( !pReader->eof ); - return leafReaderData(&pReader->leafReader); -} - -static int leavesReaderAtEnd(LeavesReader *pReader){ - return pReader->eof; -} - -/* loadSegmentLeaves() may not read all the way to SQLITE_DONE, thus -** leaving the statement handle open, which locks the table. -*/ -/* TODO(shess) This "solution" is not satisfactory. Really, there -** should be check-in function for all statement handles which -** arranges to call sqlite3_reset(). This most likely will require -** modification to control flow all over the place, though, so for now -** just punt. -** -** Note the the current system assumes that segment merges will run to -** completion, which is why this particular probably hasn't arisen in -** this case. Probably a brittle assumption. -*/ -static int leavesReaderReset(LeavesReader *pReader){ - return sqlite3_reset(pReader->pStmt); -} - -static void leavesReaderDestroy(LeavesReader *pReader){ - /* If idx is -1, that means we're using a non-cached statement - ** handle in the optimize() case, so we need to release it. - */ - if( pReader->pStmt!=NULL && pReader->idx==-1 ){ - sqlite3_finalize(pReader->pStmt); - } - leafReaderDestroy(&pReader->leafReader); - dataBufferDestroy(&pReader->rootData); - SCRAMBLE(pReader); -} - -/* Initialize pReader with the given root data (if iStartBlockid==0 -** the leaf data was entirely contained in the root), or from the -** stream of blocks between iStartBlockid and iEndBlockid, inclusive. -*/ -static int leavesReaderInit(fulltext_vtab *v, - int idx, - sqlite_int64 iStartBlockid, - sqlite_int64 iEndBlockid, - const char *pRootData, int nRootData, - LeavesReader *pReader){ - CLEAR(pReader); - pReader->idx = idx; - - dataBufferInit(&pReader->rootData, 0); - if( iStartBlockid==0 ){ - /* Entire leaf level fit in root data. */ - dataBufferReplace(&pReader->rootData, pRootData, nRootData); - leafReaderInit(pReader->rootData.pData, pReader->rootData.nData, - &pReader->leafReader); - }else{ - sqlite3_stmt *s; - int rc = sql_get_leaf_statement(v, idx, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 1, iStartBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 2, iEndBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - if( rc==SQLITE_DONE ){ - pReader->eof = 1; - return SQLITE_OK; - } - if( rc!=SQLITE_ROW ) return rc; - - pReader->pStmt = s; - leafReaderInit(sqlite3_column_blob(pReader->pStmt, 0), - sqlite3_column_bytes(pReader->pStmt, 0), - &pReader->leafReader); + *pRowid = sqlite3_column_int64(pCsr->pStmt, 0); } return SQLITE_OK; } -/* Step the current leaf forward to the next term. If we reach the -** end of the current leaf, step forward to the next leaf block. +/* +** This is the xColumn method, called by SQLite to request a value from +** the row that the supplied cursor currently points to. */ -static int leavesReaderStep(fulltext_vtab *v, LeavesReader *pReader){ - assert( !leavesReaderAtEnd(pReader) ); - leafReaderStep(&pReader->leafReader); - - if( leafReaderAtEnd(&pReader->leafReader) ){ - int rc; - if( pReader->rootData.pData ){ - pReader->eof = 1; - return SQLITE_OK; - } - rc = sqlite3_step(pReader->pStmt); - if( rc!=SQLITE_ROW ){ - pReader->eof = 1; - return rc==SQLITE_DONE ? SQLITE_OK : rc; - } - leafReaderDestroy(&pReader->leafReader); - leafReaderInit(sqlite3_column_blob(pReader->pStmt, 0), - sqlite3_column_bytes(pReader->pStmt, 0), - &pReader->leafReader); - } - return SQLITE_OK; -} - -/* Order LeavesReaders by their term, ignoring idx. Readers at eof -** always sort to the end. -*/ -static int leavesReaderTermCmp(LeavesReader *lr1, LeavesReader *lr2){ - if( leavesReaderAtEnd(lr1) ){ - if( leavesReaderAtEnd(lr2) ) return 0; - return 1; - } - if( leavesReaderAtEnd(lr2) ) return -1; - - return leafReaderTermCmp(&lr1->leafReader, - leavesReaderTerm(lr2), leavesReaderTermBytes(lr2), - 0); -} - -/* Similar to leavesReaderTermCmp(), with additional ordering by idx -** so that older segments sort before newer segments. -*/ -static int leavesReaderCmp(LeavesReader *lr1, LeavesReader *lr2){ - int c = leavesReaderTermCmp(lr1, lr2); - if( c!=0 ) return c; - return lr1->idx-lr2->idx; -} - -/* Assume that pLr[1]..pLr[nLr] are sorted. Bubble pLr[0] into its -** sorted position. -*/ -static void leavesReaderReorder(LeavesReader *pLr, int nLr){ - while( nLr>1 && leavesReaderCmp(pLr, pLr+1)>0 ){ - LeavesReader tmp = pLr[0]; - pLr[0] = pLr[1]; - pLr[1] = tmp; - nLr--; - pLr++; - } -} - -/* Initializes pReaders with the segments from level iLevel, returning -** the number of segments in *piReaders. Leaves pReaders in sorted -** order. -*/ -static int leavesReadersInit(fulltext_vtab *v, int iLevel, - LeavesReader *pReaders, int *piReaders){ - sqlite3_stmt *s; - int i, rc = sql_get_statement(v, SEGDIR_SELECT_LEVEL_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int(s, 1, iLevel); - if( rc!=SQLITE_OK ) return rc; - - i = 0; - while( (rc = sqlite3_step(s))==SQLITE_ROW ){ - sqlite_int64 iStart = sqlite3_column_int64(s, 0); - sqlite_int64 iEnd = sqlite3_column_int64(s, 1); - const char *pRootData = sqlite3_column_blob(s, 2); - int nRootData = sqlite3_column_bytes(s, 2); - - assert( i0 ){ - leavesReaderDestroy(&pReaders[i]); - } - return rc; - } - - *piReaders = i; - - /* Leave our results sorted by term, then age. */ - while( i-- ){ - leavesReaderReorder(pReaders+i, *piReaders-i); - } - return SQLITE_OK; -} - -/* Merge doclists from pReaders[nReaders] into a single doclist, which -** is written to pWriter. Assumes pReaders is ordered oldest to -** newest. -*/ -/* TODO(shess) Consider putting this inline in segmentMerge(). */ -static int leavesReadersMerge(fulltext_vtab *v, - LeavesReader *pReaders, int nReaders, - LeafWriter *pWriter){ - DLReader dlReaders[MERGE_COUNT]; - const char *pTerm = leavesReaderTerm(pReaders); - int i, nTerm = leavesReaderTermBytes(pReaders); - - assert( nReaders<=MERGE_COUNT ); - - for(i=0; i0 ){ - rc = leavesReaderStep(v, lrs+i); - if( rc!=SQLITE_OK ) goto err; - - /* Reorder by term, then by age. */ - leavesReaderReorder(lrs+i, MERGE_COUNT-i); - } - } - - for(i=0; i0 ); - - for(rc=SQLITE_OK; rc==SQLITE_OK && !leavesReaderAtEnd(pReader); - rc=leavesReaderStep(v, pReader)){ - /* TODO(shess) Really want leavesReaderTermCmp(), but that name is - ** already taken to compare the terms of two LeavesReaders. Think - ** on a better name. [Meanwhile, break encapsulation rather than - ** use a confusing name.] - */ - int c = leafReaderTermCmp(&pReader->leafReader, pTerm, nTerm, isPrefix); - if( c>0 ) break; /* Past any possible matches. */ - if( c==0 ){ - const char *pData = leavesReaderData(pReader); - int iBuffer, nData = leavesReaderDataBytes(pReader); - - /* Find the first empty buffer. */ - for(iBuffer=0; iBuffer0 ){ - assert(pBuffers!=NULL); - memcpy(p, pBuffers, nBuffers*sizeof(*pBuffers)); - sqlite3_free(pBuffers); - } - pBuffers = p; - } - dataBufferInit(&(pBuffers[nBuffers]), 0); - nBuffers++; - } - - /* At this point, must have an empty at iBuffer. */ - assert(iBufferpData, p->nData); - - /* dataBufferReset() could allow a large doclist to blow up - ** our memory requirements. - */ - if( p->nCapacity<1024 ){ - dataBufferReset(p); - }else{ - dataBufferDestroy(p); - dataBufferInit(p, 0); - } - } - } - } - } - - /* Union all the doclists together into *out. */ - /* TODO(shess) What if *out is big? Sigh. */ - if( rc==SQLITE_OK && nBuffers>0 ){ - int iBuffer; - for(iBuffer=0; iBuffer0 ){ - if( out->nData==0 ){ - dataBufferSwap(out, &(pBuffers[iBuffer])); - }else{ - docListAccumulateUnion(out, pBuffers[iBuffer].pData, - pBuffers[iBuffer].nData); - } - } - } - } - - while( nBuffers-- ){ - dataBufferDestroy(&(pBuffers[nBuffers])); - } - if( pBuffers!=NULL ) sqlite3_free(pBuffers); - - return rc; -} - -/* Call loadSegmentLeavesInt() with pData/nData as input. */ -static int loadSegmentLeaf(fulltext_vtab *v, const char *pData, int nData, - const char *pTerm, int nTerm, int isPrefix, - DataBuffer *out){ - LeavesReader reader; - int rc; - - assert( nData>1 ); - assert( *pData=='\0' ); - rc = leavesReaderInit(v, 0, 0, 0, pData, nData, &reader); - if( rc!=SQLITE_OK ) return rc; - - rc = loadSegmentLeavesInt(v, &reader, pTerm, nTerm, isPrefix, out); - leavesReaderReset(&reader); - leavesReaderDestroy(&reader); - return rc; -} - -/* Call loadSegmentLeavesInt() with the leaf nodes from iStartLeaf to -** iEndLeaf (inclusive) as input, and merge the resulting doclist into -** out. -*/ -static int loadSegmentLeaves(fulltext_vtab *v, - sqlite_int64 iStartLeaf, sqlite_int64 iEndLeaf, - const char *pTerm, int nTerm, int isPrefix, - DataBuffer *out){ - int rc; - LeavesReader reader; - - assert( iStartLeaf<=iEndLeaf ); - rc = leavesReaderInit(v, 0, iStartLeaf, iEndLeaf, NULL, 0, &reader); - if( rc!=SQLITE_OK ) return rc; - - rc = loadSegmentLeavesInt(v, &reader, pTerm, nTerm, isPrefix, out); - leavesReaderReset(&reader); - leavesReaderDestroy(&reader); - return rc; -} - -/* Taking pData/nData as an interior node, find the sequence of child -** nodes which could include pTerm/nTerm/isPrefix. Note that the -** interior node terms logically come between the blocks, so there is -** one more blockid than there are terms (that block contains terms >= -** the last interior-node term). -*/ -/* TODO(shess) The calling code may already know that the end child is -** not worth calculating, because the end may be in a later sibling -** node. Consider whether breaking symmetry is worthwhile. I suspect -** it is not worthwhile. -*/ -static void getChildrenContaining(const char *pData, int nData, - const char *pTerm, int nTerm, int isPrefix, - sqlite_int64 *piStartChild, - sqlite_int64 *piEndChild){ - InteriorReader reader; - - assert( nData>1 ); - assert( *pData!='\0' ); - interiorReaderInit(pData, nData, &reader); - - /* Scan for the first child which could contain pTerm/nTerm. */ - while( !interiorReaderAtEnd(&reader) ){ - if( interiorReaderTermCmp(&reader, pTerm, nTerm, 0)>0 ) break; - interiorReaderStep(&reader); - } - *piStartChild = interiorReaderCurrentBlockid(&reader); - - /* Keep scanning to find a term greater than our term, using prefix - ** comparison if indicated. If isPrefix is false, this will be the - ** same blockid as the starting block. - */ - while( !interiorReaderAtEnd(&reader) ){ - if( interiorReaderTermCmp(&reader, pTerm, nTerm, isPrefix)>0 ) break; - interiorReaderStep(&reader); - } - *piEndChild = interiorReaderCurrentBlockid(&reader); - - interiorReaderDestroy(&reader); - - /* Children must ascend, and if !prefix, both must be the same. */ - assert( *piEndChild>=*piStartChild ); - assert( isPrefix || *piStartChild==*piEndChild ); -} - -/* Read block at iBlockid and pass it with other params to -** getChildrenContaining(). -*/ -static int loadAndGetChildrenContaining( - fulltext_vtab *v, - sqlite_int64 iBlockid, - const char *pTerm, int nTerm, int isPrefix, - sqlite_int64 *piStartChild, sqlite_int64 *piEndChild +static int fts3ColumnMethod( + sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ + sqlite3_context *pContext, /* Context for sqlite3_result_xxx() calls */ + int iCol /* Index of column to read value from */ ){ - sqlite3_stmt *s = NULL; - int rc; + int rc; /* Return Code */ + Fts3Cursor *pCsr = (Fts3Cursor *) pCursor; + Fts3Table *p = (Fts3Table *)pCursor->pVtab; - assert( iBlockid!=0 ); - assert( pTerm!=NULL ); - assert( nTerm!=0 ); /* TODO(shess) Why not allow this? */ - assert( piStartChild!=NULL ); - assert( piEndChild!=NULL ); + /* The column value supplied by SQLite must be in range. */ + assert( iCol>=0 && iCol<=p->nColumn+1 ); - rc = sql_get_statement(v, BLOCK_SELECT_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_bind_int64(s, 1, iBlockid); - if( rc!=SQLITE_OK ) return rc; - - rc = sqlite3_step(s); - if( rc==SQLITE_DONE ) return SQLITE_ERROR; - if( rc!=SQLITE_ROW ) return rc; - - getChildrenContaining(sqlite3_column_blob(s, 0), sqlite3_column_bytes(s, 0), - pTerm, nTerm, isPrefix, piStartChild, piEndChild); - - /* We expect only one row. We must execute another sqlite3_step() - * to complete the iteration; otherwise the table will remain - * locked. */ - rc = sqlite3_step(s); - if( rc==SQLITE_ROW ) return SQLITE_ERROR; - if( rc!=SQLITE_DONE ) return rc; - - return SQLITE_OK; -} - -/* Traverse the tree represented by pData[nData] looking for -** pTerm[nTerm], placing its doclist into *out. This is internal to -** loadSegment() to make error-handling cleaner. -*/ -static int loadSegmentInt(fulltext_vtab *v, const char *pData, int nData, - sqlite_int64 iLeavesEnd, - const char *pTerm, int nTerm, int isPrefix, - DataBuffer *out){ - /* Special case where root is a leaf. */ - if( *pData=='\0' ){ - return loadSegmentLeaf(v, pData, nData, pTerm, nTerm, isPrefix, out); - }else{ - int rc; - sqlite_int64 iStartChild, iEndChild; - - /* Process pData as an interior node, then loop down the tree - ** until we find the set of leaf nodes to scan for the term. - */ - getChildrenContaining(pData, nData, pTerm, nTerm, isPrefix, - &iStartChild, &iEndChild); - while( iStartChild>iLeavesEnd ){ - sqlite_int64 iNextStart, iNextEnd; - rc = loadAndGetChildrenContaining(v, iStartChild, pTerm, nTerm, isPrefix, - &iNextStart, &iNextEnd); - if( rc!=SQLITE_OK ) return rc; - - /* If we've branched, follow the end branch, too. */ - if( iStartChild!=iEndChild ){ - sqlite_int64 iDummy; - rc = loadAndGetChildrenContaining(v, iEndChild, pTerm, nTerm, isPrefix, - &iDummy, &iNextEnd); - if( rc!=SQLITE_OK ) return rc; - } - - assert( iNextStart<=iNextEnd ); - iStartChild = iNextStart; - iEndChild = iNextEnd; - } - assert( iStartChild<=iLeavesEnd ); - assert( iEndChild<=iLeavesEnd ); - - /* Scan through the leaf segments for doclists. */ - return loadSegmentLeaves(v, iStartChild, iEndChild, - pTerm, nTerm, isPrefix, out); - } -} - -/* Call loadSegmentInt() to collect the doclist for pTerm/nTerm, then -** merge its doclist over *out (any duplicate doclists read from the -** segment rooted at pData will overwrite those in *out). -*/ -/* TODO(shess) Consider changing this to determine the depth of the -** leaves using either the first characters of interior nodes (when -** ==1, we're one level above the leaves), or the first character of -** the root (which will describe the height of the tree directly). -** Either feels somewhat tricky to me. -*/ -/* TODO(shess) The current merge is likely to be slow for large -** doclists (though it should process from newest/smallest to -** oldest/largest, so it may not be that bad). It might be useful to -** modify things to allow for N-way merging. This could either be -** within a segment, with pairwise merges across segments, or across -** all segments at once. -*/ -static int loadSegment(fulltext_vtab *v, const char *pData, int nData, - sqlite_int64 iLeavesEnd, - const char *pTerm, int nTerm, int isPrefix, - DataBuffer *out){ - DataBuffer result; - int rc; - - assert( nData>1 ); - - /* This code should never be called with buffered updates. */ - assert( v->nPendingData<0 ); - - dataBufferInit(&result, 0); - rc = loadSegmentInt(v, pData, nData, iLeavesEnd, - pTerm, nTerm, isPrefix, &result); - if( rc==SQLITE_OK && result.nData>0 ){ - if( out->nData==0 ){ - DataBuffer tmp = *out; - *out = result; - result = tmp; + rc = fts3CursorSeek(pCsr); + if( rc==SQLITE_OK ){ + if( iCol==p->nColumn+1 ){ + /* This call is a request for the "docid" column. Since "docid" is an + ** alias for "rowid", use the xRowid() method to obtain the value. + */ + sqlite3_int64 iRowid; + rc = fts3RowidMethod(pCursor, &iRowid); + sqlite3_result_int64(pContext, iRowid); + }else if( iCol==p->nColumn ){ + /* The extra column whose name is the same as the table. + ** Return a blob which is a pointer to the cursor. + */ + sqlite3_result_blob(pContext, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT); }else{ - DataBuffer merged; - DLReader readers[2]; - - dlrInit(&readers[0], DL_DEFAULT, out->pData, out->nData); - dlrInit(&readers[1], DL_DEFAULT, result.pData, result.nData); - dataBufferInit(&merged, out->nData+result.nData); - docListMerge(&merged, readers, 2); - dataBufferDestroy(out); - *out = merged; - dlrDestroy(&readers[0]); - dlrDestroy(&readers[1]); + sqlite3_result_value(pContext, sqlite3_column_value(pCsr->pStmt, iCol+1)); } } - dataBufferDestroy(&result); return rc; } -/* Scan the database and merge together the posting lists for the term -** into *out. +/* +** This function is the implementation of the xUpdate callback used by +** FTS3 virtual tables. It is invoked by SQLite each time a row is to be +** inserted, updated or deleted. */ -static int termSelect( - fulltext_vtab *v, - int iColumn, - const char *pTerm, int nTerm, /* Term to query for */ - int isPrefix, /* True for a prefix search */ - DocListType iType, - DataBuffer *out /* Write results here */ +static int fts3UpdateMethod( + sqlite3_vtab *pVtab, /* Virtual table handle */ + int nArg, /* Size of argument array */ + sqlite3_value **apVal, /* Array of arguments */ + sqlite_int64 *pRowid /* OUT: The affected (or effected) rowid */ ){ - DataBuffer doclist; - sqlite3_stmt *s; - int rc = sql_get_statement(v, SEGDIR_SELECT_ALL_STMT, &s); - if( rc!=SQLITE_OK ) return rc; - - /* This code should never be called with buffered updates. */ - assert( v->nPendingData<0 ); - - dataBufferInit(&doclist, 0); - dataBufferInit(out, 0); - - /* Traverse the segments from oldest to newest so that newer doclist - ** elements for given docids overwrite older elements. - */ - while( (rc = sqlite3_step(s))==SQLITE_ROW ){ - const char *pData = sqlite3_column_blob(s, 2); - const int nData = sqlite3_column_bytes(s, 2); - const sqlite_int64 iLeavesEnd = sqlite3_column_int64(s, 1); - rc = loadSegment(v, pData, nData, iLeavesEnd, pTerm, nTerm, isPrefix, - &doclist); - if( rc!=SQLITE_OK ) goto err; - } - if( rc==SQLITE_DONE ){ - if( doclist.nData!=0 ){ - /* TODO(shess) The old term_select_all() code applied the column - ** restrict as we merged segments, leading to smaller buffers. - ** This is probably worthwhile to bring back, once the new storage - ** system is checked in. - */ - if( iColumn==v->nColumn) iColumn = -1; - docListTrim(DL_DEFAULT, doclist.pData, doclist.nData, - iColumn, iType, out); - } - rc = SQLITE_OK; - } - - err: - dataBufferDestroy(&doclist); - return rc; + return sqlite3Fts3UpdateMethod(pVtab, nArg, apVal, pRowid); } -/****************************************************************/ -/* Used to hold hashtable data for sorting. */ -typedef struct TermData { - const char *pTerm; - int nTerm; - DLCollector *pCollector; -} TermData; - -/* Orders TermData elements in strcmp fashion ( <0 for less-than, 0 -** for equal, >0 for greater-than). +/* +** Implementation of xSync() method. Flush the contents of the pending-terms +** hash-table to the database. */ -static int termDataCmp(const void *av, const void *bv){ - const TermData *a = (const TermData *)av; - const TermData *b = (const TermData *)bv; - int n = a->nTermnTerm ? a->nTerm : b->nTerm; - int c = memcmp(a->pTerm, b->pTerm, n); - if( c!=0 ) return c; - return a->nTerm-b->nTerm; +static int fts3SyncMethod(sqlite3_vtab *pVtab){ + return sqlite3Fts3PendingTermsFlush((Fts3Table *)pVtab); } -/* Order pTerms data by term, then write a new level 0 segment using -** LeafWriter. +/* +** Implementation of xBegin() method. This is a no-op. */ -static int writeZeroSegment(fulltext_vtab *v, fts3Hash *pTerms){ - fts3HashElem *e; - int idx, rc, i, n; - TermData *pData; - LeafWriter writer; - DataBuffer dl; - - /* Determine the next index at level 0, merging as necessary. */ - rc = segdirNextIndex(v, 0, &idx); - if( rc!=SQLITE_OK ) return rc; - - n = fts3HashCount(pTerms); - pData = sqlite3_malloc(n*sizeof(TermData)); - - for(i = 0, e = fts3HashFirst(pTerms); e; i++, e = fts3HashNext(e)){ - assert( i1 ) qsort(pData, n, sizeof(*pData), termDataCmp); - - /* TODO(shess) Refactor so that we can write directly to the segment - ** DataBuffer, as happens for segment merges. - */ - leafWriterInit(0, idx, &writer); - dataBufferInit(&dl, 0); - for(i=0; inPendingData>=0 ){ - fts3HashElem *e; - for(e=fts3HashFirst(&v->pendingTerms); e; e=fts3HashNext(e)){ - dlcDelete(fts3HashData(e)); - } - fts3HashClear(&v->pendingTerms); - v->nPendingData = -1; - } +static int fts3BeginMethod(sqlite3_vtab *pVtab){ + UNUSED_PARAMETER(pVtab); + assert( ((Fts3Table *)pVtab)->nPendingData==0 ); return SQLITE_OK; } -/* If pendingTerms has data, flush it to a level-zero segment, and -** free it. +/* +** Implementation of xCommit() method. This is a no-op. The contents of +** the pending-terms hash-table have already been flushed into the database +** by fts3SyncMethod(). */ -static int flushPendingTerms(fulltext_vtab *v){ - if( v->nPendingData>=0 ){ - int rc = writeZeroSegment(v, &v->pendingTerms); - if( rc==SQLITE_OK ) clearPendingTerms(v); - return rc; - } +static int fts3CommitMethod(sqlite3_vtab *pVtab){ + UNUSED_PARAMETER(pVtab); + assert( ((Fts3Table *)pVtab)->nPendingData==0 ); return SQLITE_OK; } -/* If pendingTerms is "too big", or docid is out of order, flush it. -** Regardless, be certain that pendingTerms is initialized for use. +/* +** Implementation of xRollback(). Discard the contents of the pending-terms +** hash-table. Any changes made to the database are reverted by SQLite. */ -static int initPendingTerms(fulltext_vtab *v, sqlite_int64 iDocid){ - /* TODO(shess) Explore whether partially flushing the buffer on - ** forced-flush would provide better performance. I suspect that if - ** we ordered the doclists by size and flushed the largest until the - ** buffer was half empty, that would let the less frequent terms - ** generate longer doclists. - */ - if( iDocid<=v->iPrevDocid || v->nPendingData>kPendingThreshold ){ - int rc = flushPendingTerms(v); - if( rc!=SQLITE_OK ) return rc; - } - if( v->nPendingData<0 ){ - fts3HashInit(&v->pendingTerms, FTS3_HASH_STRING, 1); - v->nPendingData = 0; - } - v->iPrevDocid = iDocid; +static int fts3RollbackMethod(sqlite3_vtab *pVtab){ + sqlite3Fts3PendingTermsClear((Fts3Table *)pVtab); return SQLITE_OK; } -/* This function implements the xUpdate callback; it is the top-level entry - * point for inserting, deleting or updating a row in a full-text table. */ -static int fulltextUpdate(sqlite3_vtab *pVtab, int nArg, sqlite3_value **ppArg, - sqlite_int64 *pRowid){ - fulltext_vtab *v = (fulltext_vtab *) pVtab; - int rc; - - FTSTRACE(("FTS3 Update %p\n", pVtab)); - - if( nArg<2 ){ - rc = index_delete(v, sqlite3_value_int64(ppArg[0])); - if( rc==SQLITE_OK ){ - /* If we just deleted the last row in the table, clear out the - ** index data. - */ - rc = content_exists(v); - if( rc==SQLITE_ROW ){ - rc = SQLITE_OK; - }else if( rc==SQLITE_DONE ){ - /* Clear the pending terms so we don't flush a useless level-0 - ** segment when the transaction closes. - */ - rc = clearPendingTerms(v); - if( rc==SQLITE_OK ){ - rc = segdir_delete_all(v); - } - } - } - } else if( sqlite3_value_type(ppArg[0]) != SQLITE_NULL ){ - /* An update: - * ppArg[0] = old rowid - * ppArg[1] = new rowid - * ppArg[2..2+v->nColumn-1] = values - * ppArg[2+v->nColumn] = value for magic column (we ignore this) - * ppArg[2+v->nColumn+1] = value for docid - */ - sqlite_int64 rowid = sqlite3_value_int64(ppArg[0]); - if( sqlite3_value_type(ppArg[1]) != SQLITE_INTEGER || - sqlite3_value_int64(ppArg[1]) != rowid ){ - rc = SQLITE_ERROR; /* we don't allow changing the rowid */ - }else if( sqlite3_value_type(ppArg[2+v->nColumn+1]) != SQLITE_INTEGER || - sqlite3_value_int64(ppArg[2+v->nColumn+1]) != rowid ){ - rc = SQLITE_ERROR; /* we don't allow changing the docid */ - }else{ - assert( nArg==2+v->nColumn+2); - rc = index_update(v, rowid, &ppArg[2]); - } - } else { - /* An insert: - * ppArg[1] = requested rowid - * ppArg[2..2+v->nColumn-1] = values - * ppArg[2+v->nColumn] = value for magic column (we ignore this) - * ppArg[2+v->nColumn+1] = value for docid - */ - sqlite3_value *pRequestDocid = ppArg[2+v->nColumn+1]; - assert( nArg==2+v->nColumn+2); - if( SQLITE_NULL != sqlite3_value_type(pRequestDocid) && - SQLITE_NULL != sqlite3_value_type(ppArg[1]) ){ - /* TODO(shess) Consider allowing this to work if the values are - ** identical. I'm inclined to discourage that usage, though, - ** given that both rowid and docid are special columns. Better - ** would be to define one or the other as the default winner, - ** but should it be fts3-centric (docid) or SQLite-centric - ** (rowid)? - */ - rc = SQLITE_ERROR; - }else{ - if( SQLITE_NULL == sqlite3_value_type(pRequestDocid) ){ - pRequestDocid = ppArg[1]; - } - rc = index_insert(v, pRequestDocid, &ppArg[2], pRowid); - } +/* +** Helper function used by the implementation of the overloaded snippet(), +** offsets() and optimize() SQL functions. +** +** If the value passed as the third argument is a blob of size +** sizeof(Fts3Cursor*), then the blob contents are copied to the +** output variable *ppCsr and SQLITE_OK is returned. Otherwise, an error +** message is written to context pContext and SQLITE_ERROR returned. The +** string passed via zFunc is used as part of the error message. +*/ +static int fts3FunctionArg( + sqlite3_context *pContext, /* SQL function call context */ + const char *zFunc, /* Function name */ + sqlite3_value *pVal, /* argv[0] passed to function */ + Fts3Cursor **ppCsr /* OUT: Store cursor handle here */ +){ + Fts3Cursor *pRet; + if( sqlite3_value_type(pVal)!=SQLITE_BLOB + || sqlite3_value_bytes(pVal)!=sizeof(Fts3Cursor *) + ){ + char *zErr = sqlite3_mprintf("illegal first argument to %s", zFunc); + sqlite3_result_error(pContext, zErr, -1); + sqlite3_free(zErr); + return SQLITE_ERROR; } - - return rc; -} - -static int fulltextSync(sqlite3_vtab *pVtab){ - FTSTRACE(("FTS3 xSync()\n")); - return flushPendingTerms((fulltext_vtab *)pVtab); -} - -static int fulltextBegin(sqlite3_vtab *pVtab){ - fulltext_vtab *v = (fulltext_vtab *) pVtab; - FTSTRACE(("FTS3 xBegin()\n")); - - /* Any buffered updates should have been cleared by the previous - ** transaction. - */ - assert( v->nPendingData<0 ); - return clearPendingTerms(v); -} - -static int fulltextCommit(sqlite3_vtab *pVtab){ - fulltext_vtab *v = (fulltext_vtab *) pVtab; - FTSTRACE(("FTS3 xCommit()\n")); - - /* Buffered updates should have been cleared by fulltextSync(). */ - assert( v->nPendingData<0 ); - return clearPendingTerms(v); -} - -static int fulltextRollback(sqlite3_vtab *pVtab){ - FTSTRACE(("FTS3 xRollback()\n")); - return clearPendingTerms((fulltext_vtab *)pVtab); + memcpy(&pRet, sqlite3_value_blob(pVal), sizeof(Fts3Cursor *)); + *ppCsr = pRet; + return SQLITE_OK; } /* ** Implementation of the snippet() function for FTS3 */ -static void snippetFunc( - sqlite3_context *pContext, - int argc, - sqlite3_value **argv +static void fts3SnippetFunc( + sqlite3_context *pContext, /* SQLite function call context */ + int nVal, /* Size of apVal[] array */ + sqlite3_value **apVal /* Array of arguments */ ){ - fulltext_cursor *pCursor; - if( argc<1 ) return; - if( sqlite3_value_type(argv[0])!=SQLITE_BLOB || - sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){ - sqlite3_result_error(pContext, "illegal first argument to html_snippet",-1); - }else{ - const char *zStart = ""; - const char *zEnd = ""; - const char *zEllipsis = "..."; - memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor)); - if( argc>=2 ){ - zStart = (const char*)sqlite3_value_text(argv[1]); - if( argc>=3 ){ - zEnd = (const char*)sqlite3_value_text(argv[2]); - if( argc>=4 ){ - zEllipsis = (const char*)sqlite3_value_text(argv[3]); - } - } - } - snippetAllOffsets(pCursor); - snippetText(pCursor, zStart, zEnd, zEllipsis); - sqlite3_result_text(pContext, pCursor->snippet.zSnippet, - pCursor->snippet.nSnippet, SQLITE_STATIC); + Fts3Cursor *pCsr; /* Cursor handle passed through apVal[0] */ + const char *zStart = ""; + const char *zEnd = ""; + const char *zEllipsis = "..."; + + /* There must be at least one argument passed to this function (otherwise + ** the non-overloaded version would have been called instead of this one). + */ + assert( nVal>=1 ); + + if( nVal>4 ){ + sqlite3_result_error(pContext, + "wrong number of arguments to function snippet()", -1); + return; } + if( fts3FunctionArg(pContext, "snippet", apVal[0], &pCsr) ) return; + + switch( nVal ){ + case 4: zEllipsis = (const char*)sqlite3_value_text(apVal[3]); + case 3: zEnd = (const char*)sqlite3_value_text(apVal[2]); + case 2: zStart = (const char*)sqlite3_value_text(apVal[1]); + } + + sqlite3Fts3Snippet(pContext, pCsr, zStart, zEnd, zEllipsis); } /* ** Implementation of the offsets() function for FTS3 */ -static void snippetOffsetsFunc( - sqlite3_context *pContext, - int argc, - sqlite3_value **argv +static void fts3OffsetsFunc( + sqlite3_context *pContext, /* SQLite function call context */ + int nVal, /* Size of argument array */ + sqlite3_value **apVal /* Array of arguments */ ){ - fulltext_cursor *pCursor; - if( argc<1 ) return; - if( sqlite3_value_type(argv[0])!=SQLITE_BLOB || - sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){ - sqlite3_result_error(pContext, "illegal first argument to offsets",-1); - }else{ - memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor)); - snippetAllOffsets(pCursor); - snippetOffsetText(&pCursor->snippet); - sqlite3_result_text(pContext, - pCursor->snippet.zOffset, pCursor->snippet.nOffset, - SQLITE_STATIC); - } + Fts3Cursor *pCsr; /* Cursor handle passed through apVal[0] */ + + UNUSED_PARAMETER(nVal); + + assert( nVal==1 ); + if( fts3FunctionArg(pContext, "offsets", apVal[0], &pCsr) ) return; + assert( pCsr ); + sqlite3Fts3Offsets(pContext, pCsr); } -/* OptLeavesReader is nearly identical to LeavesReader, except that -** where LeavesReader is geared towards the merging of complete -** segment levels (with exactly MERGE_COUNT segments), OptLeavesReader -** is geared towards implementation of the optimize() function, and -** can merge all segments simultaneously. This version may be -** somewhat less efficient than LeavesReader because it merges into an -** accumulator rather than doing an N-way merge, but since segment -** size grows exponentially (so segment count logrithmically) this is -** probably not an immediate problem. +/* +** Implementation of the special optimize() function for FTS3. This +** function merges all segments in the database to a single segment. +** Example usage is: +** +** SELECT optimize(t) FROM t LIMIT 1; +** +** where 't' is the name of an FTS3 table. */ -/* TODO(shess): Prove that assertion, or extend the merge code to -** merge tree fashion (like the prefix-searching code does). -*/ -/* TODO(shess): OptLeavesReader and LeavesReader could probably be -** merged with little or no loss of performance for LeavesReader. The -** merged code would need to handle >MERGE_COUNT segments, and would -** also need to be able to optionally optimize away deletes. -*/ -typedef struct OptLeavesReader { - /* Segment number, to order readers by age. */ - int segment; - LeavesReader reader; -} OptLeavesReader; - -static int optLeavesReaderAtEnd(OptLeavesReader *pReader){ - return leavesReaderAtEnd(&pReader->reader); -} -static int optLeavesReaderTermBytes(OptLeavesReader *pReader){ - return leavesReaderTermBytes(&pReader->reader); -} -static const char *optLeavesReaderData(OptLeavesReader *pReader){ - return leavesReaderData(&pReader->reader); -} -static int optLeavesReaderDataBytes(OptLeavesReader *pReader){ - return leavesReaderDataBytes(&pReader->reader); -} -static const char *optLeavesReaderTerm(OptLeavesReader *pReader){ - return leavesReaderTerm(&pReader->reader); -} -static int optLeavesReaderStep(fulltext_vtab *v, OptLeavesReader *pReader){ - return leavesReaderStep(v, &pReader->reader); -} -static int optLeavesReaderTermCmp(OptLeavesReader *lr1, OptLeavesReader *lr2){ - return leavesReaderTermCmp(&lr1->reader, &lr2->reader); -} -/* Order by term ascending, segment ascending (oldest to newest), with -** exhausted readers to the end. -*/ -static int optLeavesReaderCmp(OptLeavesReader *lr1, OptLeavesReader *lr2){ - int c = optLeavesReaderTermCmp(lr1, lr2); - if( c!=0 ) return c; - return lr1->segment-lr2->segment; -} -/* Bubble pLr[0] to appropriate place in pLr[1..nLr-1]. Assumes that -** pLr[1..nLr-1] is already sorted. -*/ -static void optLeavesReaderReorder(OptLeavesReader *pLr, int nLr){ - while( nLr>1 && optLeavesReaderCmp(pLr, pLr+1)>0 ){ - OptLeavesReader tmp = pLr[0]; - pLr[0] = pLr[1]; - pLr[1] = tmp; - nLr--; - pLr++; - } -} - -/* optimize() helper function. Put the readers in order and iterate -** through them, merging doclists for matching terms into pWriter. -** Returns SQLITE_OK on success, or the SQLite error code which -** prevented success. -*/ -static int optimizeInternal(fulltext_vtab *v, - OptLeavesReader *readers, int nReaders, - LeafWriter *pWriter){ - int i, rc = SQLITE_OK; - DataBuffer doclist, merged, tmp; - - /* Order the readers. */ - i = nReaders; - while( i-- > 0 ){ - optLeavesReaderReorder(&readers[i], nReaders-i); - } - - dataBufferInit(&doclist, LEAF_MAX); - dataBufferInit(&merged, LEAF_MAX); - - /* Exhausted readers bubble to the end, so when the first reader is - ** at eof, all are at eof. - */ - while( !optLeavesReaderAtEnd(&readers[0]) ){ - - /* Figure out how many readers share the next term. */ - for(i=1; i 0 ){ - dlrDestroy(&dlReaders[nReaders]); - } - - /* Accumulated doclist to reader 0 for next pass. */ - dlrInit(&dlReaders[0], DL_DEFAULT, doclist.pData, doclist.nData); - } - - /* Destroy reader that was left in the pipeline. */ - dlrDestroy(&dlReaders[0]); - - /* Trim deletions from the doclist. */ - dataBufferReset(&merged); - docListTrim(DL_DEFAULT, doclist.pData, doclist.nData, - -1, DL_DEFAULT, &merged); - } - - /* Only pass doclists with hits (skip if all hits deleted). */ - if( merged.nData>0 ){ - rc = leafWriterStep(v, pWriter, - optLeavesReaderTerm(&readers[0]), - optLeavesReaderTermBytes(&readers[0]), - merged.pData, merged.nData); - if( rc!=SQLITE_OK ) goto err; - } - - /* Step merged readers to next term and reorder. */ - while( i-- > 0 ){ - rc = optLeavesReaderStep(v, &readers[i]); - if( rc!=SQLITE_OK ) goto err; - - optLeavesReaderReorder(&readers[i], nReaders-i); - } - } - - err: - dataBufferDestroy(&doclist); - dataBufferDestroy(&merged); - return rc; -} - -/* Implement optimize() function for FTS3. optimize(t) merges all -** segments in the fts index into a single segment. 't' is the magic -** table-named column. -*/ -static void optimizeFunc(sqlite3_context *pContext, - int argc, sqlite3_value **argv){ - fulltext_cursor *pCursor; - if( argc>1 ){ - sqlite3_result_error(pContext, "excess arguments to optimize()",-1); - }else if( sqlite3_value_type(argv[0])!=SQLITE_BLOB || - sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){ - sqlite3_result_error(pContext, "illegal first argument to optimize",-1); - }else{ - fulltext_vtab *v; - int i, rc, iMaxLevel; - OptLeavesReader *readers; - int nReaders; - LeafWriter writer; - sqlite3_stmt *s; - - memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor)); - v = cursor_vtab(pCursor); - - /* Flush any buffered updates before optimizing. */ - rc = flushPendingTerms(v); - if( rc!=SQLITE_OK ) goto err; - - rc = segdir_count(v, &nReaders, &iMaxLevel); - if( rc!=SQLITE_OK ) goto err; - if( nReaders==0 || nReaders==1 ){ - sqlite3_result_text(pContext, "Index already optimal", -1, - SQLITE_STATIC); - return; - } - - rc = sql_get_statement(v, SEGDIR_SELECT_ALL_STMT, &s); - if( rc!=SQLITE_OK ) goto err; - - readers = sqlite3_malloc(nReaders*sizeof(readers[0])); - if( readers==NULL ) goto err; - - /* Note that there will already be a segment at this position - ** until we call segdir_delete() on iMaxLevel. - */ - leafWriterInit(iMaxLevel, 0, &writer); - - i = 0; - while( (rc = sqlite3_step(s))==SQLITE_ROW ){ - sqlite_int64 iStart = sqlite3_column_int64(s, 0); - sqlite_int64 iEnd = sqlite3_column_int64(s, 1); - const char *pRootData = sqlite3_column_blob(s, 2); - int nRootData = sqlite3_column_bytes(s, 2); - - assert( i 0 ){ - leavesReaderDestroy(&readers[i].reader); - } - sqlite3_free(readers); - - /* If we've successfully gotten to here, delete the old segments - ** and flush the interior structure of the new segment. - */ - if( rc==SQLITE_OK ){ - for( i=0; i<=iMaxLevel; i++ ){ - rc = segdir_delete(v, i); - if( rc!=SQLITE_OK ) break; - } - - if( rc==SQLITE_OK ) rc = leafWriterFinalize(v, &writer); - } - - leafWriterDestroy(&writer); - - if( rc!=SQLITE_OK ) goto err; - - sqlite3_result_text(pContext, "Index optimized", -1, SQLITE_STATIC); - return; - - /* TODO(shess): Error-handling needs to be improved along the - ** lines of the dump_ functions. - */ - err: - { - char buf[512]; - sqlite3_snprintf(sizeof(buf), buf, "Error in optimize: %s", - sqlite3_errmsg(sqlite3_context_db_handle(pContext))); - sqlite3_result_error(pContext, buf, -1); - } - } -} - -#ifdef SQLITE_TEST -/* Generate an error of the form ": ". If msg is NULL, -** pull the error from the context's db handle. -*/ -static void generateError(sqlite3_context *pContext, - const char *prefix, const char *msg){ - char buf[512]; - if( msg==NULL ) msg = sqlite3_errmsg(sqlite3_context_db_handle(pContext)); - sqlite3_snprintf(sizeof(buf), buf, "%s: %s", prefix, msg); - sqlite3_result_error(pContext, buf, -1); -} - -/* Helper function to collect the set of terms in the segment into -** pTerms. The segment is defined by the leaf nodes between -** iStartBlockid and iEndBlockid, inclusive, or by the contents of -** pRootData if iStartBlockid is 0 (in which case the entire segment -** fit in a leaf). -*/ -static int collectSegmentTerms(fulltext_vtab *v, sqlite3_stmt *s, - fts3Hash *pTerms){ - const sqlite_int64 iStartBlockid = sqlite3_column_int64(s, 0); - const sqlite_int64 iEndBlockid = sqlite3_column_int64(s, 1); - const char *pRootData = sqlite3_column_blob(s, 2); - const int nRootData = sqlite3_column_bytes(s, 2); - LeavesReader reader; - int rc = leavesReaderInit(v, 0, iStartBlockid, iEndBlockid, - pRootData, nRootData, &reader); - if( rc!=SQLITE_OK ) return rc; - - while( rc==SQLITE_OK && !leavesReaderAtEnd(&reader) ){ - const char *pTerm = leavesReaderTerm(&reader); - const int nTerm = leavesReaderTermBytes(&reader); - void *oldValue = sqlite3Fts3HashFind(pTerms, pTerm, nTerm); - void *newValue = (void *)((char *)oldValue+1); - - /* From the comment before sqlite3Fts3HashInsert in fts3_hash.c, - ** the data value passed is returned in case of malloc failure. - */ - if( newValue==sqlite3Fts3HashInsert(pTerms, pTerm, nTerm, newValue) ){ - rc = SQLITE_NOMEM; - }else{ - rc = leavesReaderStep(v, &reader); - } - } - - leavesReaderDestroy(&reader); - return rc; -} - -/* Helper function to build the result string for dump_terms(). */ -static int generateTermsResult(sqlite3_context *pContext, fts3Hash *pTerms){ - int iTerm, nTerms, nResultBytes, iByte; - char *result; - TermData *pData; - fts3HashElem *e; - - /* Iterate pTerms to generate an array of terms in pData for - ** sorting. - */ - nTerms = fts3HashCount(pTerms); - assert( nTerms>0 ); - pData = sqlite3_malloc(nTerms*sizeof(TermData)); - if( pData==NULL ) return SQLITE_NOMEM; - - nResultBytes = 0; - for(iTerm = 0, e = fts3HashFirst(pTerms); e; iTerm++, e = fts3HashNext(e)){ - nResultBytes += fts3HashKeysize(e)+1; /* Term plus trailing space */ - assert( iTerm0 ); /* nTerms>0, nResultsBytes must be, too. */ - result = sqlite3_malloc(nResultBytes); - if( result==NULL ){ - sqlite3_free(pData); - return SQLITE_NOMEM; - } - - if( nTerms>1 ) qsort(pData, nTerms, sizeof(*pData), termDataCmp); - - /* Read the terms in order to build the result. */ - iByte = 0; - for(iTerm=0; iTermbase.pVtab; + assert( p ); - if( rc!=SQLITE_OK ){ - generateError(pContext, "dump_terms", NULL); - return; - } + rc = sqlite3Fts3Optimize(p); - /* Collect the terms for each segment. */ - sqlite3Fts3HashInit(&terms, FTS3_HASH_STRING, 1); - while( (rc = sqlite3_step(s))==SQLITE_ROW ){ - rc = collectSegmentTerms(v, s, &terms); - if( rc!=SQLITE_OK ) break; - } - - if( rc!=SQLITE_DONE ){ - sqlite3_reset(s); - generateError(pContext, "dump_terms", NULL); - }else{ - const int nTerms = fts3HashCount(&terms); - if( nTerms>0 ){ - rc = generateTermsResult(pContext, &terms); - if( rc==SQLITE_NOMEM ){ - generateError(pContext, "dump_terms", "out of memory"); - }else{ - assert( rc==SQLITE_OK ); - } - }else if( argc==3 ){ - /* The specific segment asked for could not be found. */ - generateError(pContext, "dump_terms", "segment not found"); - }else{ - /* No segments found. */ - /* TODO(shess): It should be impossible to reach this. This - ** case can only happen for an empty table, in which case - ** SQLite has no rows to call this function on. - */ - sqlite3_result_null(pContext); - } - } - sqlite3Fts3HashClear(&terms); + switch( rc ){ + case SQLITE_OK: + sqlite3_result_text(pContext, "Index optimized", -1, SQLITE_STATIC); + break; + case SQLITE_DONE: + sqlite3_result_text(pContext, "Index already optimal", -1, SQLITE_STATIC); + break; + default: + sqlite3_result_error_code(pContext, rc); + break; } } -/* Expand the DL_DEFAULT doclist in pData into a text result in -** pContext. -*/ -static void createDoclistResult(sqlite3_context *pContext, - const char *pData, int nData){ - DataBuffer dump; - DLReader dlReader; - - assert( pData!=NULL && nData>0 ); - - dataBufferInit(&dump, 0); - dlrInit(&dlReader, DL_DEFAULT, pData, nData); - for( ; !dlrAtEnd(&dlReader); dlrStep(&dlReader) ){ - char buf[256]; - PLReader plReader; - - plrInit(&plReader, &dlReader); - if( DL_DEFAULT==DL_DOCIDS || plrAtEnd(&plReader) ){ - sqlite3_snprintf(sizeof(buf), buf, "[%lld] ", dlrDocid(&dlReader)); - dataBufferAppend(&dump, buf, strlen(buf)); - }else{ - int iColumn = plrColumn(&plReader); - - sqlite3_snprintf(sizeof(buf), buf, "[%lld %d[", - dlrDocid(&dlReader), iColumn); - dataBufferAppend(&dump, buf, strlen(buf)); - - for( ; !plrAtEnd(&plReader); plrStep(&plReader) ){ - if( plrColumn(&plReader)!=iColumn ){ - iColumn = plrColumn(&plReader); - sqlite3_snprintf(sizeof(buf), buf, "] %d[", iColumn); - assert( dump.nData>0 ); - dump.nData--; /* Overwrite trailing space. */ - assert( dump.pData[dump.nData]==' '); - dataBufferAppend(&dump, buf, strlen(buf)); - } - if( DL_DEFAULT==DL_POSITIONS_OFFSETS ){ - sqlite3_snprintf(sizeof(buf), buf, "%d,%d,%d ", - plrPosition(&plReader), - plrStartOffset(&plReader), plrEndOffset(&plReader)); - }else if( DL_DEFAULT==DL_POSITIONS ){ - sqlite3_snprintf(sizeof(buf), buf, "%d ", plrPosition(&plReader)); - }else{ - assert( NULL=="Unhandled DL_DEFAULT value"); - } - dataBufferAppend(&dump, buf, strlen(buf)); - } - plrDestroy(&plReader); - - assert( dump.nData>0 ); - dump.nData--; /* Overwrite trailing space. */ - assert( dump.pData[dump.nData]==' '); - dataBufferAppend(&dump, "]] ", 3); - } - } - dlrDestroy(&dlReader); - - assert( dump.nData>0 ); - dump.nData--; /* Overwrite trailing space. */ - assert( dump.pData[dump.nData]==' '); - dump.pData[dump.nData] = '\0'; - assert( dump.nData>0 ); - - /* Passes ownership of dump's buffer to pContext. */ - sqlite3_result_text(pContext, dump.pData, dump.nData, sqlite3_free); - dump.pData = NULL; - dump.nData = dump.nCapacity = 0; -} - -/* Implements dump_doclist() for use in inspecting the fts3 index from -** tests. TEXT result containing a string representation of the -** doclist for the indicated term. dump_doclist(t, term, level, idx) -** dumps the doclist for term from the segment specified by level, idx -** (in %_segdir), while dump_doclist(t, term) dumps the logical -** doclist for the term across all segments. The per-segment doclist -** can contain deletions, while the full-index doclist will not -** (deletions are omitted). -** -** Result formats differ with the setting of DL_DEFAULTS. Examples: -** -** DL_DOCIDS: [1] [3] [7] -** DL_POSITIONS: [1 0[0 4] 1[17]] [3 1[5]] -** DL_POSITIONS_OFFSETS: [1 0[0,0,3 4,23,26] 1[17,102,105]] [3 1[5,20,23]] -** -** In each case the number after the outer '[' is the docid. In the -** latter two cases, the number before the inner '[' is the column -** associated with the values within. For DL_POSITIONS the numbers -** within are the positions, for DL_POSITIONS_OFFSETS they are the -** position, the start offset, and the end offset. -*/ -static void dumpDoclistFunc( - sqlite3_context *pContext, - int argc, sqlite3_value **argv -){ - fulltext_cursor *pCursor; - if( argc!=2 && argc!=4 ){ - generateError(pContext, "dump_doclist", "incorrect arguments"); - }else if( sqlite3_value_type(argv[0])!=SQLITE_BLOB || - sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){ - generateError(pContext, "dump_doclist", "illegal first argument"); - }else if( sqlite3_value_text(argv[1])==NULL || - sqlite3_value_text(argv[1])[0]=='\0' ){ - generateError(pContext, "dump_doclist", "empty second argument"); - }else{ - const char *pTerm = (const char *)sqlite3_value_text(argv[1]); - const int nTerm = strlen(pTerm); - fulltext_vtab *v; - int rc; - DataBuffer doclist; - - memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor)); - v = cursor_vtab(pCursor); - - dataBufferInit(&doclist, 0); - - /* termSelect() yields the same logical doclist that queries are - ** run against. - */ - if( argc==2 ){ - rc = termSelect(v, v->nColumn, pTerm, nTerm, 0, DL_DEFAULT, &doclist); - }else{ - sqlite3_stmt *s = NULL; - - /* Get our specific segment's information. */ - rc = sql_get_statement(v, SEGDIR_SELECT_SEGMENT_STMT, &s); - if( rc==SQLITE_OK ){ - rc = sqlite3_bind_int(s, 1, sqlite3_value_int(argv[2])); - if( rc==SQLITE_OK ){ - rc = sqlite3_bind_int(s, 2, sqlite3_value_int(argv[3])); - } - } - - if( rc==SQLITE_OK ){ - rc = sqlite3_step(s); - - if( rc==SQLITE_DONE ){ - dataBufferDestroy(&doclist); - generateError(pContext, "dump_doclist", "segment not found"); - return; - } - - /* Found a segment, load it into doclist. */ - if( rc==SQLITE_ROW ){ - const sqlite_int64 iLeavesEnd = sqlite3_column_int64(s, 1); - const char *pData = sqlite3_column_blob(s, 2); - const int nData = sqlite3_column_bytes(s, 2); - - /* loadSegment() is used by termSelect() to load each - ** segment's data. - */ - rc = loadSegment(v, pData, nData, iLeavesEnd, pTerm, nTerm, 0, - &doclist); - if( rc==SQLITE_OK ){ - rc = sqlite3_step(s); - - /* Should not have more than one matching segment. */ - if( rc!=SQLITE_DONE ){ - sqlite3_reset(s); - dataBufferDestroy(&doclist); - generateError(pContext, "dump_doclist", "invalid segdir"); - return; - } - rc = SQLITE_OK; - } - } - } - - sqlite3_reset(s); - } - - if( rc==SQLITE_OK ){ - if( doclist.nData>0 ){ - createDoclistResult(pContext, doclist.pData, doclist.nData); - }else{ - /* TODO(shess): This can happen if the term is not present, or - ** if all instances of the term have been deleted and this is - ** an all-index dump. It may be interesting to distinguish - ** these cases. - */ - sqlite3_result_text(pContext, "", 0, SQLITE_STATIC); - } - }else if( rc==SQLITE_NOMEM ){ - /* Handle out-of-memory cases specially because if they are - ** generated in fts3 code they may not be reflected in the db - ** handle. - */ - /* TODO(shess): Handle this more comprehensively. - ** sqlite3ErrStr() has what I need, but is internal. - */ - generateError(pContext, "dump_doclist", "out of memory"); - }else{ - generateError(pContext, "dump_doclist", NULL); - } - - dataBufferDestroy(&doclist); - } -} -#endif - /* ** This routine implements the xFindFunction method for the FTS3 ** virtual table. */ -static int fulltextFindFunction( - sqlite3_vtab *pVtab, - int nArg, - const char *zName, - void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), - void **ppArg +static int fts3FindFunctionMethod( + sqlite3_vtab *pVtab, /* Virtual table handle */ + int nArg, /* Number of SQL function arguments */ + const char *zName, /* Name of SQL function */ + void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */ + void **ppArg /* Unused */ ){ - if( strcmp(zName,"snippet")==0 ){ - *pxFunc = snippetFunc; - return 1; - }else if( strcmp(zName,"offsets")==0 ){ - *pxFunc = snippetOffsetsFunc; - return 1; - }else if( strcmp(zName,"optimize")==0 ){ - *pxFunc = optimizeFunc; - return 1; -#ifdef SQLITE_TEST - /* NOTE(shess): These functions are present only for testing - ** purposes. No particular effort is made to optimize their - ** execution or how they build their results. - */ - }else if( strcmp(zName,"dump_terms")==0 ){ - /* fprintf(stderr, "Found dump_terms\n"); */ - *pxFunc = dumpTermsFunc; - return 1; - }else if( strcmp(zName,"dump_doclist")==0 ){ - /* fprintf(stderr, "Found dump_doclist\n"); */ - *pxFunc = dumpDoclistFunc; - return 1; -#endif + struct Overloaded { + const char *zName; + void (*xFunc)(sqlite3_context*,int,sqlite3_value**); + } aOverload[] = { + { "snippet", fts3SnippetFunc }, + { "offsets", fts3OffsetsFunc }, + { "optimize", fts3OptimizeFunc }, + }; + int i; /* Iterator variable */ + + UNUSED_PARAMETER(pVtab); + UNUSED_PARAMETER(nArg); + UNUSED_PARAMETER(ppArg); + + for(i=0; izDb, p->zName, zName - , p->zDb, p->zName, zName + , p->zDb, p->zName, zName + , p->zDb, p->zName, zName , p->zDb, p->zName, zName ); if( zSql ){ @@ -104592,29 +100546,34 @@ static int fulltextRename( static const sqlite3_module fts3Module = { /* iVersion */ 0, - /* xCreate */ fulltextCreate, - /* xConnect */ fulltextConnect, - /* xBestIndex */ fulltextBestIndex, - /* xDisconnect */ fulltextDisconnect, - /* xDestroy */ fulltextDestroy, - /* xOpen */ fulltextOpen, + /* xCreate */ fts3CreateMethod, + /* xConnect */ fts3ConnectMethod, + /* xBestIndex */ fts3BestIndexMethod, + /* xDisconnect */ fts3DisconnectMethod, + /* xDestroy */ fts3DestroyMethod, + /* xOpen */ fts3OpenMethod, /* xClose */ fulltextClose, - /* xFilter */ fulltextFilter, - /* xNext */ fulltextNext, - /* xEof */ fulltextEof, - /* xColumn */ fulltextColumn, - /* xRowid */ fulltextRowid, - /* xUpdate */ fulltextUpdate, - /* xBegin */ fulltextBegin, - /* xSync */ fulltextSync, - /* xCommit */ fulltextCommit, - /* xRollback */ fulltextRollback, - /* xFindFunction */ fulltextFindFunction, - /* xRename */ fulltextRename, + /* xFilter */ fts3FilterMethod, + /* xNext */ fts3NextMethod, + /* xEof */ fts3EofMethod, + /* xColumn */ fts3ColumnMethod, + /* xRowid */ fts3RowidMethod, + /* xUpdate */ fts3UpdateMethod, + /* xBegin */ fts3BeginMethod, + /* xSync */ fts3SyncMethod, + /* xCommit */ fts3CommitMethod, + /* xRollback */ fts3RollbackMethod, + /* xFindFunction */ fts3FindFunctionMethod, + /* xRename */ fts3RenameMethod, }; +/* +** This function is registered as the module destructor (called when an +** FTS3 enabled database connection is closed). It frees the memory +** allocated for the tokenizer hash table. +*/ static void hashDestroy(void *p){ - fts3Hash *pHash = (fts3Hash *)p; + Fts3Hash *pHash = (Fts3Hash *)p; sqlite3Fts3HashClear(pHash); sqlite3_free(pHash); } @@ -104634,8 +100593,6 @@ SQLITE_PRIVATE void sqlite3Fts3SimpleTokenizerModule(sqlite3_tokenizer_module co SQLITE_PRIVATE void sqlite3Fts3PorterTokenizerModule(sqlite3_tokenizer_module const**ppModule); SQLITE_PRIVATE void sqlite3Fts3IcuTokenizerModule(sqlite3_tokenizer_module const**ppModule); -SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, fts3Hash *, const char *); - /* ** Initialise the fts3 extension. If this extension is built as part ** of the sqlite library, then this function is called directly by @@ -104644,7 +100601,7 @@ SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, fts3Hash *, const char *) */ SQLITE_PRIVATE int sqlite3Fts3Init(sqlite3 *db){ int rc = SQLITE_OK; - fts3Hash *pHash = 0; + Fts3Hash *pHash = 0; const sqlite3_tokenizer_module *pSimple = 0; const sqlite3_tokenizer_module *pPorter = 0; const sqlite3_tokenizer_module *pIcu = 0; @@ -104656,7 +100613,7 @@ SQLITE_PRIVATE int sqlite3Fts3Init(sqlite3 *db){ #endif /* Allocate and initialise the hash-table used to store tokenizers. */ - pHash = sqlite3_malloc(sizeof(fts3Hash)); + pHash = sqlite3_malloc(sizeof(Fts3Hash)); if( !pHash ){ rc = SQLITE_NOMEM; }else{ @@ -104666,7 +100623,7 @@ SQLITE_PRIVATE int sqlite3Fts3Init(sqlite3 *db){ /* Load the built-in tokenizers into the hash table */ if( rc==SQLITE_OK ){ if( sqlite3Fts3HashInsert(pHash, "simple", 7, (void *)pSimple) - || sqlite3Fts3HashInsert(pHash, "porter", 7, (void *)pPorter) + || sqlite3Fts3HashInsert(pHash, "porter", 7, (void *)pPorter) || (pIcu && sqlite3Fts3HashInsert(pHash, "icu", 4, (void *)pIcu)) ){ rc = SQLITE_NOMEM; @@ -104677,19 +100634,15 @@ SQLITE_PRIVATE int sqlite3Fts3Init(sqlite3 *db){ sqlite3Fts3ExprInitTestInterface(db); #endif - /* Create the virtual table wrapper around the hash-table and overload + /* Create the virtual table wrapper around the hash-table and overload ** the two scalar functions. If this is successful, register the ** module with sqlite. */ - if( SQLITE_OK==rc + if( SQLITE_OK==rc && SQLITE_OK==(rc = sqlite3Fts3InitHashTable(db, pHash, "fts3_tokenizer")) && SQLITE_OK==(rc = sqlite3_overload_function(db, "snippet", -1)) - && SQLITE_OK==(rc = sqlite3_overload_function(db, "offsets", -1)) - && SQLITE_OK==(rc = sqlite3_overload_function(db, "optimize", -1)) -#ifdef SQLITE_TEST - && SQLITE_OK==(rc = sqlite3_overload_function(db, "dump_terms", -1)) - && SQLITE_OK==(rc = sqlite3_overload_function(db, "dump_doclist", -1)) -#endif + && SQLITE_OK==(rc = sqlite3_overload_function(db, "offsets", 1)) + && SQLITE_OK==(rc = sqlite3_overload_function(db, "optimize", 1)) ){ return sqlite3_create_module_v2( db, "fts3", &fts3Module, (void *)pHash, hashDestroy @@ -104707,7 +100660,7 @@ SQLITE_PRIVATE int sqlite3Fts3Init(sqlite3 *db){ #if !SQLITE_CORE SQLITE_API int sqlite3_extension_init( - sqlite3 *db, + sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ @@ -104716,7 +100669,7 @@ SQLITE_API int sqlite3_extension_init( } #endif -#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) */ +#endif /************** End of fts3.c ************************************************/ /************** Begin file fts3_expr.c ***************************************/ @@ -104733,15 +100686,14 @@ SQLITE_API int sqlite3_extension_init( ****************************************************************************** ** ** This module contains code that implements a parser for fts3 query strings -** (the right-hand argument to the MATCH operator). Because the supported +** (the right-hand argument to the MATCH operator). Because the supported ** syntax is relatively simple, the whole tokenizer/parser system is -** hand-coded. The public interface to this module is declared in source -** code file "fts3_expr.h". +** hand-coded. */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) /* -** By default, this module parses the legacy syntax that has been +** By default, this module parses the legacy syntax that has been ** traditionally used by fts3. Or, if SQLITE_ENABLE_FTS3_PARENTHESIS ** is defined, then it uses the new syntax. The differences between ** the new and the old syntaxes are: @@ -104750,7 +100702,7 @@ SQLITE_API int sqlite3_extension_init( ** ** b) The new syntax supports the AND and NOT operators. The old does not. ** -** c) The old syntax supports the "-" token qualifier. This is not +** c) The old syntax supports the "-" token qualifier. This is not ** supported by the new syntax (it is replaced by the NOT operator). ** ** d) When using the old syntax, the OR operator has a greater precedence @@ -104759,14 +100711,36 @@ SQLITE_API int sqlite3_extension_init( ** ** If compiled with SQLITE_TEST defined, then this module exports the ** symbol "int sqlite3_fts3_enable_parentheses". Setting this variable -** to zero causes the module to use the old syntax. If it is set to +** to zero causes the module to use the old syntax. If it is set to ** non-zero the new syntax is activated. This is so both syntaxes can ** be tested using a single build of testfixture. +** +** The following describes the syntax supported by the fts3 MATCH +** operator in a similar format to that used by the lemon parser +** generator. This module does not use actually lemon, it uses a +** custom parser. +** +** query ::= andexpr (OR andexpr)*. +** +** andexpr ::= notexpr (AND? notexpr)*. +** +** notexpr ::= nearexpr (NOT nearexpr|-TOKEN)*. +** notexpr ::= LP query RP. +** +** nearexpr ::= phrase (NEAR distance_opt nearexpr)*. +** +** distance_opt ::= . +** distance_opt ::= / INTEGER. +** +** phrase ::= TOKEN. +** phrase ::= COLUMN:TOKEN. +** phrase ::= "TOKEN TOKEN TOKEN...". */ + #ifdef SQLITE_TEST SQLITE_API int sqlite3_fts3_enable_parentheses = 0; #else -# ifdef SQLITE_ENABLE_FTS3_PARENTHESIS +# ifdef SQLITE_ENABLE_FTS3_PARENTHESIS # define sqlite3_fts3_enable_parentheses 1 # else # define sqlite3_fts3_enable_parentheses 0 @@ -104790,7 +100764,7 @@ struct ParseContext { }; /* -** This function is equivalent to the standard isspace() function. +** This function is equivalent to the standard isspace() function. ** ** The standard isspace() can be awkward to use safely, because although it ** is defined to accept an argument of type int, its behaviour when passed @@ -104810,7 +100784,7 @@ static int fts3isspace(char c){ ** structure of type FTSQUERY_PHRASE containing a phrase consisting of this ** single token and set *ppExpr to point to it. If the end of the buffer is ** reached before a token is found, set *ppExpr to zero. It is the -** responsibility of the caller to eventually deallocate the allocated +** responsibility of the caller to eventually deallocate the allocated ** Fts3Expr structure (if any) by passing it to sqlite3_free(). ** ** Return SQLITE_OK if successful, or SQLITE_NOMEM if a memory allocation @@ -104867,7 +100841,7 @@ static int getNextToken( pModule->xClose(pCursor); } - + *pnConsumed = nConsumed; *ppExpr = pRet; return rc; @@ -104890,7 +100864,7 @@ void *fts3ReallocOrFree(void *pOrig, int nNew){ ** Buffer zInput, length nInput, contains the contents of a quoted string ** that appeared as part of an fts3 query expression. Neither quote character ** is included in the buffer. This function attempts to tokenize the entire -** input buffer and create an Fts3Expr structure of type FTSQUERY_PHRASE +** input buffer and create an Fts3Expr structure of type FTSQUERY_PHRASE ** containing the results. ** ** If successful, SQLITE_OK is returned and *ppExpr set to point at the @@ -104949,7 +100923,7 @@ static int getNextString( if( rc==SQLITE_DONE ){ int jj; - char *zNew; + char *zNew = NULL; int nNew = 0; int nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase); nByte += (p?(p->pPhrase->nToken-1):0) * sizeof(struct PhraseToken); @@ -104994,7 +100968,7 @@ no_mem: static int fts3ExprParse(ParseContext *, const char *, int, Fts3Expr **, int *); /* -** The output variable *ppExpr is populated with an allocated Fts3Expr +** The output variable *ppExpr is populated with an allocated Fts3Expr ** structure, or set to 0 if the end of the input buffer is reached. ** ** Returns an SQLite error code. SQLITE_OK if everything works, SQLITE_NOMEM @@ -105008,7 +100982,7 @@ static int getNextNode( int *pnConsumed /* OUT: Number of bytes consumed */ ){ static const struct Fts3Keyword { - char z[4]; /* Keyword text */ + char *z; /* Keyword text */ unsigned char n; /* Length of the keyword */ unsigned char parenOnly; /* Only valid in paren mode */ unsigned char eType; /* Keyword code */ @@ -105028,7 +101002,7 @@ static int getNextNode( int nInput = n; /* Skip over any whitespace before checking for a keyword, an open or - ** close bracket, or a quoted string. + ** close bracket, or a quoted string. */ while( nInput>0 && fts3isspace(*zInput) ){ nInput--; @@ -105064,18 +101038,21 @@ static int getNextNode( /* At this point this is probably a keyword. But for that to be true, ** the next byte must contain either whitespace, an open or close - ** parenthesis, a quote character, or EOF. + ** parenthesis, a quote character, or EOF. */ cNext = zInput[nKey]; - if( fts3isspace(cNext) + if( fts3isspace(cNext) || cNext=='"' || cNext=='(' || cNext==')' || cNext==0 ){ pRet = (Fts3Expr *)sqlite3_malloc(sizeof(Fts3Expr)); + if( !pRet ){ + return SQLITE_NOMEM; + } memset(pRet, 0, sizeof(Fts3Expr)); pRet->eType = pKey->eType; pRet->nNear = nNear; *ppExpr = pRet; - *pnConsumed = (zInput - z) + nKey; + *pnConsumed = (int)((zInput - z) + nKey); return SQLITE_OK; } @@ -105095,14 +101072,14 @@ static int getNextNode( if( rc==SQLITE_OK && !*ppExpr ){ rc = SQLITE_DONE; } - *pnConsumed = (zInput - z) + 1 + nConsumed; + *pnConsumed = (int)((zInput - z) + 1 + nConsumed); return rc; } - + /* Check for a close bracket. */ if( *zInput==')' ){ pParse->nNest--; - *pnConsumed = (zInput - z) + 1; + *pnConsumed = (int)((zInput - z) + 1); return SQLITE_DONE; } } @@ -105114,7 +101091,7 @@ static int getNextNode( */ if( *zInput=='"' ){ for(ii=1; iiiDefaultCol; iColLen = 0; for(ii=0; iinCol; ii++){ const char *zStr = pParse->azCol[ii]; - int nStr = strlen(zStr); - if( nInput>nStr && zInput[nStr]==':' - && sqlite3_strnicmp(zStr, zInput, nStr)==0 + int nStr = (int)strlen(zStr); + if( nInput>nStr && zInput[nStr]==':' + && sqlite3_strnicmp(zStr, zInput, nStr)==0 ){ iCol = ii; - iColLen = ((zInput - z) + nStr + 1); + iColLen = (int)((zInput - z) + nStr + 1); break; } } @@ -105184,7 +101161,7 @@ static int opPrecedence(Fts3Expr *p){ } /* -** Argument ppHead contains a pointer to the current head of a query +** Argument ppHead contains a pointer to the current head of a query ** expression tree being parsed. pPrev is the expression node most recently ** inserted into the tree. This function adds pNew, which is always a binary ** operator node, into the expression tree based on the relative precedence @@ -105214,7 +101191,7 @@ static void insertBinaryOperator( /* ** Parse the fts3 query expression found in buffer z, length n. This function -** returns either when the end of the buffer is reached or an unmatched +** returns either when the end of the buffer is reached or an unmatched ** closing bracket - ')' - is encountered. ** ** If successful, SQLITE_OK is returned, *ppExpr is set to point to the @@ -105243,8 +101220,8 @@ static int fts3ExprParse( if( rc==SQLITE_OK ){ int isPhrase; - if( !sqlite3_fts3_enable_parentheses - && p->eType==FTSQUERY_PHRASE && p->pPhrase->isNot + if( !sqlite3_fts3_enable_parentheses + && p->eType==FTSQUERY_PHRASE && p->pPhrase->isNot ){ /* Create an implicit NOT operator. */ Fts3Expr *pNot = sqlite3_malloc(sizeof(Fts3Expr)); @@ -105276,7 +101253,7 @@ static int fts3ExprParse( rc = SQLITE_ERROR; goto exprparse_out; } - + if( isPhrase && !isRequirePhrase ){ /* Insert an implicit AND operator. */ Fts3Expr *pAnd; @@ -105310,7 +101287,7 @@ static int fts3ExprParse( rc = SQLITE_ERROR; goto exprparse_out; } - + if( isPhrase ){ if( pRet ){ assert( pPrev && pPrev->pLeft && pPrev->pRight==0 ); @@ -105378,7 +101355,7 @@ exprparse_out: ** The first parameter, pTokenizer, is passed the fts3 tokenizer module to ** use to normalize query tokens while parsing the expression. The azCol[] ** array, which is assumed to contain nCol entries, should contain the names -** of each column in the target fts3 table, in order from left to right. +** of each column in the target fts3 table, in order from left to right. ** Column names must be nul-terminated strings. ** ** The iDefaultCol parameter should be passed the index of the table column @@ -105408,7 +101385,7 @@ SQLITE_PRIVATE int sqlite3Fts3ExprParse( return SQLITE_OK; } if( n<0 ){ - n = strlen(z); + n = (int)strlen(z); } rc = fts3ExprParse(&sParse, z, n, ppExpr, &nParsed); @@ -105445,8 +101422,8 @@ SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *p){ ** Function to query the hash-table of tokenizers (see README.tokenizers). */ static int queryTestTokenizer( - sqlite3 *db, - const char *zName, + sqlite3 *db, + const char *zName, const sqlite3_tokenizer_module **pp ){ int rc; @@ -105462,7 +101439,7 @@ static int queryTestTokenizer( sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC); if( SQLITE_ROW==sqlite3_step(pStmt) ){ if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){ - memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp)); + memcpy((void *)pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp)); } } @@ -105472,7 +101449,7 @@ static int queryTestTokenizer( /* ** This function is part of the test interface for the query parser. It ** writes a text representation of the query expression pExpr into the -** buffer pointed to by argument zBuf. It is assumed that zBuf is large +** buffer pointed to by argument zBuf. It is assumed that zBuf is large ** enough to store the required text representation. */ static void exprToString(Fts3Expr *pExpr, char *zBuf){ @@ -105514,7 +101491,7 @@ static void exprToString(Fts3Expr *pExpr, char *zBuf){ } /* -** This is the implementation of a scalar SQL function used to test the +** This is the implementation of a scalar SQL function used to test the ** expression parser. It should be called as follows: ** ** fts3_exprtest(, , , ...); @@ -105544,7 +101521,7 @@ static void fts3ExprTest( sqlite3 *db = sqlite3_context_db_handle(context); if( argc<3 ){ - sqlite3_result_error(context, + sqlite3_result_error(context, "Usage: fts3_exprtest(tokenizer, expr, col1, ...", -1 ); return; @@ -105603,8 +101580,8 @@ exprtest_out: } /* -** Register the query expression parser test function fts3_exprtest() -** with database connection db. +** Register the query expression parser test function fts3_exprtest() +** with database connection db. */ SQLITE_PRIVATE void sqlite3Fts3ExprInitTestInterface(sqlite3* db){ sqlite3_create_function( @@ -105664,13 +101641,13 @@ static void fts3HashFree(void *p){ ** fields of the Hash structure. ** ** "pNew" is a pointer to the hash table that is to be initialized. -** keyClass is one of the constants -** FTS3_HASH_BINARY or FTS3_HASH_STRING. The value of keyClass +** keyClass is one of the constants +** FTS3_HASH_BINARY or FTS3_HASH_STRING. The value of keyClass ** determines what kind of key the hash table will use. "copyKey" is ** true if the hash table should make its own private copy of keys and ** false if it should just use the supplied pointer. */ -SQLITE_PRIVATE void sqlite3Fts3HashInit(fts3Hash *pNew, int keyClass, int copyKey){ +SQLITE_PRIVATE void sqlite3Fts3HashInit(Fts3Hash *pNew, char keyClass, char copyKey){ assert( pNew!=0 ); assert( keyClass>=FTS3_HASH_STRING && keyClass<=FTS3_HASH_BINARY ); pNew->keyClass = keyClass; @@ -105685,8 +101662,8 @@ SQLITE_PRIVATE void sqlite3Fts3HashInit(fts3Hash *pNew, int keyClass, int copyKe ** Call this routine to delete a hash table or to reset a hash table ** to the empty state. */ -SQLITE_PRIVATE void sqlite3Fts3HashClear(fts3Hash *pH){ - fts3HashElem *elem; /* For looping over all elements of the table */ +SQLITE_PRIVATE void sqlite3Fts3HashClear(Fts3Hash *pH){ + Fts3HashElem *elem; /* For looping over all elements of the table */ assert( pH!=0 ); elem = pH->first; @@ -105695,7 +101672,7 @@ SQLITE_PRIVATE void sqlite3Fts3HashClear(fts3Hash *pH){ pH->ht = 0; pH->htsize = 0; while( elem ){ - fts3HashElem *next_elem = elem->next; + Fts3HashElem *next_elem = elem->next; if( pH->copyKey && elem->pKey ){ fts3HashFree(elem->pKey); } @@ -105742,7 +101719,7 @@ static int fts3BinCompare(const void *pKey1, int n1, const void *pKey2, int n2){ /* ** Return a pointer to the appropriate hash function given the key class. ** -** The C syntax in this function definition may be unfamilar to some +** The C syntax in this function definition may be unfamilar to some ** programmers, so we provide the following additional explanation: ** ** The name of the function is "ftsHashFunction". The function takes a @@ -105778,11 +101755,11 @@ static int (*ftsCompareFunction(int keyClass))(const void*,int,const void*,int){ /* Link an element into the hash table */ static void fts3HashInsertElement( - fts3Hash *pH, /* The complete hash table */ + Fts3Hash *pH, /* The complete hash table */ struct _fts3ht *pEntry, /* The entry into which pNew is inserted */ - fts3HashElem *pNew /* The element to be inserted */ + Fts3HashElem *pNew /* The element to be inserted */ ){ - fts3HashElem *pHead; /* First element already in pEntry */ + Fts3HashElem *pHead; /* First element already in pEntry */ pHead = pEntry->chain; if( pHead ){ pNew->next = pHead; @@ -105802,17 +101779,19 @@ static void fts3HashInsertElement( /* Resize the hash table so that it cantains "new_size" buckets. -** "new_size" must be a power of 2. The hash table might fail +** "new_size" must be a power of 2. The hash table might fail ** to resize if sqliteMalloc() fails. +** +** Return non-zero if a memory allocation error occurs. */ -static void fts3Rehash(fts3Hash *pH, int new_size){ +static int fts3Rehash(Fts3Hash *pH, int new_size){ struct _fts3ht *new_ht; /* The new hash table */ - fts3HashElem *elem, *next_elem; /* For looping over existing elements */ + Fts3HashElem *elem, *next_elem; /* For looping over existing elements */ int (*xHash)(const void*,int); /* The hash function */ assert( (new_size & (new_size-1))==0 ); new_ht = (struct _fts3ht *)fts3HashMalloc( new_size*sizeof(struct _fts3ht) ); - if( new_ht==0 ) return; + if( new_ht==0 ) return 1; fts3HashFree(pH->ht); pH->ht = new_ht; pH->htsize = new_size; @@ -105822,19 +101801,20 @@ static void fts3Rehash(fts3Hash *pH, int new_size){ next_elem = elem->next; fts3HashInsertElement(pH, &new_ht[h], elem); } + return 0; } /* This function (for internal use only) locates an element in an ** hash table that matches the given key. The hash for this key has ** already been computed and is passed as the 4th parameter. */ -static fts3HashElem *fts3FindElementByHash( - const fts3Hash *pH, /* The pH to be searched */ +static Fts3HashElem *fts3FindElementByHash( + const Fts3Hash *pH, /* The pH to be searched */ const void *pKey, /* The key we are searching for */ int nKey, int h /* The hash for this key. */ ){ - fts3HashElem *elem; /* Used to loop thru the element list */ + Fts3HashElem *elem; /* Used to loop thru the element list */ int count; /* Number of elements left to test */ int (*xCompare)(const void*,int,const void*,int); /* comparison function */ @@ -105844,7 +101824,7 @@ static fts3HashElem *fts3FindElementByHash( count = pEntry->count; xCompare = ftsCompareFunction(pH->keyClass); while( count-- && elem ){ - if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){ + if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){ return elem; } elem = elem->next; @@ -105857,13 +101837,13 @@ static fts3HashElem *fts3FindElementByHash( ** element and a hash on the element's key. */ static void fts3RemoveElementByHash( - fts3Hash *pH, /* The pH containing "elem" */ - fts3HashElem* elem, /* The element to be removed from the pH */ + Fts3Hash *pH, /* The pH containing "elem" */ + Fts3HashElem* elem, /* The element to be removed from the pH */ int h /* Hash value for the element */ ){ struct _fts3ht *pEntry; if( elem->prev ){ - elem->prev->next = elem->next; + elem->prev->next = elem->next; }else{ pH->first = elem->next; } @@ -105894,9 +101874,9 @@ static void fts3RemoveElementByHash( ** that matches pKey,nKey. Return the data for this element if it is ** found, or NULL if there is no match. */ -SQLITE_PRIVATE void *sqlite3Fts3HashFind(const fts3Hash *pH, const void *pKey, int nKey){ +SQLITE_PRIVATE void *sqlite3Fts3HashFind(const Fts3Hash *pH, const void *pKey, int nKey){ int h; /* A hash on key */ - fts3HashElem *elem; /* The element that matches key */ + Fts3HashElem *elem; /* The element that matches key */ int (*xHash)(const void*,int); /* The hash function */ if( pH==0 || pH->ht==0 ) return 0; @@ -105924,15 +101904,15 @@ SQLITE_PRIVATE void *sqlite3Fts3HashFind(const fts3Hash *pH, const void *pKey, i ** element corresponding to "key" is removed from the hash table. */ SQLITE_PRIVATE void *sqlite3Fts3HashInsert( - fts3Hash *pH, /* The hash table to insert into */ + Fts3Hash *pH, /* The hash table to insert into */ const void *pKey, /* The key */ int nKey, /* Number of bytes in the key */ void *data /* The data */ ){ int hraw; /* Raw hash value of the key */ int h; /* the hash of the key modulo hash table size */ - fts3HashElem *elem; /* Used to loop thru the element list */ - fts3HashElem *new_elem; /* New element added to the pH */ + Fts3HashElem *elem; /* Used to loop thru the element list */ + Fts3HashElem *new_elem; /* New element added to the pH */ int (*xHash)(const void*,int); /* The hash function */ assert( pH!=0 ); @@ -105952,14 +101932,14 @@ SQLITE_PRIVATE void *sqlite3Fts3HashInsert( return old_data; } if( data==0 ) return 0; - if( pH->htsize==0 ){ - fts3Rehash(pH,8); - if( pH->htsize==0 ){ - pH->count = 0; - return data; - } + if( (pH->htsize==0 && fts3Rehash(pH,8)) + || (pH->count>=pH->htsize && fts3Rehash(pH, pH->htsize*2)) + ){ + pH->count = 0; + return data; } - new_elem = (fts3HashElem*)fts3HashMalloc( sizeof(fts3HashElem) ); + assert( pH->htsize>0 ); + new_elem = (Fts3HashElem*)fts3HashMalloc( sizeof(Fts3HashElem) ); if( new_elem==0 ) return data; if( pH->copyKey && pKey!=0 ){ new_elem->pKey = fts3HashMalloc( nKey ); @@ -105973,9 +101953,6 @@ SQLITE_PRIVATE void *sqlite3Fts3HashInsert( } new_elem->nKey = nKey; pH->count++; - if( pH->count > pH->htsize ){ - fts3Rehash(pH,pH->htsize*2); - } assert( pH->htsize>0 ); assert( (pH->htsize & (pH->htsize-1))==0 ); h = hraw & (pH->htsize-1); @@ -106038,10 +102015,6 @@ typedef struct porter_tokenizer_cursor { } porter_tokenizer_cursor; -/* Forward declaration */ -static const sqlite3_tokenizer_module porterTokenizerModule; - - /* ** Create a new tokenizer instance. */ @@ -106050,6 +102023,10 @@ static int porterCreate( sqlite3_tokenizer **ppTokenizer ){ porter_tokenizer *t; + + UNUSED_PARAMETER(argc); + UNUSED_PARAMETER(argv); + t = (porter_tokenizer *) sqlite3_malloc(sizeof(*t)); if( t==NULL ) return SQLITE_NOMEM; memset(t, 0, sizeof(*t)); @@ -106068,7 +102045,7 @@ static int porterDestroy(sqlite3_tokenizer *pTokenizer){ /* ** Prepare to begin tokenizing a particular string. The input ** string to be tokenized is zInput[0..nInput-1]. A cursor -** used to incrementally tokenize this string is returned in +** used to incrementally tokenize this string is returned in ** *ppCursor. */ static int porterOpen( @@ -106078,6 +102055,8 @@ static int porterOpen( ){ porter_tokenizer_cursor *c; + UNUSED_PARAMETER(pTokenizer); + c = (porter_tokenizer_cursor *) sqlite3_malloc(sizeof(*c)); if( c==NULL ) return SQLITE_NOMEM; @@ -106119,7 +102098,7 @@ static const char cType[] = { /* ** isConsonant() and isVowel() determine if their first character in ** the string they point to is a consonant or a vowel, according -** to Porter ruls. +** to Porter ruls. ** ** A consonate is any letter other than 'a', 'e', 'i', 'o', or 'u'. ** 'Y' is a consonant unless it follows another consonant, @@ -106218,7 +102197,7 @@ static int hasVowel(const char *z){ ** the first two characters of z[]. */ static int doubleConsonant(const char *z){ - return isConsonant(z) && z[0]==z[1] && isConsonant(z+1); + return isConsonant(z) && z[0]==z[1]; } /* @@ -106231,19 +102210,19 @@ static int doubleConsonant(const char *z){ */ static int star_oh(const char *z){ return - z[0]!=0 && isConsonant(z) && + isConsonant(z) && z[0]!='w' && z[0]!='x' && z[0]!='y' && - z[1]!=0 && isVowel(z+1) && - z[2]!=0 && isConsonant(z+2); + isVowel(z+1) && + isConsonant(z+2); } /* ** If the word ends with zFrom and xCond() is true for the stem -** of the word that preceeds the zFrom ending, then change the +** of the word that preceeds the zFrom ending, then change the ** ending to zTo. ** ** The input word *pz and zFrom are both in reverse order. zTo -** is in normal order. +** is in normal order. ** ** Return TRUE if zFrom matches. Return FALSE if zFrom does not ** match. Not that TRUE is returned even if xCond() fails and @@ -106278,7 +102257,7 @@ static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ int i, mx, j; int hasDigit = 0; for(i=0; i='A' && c<='Z' ){ zOut[i] = c - 'A' + 'a'; }else{ @@ -106312,9 +102291,9 @@ static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ ** word contains digits, 3 bytes are taken from the beginning and ** 3 bytes from the end. For long words without digits, 10 bytes ** are taken from each end. US-ASCII case folding still applies. -** -** If the input word contains not digits but does characters not -** in [a-zA-Z] then no stemming is attempted and this routine just +** +** If the input word contains not digits but does characters not +** in [a-zA-Z] then no stemming is attempted and this routine just ** copies the input into the input into the output with US-ASCII ** case folding. ** @@ -106322,7 +102301,7 @@ static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ ** no chance of overflowing the zOut buffer. */ static void porter_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ - int i, j, c; + int i, j; char zReverse[28]; char *z, *z2; if( nIn<3 || nIn>=sizeof(zReverse)-7 ){ @@ -106332,7 +102311,7 @@ static void porter_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ return; } for(i=0, j=sizeof(zReverse)-6; i='A' && c<='Z' ){ zReverse[j] = c + 'a' - 'A'; }else if( c>='a' && c<='z' ){ @@ -106359,11 +102338,11 @@ static void porter_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ } } - /* Step 1b */ + /* Step 1b */ z2 = z; if( stem(&z, "dee", "ee", m_gt_0) ){ /* Do nothing. The work was all in the test */ - }else if( + }else if( (stem(&z, "gni", "", hasVowel) || stem(&z, "de", "", hasVowel)) && z!=z2 ){ @@ -106531,7 +102510,7 @@ static void porter_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){ /* z[] is now the stemmed word in reverse order. Flip it back ** around into forward order and return. */ - *pnOut = i = strlen(z); + *pnOut = i = (int)strlen(z); zOut[i] = 0; while( *z ){ zOut[--i] = *(z++); @@ -106660,7 +102639,7 @@ SQLITE_PRIVATE void sqlite3Fts3PorterTokenizerModule( /* -** Implementation of the SQL scalar function for accessing the underlying +** Implementation of the SQL scalar function for accessing the underlying ** hash table. This function may be called as follows: ** ** SELECT (); @@ -106684,14 +102663,14 @@ static void scalarFunc( int argc, sqlite3_value **argv ){ - fts3Hash *pHash; + Fts3Hash *pHash; void *pPtr = 0; const unsigned char *zName; int nName; assert( argc==1 || argc==2 ); - pHash = (fts3Hash *)sqlite3_user_data(context); + pHash = (Fts3Hash *)sqlite3_user_data(context); zName = sqlite3_value_text(argv[0]); nName = sqlite3_value_bytes(argv[0])+1; @@ -106722,11 +102701,132 @@ static void scalarFunc( sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT); } +static int fts3IsIdChar(char c){ + static const char isFtsIdChar[] = { + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 0x */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 1x */ + 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 2x */ + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, /* 3x */ + 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 4x */ + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, /* 5x */ + 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 6x */ + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, /* 7x */ + }; + return (c&0x80 || isFtsIdChar[(int)(c)]); +} + +SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *zStr, int *pn){ + const char *z1; + const char *z2 = 0; + + /* Find the start of the next token. */ + z1 = zStr; + while( z2==0 ){ + switch( *z1 ){ + case '\0': return 0; /* No more tokens here */ + case '\'': + case '"': + case '`': { + z2 = &z1[1]; + while( *z2 && (z2[0]!=*z1 || z2[1]==*z1) ) z2++; + if( *z2 ) z2++; + break; + } + case '[': + z2 = &z1[1]; + while( *z2 && z2[0]!=']' ) z2++; + if( *z2 ) z2++; + break; + + default: + if( fts3IsIdChar(*z1) ){ + z2 = &z1[1]; + while( fts3IsIdChar(*z2) ) z2++; + }else{ + z1++; + } + } + } + + *pn = (int)(z2-z1); + return z1; +} + +SQLITE_PRIVATE int sqlite3Fts3InitTokenizer( + Fts3Hash *pHash, /* Tokenizer hash table */ + const char *zArg, /* Possible tokenizer specification */ + sqlite3_tokenizer **ppTok, /* OUT: Tokenizer (if applicable) */ + const char **pzTokenizer, /* OUT: Set to zArg if is tokenizer */ + char **pzErr /* OUT: Set to malloced error message */ +){ + int rc; + char *z = (char *)zArg; + int n; + char *zCopy; + char *zEnd; /* Pointer to nul-term of zCopy */ + sqlite3_tokenizer_module *m; + + if( !z ){ + zCopy = sqlite3_mprintf("simple"); + }else{ + if( sqlite3_strnicmp(z, "tokenize", 8) || fts3IsIdChar(z[8])){ + return SQLITE_OK; + } + zCopy = sqlite3_mprintf("%s", &z[8]); + *pzTokenizer = zArg; + } + if( !zCopy ){ + return SQLITE_NOMEM; + } + + zEnd = &zCopy[strlen(zCopy)]; + + z = (char *)sqlite3Fts3NextToken(zCopy, &n); + z[n] = '\0'; + sqlite3Fts3Dequote(z); + + m = (sqlite3_tokenizer_module *)sqlite3Fts3HashFind(pHash, z, (int)strlen(z)+1); + if( !m ){ + *pzErr = sqlite3_mprintf("unknown tokenizer: %s", z); + rc = SQLITE_ERROR; + }else{ + char const **aArg = 0; + int iArg = 0; + z = &z[n+1]; + while( zxCreate(iArg, aArg, ppTok); + assert( rc!=SQLITE_OK || *ppTok ); + if( rc!=SQLITE_OK ){ + *pzErr = sqlite3_mprintf("unknown tokenizer"); + }else{ + (*ppTok)->pModule = m; + } + sqlite3_free((void *)aArg); + } + + sqlite3_free(zCopy); + return rc; +} + + #ifdef SQLITE_TEST /* -** Implementation of a special SQL scalar function for testing tokenizers +** Implementation of a special SQL scalar function for testing tokenizers ** designed to be used in concert with the Tcl testing framework. This ** function must be called with two arguments: ** @@ -106739,9 +102839,9 @@ static void scalarFunc( ** ** The return value is a string that may be interpreted as a Tcl ** list. For each token in the , three elements are -** added to the returned list. The first is the token position, the +** added to the returned list. The first is the token position, the ** second is the token text (folded, stemmed, etc.) and the third is the -** substring of associated with the token. For example, +** substring of associated with the token. For example, ** using the built-in "simple" tokenizer: ** ** SELECT fts_tokenizer_test('simple', 'I don't see how'); @@ -106749,14 +102849,14 @@ static void scalarFunc( ** will return the string: ** ** "{0 i I 1 dont don't 2 see see 3 how how}" -** +** */ static void testFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ - fts3Hash *pHash; + Fts3Hash *pHash; sqlite3_tokenizer_module *p; sqlite3_tokenizer *pTokenizer = 0; sqlite3_tokenizer_cursor *pCsr = 0; @@ -106789,7 +102889,7 @@ static void testFunc( zArg = (const char *)sqlite3_value_text(argv[1]); } - pHash = (fts3Hash *)sqlite3_user_data(context); + pHash = (Fts3Hash *)sqlite3_user_data(context); p = (sqlite3_tokenizer_module *)sqlite3Fts3HashFind(pHash, zName, nName+1); if( !p ){ @@ -106841,8 +102941,8 @@ finish: static int registerTokenizer( - sqlite3 *db, - char *zName, + sqlite3 *db, + char *zName, const sqlite3_tokenizer_module *p ){ int rc; @@ -106863,8 +102963,8 @@ int registerTokenizer( static int queryTokenizer( - sqlite3 *db, - char *zName, + sqlite3 *db, + char *zName, const sqlite3_tokenizer_module **pp ){ int rc; @@ -106880,7 +102980,7 @@ int queryTokenizer( sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC); if( SQLITE_ROW==sqlite3_step(pStmt) ){ if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){ - memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp)); + memcpy((void *)pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp)); } } @@ -106917,6 +103017,9 @@ static void intTestFunc( const sqlite3_tokenizer_module *p2; sqlite3 *db = (sqlite3 *)sqlite3_user_data(context); + UNUSED_PARAMETER(argc); + UNUSED_PARAMETER(argv); + /* Test the query function */ sqlite3Fts3SimpleTokenizerModule(&p1); rc = queryTokenizer(db, "simple", &p2); @@ -106942,23 +103045,23 @@ static void intTestFunc( /* ** Set up SQL objects in database db used to access the contents of ** the hash table pointed to by argument pHash. The hash table must -** been initialised to use string keys, and to take a private copy +** been initialised to use string keys, and to take a private copy ** of the key when a value is inserted. i.e. by a call similar to: ** ** sqlite3Fts3HashInit(pHash, FTS3_HASH_STRING, 1); ** ** This function adds a scalar function (see header comment above ** scalarFunc() in this file for details) and, if ENABLE_TABLE is -** defined at compilation time, a temporary virtual table (see header -** comment above struct HashTableVtab) to the database schema. Both +** defined at compilation time, a temporary virtual table (see header +** comment above struct HashTableVtab) to the database schema. Both ** provide read/write access to the contents of *pHash. ** ** The third argument to this function, zName, is used as the name ** of both the scalar and, if created, the virtual table. */ SQLITE_PRIVATE int sqlite3Fts3InitHashTable( - sqlite3 *db, - fts3Hash *pHash, + sqlite3 *db, + Fts3Hash *pHash, const char *zName ){ int rc = SQLITE_OK; @@ -106976,13 +103079,13 @@ SQLITE_PRIVATE int sqlite3Fts3InitHashTable( } #endif - if( rc!=SQLITE_OK - || (rc = sqlite3_create_function(db, zName, 1, any, p, scalarFunc, 0, 0)) - || (rc = sqlite3_create_function(db, zName, 2, any, p, scalarFunc, 0, 0)) + if( SQLITE_OK!=rc + || SQLITE_OK!=(rc = sqlite3_create_function(db, zName, 1, any, p, scalarFunc, 0, 0)) + || SQLITE_OK!=(rc = sqlite3_create_function(db, zName, 2, any, p, scalarFunc, 0, 0)) #ifdef SQLITE_TEST - || (rc = sqlite3_create_function(db, zTest, 2, any, p, testFunc, 0, 0)) - || (rc = sqlite3_create_function(db, zTest, 3, any, p, testFunc, 0, 0)) - || (rc = sqlite3_create_function(db, zTest2, 0, any, pdb, intTestFunc, 0, 0)) + || SQLITE_OK!=(rc = sqlite3_create_function(db, zTest, 2, any, p, testFunc, 0, 0)) + || SQLITE_OK!=(rc = sqlite3_create_function(db, zTest, 3, any, p, testFunc, 0, 0)) + || SQLITE_OK!=(rc = sqlite3_create_function(db, zTest2, 0, any, pdb, intTestFunc, 0, 0)) #endif ); @@ -107040,9 +103143,6 @@ typedef struct simple_tokenizer_cursor { } simple_tokenizer_cursor; -/* Forward declaration */ -static const sqlite3_tokenizer_module simpleTokenizerModule; - static int simpleDelim(simple_tokenizer *t, unsigned char c){ return c<0x80 && t->delim[c]; } @@ -107066,7 +103166,7 @@ static int simpleCreate( ** information on the initial create. */ if( argc>1 ){ - int i, n = strlen(argv[1]); + int i, n = (int)strlen(argv[1]); for(i=0; idelim[i] = !isalnum(i); + t->delim[i] = !isalnum(i) ? -1 : 0; } } @@ -107099,7 +103199,7 @@ static int simpleDestroy(sqlite3_tokenizer *pTokenizer){ /* ** Prepare to begin tokenizing a particular string. The input ** string to be tokenized is pInput[0..nBytes-1]. A cursor -** used to incrementally tokenize this string is returned in +** used to incrementally tokenize this string is returned in ** *ppCursor. */ static int simpleOpen( @@ -107109,6 +103209,8 @@ static int simpleOpen( ){ simple_tokenizer_cursor *c; + UNUSED_PARAMETER(pTokenizer); + c = (simple_tokenizer_cursor *) sqlite3_malloc(sizeof(*c)); if( c==NULL ) return SQLITE_NOMEM; @@ -107182,7 +103284,7 @@ static int simpleNext( ** case-insensitivity. */ unsigned char ch = p[iStartOffset+i]; - c->pToken[i] = ch<0x80 ? tolower(ch) : ch; + c->pToken[i] = (char)(ch<0x80 ? tolower(ch) : ch); } *ppToken = c->pToken; *pnBytes = n; @@ -107221,6 +103323,2909 @@ SQLITE_PRIVATE void sqlite3Fts3SimpleTokenizerModule( #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) */ /************** End of fts3_tokenizer1.c *************************************/ +/************** Begin file fts3_write.c **************************************/ +/* +** 2009 Oct 23 +** +** The author disclaims copyright to this source code. In place of +** a legal notice, here is a blessing: +** +** May you do good and not evil. +** May you find forgiveness for yourself and forgive others. +** May you share freely, never taking more than you give. +** +****************************************************************************** +** +** This file is part of the SQLite FTS3 extension module. Specifically, +** this file contains code to insert, update and delete rows from FTS3 +** tables. It also contains code to merge FTS3 b-tree segments. Some +** of the sub-routines used to merge segments are also used by the query +** code in fts3.c. +*/ + +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) + + +typedef struct PendingList PendingList; +typedef struct SegmentNode SegmentNode; +typedef struct SegmentWriter SegmentWriter; + +/* +** Data structure used while accumulating terms in the pending-terms hash +** table. The hash table entry maps from term (a string) to a malloced +** instance of this structure. +*/ +struct PendingList { + int nData; + char *aData; + int nSpace; + sqlite3_int64 iLastDocid; + sqlite3_int64 iLastCol; + sqlite3_int64 iLastPos; +}; + +/* +** An instance of this structure is used to iterate through the terms on +** a contiguous set of segment b-tree leaf nodes. Although the details of +** this structure are only manipulated by code in this file, opaque handles +** of type Fts3SegReader* are also used by code in fts3.c to iterate through +** terms when querying the full-text index. See functions: +** +** sqlite3Fts3SegReaderNew() +** sqlite3Fts3SegReaderFree() +** sqlite3Fts3SegReaderIterate() +*/ +struct Fts3SegReader { + int iIdx; /* Index within level */ + sqlite3_int64 iStartBlock; + sqlite3_int64 iEndBlock; + sqlite3_stmt *pStmt; /* SQL Statement to access leaf nodes */ + char *aNode; /* Pointer to node data (or NULL) */ + int nNode; /* Size of buffer at aNode (or 0) */ + int nTermAlloc; /* Allocated size of zTerm buffer */ + + /* Variables set by fts3SegReaderNext(). These may be read directly + ** by the caller. They are valid from the time SegmentReaderNew() returns + ** until SegmentReaderNext() returns something other than SQLITE_OK + ** (i.e. SQLITE_DONE). + */ + int nTerm; /* Number of bytes in current term */ + char *zTerm; /* Pointer to current term */ + char *aDoclist; /* Pointer to doclist of current entry */ + int nDoclist; /* Size of doclist in current entry */ + + /* The following variables are used to iterate through the current doclist */ + char *pOffsetList; + sqlite3_int64 iDocid; +}; + +/* +** An instance of this structure is used to create a segment b-tree in the +** database. The internal details of this type are only accessed by the +** following functions: +** +** fts3SegWriterAdd() +** fts3SegWriterFlush() +** fts3SegWriterFree() +*/ +struct SegmentWriter { + SegmentNode *pTree; /* Pointer to interior tree structure */ + sqlite3_int64 iFirst; /* First slot in %_segments written */ + sqlite3_int64 iFree; /* Next free slot in %_segments */ + char *zTerm; /* Pointer to previous term buffer */ + int nTerm; /* Number of bytes in zTerm */ + int nMalloc; /* Size of malloc'd buffer at zMalloc */ + char *zMalloc; /* Malloc'd space (possibly) used for zTerm */ + int nSize; /* Size of allocation at aData */ + int nData; /* Bytes of data in aData */ + char *aData; /* Pointer to block from malloc() */ +}; + +/* +** Type SegmentNode is used by the following three functions to create +** the interior part of the segment b+-tree structures (everything except +** the leaf nodes). These functions and type are only ever used by code +** within the fts3SegWriterXXX() family of functions described above. +** +** fts3NodeAddTerm() +** fts3NodeWrite() +** fts3NodeFree() +*/ +struct SegmentNode { + SegmentNode *pParent; /* Parent node (or NULL for root node) */ + SegmentNode *pRight; /* Pointer to right-sibling */ + SegmentNode *pLeftmost; /* Pointer to left-most node of this depth */ + int nEntry; /* Number of terms written to node so far */ + char *zTerm; /* Pointer to previous term buffer */ + int nTerm; /* Number of bytes in zTerm */ + int nMalloc; /* Size of malloc'd buffer at zMalloc */ + char *zMalloc; /* Malloc'd space (possibly) used for zTerm */ + int nData; /* Bytes of valid data so far */ + char *aData; /* Node data */ +}; + +/* +** Valid values for the second argument to fts3SqlStmt(). +*/ +#define SQL_DELETE_CONTENT 0 +#define SQL_IS_EMPTY 1 +#define SQL_DELETE_ALL_CONTENT 2 +#define SQL_DELETE_ALL_SEGMENTS 3 +#define SQL_DELETE_ALL_SEGDIR 4 +#define SQL_SELECT_CONTENT_BY_ROWID 5 +#define SQL_NEXT_SEGMENT_INDEX 6 +#define SQL_INSERT_SEGMENTS 7 +#define SQL_NEXT_SEGMENTS_ID 8 +#define SQL_INSERT_SEGDIR 9 +#define SQL_SELECT_LEVEL 10 +#define SQL_SELECT_ALL_LEVEL 11 +#define SQL_SELECT_LEVEL_COUNT 12 +#define SQL_SELECT_SEGDIR_COUNT_MAX 13 +#define SQL_DELETE_SEGDIR_BY_LEVEL 14 +#define SQL_DELETE_SEGMENTS_RANGE 15 +#define SQL_CONTENT_INSERT 16 +#define SQL_GET_BLOCK 17 + +/* +** This function is used to obtain an SQLite prepared statement handle +** for the statement identified by the second argument. If successful, +** *pp is set to the requested statement handle and SQLITE_OK returned. +** Otherwise, an SQLite error code is returned and *pp is set to 0. +** +** If argument apVal is not NULL, then it must point to an array with +** at least as many entries as the requested statement has bound +** parameters. The values are bound to the statements parameters before +** returning. +*/ +static int fts3SqlStmt( + Fts3Table *p, /* Virtual table handle */ + int eStmt, /* One of the SQL_XXX constants above */ + sqlite3_stmt **pp, /* OUT: Statement handle */ + sqlite3_value **apVal /* Values to bind to statement */ +){ + const char *azSql[] = { +/* 0 */ "DELETE FROM %Q.'%q_content' WHERE rowid = ?", +/* 1 */ "SELECT NOT EXISTS(SELECT docid FROM %Q.'%q_content' WHERE rowid!=?)", +/* 2 */ "DELETE FROM %Q.'%q_content'", +/* 3 */ "DELETE FROM %Q.'%q_segments'", +/* 4 */ "DELETE FROM %Q.'%q_segdir'", +/* 5 */ "SELECT * FROM %Q.'%q_content' WHERE rowid=?", +/* 6 */ "SELECT coalesce(max(idx)+1, 0) FROM %Q.'%q_segdir' WHERE level=?", +/* 7 */ "INSERT INTO %Q.'%q_segments'(blockid, block) VALUES(?, ?)", +/* 8 */ "SELECT coalesce(max(blockid)+1, 1) FROM %Q.'%q_segments'", +/* 9 */ "INSERT INTO %Q.'%q_segdir' VALUES(?,?,?,?,?,?)", + + /* Return segments in order from oldest to newest.*/ +/* 10 */ "SELECT idx, start_block, leaves_end_block, end_block, root " + "FROM %Q.'%q_segdir' WHERE level = ? ORDER BY idx ASC", +/* 11 */ "SELECT idx, start_block, leaves_end_block, end_block, root " + "FROM %Q.'%q_segdir' ORDER BY level DESC, idx ASC", + +/* 12 */ "SELECT count(*) FROM %Q.'%q_segdir' WHERE level = ?", +/* 13 */ "SELECT count(*), max(level) FROM %Q.'%q_segdir'", + +/* 14 */ "DELETE FROM %Q.'%q_segdir' WHERE level = ?", +/* 15 */ "DELETE FROM %Q.'%q_segments' WHERE blockid BETWEEN ? AND ?", +/* 16 */ "INSERT INTO %Q.'%q_content' VALUES(%z)", +/* 17 */ "SELECT block FROM %Q.'%q_segments' WHERE blockid = ?", + }; + int rc = SQLITE_OK; + sqlite3_stmt *pStmt; + + assert( SizeofArray(azSql)==SizeofArray(p->aStmt) ); + assert( eStmt=0 ); + + pStmt = p->aStmt[eStmt]; + if( !pStmt ){ + char *zSql; + if( eStmt==SQL_CONTENT_INSERT ){ + int i; /* Iterator variable */ + char *zVarlist; /* The "?, ?, ..." string */ + zVarlist = (char *)sqlite3_malloc(2*p->nColumn+2); + if( !zVarlist ){ + *pp = 0; + return SQLITE_NOMEM; + } + zVarlist[0] = '?'; + zVarlist[p->nColumn*2+1] = '\0'; + for(i=1; i<=p->nColumn; i++){ + zVarlist[i*2-1] = ','; + zVarlist[i*2] = '?'; + } + zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName, zVarlist); + }else{ + zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName); + } + if( !zSql ){ + rc = SQLITE_NOMEM; + }else{ + rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, NULL); + sqlite3_free(zSql); + assert( rc==SQLITE_OK || pStmt==0 ); + p->aStmt[eStmt] = pStmt; + } + } + if( apVal ){ + int i; + int nParam = sqlite3_bind_parameter_count(pStmt); + for(i=0; rc==SQLITE_OK && inSpace = 100; + p->aData = (char *)&p[1]; + p->nData = 0; + } + else if( p->nData+FTS3_VARINT_MAX+1>p->nSpace ){ + int nNew = p->nSpace * 2; + p = sqlite3_realloc(p, sizeof(*p) + nNew); + if( !p ){ + sqlite3_free(*pp); + *pp = 0; + return SQLITE_NOMEM; + } + p->nSpace = nNew; + p->aData = (char *)&p[1]; + } + + /* Append the new serialized varint to the end of the list. */ + p->nData += sqlite3Fts3PutVarint(&p->aData[p->nData], i); + p->aData[p->nData] = '\0'; + *pp = p; + return SQLITE_OK; +} + +/* +** Add a docid/column/position entry to a PendingList structure. Non-zero +** is returned if the structure is sqlite3_realloced as part of adding +** the entry. Otherwise, zero. +** +** If an OOM error occurs, *pRc is set to SQLITE_NOMEM before returning. +** Zero is always returned in this case. Otherwise, if no OOM error occurs, +** it is set to SQLITE_OK. +*/ +static int fts3PendingListAppend( + PendingList **pp, /* IN/OUT: PendingList structure */ + sqlite3_int64 iDocid, /* Docid for entry to add */ + sqlite3_int64 iCol, /* Column for entry to add */ + sqlite3_int64 iPos, /* Position of term for entry to add */ + int *pRc /* OUT: Return code */ +){ + PendingList *p = *pp; + int rc = SQLITE_OK; + + assert( !p || p->iLastDocid<=iDocid ); + + if( !p || p->iLastDocid!=iDocid ){ + sqlite3_int64 iDelta = iDocid - (p ? p->iLastDocid : 0); + if( p ){ + assert( p->nDatanSpace ); + assert( p->aData[p->nData]==0 ); + p->nData++; + } + if( SQLITE_OK!=(rc = fts3PendingListAppendVarint(&p, iDelta)) ){ + goto pendinglistappend_out; + } + p->iLastCol = -1; + p->iLastPos = 0; + p->iLastDocid = iDocid; + } + if( iCol>0 && p->iLastCol!=iCol ){ + if( SQLITE_OK!=(rc = fts3PendingListAppendVarint(&p, 1)) + || SQLITE_OK!=(rc = fts3PendingListAppendVarint(&p, iCol)) + ){ + goto pendinglistappend_out; + } + p->iLastCol = iCol; + p->iLastPos = 0; + } + if( iCol>=0 ){ + assert( iPos>p->iLastPos || (iPos==0 && p->iLastPos==0) ); + rc = fts3PendingListAppendVarint(&p, 2+iPos-p->iLastPos); + if( rc==SQLITE_OK ){ + p->iLastPos = iPos; + } + } + + pendinglistappend_out: + *pRc = rc; + if( p!=*pp ){ + *pp = p; + return 1; + } + return 0; +} + +/* +** Tokenize the nul-terminated string zText and add all tokens to the +** pending-terms hash-table. The docid used is that currently stored in +** p->iPrevDocid, and the column is specified by argument iCol. +** +** If successful, SQLITE_OK is returned. Otherwise, an SQLite error code. +*/ +static int fts3PendingTermsAdd(Fts3Table *p, const char *zText, int iCol){ + int rc; + int iStart; + int iEnd; + int iPos; + + char const *zToken; + int nToken; + + sqlite3_tokenizer *pTokenizer = p->pTokenizer; + sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; + sqlite3_tokenizer_cursor *pCsr; + int (*xNext)(sqlite3_tokenizer_cursor *pCursor, + const char**,int*,int*,int*,int*); + + assert( pTokenizer && pModule ); + + rc = pModule->xOpen(pTokenizer, zText, -1, &pCsr); + if( rc!=SQLITE_OK ){ + return rc; + } + pCsr->pTokenizer = pTokenizer; + + xNext = pModule->xNext; + while( SQLITE_OK==rc + && SQLITE_OK==(rc = xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos)) + ){ + PendingList *pList; + + /* Positions cannot be negative; we use -1 as a terminator internally. + ** Tokens must have a non-zero length. + */ + if( iPos<0 || !zToken || nToken<=0 ){ + rc = SQLITE_ERROR; + break; + } + + pList = (PendingList *)fts3HashFind(&p->pendingTerms, zToken, nToken); + if( pList ){ + p->nPendingData -= (pList->nData + nToken + sizeof(Fts3HashElem)); + } + if( fts3PendingListAppend(&pList, p->iPrevDocid, iCol, iPos, &rc) ){ + if( pList==fts3HashInsert(&p->pendingTerms, zToken, nToken, pList) ){ + /* Malloc failed while inserting the new entry. This can only + ** happen if there was no previous entry for this token. + */ + assert( 0==fts3HashFind(&p->pendingTerms, zToken, nToken) ); + sqlite3_free(pList); + rc = SQLITE_NOMEM; + } + } + if( rc==SQLITE_OK ){ + p->nPendingData += (pList->nData + nToken + sizeof(Fts3HashElem)); + } + } + + pModule->xClose(pCsr); + return (rc==SQLITE_DONE ? SQLITE_OK : rc); +} + +/* +** Calling this function indicates that subsequent calls to +** fts3PendingTermsAdd() are to add term/position-list pairs for the +** contents of the document with docid iDocid. +*/ +static int fts3PendingTermsDocid(Fts3Table *p, sqlite_int64 iDocid){ + /* TODO(shess) Explore whether partially flushing the buffer on + ** forced-flush would provide better performance. I suspect that if + ** we ordered the doclists by size and flushed the largest until the + ** buffer was half empty, that would let the less frequent terms + ** generate longer doclists. + */ + if( iDocid<=p->iPrevDocid || p->nPendingData>FTS3_MAX_PENDING_DATA ){ + int rc = sqlite3Fts3PendingTermsFlush(p); + if( rc!=SQLITE_OK ) return rc; + } + p->iPrevDocid = iDocid; + return SQLITE_OK; +} + +SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *p){ + Fts3HashElem *pElem; + for(pElem=fts3HashFirst(&p->pendingTerms); pElem; pElem=fts3HashNext(pElem)){ + sqlite3_free(fts3HashData(pElem)); + } + fts3HashClear(&p->pendingTerms); + p->nPendingData = 0; +} + +/* +** This function is called by the xUpdate() method as part of an INSERT +** operation. It adds entries for each term in the new record to the +** pendingTerms hash table. +** +** Argument apVal is the same as the similarly named argument passed to +** fts3InsertData(). Parameter iDocid is the docid of the new row. +*/ +static int fts3InsertTerms(Fts3Table *p, sqlite3_value **apVal){ + int i; /* Iterator variable */ + for(i=2; inColumn+2; i++){ + const char *zText = (const char *)sqlite3_value_text(apVal[i]); + if( zText ){ + int rc = fts3PendingTermsAdd(p, zText, i-2); + if( rc!=SQLITE_OK ){ + return rc; + } + } + } + return SQLITE_OK; +} + +/* +** This function is called by the xUpdate() method for an INSERT operation. +** The apVal parameter is passed a copy of the apVal argument passed by +** SQLite to the xUpdate() method. i.e: +** +** apVal[0] Not used for INSERT. +** apVal[1] rowid +** apVal[2] Left-most user-defined column +** ... +** apVal[p->nColumn+1] Right-most user-defined column +** apVal[p->nColumn+2] Hidden column with same name as table +** apVal[p->nColumn+3] Hidden "docid" column (alias for rowid) +*/ +static int fts3InsertData( + Fts3Table *p, /* Full-text table */ + sqlite3_value **apVal, /* Array of values to insert */ + sqlite3_int64 *piDocid /* OUT: Docid for row just inserted */ +){ + int rc; /* Return code */ + sqlite3_stmt *pContentInsert; /* INSERT INTO %_content VALUES(...) */ + + /* Locate the statement handle used to insert data into the %_content + ** table. The SQL for this statement is: + ** + ** INSERT INTO %_content VALUES(?, ?, ?, ...) + ** + ** The statement features N '?' variables, where N is the number of user + ** defined columns in the FTS3 table, plus one for the docid field. + */ + rc = fts3SqlStmt(p, SQL_CONTENT_INSERT, &pContentInsert, &apVal[1]); + if( rc!=SQLITE_OK ){ + return rc; + } + + /* There is a quirk here. The users INSERT statement may have specified + ** a value for the "rowid" field, for the "docid" field, or for both. + ** Which is a problem, since "rowid" and "docid" are aliases for the + ** same value. For example: + ** + ** INSERT INTO fts3tbl(rowid, docid) VALUES(1, 2); + ** + ** In FTS3, this is an error. It is an error to specify non-NULL values + ** for both docid and some other rowid alias. + */ + if( SQLITE_NULL!=sqlite3_value_type(apVal[3+p->nColumn]) ){ + if( SQLITE_NULL==sqlite3_value_type(apVal[0]) + && SQLITE_NULL!=sqlite3_value_type(apVal[1]) + ){ + /* A rowid/docid conflict. */ + return SQLITE_ERROR; + } + rc = sqlite3_bind_value(pContentInsert, 1, apVal[3+p->nColumn]); + if( rc!=SQLITE_OK ) return rc; + } + + /* Execute the statement to insert the record. Set *piDocid to the + ** new docid value. + */ + sqlite3_step(pContentInsert); + rc = sqlite3_reset(pContentInsert); + + *piDocid = sqlite3_last_insert_rowid(p->db); + return rc; +} + + + +/* +** Remove all data from the FTS3 table. Clear the hash table containing +** pending terms. +*/ +static int fts3DeleteAll(Fts3Table *p){ + int rc; /* Return code */ + + /* Discard the contents of the pending-terms hash table. */ + sqlite3Fts3PendingTermsClear(p); + + /* Delete everything from the %_content, %_segments and %_segdir tables. */ + rc = fts3SqlExec(p, SQL_DELETE_ALL_CONTENT, 0); + if( rc==SQLITE_OK ){ + rc = fts3SqlExec(p, SQL_DELETE_ALL_SEGMENTS, 0); + } + if( rc==SQLITE_OK ){ + rc = fts3SqlExec(p, SQL_DELETE_ALL_SEGDIR, 0); + } + return rc; +} + +/* +** The first element in the apVal[] array is assumed to contain the docid +** (an integer) of a row about to be deleted. Remove all terms from the +** full-text index. +*/ +static int fts3DeleteTerms(Fts3Table *p, sqlite3_value **apVal){ + int rc; + sqlite3_stmt *pSelect; + + rc = fts3SqlStmt(p, SQL_SELECT_CONTENT_BY_ROWID, &pSelect, apVal); + if( rc==SQLITE_OK ){ + if( SQLITE_ROW==sqlite3_step(pSelect) ){ + int i; + for(i=1; i<=p->nColumn; i++){ + const char *zText = (const char *)sqlite3_column_text(pSelect, i); + rc = fts3PendingTermsAdd(p, zText, -1); + if( rc!=SQLITE_OK ){ + sqlite3_reset(pSelect); + return rc; + } + } + } + rc = sqlite3_reset(pSelect); + }else{ + sqlite3_reset(pSelect); + } + return rc; +} + +/* +** Forward declaration to account for the circular dependency between +** functions fts3SegmentMerge() and fts3AllocateSegdirIdx(). +*/ +static int fts3SegmentMerge(Fts3Table *, int); + +/* +** This function allocates a new level iLevel index in the segdir table. +** Usually, indexes are allocated within a level sequentially starting +** with 0, so the allocated index is one greater than the value returned +** by: +** +** SELECT max(idx) FROM %_segdir WHERE level = :iLevel +** +** However, if there are already FTS3_MERGE_COUNT indexes at the requested +** level, they are merged into a single level (iLevel+1) segment and the +** allocated index is 0. +** +** If successful, *piIdx is set to the allocated index slot and SQLITE_OK +** returned. Otherwise, an SQLite error code is returned. +*/ +static int fts3AllocateSegdirIdx(Fts3Table *p, int iLevel, int *piIdx){ + int rc; /* Return Code */ + sqlite3_stmt *pNextIdx; /* Query for next idx at level iLevel */ + int iNext = 0; /* Result of query pNextIdx */ + + /* Set variable iNext to the next available segdir index at level iLevel. */ + rc = fts3SqlStmt(p, SQL_NEXT_SEGMENT_INDEX, &pNextIdx, 0); + if( rc==SQLITE_OK ){ + sqlite3_bind_int(pNextIdx, 1, iLevel); + if( SQLITE_ROW==sqlite3_step(pNextIdx) ){ + iNext = sqlite3_column_int(pNextIdx, 0); + } + rc = sqlite3_reset(pNextIdx); + } + + if( rc==SQLITE_OK ){ + /* If iNext is FTS3_MERGE_COUNT, indicating that level iLevel is already + ** full, merge all segments in level iLevel into a single iLevel+1 + ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise, + ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext. + */ + if( iNext>=FTS3_MERGE_COUNT ){ + rc = fts3SegmentMerge(p, iLevel); + *piIdx = 0; + }else{ + *piIdx = iNext; + } + } + + return rc; +} + +/* +** Move the iterator passed as the first argument to the next term in the +** segment. If successful, SQLITE_OK is returned. If there is no next term, +** SQLITE_DONE. Otherwise, an SQLite error code. +*/ +static int fts3SegReaderNext(Fts3SegReader *pReader){ + char *pNext; /* Cursor variable */ + int nPrefix; /* Number of bytes in term prefix */ + int nSuffix; /* Number of bytes in term suffix */ + + if( !pReader->aDoclist ){ + pNext = pReader->aNode; + }else{ + pNext = &pReader->aDoclist[pReader->nDoclist]; + } + + if( !pNext || pNext>=&pReader->aNode[pReader->nNode] ){ + int rc; + if( !pReader->pStmt ){ + pReader->aNode = 0; + return SQLITE_OK; + } + rc = sqlite3_step(pReader->pStmt); + if( rc!=SQLITE_ROW ){ + pReader->aNode = 0; + return (rc==SQLITE_DONE ? SQLITE_OK : rc); + } + pReader->nNode = sqlite3_column_bytes(pReader->pStmt, 0); + pReader->aNode = (char *)sqlite3_column_blob(pReader->pStmt, 0); + pNext = pReader->aNode; + } + + pNext += sqlite3Fts3GetVarint32(pNext, &nPrefix); + pNext += sqlite3Fts3GetVarint32(pNext, &nSuffix); + + if( nPrefix+nSuffix>pReader->nTermAlloc ){ + int nNew = (nPrefix+nSuffix)*2; + char *zNew = sqlite3_realloc(pReader->zTerm, nNew); + if( !zNew ){ + return SQLITE_NOMEM; + } + pReader->zTerm = zNew; + pReader->nTermAlloc = nNew; + } + memcpy(&pReader->zTerm[nPrefix], pNext, nSuffix); + pReader->nTerm = nPrefix+nSuffix; + pNext += nSuffix; + pNext += sqlite3Fts3GetVarint32(pNext, &pReader->nDoclist); + assert( pNext<&pReader->aNode[pReader->nNode] ); + pReader->aDoclist = pNext; + pReader->pOffsetList = 0; + return SQLITE_OK; +} + +/* +** Set the SegReader to point to the first docid in the doclist associated +** with the current term. +*/ +static void fts3SegReaderFirstDocid(Fts3SegReader *pReader){ + int n; + assert( pReader->aDoclist ); + assert( !pReader->pOffsetList ); + n = sqlite3Fts3GetVarint(pReader->aDoclist, &pReader->iDocid); + pReader->pOffsetList = &pReader->aDoclist[n]; +} + +/* +** Advance the SegReader to point to the next docid in the doclist +** associated with the current term. +** +** If arguments ppOffsetList and pnOffsetList are not NULL, then +** *ppOffsetList is set to point to the first column-offset list +** in the doclist entry (i.e. immediately past the docid varint). +** *pnOffsetList is set to the length of the set of column-offset +** lists, not including the nul-terminator byte. For example: +*/ +static void fts3SegReaderNextDocid( + Fts3SegReader *pReader, + char **ppOffsetList, + int *pnOffsetList +){ + char *p = pReader->pOffsetList; + char c = 0; + + /* Pointer p currently points at the first byte of an offset list. The + ** following two lines advance it to point one byte past the end of + ** the same offset list. + */ + while( *p | c ) c = *p++ & 0x80; + p++; + + /* If required, populate the output variables with a pointer to and the + ** size of the previous offset-list. + */ + if( ppOffsetList ){ + *ppOffsetList = pReader->pOffsetList; + *pnOffsetList = (int)(p - pReader->pOffsetList - 1); + } + + /* If there are no more entries in the doclist, set pOffsetList to + ** NULL. Otherwise, set Fts3SegReader.iDocid to the next docid and + ** Fts3SegReader.pOffsetList to point to the next offset list before + ** returning. + */ + if( p>=&pReader->aDoclist[pReader->nDoclist] ){ + pReader->pOffsetList = 0; + }else{ + sqlite3_int64 iDelta; + pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta); + pReader->iDocid += iDelta; + } +} + +/* +** Free all allocations associated with the iterator passed as the +** second argument. +*/ +SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3Table *p, Fts3SegReader *pReader){ + if( pReader ){ + if( pReader->pStmt ){ + /* Move the leaf-range SELECT statement to the aLeavesStmt[] array, + ** so that it can be reused when required by another query. + */ + assert( p->nLeavesStmtnLeavesTotal ); + sqlite3_reset(pReader->pStmt); + p->aLeavesStmt[p->nLeavesStmt++] = pReader->pStmt; + } + sqlite3_free(pReader->zTerm); + sqlite3_free(pReader); + } +} + +/* +** Allocate a new SegReader object. +*/ +SQLITE_PRIVATE int sqlite3Fts3SegReaderNew( + Fts3Table *p, /* Virtual table handle */ + int iAge, /* Segment "age". */ + sqlite3_int64 iStartLeaf, /* First leaf to traverse */ + sqlite3_int64 iEndLeaf, /* Final leaf to traverse */ + sqlite3_int64 iEndBlock, /* Final block of segment */ + const char *zRoot, /* Buffer containing root node */ + int nRoot, /* Size of buffer containing root node */ + Fts3SegReader **ppReader /* OUT: Allocated Fts3SegReader */ +){ + int rc = SQLITE_OK; /* Return code */ + Fts3SegReader *pReader; /* Newly allocated SegReader object */ + int nExtra = 0; /* Bytes to allocate segment root node */ + + if( iStartLeaf==0 ){ + nExtra = nRoot; + } + + pReader = (Fts3SegReader *)sqlite3_malloc(sizeof(Fts3SegReader) + nExtra); + if( !pReader ){ + return SQLITE_NOMEM; + } + memset(pReader, 0, sizeof(Fts3SegReader)); + pReader->iStartBlock = iStartLeaf; + pReader->iIdx = iAge; + pReader->iEndBlock = iEndBlock; + + if( nExtra ){ + /* The entire segment is stored in the root node. */ + pReader->aNode = (char *)&pReader[1]; + pReader->nNode = nRoot; + memcpy(pReader->aNode, zRoot, nRoot); + }else{ + /* If the text of the SQL statement to iterate through a contiguous + ** set of entries in the %_segments table has not yet been composed, + ** compose it now. + */ + if( !p->zSelectLeaves ){ + p->zSelectLeaves = sqlite3_mprintf( + "SELECT block FROM %Q.'%q_segments' WHERE blockid BETWEEN ? AND ? " + "ORDER BY blockid", p->zDb, p->zName + ); + if( !p->zSelectLeaves ){ + rc = SQLITE_NOMEM; + goto finished; + } + } + + /* If there are no free statements in the aLeavesStmt[] array, prepare + ** a new statement now. Otherwise, reuse a prepared statement from + ** aLeavesStmt[]. + */ + if( p->nLeavesStmt==0 ){ + if( p->nLeavesTotal==p->nLeavesAlloc ){ + int nNew = p->nLeavesAlloc + 16; + sqlite3_stmt **aNew = (sqlite3_stmt **)sqlite3_realloc( + p->aLeavesStmt, nNew*sizeof(sqlite3_stmt *) + ); + if( !aNew ){ + rc = SQLITE_NOMEM; + goto finished; + } + p->nLeavesAlloc = nNew; + p->aLeavesStmt = aNew; + } + rc = sqlite3_prepare_v2(p->db, p->zSelectLeaves, -1, &pReader->pStmt, 0); + if( rc!=SQLITE_OK ){ + goto finished; + } + p->nLeavesTotal++; + }else{ + pReader->pStmt = p->aLeavesStmt[--p->nLeavesStmt]; + } + + /* Bind the start and end leaf blockids to the prepared SQL statement. */ + sqlite3_bind_int64(pReader->pStmt, 1, iStartLeaf); + sqlite3_bind_int64(pReader->pStmt, 2, iEndLeaf); + } + rc = fts3SegReaderNext(pReader); + + finished: + if( rc==SQLITE_OK ){ + *ppReader = pReader; + }else{ + sqlite3Fts3SegReaderFree(p, pReader); + } + return rc; +} + + +/* +** The second argument to this function is expected to be a statement of +** the form: +** +** SELECT +** idx, -- col 0 +** start_block, -- col 1 +** leaves_end_block, -- col 2 +** end_block, -- col 3 +** root -- col 4 +** FROM %_segdir ... +** +** This function allocates and initializes a Fts3SegReader structure to +** iterate through the terms stored in the segment identified by the +** current row that pStmt is pointing to. +** +** If successful, the Fts3SegReader is left pointing to the first term +** in the segment and SQLITE_OK is returned. Otherwise, an SQLite error +** code is returned. +*/ +static int fts3SegReaderNew( + Fts3Table *p, /* Virtual table handle */ + sqlite3_stmt *pStmt, /* See above */ + int iAge, /* Segment "age". */ + Fts3SegReader **ppReader /* OUT: Allocated Fts3SegReader */ +){ + return sqlite3Fts3SegReaderNew(p, iAge, + sqlite3_column_int64(pStmt, 1), + sqlite3_column_int64(pStmt, 2), + sqlite3_column_int64(pStmt, 3), + sqlite3_column_blob(pStmt, 4), + sqlite3_column_bytes(pStmt, 4), + ppReader + ); +} + +/* +** Compare the entries pointed to by two Fts3SegReader structures. +** Comparison is as follows: +** +** 1) EOF is greater than not EOF. +** +** 2) The current terms (if any) are compared with memcmp(). If one +** term is a prefix of another, the longer term is considered the +** larger. +** +** 3) By segment age. An older segment is considered larger. +*/ +static int fts3SegReaderCmp(Fts3SegReader *pLhs, Fts3SegReader *pRhs){ + int rc; + if( pLhs->aNode && pRhs->aNode ){ + int rc2 = pLhs->nTerm - pRhs->nTerm; + if( rc2<0 ){ + rc = memcmp(pLhs->zTerm, pRhs->zTerm, pLhs->nTerm); + }else{ + rc = memcmp(pLhs->zTerm, pRhs->zTerm, pRhs->nTerm); + } + if( rc==0 ){ + rc = rc2; + } + }else{ + rc = (pLhs->aNode==0) - (pRhs->aNode==0); + } + if( rc==0 ){ + rc = pRhs->iIdx - pLhs->iIdx; + } + assert( rc!=0 ); + return rc; +} + +/* +** A different comparison function for SegReader structures. In this +** version, it is assumed that each SegReader points to an entry in +** a doclist for identical terms. Comparison is made as follows: +** +** 1) EOF (end of doclist in this case) is greater than not EOF. +** +** 2) By current docid. +** +** 3) By segment age. An older segment is considered larger. +*/ +static int fts3SegReaderDoclistCmp(Fts3SegReader *pLhs, Fts3SegReader *pRhs){ + int rc = (pLhs->pOffsetList==0)-(pRhs->pOffsetList==0); + if( rc==0 ){ + if( pLhs->iDocid==pRhs->iDocid ){ + rc = pRhs->iIdx - pLhs->iIdx; + }else{ + rc = (pLhs->iDocid > pRhs->iDocid) ? 1 : -1; + } + } + assert( pLhs->aNode && pRhs->aNode ); + return rc; +} + +/* +** Compare the term that the Fts3SegReader object passed as the first argument +** points to with the term specified by arguments zTerm and nTerm. +** +** If the pSeg iterator is already at EOF, return 0. Otherwise, return +** -ve if the pSeg term is less than zTerm/nTerm, 0 if the two terms are +** equal, or +ve if the pSeg term is greater than zTerm/nTerm. +*/ +static int fts3SegReaderTermCmp( + Fts3SegReader *pSeg, /* Segment reader object */ + const char *zTerm, /* Term to compare to */ + int nTerm /* Size of term zTerm in bytes */ +){ + int res = 0; + if( pSeg->aNode ){ + if( pSeg->nTerm>nTerm ){ + res = memcmp(pSeg->zTerm, zTerm, nTerm); + }else{ + res = memcmp(pSeg->zTerm, zTerm, pSeg->nTerm); + } + if( res==0 ){ + res = pSeg->nTerm-nTerm; + } + } + return res; +} + +/* +** Argument apSegment is an array of nSegment elements. It is known that +** the final (nSegment-nSuspect) members are already in sorted order +** (according to the comparison function provided). This function shuffles +** the array around until all entries are in sorted order. +*/ +static void fts3SegReaderSort( + Fts3SegReader **apSegment, /* Array to sort entries of */ + int nSegment, /* Size of apSegment array */ + int nSuspect, /* Unsorted entry count */ + int (*xCmp)(Fts3SegReader *, Fts3SegReader *) /* Comparison function */ +){ + int i; /* Iterator variable */ + + assert( nSuspect<=nSegment ); + + if( nSuspect==nSegment ) nSuspect--; + for(i=nSuspect-1; i>=0; i--){ + int j; + for(j=i; j<(nSegment-1); j++){ + Fts3SegReader *pTmp; + if( xCmp(apSegment[j], apSegment[j+1])<0 ) break; + pTmp = apSegment[j+1]; + apSegment[j+1] = apSegment[j]; + apSegment[j] = pTmp; + } + } + +#ifndef NDEBUG + /* Check that the list really is sorted now. */ + for(i=0; i<(nSuspect-1); i++){ + assert( xCmp(apSegment[i], apSegment[i+1])<0 ); + } +#endif +} + +/* +** Insert a record into the %_segments table. +*/ +static int fts3WriteSegment( + Fts3Table *p, /* Virtual table handle */ + sqlite3_int64 iBlock, /* Block id for new block */ + char *z, /* Pointer to buffer containing block data */ + int n /* Size of buffer z in bytes */ +){ + sqlite3_stmt *pStmt; + int rc = fts3SqlStmt(p, SQL_INSERT_SEGMENTS, &pStmt, 0); + if( rc==SQLITE_OK ){ + sqlite3_bind_int64(pStmt, 1, iBlock); + rc = sqlite3_bind_blob(pStmt, 2, z, n, SQLITE_STATIC); + if( rc==SQLITE_OK ){ + sqlite3_step(pStmt); + rc = sqlite3_reset(pStmt); + } + } + return rc; +} + +/* +** Insert a record into the %_segdir table. +*/ +static int fts3WriteSegdir( + Fts3Table *p, /* Virtual table handle */ + int iLevel, /* Value for "level" field */ + int iIdx, /* Value for "idx" field */ + sqlite3_int64 iStartBlock, /* Value for "start_block" field */ + sqlite3_int64 iLeafEndBlock, /* Value for "leaves_end_block" field */ + sqlite3_int64 iEndBlock, /* Value for "end_block" field */ + char *zRoot, /* Blob value for "root" field */ + int nRoot /* Number of bytes in buffer zRoot */ +){ + sqlite3_stmt *pStmt; + int rc = fts3SqlStmt(p, SQL_INSERT_SEGDIR, &pStmt, 0); + if( rc==SQLITE_OK ){ + sqlite3_bind_int(pStmt, 1, iLevel); + sqlite3_bind_int(pStmt, 2, iIdx); + sqlite3_bind_int64(pStmt, 3, iStartBlock); + sqlite3_bind_int64(pStmt, 4, iLeafEndBlock); + sqlite3_bind_int64(pStmt, 5, iEndBlock); + rc = sqlite3_bind_blob(pStmt, 6, zRoot, nRoot, SQLITE_STATIC); + if( rc==SQLITE_OK ){ + sqlite3_step(pStmt); + rc = sqlite3_reset(pStmt); + } + } + return rc; +} + +/* +** Return the size of the common prefix (if any) shared by zPrev and +** zNext, in bytes. For example, +** +** fts3PrefixCompress("abc", 3, "abcdef", 6) // returns 3 +** fts3PrefixCompress("abX", 3, "abcdef", 6) // returns 2 +** fts3PrefixCompress("abX", 3, "Xbcdef", 6) // returns 0 +*/ +static int fts3PrefixCompress( + const char *zPrev, /* Buffer containing previous term */ + int nPrev, /* Size of buffer zPrev in bytes */ + const char *zNext, /* Buffer containing next term */ + int nNext /* Size of buffer zNext in bytes */ +){ + int n; + UNUSED_PARAMETER(nNext); + for(n=0; nnData; /* Current size of node in bytes */ + int nReq = nData; /* Required space after adding zTerm */ + int nPrefix; /* Number of bytes of prefix compression */ + int nSuffix; /* Suffix length */ + + nPrefix = fts3PrefixCompress(pTree->zTerm, pTree->nTerm, zTerm, nTerm); + nSuffix = nTerm-nPrefix; + + nReq += sqlite3Fts3VarintLen(nPrefix)+sqlite3Fts3VarintLen(nSuffix)+nSuffix; + if( nReq<=p->nNodeSize || !pTree->zTerm ){ + + if( nReq>p->nNodeSize ){ + /* An unusual case: this is the first term to be added to the node + ** and the static node buffer (p->nNodeSize bytes) is not large + ** enough. Use a separately malloced buffer instead This wastes + ** p->nNodeSize bytes, but since this scenario only comes about when + ** the database contain two terms that share a prefix of almost 2KB, + ** this is not expected to be a serious problem. + */ + assert( pTree->aData==(char *)&pTree[1] ); + pTree->aData = (char *)sqlite3_malloc(nReq); + if( !pTree->aData ){ + return SQLITE_NOMEM; + } + } + + if( pTree->zTerm ){ + /* There is no prefix-length field for first term in a node */ + nData += sqlite3Fts3PutVarint(&pTree->aData[nData], nPrefix); + } + + nData += sqlite3Fts3PutVarint(&pTree->aData[nData], nSuffix); + memcpy(&pTree->aData[nData], &zTerm[nPrefix], nSuffix); + pTree->nData = nData + nSuffix; + pTree->nEntry++; + + if( isCopyTerm ){ + if( pTree->nMalloczMalloc, nTerm*2); + if( !zNew ){ + return SQLITE_NOMEM; + } + pTree->nMalloc = nTerm*2; + pTree->zMalloc = zNew; + } + pTree->zTerm = pTree->zMalloc; + memcpy(pTree->zTerm, zTerm, nTerm); + pTree->nTerm = nTerm; + }else{ + pTree->zTerm = (char *)zTerm; + pTree->nTerm = nTerm; + } + return SQLITE_OK; + } + } + + /* If control flows to here, it was not possible to append zTerm to the + ** current node. Create a new node (a right-sibling of the current node). + ** If this is the first node in the tree, the term is added to it. + ** + ** Otherwise, the term is not added to the new node, it is left empty for + ** now. Instead, the term is inserted into the parent of pTree. If pTree + ** has no parent, one is created here. + */ + pNew = (SegmentNode *)sqlite3_malloc(sizeof(SegmentNode) + p->nNodeSize); + if( !pNew ){ + return SQLITE_NOMEM; + } + memset(pNew, 0, sizeof(SegmentNode)); + pNew->nData = 1 + FTS3_VARINT_MAX; + pNew->aData = (char *)&pNew[1]; + + if( pTree ){ + SegmentNode *pParent = pTree->pParent; + rc = fts3NodeAddTerm(p, &pParent, isCopyTerm, zTerm, nTerm); + if( pTree->pParent==0 ){ + pTree->pParent = pParent; + } + pTree->pRight = pNew; + pNew->pLeftmost = pTree->pLeftmost; + pNew->pParent = pParent; + pNew->zMalloc = pTree->zMalloc; + pNew->nMalloc = pTree->nMalloc; + pTree->zMalloc = 0; + }else{ + pNew->pLeftmost = pNew; + rc = fts3NodeAddTerm(p, &pNew, isCopyTerm, zTerm, nTerm); + } + + *ppTree = pNew; + return rc; +} + +/* +** Helper function for fts3NodeWrite(). +*/ +static int fts3TreeFinishNode( + SegmentNode *pTree, + int iHeight, + sqlite3_int64 iLeftChild +){ + int nStart; + assert( iHeight>=1 && iHeight<128 ); + nStart = FTS3_VARINT_MAX - sqlite3Fts3VarintLen(iLeftChild); + pTree->aData[nStart] = (char)iHeight; + sqlite3Fts3PutVarint(&pTree->aData[nStart+1], iLeftChild); + return nStart; +} + +/* +** Write the buffer for the segment node pTree and all of its peers to the +** database. Then call this function recursively to write the parent of +** pTree and its peers to the database. +** +** Except, if pTree is a root node, do not write it to the database. Instead, +** set output variables *paRoot and *pnRoot to contain the root node. +** +** If successful, SQLITE_OK is returned and output variable *piLast is +** set to the largest blockid written to the database (or zero if no +** blocks were written to the db). Otherwise, an SQLite error code is +** returned. +*/ +static int fts3NodeWrite( + Fts3Table *p, /* Virtual table handle */ + SegmentNode *pTree, /* SegmentNode handle */ + int iHeight, /* Height of this node in tree */ + sqlite3_int64 iLeaf, /* Block id of first leaf node */ + sqlite3_int64 iFree, /* Block id of next free slot in %_segments */ + sqlite3_int64 *piLast, /* OUT: Block id of last entry written */ + char **paRoot, /* OUT: Data for root node */ + int *pnRoot /* OUT: Size of root node in bytes */ +){ + int rc = SQLITE_OK; + + if( !pTree->pParent ){ + /* Root node of the tree. */ + int nStart = fts3TreeFinishNode(pTree, iHeight, iLeaf); + *piLast = iFree-1; + *pnRoot = pTree->nData - nStart; + *paRoot = &pTree->aData[nStart]; + }else{ + SegmentNode *pIter; + sqlite3_int64 iNextFree = iFree; + sqlite3_int64 iNextLeaf = iLeaf; + for(pIter=pTree->pLeftmost; pIter && rc==SQLITE_OK; pIter=pIter->pRight){ + int nStart = fts3TreeFinishNode(pIter, iHeight, iNextLeaf); + int nWrite = pIter->nData - nStart; + + rc = fts3WriteSegment(p, iNextFree, &pIter->aData[nStart], nWrite); + iNextFree++; + iNextLeaf += (pIter->nEntry+1); + } + if( rc==SQLITE_OK ){ + assert( iNextLeaf==iFree ); + rc = fts3NodeWrite( + p, pTree->pParent, iHeight+1, iFree, iNextFree, piLast, paRoot, pnRoot + ); + } + } + + return rc; +} + +/* +** Free all memory allocations associated with the tree pTree. +*/ +static void fts3NodeFree(SegmentNode *pTree){ + if( pTree ){ + SegmentNode *p = pTree->pLeftmost; + fts3NodeFree(p->pParent); + while( p ){ + SegmentNode *pRight = p->pRight; + if( p->aData!=(char *)&p[1] ){ + sqlite3_free(p->aData); + } + assert( pRight==0 || p->zMalloc==0 ); + sqlite3_free(p->zMalloc); + sqlite3_free(p); + p = pRight; + } + } +} + +/* +** Add a term to the segment being constructed by the SegmentWriter object +** *ppWriter. When adding the first term to a segment, *ppWriter should +** be passed NULL. This function will allocate a new SegmentWriter object +** and return it via the input/output variable *ppWriter in this case. +** +** If successful, SQLITE_OK is returned. Otherwise, an SQLite error code. +*/ +static int fts3SegWriterAdd( + Fts3Table *p, /* Virtual table handle */ + SegmentWriter **ppWriter, /* IN/OUT: SegmentWriter handle */ + int isCopyTerm, /* True if buffer zTerm must be copied */ + const char *zTerm, /* Pointer to buffer containing term */ + int nTerm, /* Size of term in bytes */ + const char *aDoclist, /* Pointer to buffer containing doclist */ + int nDoclist /* Size of doclist in bytes */ +){ + int nPrefix; /* Size of term prefix in bytes */ + int nSuffix; /* Size of term suffix in bytes */ + int nReq; /* Number of bytes required on leaf page */ + int nData; + SegmentWriter *pWriter = *ppWriter; + + if( !pWriter ){ + int rc; + sqlite3_stmt *pStmt; + + /* Allocate the SegmentWriter structure */ + pWriter = (SegmentWriter *)sqlite3_malloc(sizeof(SegmentWriter)); + if( !pWriter ) return SQLITE_NOMEM; + memset(pWriter, 0, sizeof(SegmentWriter)); + *ppWriter = pWriter; + + /* Allocate a buffer in which to accumulate data */ + pWriter->aData = (char *)sqlite3_malloc(p->nNodeSize); + if( !pWriter->aData ) return SQLITE_NOMEM; + pWriter->nSize = p->nNodeSize; + + /* Find the next free blockid in the %_segments table */ + rc = fts3SqlStmt(p, SQL_NEXT_SEGMENTS_ID, &pStmt, 0); + if( rc!=SQLITE_OK ) return rc; + if( SQLITE_ROW==sqlite3_step(pStmt) ){ + pWriter->iFree = sqlite3_column_int64(pStmt, 0); + pWriter->iFirst = pWriter->iFree; + } + rc = sqlite3_reset(pStmt); + if( rc!=SQLITE_OK ) return rc; + } + nData = pWriter->nData; + + nPrefix = fts3PrefixCompress(pWriter->zTerm, pWriter->nTerm, zTerm, nTerm); + nSuffix = nTerm-nPrefix; + + /* Figure out how many bytes are required by this new entry */ + nReq = sqlite3Fts3VarintLen(nPrefix) + /* varint containing prefix size */ + sqlite3Fts3VarintLen(nSuffix) + /* varint containing suffix size */ + nSuffix + /* Term suffix */ + sqlite3Fts3VarintLen(nDoclist) + /* Size of doclist */ + nDoclist; /* Doclist data */ + + if( nData>0 && nData+nReq>p->nNodeSize ){ + int rc; + + /* The current leaf node is full. Write it out to the database. */ + rc = fts3WriteSegment(p, pWriter->iFree++, pWriter->aData, nData); + if( rc!=SQLITE_OK ) return rc; + + /* Add the current term to the interior node tree. The term added to + ** the interior tree must: + ** + ** a) be greater than the largest term on the leaf node just written + ** to the database (still available in pWriter->zTerm), and + ** + ** b) be less than or equal to the term about to be added to the new + ** leaf node (zTerm/nTerm). + ** + ** In other words, it must be the prefix of zTerm 1 byte longer than + ** the common prefix (if any) of zTerm and pWriter->zTerm. + */ + assert( nPrefixpTree, isCopyTerm, zTerm, nPrefix+1); + if( rc!=SQLITE_OK ) return rc; + + nData = 0; + pWriter->nTerm = 0; + + nPrefix = 0; + nSuffix = nTerm; + nReq = 1 + /* varint containing prefix size */ + sqlite3Fts3VarintLen(nTerm) + /* varint containing suffix size */ + nTerm + /* Term suffix */ + sqlite3Fts3VarintLen(nDoclist) + /* Size of doclist */ + nDoclist; /* Doclist data */ + } + + /* If the buffer currently allocated is too small for this entry, realloc + ** the buffer to make it large enough. + */ + if( nReq>pWriter->nSize ){ + char *aNew = sqlite3_realloc(pWriter->aData, nReq); + if( !aNew ) return SQLITE_NOMEM; + pWriter->aData = aNew; + pWriter->nSize = nReq; + } + assert( nData+nReq<=pWriter->nSize ); + + /* Append the prefix-compressed term and doclist to the buffer. */ + nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nPrefix); + nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nSuffix); + memcpy(&pWriter->aData[nData], &zTerm[nPrefix], nSuffix); + nData += nSuffix; + nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nDoclist); + memcpy(&pWriter->aData[nData], aDoclist, nDoclist); + pWriter->nData = nData + nDoclist; + + /* Save the current term so that it can be used to prefix-compress the next. + ** If the isCopyTerm parameter is true, then the buffer pointed to by + ** zTerm is transient, so take a copy of the term data. Otherwise, just + ** store a copy of the pointer. + */ + if( isCopyTerm ){ + if( nTerm>pWriter->nMalloc ){ + char *zNew = sqlite3_realloc(pWriter->zMalloc, nTerm*2); + if( !zNew ){ + return SQLITE_NOMEM; + } + pWriter->nMalloc = nTerm*2; + pWriter->zMalloc = zNew; + pWriter->zTerm = zNew; + } + assert( pWriter->zTerm==pWriter->zMalloc ); + memcpy(pWriter->zTerm, zTerm, nTerm); + }else{ + pWriter->zTerm = (char *)zTerm; + } + pWriter->nTerm = nTerm; + + return SQLITE_OK; +} + +/* +** Flush all data associated with the SegmentWriter object pWriter to the +** database. This function must be called after all terms have been added +** to the segment using fts3SegWriterAdd(). If successful, SQLITE_OK is +** returned. Otherwise, an SQLite error code. +*/ +static int fts3SegWriterFlush( + Fts3Table *p, /* Virtual table handle */ + SegmentWriter *pWriter, /* SegmentWriter to flush to the db */ + int iLevel, /* Value for 'level' column of %_segdir */ + int iIdx /* Value for 'idx' column of %_segdir */ +){ + int rc; /* Return code */ + if( pWriter->pTree ){ + sqlite3_int64 iLast = 0; /* Largest block id written to database */ + sqlite3_int64 iLastLeaf; /* Largest leaf block id written to db */ + char *zRoot = NULL; /* Pointer to buffer containing root node */ + int nRoot = 0; /* Size of buffer zRoot */ + + iLastLeaf = pWriter->iFree; + rc = fts3WriteSegment(p, pWriter->iFree++, pWriter->aData, pWriter->nData); + if( rc==SQLITE_OK ){ + rc = fts3NodeWrite(p, pWriter->pTree, 1, + pWriter->iFirst, pWriter->iFree, &iLast, &zRoot, &nRoot); + } + if( rc==SQLITE_OK ){ + rc = fts3WriteSegdir( + p, iLevel, iIdx, pWriter->iFirst, iLastLeaf, iLast, zRoot, nRoot); + } + }else{ + /* The entire tree fits on the root node. Write it to the segdir table. */ + rc = fts3WriteSegdir( + p, iLevel, iIdx, 0, 0, 0, pWriter->aData, pWriter->nData); + } + return rc; +} + +/* +** Release all memory held by the SegmentWriter object passed as the +** first argument. +*/ +static void fts3SegWriterFree(SegmentWriter *pWriter){ + if( pWriter ){ + sqlite3_free(pWriter->aData); + sqlite3_free(pWriter->zMalloc); + fts3NodeFree(pWriter->pTree); + sqlite3_free(pWriter); + } +} + +/* +** The first value in the apVal[] array is assumed to contain an integer. +** This function tests if there exist any documents with docid values that +** are different from that integer. i.e. if deleting the document with docid +** apVal[0] would mean the FTS3 table were empty. +** +** If successful, *pisEmpty is set to true if the table is empty except for +** document apVal[0], or false otherwise, and SQLITE_OK is returned. If an +** error occurs, an SQLite error code is returned. +*/ +static int fts3IsEmpty(Fts3Table *p, sqlite3_value **apVal, int *pisEmpty){ + sqlite3_stmt *pStmt; + int rc; + rc = fts3SqlStmt(p, SQL_IS_EMPTY, &pStmt, apVal); + if( rc==SQLITE_OK ){ + if( SQLITE_ROW==sqlite3_step(pStmt) ){ + *pisEmpty = sqlite3_column_int(pStmt, 0); + } + rc = sqlite3_reset(pStmt); + } + return rc; +} + +/* +** Set *pnSegment to the number of segments of level iLevel in the database. +** +** Return SQLITE_OK if successful, or an SQLite error code if not. +*/ +static int fts3SegmentCount(Fts3Table *p, int iLevel, int *pnSegment){ + sqlite3_stmt *pStmt; + int rc; + + assert( iLevel>=0 ); + rc = fts3SqlStmt(p, SQL_SELECT_LEVEL_COUNT, &pStmt, 0); + if( rc!=SQLITE_OK ) return rc; + sqlite3_bind_int(pStmt, 1, iLevel); + if( SQLITE_ROW==sqlite3_step(pStmt) ){ + *pnSegment = sqlite3_column_int(pStmt, 0); + } + return sqlite3_reset(pStmt); +} + +/* +** Set *pnSegment to the total number of segments in the database. Set +** *pnMax to the largest segment level in the database (segment levels +** are stored in the 'level' column of the %_segdir table). +** +** Return SQLITE_OK if successful, or an SQLite error code if not. +*/ +static int fts3SegmentCountMax(Fts3Table *p, int *pnSegment, int *pnMax){ + sqlite3_stmt *pStmt; + int rc; + + rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_COUNT_MAX, &pStmt, 0); + if( rc!=SQLITE_OK ) return rc; + if( SQLITE_ROW==sqlite3_step(pStmt) ){ + *pnSegment = sqlite3_column_int(pStmt, 0); + *pnMax = sqlite3_column_int(pStmt, 1); + } + return sqlite3_reset(pStmt); +} + +/* +** This function is used after merging multiple segments into a single large +** segment to delete the old, now redundant, segment b-trees. Specifically, +** it: +** +** 1) Deletes all %_segments entries for the segments associated with +** each of the SegReader objects in the array passed as the third +** argument, and +** +** 2) deletes all %_segdir entries with level iLevel, or all %_segdir +** entries regardless of level if (iLevel<0). +** +** SQLITE_OK is returned if successful, otherwise an SQLite error code. +*/ +static int fts3DeleteSegdir( + Fts3Table *p, /* Virtual table handle */ + int iLevel, /* Level of %_segdir entries to delete */ + Fts3SegReader **apSegment, /* Array of SegReader objects */ + int nReader /* Size of array apSegment */ +){ + int rc; /* Return Code */ + int i; /* Iterator variable */ + sqlite3_stmt *pDelete; /* SQL statement to delete rows */ + + rc = fts3SqlStmt(p, SQL_DELETE_SEGMENTS_RANGE, &pDelete, 0); + for(i=0; rc==SQLITE_OK && iiStartBlock ){ + sqlite3_bind_int64(pDelete, 1, pSegment->iStartBlock); + sqlite3_bind_int64(pDelete, 2, pSegment->iEndBlock); + sqlite3_step(pDelete); + rc = sqlite3_reset(pDelete); + } + } + if( rc!=SQLITE_OK ){ + return rc; + } + + if( iLevel>=0 ){ + rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_BY_LEVEL, &pDelete, 0); + if( rc==SQLITE_OK ){ + sqlite3_bind_int(pDelete, 1, iLevel); + sqlite3_step(pDelete); + rc = sqlite3_reset(pDelete); + } + }else{ + rc = fts3SqlExec(p, SQL_DELETE_ALL_SEGDIR, 0); + } + + return rc; +} + +/* +** When this function is called, buffer *ppList (size *pnList bytes) contains +** a position list that may (or may not) feature multiple columns. This +** function adjusts the pointer *ppList and the length *pnList so that they +** identify the subset of the position list that corresponds to column iCol. +** +** If there are no entries in the input position list for column iCol, then +** *pnList is set to zero before returning. +*/ +static void fts3ColumnFilter( + int iCol, /* Column to filter on */ + char **ppList, /* IN/OUT: Pointer to position list */ + int *pnList /* IN/OUT: Size of buffer *ppList in bytes */ +){ + char *pList = *ppList; + int nList = *pnList; + char *pEnd = &pList[nList]; + int iCurrent = 0; + char *p = pList; + + assert( iCol>=0 ); + while( 1 ){ + char c = 0; + while( pflags & FTS3_SEGMENT_IGNORE_EMPTY); + int isRequirePos = (pFilter->flags & FTS3_SEGMENT_REQUIRE_POS); + int isColFilter = (pFilter->flags & FTS3_SEGMENT_COLUMN_FILTER); + int isPrefix = (pFilter->flags & FTS3_SEGMENT_PREFIX); + + /* If there are zero segments, this function is a no-op. This scenario + ** comes about only when reading from an empty database. + */ + if( nSegment==0 ) goto finished; + + /* If the Fts3SegFilter defines a specific term (or term prefix) to search + ** for, then advance each segment iterator until it points to a term of + ** equal or greater value than the specified term. This prevents many + ** unnecessary merge/sort operations for the case where single segment + ** b-tree leaf nodes contain more than one term. + */ + if( pFilter->zTerm ){ + int nTerm = pFilter->nTerm; + const char *zTerm = pFilter->zTerm; + for(i=0; iaNode ){ + int nTerm = apSegment[0]->nTerm; + char *zTerm = apSegment[0]->zTerm; + int nMerge = 1; + + /* If this is a prefix-search, and if the term that apSegment[0] points + ** to does not share a suffix with pFilter->zTerm/nTerm, then all + ** required callbacks have been made. In this case exit early. + ** + ** Similarly, if this is a search for an exact match, and the first term + ** of segment apSegment[0] is not a match, exit early. + */ + if( pFilter->zTerm ){ + if( nTermnTerm + || (!isPrefix && nTerm>pFilter->nTerm) + || memcmp(zTerm, pFilter->zTerm, pFilter->nTerm) + ){ + goto finished; + } + } + + while( nMergeaNode + && apSegment[nMerge]->nTerm==nTerm + && 0==memcmp(zTerm, apSegment[nMerge]->zTerm, nTerm) + ){ + nMerge++; + } + + if( nMerge==1 && !isIgnoreEmpty && !isColFilter && isRequirePos ){ + Fts3SegReader *p0 = apSegment[0]; + rc = xFunc(p, pContext, zTerm, nTerm, p0->aDoclist, p0->nDoclist); + if( rc!=SQLITE_OK ) goto finished; + }else{ + int nDoclist = 0; /* Size of doclist */ + sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */ + + /* The current term of the first nMerge entries in the array + ** of Fts3SegReader objects is the same. The doclists must be merged + ** and a single term added to the new segment. + */ + for(i=0; ipOffsetList ){ + int j; /* Number of segments that share a docid */ + char *pList; + int nList; + int nByte; + sqlite3_int64 iDocid = apSegment[0]->iDocid; + fts3SegReaderNextDocid(apSegment[0], &pList, &nList); + j = 1; + while( jpOffsetList + && apSegment[j]->iDocid==iDocid + ){ + fts3SegReaderNextDocid(apSegment[j], 0, 0); + j++; + } + + if( isColFilter ){ + fts3ColumnFilter(pFilter->iCol, &pList, &nList); + } + + if( !isIgnoreEmpty || nList>0 ){ + nByte = sqlite3Fts3VarintLen(iDocid-iPrev) + (isRequirePos?nList+1:0); + if( nDoclist+nByte>nAlloc ){ + char *aNew; + nAlloc = nDoclist+nByte*2; + aNew = sqlite3_realloc(aBuffer, nAlloc); + if( !aNew ){ + rc = SQLITE_NOMEM; + goto finished; + } + aBuffer = aNew; + } + nDoclist += sqlite3Fts3PutVarint(&aBuffer[nDoclist], iDocid-iPrev); + iPrev = iDocid; + if( isRequirePos ){ + memcpy(&aBuffer[nDoclist], pList, nList); + nDoclist += nList; + aBuffer[nDoclist++] = '\0'; + } + } + + fts3SegReaderSort(apSegment, nMerge, j, fts3SegReaderDoclistCmp); + } + + if( nDoclist>0 ){ + rc = xFunc(p, pContext, zTerm, nTerm, aBuffer, nDoclist); + if( rc!=SQLITE_OK ) goto finished; + } + } + + /* If there is a term specified to filter on, and this is not a prefix + ** search, return now. The callback that corresponds to the required + ** term (if such a term exists in the index) has already been made. + */ + if( pFilter->zTerm && !isPrefix ){ + goto finished; + } + + for(i=0; i0 ); + assert( iNewLevel>=0 ); + + /* Allocate space for an array of pointers to segment iterators. */ + apSegment = (Fts3SegReader**)sqlite3_malloc(sizeof(Fts3SegReader *)*nSegment); + if( !apSegment ){ + return SQLITE_NOMEM; + } + memset(apSegment, 0, sizeof(Fts3SegReader *)*nSegment); + + /* Allocate a Fts3SegReader structure for each segment being merged. A + ** Fts3SegReader stores the state data required to iterate through all + ** entries on all leaves of a single segment. + */ + assert( SQL_SELECT_LEVEL+1==SQL_SELECT_ALL_LEVEL); + rc = fts3SqlStmt(p, SQL_SELECT_LEVEL+(iLevel<0), &pStmt, 0); + if( rc!=SQLITE_OK ) goto finished; + sqlite3_bind_int(pStmt, 1, iLevel); + for(i=0; SQLITE_ROW==(sqlite3_step(pStmt)); i++){ + rc = fts3SegReaderNew(p, pStmt, i, &apSegment[i]); + if( rc!=SQLITE_OK ){ + goto finished; + } + } + rc = sqlite3_reset(pStmt); + pStmt = 0; + if( rc!=SQLITE_OK ) goto finished; + + memset(&filter, 0, sizeof(Fts3SegFilter)); + filter.flags = FTS3_SEGMENT_REQUIRE_POS; + filter.flags |= (iLevel<0 ? FTS3_SEGMENT_IGNORE_EMPTY : 0); + rc = sqlite3Fts3SegReaderIterate(p, apSegment, nSegment, + &filter, fts3MergeCallback, (void *)&pWriter + ); + if( rc!=SQLITE_OK ) goto finished; + + rc = fts3DeleteSegdir(p, iLevel, apSegment, nSegment); + if( rc==SQLITE_OK ){ + rc = fts3SegWriterFlush(p, pWriter, iNewLevel, iIdx); + } + + finished: + fts3SegWriterFree(pWriter); + if( apSegment ){ + for(i=0; ipendingTerms); + if( nElem==0 ){ + assert( p->nPendingData==0 ); + return SQLITE_OK; + } + + /* Determine the next index at level 0, merging as necessary. */ + rc = fts3AllocateSegdirIdx(p, 0, &idx); + if( rc!=SQLITE_OK ){ + return rc; + } + + apElem = sqlite3_malloc(nElem*sizeof(Fts3HashElem *)); + if( !apElem ){ + return SQLITE_NOMEM; + } + + i = 0; + for(pElem=fts3HashFirst(&p->pendingTerms); pElem; pElem=fts3HashNext(pElem)){ + apElem[i++] = pElem; + } + assert( i==nElem ); + + /* TODO(shess) Should we allow user-defined collation sequences, + ** here? I think we only need that once we support prefix searches. + ** Also, should we be using qsort()? + */ + if( nElem>1 ){ + qsort(apElem, nElem, sizeof(Fts3HashElem *), qsortCompare); + } + + + /* Write the segment tree into the database. */ + for(i=0; rc==SQLITE_OK && iaData, pList->nData+1); + } + if( rc==SQLITE_OK ){ + rc = fts3SegWriterFlush(p, pWriter, 0, idx); + } + + /* Free all allocated resources before returning */ + fts3SegWriterFree(pWriter); + sqlite3_free(apElem); + sqlite3Fts3PendingTermsClear(p); + return rc; +} + +/* +** This function does the work for the xUpdate method of FTS3 virtual +** tables. +*/ +SQLITE_PRIVATE int sqlite3Fts3UpdateMethod( + sqlite3_vtab *pVtab, /* FTS3 vtab object */ + int nArg, /* Size of argument array */ + sqlite3_value **apVal, /* Array of arguments */ + sqlite_int64 *pRowid /* OUT: The affected (or effected) rowid */ +){ + Fts3Table *p = (Fts3Table *)pVtab; + int rc = SQLITE_OK; /* Return Code */ + int isRemove = 0; /* True for an UPDATE or DELETE */ + sqlite3_int64 iRemove = 0; /* Rowid removed by UPDATE or DELETE */ + + /* If this is a DELETE or UPDATE operation, remove the old record. */ + if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){ + int isEmpty; + rc = fts3IsEmpty(p, apVal, &isEmpty); + if( rc==SQLITE_OK ){ + if( isEmpty ){ + /* Deleting this row means the whole table is empty. In this case + ** delete the contents of all three tables and throw away any + ** data in the pendingTerms hash table. + */ + rc = fts3DeleteAll(p); + }else{ + isRemove = 1; + iRemove = sqlite3_value_int64(apVal[0]); + rc = fts3PendingTermsDocid(p, iRemove); + if( rc==SQLITE_OK ){ + rc = fts3DeleteTerms(p, apVal); + if( rc==SQLITE_OK ){ + rc = fts3SqlExec(p, SQL_DELETE_CONTENT, apVal); + } + } + } + } + } + + /* If this is an INSERT or UPDATE operation, insert the new record. */ + if( nArg>1 && rc==SQLITE_OK ){ + rc = fts3InsertData(p, apVal, pRowid); + if( rc==SQLITE_OK && (!isRemove || *pRowid!=iRemove) ){ + rc = fts3PendingTermsDocid(p, *pRowid); + } + if( rc==SQLITE_OK ){ + rc = fts3InsertTerms(p, apVal); + } + } + + return rc; +} + +/* +** Flush any data in the pending-terms hash table to disk. If successful, +** merge all segments in the database (including the new segment, if +** there was any data to flush) into a single segment. +*/ +SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *p){ + int rc; + rc = sqlite3_exec(p->db, "SAVEPOINT fts3", 0, 0, 0); + if( rc==SQLITE_OK ){ + rc = sqlite3Fts3PendingTermsFlush(p); + if( rc==SQLITE_OK ){ + rc = fts3SegmentMerge(p, -1); + } + if( rc==SQLITE_OK ){ + rc = sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0); + }else{ + sqlite3_exec(p->db, "ROLLBACK TO fts3 ; RELEASE fts3", 0, 0, 0); + } + } + return rc; +} + +#endif + +/************** End of fts3_write.c ******************************************/ +/************** Begin file fts3_snippet.c ************************************/ +/* +** 2009 Oct 23 +** +** The author disclaims copyright to this source code. In place of +** a legal notice, here is a blessing: +** +** May you do good and not evil. +** May you find forgiveness for yourself and forgive others. +** May you share freely, never taking more than you give. +** +****************************************************************************** +*/ + +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) + + +typedef struct Snippet Snippet; + +/* +** An instance of the following structure keeps track of generated +** matching-word offset information and snippets. +*/ +struct Snippet { + int nMatch; /* Total number of matches */ + int nAlloc; /* Space allocated for aMatch[] */ + struct snippetMatch { /* One entry for each matching term */ + char snStatus; /* Status flag for use while constructing snippets */ + short int nByte; /* Number of bytes in the term */ + short int iCol; /* The column that contains the match */ + short int iTerm; /* The index in Query.pTerms[] of the matching term */ + int iToken; /* The index of the matching document token */ + int iStart; /* The offset to the first character of the term */ + } *aMatch; /* Points to space obtained from malloc */ + char *zOffset; /* Text rendering of aMatch[] */ + int nOffset; /* strlen(zOffset) */ + char *zSnippet; /* Snippet text */ + int nSnippet; /* strlen(zSnippet) */ +}; + + +/* It is not safe to call isspace(), tolower(), or isalnum() on +** hi-bit-set characters. This is the same solution used in the +** tokenizer. +*/ +static int fts3snippetIsspace(char c){ + return (c&0x80)==0 ? isspace(c) : 0; +} + + +/* +** A StringBuffer object holds a zero-terminated string that grows +** arbitrarily by appending. Space to hold the string is obtained +** from sqlite3_malloc(). After any memory allocation failure, +** StringBuffer.z is set to NULL and no further allocation is attempted. +*/ +typedef struct StringBuffer { + char *z; /* Text of the string. Space from malloc. */ + int nUsed; /* Number bytes of z[] used, not counting \000 terminator */ + int nAlloc; /* Bytes allocated for z[] */ +} StringBuffer; + + +/* +** Initialize a new StringBuffer. +*/ +static void fts3SnippetSbInit(StringBuffer *p){ + p->nAlloc = 100; + p->nUsed = 0; + p->z = sqlite3_malloc( p->nAlloc ); +} + +/* +** Append text to the string buffer. +*/ +static void fts3SnippetAppend(StringBuffer *p, const char *zNew, int nNew){ + if( p->z==0 ) return; + if( nNew<0 ) nNew = (int)strlen(zNew); + if( p->nUsed + nNew >= p->nAlloc ){ + int nAlloc; + char *zNew; + + nAlloc = p->nUsed + nNew + p->nAlloc; + zNew = sqlite3_realloc(p->z, nAlloc); + if( zNew==0 ){ + sqlite3_free(p->z); + p->z = 0; + return; + } + p->z = zNew; + p->nAlloc = nAlloc; + } + memcpy(&p->z[p->nUsed], zNew, nNew); + p->nUsed += nNew; + p->z[p->nUsed] = 0; +} + +/* If the StringBuffer ends in something other than white space, add a +** single space character to the end. +*/ +static void fts3SnippetAppendWhiteSpace(StringBuffer *p){ + if( p->z && p->nUsed && !fts3snippetIsspace(p->z[p->nUsed-1]) ){ + fts3SnippetAppend(p, " ", 1); + } +} + +/* Remove white space from the end of the StringBuffer */ +static void fts3SnippetTrimWhiteSpace(StringBuffer *p){ + if( p->z ){ + while( p->nUsed && fts3snippetIsspace(p->z[p->nUsed-1]) ){ + p->nUsed--; + } + p->z[p->nUsed] = 0; + } +} + +/* +** Release all memory associated with the Snippet structure passed as +** an argument. +*/ +static void fts3SnippetFree(Snippet *p){ + if( p ){ + sqlite3_free(p->aMatch); + sqlite3_free(p->zOffset); + sqlite3_free(p->zSnippet); + sqlite3_free(p); + } +} + +/* +** Append a single entry to the p->aMatch[] log. +*/ +static int snippetAppendMatch( + Snippet *p, /* Append the entry to this snippet */ + int iCol, int iTerm, /* The column and query term */ + int iToken, /* Matching token in document */ + int iStart, int nByte /* Offset and size of the match */ +){ + int i; + struct snippetMatch *pMatch; + if( p->nMatch+1>=p->nAlloc ){ + struct snippetMatch *pNew; + p->nAlloc = p->nAlloc*2 + 10; + pNew = sqlite3_realloc(p->aMatch, p->nAlloc*sizeof(p->aMatch[0]) ); + if( pNew==0 ){ + p->aMatch = 0; + p->nMatch = 0; + p->nAlloc = 0; + return SQLITE_NOMEM; + } + p->aMatch = pNew; + } + i = p->nMatch++; + pMatch = &p->aMatch[i]; + pMatch->iCol = (short)iCol; + pMatch->iTerm = (short)iTerm; + pMatch->iToken = iToken; + pMatch->iStart = iStart; + pMatch->nByte = (short)nByte; + return SQLITE_OK; +} + +/* +** Sizing information for the circular buffer used in snippetOffsetsOfColumn() +*/ +#define FTS3_ROTOR_SZ (32) +#define FTS3_ROTOR_MASK (FTS3_ROTOR_SZ-1) + +/* +** Function to iterate through the tokens of a compiled expression. +** +** Except, skip all tokens on the right-hand side of a NOT operator. +** This function is used to find tokens as part of snippet and offset +** generation and we do nt want snippets and offsets to report matches +** for tokens on the RHS of a NOT. +*/ +static int fts3NextExprToken(Fts3Expr **ppExpr, int *piToken){ + Fts3Expr *p = *ppExpr; + int iToken = *piToken; + if( iToken<0 ){ + /* In this case the expression p is the root of an expression tree. + ** Move to the first token in the expression tree. + */ + while( p->pLeft ){ + p = p->pLeft; + } + iToken = 0; + }else{ + assert(p && p->eType==FTSQUERY_PHRASE ); + if( iToken<(p->pPhrase->nToken-1) ){ + iToken++; + }else{ + iToken = 0; + while( p->pParent && p->pParent->pLeft!=p ){ + assert( p->pParent->pRight==p ); + p = p->pParent; + } + p = p->pParent; + if( p ){ + assert( p->pRight!=0 ); + p = p->pRight; + while( p->pLeft ){ + p = p->pLeft; + } + } + } + } + + *ppExpr = p; + *piToken = iToken; + return p?1:0; +} + +/* +** Return TRUE if the expression node pExpr is located beneath the +** RHS of a NOT operator. +*/ +static int fts3ExprBeneathNot(Fts3Expr *p){ + Fts3Expr *pParent; + while( p ){ + pParent = p->pParent; + if( pParent && pParent->eType==FTSQUERY_NOT && pParent->pRight==p ){ + return 1; + } + p = pParent; + } + return 0; +} + +/* +** Add entries to pSnippet->aMatch[] for every match that occurs against +** document zDoc[0..nDoc-1] which is stored in column iColumn. +*/ +static int snippetOffsetsOfColumn( + Fts3Cursor *pCur, /* The fulltest search cursor */ + Snippet *pSnippet, /* The Snippet object to be filled in */ + int iColumn, /* Index of fulltext table column */ + const char *zDoc, /* Text of the fulltext table column */ + int nDoc /* Length of zDoc in bytes */ +){ + const sqlite3_tokenizer_module *pTModule; /* The tokenizer module */ + sqlite3_tokenizer *pTokenizer; /* The specific tokenizer */ + sqlite3_tokenizer_cursor *pTCursor; /* Tokenizer cursor */ + Fts3Table *pVtab; /* The full text index */ + int nColumn; /* Number of columns in the index */ + int i, j; /* Loop counters */ + int rc; /* Return code */ + unsigned int match, prevMatch; /* Phrase search bitmasks */ + const char *zToken; /* Next token from the tokenizer */ + int nToken; /* Size of zToken */ + int iBegin, iEnd, iPos; /* Offsets of beginning and end */ + + /* The following variables keep a circular buffer of the last + ** few tokens */ + unsigned int iRotor = 0; /* Index of current token */ + int iRotorBegin[FTS3_ROTOR_SZ]; /* Beginning offset of token */ + int iRotorLen[FTS3_ROTOR_SZ]; /* Length of token */ + + pVtab = (Fts3Table *)pCur->base.pVtab; + nColumn = pVtab->nColumn; + pTokenizer = pVtab->pTokenizer; + pTModule = pTokenizer->pModule; + rc = pTModule->xOpen(pTokenizer, zDoc, nDoc, &pTCursor); + if( rc ) return rc; + pTCursor->pTokenizer = pTokenizer; + + prevMatch = 0; + while( (rc = pTModule->xNext(pTCursor, &zToken, &nToken, + &iBegin, &iEnd, &iPos))==SQLITE_OK ){ + Fts3Expr *pIter = pCur->pExpr; + int iIter = -1; + iRotorBegin[iRotor&FTS3_ROTOR_MASK] = iBegin; + iRotorLen[iRotor&FTS3_ROTOR_MASK] = iEnd-iBegin; + match = 0; + for(i=0; i<(FTS3_ROTOR_SZ-1) && fts3NextExprToken(&pIter, &iIter); i++){ + int nPhrase; /* Number of tokens in current phrase */ + struct PhraseToken *pToken; /* Current token */ + int iCol; /* Column index */ + + if( fts3ExprBeneathNot(pIter) ) continue; + nPhrase = pIter->pPhrase->nToken; + pToken = &pIter->pPhrase->aToken[iIter]; + iCol = pIter->pPhrase->iColumn; + if( iCol>=0 && iColn>nToken ) continue; + if( !pToken->isPrefix && pToken->nn<=nToken ); + if( memcmp(pToken->z, zToken, pToken->n) ) continue; + if( iIter>0 && (prevMatch & (1<=0; j--){ + int k = (iRotor-j) & FTS3_ROTOR_MASK; + rc = snippetAppendMatch(pSnippet, iColumn, i-j, iPos-j, + iRotorBegin[k], iRotorLen[k]); + if( rc ) goto end_offsets_of_column; + } + } + } + prevMatch = match<<1; + iRotor++; + } +end_offsets_of_column: + pTModule->xClose(pTCursor); + return rc==SQLITE_DONE ? SQLITE_OK : rc; +} + +/* +** Remove entries from the pSnippet structure to account for the NEAR +** operator. When this is called, pSnippet contains the list of token +** offsets produced by treating all NEAR operators as AND operators. +** This function removes any entries that should not be present after +** accounting for the NEAR restriction. For example, if the queried +** document is: +** +** "A B C D E A" +** +** and the query is: +** +** A NEAR/0 E +** +** then when this function is called the Snippet contains token offsets +** 0, 4 and 5. This function removes the "0" entry (because the first A +** is not near enough to an E). +** +** When this function is called, the value pointed to by parameter piLeft is +** the integer id of the left-most token in the expression tree headed by +** pExpr. This function increments *piLeft by the total number of tokens +** in the expression tree headed by pExpr. +** +** Return 1 if any trimming occurs. Return 0 if no trimming is required. +*/ +static int trimSnippetOffsets( + Fts3Expr *pExpr, /* The search expression */ + Snippet *pSnippet, /* The set of snippet offsets to be trimmed */ + int *piLeft /* Index of left-most token in pExpr */ +){ + if( pExpr ){ + if( trimSnippetOffsets(pExpr->pLeft, pSnippet, piLeft) ){ + return 1; + } + + switch( pExpr->eType ){ + case FTSQUERY_PHRASE: + *piLeft += pExpr->pPhrase->nToken; + break; + case FTSQUERY_NEAR: { + /* The right-hand-side of a NEAR operator is always a phrase. The + ** left-hand-side is either a phrase or an expression tree that is + ** itself headed by a NEAR operator. The following initializations + ** set local variable iLeft to the token number of the left-most + ** token in the right-hand phrase, and iRight to the right most + ** token in the same phrase. For example, if we had: + ** + ** MATCH '"abc def" NEAR/2 "ghi jkl"' + ** + ** then iLeft will be set to 2 (token number of ghi) and nToken will + ** be set to 4. + */ + Fts3Expr *pLeft = pExpr->pLeft; + Fts3Expr *pRight = pExpr->pRight; + int iLeft = *piLeft; + int nNear = pExpr->nNear; + int nToken = pRight->pPhrase->nToken; + int jj, ii; + if( pLeft->eType==FTSQUERY_NEAR ){ + pLeft = pLeft->pRight; + } + assert( pRight->eType==FTSQUERY_PHRASE ); + assert( pLeft->eType==FTSQUERY_PHRASE ); + nToken += pLeft->pPhrase->nToken; + + for(ii=0; iinMatch; ii++){ + struct snippetMatch *p = &pSnippet->aMatch[ii]; + if( p->iTerm==iLeft ){ + int isOk = 0; + /* Snippet ii is an occurence of query term iLeft in the document. + ** It occurs at position (p->iToken) of the document. We now + ** search for an instance of token (iLeft-1) somewhere in the + ** range (p->iToken - nNear)...(p->iToken + nNear + nToken) within + ** the set of snippetMatch structures. If one is found, proceed. + ** If one cannot be found, then remove snippets ii..(ii+N-1) + ** from the matching snippets, where N is the number of tokens + ** in phrase pRight->pPhrase. + */ + for(jj=0; isOk==0 && jjnMatch; jj++){ + struct snippetMatch *p2 = &pSnippet->aMatch[jj]; + if( p2->iTerm==(iLeft-1) ){ + if( p2->iToken>=(p->iToken-nNear-1) + && p2->iToken<(p->iToken+nNear+nToken) + ){ + isOk = 1; + } + } + } + if( !isOk ){ + int kk; + for(kk=0; kkpPhrase->nToken; kk++){ + pSnippet->aMatch[kk+ii].iTerm = -2; + } + return 1; + } + } + if( p->iTerm==(iLeft-1) ){ + int isOk = 0; + for(jj=0; isOk==0 && jjnMatch; jj++){ + struct snippetMatch *p2 = &pSnippet->aMatch[jj]; + if( p2->iTerm==iLeft ){ + if( p2->iToken<=(p->iToken+nNear+1) + && p2->iToken>(p->iToken-nNear-nToken) + ){ + isOk = 1; + } + } + } + if( !isOk ){ + int kk; + for(kk=0; kkpPhrase->nToken; kk++){ + pSnippet->aMatch[ii-kk].iTerm = -2; + } + return 1; + } + } + } + break; + } + } + + if( trimSnippetOffsets(pExpr->pRight, pSnippet, piLeft) ){ + return 1; + } + } + return 0; +} + +/* +** Compute all offsets for the current row of the query. +** If the offsets have already been computed, this routine is a no-op. +*/ +static int snippetAllOffsets(Fts3Cursor *pCsr, Snippet **ppSnippet){ + Fts3Table *p = (Fts3Table *)pCsr->base.pVtab; /* The FTS3 virtual table */ + int nColumn; /* Number of columns. Docid does count */ + int iColumn; /* Index of of a column */ + int i; /* Loop index */ + int iFirst; /* First column to search */ + int iLast; /* Last coumn to search */ + int iTerm = 0; + Snippet *pSnippet; + int rc = SQLITE_OK; + + if( pCsr->pExpr==0 ){ + return SQLITE_OK; + } + + pSnippet = (Snippet *)sqlite3_malloc(sizeof(Snippet)); + *ppSnippet = pSnippet; + if( !pSnippet ){ + return SQLITE_NOMEM; + } + memset(pSnippet, 0, sizeof(Snippet)); + + nColumn = p->nColumn; + iColumn = (pCsr->eSearch - 2); + if( iColumn<0 || iColumn>=nColumn ){ + /* Look for matches over all columns of the full-text index */ + iFirst = 0; + iLast = nColumn-1; + }else{ + /* Look for matches in the iColumn-th column of the index only */ + iFirst = iColumn; + iLast = iColumn; + } + for(i=iFirst; rc==SQLITE_OK && i<=iLast; i++){ + const char *zDoc; + int nDoc; + zDoc = (const char*)sqlite3_column_text(pCsr->pStmt, i+1); + nDoc = sqlite3_column_bytes(pCsr->pStmt, i+1); + if( zDoc==0 && sqlite3_column_type(pCsr->pStmt, i+1)!=SQLITE_NULL ){ + rc = SQLITE_NOMEM; + }else{ + rc = snippetOffsetsOfColumn(pCsr, pSnippet, i, zDoc, nDoc); + } + } + + while( trimSnippetOffsets(pCsr->pExpr, pSnippet, &iTerm) ){ + iTerm = 0; + } + + return rc; +} + +/* +** Convert the information in the aMatch[] array of the snippet +** into the string zOffset[0..nOffset-1]. This string is used as +** the return of the SQL offsets() function. +*/ +static void snippetOffsetText(Snippet *p){ + int i; + int cnt = 0; + StringBuffer sb; + char zBuf[200]; + if( p->zOffset ) return; + fts3SnippetSbInit(&sb); + for(i=0; inMatch; i++){ + struct snippetMatch *pMatch = &p->aMatch[i]; + if( pMatch->iTerm>=0 ){ + /* If snippetMatch.iTerm is less than 0, then the match was + ** discarded as part of processing the NEAR operator (see the + ** trimSnippetOffsetsForNear() function for details). Ignore + ** it in this case + */ + zBuf[0] = ' '; + sqlite3_snprintf(sizeof(zBuf)-1, &zBuf[cnt>0], "%d %d %d %d", + pMatch->iCol, pMatch->iTerm, pMatch->iStart, pMatch->nByte); + fts3SnippetAppend(&sb, zBuf, -1); + cnt++; + } + } + p->zOffset = sb.z; + p->nOffset = sb.z ? sb.nUsed : 0; +} + +/* +** zDoc[0..nDoc-1] is phrase of text. aMatch[0..nMatch-1] are a set +** of matching words some of which might be in zDoc. zDoc is column +** number iCol. +** +** iBreak is suggested spot in zDoc where we could begin or end an +** excerpt. Return a value similar to iBreak but possibly adjusted +** to be a little left or right so that the break point is better. +*/ +static int wordBoundary( + int iBreak, /* The suggested break point */ + const char *zDoc, /* Document text */ + int nDoc, /* Number of bytes in zDoc[] */ + struct snippetMatch *aMatch, /* Matching words */ + int nMatch, /* Number of entries in aMatch[] */ + int iCol /* The column number for zDoc[] */ +){ + int i; + if( iBreak<=10 ){ + return 0; + } + if( iBreak>=nDoc-10 ){ + return nDoc; + } + for(i=0; ALWAYS(i0 && aMatch[i-1].iStart+aMatch[i-1].nByte>=iBreak ){ + return aMatch[i-1].iStart; + } + } + for(i=1; i<=10; i++){ + if( fts3snippetIsspace(zDoc[iBreak-i]) ){ + return iBreak - i + 1; + } + if( fts3snippetIsspace(zDoc[iBreak+i]) ){ + return iBreak + i + 1; + } + } + return iBreak; +} + + + +/* +** Allowed values for Snippet.aMatch[].snStatus +*/ +#define SNIPPET_IGNORE 0 /* It is ok to omit this match from the snippet */ +#define SNIPPET_DESIRED 1 /* We want to include this match in the snippet */ + +/* +** Generate the text of a snippet. +*/ +static void snippetText( + Fts3Cursor *pCursor, /* The cursor we need the snippet for */ + Snippet *pSnippet, + const char *zStartMark, /* Markup to appear before each match */ + const char *zEndMark, /* Markup to appear after each match */ + const char *zEllipsis /* Ellipsis mark */ +){ + int i, j; + struct snippetMatch *aMatch; + int nMatch; + int nDesired; + StringBuffer sb; + int tailCol; + int tailOffset; + int iCol; + int nDoc; + const char *zDoc; + int iStart, iEnd; + int tailEllipsis = 0; + int iMatch; + + + sqlite3_free(pSnippet->zSnippet); + pSnippet->zSnippet = 0; + aMatch = pSnippet->aMatch; + nMatch = pSnippet->nMatch; + fts3SnippetSbInit(&sb); + + for(i=0; i0; i++){ + if( aMatch[i].snStatus!=SNIPPET_DESIRED ) continue; + nDesired--; + iCol = aMatch[i].iCol; + zDoc = (const char*)sqlite3_column_text(pCursor->pStmt, iCol+1); + nDoc = sqlite3_column_bytes(pCursor->pStmt, iCol+1); + iStart = aMatch[i].iStart - 40; + iStart = wordBoundary(iStart, zDoc, nDoc, aMatch, nMatch, iCol); + if( iStart<=10 ){ + iStart = 0; + } + if( iCol==tailCol && iStart<=tailOffset+20 ){ + iStart = tailOffset; + } + if( (iCol!=tailCol && tailCol>=0) || iStart!=tailOffset ){ + fts3SnippetTrimWhiteSpace(&sb); + fts3SnippetAppendWhiteSpace(&sb); + fts3SnippetAppend(&sb, zEllipsis, -1); + fts3SnippetAppendWhiteSpace(&sb); + } + iEnd = aMatch[i].iStart + aMatch[i].nByte + 40; + iEnd = wordBoundary(iEnd, zDoc, nDoc, aMatch, nMatch, iCol); + if( iEnd>=nDoc-10 ){ + iEnd = nDoc; + tailEllipsis = 0; + }else{ + tailEllipsis = 1; + } + while( iMatchzSnippet = sb.z; + pSnippet->nSnippet = sb.z ? sb.nUsed : 0; +} + +SQLITE_PRIVATE void sqlite3Fts3Offsets( + sqlite3_context *pCtx, /* SQLite function call context */ + Fts3Cursor *pCsr /* Cursor object */ +){ + Snippet *p; /* Snippet structure */ + int rc = snippetAllOffsets(pCsr, &p); + if( rc==SQLITE_OK ){ + snippetOffsetText(p); + if( p->zOffset ){ + sqlite3_result_text(pCtx, p->zOffset, p->nOffset, SQLITE_TRANSIENT); + }else{ + sqlite3_result_error_nomem(pCtx); + } + }else{ + sqlite3_result_error_nomem(pCtx); + } + fts3SnippetFree(p); +} + +SQLITE_PRIVATE void sqlite3Fts3Snippet( + sqlite3_context *pCtx, /* SQLite function call context */ + Fts3Cursor *pCsr, /* Cursor object */ + const char *zStart, /* Snippet start text - "" */ + const char *zEnd, /* Snippet end text - "" */ + const char *zEllipsis /* Snippet ellipsis text - "..." */ +){ + Snippet *p; /* Snippet structure */ + int rc = snippetAllOffsets(pCsr, &p); + if( rc==SQLITE_OK ){ + snippetText(pCsr, p, zStart, zEnd, zEllipsis); + if( p->zSnippet ){ + sqlite3_result_text(pCtx, p->zSnippet, p->nSnippet, SQLITE_TRANSIENT); + }else{ + sqlite3_result_error_nomem(pCtx); + } + }else{ + sqlite3_result_error_nomem(pCtx); + } + fts3SnippetFree(p); +} + +#endif + +/************** End of fts3_snippet.c ****************************************/ /************** Begin file rtree.c *******************************************/ /* ** 2001 September 15 @@ -107235,27 +106240,25 @@ SQLITE_PRIVATE void sqlite3Fts3SimpleTokenizerModule( ************************************************************************* ** This file contains code for implementations of the r-tree and r*-tree ** algorithms packaged as an SQLite virtual table module. -** -** $Id: rtree.c,v 1.14 2009/08/06 18:36:47 danielk1977 Exp $ */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_RTREE) /* ** This file contains an implementation of a couple of different variants -** of the r-tree algorithm. See the README file for further details. The +** of the r-tree algorithm. See the README file for further details. The ** same data-structure is used for all, but the algorithms for insert and -** delete operations vary. The variants used are selected at compile time +** delete operations vary. The variants used are selected at compile time ** by defining the following symbols: */ -/* Either, both or none of the following may be set to activate +/* Either, both or none of the following may be set to activate ** r*tree variant algorithms. */ #define VARIANT_RSTARTREE_CHOOSESUBTREE 0 #define VARIANT_RSTARTREE_REINSERT 1 -/* +/* ** Exactly one of the following must be set to 1. */ #define VARIANT_GUTTMAN_QUADRATIC_SPLIT 0 @@ -107303,12 +106306,12 @@ typedef union RtreeCoord RtreeCoord; #define RTREE_MAX_DIMENSIONS 5 /* Size of hash table Rtree.aHash. This hash table is not expected to -** ever contain very many entries, so a fixed number of buckets is +** ever contain very many entries, so a fixed number of buckets is ** used. */ #define HASHSIZE 128 -/* +/* ** An rtree virtual-table object. */ struct Rtree { @@ -107319,13 +106322,13 @@ struct Rtree { int nBytesPerCell; /* Bytes consumed per cell */ int iDepth; /* Current depth of the r-tree structure */ char *zDb; /* Name of database containing r-tree table */ - char *zName; /* Name of r-tree table */ - RtreeNode *aHash[HASHSIZE]; /* Hash table of in-memory nodes. */ + char *zName; /* Name of r-tree table */ + RtreeNode *aHash[HASHSIZE]; /* Hash table of in-memory nodes. */ int nBusy; /* Current number of users of this structure */ /* List of nodes removed during a CondenseTree operation. List is ** linked together via the pointer normally used for hash chains - - ** RtreeNode.pNext. RtreeNode.iNode stores the depth of the sub-tree + ** RtreeNode.pNext. RtreeNode.iNode stores the depth of the sub-tree ** headed by the node (leaf nodes have RtreeNode.iNode==0). */ RtreeNode *pDeleted; @@ -107354,7 +106357,7 @@ struct Rtree { #define RTREE_COORD_INT32 1 /* -** The minimum number of cells allowed for a node is a third of the +** The minimum number of cells allowed for a node is a third of the ** maximum. In Gutman's notation: ** ** m = M/3 @@ -107366,7 +106369,7 @@ struct Rtree { #define RTREE_REINSERT(p) RTREE_MINCELLS(p) #define RTREE_MAXCELLS 51 -/* +/* ** An rtree cursor object. */ struct RtreeCursor { @@ -107410,7 +106413,7 @@ struct RtreeConstraint { #define RTREE_GE 0x44 #define RTREE_GT 0x45 -/* +/* ** An rtree structure node. ** ** Data format (RtreeNode.zData): @@ -107419,7 +106422,7 @@ struct RtreeConstraint { ** of the node contain the tree depth as a big-endian integer. ** For non-root nodes, the first 2 bytes are left unused. ** -** 2. The next 2 bytes contain the number of entries currently +** 2. The next 2 bytes contain the number of entries currently ** stored in the node. ** ** 3. The remainder of the node contains the node entries. Each entry @@ -107438,7 +106441,7 @@ struct RtreeNode { }; #define NCELL(pNode) readInt16(&(pNode)->zData[2]) -/* +/* ** Structure to store a deserialized rtree record. */ struct RtreeCell { @@ -107462,22 +106465,22 @@ static int readInt16(u8 *p){ } static void readCoord(u8 *p, RtreeCoord *pCoord){ u32 i = ( - (((u32)p[0]) << 24) + - (((u32)p[1]) << 16) + - (((u32)p[2]) << 8) + + (((u32)p[0]) << 24) + + (((u32)p[1]) << 16) + + (((u32)p[2]) << 8) + (((u32)p[3]) << 0) ); *(u32 *)pCoord = i; } static i64 readInt64(u8 *p){ return ( - (((i64)p[0]) << 56) + - (((i64)p[1]) << 48) + - (((i64)p[2]) << 40) + - (((i64)p[3]) << 32) + - (((i64)p[4]) << 24) + - (((i64)p[5]) << 16) + - (((i64)p[6]) << 8) + + (((i64)p[0]) << 56) + + (((i64)p[1]) << 48) + + (((i64)p[2]) << 40) + + (((i64)p[3]) << 32) + + (((i64)p[4]) << 24) + + (((i64)p[5]) << 16) + + (((i64)p[6]) << 8) + (((i64)p[7]) << 0) ); } @@ -107540,7 +106543,7 @@ static void nodeZero(Rtree *pRtree, RtreeNode *p){ */ static int nodeHash(i64 iNode){ return ( - (iNode>>56) ^ (iNode>>48) ^ (iNode>>40) ^ (iNode>>32) ^ + (iNode>>56) ^ (iNode>>48) ^ (iNode>>40) ^ (iNode>>32) ^ (iNode>>24) ^ (iNode>>16) ^ (iNode>> 8) ^ (iNode>> 0) ) % HASHSIZE; } @@ -107669,9 +106672,9 @@ nodeAcquire( ** Overwrite cell iCell of node pNode with the contents of pCell. */ static void nodeOverwriteCell( - Rtree *pRtree, - RtreeNode *pNode, - RtreeCell *pCell, + Rtree *pRtree, + RtreeNode *pNode, + RtreeCell *pCell, int iCell ){ int ii; @@ -107703,9 +106706,9 @@ static void nodeDeleteCell(Rtree *pRtree, RtreeNode *pNode, int iCell){ */ static int nodeInsertCell( - Rtree *pRtree, - RtreeNode *pNode, - RtreeCell *pCell + Rtree *pRtree, + RtreeNode *pNode, + RtreeCell *pCell ){ int nCell; /* Current number of cells in pNode */ int nMaxCell; /* Maximum number of cells for pNode */ @@ -107782,8 +106785,8 @@ nodeRelease(Rtree *pRtree, RtreeNode *pNode){ ** an internal node, then the 64-bit integer is a child page number. */ static i64 nodeGetRowid( - Rtree *pRtree, - RtreeNode *pNode, + Rtree *pRtree, + RtreeNode *pNode, int iCell ){ assert( iCellzDb, pRtree->zName, + pRtree->zDb, pRtree->zName, pRtree->zDb, pRtree->zName, pRtree->zDb, pRtree->zName ); @@ -107916,7 +106919,7 @@ static int rtreeDestroy(sqlite3_vtab *pVtab){ return rc; } -/* +/* ** Rtree virtual table module xOpen method. */ static int rtreeOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){ @@ -107934,7 +106937,7 @@ static int rtreeOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){ return rc; } -/* +/* ** Rtree virtual table module xClose method. */ static int rtreeClose(sqlite3_vtab_cursor *cur){ @@ -107950,7 +106953,7 @@ static int rtreeClose(sqlite3_vtab_cursor *cur){ /* ** Rtree virtual table module xEof method. ** -** Return non-zero if the cursor does not currently point to a valid +** Return non-zero if the cursor does not currently point to a valid ** record (i.e if the scan has finished), or zero otherwise. */ static int rtreeEof(sqlite3_vtab_cursor *cur){ @@ -107958,10 +106961,10 @@ static int rtreeEof(sqlite3_vtab_cursor *cur){ return (pCsr->pNode==0); } -/* +/* ** Cursor pCursor currently points to a cell in a non-leaf page. ** Return true if the sub-tree headed by the cell is filtered -** (excluded) by the constraints in the pCursor->aConstraint[] +** (excluded) by the constraints in the pCursor->aConstraint[] ** array, or false otherwise. */ static int testRtreeCell(Rtree *pRtree, RtreeCursor *pCursor){ @@ -107975,14 +106978,14 @@ static int testRtreeCell(Rtree *pRtree, RtreeCursor *pCursor){ double cell_min = DCOORD(cell.aCoord[(p->iCoord>>1)*2]); double cell_max = DCOORD(cell.aCoord[(p->iCoord>>1)*2+1]); - assert(p->op==RTREE_LE || p->op==RTREE_LT || p->op==RTREE_GE + assert(p->op==RTREE_LE || p->op==RTREE_LT || p->op==RTREE_GE || p->op==RTREE_GT || p->op==RTREE_EQ ); switch( p->op ){ case RTREE_LE: case RTREE_LT: bRes = p->rValuerValue>cell_max; break; - case RTREE_EQ: + case RTREE_EQ: bRes = (p->rValue>cell_max || p->rValueaConstraint[] array, or false otherwise. ** ** This function assumes that the cell is part of a leaf node. @@ -108007,7 +107010,7 @@ static int testRtreeEntry(Rtree *pRtree, RtreeCursor *pCursor){ RtreeConstraint *p = &pCursor->aConstraint[ii]; double coord = DCOORD(cell.aCoord[p->iCoord]); int res; - assert(p->op==RTREE_LE || p->op==RTREE_LT || p->op==RTREE_GE + assert(p->op==RTREE_LE || p->op==RTREE_LT || p->op==RTREE_GE || p->op==RTREE_GT || p->op==RTREE_EQ ); switch( p->op ){ @@ -108027,12 +107030,12 @@ static int testRtreeEntry(Rtree *pRtree, RtreeCursor *pCursor){ /* ** Cursor pCursor currently points at a node that heads a sub-tree of ** height iHeight (if iHeight==0, then the node is a leaf). Descend -** to point to the left-most cell of the sub-tree that matches the +** to point to the left-most cell of the sub-tree that matches the ** configured constraints. */ static int descendToCell( - Rtree *pRtree, - RtreeCursor *pCursor, + Rtree *pRtree, + RtreeCursor *pCursor, int iHeight, int *pEof /* OUT: Set to true if cannot descend */ ){ @@ -108087,7 +107090,7 @@ static int descendToCell( } /* -** One of the cells in node pNode is guaranteed to have a 64-bit +** One of the cells in node pNode is guaranteed to have a 64-bit ** integer value equal to iRowid. Return the index of this cell. */ static int nodeRowidIndex(Rtree *pRtree, RtreeNode *pNode, i64 iRowid){ @@ -108110,7 +107113,7 @@ static int nodeParentIndex(Rtree *pRtree, RtreeNode *pNode){ return -1; } -/* +/* ** Rtree virtual table module xNext method. */ static int rtreeNext(sqlite3_vtab_cursor *pVtabCursor){ @@ -108148,7 +107151,7 @@ static int rtreeNext(sqlite3_vtab_cursor *pVtabCursor){ return rc; } -/* +/* ** Rtree virtual table module xRowid method. */ static int rtreeRowid(sqlite3_vtab_cursor *pVtabCursor, sqlite_int64 *pRowid){ @@ -108161,7 +107164,7 @@ static int rtreeRowid(sqlite3_vtab_cursor *pVtabCursor, sqlite_int64 *pRowid){ return SQLITE_OK; } -/* +/* ** Rtree virtual table module xColumn method. */ static int rtreeColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){ @@ -108185,8 +107188,8 @@ static int rtreeColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){ return SQLITE_OK; } -/* -** Use nodeAcquire() to obtain the leaf node containing the record with +/* +** Use nodeAcquire() to obtain the leaf node containing the record with ** rowid iRowid. If successful, set *ppLeaf to point to the node and ** return SQLITE_OK. If there is no such record in the table, set ** *ppLeaf to 0 and return SQLITE_OK. If an error occurs, set *ppLeaf @@ -108207,11 +107210,11 @@ static int findLeafNode(Rtree *pRtree, i64 iRowid, RtreeNode **ppLeaf){ } -/* +/* ** Rtree virtual table module xFilter method. */ static int rtreeFilter( - sqlite3_vtab_cursor *pVtabCursor, + sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ @@ -108233,13 +107236,13 @@ static int rtreeFilter( RtreeNode *pLeaf; /* Leaf on which the required cell resides */ i64 iRowid = sqlite3_value_int64(argv[0]); rc = findLeafNode(pRtree, iRowid, &pLeaf); - pCsr->pNode = pLeaf; + pCsr->pNode = pLeaf; if( pLeaf && rc==SQLITE_OK ){ pCsr->iCell = nodeRowidIndex(pRtree, pLeaf, iRowid); } }else{ - /* Normal case - r-tree scan. Set up the RtreeCursor.aConstraint array - ** with the configured constraints. + /* Normal case - r-tree scan. Set up the RtreeCursor.aConstraint array + ** with the configured constraints. */ if( argc>0 ){ pCsr->aConstraint = sqlite3_malloc(sizeof(RtreeConstraint)*argc); @@ -108256,7 +107259,7 @@ static int rtreeFilter( } } } - + if( rc==SQLITE_OK ){ pCsr->pNode = 0; rc = nodeAcquire(pRtree, 1, 0, &pRoot); @@ -108287,7 +107290,7 @@ static int rtreeFilter( /* ** Rtree virtual table module xBestIndex method. There are three -** table scan strategies to choose from (in order from most to +** table scan strategies to choose from (in order from most to ** least desirable): ** ** idxNum idxStr Strategy @@ -108298,8 +107301,8 @@ static int rtreeFilter( ** ------------------------------------------------ ** ** If strategy 1 or 3 is used, then idxStr is not meaningful. If strategy -** 2 is used, idxStr is formatted to contain 2 bytes for each -** constraint used. The first two bytes of idxStr correspond to +** 2 is used, idxStr is formatted to contain 2 bytes for each +** constraint used. The first two bytes of idxStr correspond to ** the constraint in sqlite3_index_info.aConstraintUsage[] with ** (argvIndex==1) etc. ** @@ -108343,10 +107346,10 @@ static int rtreeBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo){ pIdxInfo->aConstraintUsage[jj].omit = 1; /* This strategy involves a two rowid lookups on an B-Tree structures - ** and then a linear search of an R-Tree node. This should be - ** considered almost as quick as a direct rowid lookup (for which + ** and then a linear search of an R-Tree node. This should be + ** considered almost as quick as a direct rowid lookup (for which ** sqlite uses an internal cost of 0.0). - */ + */ pIdxInfo->estimatedCost = 10.0; return SQLITE_OK; } @@ -108459,8 +107462,8 @@ static int cellContains(Rtree *pRtree, RtreeCell *p1, RtreeCell *p2){ for(ii=0; ii<(pRtree->nDim*2); ii+=2){ RtreeCoord *a1 = &p1->aCoord[ii]; RtreeCoord *a2 = &p2->aCoord[ii]; - if( (!isInt && (a2[0].fa1[1].f)) - || ( isInt && (a2[0].ia1[1].i)) + if( (!isInt && (a2[0].fa1[1].f)) + || ( isInt && (a2[0].ia1[1].i)) ){ return 0; } @@ -108482,10 +107485,10 @@ static float cellGrowth(Rtree *pRtree, RtreeCell *p, RtreeCell *pCell){ #if VARIANT_RSTARTREE_CHOOSESUBTREE || VARIANT_RSTARTREE_SPLIT static float cellOverlap( - Rtree *pRtree, - RtreeCell *p, - RtreeCell *aCell, - int nCell, + Rtree *pRtree, + RtreeCell *p, + RtreeCell *aCell, + int nCell, int iExclude ){ int ii; @@ -108517,11 +107520,11 @@ static float cellOverlap( #if VARIANT_RSTARTREE_CHOOSESUBTREE static float cellOverlapEnlargement( - Rtree *pRtree, - RtreeCell *p, - RtreeCell *pInsert, - RtreeCell *aCell, - int nCell, + Rtree *pRtree, + RtreeCell *p, + RtreeCell *pInsert, + RtreeCell *aCell, + int nCell, int iExclude ){ float before; @@ -108595,8 +107598,8 @@ static int ChooseLeaf( overlap = cellOverlapEnlargement(pRtree,&cell,pCell,aCell,nCell,iCell); } #endif - if( (iCell==0) - || (overlap1 ){ @@ -108889,7 +107892,7 @@ static void SortByDistance( /* ** Arguments aIdx, aCell and aSpare all point to arrays of size -** nIdx. The aIdx array contains the set of integers from 0 to +** nIdx. The aIdx array contains the set of integers from 0 to ** (nIdx-1) in no particular order. This function sorts the values ** in aIdx according to dimension iDim of the cells in aCell. The ** minimum value of dimension iDim is considered first, the @@ -108900,10 +107903,10 @@ static void SortByDistance( */ static void SortByDimension( Rtree *pRtree, - int *aIdx, - int nIdx, - int iDim, - RtreeCell *aCell, + int *aIdx, + int nIdx, + int iDim, + RtreeCell *aCell, int *aSpare ){ if( nIdx>1 ){ @@ -109001,8 +108004,8 @@ static int splitNodeStartree( int nLeft; for( - nLeft=RTREE_MINCELLS(pRtree); - nLeft<=(nCell-RTREE_MINCELLS(pRtree)); + nLeft=RTREE_MINCELLS(pRtree); + nLeft<=(nCell-RTREE_MINCELLS(pRtree)); nLeft++ ){ RtreeCell left; @@ -109092,8 +108095,8 @@ static int splitNodeGuttman( for(i=nCell-2; i>0; i--){ RtreeCell *pNext; pNext = PickNext(pRtree, aCell, nCell, pBboxLeft, pBboxRight, aiUsed); - float diff = - cellGrowth(pRtree, pBboxLeft, pNext) - + float diff = + cellGrowth(pRtree, pBboxLeft, pNext) - cellGrowth(pRtree, pBboxRight, pNext) ; if( (RTREE_MINCELLS(pRtree)-NCELL(pRight)==i) @@ -109113,9 +108116,9 @@ static int splitNodeGuttman( #endif static int updateMapping( - Rtree *pRtree, - i64 iRowid, - RtreeNode *pNode, + Rtree *pRtree, + i64 iRowid, + RtreeNode *pNode, int iHeight ){ int (*xSetMapping)(Rtree *, sqlite3_int64, sqlite3_int64); @@ -109151,7 +108154,7 @@ static int SplitNode( RtreeCell leftbbox; RtreeCell rightbbox; - /* Allocate an array and populate it with a copy of pCell and + /* Allocate an array and populate it with a copy of pCell and ** all cells from node pLeft. Then zero the original node. */ aCell = sqlite3_malloc((sizeof(RtreeCell)+sizeof(int))*(nCell+1)); @@ -109287,7 +108290,7 @@ static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){ iCell = nodeParentIndex(pRtree, pNode); pParent = pNode->pParent; pNode->pParent = 0; - if( SQLITE_OK!=(rc = deleteCell(pRtree, pParent, iCell, iHeight+1)) + if( SQLITE_OK!=(rc = deleteCell(pRtree, pParent, iCell, iHeight+1)) || SQLITE_OK!=(rc = nodeRelease(pRtree, pParent)) ){ return rc; @@ -109306,7 +108309,7 @@ static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){ if( SQLITE_OK!=(rc = sqlite3_reset(pRtree->pDeleteParent)) ){ return rc; } - + /* Remove the node from the in-memory hash table and link it into ** the Rtree.pDeleted list. Its contents will be re-inserted later on. */ @@ -109322,7 +108325,7 @@ static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){ static void fixBoundingBox(Rtree *pRtree, RtreeNode *pNode){ RtreeNode *pParent = pNode->pParent; if( pParent ){ - int ii; + int ii; int nCell = NCELL(pNode); RtreeCell box; /* Bounding box for pNode */ nodeGetCell(pRtree, pNode, 0, &box); @@ -109361,7 +108364,7 @@ static int deleteCell(Rtree *pRtree, RtreeNode *pNode, int iCell, int iHeight){ */ if( pNode->iNode!=1 ){ RtreeNode *pParent = pNode->pParent; - if( (pParent->iNode!=1 || NCELL(pParent)!=1) + if( (pParent->iNode!=1 || NCELL(pParent)!=1) && (NCELL(pNode)nDim; iDim++){ - float coord = DCOORD(aCell[ii].aCoord[iDim*2+1]) - + float coord = DCOORD(aCell[ii].aCoord[iDim*2+1]) - DCOORD(aCell[ii].aCoord[iDim*2]); aDistance[ii] += (coord-aCenterCoord[iDim])*(coord-aCenterCoord[iDim]); } @@ -109473,7 +108476,7 @@ static int Reinsert( } /* -** Insert cell pCell into node pNode. Node pNode is the head of a +** Insert cell pCell into node pNode. Node pNode is the head of a ** subtree iHeight high (leaf nodes have iHeight==0). */ static int rtreeInsertCell( @@ -109566,9 +108569,9 @@ static int hashIsEmpty(Rtree *pRtree){ ** The xUpdate method for rtree module virtual tables. */ static int rtreeUpdate( - sqlite3_vtab *pVtab, - int nData, - sqlite3_value **azData, + sqlite3_vtab *pVtab, + int nData, + sqlite3_value **azData, sqlite_int64 *pRowid ){ Rtree *pRtree = (Rtree *)pVtab; @@ -109592,8 +108595,8 @@ static int rtreeUpdate( /* Obtain a reference to the root node to initialise Rtree.iDepth */ rc = nodeAcquire(pRtree, 1, 0, &pRoot); - /* Obtain a reference to the leaf node that contains the entry - ** about to be deleted. + /* Obtain a reference to the leaf node that contains the entry + ** about to be deleted. */ if( rc==SQLITE_OK ){ iDelete = sqlite3_value_int64(azData[0]); @@ -109619,11 +108622,11 @@ static int rtreeUpdate( } /* Check if the root node now has exactly one child. If so, remove - ** it, schedule the contents of the child for reinsertion and + ** it, schedule the contents of the child for reinsertion and ** reduce the tree height by one. ** ** This is equivalent to copying the contents of the child into - ** the root node (the operation that Gutman's paper says to perform + ** the root node (the operation that Gutman's paper says to perform ** in this scenario). */ if( rc==SQLITE_OK && pRtree->iDepth>0 ){ @@ -109734,8 +108737,8 @@ static int rtreeRename(sqlite3_vtab *pVtab, const char *zNewName){ "ALTER TABLE %Q.'%q_node' RENAME TO \"%w_node\";" "ALTER TABLE %Q.'%q_parent' RENAME TO \"%w_parent\";" "ALTER TABLE %Q.'%q_rowid' RENAME TO \"%w_rowid\";" - , pRtree->zDb, pRtree->zName, zNewName - , pRtree->zDb, pRtree->zName, zNewName + , pRtree->zDb, pRtree->zName, zNewName + , pRtree->zDb, pRtree->zName, zNewName , pRtree->zDb, pRtree->zName, zNewName ); if( zSql ){ @@ -109769,10 +108772,10 @@ static sqlite3_module rtreeModule = { }; static int rtreeSqlInit( - Rtree *pRtree, - sqlite3 *db, - const char *zDb, - const char *zPrefix, + Rtree *pRtree, + sqlite3 *db, + const char *zDb, + const char *zPrefix, int isCreate ){ int rc = SQLITE_OK; @@ -109830,7 +108833,7 @@ static int rtreeSqlInit( for(i=0; i*2 coordinates. */ static void rtreenode(sqlite3_context *ctx, int nArg, sqlite3_value **apArg){ @@ -110027,15 +109030,15 @@ static void rtreenode(sqlite3_context *ctx, int nArg, sqlite3_value **apArg){ zText = sqlite3_mprintf("{%s}", zCell); } } - + sqlite3_result_text(ctx, zText, -1, sqlite3_free); } static void rtreedepth(sqlite3_context *ctx, int nArg, sqlite3_value **apArg){ - if( sqlite3_value_type(apArg[0])!=SQLITE_BLOB + if( sqlite3_value_type(apArg[0])!=SQLITE_BLOB || sqlite3_value_bytes(apArg[0])<2 ){ - sqlite3_result_error(ctx, "Invalid argument to rtreedepth()", -1); + sqlite3_result_error(ctx, "Invalid argument to rtreedepth()", -1); }else{ u8 *zBlob = (u8 *)sqlite3_value_blob(apArg[0]); sqlite3_result_int(ctx, readInt16(zBlob)); @@ -110044,7 +109047,7 @@ static void rtreedepth(sqlite3_context *ctx, int nArg, sqlite3_value **apArg){ /* ** Register the r-tree module with database handle db. This creates the -** virtual table module "rtree" and the debugging/analysis scalar +** virtual table module "rtree" and the debugging/analysis scalar ** function "rtreenode". */ SQLITE_PRIVATE int sqlite3RtreeInit(sqlite3 *db){ @@ -110098,9 +109101,9 @@ SQLITE_API int sqlite3_extension_init( ************************************************************************* ** $Id: icu.c,v 1.7 2007/12/13 21:54:11 drh Exp $ ** -** This file implements an integration between the ICU library -** ("International Components for Unicode", an open-source library -** for handling unicode data) and SQLite. The integration uses +** This file implements an integration between the ICU library +** ("International Components for Unicode", an open-source library +** for handling unicode data) and SQLite. The integration uses ** ICU to provide the following to SQLite: ** ** * An implementation of the SQL regexp() function (and hence REGEXP @@ -110111,7 +109114,7 @@ SQLITE_API int sqlite3_extension_init( ** ** * Integration of ICU and SQLite collation seqences. ** -** * An implementation of the LIKE operator that uses ICU to +** * An implementation of the LIKE operator that uses ICU to ** provide case-independent matching. */ @@ -110146,7 +109149,7 @@ static void xFree(void *p){ /* ** Compare two UTF-8 strings for equality where the first string is -** a "LIKE" expression. Return true (1) if they are the same and +** a "LIKE" expression. Return true (1) if they are the same and ** false (0) if they are different. */ static int icuLikeCompare( @@ -110181,7 +109184,7 @@ static int icuLikeCompare( uint8_t c; /* Skip any MATCH_ALL or MATCH_ONE characters that follow a - ** MATCH_ALL. For each MATCH_ONE, skip one character in the + ** MATCH_ALL. For each MATCH_ONE, skip one character in the ** test string. */ while( (c=zPattern[iPattern]) == MATCH_ALL || c == MATCH_ONE ){ @@ -110234,15 +109237,15 @@ static int icuLikeCompare( ** ** A LIKE B ** -** is implemented as like(B, A). If there is an escape character E, +** is implemented as like(B, A). If there is an escape character E, ** ** A LIKE B ESCAPE E ** ** is mapped to like(B, A, E). */ static void icuLikeFunc( - sqlite3_context *context, - int argc, + sqlite3_context *context, + int argc, sqlite3_value **argv ){ const unsigned char *zA = sqlite3_value_text(argv[0]); @@ -110268,7 +109271,7 @@ static void icuLikeFunc( if( zE==0 ) return; U8_NEXT(zE, i, nE, uEsc); if( i!=nE){ - sqlite3_result_error(context, + sqlite3_result_error(context, "ESCAPE expression must be a single character", -1); return; } @@ -110283,7 +109286,7 @@ static void icuLikeFunc( ** This function is called when an ICU function called from within ** the implementation of an SQL scalar function returns an error. ** -** The scalar function context passed as the first argument is +** The scalar function context passed as the first argument is ** loaded with an error message based on the following two args. */ static void icuFunctionError( @@ -110309,7 +109312,7 @@ static void icuRegexpDelete(void *p){ /* ** Implementation of SQLite REGEXP operator. This scalar function takes ** two arguments. The first is a regular expression pattern to compile -** the second is a string to match against that pattern. If either +** the second is a string to match against that pattern. If either ** argument is an SQL NULL, then NULL Is returned. Otherwise, the result ** is 1 if the string matches the pattern, or 0 otherwise. ** @@ -110331,8 +109334,8 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){ UBool res; const UChar *zString = sqlite3_value_text16(apArg[1]); - /* If the left hand side of the regexp operator is NULL, - ** then the result is also NULL. + /* If the left hand side of the regexp operator is NULL, + ** then the result is also NULL. */ if( !zString ){ return; @@ -110370,7 +109373,7 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){ } /* Set the text that the regular expression operates on to a NULL - ** pointer. This is not really necessary, but it is tidier than + ** pointer. This is not really necessary, but it is tidier than ** leaving the regular expression object configured with an invalid ** pointer after this function returns. */ @@ -110381,7 +109384,7 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){ } /* -** Implementations of scalar functions for case mapping - upper() and +** Implementations of scalar functions for case mapping - upper() and ** lower(). Function upper() converts its input to upper-case (ABC). ** Function lower() converts to lower-case (abc). ** @@ -110389,7 +109392,7 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){ ** "language specific". Refer to ICU documentation for the differences ** between the two. ** -** To utilise "general" case mapping, the upper() or lower() scalar +** To utilise "general" case mapping, the upper() or lower() scalar ** functions are invoked with one argument: ** ** upper('ABC') -> 'abc' @@ -110402,7 +109405,7 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){ ** of upper() or lower(). ** ** lower('I', 'en_us') -> 'i' -** lower('I', 'tr_tr') -> 'ı' (small dotless i) +** lower('I', 'tr_tr') -> 'Ä+' (small dotless i) ** ** http://www.icu-project.org/userguide/posix.html#case_mappings */ @@ -110481,7 +109484,7 @@ static int icuCollationColl( /* ** Implementation of the scalar function icu_load_collation(). ** -** This scalar function is used to add ICU collation based collation +** This scalar function is used to add ICU collation based collation ** types to an SQLite database connection. It is intended to be called ** as follows: ** @@ -110492,8 +109495,8 @@ static int icuCollationColl( ** collation sequence to create. */ static void icuLoadCollation( - sqlite3_context *p, - int nArg, + sqlite3_context *p, + int nArg, sqlite3_value **apArg ){ sqlite3 *db = (sqlite3 *)sqlite3_user_data(p); @@ -110518,7 +109521,7 @@ static void icuLoadCollation( } assert(p); - rc = sqlite3_create_collation_v2(db, zName, SQLITE_UTF16, (void *)pUCollator, + rc = sqlite3_create_collation_v2(db, zName, SQLITE_UTF16, (void *)pUCollator, icuCollationColl, icuCollationDel ); if( rc!=SQLITE_OK ){ @@ -110571,7 +109574,7 @@ SQLITE_PRIVATE int sqlite3IcuInit(sqlite3 *db){ #if !SQLITE_CORE SQLITE_API int sqlite3_extension_init( - sqlite3 *db, + sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ @@ -110596,7 +109599,7 @@ SQLITE_API int sqlite3_extension_init( ** ************************************************************************* ** This file implements a tokenizer for fts3 based on the ICU library. -** +** ** $Id: fts3_icu.c,v 1.3 2008/09/01 18:34:20 danielk1977 Exp $ */ @@ -110671,7 +109674,7 @@ static int icuDestroy(sqlite3_tokenizer *pTokenizer){ /* ** Prepare to begin tokenizing a particular string. The input ** string to be tokenized is pInput[0..nBytes-1]. A cursor -** used to incrementally tokenize this string is returned in +** used to incrementally tokenize this string is returned in ** *ppCursor. */ static int icuOpen( @@ -110710,7 +109713,7 @@ static int icuOpen( pCsr->aOffset = (int *)&pCsr->aChar[nChar]; pCsr->aOffset[iOut] = iInput; - U8_NEXT(zInput, iInput, nInput, c); + U8_NEXT(zInput, iInput, nInput, c); while( c>0 ){ int isError = 0; c = u_foldCase(c, opt); diff --git a/harbour/external/sqlite3/sqlite3.h b/harbour/external/sqlite3/sqlite3.h index fc015b08e1..e614a604e4 100644 --- a/harbour/external/sqlite3/sqlite3.h +++ b/harbour/external/sqlite3/sqlite3.h @@ -119,9 +119,9 @@ extern "C" { ** ** Requirements: [H10011] [H10014] */ -#define SQLITE_VERSION "3.6.20" -#define SQLITE_VERSION_NUMBER 3006020 -#define SQLITE_SOURCE_ID "2009-11-04 13:30:02 eb7a544fe49d1626bacecfe53ddc03fe082e3243" +#define SQLITE_VERSION "3.6.21" +#define SQLITE_VERSION_NUMBER 3006021 +#define SQLITE_SOURCE_ID "2009-12-07 16:39:13 1ed88e9d01e9eda5cbc622e7614277f29bcc551c" /* ** CAPI3REF: Run-Time Library Version Numbers {H10020} @@ -825,7 +825,7 @@ struct sqlite3_vfs { ** The sqlite3_initialize() routine initializes the ** SQLite library. The sqlite3_shutdown() routine ** deallocates any resources that were allocated by sqlite3_initialize(). -** This routines are designed to aid in process initialization and +** These routines are designed to aid in process initialization and ** shutdown on embedded systems. Workstation applications using ** SQLite normally do not need to invoke either of these routines. ** @@ -1295,6 +1295,9 @@ SQLITE_API int sqlite3_extended_result_codes(sqlite3*, int onoff); ** For the purposes of this routine, an [INSERT] is considered to ** be successful even if it is subsequently rolled back. ** +** This function is accessible to SQL statements via the +** [last_insert_rowid() SQL function]. +** ** Requirements: ** [H12221] [H12223] ** @@ -1352,8 +1355,8 @@ SQLITE_API sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*); ** However, the number returned does not include changes ** caused by subtriggers since those have their own context. ** -** See also the [sqlite3_total_changes()] interface and the -** [count_changes pragma]. +** See also the [sqlite3_total_changes()] interface, the +** [count_changes pragma], and the [changes() SQL function]. ** ** Requirements: ** [H12241] [H12243] @@ -1380,8 +1383,8 @@ SQLITE_API int sqlite3_changes(sqlite3*); ** completed (when the statement handle is passed to [sqlite3_reset()] or ** [sqlite3_finalize()]). ** -** See also the [sqlite3_changes()] interface and the -** [count_changes pragma]. +** See also the [sqlite3_changes()] interface, the +** [count_changes pragma], and the [total_changes() SQL function]. ** ** Requirements: ** [H12261] [H12263] @@ -2395,7 +2398,7 @@ SQLITE_API int sqlite3_limit(sqlite3*, int id, int newVal); ** In the "v2" interfaces, the prepared statement ** that is returned (the [sqlite3_stmt] object) contains a copy of the ** original SQL text. This causes the [sqlite3_step()] interface to -** behave a differently in three ways: +** behave differently in three ways: ** **
      **
    1. @@ -4142,6 +4145,8 @@ SQLITE_API int sqlite3_table_column_metadata( ** {H12606} Extension loading must be enabled using ** [sqlite3_enable_load_extension()] prior to calling this API, ** otherwise an error will be returned. +** +** See also the [load_extension() SQL function]. */ SQLITE_API int sqlite3_load_extension( sqlite3 *db, /* Load the extension into this database connection */