In this chapter
Four query-shaping patterns that silently hide an existing index — a function wrapped around an indexed column, a query needing a column the index doesn't cover, and LIKE's surprising case-sensitivity gap next to GLOB — each verified live against the exact same table.
The Problem in Real Life
Sarah adds an index, checks EXPLAIN QUERY PLAN, and stares at the result. Still a SCAN. She adds another index. Still a SCAN.
"The index is right there," she mutters. "Why won't it use it?"
Mike watches her scroll through the query itself, and slowly it clicks for her — the index isn't the problem. The query is shaped in a way that hides it from the planner entirely.
The index exists. I just wrote the query in a way that made it invisible.
Sarah
A Query That Hides Its Index vs. One That Doesn't
An index existing isn't the same as it being used
The query has to expose the indexed column in a form the planner can recognize — otherwise the index sits there unused.
Functions on a column hide it from the index
strftime(OrderDate), UPPER(Name), or any other wrapping function breaks the direct match an index needs.
A covering index answers the whole query alone
Include every column a specific query needs, not just the filtered one, and SQLite never has to revisit the table.
LIKE and GLOB aren't interchangeable near an index
GLOB's guaranteed case-sensitivity makes it index-eligible in cases where LIKE's default case-insensitivity forces a full scan.
Writing Queries the Planner Can Actually Use
An index only helps if the query is written in a way the planner can actually match against it. Query optimization here means reshaping a query — not the schema, not the indexes — so the planner can see and use the shortcuts that already exist.
A few specific patterns come up again and again: wrapping an indexed column in a function, selecting columns an index doesn't cover, and pattern-matching in a way that breaks the index's sort order. Each one silently turns a SEARCH back into a SCAN, with no error, no warning — just a slower query.
OrderDate has an index, but this query doesn't filter on OrderDate directly — it filters on the result of calling strftime() on it. SQLite would have to run that function on every single row just to check, so it can't use the index's sort order at all.
EXPLAIN QUERY PLANSELECT * FROM AllOrders WHERE strftime('%Y', OrderDate) = '2020';
SCAN AllOrders — the index sits there unused.
Same intent — "orders from 2020" — but expressed as a direct range on the actual column. Nothing hides the column from the planner now.
EXPLAIN QUERY PLANSELECT * FROM AllOrders WHERE OrderDate >= '2020-01-01' AND OrderDate < '2021-01-01';
SEARCH AllOrders USING INDEX idx_allorders_orderdate (OrderDate>? AND OrderDate<?) — the exact same question, answered through the index.
The index on OrderDate finds the right rows, but this query also needs Status — a column the index doesn't include. SQLite still has to visit the actual table row for that piece, on top of the index lookup.
SELECT OrderDate, Status FROM AllOrdersWHERE OrderDate >= '2020-06-01' AND OrderDate < '2020-07-01';
SEARCH AllOrders USING INDEX idx_allorders_orderdate (OrderDate>? AND OrderDate<?) — a real index search, but not the fastest version of one.
A covering index includes every column a specific query needs, not just the one it filters on. Once OrderDate and Status both live in the index, SQLite can answer the whole query from the index alone.
CREATE INDEX idx_allorders_covering ON AllOrders (OrderDate, Status);EXPLAIN QUERY PLANSELECT OrderDate, Status FROM AllOrdersWHERE OrderDate >= '2020-06-01' AND OrderDate < '2020-07-01';
SEARCH AllOrders USING COVERING INDEX idx_allorders_covering — no trip back to the table at all.
A leading wildcard (%elivered) can never use a sorted index — there's no fixed starting point to jump to. A trailing wildcard (Deliv%) looks like it should work the same way a range does — but SQLite's LIKE is case-insensitive by default, and a case-insensitive match can't be proven safe against a case-sensitive sorted index, so it falls back to a scan too.
CREATE INDEX idx_allorders_status ON AllOrders (Status);EXPLAIN QUERY PLANSELECT * FROM AllOrders WHERE Status LIKE '%elivered';EXPLAIN QUERY PLANSELECT * FROM AllOrders WHERE Status LIKE 'Deliv%';
Both queries return SCAN AllOrders — for two genuinely different reasons.
GLOB uses the same kind of prefix pattern as LIKE, but it's always case-sensitive — so SQLite can safely treat it as a range against the sorted index, with no ambiguity to rule out.
EXPLAIN QUERY PLANSELECT * FROM AllOrders WHERE Status GLOB 'Deliv*';
SEARCH AllOrders USING INDEX idx_allorders_status (Status>? AND Status<?) — the identical prefix pattern, but through GLOB instead of LIKE, actually reaches the index.
None of these fixes touched the schema or added a new index type — they reshaped the query so the planner could actually see the shortcut that already existed. That's the whole discipline of query optimization: writing conditions in the plain, direct form the planner is built to recognize.
It's also a humbling one. "The index exists" and "the query can use it" are two different claims, and the gap between them is exactly where slow queries hide in real systems.
Key Takeaway
An index only helps if the query exposes the column in a form the planner recognizes — wrapping it in a function, needing a column outside the index, or using a pattern the index can't prove safe all silently turn a fast SEARCH back into a full SCAN.
Why This Matters
Most real-world 'the index isn't working' complaints aren't missing-index problems at all — they're queries shaped in a way that hides an index that's already there. Recognizing these patterns is often a five-minute fix instead of a schema change.
GreenMart's reports now use the indexes that already exist, just written in a form the planner can actually match. Last stop in this Act: what's really happening underneath all of this, at the storage engine level.
