Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Mateusz Herych - LIKE '%smth%' is not the way
Search
Base Lab
February 12, 2014
Programming
0
140
Mateusz Herych - LIKE '%smth%' is not the way
Droidcon IT, Turin Feb 2014
Base Lab
February 12, 2014
Tweet
Share
More Decks by Base Lab
See All by Base Lab
Szymon Sobczak - Hadoop + Storm
baselab
0
100
Slawek Skowron - Monitoring @ Scale
baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
110
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Jerzy Chałupski - Offline mode in Android apps
baselab
3
480
Jerzy Chałupski - Data model on Android
baselab
4
230
Other Decks in Programming
See All in Programming
Namespace and Its Future
tagomoris
6
540
【第4回】関東Kaggler会「Kaggleは執筆に役立つ」
mipypf
0
880
MCPで実現するAIエージェント駆動のNext.jsアプリデバッグ手法
nyatinte
7
910
AIコーディングAgentとの向き合い方
eycjur
0
240
コーディングは技術者(エンジニア)の嗜みでして / Learning the System Development Mindset from Rock Lady
mackey0225
2
580
State of CSS 2025
benjaminkott
1
120
The State of Fluid (2025)
s2b
0
200
AI OCR API on Lambdaを Datadogで可視化してみた
nealle
0
180
デザインシステムが必須の時代に
yosuke_furukawa
PRO
2
110
フロントエンドのmonorepo化と責務分離のリアーキテクト
kajitack
2
140
一人でAIプロダクトを作るための工夫 〜技術選定・開発プロセス編〜 / I want AI to work harder
rkaga
13
2.8k
Infer入門
riru
4
1.6k
Featured
See All Featured
Side Projects
sachag
455
43k
Gamification - CAS2011
davidbonilla
81
5.4k
A Tale of Four Properties
chriscoyier
160
23k
Typedesign – Prime Four
hannesfritz
42
2.8k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
How to Ace a Technical Interview
jacobian
279
23k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
A better future with KSS
kneath
239
17k
4 Signs Your Business is Dying
shpigford
184
22k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.8k
Site-Speed That Sticks
csswizardry
10
790
Rails Girls Zürich Keynote
gr2m
95
14k
Transcript
None
Mateusz Herych Android Developer - Base CRM Co-organizer - GDG
Krakow Co-organizer - KrakDroid
Stats
LIKE ‘%smth%’
LIKE ‘%smth%’ is not the way.
Search
Search Offline.
Why?
Why? Let the backend guys do the job
Why? Internet is not everywhere.
Why? Internet is not everywhere. It takes time. (especially SSL)
Why? Internet is not everywhere. It takes time. (especially SSL)
And sometimes it’s shitty.
Why? Internet is not everywhere. It takes time. (especially SSL)
And sometimes it’s shitty.
Sure, some apps don’ t really need it You need
an Internet to order that taxi anyway
Do you keep offline content? Let your users navigate fast.
Did I say fast?
How? Let’s go deeper.
Context
CRM - Contacts - Deals - Notes - ...
CRM - Contacts (~100) - Deals (~50) - Notes (~100)
- ... 2009
select id from deals where name LIKE ‘% something%’
CRM - Contacts (~40K) - Deals (~20K) - Notes (~300K)
- ...
None
HOW DOES “LIKE” WORKS LIKE?
Docs saying
I tried to put all the conditions that need to
be satisfied so SQLite can use indices combined with LIKE operator. Docs saying
They didn’t fit. Docs saying
http://www.sqlite. org/optoverview.html Docs saying
Hey, you, SQLite! EXPLAIN (my) QUERY PLAN
PRAGMA case_sensitive_like=1;
PRAGMA case_sensitive_like=1; CREATE INDEX search_index on deals(name);
PRAGMA case_sensitive_like=1; CREATE INDEX search_index on deals(name); SELECT id FROM
deals WHERE name LIKE ‘Some%’;
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘Some%’; SEARCH TABLE deals USING COVERING INDEX search_index (name>? AND name<?) (~31250 rows)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘%Some%’;
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘%Some%’; SCAN TABLE deals (~500000 rows)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘%Some%’; SCAN TABLE deals (~500000 rows) (And then you die)
first_name || ‘ ‘ || last_name? UNIONs, complicated VIEWs? Like
is NOT the way to go.
What people think SQLite is
What SQLite really is
SQLite is powerful Not kidding.
FTS3 Full Text Search
CREATE VIRTUAL TABLE search USING fts3 (tokens)
? CREATE VIRTUAL TABLE search USING fts3 (tokens INT)
Nope. PRAGMA table_info(search); cid|name|type|notnull|dflt_value|pk 0|word||0||0
All is TEXT, except for hidden rowid.
What is virtual table? Imagine it’s a Java interface. interface
VirtualTable { void insert(Params p); void update(Params p); // etc, also createTable. }
What is a virtual table? class Fts3 implements VirtualTable {
// … }
None
MATCH Let’s go make some magic.
SELECT * FROM search WHERE content MATCH ‘something’
SELECT rowid, * FROM search WHERE content MATCH ‘something’ rowid|word
1|something 2|not something special 3|SoMeThInG
SELECT rowid, * FROM search WHERE content MATCH ‘some* spe*’
rowid|word 2|not something special
CREATE VIRTUAL TABLE search USING fts3 (author, lyrics)
SELECT * FROM search WHERE lyrics MATCH ‘author:Giorgio Synthesizer author
|lyrics Giorgio Moroder|..Why don’t I use a synthesizer...
Cool?
Cool? Look at this.
SELECT * FROM search WHERE lyrics MATCH ‘why NEAR synthesizer’
author |lyrics Giorgio Moroder|..Why don’t I use synthesizer...
SELECT * FROM search WHERE lyrics MATCH ‘why NEAR/3 synthesizer’
author |lyrics Giorgio Moroder|..Why don’t I use synthesizer...
Tips.
1. Your FTS vtable should contain only tokens. Eventually divided
into sections.
2. Link your FTS table’s records with other table (containing
real object’s id and type) using rowid.
3. Remember. FTS is fast enough for searching purposes. But
it’s always slower than ‘=’ based query on indexed field.
4. EXPLAIN QUERY PLAN doesn’t work for fts tables. Try
to measure it with .timer ON.
5. ???
6. QUESTIONS TIME!