Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Mateusz Herych - LIKE '%smth%' is not the way
Search
Base Lab
February 12, 2014
Programming
0
130
Mateusz Herych - LIKE '%smth%' is not the way
Droidcon IT, Turin Feb 2014
Base Lab
February 12, 2014
Tweet
Share
More Decks by Base Lab
See All by Base Lab
Szymon Sobczak - Hadoop + Storm
baselab
0
90
Slawek Skowron - Monitoring @ Scale
baselab
0
110
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
94
Tomasz Nowak - Web Application Testing made easy
baselab
0
280
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Jerzy Chałupski - Offline mode in Android apps
baselab
3
460
Jerzy Chałupski - Data model on Android
baselab
4
200
Other Decks in Programming
See All in Programming
C# 大統一理論推進委員会 会員のための Unity Package Manager プロジェクト構成案
monry
PRO
0
580
とにかくHTTP3をライトニングに話す / Anyway, I'll talk to Lightning about HTTP3.
seike460
PRO
0
120
GitHub Copilot Tips and Tricks
yuichielectric
26
7.4k
IntelliJ IDEA を知らなかった 自分に教えたい小ネタ集 / IntelliJ IDEA Hints for My Past Self
mackey0225
3
160
Understanding Ast By Looking
inouehi
0
120
PHPでOfficeファイルを取り扱う! PHP Officeライブラリを プロダクトに組み込んだ話
hirobe1999
0
840
生成 AI の中身を覗いてみよう〜基礎から医療現場での応用まで〜
soh9834
2
760
イベントストーミングによるオブジェクトモデリング・オブジェクト指向プログラミングの適用・開発プロセスの変遷・アーキテクチャの変革 / Object modeling with Event Storming.
nrslib
12
2.9k
設計の知識と技能で駆動するソフトウェア開発
masuda220
PRO
18
10k
Open Source Swiftc Workshop
kitasuke
1
180
もうすぐ新年度、Babylon.jsがお勧めな3個の理由
hideg
0
160
Kotlinを用いたDSL的な設計手法と使用上の注意
kohii00
3
530
Featured
See All Featured
What's new in Ruby 2.0
geeforr
335
31k
Building Better People: How to give real-time feedback that sticks.
wjessup
350
18k
Building Your Own Lightsaber
phodgson
97
5.6k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
113
18k
Why Our Code Smells
bkeepers
PRO
330
56k
Docker and Python
trallard
33
2.6k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
657
120k
Build your cross-platform service in a week with App Engine
jlugia
223
17k
RailsConf 2023
tenderlove
0
510
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
28
5.9k
Web Components: a chance to create the future
zenorocha
304
41k
Stop Working from a Prison Cell
hatefulcrawdad
265
19k
Transcript
None
Mateusz Herych Android Developer - Base CRM Co-organizer - GDG
Krakow Co-organizer - KrakDroid
Stats
LIKE ‘%smth%’
LIKE ‘%smth%’ is not the way.
Search
Search Offline.
Why?
Why? Let the backend guys do the job
Why? Internet is not everywhere.
Why? Internet is not everywhere. It takes time. (especially SSL)
Why? Internet is not everywhere. It takes time. (especially SSL)
And sometimes it’s shitty.
Why? Internet is not everywhere. It takes time. (especially SSL)
And sometimes it’s shitty.
Sure, some apps don’ t really need it You need
an Internet to order that taxi anyway
Do you keep offline content? Let your users navigate fast.
Did I say fast?
How? Let’s go deeper.
Context
CRM - Contacts - Deals - Notes - ...
CRM - Contacts (~100) - Deals (~50) - Notes (~100)
- ... 2009
select id from deals where name LIKE ‘% something%’
CRM - Contacts (~40K) - Deals (~20K) - Notes (~300K)
- ...
None
HOW DOES “LIKE” WORKS LIKE?
Docs saying
I tried to put all the conditions that need to
be satisfied so SQLite can use indices combined with LIKE operator. Docs saying
They didn’t fit. Docs saying
http://www.sqlite. org/optoverview.html Docs saying
Hey, you, SQLite! EXPLAIN (my) QUERY PLAN
PRAGMA case_sensitive_like=1;
PRAGMA case_sensitive_like=1; CREATE INDEX search_index on deals(name);
PRAGMA case_sensitive_like=1; CREATE INDEX search_index on deals(name); SELECT id FROM
deals WHERE name LIKE ‘Some%’;
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘Some%’; SEARCH TABLE deals USING COVERING INDEX search_index (name>? AND name<?) (~31250 rows)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘%Some%’;
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘%Some%’; SCAN TABLE deals (~500000 rows)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE
‘%Some%’; SCAN TABLE deals (~500000 rows) (And then you die)
first_name || ‘ ‘ || last_name? UNIONs, complicated VIEWs? Like
is NOT the way to go.
What people think SQLite is
What SQLite really is
SQLite is powerful Not kidding.
FTS3 Full Text Search
CREATE VIRTUAL TABLE search USING fts3 (tokens)
? CREATE VIRTUAL TABLE search USING fts3 (tokens INT)
Nope. PRAGMA table_info(search); cid|name|type|notnull|dflt_value|pk 0|word||0||0
All is TEXT, except for hidden rowid.
What is virtual table? Imagine it’s a Java interface. interface
VirtualTable { void insert(Params p); void update(Params p); // etc, also createTable. }
What is a virtual table? class Fts3 implements VirtualTable {
// … }
None
MATCH Let’s go make some magic.
SELECT * FROM search WHERE content MATCH ‘something’
SELECT rowid, * FROM search WHERE content MATCH ‘something’ rowid|word
1|something 2|not something special 3|SoMeThInG
SELECT rowid, * FROM search WHERE content MATCH ‘some* spe*’
rowid|word 2|not something special
CREATE VIRTUAL TABLE search USING fts3 (author, lyrics)
SELECT * FROM search WHERE lyrics MATCH ‘author:Giorgio Synthesizer author
|lyrics Giorgio Moroder|..Why don’t I use a synthesizer...
Cool?
Cool? Look at this.
SELECT * FROM search WHERE lyrics MATCH ‘why NEAR synthesizer’
author |lyrics Giorgio Moroder|..Why don’t I use synthesizer...
SELECT * FROM search WHERE lyrics MATCH ‘why NEAR/3 synthesizer’
author |lyrics Giorgio Moroder|..Why don’t I use synthesizer...
Tips.
1. Your FTS vtable should contain only tokens. Eventually divided
into sections.
2. Link your FTS table’s records with other table (containing
real object’s id and type) using rowid.
3. Remember. FTS is fast enough for searching purposes. But
it’s always slower than ‘=’ based query on indexed field.
4. EXPLAIN QUERY PLAN doesn’t work for fts tables. Try
to measure it with .timer ON.
5. ???
6. QUESTIONS TIME!