Inside the room- DC SF edition

Transcript

1. Inside the room Effie Barak (@CodingChick)
3. Talk Outline: 1. Intro 2. Threading in Room 3. Triggers 4. Transactions 5. Large queries
4. Intro to SQLite
5. Embedded SQL database engine
11. Intro to Android and Android x APIs
12. Android package android.database.sqlite SQLiteStatement SQLiteCursor SQLiteDatabase SQLiteOpenHelper SQLiteQuery
13. Android X androidx.sqlite.db SupportSQLiteDatabase SupportSQLiteOpenHelper SupportSQLiteQuery
14. /** * A database abstraction which removes the framework dependency and allows swapping underlying * sql versions. It mimics the behavior of {@link android.database.sqlite.SQLiteDatabase} */ public interface SupportSQLiteDatabase extends Closeable {
15. Android X androidx.sqlite.db.framework FrameworkSQLiteDatabase FrameworkSQLiteOpenHelper FrameworkSQLiteStatement
16. /** * Delegates all calls to an implementation of {@link SQLiteDatabase}. */ class FrameworkSQLiteDatabase implements SupportSQLiteDatabase {
17. private final SQLiteDatabase mDelegate; @Override public void beginTransaction() { mDelegate.beginTransaction(); } @Override public void beginTransactionNonExclusive() { mDelegate.beginTransactionNonExclusive(); }
18. Intro to Room Architecture
19. Isolated modules 4 room-common 4 room-compiler 4 room-runtime 4 room-ktx 4 room-rxjava2
20. Threads, processes etc
21. Locking in SQLite
23. Types of locks
24. Types of locks 4 Unlocked
25. Types of locks 4 Unlocked 4 Shared
26. Types of locks 4 Unlocked 4 Shared 4 Reserved
27. Types of locks 4 Unlocked 4 Shared 4 Reserved 4 Pending
28. Types of locks 4 Unlocked 4 Shared 4 Reserved 4 Pending 4 Exclusive
29. Reading from the database
30. Writing to the database 1. Get share lock 2. Get reserved lock 3. Write to memory 4. Write to rollback journal
31. The rollback journal
32. Writing to the database 1. Obtain a PENDING lock 2. Obtain an EXCLUSIVE lock 3. Wait for SHARED locks to be done 4. Write to database
33. Write-Ahead Log (WAL)
34. WAL continued
35. Writes with WAL 1. Append to WAL 2. COMMIT 3. Continue writing 4. Checkpoint- write to file
36. Reads with WAL 1. Check WAL 2. Check DB
37. Database connections
39. SQLite connections
40. Android APIs
41. Only one write connection and multiple reads
42. Get a connection to SQLite
43. package android.database.sqlite; public abstract class SQLiteOpenHelper { public SQLiteDatabase getWritableDatabase() { synchronized (this) { return getDatabaseLocked(true); } } public SQLiteDatabase getReadableDatabase() { synchronized (this) { return getDatabaseLocked(false); } } }
44. public final class SQLiteDatabase extends SQLiteClosable { // The connection pool for the database, null when closed. // The pool itself is thread-safe, but the reference to it can only be acquired // when the lock is held. // INVARIANT: Guarded by mLock. private SQLiteConnectionPool mConnectionPoolLocked; }
45. Change SQLite settings public final class SQLiteDatabase extends SQLiteClosable { public boolean enableWriteAheadLogging() { ... mConfigurationLocked.openFlags |= ENABLE_WRITE_AHEAD_LOGGING; try { mConnectionPoolLocked.reconfigure(mConfigurationLocked); } catch (RuntimeException ex) { mConfigurationLocked.openFlags &= ~ENABLE_WRITE_AHEAD_LOGGING; throw ex; } } }
46. public final class SQLiteConnection ... { private void setWalModeFromConfiguration() { ... setJournalMode("WAL"); ... } } }
47. private void setJournalMode(String newValue) { String value = executeForString("PRAGMA journal_mode", null, null); if (!value.equalsIgnoreCase(newValue)) { try { String result = executeForString("PRAGMA journal_mode=" + newValue, null, null); if (result.equalsIgnoreCase(newValue)) { return; } ... }
48. Where Room sets up WAL public abstract class RoomDatabase { public void init(...) { boolean wal = false; if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN) { wal = configuration.journalMode == JournalMode.WRITE_AHEAD_LOGGING; mOpenHelper.setWriteAheadLoggingEnabled(wal); } ... } }
49. Room with threads
50. Queries (reads) in Room
51. Writes in Room?
52. Transactions
53. Transactions in SQLite
54. SQLite BEGIN TRANSACTION
55. SQLite COMMIT
56. Transaction on journal vs WAL
57. Transactions / writes with Android API
58. Room wraps a lot of things in transactions
59. Multiple write statements - same transaction val preparedQuery = prepareQuery("INSERT...") beginTransaction input.forEach { bind(preparedQuery, it) } endTransaction
60. Triggers
61. Triggers in SQLite
62. Triggers for RxJava and LiveData class RoomTrackingLiveData<T> extends LiveData<T> { final Runnable mInvalidationRunnable = new Runnable() { ... public void run() { boolean isActive = hasActiveObservers(); if (mInvalid.compareAndSet(false, true)) { if (isActive) { mDatabase.getQueryExecutor().execute(mRefreshRunnable); } } } }; }
63. class RoomTrackingLiveData<T> extends LiveData<T> { final Runnable mRefreshRunnable = new Runnable() { public void run() { ... mDatabase.getInvalidationTracker().addWeakObserver(mObserver); }
64. Invalidation Tracking public class InvalidationTracker { static class ObservedTableTracker { static final int NO_OP = 0; // don't change trigger state for this table static final int ADD = 1; // add triggers for this table static final int REMOVE = 2; // remove triggers for this table
65. public class InvalidationTracker { void syncTriggers(SupportSQLiteDatabase database) { try { for (int tableId = 0; tableId < limit; tableId++) { switch (tablesToSync[tableId]) { case ObservedTableTracker.ADD: startTrackingTable(database, tableId); break; case ObservedTableTracker.REMOVE: stopTrackingTable(database, tableId); break; } } }
66. private void startTrackingTable(SupportSQLiteDatabase writableDb, int tableId) { final String tableName = mTableNames[tableId]; StringBuilder stringBuilder = new StringBuilder(); for (String trigger : TRIGGERS) { stringBuilder.setLength(0); stringBuilder.append("CREATE TEMP TRIGGER IF NOT EXISTS "); appendTriggerName(stringBuilder, tableName, trigger); stringBuilder.append(" AFTER ") .append(trigger) .append(" ON `") .append(tableName) .append("` BEGIN INSERT OR REPLACE INTO ") .append(UPDATE_TABLE_NAME) .append(" VALUES(null, ") .append(tableId) .append("); END"); writableDb.execSQL(stringBuilder.toString()); } }
67. private Set<Integer> checkUpdatedTable() { HashSet<Integer> invalidatedTableIds = new HashSet<>(); Cursor cursor = mDatabase.query(new SimpleSQLiteQuery(SELECT_UPDATED_TABLES_SQL)); //noinspection TryFinallyCanBeTryWithResources try { while (cursor.moveToNext()) { final int tableId = cursor.getInt(0); invalidatedTableIds.add(tableId); } } finally { cursor.close(); } if (!invalidatedTableIds.isEmpty()) { mCleanupStatement.executeUpdateDelete(); } return invalidatedTableIds; }
68. public void endTransaction() { mInvalidationTracker.refreshVersionsAsync(); }
69. AFTER INSERT/UPDATE/DELETE ON <table> BEGIN UPDATE SET invalidated = 1 WHERE tableId = <id> AND invalidated = 0; END
70. Transactions with Room
71. TransactionExecutor
72. class TransactionExecutor implements Executor { private final Executor mExecutor; private final ArrayDeque<Runnable> mTasks = new ArrayDeque<>(); private Runnable mActive; TransactionExecutor(@NonNull Executor executor) { mExecutor = executor; } public synchronized void execute(final Runnable command) { mTasks.offer(new Runnable() { public void run() { try { command.run(); } finally { scheduleNext(); } } }); if (mActive == null) { scheduleNext(); } } @SuppressWarnings("WeakerAccess") synchronized void scheduleNext() { if ((mActive = mTasks.poll()) != null) { mExecutor.execute(mActive); } } }
73. In RxJava public class RxRoom { @RestrictTo(RestrictTo.Scope.LIBRARY_GROUP_PREFIX) public static <T> Observable<T> createObservable(final RoomDatabase database, final boolean inTransaction, final String[] tableNames, final Callable<T> callable) { Scheduler scheduler = Schedulers.from(getExecutor(database, inTransaction)); final Maybe<T> maybe = Maybe.fromCallable(callable); return createObservable(database, tableNames) .subscribeOn(scheduler) .unsubscribeOn(scheduler) .observeOn(scheduler) .flatMapMaybe(new Function<Object, MaybeSource<T>>() { @Override public MaybeSource<T> apply(Object o) throws Exception { return maybe; } }); } private static Executor getExecutor(RoomDatabase database, boolean inTransaction) { if (inTransaction) { return database.getTransactionExecutor(); } else { return database.getQueryExecutor(); } } }
74. In Coroutines class CoroutinesRoom private constructor() { companion object { @JvmStatic suspend fun <R> execute( db: RoomDatabase, inTransaction: Boolean, callable: Callable<R> ): R { if (db.isOpen && db.inTransaction()) { return callable.call() } // Use the transaction dispatcher if we are on a transaction coroutine, otherwise // use the database dispatchers. val context = coroutineContext[TransactionElement]?.transactionDispatcher ?: if (inTransaction) db.transactionDispatcher else db.queryDispatcher return withContext(context) { callable.call() } } } }
75. Transaction APIs in Room public abstract class RoomDatabase { @Deprecated public void beginTransaction() { assertNotMainThread(); SupportSQLiteDatabase database = mOpenHelper.getWritableDatabase(); mInvalidationTracker.syncTriggers(database); database.beginTransaction(); } /** * Wrapper for {@link SupportSQLiteDatabase#endTransaction()}. * * @deprecated Use {@link #runInTransaction(Runnable)} */ @Deprecated public void endTransaction() { mOpenHelper.getWritableDatabase().endTransaction(); if (!inTransaction()) { // enqueue refresh only if we are NOT in a transaction. Otherwise, wait for the last // endTransaction call to do it. mInvalidationTracker.refreshVersionsAsync(); } } }
76. public abstract class RoomDatabase { public void runInTransaction(@NonNull Runnable body) { beginTransaction(); try { body.run(); setTransactionSuccessful(); } finally { endTransaction(); } } }
77. What can developers Do?
78. Conclusions
79. Large queries and paging in Room
80. Cursor - Android API package android.database.sqlite; public class SQLiteCursor extends AbstractWindowedCursor { private void fillWindow(int requiredPos) { ... } }
81. So large queries are...
82. Why is this a problem? 1. SQLiteCursor isn't in a transaction
83. Why is this a problem? 1. SQLiteCursor isn't in a transaction 2. SQLiteCursor cursorPickFillWindowStartPosition
84. /** * Picks a start position for {@link Cursor#fillWindow} such that the * window will contain the requested row and a useful range of rows * around it. * * When the data set is too large to fit in a cursor window, seeking the * cursor can become a very expensive operation since we have to run the * query again when we move outside the bounds of the current window. * * We try to choose a start position for the cursor window such that * 1/3 of the window's capacity is used to hold rows before the requested * position and 2/3 of the window's capacity is used to hold rows after the * requested position. * * @param cursorPosition The row index of the row we want to get. * @param cursorWindowCapacity The estimated number of rows that can fit in * a cursor window, or 0 if unknown. * @return The recommended start position, always less than or equal to * the requested row. * @hide */ public static int cursorPickFillWindowStartPosition( int cursorPosition, int cursorWindowCapacity) { return Math.max(cursorPosition - cursorWindowCapacity / 3, 0); }
85. Solution #1: @Transaction https://medium.com/androiddevelopers/7-pro-tips- for-room-fbadea4bfbd1
86. Solution #2: Write smaller queries!
87. Solution #3: Integrate with paging library package androidx.room.paging; public abstract class LimitOffsetDataSource<T> extends PositionalDataSource<T> { public List<T> loadRange(int startPosition, int loadCount) { final RoomSQLiteQuery sqLiteQuery = getSQLiteQuery(startPosition, loadCount); if (mInTransaction) { mDb.beginTransaction(); Cursor cursor = null; //noinspection TryFinallyCanBeTryWithResources try { cursor = mDb.query(sqLiteQuery); List<T> rows = convertRows(cursor); mDb.setTransactionSuccessful(); return rows; } finally { if (cursor != null) { cursor.close(); } mDb.endTransaction(); sqLiteQuery.release(); } } else { Cursor cursor = mDb.query(sqLiteQuery); //noinspection TryFinallyCanBeTryWithResources try { return convertRows(cursor); } finally { cursor.close(); sqLiteQuery.release(); } } } }
88. mLimitOffsetQuery = "SELECT * FROM ( " + mSourceQuery.getSql() + " ) LIMIT ? OFFSET ?";
89. Paging + LiveData + Room support LiveData<PagedList<User>> users = new LivePagedListBuilder<>( userDao.loadUsersByAgeDesc(), /*page size*/ 20).build(); https://github.com/googlesamples/android- architecture-components/tree/master/PagingSample
90. Thanks! Effie Barak (@CodingChick)