OctoLab Cookbook: how to generate a unique key for a sequence

PHP tips and tricks: work with IteratorAggregate, Generator, SplQueue and SplDoublyLinkedList.

Transcript

1. Episode #1.0.2 (2016-05-22) Level: Intermediate OctoLab Cookbook How to generate a unique key for a sequence
2. The context
3. timestamp payment currency 18.03.2016 09:22:23 100 USD 18.03.2016 09:21:02 50 EUR 18.03.2016 09:19:46 10 USD 08.03.2016 09:31:03 10 USD 07.03.2016 22:52:17 145 USD 18.02.2016 23:17:25 35 EUR Import CSV from tinkoff.ru to TCA application
4. The problem
5. Merging timestamp payment currency 18.03.2016 09:22:23 100 USD 18.03.2016 09:21:02 50 EUR 18.03.2016 09:19:46 10 USD timestamp payment currency 18.03.2016 09:19:46 10 USD 08.03.2016 09:31:03 10 USD 07.03.2016 22:52:17 145 USD 18.02.2016 23:17:25 35 EUR timestamp payment currency 18.02.2016 23:17:25 35 EUR 17.02.2016 19:49:59 11 USD 02.02.2016 17:58:36 9 USD a first imported CSV file a second imported CSV file a third imported CSV file
6. The solution
7. - first iteration: simple, unique key constraint $table->addUniqueIndex([‘timestamp’, ‘payment’, ‘currency’]);
8. timestamp payment currency 22.02.2016 20:36:33 15 USD 22.02.2016 19:25:14 25 USD 22.02.2016 00:00:00 0,1 USD 22.02.2016 00:00:00 0,1 USD 21.02.2016 21:19:38 10 USD 21.02.2016 17:04:00 10 USD valid internal operations look exactly the same
9. - second iteration: buffering and forward hashing $table->addColumn(‘f_hash’, ‘string’, …) $table->addUniqueIndex([‘f_hash’]);
11. column forward hash row 1 08bb542ead91978a26317a4aa8caf1e5d355706e row 2 e993895c5fd4fcbb22f9af9c7d341c47db87a8a7 row 3 7bb25dfeeaec0235cc5cdc4643a09e753687bded row 4 NULL buffer size = 2 column forward hash row 1 bd74be8fda860811368dfc3dd170f34652c73dea row 2 eaf057a4aac69217b5cc2c996679e7d1e11f411d row 3 NULL row 4 NULL buffer size = 3
12. - third iteration: bidirectional hashing $table->addColumn(‘b_hash’, ‘string’, …) $table->addUniqueIndex([‘b_hash’]);
14. column forward hash backward hash row 1 … NULL row 2 … 0b48e15756c28dc8a00d54594a1c25cabe707973 row 3 … af8d00a06f6e58ec06a650433df597cf4140d78b row 4 NULL 219fe3369d071daadedf81fde031e0f1e74f571b buffer size = 2 not-nullable combination
15. Limitations • number of rows in a CSV file must be greater than or equal to the buffer size Benefits • import CSV files in any order • natural flow of the code with minimal changes in the original iteration
16. • StringifierInterface and CsvRowStringifier • BufferInterface and HashedBuffer • BufferedIterator • changes in the original iteration The real-world example
17. Thank you for your attention! Have questions? Kamil Samigullin a some developer kamil@samigullin.info @ikamilsk github.com/kamilsk linkedin.com/in/kamilsk