Slide 1
Slide 1 text
Chunk Computing
@matyo91
27-08-2024
In PHP
Slide 2
Slide 2 text
Why?
@matyo91
27-08-2024
Slide 3
Slide 3 text
address the inherent limitations
of memory making it easier to
manage, understand, and use
information
“
@matyo91
27-08-2024
Slide 4
Slide 4 text
What is “Chunking"?
@matyo91
27-08-2024
Slide 5
Slide 5 text
A "chunk" is a smaller,
manageable segment of a larger
data set used for easier data
transfer and processing.
“
@matyo91
27-08-2024
Slide 6
Slide 6 text
@matyo91
27-08-2024
Slide 7
Slide 7 text
Simplifies complex information.
Enhances learning and memory.
Applicable across various fields.
Key Takeaways
@matyo91
@matyo91
27-08-2024
Slide 8
Slide 8 text
File Splitting: File-sharing services like BitTorrent break large files
into smaller ‘chunks’ for faster, more efficient data transfer.
Video Streaming: Platforms like YouTube and Netflix split videos into
‘chunks’ to enable smoother streaming without waiting for the entire
video to load.
Big Data Processing: Tools like Apache Hadoop process large
datasets in parallel by dividing them into smaller ‘chunks,’ speeding
up analysis.
AI Model Training: In AI, data is split into smaller batches during
training, enabling frequent updates, faster learning, and efficient
handling of large datasets.
Examples
@matyo91
@matyo91
27-08-2024
Slide 9
Slide 9 text
In Memory Management
@matyo91
27-08-2024
Slide 10
Slide 10 text
Dynamic Allocation: Memory is allocated from heaps, which can be
time-consuming.
Performance Impact: Frequent heap management calls may slow
down systems.
Chunking Strategy:
Groups related allocations to minimize calls.
Example: Allocate/free multiple objects at once (e.g., 8 at a time),
reducing overhead.
Chunk Memory Management
@matyo91
@matyo91
27-08-2024
Slide 11
Slide 11 text
Traditional iteration
@matyo91
@matyo91
27-08-2024
Slide 12
Slide 12 text
array_chunk : Splitting a Simple
@matyo91
@matyo91
27-08-2024
Slide 13
Slide 13 text
array_chunk : Splitting a Simple
@matyo91
@matyo91
27-08-2024
Slide 14
Slide 14 text
array_chunk : Preserving Keys
@matyo91
@matyo91
27-08-2024
Slide 15
Slide 15 text
array_chunk : Preserving Keys
@matyo91
@matyo91
27-08-2024
Slide 16
Slide 16 text
chunk_split
@matyo91
@matyo91
27-08-2024
Slide 17
Slide 17 text
chunk_split
@matyo91
@matyo91
27-08-2024
Slide 18
Slide 18 text
Threaded::chunk
@matyo91
@matyo91
27-08-2024
Slide 19
Slide 19 text
fread
@matyo91
@matyo91
27-08-2024
Slide 20
Slide 20 text
SplFileObject
@matyo91
@matyo91
27-08-2024
Slide 21
Slide 21 text
Memory Management Functions
@matyo91
@matyo91
27-08-2024
Slide 22
Slide 22 text
Memory Management Functions
@matyo91
@matyo91
27-08-2024
Slide 23
Slide 23 text
Memory Management Functions
@matyo91
@matyo91
27-08-2024
Slide 24
Slide 24 text
Generators
@matyo91
@matyo91
27-08-2024
Slide 25
Slide 25 text
unset
@matyo91
@matyo91
27-08-2024
Slide 26
Slide 26 text
loophp/collection : Chunk
@matyo91
@matyo91
27-08-2024
Slide 27
Slide 27 text
loophp/collection : Chunk
@matyo91
@matyo91
27-08-2024
Slide 28
Slide 28 text
loophp/collection : Chunk
@matyo91
@matyo91
27-08-2024
Slide 29
Slide 29 text
In HTTP Message Transmission
@matyo91
27-08-2024
Slide 30
Slide 30 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 31
Slide 31 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 32
Slide 32 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 33
Slide 33 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 34
Slide 34 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 35
Slide 35 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 36
Slide 36 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 37
Slide 37 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 38
Slide 38 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 39
Slide 39 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 40
Slide 40 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 41
Slide 41 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 42
Slide 42 text
Http Chunk
@matyo91
@matyo91
27-08-2024
Slide 43
Slide 43 text
Http Chunked Response : Transfert-Encoding: chunked
Available since 1997 : HTTP 1.1 (RFC 2068)
@matyo91
@matyo91
27-08-2024
Http Chunk
Slide 44
Slide 44 text
@matyo91
@matyo91
27-08-2024
Symfony: chunk server
Slide 45
Slide 45 text
@matyo91
@matyo91
27-08-2024
Symfony: get entire response
Slide 46
Slide 46 text
@matyo91
@matyo91
27-08-2024
Symfony: each chunk individually
Slide 47
Slide 47 text
@matyo91
@matyo91
27-08-2024
Symfony: Http chunks
Slide 48
Slide 48 text
@matyo91
@matyo91
27-08-2024
Allison Guilhem AsyncDecoratorTrait
Slide 49
Slide 49 text
@matyo91
@matyo91
27-08-2024
Allison Guilhem AsyncDecoratorTrait
Slide 50
Slide 50 text
@matyo91
@matyo91
27-08-2024
Allison Guilhem AsyncDecoratorTrait
Slide 51
Slide 51 text
@matyo91
@matyo91
27-08-2024
Output Buffering
Slide 52
Slide 52 text
@matyo91
@matyo91
27-08-2024
Sending JSON Data
Slide 53
Slide 53 text
@matyo91
@matyo91
27-08-2024
Handling Large File Uploads
Slide 54
Slide 54 text
@matyo91
@matyo91
27-08-2024
Server-Sent Events
Slide 55
Slide 55 text
In Data Deduplication,
Synchronisation, and Compression
@matyo91
27-08-2024
Slide 56
Slide 56 text
@matyo91
@matyo91
27-08-2024
Data Deduplication
Slide 57
Slide 57 text
@matyo91
@matyo91
27-08-2024
Data Synchronization
Slide 58
Slide 58 text
@matyo91
@matyo91
27-08-2024
Remote Data Compression
Slide 59
Slide 59 text
@matyo91
@matyo91
27-08-2024
Content-Defined Chunking
Slide 60
Slide 60 text
@matyo91
@matyo91
27-08-2024
Chunking with Cloud Services
Slide 61
Slide 61 text
@matyo91
@matyo91
27-08-2024
alexandre-daubois : Lazy streams
Slide 62
Slide 62 text
@matyo91
@matyo91
27-08-2024
alexandre-daubois : Lazy streams
Slide 63
Slide 63 text
@matyo91
@matyo91
27-08-2024
alexandre-daubois : Lazy streams
Slide 64
Slide 64 text
@matyo91
@matyo91
27-08-2024
matyo91 : Cached offers
Slide 65
Slide 65 text
Chunk Versus
@matyo91
27-08-2024
Slide 66
Slide 66 text
Chunk: A chunk is a small piece of binary data transferred from
source to destination via streams. It carries all necessary
information for processing.
Buffer: A buffer is temporary RAM storage that manages chunk
data. When full, it sends data for processing, acting as an
intermediary between data reception and processing.
@matyo91
@matyo91
27-08-2024
Chunk vs Buffer Data
Slide 67
Slide 67 text
@matyo91
@matyo91
27-08-2024
Chunk vs Buffer Data
Chunks Buffer
A piece of binary data. A class for managing binary data.
Contains binary data.
Manages data in binary and other
formats.
In octet format, it must be handled carefully to avoid
data corruption.
Handles exceptions when managing
chunk data.
Independent container, managed by Buffer.
Dependent on Chunks or streams for
data transfer.
Used for retrieving data externally. Used to retrieve and format data.
Slide 68
Slide 68 text
@matyo91
@matyo91
27-08-2024
Chunk vs Buffer Data
Chunks Buffer
Uncontrolled flow. Controlled flow via Buffer and streams.
Cannot be created. Can be created to contain Chunks.
Example: .
Underlined part is a chunk.
Example: .
Entire contents are a buffer.
Stores one byte of data. Size depends on RAM and hardware.
Uncontrolled flow. Controlled flow via Buffer and streams.
Slide 69
Slide 69 text
Thunk is a subroutine used to inject a calculation into another
subroutine. Thunks are primarily used to delay a calculation until its
result is needed, or to insert operations at the beginning or end of the
other subroutine. They have many other applications in compiler code
generation and modular programming.
@matyo91
@matyo91
27-08-2024
Chunk vs Thunk
Slide 70
Slide 70 text
@matyo91
@matyo91
27-08-2024
Chunk vs Thunk
Slide 71
Slide 71 text
Hello!
I Am Mathieu Ledru
You can contact me at @matyo91
27-08-2024 @matyo91
Thanks!
Any questions?