Parsing Binary Data

A lighting talk about parsing binary data in ruby with bindata gem.

Transcript

1. Parsing and Writing
   Binary Data in Ruby
   Jan Krutisch
   @halfbyte
   jan.krutisch@mindmatters.de
   

   View Slide

2. señor developer
   

   View Slide

3. View Slide

4. Protracker "mod" files
   

   View Slide

5. Binary Format
   

   View Slide

6. View Slide

7. ‣ Instrument Metadata
   ‣ Sample Data
   ‣ Pattern Data
   ‣ Sequence Data
   Mod File Contents
   

   View Slide

8. ‣ Instrument Metadata
   ‣ Sample Data
   ‣ Pattern Data
   ‣ Sequence Data
   Mod File Contents
   Array of Structs (=Array of Hashes)
   

   View Slide

9. ‣ Instrument Metadata
   ‣ Sample Data
   ‣ Pattern Data
   ‣ Sequence Data
   Mod File Contents
   Array of Structs (=Array of Hashes)
   Array of Blobs
   

   View Slide

10. ‣ Instrument Metadata
    ‣ Sample Data
    ‣ Pattern Data
    ‣ Sequence Data
    Mod File Contents
    Array of Structs (=Array of Hashes)
    Array of Blobs
    Arrays of Arrays of Arrays of Structs
    

    View Slide

11. ‣ Instrument Metadata
    ‣ Sample Data
    ‣ Pattern Data
    ‣ Sequence Data
    Mod File Contents
    Array of Structs (=Array of Hashes)
    Array of Blobs
    Arrays of Arrays of Arrays of Structs
    Array of Integers
    

    View Slide

12. Simple Approach
    

    View Slide

13. Serializer / Deserializer
    

    View Slide

14. Different Datatypes
    

    View Slide

15. Protracker Datatypes
    

    View Slide

16. Protracker Datatypes
    ‣ 8Bit signed binary blobs
    

    View Slide

17. Protracker Datatypes
    ‣ 8Bit signed binary blobs
    ‣ UInt8
    

    View Slide

18. Protracker Datatypes
    ‣ 8Bit signed binary blobs
    ‣ UInt8
    ‣ UInt16 (Endianness?)
    

    View Slide

19. Protracker Datatypes
    ‣ 8Bit signed binary blobs
    ‣ UInt8
    ‣ UInt16 (Endianness?)
    ‣ \0-Terminated/Padded Strings
    

    View Slide

20. Protracker Datatypes
    ‣ 8Bit signed binary blobs
    ‣ UInt8
    ‣ UInt16 (Endianness?)
    ‣ \0-Terminated/Padded Strings
    ‣ Int4 (Yes, that‘s a signed halfbyte)
    

    View Slide

21. And this...
    

    View Slide

22. View Slide

23. Procedural Mess
    

    View Slide

24. Declarative
    

    View Slide

25. gem "bindata"
    

    View Slide

26. class ProtrackerModule < BinData::Record
    endian :big
    string :name, :length => 20, :trim_padding => true
    array :samples, :type => Sample, :length => 31
    uint8 :pattern_table_length
    uint8 :unused
    array :pattern_table, :type => :uint8, :initial_length => 128
    [...]
    

    View Slide

27. class ProtrackerModule < BinData::Record
    endian :big
    string :name, :length => 20, :trim_padding => true
    array :samples, :type => Sample, :length => 31
    uint8 :pattern_table_length
    uint8 :unused
    array :pattern_table, :type => :uint8, :initial_length => 128
    [...]
    

    View Slide

28. class ProtrackerModule < BinData::Record
    endian :big
    string :name, :length => 20, :trim_padding => true
    array :samples, :type => Sample, :length => 31
    uint8 :pattern_table_length
    uint8 :unused
    array :pattern_table, :type => :uint8, :initial_length => 128
    [...]
    

    View Slide

29. class ProtrackerModule < BinData::Record
    endian :big
    string :name, :length => 20, :trim_padding => true
    array :samples, :type => Sample, :length => 31
    uint8 :pattern_table_length
    uint8 :unused
    array :pattern_table, :type => :uint8, :initial_length => 128
    [...]
    

    View Slide

30. class ProtrackerModule < BinData::Record
    endian :big
    string :name, :length => 20, :trim_padding => true
    array :samples, :type => Sample, :length => 31
    uint8 :pattern_table_length
    uint8 :unused
    array :pattern_table, :type => :uint8, :initial_length => 128
    [...]
    

    View Slide

31. array :patterns,
    :type => Pattern,
    :initial_length => lambda { pattern_table.max + 1 }
    

    View Slide

32. View Slide

33. Demo
    

    View Slide

34. señor developer
    http://senordevelopershop.spreadshirt.de/
    (europe)
    http://senordeveloper.spreadshirt.com/
    (intl)
    

    View Slide