Parser Combinator in Swift

Slide 1

Slide 1 text

Parser Combinator in Swift 2016/03/02-04 try! Swift (#tryswiftconf) Yasuhiro Inami / @inamiy

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Parser Combinator

Slide 4

Slide 4 text

Parser Takes input data (frequently text) and builds a data structure, e.g. abstract syntax tree (AST) • Lexer • Strings/Bytes → Tokens • Parser • Tokens → Syntax Tree

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Bottom-Up

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Top-Down

Slide 11

Slide 11 text

Parsing Algorithms • Bottom-Up • Operator Precedence Parsing • LR(k) (SLR / LALR / etc) • Top-Down • Recursive-descent / LL(k) (Table-driven) • Predictive / Backtracking • Packrat (Recursive-descent + memoization)

Slide 12

Slide 12 text

Combinator • Lambda expression without free variables • I = λx.x = { x in x } • K = λxy.x = { x, y in x } • S = λxyz.xz(yz) = { x, y, z in x(z)(y(z))} • Y = S(K(SII))(S(S(KS)K)(K(SII))) • ι = λx.xSK

Slide 13

Slide 13 text

Parser Combinator Higher-order function which takes: • Input: Parser(s) • Output: New Parser Combining simple parsers to construct more complex parsers → Functional Programming approach

Slide 14

Slide 14 text

Simple Parser in Swift

Slide 15

Slide 15 text

Parser Monad struct Parser { let parse: String -> (A, String)? } • Container of state-transforming function • Input: String • Output: Optional tuple of "output" & "remaining input"

Slide 16

Slide 16 text

// Applicative pure func pure(a: A) -> Parser { return Parser { (a, $0) } } // Alternative empty func empty () -> Parser { return Parser { _ in nil } }

Slide 17

Slide 17 text

// Monad bind (>>=) func >>- (p: Parser , f: A -> Parser) -> Parser { return Parser { input in if let (result, input2) = p.parse(input) { return f(result).parse(input2) } else { return nil } } } // Functor fmap (<$>) func <^> (f: A -> B, p: Parser ) -> Parser { return p >>- { a in pure(f(a)) } }

Slide 18

Slide 18 text

// Alternative choice (associative operation) func <|> (p: Parser , q: Parser ) -> Parser { return Parser { input in if let (result, input2) = p.parse(input) { return (result, input2) } else { return q.parse(input) } } }

Slide 19

Slide 19 text

// Applicative sequential application func <*> (p: Parser B>, q: Parser ) -> Parser { return p >>- { f in f <^> q } } // Sequence actions, discarding the value of the second argument func <* (p: Parser , q: Parser) -> Parser { return const <^> p <*> q } // Sequence actions, discarding the value of the first argument func *> (p: Parser , q: Parser) -> Parser { return const(id) <^> p <*> q }

Slide 20

Slide 20 text

Parse 1 character func satisfy(predicate: Character -> Bool) -> Parser { return Parser { input in if let (head, tail) = uncons(input) where predicate(head) { return (tail, head) } else { return nil } } }

Slide 21

Slide 21 text

Parse 1 character func any() -> Parser { return satisfy(const(true)) } func digit() -> Parser { return satisfy { "0"..."9" ~= $0 } } func char(c: Character) -> Parser { return satisfy { $0 == c } }

Slide 22

Slide 22 text

Parse string func string(str: String) -> Parser { if let (head, tail) = uncons(str) { return char(head) *> string(tail) *> pure(str) } else { return pure("") } }

Slide 23

Slide 23 text

Combine parsers func many(p: Parser ) -> Parser<[A]> { return many1(p) <|> pure([]) } func many1 (p: Parser ) -> Parser<[A]> { return cons <^> p <*> many(p) }

Slide 24

Slide 24 text

Combine parsers func skipMany(p: Parser ) -> Parser<()> { return skipMany1(p) <|> pure(()) } func skipMany1 (p: Parser ) -> Parser<()> { return p *> skipMany(p) } func skipSpaces() -> Parser<()> { return skipMany(space) }

Slide 25

Slide 25 text

Parse tokens func symbol(str: String) -> Parser { return skipSpaces() *> string(str) <* skipSpaces() } func natural() -> Parser { return skipSpaces() *> ({ Int(String($0))! } <^> many1(digit())) <* skipSpaces() }

Slide 26

Slide 26 text

Let's play!

Slide 27

Slide 27 text

Simple Arithmetics Backus–Naur Form (BNF) ::= + | ::= * | ::= ( ) | ::= '0' | '1' | '2' | ...

Slide 28

Slide 28 text

Simple Arithmetics func expr() -> Parser { return term() >>- { t in // uses right recursion (symbol("+") *> expr() >>- { e in pure(t + e) }) <|> pure(t) } } func term() -> Parser { return factor() >>- { f in (symbol("*") *> term() >>- { t in pure(f * t) }) <|> pure(f) } } func factor() -> Parser { return (symbol("(") *> expr() <* symbol(")")) <|> natural() }

Slide 29

Slide 29 text

let (ans, _) = expr().parse(" ( 12 + 3 ) * 4+5")! expect(ans) == 65

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

More?

Slide 32

Slide 32 text

TryParsec https://github.com/inamiy/TryParsec

Slide 33

Slide 33 text

TryParsec • Monadic Parser Combinator for ✨✨ try! Swift ✨✨ • Inspired by Haskell's Attoparsec / Aeson • Supports CSV / XML / JSON • LL(*) with backtracking by default • Doesn't try, but please try! :)

Slide 34

Slide 34 text

TryParsec • Basic Operators: >>-, <^>, <*>, *>, <*, <|>, > • Combinators: many, many1, manyTill, zeroOrOne, skipMany, skipMany1, sepBy, sepBy1, sepEndBy, sepEndBy1, count, chainl, chainl1, chainr, chainr1 • Text (UnicodeScalarView): peek, endOfInput, satisfy, skip, skipWhile, take, takeWhile, any, char, not, string, asciiCI, oneOf, noneOf, space, skipSpaces, number... (etc)

Slide 35

Slide 35 text

Parse JSON

Slide 36

Slide 36 text

enum JSON enum JSON { case String(Swift.String) case Number(Double) case Bool(Swift.Bool) case Null case Array([JSON]) case Object([Swift.String : JSON]) }

Slide 37

Slide 37 text

JSON example { "string": "hello", "num": 123.45, "bool": true, "null": null, "array": ["hello", 9.99, false], "dict": { "key": "value" }, "object": { "enabled": true } }

Slide 38

Slide 38 text

Parse JSON (to AST) let json = parseJSON(jsonString).value print(json) Result: JSON.Object(["null": .Null, "num": .Number(123.45), "bool": .Bool(true), "string": .String("hello"), "array": .Array([.String("hello"), .Number(9.99), .Bool(false)]), "dict": .Object(["key": .String("value")]), "object": .Object(["enabled": .Bool(true)]) ])

Slide 39

Slide 39 text

JSON Decoding

Slide 40

Slide 40 text

struct Model struct Model { let string: String let num: Double let bool: Bool let null: Any? let array: [Any] let dict: [String : Any] let subModel: SubModel let dummy: Bool? // doesn't exist in raw JSON }

Slide 41

Slide 41 text

FromJSON (Protocol) extension Model: FromJSON { static func fromJSON(json: JSON) -> Result { return fromJSONObject(json) { curry(self.init) <^> $0 !! "string" <*> $0 !! "num" <*> $0 !! "bool" <*> $0 !! "null" <*> $0 !! "array" <*> $0 !! "dict" <*> $0 !! "object" // mapping to SubModel <*> $0 !? "dummy" // doesn't exist in raw JSON } } }

Slide 42

Slide 42 text

Decode from JSON String let model = decode(jsonString).value print(model) Result: Model(string: "hello", num: 123.45, bool: true, null: nil, array: ["hello", 9.99, false], dict: ["key": "value"], subModel: SubModel(enabled: true), dummy: nil)

Slide 43

Slide 43 text

JSON Encoding

Slide 44

Slide 44 text

ToJSON (Protocol) extension Model: ToJSON { static func toJSON(model: Model) -> JSON { return toJSONObject([ "string" ~ model.string, "num" ~ model.num, "bool" ~ model.bool, "null" ~ model.null, "array" ~ model.array, "dict" ~ model.dict, "subModel" ~ model.subModel ]) } }

Slide 45

Slide 45 text

Encode to JSON String let jsonString = encode(model) print(jsonString) Result: "{ \"bool\" : true, \"null\" : null, \"num\" : 123.45, \"string\" : \"hello\", \"array\" : [ \"hello\", 9.99, false ], \"dict\" : { \"key\" : \"value\" }, \"subModel\" : { \"enabled\" : true } }"

Slide 46

Slide 46 text

Summary (TryParsec) • Supports CSV / XML / JSON • Simple, readable, and easy to create your own parsers • Caveats • Needs performance improvements • FromJSON / ToJSON doesn't work in some nested structure • Swift 3 (with higher kinded types support) will surely solve this problem!

Slide 47

Slide 47 text

Thanks! https://github.com/inamiy/TryParsec