SimplePEG и Tchaikovsky - универсальная валидация

Алексей Охрименко (Senior JavaScript Developer at IPONWEB) @ Moscow Python Meetup 55

"Как создать универсальную валидацию для Backend и Frontend и как валидировать сложные текстовые поля".

Видео: http://www.moscowpython.ru/meetup/55/simplepeg-tchaikovsky/

Transcript

1. SIMPLEPEG И TCHAIKOVSKY
   УНИВЕРСАЛЬНАЯ ВАЛИДАЦИЯ
   Алексей Охрименко - IPONWEB
   1
   

   View Slide

2. Алексей
   Охрименко
   Twitter (@Ai_boy)
   2
   

   View Slide

3. IPONWEB RTB
   3
   

   View Slide

4. View Slide

5. View Slide

6. Валидация на клиенте и сервере
   постоянно разъезжается…
   6
   

   View Slide

7. Tchaikovsky
   7
   

   View Slide

8. true || false
   8
   

   View Slide

9. tchaikovsky.validate({a: 1}, “a == 1”)
   9
   

   View Slide

10. tchaikovsky.validate({a: 1}, “a == 1”)
    10
    

    View Slide

11. tchaikovsky.validate({a: 1}, “a == 1”)
    11
    

    View Slide

12. obj.id == null
    12
    

    View Slide

13. obj.start < obj.end
    13
    

    View Slide

14. obj.count > 0 and obj.id > 10
    14
    

    View Slide

15. obj.adress is email
    15
    

    View Slide

16. obj.type in [‘open’,’closed’]
    16
    

    View Slide

17. if obj.end_date != null
    then obj.count > 0
    else true
    17
    

    View Slide

18. Config
    Client Server
    

    View Slide

19. Tchaikovsky
    Python
    
    JavaScript
    
    C#
    
    Java
    
    Go
    
    Ruby
    19
    

    View Slide

20. Tchaikovsky
    Python
    
    JavaScript
    
    Lua
    
    C#
    
    Java
    
    Go
    
    Ruby
    20
    

    View Slide

21. Tchaikovsky - based on
    21
    

    View Slide

22. Tchaikovsky - based on
    SimplePEG
    22
    

    View Slide

23. Tchaikovsky - based on
    SimplePEG
    Visitor
    23
    

    View Slide

24. Почему не регулярки и
    обычно проверки в коде?
    24
    

    View Slide

25. Каждый раз когда вы пытаетесь
    решить проблему с помощью
    регулярки…
    

    View Slide

26. Каждый раз когда вы пытаетесь
    решить проблему с помощью
    регулярки - у вас становится на 1
    проблему больше
    

    View Slide

27. View Slide

28. RFC 822, 2822, 5322
    • https://tools.ietf.org/html/rfc822 ( 1982 год )
    
    • https://tools.ietf.org/html/rfc2822 ( 2001 год )
    
    • https://tools.ietf.org/html/rfc5322 ( 2008 год )
    

    View Slide

29. ReDOS
    29
    

    View Slide

30. ReDOS
    

    View Slide

31. Поддержка сложных регулярок
    

    View Slide

32. Поддержка сложных регулярок
    

    View Slide

33. Tchaikovsky
    SimplePEG
    33
    

    View Slide

34. Что такое парсеры и
    грамматики?
    34
    

    View Slide

35. Любая сложная задача состоит
    из плохо сформированных
    вопросов
    

    View Slide

36. View Slide

37. View Slide

38. парсер и грамматика
    

    View Slide

39. class State(object):
    text = ""
    position = 0
    class Node(object):
    match = ""
    children = None
    start_position = None
    end_position = None
    

    View Slide

40. def to_json(self):
    """returns json string"""
    return json.dumps(self, default=lambda o: o.__dict__, sort_keys=False, indent=2)
    

    View Slide

41. def string(rule, state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule,
    start_position=start_position,
    end_position=state.position
    )
    else:
    return False
    

    View Slide

42. def string(rule, state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule,
    start_position=start_position,
    end_position=state.position
    )
    else:
    return False
    

    View Slide

43. def string(rule, state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule,
    start_position=start_position,
    end_position=state.position
    )
    else:
    return False
    

    View Slide

44. def string(rule, state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule,
    start_position=start_position,
    end_position=state.position
    )
    else:
    return False
    

    View Slide

45. def string(rule, state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule,
    start_position=start_position,
    end_position=state.position
    )
    else:
    return False
    

    View Slide

46. print( string(‘a’, {}) )
    

    View Slide

47. print( string(‘a’, {}) )
    {
    match = 'a',
    start_position: 0,
    end_position: 1
    }
    

    View Slide

48. FP спешит на помощь
    

    View Slide

49. Каррирование и замыкание
    

    View Slide

50. def sum(a, b):
    return a + b
    

    View Slide

51. def sum(a):
    def _(b):
    return a + b
    return _
    

    View Slide

52. def sum(a):
    def _(b):
    return a + b
    return _
    

    View Slide

53. def string(rule):
    def _(state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule
    )
    else:
    return False
    return _
    

    View Slide

54. def string(rule):
    def _(state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule
    )
    else:
    return False
    return _
    

    View Slide

55. def string(rule):
    def _(state):
    if state.text[state.position:state.position+len(rule)] == rule:
    start_position = state.position
    state.position += len(rule)
    return Node(
    type='string',
    match=rule
    )
    else:
    return False
    return _
    

    View Slide

56. View Slide

57. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

58. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

59. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

60. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

61. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

62. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

63. def sequence(parsers):
    def _(state):
    asts = []
    start_position = state.position
    for parser in parsers:
    ast = parser(state)
    if ast: asts.append(ast)
    else: return False
    return Node(
    type='sequence',
    children=asts
    )
    return _
    

    View Slide

64. parser = sequence([
    string('a'),
    sequence([
    string('b')
    string('c')
    ])
    ])
    

    View Slide

65. parser = sequence([
    string('a'),
    sequence([
    string('b')
    string('c')
    ])
    ])
    parser({})
    

    View Slide

66. parser = sequence([
    string('a'),
    sequence([
    string('b')
    string('c')
    ])
    ])
    parser({})
    {
    type: 'sequence',
    children: [
    {
    type: 'string',
    match: 'a'
    },
    {
    type: 'sequence',
    children: [
    {
    type: 'string',
    match: 'b'
    },
    {
    type: 'string',
    match: 'c'
    },
    ]
    }
    ]
    }
    

    View Slide

67. View Slide

68. View Slide

69. PEG
    69
    

    View Slide

70. PEG - абстрактная
    формальная грамматика
    70
    

    View Slide

71. PEG - как интерфейс
    или абстрактный класс
    71
    

    View Slide

72. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    PEG
    (7 / 3)
    

    View Slide

73. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    Rule
    e -> e
    

    View Slide

74. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    Sequence
    e1 e2 e3
    

    View Slide

75. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    Ordered choice
    e1 / e2
    

    View Slide

76. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    Zero-or-more
    e*
    

    View Slide

77. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    One-or-more
    e+
    

    View Slide

78. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    Optional
    e?
    

    View Slide

79. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    And-predicate
    &e
    

    View Slide

80. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    Not-predicate
    !e
    

    View Slide

81. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    String
    «something»
    

    View Slide

82. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    RegExp
    [a-zA-Z0-9]
    

    View Slide

83. // Rule
    // Parsing Expression
    // Parsing Expression
    // Sequence: e1 e2 e3
    // Ordered choice: e1 / e2
    // Zero-or-more: e*
    // One-or-more: e+
    // Optional: e?
    // And-predicate: &e
    // Not-predicate: !e
    // Atomic Parsing Expression:
    // String
    // RegExp
    // EOF
    EOF
    end of file
    

    View Slide

84. Tchaikovsky
    Visitor
    84
    

    View Slide

85. Visitor
    85
    

    View Slide

86. View Slide

87. Visitor
    

    View Slide

88. Visitor
    

    View Slide

89. Visitor
    

    View Slide

90. Visitor
    

    View Slide

91. Visitor
    

    View Slide

92. View Slide

93. def visit(self, node):
    if node.children:
    children = [self.visit(child) for child in node.children]
    node.children = children
    return getattr(self, node.action)(node)
    

    View Slide

94. def visit(self, node):
    if node.children:
    children = [self.visit(child) for child in node.children]
    node.children = children
    return getattr(self, node.action)(node)
    

    View Slide

95. def visit(self, node):
    if node.children:
    children = [self.visit(child) for child in node.children]
    node.children = children
    return getattr(self, node.action)(node)
    

    View Slide

96. class Visitor(object):
    def string(self, node):
    return node
    def sequence(self, node):
    return node
    

    View Slide

97. fn()
    fn() fn()
    fn() fn()
    

    View Slide

98. sequence([
    string('a'),
    sequence([
    string('b')
    ])
    ])
    fn()
    fn() fn()
    fn() fn()
    

    View Slide

99. Текст AST Rec fn()
    

    View Slide

100. Текст AST Rec fn() AST
     Текст
     

     View Slide

101. Текст AST Rec fn() AST
     Текст
     SimplePEG
     

     View Slide

102. Текст AST Rec fn() AST
     Tchaikovsky
     Текст AST Rec fn() AST
     

     View Slide

103. Как писать свои
     грамматики?
     103
     

     View Slide

104. View Slide

105. Готовые грамматики
     105
     

     View Slide

106. View Slide

107. Только валидация?
     

     View Slide

108. THE END
     108
     

     View Slide

109. Алексей
     Охрименко
     IPONWEB
     
     Twitter (@Ai_boy)
     http://bit.ly/2wZwdAG
     109
     

     View Slide