UP Lecture 03

Compilers
Lexical Analysis
(202401)

Transcript

1. jgs Compilers Lecture 03: Lexical Analysis Dr. Javier Gonzalez-Sanchez jgonzalezs@up.edu.mx
2. Dr. Javier Gonzalez-Sanchez | Compilers | 2 jgs
3. Dr. Javier Gonzalez-Sanchez | Compilers | 3 jgs Review
4. Dr. Javier Gonzalez-Sanchez | Compilers | 4 jgs Review
5. Dr. Javier Gonzalez-Sanchez | Compilers | 5 jgs Review
6. Dr. Javier Gonzalez-Sanchez | Compilers | 6 jgs Review
7. Dr. Javier Gonzalez-Sanchez | Compilers | 7 jgs Review
8. jgs Test Yourselves
9. Dr. Javier Gonzalez-Sanchez | Compilers | 9 jgs Lexical Analysis int x = 5; float y = "hello; String @z = "9.5"; int x = cse340; if ( x > 14) while (5 == 5) if (int a) a = 1; x = x; for ( ; ; ); y = 13.45.0; int me = 99999000001111222000000111111222223443483045830948; while { x != 9} (); int {x} = 10;
10. Dr. Javier Gonzalez-Sanchez | Compilers | 10 jgs Lexical Analysis int x = 5; float y = "hello; String@z="9.5”;intx=cse340;if(x> 14) while (5 == 5) if (int a) a = 1; x = x; for ( ; ; );y = 13.45.0;int me =99999000001111222000000111111222 223443483045830948;while { x != 9} ();int {x} = 10; "hello "world" bye" # of STRINGS ? ? ? ? ? ? ? ? ?
11. Dr. Javier Gonzalez-Sanchez | Compilers | 11 jgs Lexical Analysis int x = 5; float y = "hello; String@z="9.5”;intx=cse340;if(x> 14) while (5 == 5) if (int a) a = 1; x = x; for ( ; ; );y = 13.45.0;int me =99999000001111222000000111111222 223443483045830948;while { x != 9} ();int {x} = 10; ”hello "world" bye" # of STRINGS 9 12 3 18 12 2 6 12 3
12. jgs Previously …
13. Dr. Javier Gonzalez-Sanchez | Compilers | 13 jgs High-Level Languages X,E,G,O,O #e1,I,I,0,7 @ OPR 19, AX STO x, AX LIT 5, AX OPR 21, AX LOD #e1,AX CAL 1, AX OPR 0, AX 5 Virtual Machine (interpreter) // sorce code int x; int foo () { read (x); print (5); } main () { foo (); } Lexer Parser Semantic Analyzer Code Generation 01001010101000010 01010100101010010 10100100000011011 11010010110101111 00010010101010010 10101001010101011 Assembler compilation execution
14. Dr. Javier Gonzalez-Sanchez | Compilers | 14 jgs Key Ideas Lexical Alphabet Symbol String Word Token Rules Regular Expression Deterministic Finite Automata
15. Dr. Javier Gonzalez-Sanchez | Compilers | 15 jgs Key Ideas Lexical Alphabet Symbol String Word Token Rules Regular Expression Deterministic Finite Automata
16. jgs Lexer
17. Dr. Javier Gonzalez-Sanchez | Compilers | 17 jgs Lexical Analysis | Steps a) Read a text FILE line by line b) For each LINE: § Read character by character. § Create sets of consecutive characters (STRING). Try to group the bigger amount of characters as possible. § Start a new set each time that you need. Take care of: Whitespace, Delimiter, Operator, End of Line and others special characters. c) For each STRING: verify if it is a valid WORD. d) Create a VECTOR and store the STRINGs and WORDs.
18. jgs Deterministic Finite Automata
19. Dr. Javier Gonzalez-Sanchez | Compilers | 19 jgs Deterministic Finite Automata § It is a finite state machine that accepts/rejects finite strings of symbols and produces a unique result for each input string. § In the automaton, there are three states (denoted graphically by circles) and transition arrows connecting one state with other. § Upon reading a symbol, a DFA jumps deterministically from a state to another by following the transition arrow.
20. Dr. Javier Gonzalez-Sanchez | Compilers | 20 jgs DFA | Examples binary 0b(0|1)+ string “([a-z] | [A-Z] | [0-9])*”
21. Dr. Javier Gonzalez-Sanchez | Compilers | 21 jgs DFA | Examples Operator Start Delimiter Integer Float ID String Char {+,-,*,/,%, <,>,=,!,…} {(, ), {, }, [, ]} {0-9} {$, _, 0-9, a-z} { ‘ } { ‘ } {0-9} {0-9} {0-9} {\.} {\.} { “ } { “ } {.} {.} {a-z} {_} {$}
22. Dr. Javier Gonzalez-Sanchez | Compilers | 22 jgs Drafting a Lexer § ID = § Operator = § Delimiter = § Keywords =
23. Dr. Javier Gonzalez-Sanchez | Compilers | 23 jgs Drafting a Lexer § Float = § Integer = § Hexadecimal = § Octal = § Binary = § String = § Char =
24. Dr. Javier Gonzalez-Sanchez | Compilers | 24 jgs Let’s Work
25. Dr. Javier Gonzalez-Sanchez | Compilers | 25 jgs Homework A Complete the DFA for ALL tokens in our language
26. Dr. Javier Gonzalez-Sanchez | Compilers | 26 jgs Homework B Review the following topics: • Regular Expressions • Deterministic Finite Automata
27. Dr. Javier Gonzalez-Sanchez | Compilers | 27 jgs Questions
28. jgs Compilers Javier Gonzalez-Sanchez, Ph.D. jgonzalezs@up.edu.mx Spring 2024 Copyright. These slides can only be used as study material for the Compilers course at Universidad Panamericana. They cannot be distributed or used for another purpose.