Code Completion with Scale

Transcript

1. Better Autocomplete Code Completion Servers Abhishek Gahlot
2. Introduction Why Code Completion?
3. Why Code Completion? HackerRank uses Web Based Editors Ease of use Switching between browser and editor Low latency for completion and docs
4. Previous Solutions • Eclim - Memory Issue and Overkill • Tries - Not Comprehensive • Inbuilt keywords - Doesn’t do much
5. So…How does it work? Parser, Grammar, AST
6. Code Completion • Parser, Grammar, Abstract Syntax Tree (AST) • Can Handle Errors, Partial code • Should be Fast Enough
7. Parser • Most parsers check inputs strictly and raise errors • We need parsers that can tolerate errors and give some AST • Heuristics and general approaches to make it tolerant
8. Abstract Syntax Tree (AST)
9. What we’ve Done… Our Solution
10. Current Solution • Separate Daemon for all Languages • Modular with interfaces • Ability to sync data and projects
11. Architecture
12. Language Support Python/Python3 C/C++/ObjectiveC Javascript Go Java Swift C# PHP
13. C/C++ • Clang • Ability to support both C and C++
14. Python and Python 3 • No good option available. • Javaparser ( https://github.com/javaparser/ javaparser) • Make java completion faster than available solution
15. Java • No good option available. • Javaparser ( https://github.com/javaparser/ javaparser) • Make java completion faster than available solution
16. Javascript, Go, and PHP • Javascript is done using Tern ( Acorn Parser ) • Go is done using Gocode • PHP uses language server.
17. Swift • Open Source • SourceKitten + Swift = Autocompletion on Linux
18. Communication over Sockets • All daemons connect to node app via Sockets • Backend is connected with web via websockets • File write time
19. Project Based Completion • Project Type Questions • Ability to sync only tokens required to apply server side instead of sending whole files • Differential Synchronization
20. Further Optimization Latency | Packets | Geolocation
21. Network Latency • Packet Journey of autocompletion • Websocket for connection
22. Geolocation Routing • Possible approaches done at Application Level • Geographical latency for users
23. Comparing Consumption How the Current solution scales
24. Previous Consumption 25 m4.large EC2 Instances
25. Current Consumption 5 M4.large Instances
26. Python
27. Javascript
28. Alternate Ways • Antlr • Language server protocol
29. Questions?
30. Abhishek Gahlot me@abhishek.it Thank You!