CLIFuzzer: Mining Grammars for Commandline Invocations Rahul Gopinath Abhilash Gupta Andreas Zeller CISPA Helmholtz Center for Information Security CISPA Helmholtz Center for Information Security University of Sydney 

CLIFuzzer: Mining Grammars for Commandline Invocations Rahul Gopinath Abhilash Gupta Andreas Zeller CISPA Helmholtz Center for Information Security CISPA Helmholtz Center for Information Security University of Sydney 

3 Command Line Utility $ ls -a --color=always . Google Chrome.app Microsoft Word.app TextMate.app .. Google Docs.app Numbers.app UTM.app .DS_Store Google Drive.app OneDrive.app Utilities .localized Google Sheets.app Opera.app VMware Fusion Tech Pr Barrier.app Google Slides.app Pages.app VMware Fusion.app Cisco Keynote.app Parallels Desktop.app Visual Studio Code.ap Cisco Jabber.app LanguageTool.app Piezo.app Zotero.app Dato.app Microsoft Edge.app Safari.app iMovie.app DiffusionBee.app Microsoft Excel.app Self Service.app scanserver.app Docker.app Microsoft OneNote.app Skype.app zoom.us.app Firefox.app Microsoft Outlook.app Slack.app Free Ruler.app Microsoft PowerPoint.app Stats.app GarageBand.app Microsoft Teams.app TeX 

4 Command Line Utility $ ls -a --color=always . Google Chrome.app Microsoft Word.app TextMate.app .. Google Docs.app Numbers.app UTM.app .DS_Store Google Drive.app OneDrive.app Utilities .localized Google Sheets.app Opera.app VMware Fusion Tech Pr Barrier.app Google Slides.app Pages.app VMware Fusion.app Cisco Keynote.app Parallels Desktop.app Visual Studio Code.ap Cisco Jabber.app LanguageTool.app Piezo.app Zotero.app Dato.app Microsoft Edge.app Safari.app iMovie.app DiffusionBee.app Microsoft Excel.app Self Service.app scanserver.app Docker.app Microsoft OneNote.app Skype.app zoom.us.app Firefox.app Microsoft Outlook.app Slack.app Free Ruler.app Microsoft PowerPoint.app Stats.app GarageBand.app Microsoft Teams.app TeX $ command [configuration options][arguments] 

5 (CACM '90) 

$ ls xldjafljdj;jfafiioequreqrin,mnewioqr;e3kekjdfjdafj ls: xldjafljdj: No such file or directory zsh: exit 1 ls xldjafljdj zsh: command not found: jfafiioequreqrin,mnewioqr zsh: exit 127 jfafiioequreqrin,mnewioqr zsh: command not found: e3kekjdfjdafj zsh: exit 127 e3kekjdfjdafj 

7 Command Line Arugment Processing 

9 {'': [ '()*'], '': [' -h', ' --help', ' --version', ' -v', ' --verbose'], '': [ ' foo.py'], '': ['+'], '': [ /0-9/, /a-z/, /A-Z/ '[', '\\', ']', '^', '_', '`', '{','|','}', '~'], '': [''], '': ['(-)?+'], '': [/0-9/], '': [''], '': [''], '': [''], '': ['']} Getopt to Context Free Grammar 

10 Argument and Option Processing Libc functions 

11 Command Line Invocation Grammar Convert getopt to CFG Extract Argument Types Extract Option Types 

12 Evaluation ▪ 44 utilities in Ubuntu 20.04.3 LTS ▪ Use getopt, getopt_long or getopt_long_only to parse their options ▪ Take in a file or stdin as argument ▪ Test inputs ▪ 60 generated files of large lengths (~ 100KB and 10MB) using different seed values ▪ 3 large publicly available text files* ▪ 3 large publicly available media (1 image and 2 audio) files as bc bison cat col colcrt column colrm comm cmp cut dc diff expand fmt fold gdb grep head join look m4 nl nm od paste pr ptx rev sdiff spell strings strip sort tac tail tee tr troff tsort unexpand uniq wc xargs 

22 37 41 85 11 39 80 32 34 58 29 17.31 30.89 26.06 83.32 10.51 30.34 56.86 16.49 31.11 49.84 29.70 0.00 22.50 45.00 67.50 90.00 as bison column dc gdb ptx spell tac tee troff tsort % coverage achieved CLI Utility CLIFuzzer AFL++ Fig: Coverage achieved by AFL++ and MyFuzzer on different utilities that report errors* * -> AFL++ ran for 3 hours. CLIFuzzer ran 3000 invocations. CLIFuzzer took 1-1.5 hours to run. 

14 as bc bison cat col colcrt column colrm comm cmp cut dc diff expand fmt fold gdb grep head join look m4 nl nm od paste pr ptx rev sdiff spell strings strip sort tac tail tee tr troff tsort unexpand uniq wc xargs column (v2.37.2) tac (v9.0) tee (v9.0) tsort (v9.0) * * = crash, = hang as (v2.37) bison (v3.8) dc (v1.41) gdb (v11.1) ptx (v9.0) spell (v1.1) troff (v1.22.4) * * * Parameter Interactions General Failures 

15 

16