Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CLIFuzzer: Mining Grammars for Command-Line Invocations

CLIFuzzer: Mining Grammars for Command-Line Invocations

ESEC/FSE 2022

Rahul Gopinath

May 23, 2022
Tweet

More Decks by Rahul Gopinath

Other Decks in Research

Transcript

  1. CLIFuzzer: Mining Grammars
    for Commandline Invocations
    Rahul Gopinath
    Abhilash Gupta
    Andreas Zeller CISPA Helmholtz Center for Information Security
    CISPA Helmholtz Center for Information Security
    University of Sydney

    View full-size slide

  2. CLIFuzzer: Mining Grammars
    for Commandline Invocations
    Rahul Gopinath
    Abhilash Gupta
    Andreas Zeller CISPA Helmholtz Center for Information Security
    CISPA Helmholtz Center for Information Security
    University of Sydney

    View full-size slide

  3. 3
    Command Line Utility
    $ ls -a --color=always
    . Google Chrome.app Microsoft Word.app TextMate.app
    .. Google Docs.app Numbers.app UTM.app
    .DS_Store Google Drive.app OneDrive.app Utilities
    .localized Google Sheets.app Opera.app VMware Fusion Tech Pr
    Barrier.app Google Slides.app Pages.app VMware Fusion.app
    Cisco Keynote.app Parallels Desktop.app Visual Studio Code.ap
    Cisco Jabber.app LanguageTool.app Piezo.app Zotero.app
    Dato.app Microsoft Edge.app Safari.app iMovie.app
    DiffusionBee.app Microsoft Excel.app Self Service.app scanserver.app
    Docker.app Microsoft OneNote.app Skype.app zoom.us.app
    Firefox.app Microsoft Outlook.app Slack.app
    Free Ruler.app Microsoft PowerPoint.app Stats.app
    GarageBand.app Microsoft Teams.app TeX

    View full-size slide

  4. 4
    Command Line Utility
    $ ls -a --color=always
    . Google Chrome.app Microsoft Word.app TextMate.app
    .. Google Docs.app Numbers.app UTM.app
    .DS_Store Google Drive.app OneDrive.app Utilities
    .localized Google Sheets.app Opera.app VMware Fusion Tech Pr
    Barrier.app Google Slides.app Pages.app VMware Fusion.app
    Cisco Keynote.app Parallels Desktop.app Visual Studio Code.ap
    Cisco Jabber.app LanguageTool.app Piezo.app Zotero.app
    Dato.app Microsoft Edge.app Safari.app iMovie.app
    DiffusionBee.app Microsoft Excel.app Self Service.app scanserver.app
    Docker.app Microsoft OneNote.app Skype.app zoom.us.app
    Firefox.app Microsoft Outlook.app Slack.app
    Free Ruler.app Microsoft PowerPoint.app Stats.app
    GarageBand.app Microsoft Teams.app TeX
    $ command [configuration options][arguments]

    View full-size slide

  5. $ ls xldjafljdj;jfafiioequreqrin,mnewioqr;e3kekjdfjdafj
    ls: xldjafljdj: No such file or directory
    zsh: exit 1 ls xldjafljdj
    zsh: command not found: jfafiioequreqrin,mnewioqr
    zsh: exit 127 jfafiioequreqrin,mnewioqr
    zsh: command not found: e3kekjdfjdafj
    zsh: exit 127 e3kekjdfjdafj

    View full-size slide

  6. 7
    Command Line Arugment Processing

    View full-size slide

  7. 9
    {'': [
    '()*'],
    '': [' -h',
    ' --help',
    ' --version',
    ' -v',
    ' --verbose'],
    '': [
    ' foo.py'],
    '': ['+'],
    '': [
    /0-9/, /a-z/, /A-Z/
    '[', '\\', ']', '^', '_',
    '`', '{','|','}', '~'],
    '': [''],
    '': ['(-)?+'],
    '': [/0-9/],
    '': [''],
    '': [''],
    '': [''],
    '': ['']}
    Getopt to Context Free Grammar

    View full-size slide

  8. 10
    Argument and Option Processing
    Libc functions

    View full-size slide

  9. 11
    Command Line Invocation Grammar
    Convert getopt to CFG Extract Argument Types Extract Option Types

    View full-size slide

  10. 12
    Evaluation
    ▪ 44 utilities in Ubuntu 20.04.3 LTS
    ▪ Use getopt, getopt_long or getopt_long_only to parse their options
    ▪ Take in a file or stdin as argument
    ▪ Test inputs
    ▪ 60 generated files of large lengths (~ 100KB and 10MB) using different seed
    values
    ▪ 3 large publicly available text files*
    ▪ 3 large publicly available media (1 image and 2 audio) files
    as bc bison cat col colcrt column colrm comm cmp cut
    dc diff expand fmt fold gdb grep head join look m4
    nl nm od paste pr ptx rev sdiff spell strings strip
    sort tac tail tee tr troff tsort unexpand uniq wc xargs

    View full-size slide

  11. 22
    37
    41
    85
    11
    39
    80
    32
    34
    58
    29
    17.31
    30.89
    26.06
    83.32
    10.51
    30.34
    56.86
    16.49
    31.11
    49.84
    29.70
    0.00
    22.50
    45.00
    67.50
    90.00
    as bison column dc gdb ptx spell tac tee troff tsort
    % coverage achieved
    CLI Utility
    CLIFuzzer AFL++
    Fig: Coverage achieved by AFL++ and MyFuzzer on different utilities that report errors*
    * -> AFL++ ran for 3 hours. CLIFuzzer ran 3000 invocations. CLIFuzzer took 1-1.5 hours to run.

    View full-size slide

  12. 14
    as bc bison cat col colcrt column colrm comm cmp cut
    dc diff expand fmt fold gdb grep head join look m4
    nl nm od paste pr ptx rev sdiff spell strings strip
    sort tac tail tee tr troff tsort unexpand uniq wc xargs
    column
    (v2.37.2)
    tac (v9.0) tee (v9.0) tsort (v9.0)
    * *
    = crash, = hang
    as (v2.37) bison (v3.8) dc (v1.41) gdb (v11.1) ptx (v9.0) spell (v1.1) troff (v1.22.4)
    * * *
    Parameter Interactions
    General Failures

    View full-size slide