Moving Target Defense Quantification

Cad49e6ffc6048dc9c53c77a907632dc?s=47 Daniel Bilar
August 31, 2015

Moving Target Defense Quantification

Can we prioritize the quantification of MTD techniques? That is, which MTD properties are most important to quantify and why?
What is in good shape with respect to existing and imminent attempts to quantify those properties?
What kind of work is still missing and why is it not being done? This includes both empirical and analytic work on existing MTD quantification ideas as well as completely new scientific approaches to MTD quantification

Cad49e6ffc6048dc9c53c77a907632dc?s=128

Daniel Bilar

August 31, 2015
Tweet

Transcript

  1. Moving'Target'Defense' Quan2fica2on'Workshop' GMU,'Fairfax'(VA)' Aug'31D'Sep'1,'2015' ' Daniel'Bilar' dbilar@acm.org'

  2. Some'WS'ques2ons,'varia' 1.  Can'we'priori2ze'the'quan2fica2on'of'MTD' techniques?'That'is,'which'MTD'proper2es'are'most' important'to'quan2fy'and'why?' 2.  What'is'in'good'shape'with'respect'to'exis2ng'and' imminent'aVempts'to'quan2fy'those'proper2es?' 3.  What'kind'of'work'is's2ll'missing'and'why'is'it'not'

    being'done?''This'includes'both'empirical'and'analy2c' work'on'exis2ng'MTD'quan2fica2on'ideas'as'well'as' completely'new'scien2fic'approaches'to'MTD' quan2fica2on' ' Thanks'to'all'par2cipants:'GMU,'CMU,'MIT'LL,'DHS,'DoD/ IC,'AFRL,'NSF,'BAE,'Siege,'Dartmouth,'JHU/APL,'UNCC,' USMA,'Apertus,'W&M,'KSU,'BBN,'FIT,'UCR,'ARL'' 2'
  3. Current'scien2fic/engineering'effort' •  Been'doing'MTD'R&D'~2000D'(QSRA,'ABCDDACP'etc)' •  Since'2013,'R'&'D'PoC'An2DFragile'Soeware'System' – Informally'an'an2Dfragile'system'is'a'system'which,' when'stressed,'emerges'stronger'for'the'wear' •  Can'think'of'it'as'next'gen'MTD.'Not'just'resist/ mission'fightDthrough'of'system'v1'but'improve'

    system'v1'to'v2'by'means'of'aVack'info'absorp2on! AFSS:%A%‘system’%that%‘absorbs’%‘stressor'%then% 'recons5tutes’%‘itself’%more%‘resilient'%than%before% •  Words'in'‘'‘'are'placeholders'for'quan2fied'en22es' •  See'also''10y'efforts'CMU'[Helix]'&'UVA'[Rainbow]' 3'
  4. Past'MTDDrelated'Work' •  2000D2003:%Abstract%MTD%a=ack%surface%risk%management' QSRA'Risk'analysis'of'computer'networks'via'mul2Dfactor'risk' metrics,'and'manage'risk'by'soeware'subs2tu2on;'subject'to' cost,'func2onality'and'risk'tolerance'constraints'' ' •  2008D2011:'Concrete%MTD%via%subversion%of%adversarial% decision%structures%%

    ABCDFACP:'Deployed'porlolio'of'‘baits’'(files,'shares,'processes,' etc),'probabilis2cally'iden2fy'suspicious'par2cipants'through' aggregate'suspicious'behavior,'subvert'decision'structure'with' s2muli'and'goad'into'a'posi2on'favorable'to'the'defense' ' •  2011D2014:'Quan5fica5on%/%algo%predic5on%work%at%OS%/CPU% event%/%soNware%surface%level% As'PI'at'defense'contractor'(workflow,'offensive'measurements,' MTD'effec2veness,'scien2fic'quan2fica2on'etc)' 4'
  5. Problems:'Scien2fic'm.o' Reproducibility/(int/ext)%Valida5on%“poor%methods%get% results”! •  Need'to'avoid'going'down'm.o.'road'of'say'psychology.' In'2015' Of!the!100!prominent!papers!analyzed,!only!39%!could!be! replicated!unambiguously! Bohannon'(2015)'“Many'psychology'papers'fail'replica2on'test hVp://www.sciencemag.org/content/349/6251/910.summary'

    •  Similar'depressing'assessments'in'sociology,'medicine'' hVp://www.collec2veDevolu2on.com/2015/05/16/ editorDinDchiefDofDworldsDbestDknownDmedicalDjournalD halfDofDallDtheDliteratureDisDfalse/'''(Bright'spot:' Bioinforma2cs)' 5'
  6. Quan2fied'Computer'Security' •  Status'quo'six'years'ago:' Meta>survey!of!90!papers!between!1981!and!2008! with!respect!to!security!perspecCve,!target!of! quanCficaCon,!underlying!assumpCons!and!type!of! validaCon.!The$result$shows$how$the$validity$of$ most$methods$is$s3ll$strikingly$unclear.!Despite! applying!a!number!of!techniques!from!fields!such!as! computer!science,!economics!and!reliability!theory!

    to!the!problem!it$is$unclear$what$valid$results$exist$ with$respect$to$opera3onal$security! ! Verendel'(2009)'“Quan2fied'Security'is'a'weak'hypothesis”'' 'hVp://publica2ons.lib.chalmers.se/publica2on/108725'' 6'
  7. Need:'MTD'quan2fica2on'as'Science'I' •  Measurement%metadata% – Provenance,'life'decay'rate,'std'error,'confidence' intervals,'proxies'(Syversen)'' •  Solid%experimental%regimes%(pioneer'Maxion/ Killourhy'at'CMU)'with'all'accoutrements' – Simula2on,'runmycode,'baselines,'SPEC'–like' benchmarks/test'suites,'NIST'ACT/CCM,'etc'

    •  Mission%and%Adversary%Parameteriza5on% – Defense'against'whom,'against'which'methods,'for' how'long,'at'what'costs/risks,'etc' % 7'
  8. Need:'MTD'quan2fica2on'as'Science'II' •  No%blind%(poten5ally%spurious)%correla5on%fishing% (even'with'modern'methods'such'as'MINE' [Reshef2011])' –  Educe/validate'with'genera2ve'(noisy)'appropriate' model'(see'[Lipson2015]'Eureqa'talk'wrt'bio'systems)' •  Laws%of%Cyber:'For'cyber'Newton’s'Laws,'need'

    (nonDenerge2c)'invariants'for'conserva2on'laws'(a' la'Noether).'Exists'for'isolated'components,'no' unifying'framework'yet' •  Composi5on%/%differen5al%security%(not%from% scratch)%:%Possible,'but'will'require'smarter'reD engineering'efforts'(eg'LANGSEC)% 8'
  9. Need:'Opera2onal'm.o' Opera5onal%MTD%introduces%(controlled?)%instabili5es% (latencies,%availability)%%&%increases%a=ack%surface% %F'Tom'Longstaff'(JHU'APL)'pointg% 1.  MTD%as%distributed%database%(CAP,'consistency'vs' availability)'from'good'guy'PoV'[Doyle'CSER'2014]')' –  MTD'as'poten2al'selfDDoS' 2. 

    MTD%as%control%system% –  Neuralgic'points'(eg'rendezvous'points)' –  Gabriella'Barrantes'ref'(see'Fig'1'[RoQ07])' 3.  MTD%as%informa5on%leak%/'side'channel'/'asymmetric' adversarial'learning'problem' –  Strict?'On'average?'Don’t'know'yet' –  Quan2fica2on:'NSA'SoS'2014'[Alvim14]'general'leakage'bounds' robust'wrt'opera2onal'scenarios;'generalized'Shannon'channel' capacity' 9'
  10. Assessing'MTD'technique'effec2veness' 10' My'2013'working'metrics'' 'At'WS:'Hamed'Okhravi'(MIT'LL)'suggest'refinement'of'3rd' point'–'predictable'oscilla2ons'between'points'may'be'OK'

  11. LongDterm:'Incen2ve'Structures' 11'

  12. Addi2onal'slides' •  Thanks'for'the'considera2on'of'these'ideas':D' ' •  I’ll'be'happy'to'take'ques2ons'or'comments' •  I'have'a'request/ques2on'for'par2cipants' – IARPA'CyberDaVack'Automated'Unconven2onal' Sensor'Environment'(CAUSE)'please'talk'about'

    public'part'here'a'bit':D' 12'
  13. References'I' •  [ABCD]'Adversarial'Bai2ng'Control'Decep2on hVp://www.docdroid.net/agqz/bilarDfinalDiccc3june2011D slides.pdf.html'' •  [Alvim14]'Informa2on'flow'leakage'bounds hVp://users.cis.fiu.edu/~smithg/papers/csf14.pdf'' •  [BMC13]'Conficker'adversarial'dynamics

    hVps://speakerdeck.com/dbilar/adversarialDdynamicsDconfickerD caseDstudy'' •  [CAP11]'CAP'Perspec2ves' hVps://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf'' •  [Doyle'CSER'14]'Resilience'distributed' hVp://www.cds.caltech.edu/~yw4ng/Files/2014CSER.pdf' •  [Maxion11]'Proper'cybersecurity'science' www.cs.cmu.edu/~maxion/pubs/Maxion12.pdf'' •  [RoQ]'Subversion'/'degrada2on'subsystems' hVp://www.docdroid.net/agqk/bilarDieeespD degrada2onsubsystem.pdf.html'' 13'
  14. References'II' •  [ANT10]'Mechanism'design'meets'CS' hVp://cacm.acm.org/magazines/2010/8/96622D mechanismDdesignDmeetsDcomputerDscience/fulltext'' •  [JL95]'Shi'and'China hVp://www.zonebooks.org/2tles/JULL_PRO.html'' •  [Doyle'CSER'14]'Resilience'distributed'system'

    hVp://www.cds.caltech.edu/~yw4ng/Files/2014CSER.pdf' •  [Reshef2011]'2'var'generalized'assoc'MIC'/'MINE' hVp://www.exploredata.net/'' •  [Helix]'SelfDregenera2ve' hVp://link.springer.com/chapter/ 10.1007%2F978D1D4614D5416D8_7'' •  [Rainbow]'SelfDadapta2on'' hVp://www.springer.com/cda/content/document/ cda_downloaddocument/9780387898278Dc2.pdf? SGWID=0D0D45D734916Dp173871105'' 14'
  15. Appendix' •  Par2cpants/Goals' •  LANGSEC' •  CAP' 15'

  16. 16'

  17. CAP'“uniformity'of'informa2on”' •  What'different'parts'of'a'system'can'agree'upon'at'a'given'moment,'and' whether'that'informa2on'is'available'to'others'or'not,'given'the'effect'of' system'boundaries'(or'"par22ons")'that'prevent'knowledge'from' spreading.' •  The'no2on'of'bringing'about'consistency'hinges'on'the'concept'of' availability,'even'in'the'trivial'case'where'data'are'consistently' unavailable,'so'these'proper2es'are'inseparable.'To'define'availability,'we'

    need'an'independent'measure'of'2me.'Without'availability,'we'cannot' define'"simultaneous"'or'"consistency".' •  All'consistency'is'eventual'in'real'2me,'i.e.'the'user'has'to'wait'for'it'(Re:' ACID'versus'BASE'in'databases).'Distributed'consistency'of'informa2on'is' a'form'of'equilibra2on'of'the'total'system.'This'is'the'same'concept'of' equilibrium'as'in'thermodynamics.' •  Systems%that%are%changing%so%fast%that%informa5on%cannot%travel%to%all% parts%of%the%system%before%another%change%enters,%cannot%be%globally% consistent,%as%equilibra5on%takes%longer%than%this.%This%is%the%tradeFoff% between%availability%and%consistency.' 17' hVp://markburgess.org/blog_cap.html''