Moving Target Defense Quantification

Can we prioritize the quantification of MTD techniques? That is, which MTD properties are most important to quantify and why?
What is in good shape with respect to existing and imminent attempts to quantify those properties?
What kind of work is still missing and why is it not being done? This includes both empirical and analytic work on existing MTD quantification ideas as well as completely new scientific approaches to MTD quantification

Transcript

1. Moving'Target'Defense' Quan2fica2on'Workshop' GMU,'Fairfax'(VA)' Aug'31D'Sep'1,'2015' ' Daniel'Bilar' dbilar@acm.org'
2. Some'WS'ques2ons,'varia' 1.  Can'we'priori2ze'the'quan2fica2on'of'MTD' techniques?'That'is,'which'MTD'proper2es'are'most' important'to'quan2fy'and'why?' 2.  What'is'in'good'shape'with'respect'to'exis2ng'and' imminent'aVempts'to'quan2fy'those'proper2es?' 3.  What'kind'of'work'is's2ll'missing'and'why'is'it'not' being'done?''This'includes'both'empirical'and'analy2c' work'on'exis2ng'MTD'quan2fica2on'ideas'as'well'as' completely'new'scien2fic'approaches'to'MTD' quan2fica2on' ' Thanks'to'all'par2cipants:'GMU,'CMU,'MIT'LL,'DHS,'DoD/ IC,'AFRL,'NSF,'BAE,'Siege,'Dartmouth,'JHU/APL,'UNCC,' USMA,'Apertus,'W&M,'KSU,'BBN,'FIT,'UCR,'ARL'' 2'
3. Current'scien2fic/engineering'effort' •  Been'doing'MTD'R&D'~2000D'(QSRA,'ABCDDACP'etc)' •  Since'2013,'R'&'D'PoC'An2DFragile'Soeware'System' – Informally'an'an2Dfragile'system'is'a'system'which,' when'stressed,'emerges'stronger'for'the'wear' •  Can'think'of'it'as'next'gen'MTD.'Not'just'resist/ mission'fightDthrough'of'system'v1'but'improve' system'v1'to'v2'by'means'of'aVack'info'absorp2on! AFSS:%A%‘system’%that%‘absorbs’%‘stressor'%then% 'recons5tutes’%‘itself’%more%‘resilient'%than%before% •  Words'in'‘'‘'are'placeholders'for'quan2fied'en22es' •  See'also''10y'efforts'CMU'[Helix]'&'UVA'[Rainbow]' 3'
4. Past'MTDDrelated'Work' •  2000D2003:%Abstract%MTD%a=ack%surface%risk%management' QSRA'Risk'analysis'of'computer'networks'via'mul2Dfactor'risk' metrics,'and'manage'risk'by'soeware'subs2tu2on;'subject'to' cost,'func2onality'and'risk'tolerance'constraints'' ' •  2008D2011:'Concrete%MTD%via%subversion%of%adversarial% decision%structures%% ABCDFACP:'Deployed'porlolio'of'‘baits’'(files,'shares,'processes,' etc),'probabilis2cally'iden2fy'suspicious'par2cipants'through' aggregate'suspicious'behavior,'subvert'decision'structure'with' s2muli'and'goad'into'a'posi2on'favorable'to'the'defense' ' •  2011D2014:'Quan5fica5on%/%algo%predic5on%work%at%OS%/CPU% event%/%soNware%surface%level% As'PI'at'defense'contractor'(workflow,'offensive'measurements,' MTD'effec2veness,'scien2fic'quan2fica2on'etc)' 4'
5. Problems:'Scien2fic'm.o' Reproducibility/(int/ext)%Valida5on%“poor%methods%get% results”! •  Need'to'avoid'going'down'm.o.'road'of'say'psychology.' In'2015' Of!the!100!prominent!papers!analyzed,!only!39%!could!be! replicated!unambiguously! Bohannon'(2015)'“Many'psychology'papers'fail'replica2on'test hVp://www.sciencemag.org/content/349/6251/910.summary' •  Similar'depressing'assessments'in'sociology,'medicine'' hVp://www.collec2veDevolu2on.com/2015/05/16/ editorDinDchiefDofDworldsDbestDknownDmedicalDjournalD halfDofDallDtheDliteratureDisDfalse/'''(Bright'spot:' Bioinforma2cs)' 5'
6. Quan2fied'Computer'Security' •  Status'quo'six'years'ago:' Meta>survey!of!90!papers!between!1981!and!2008! with!respect!to!security!perspecCve,!target!of! quanCficaCon,!underlying!assumpCons!and!type!of! validaCon.!The$result$shows$how$the$validity$of$ most$methods$is$s3ll$strikingly$unclear.!Despite! applying!a!number!of!techniques!from!fields!such!as! computer!science,!economics!and!reliability!theory! to!the!problem!it$is$unclear$what$valid$results$exist$ with$respect$to$opera3onal$security! ! Verendel'(2009)'“Quan2fied'Security'is'a'weak'hypothesis”'' 'hVp://publica2ons.lib.chalmers.se/publica2on/108725'' 6'
7. Need:'MTD'quan2fica2on'as'Science'I' •  Measurement%metadata% – Provenance,'life'decay'rate,'std'error,'confidence' intervals,'proxies'(Syversen)'' •  Solid%experimental%regimes%(pioneer'Maxion/ Killourhy'at'CMU)'with'all'accoutrements' – Simula2on,'runmycode,'baselines,'SPEC'–like' benchmarks/test'suites,'NIST'ACT/CCM,'etc' •  Mission%and%Adversary%Parameteriza5on% – Defense'against'whom,'against'which'methods,'for' how'long,'at'what'costs/risks,'etc' % 7'
8. Need:'MTD'quan2fica2on'as'Science'II' •  No%blind%(poten5ally%spurious)%correla5on%fishing% (even'with'modern'methods'such'as'MINE' [Reshef2011])' –  Educe/validate'with'genera2ve'(noisy)'appropriate' model'(see'[Lipson2015]'Eureqa'talk'wrt'bio'systems)' •  Laws%of%Cyber:'For'cyber'Newton’s'Laws,'need' (nonDenerge2c)'invariants'for'conserva2on'laws'(a' la'Noether).'Exists'for'isolated'components,'no' unifying'framework'yet' •  Composi5on%/%differen5al%security%(not%from% scratch)%:%Possible,'but'will'require'smarter'reD engineering'efforts'(eg'LANGSEC)% 8'
9. Need:'Opera2onal'm.o' Opera5onal%MTD%introduces%(controlled?)%instabili5es% (latencies,%availability)%%&%increases%a=ack%surface% %F'Tom'Longstaff'(JHU'APL)'pointg% 1.  MTD%as%distributed%database%(CAP,'consistency'vs' availability)'from'good'guy'PoV'[Doyle'CSER'2014]')' –  MTD'as'poten2al'selfDDoS' 2.  MTD%as%control%system% –  Neuralgic'points'(eg'rendezvous'points)' –  Gabriella'Barrantes'ref'(see'Fig'1'[RoQ07])' 3.  MTD%as%informa5on%leak%/'side'channel'/'asymmetric' adversarial'learning'problem' –  Strict?'On'average?'Don’t'know'yet' –  Quan2fica2on:'NSA'SoS'2014'[Alvim14]'general'leakage'bounds' robust'wrt'opera2onal'scenarios;'generalized'Shannon'channel' capacity' 9'
10. Assessing'MTD'technique'effec2veness' 10' My'2013'working'metrics'' 'At'WS:'Hamed'Okhravi'(MIT'LL)'suggest'refinement'of'3rd' point'–'predictable'oscilla2ons'between'points'may'be'OK'
11. LongDterm:'Incen2ve'Structures' 11'
12. Addi2onal'slides' •  Thanks'for'the'considera2on'of'these'ideas':D' ' •  I’ll'be'happy'to'take'ques2ons'or'comments' •  I'have'a'request/ques2on'for'par2cipants' – IARPA'CyberDaVack'Automated'Unconven2onal' Sensor'Environment'(CAUSE)'please'talk'about' public'part'here'a'bit':D' 12'
13. References'I' •  [ABCD]'Adversarial'Bai2ng'Control'Decep2on hVp://www.docdroid.net/agqz/bilarDfinalDiccc3june2011D slides.pdf.html'' •  [Alvim14]'Informa2on'flow'leakage'bounds hVp://users.cis.fiu.edu/~smithg/papers/csf14.pdf'' •  [BMC13]'Conficker'adversarial'dynamics hVps://speakerdeck.com/dbilar/adversarialDdynamicsDconfickerD caseDstudy'' •  [CAP11]'CAP'Perspec2ves' hVps://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf'' •  [Doyle'CSER'14]'Resilience'distributed' hVp://www.cds.caltech.edu/~yw4ng/Files/2014CSER.pdf' •  [Maxion11]'Proper'cybersecurity'science' www.cs.cmu.edu/~maxion/pubs/Maxion12.pdf'' •  [RoQ]'Subversion'/'degrada2on'subsystems' hVp://www.docdroid.net/agqk/bilarDieeespD degrada2onsubsystem.pdf.html'' 13'
14. References'II' •  [ANT10]'Mechanism'design'meets'CS' hVp://cacm.acm.org/magazines/2010/8/96622D mechanismDdesignDmeetsDcomputerDscience/fulltext'' •  [JL95]'Shi'and'China hVp://www.zonebooks.org/2tles/JULL_PRO.html'' •  [Doyle'CSER'14]'Resilience'distributed'system' hVp://www.cds.caltech.edu/~yw4ng/Files/2014CSER.pdf' •  [Reshef2011]'2'var'generalized'assoc'MIC'/'MINE' hVp://www.exploredata.net/'' •  [Helix]'SelfDregenera2ve' hVp://link.springer.com/chapter/ 10.1007%2F978D1D4614D5416D8_7'' •  [Rainbow]'SelfDadapta2on'' hVp://www.springer.com/cda/content/document/ cda_downloaddocument/9780387898278Dc2.pdf? SGWID=0D0D45D734916Dp173871105'' 14'
15. Appendix' •  Par2cpants/Goals' •  LANGSEC' •  CAP' 15'
16. 16'
17. CAP'“uniformity'of'informa2on”' •  What'different'parts'of'a'system'can'agree'upon'at'a'given'moment,'and' whether'that'informa2on'is'available'to'others'or'not,'given'the'effect'of' system'boundaries'(or'"par22ons")'that'prevent'knowledge'from' spreading.' •  The'no2on'of'bringing'about'consistency'hinges'on'the'concept'of' availability,'even'in'the'trivial'case'where'data'are'consistently' unavailable,'so'these'proper2es'are'inseparable.'To'define'availability,'we' need'an'independent'measure'of'2me.'Without'availability,'we'cannot' define'"simultaneous"'or'"consistency".' •  All'consistency'is'eventual'in'real'2me,'i.e.'the'user'has'to'wait'for'it'(Re:' ACID'versus'BASE'in'databases).'Distributed'consistency'of'informa2on'is' a'form'of'equilibra2on'of'the'total'system.'This'is'the'same'concept'of' equilibrium'as'in'thermodynamics.' •  Systems%that%are%changing%so%fast%that%informa5on%cannot%travel%to%all% parts%of%the%system%before%another%change%enters,%cannot%be%globally% consistent,%as%equilibra5on%takes%longer%than%this.%This%is%the%tradeFoff% between%availability%and%consistency.' 17' hVp://markburgess.org/blog_cap.html''