Share
Embed
Moving Target Defense Quantification
Can we prioritize the quantification of MTD techniques? That is, which MTD properties are most important to quantify and why?
What is in good shape with respect to existing and imminent attempts to quantify those properties?
What kind of work is still missing and why is it not being done? This includes both empirical and analytic work on existing MTD quantification ideas as well as completely new scientific approaches to MTD quantification
Daniyel Yaacov
August 31, 2015
Transcript
17 slides
-
Some'WS'ques2ons,'varia' 1. Can'we'priori2ze'the'quan2fica2on'of'MTD' techniques?'That'is,'which'MTD'proper2es'are'most' important'to'quan2fy'and'why?' 2. What'is'in'good'shape'with'respect'to'exis2ng'and' imminent'aVempts'to'quan2fy'those'proper2es?' 3. What'kind'of'work'is's2ll'missing'and'why'is'it'not'
being'done?''This'includes'both'empirical'and'analy2c' work'on'exis2ng'MTD'quan2fica2on'ideas'as'well'as' completely'new'scien2fic'approaches'to'MTD' quan2fica2on' ' Thanks'to'all'par2cipants:'GMU,'CMU,'MIT'LL,'DHS,'DoD/ IC,'AFRL,'NSF,'BAE,'Siege,'Dartmouth,'JHU/APL,'UNCC,' USMA,'Apertus,'W&M,'KSU,'BBN,'FIT,'UCR,'ARL'' 2' -
Current'scien2fic/engineering'effort' • Been'doing'MTD'R&D'~2000D'(QSRA,'ABCDDACP'etc)' • Since'2013,'R'&'D'PoC'An2DFragile'Soeware'System' – Informally'an'an2Dfragile'system'is'a'system'which,' when'stressed,'emerges'stronger'for'the'wear' • Can'think'of'it'as'next'gen'MTD.'Not'just'resist/ mission'fightDthrough'of'system'v1'but'improve'
system'v1'to'v2'by'means'of'aVack'info'absorp2on! AFSS:%A%‘system’%that%‘absorbs’%‘stressor'%then% 'recons5tutes’%‘itself’%more%‘resilient'%than%before% • Words'in'‘'‘'are'placeholders'for'quan2fied'en22es' • See'also''10y'efforts'CMU'[Helix]'&'UVA'[Rainbow]' 3' -
Past'MTDDrelated'Work' • 2000D2003:%Abstract%MTD%a=ack%surface%risk%management' QSRA'Risk'analysis'of'computer'networks'via'mul2Dfactor'risk' metrics,'and'manage'risk'by'soeware'subs2tu2on;'subject'to' cost,'func2onality'and'risk'tolerance'constraints'' ' • 2008D2011:'Concrete%MTD%via%subversion%of%adversarial% decision%structures%%
ABCDFACP:'Deployed'porlolio'of'‘baits’'(files,'shares,'processes,' etc),'probabilis2cally'iden2fy'suspicious'par2cipants'through' aggregate'suspicious'behavior,'subvert'decision'structure'with' s2muli'and'goad'into'a'posi2on'favorable'to'the'defense' ' • 2011D2014:'Quan5fica5on%/%algo%predic5on%work%at%OS%/CPU% event%/%soNware%surface%level% As'PI'at'defense'contractor'(workflow,'offensive'measurements,' MTD'effec2veness,'scien2fic'quan2fica2on'etc)' 4' -
Problems:'Scien2fic'm.o' Reproducibility/(int/ext)%Valida5on%“poor%methods%get% results”! • Need'to'avoid'going'down'm.o.'road'of'say'psychology.' In'2015' Of!the!100!prominent!papers!analyzed,!only!39%!could!be! replicated!unambiguously! Bohannon'(2015)'“Many'psychology'papers'fail'replica2on'test hVp://www.sciencemag.org/content/349/6251/910.summary'
• Similar'depressing'assessments'in'sociology,'medicine'' hVp://www.collec2veDevolu2on.com/2015/05/16/ editorDinDchiefDofDworldsDbestDknownDmedicalDjournalD halfDofDallDtheDliteratureDisDfalse/'''(Bright'spot:' Bioinforma2cs)' 5' -
Quan2fied'Computer'Security' • Status'quo'six'years'ago:' Meta>survey!of!90!papers!between!1981!and!2008! with!respect!to!security!perspecCve,!target!of! quanCficaCon,!underlying!assumpCons!and!type!of! validaCon.!The$result$shows$how$the$validity$of$ most$methods$is$s3ll$strikingly$unclear.!Despite! applying!a!number!of!techniques!from!fields!such!as! computer!science,!economics!and!reliability!theory!
to!the!problem!it$is$unclear$what$valid$results$exist$ with$respect$to$opera3onal$security! ! Verendel'(2009)'“Quan2fied'Security'is'a'weak'hypothesis”'' 'hVp://publica2ons.lib.chalmers.se/publica2on/108725'' 6' -
Need:'MTD'quan2fica2on'as'Science'I' • Measurement%metadata% – Provenance,'life'decay'rate,'std'error,'confidence' intervals,'proxies'(Syversen)'' • Solid%experimental%regimes%(pioneer'Maxion/ Killourhy'at'CMU)'with'all'accoutrements' – Simula2on,'runmycode,'baselines,'SPEC'–like' benchmarks/test'suites,'NIST'ACT/CCM,'etc'
• Mission%and%Adversary%Parameteriza5on% – Defense'against'whom,'against'which'methods,'for' how'long,'at'what'costs/risks,'etc' % 7' -
Need:'MTD'quan2fica2on'as'Science'II' • No%blind%(poten5ally%spurious)%correla5on%fishing% (even'with'modern'methods'such'as'MINE' [Reshef2011])' – Educe/validate'with'genera2ve'(noisy)'appropriate' model'(see'[Lipson2015]'Eureqa'talk'wrt'bio'systems)' • Laws%of%Cyber:'For'cyber'Newton’s'Laws,'need'
(nonDenerge2c)'invariants'for'conserva2on'laws'(a' la'Noether).'Exists'for'isolated'components,'no' unifying'framework'yet' • Composi5on%/%differen5al%security%(not%from% scratch)%:%Possible,'but'will'require'smarter'reD engineering'efforts'(eg'LANGSEC)% 8' -
Need:'Opera2onal'm.o' Opera5onal%MTD%introduces%(controlled?)%instabili5es% (latencies,%availability)%%&%increases%a=ack%surface% %F'Tom'Longstaff'(JHU'APL)'pointg% 1. MTD%as%distributed%database%(CAP,'consistency'vs' availability)'from'good'guy'PoV'[Doyle'CSER'2014]')' – MTD'as'poten2al'selfDDoS' 2.
MTD%as%control%system% – Neuralgic'points'(eg'rendezvous'points)' – Gabriella'Barrantes'ref'(see'Fig'1'[RoQ07])' 3. MTD%as%informa5on%leak%/'side'channel'/'asymmetric' adversarial'learning'problem' – Strict?'On'average?'Don’t'know'yet' – Quan2fica2on:'NSA'SoS'2014'[Alvim14]'general'leakage'bounds' robust'wrt'opera2onal'scenarios;'generalized'Shannon'channel' capacity' 9' -
References'I' • [ABCD]'Adversarial'Bai2ng'Control'Decep2on hVp://www.docdroid.net/agqz/bilarDfinalDiccc3june2011D slides.pdf.html'' • [Alvim14]'Informa2on'flow'leakage'bounds hVp://users.cis.fiu.edu/~smithg/papers/csf14.pdf'' • [BMC13]'Conficker'adversarial'dynamics
hVps://speakerdeck.com/dbilar/adversarialDdynamicsDconfickerD caseDstudy'' • [CAP11]'CAP'Perspec2ves' hVps://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf'' • [Doyle'CSER'14]'Resilience'distributed' hVp://www.cds.caltech.edu/~yw4ng/Files/2014CSER.pdf' • [Maxion11]'Proper'cybersecurity'science' www.cs.cmu.edu/~maxion/pubs/Maxion12.pdf'' • [RoQ]'Subversion'/'degrada2on'subsystems' hVp://www.docdroid.net/agqk/bilarDieeespD degrada2onsubsystem.pdf.html'' 13' -
References'II' • [ANT10]'Mechanism'design'meets'CS' hVp://cacm.acm.org/magazines/2010/8/96622D mechanismDdesignDmeetsDcomputerDscience/fulltext'' • [JL95]'Shi'and'China hVp://www.zonebooks.org/2tles/JULL_PRO.html'' • [Doyle'CSER'14]'Resilience'distributed'system'
hVp://www.cds.caltech.edu/~yw4ng/Files/2014CSER.pdf' • [Reshef2011]'2'var'generalized'assoc'MIC'/'MINE' hVp://www.exploredata.net/'' • [Helix]'SelfDregenera2ve' hVp://link.springer.com/chapter/ 10.1007%2F978D1D4614D5416D8_7'' • [Rainbow]'SelfDadapta2on'' hVp://www.springer.com/cda/content/document/ cda_downloaddocument/9780387898278Dc2.pdf? SGWID=0D0D45D734916Dp173871105'' 14' -
CAP'“uniformity'of'informa2on”' • What'different'parts'of'a'system'can'agree'upon'at'a'given'moment,'and' whether'that'informa2on'is'available'to'others'or'not,'given'the'effect'of' system'boundaries'(or'"par22ons")'that'prevent'knowledge'from' spreading.' • The'no2on'of'bringing'about'consistency'hinges'on'the'concept'of' availability,'even'in'the'trivial'case'where'data'are'consistently' unavailable,'so'these'proper2es'are'inseparable.'To'define'availability,'we'
need'an'independent'measure'of'2me.'Without'availability,'we'cannot' define'"simultaneous"'or'"consistency".' • All'consistency'is'eventual'in'real'2me,'i.e.'the'user'has'to'wait'for'it'(Re:' ACID'versus'BASE'in'databases).'Distributed'consistency'of'informa2on'is' a'form'of'equilibra2on'of'the'total'system.'This'is'the'same'concept'of' equilibrium'as'in'thermodynamics.' • Systems%that%are%changing%so%fast%that%informa5on%cannot%travel%to%all% parts%of%the%system%before%another%change%enters,%cannot%be%globally% consistent,%as%equilibra5on%takes%longer%than%this.%This%is%the%tradeFoff% between%availability%and%consistency.' 17' hVp://markburgess.org/blog_cap.html''