Processing Smart City Data in the Cloud with Domain-Specific Languages

Bdcf8af7892cb0147cb22828d37e872f?s=47 Michel Krämer
December 11, 2014

Processing Smart City Data in the Cloud with Domain-Specific Languages

In this presentation that I gave at the Smart City Clouds Workshop co-located with the Utility and Cloud Computing Conference UCC 2014 I talked about a novel user interface that allows domain experts such as urban planners to harness the capabilities of Cloud Computing. The user interface is based on Domain-Specific Languages (DSLs) that are really readable and understandable even for users with no background in computer science.

My DSLs also hide the technical details of Cloud Computing and enable the users to specify what should be done instead of how it should be done.

In this talk I specifically focused on the modelling method that I use to specify new Domain-Specific Languages.

Bdcf8af7892cb0147cb22828d37e872f?s=128

Michel Krämer

December 11, 2014
Tweet

Transcript

  1. PROCESSING SMART CITY DATA IN THE CLOUD WITH DOMAIN-SPECIFIC LANGUAGES

    MICHEL KRÄMER
  2. None
  3. FP7dproject,d2012-2016 http://iqmulus.eu IQmulus PartiallydfundeddbydthedEuropeandCommission

  4. As an urban planner I would like to update my

    existing 3D city model based on analysing recent LIDAR point clouds. " " -- Random IQmulus user
  5. None
  6. None
  7. None
  8. None
  9. WAT?

  10. PROBLEM STATEMENT

  11. PROBLEM STATEMENT Current solutions are not scalable

  12. PROBLEM STATEMENT Cloud is scalable! Current solutions are not scalable

  13. PROBLEM STATEMENT Cloud is scalable! Current UIs are too complicated

    Current solutions are not scalable
  14. PROBLEMySTATEMENT Cloudyis scalable! CurrentyUIsyare tooycomplicated Cloudyis easy! Currentysolutions areynotyscalable

  15. THANKS FOR YOUR ATTENTION! QUESTIONS?

  16. PROBLEMySTATEMENT Cloudyis scalable! CurrentyUIsyare tooycomplicated Cloudyis easy! Currentysolutions areynotyscalable

  17. PROBLEM?STATEMENT Cloud?is scalable! Current?UIs?are too?complicated Current?solutions are?not?scalable ?

  18. None
  19. HOW ABOUT THIS?

  20. with recent PointCloud do exclude NonStaticObjects select added Trees and

    added FacadeElements add to CityModel end
  21. DSL3MODELLING3METHOD 1 Analyse3application3domain 2 Create3user3stories 3 Analyse3user3stories 4 Create3domain3model

  22. 5 Identify relevant verbs 6 Build sample DSL scripts 7

    Derive grammar 8 Review and reiterate
  23. CREATEDUSERDSTORIES 2 AsDanDurbanDplannerDIDwouldDlikeDtoDupdate myDexistingD3DDcityDmodelDbasedDon analysingDrecentDLIDARDpointDclouds. L L --DExample

  24. ANALYSECUSERCSTORIES 3 TopographicCobject UrbanCfurniture LIDARCpointCcloud Tree Car People Roof CableCnetwork

    TrafficClight UrbanCarea 3DCcityCmodel RubbishCbin BusCstop Antenna StreetCedge Image Growth Object Bike FacadeCelement ...
  25. DOMAINdMODEL 4 Topographicdobject NonStaticObject People Bike Car DataSet PointCloud CityModel

    Image ... StaticObject Facade Tree Roof Antenna ...
  26. IDENTIFYzRELEVANTzVERBS 5 capture detect remove exclude create monitor characterize visualise

    update foresee include
  27. BUILD SAMPLE SCRIPTS 6 with recent PointCloud do exclude NonStaticObjects

    select added Trees and added FacadeElements add to CityModel end
  28. BUILD SAMPLE SCRIPTS 6 applyGSplineInterpolation GGGGwithG[PointCloud2010] GGGGusingGformat:GcRGBc GGGGGGGGandGtolerance:G0.3 GGGGGGGGandGiterations:G6 GGGGGGGGasGsurface2010

    applyGTrimSurfaceWithPoints GGGGwithG[PointCloud2010]GandGsurface2010 GGGGusingGsensibility:G7 GGGGasGtrimmedSurface2010
  29. BUILD SAMPLE SCRIPTS 6 withD[SatelliteImage]Ddo DDDDapplyDImageEnhancement DDDDDDDDusingDfactor:D2.0DandDoffset:D0.0 DDDDapplyDFloodedAreaDetection DDDDDDDDusingDupperThresholdWater:D10 DDDDDDDDandDlowerThresholdWater:D0

    DDDDDDDDandDupperThresholdSoilHigh:D11 DDDDDDDDandDupperThresholdSoilMedium:D13 DDDDDDDDandDupperThresholdSoil:D15 end
  30. DERIVE GRAMMAR 7 start = SP* statements SP* statements =

    statement ( SP+ statement )* statement = block / process block = with SP+ statements SP+ "end" with = "with" SP+ dataset SP+ "do" dataset = "recent" SP+ ID / ID ...
  31. DERIVE GRAMMAR (FINAL) 7 { <<var<ast'uilder<R<options1ast'uilderE <<function<asty_<{ <<<<return<ast'uilder1posyliney_*<columny__E <<} <<function<extract9istylist*<index_<{

    <<<<var<result<R<[]E <<<<for<yvar<i<R<WE<i<F<list1lengthE<PPi_<{ <<<<<<result1pushylist[i][index]_E <<<<} <<<<return<resultE <<} <<function<build9istyfirst*<rest*<index_<{ <<<<return<[first]1concatyextract9istyrest*<index__E <<} } start <<R<workflow<?<empty_workflow empty_workflow <<R<SPS<{ <<<<return<asty_1workflowy[]_E <<} workflow <<R<SPS<sOstatements<SPS<{ <<<<return<asty_1workflowys_E <<} statements <<R<firstOstatement<moreOy<SPP<statement<_S<{ <<<<return<build9istyfirst*<more*<I_E <<} statement <<R<with <<?<process_statement process_statement <<R<pOprocess<datasetsOprocess_statement_withM <<<<paramsOprocess_statement_usingM<nameOprocess_statement_asM<{ <<<<return<asty_1processyp*<datasets*<params*<name_E <<} process_statement_with <<R<SPP<WYTZ<SPP<firstOdataset_expression<moreOy <<<<SPP<jN!<SPP<dataset_expression<_S<{ <<<<return<build9istyfirst*<more*<H_E <<} process_statement_using <<R<SPP<USYNV<SPP<pOparams<{ <<<<return<pE <<} process_statement_as <<R<SPP<jS<SPP<nOname<{ <<<<return<nE <<} with <<R<WYTZ<SPP<dsOdataset_expression<SPP<!O<SPP <<<<sOstatements<SPP<\N!<{ <<<<return<asty_1withyds*<s_E <<} dataset_expression <<R<dsOdataset<y<SPP<Oz<SPP<dataset<_M<{ <<<<return<dsE <<} recent <<R<R\-\NT<SPP<nameONjK\<{ <<<<return<asty_1recentyname_E <<} latest <<R<9jT\ST<SPP<nameONjK\<{ <<<<return<asty_1latestyname_E <<} dataset <<R<recent <<?<latest <<?<placeholder <<?<NjK\ params <<R<firstOparam<moreOy<SPP<jN!<SPP<param<_S<{ <<<<return<build9istyfirst*<more*<H_E <<} param <<R<h{h<SPS<tuple_ids<SPS<hOh<SPS<tuple_expression<SPS<h}h <<?<keyONjK\<SPS<hOh<SPS<valueOexpression<{ <<<<return<asty_1paramykey*<value_E <<} tuple_expression <<R<expression<y<h*h<SPS<expression<_S expression <<R<placeholder <<?<NUK'\R <<?<string <<?<NjK\ tuple_ids <<R<NjK\<y<h*h<SPS<NjK\<_S placeholder <<R<h[h<nameONjK\<h]h<{ <<<<return<asty_1placeholderyname_E <<} ref <<R<nONjK\<SPS<h1h<SPS<pOref<{<return<asty_1objectRefyn*<p_E<} <<?<nONjK\<{<return<asty_1refyn_E<} name <<R<nONjK\<{ <<<<return<asty_1nameyn_E <<} string <<R</h/<sOstring_characterS</h/<{ <<<<return<texty_1sliceyI*<:I_E <<} <<?<h/h<sOstring_characterS<h/h<{ <<<<return<texty_1sliceyI*<:I_E <<} string_character <<R<g[h\\\r\n]<1 <<?<h\\h<\S-jP\_-ZjRj-T\R store_process <<R<STOR\<rOySPP<ref_M<dsOySPP<TO<SPP<dataset_M<{ <<<<if<yr_<{ <<<<<<r<R<r[I]E <<<<} <<<<if<yds_<{ <<<<<<ds<R<ds[H]E <<<<} <<<<return<asty_1storeyds*<r_E <<} visualize_process <<R<VYSUj9YZ\<{ <<<<return<asty_1visualizey_E <<} apply_process <<R<y<jPP9Y<?<-j9-U9jT\<_<SPP<nameONjK\<{ <<<<return<asty_1applyyname_E <<} process <<R<apply_process <<?<store_process <<?<visualize_process <<?<urban_process <<?<marine_process <<?<land_process add_process <<R<j!!<SPP<TO<SPP<dsOdataset<{ <<<<return<asty_1addyds_E <<} <<?<j!!<SPP<rOref<SPP<TO<SPP<dsOdataset<{ <<<<return<asty_1addyds*<r_E <<} exclude_process <<R<\X-9U!\<y<SPP<urban_dataset_param<_M<SPP<nameONjK\<{ <<<<return<asty_1excludeyname_E <<} urban_process <<R<add_process <<?<exclude_process <<?<XOYN<SPP<NjK\<y<SPP<jN!<SPP<NjK\<_S <<?<S\9\-T<SPP<urban_dataset_param<SPP<NjK\<y<SPP <<<<jN!<SPP<urban_dataset_param<SPP<NjK\<_S urban_dataset_param <<R<j!!\! <<?<j99 <<?<-\RTjYN <<?<!\zORK\! marine_process <<R<V\N\RjT\<SPP<NjK\ land_process <<R<-O9ORYZ\<SPP<NjK\<SPP<dataset ?SS<NjK\S?Y!\NTYzY\RS<SSSSSSSSSSS? NjK\ <<R<gReservedWord<firstO[_a:zj:Z]<moreONjK\_KOR\S<{ <<<<return<first<P<more1joinyhh_E <<} NjK\_KOR\ <<R<[_a:zj:ZW:G] ?SS<R\S\RV\!<WOR!S<SSSSSSSSSSSSSS? ReservedWord <<R<Jeyword Jeyword <<R<j!! <<?<j!!\! <<?<j99 <<?<jN! <<?<jPP9Y <<?<jS <<?<-j9-U9jT\ <<?<-\RTjYN <<?<-O9ORYZ\ <<?<!\zORK\! <<?<!O <<?<\N! <<?<\X-9U!\ <<?<V\N\RjT\ <<?<XOYN <<?<9jT\ST <<?<Oz <<?<R\-\NT <<?<S\9\-T <<?<STOR\ <<?<TO <<?<USYNV <<?<VYSUj9YZ\ <<?<WYTZ ?SS<TOJ\NS<SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS? NUK'\R <<R<[W:G]P<y<h1h<[W:G]P<_M<{ <<<<return<parsezloatytexty__E <<} \S-jP\_-ZjRj-T\R <<R<[/h\\bfnrtv] SP <<R<[<\t\n\r] ?SS<J\YWOR!<TOJ\NS<SSSSSSSSSSSSSSSSSSSSSSSSSS? j!!<<<<<<<<<<<<R<haddh<<<<<<<<<<<<gNjK\_KOR\ j!!\!<<<<<<<<<<R<haddedh<<<<<<<<<<gNjK\_KOR\ j99<<<<<<<<<<<<R<hallh<<<<<<<<<<<<gNjK\_KOR\ jN!<<<<<<<<<<<<R<handh<<<<<<<<<<<<gNjK\_KOR\ jPP9Y<<<<<<<<<<R<happlyh<<<<<<<<<<gNjK\_KOR\ jS<<<<<<<<<<<<<R<hash<<<<<<<<<<<<<gNjK\_KOR\ -j9-U9jT\<<<<<<R<hcalculateh<<<<<<gNjK\_KOR\ -\RTjYN<<<<<<<<R<hcertainh<<<<<<<<gNjK\_KOR\ -O9ORYZ\<<<<<<<R<hcolorizeh<<<<<<<gNjK\_KOR\ <<<<<<<<<<<<<<<?<hcolouriseh<<<<<<gNjK\_KOR\ !\zORK\!<<<<<<<R<hdeformedh<<<<<<<gNjK\_KOR\ !O<<<<<<<<<<<<<R<hdoh<<<<<<<<<<<<<gNjK\_KOR\ \N!<<<<<<<<<<<<R<hendh<<<<<<<<<<<<gNjK\_KOR\ \X-9U!\<<<<<<<<R<hexcludeh<<<<<<<<gNjK\_KOR\ V\N\RjT\<<<<<<<R<hgenerateh<<<<<<<gNjK\_KOR\ XOYN<<<<<<<<<<<R<hjoinh<<<<<<<<<<<gNjK\_KOR\ 9jT\ST<<<<<<<<<R<hlatesth<<<<<<<<<gNjK\_KOR\ Oz<<<<<<<<<<<<<R<hofh<<<<<<<<<<<<<gNjK\_KOR\ R\-\NT<<<<<<<<<R<hrecenth<<<<<<<<<gNjK\_KOR\ S\9\-T<<<<<<<<<R<hselecth<<<<<<<<<gNjK\_KOR\ STOR\<<<<<<<<<<R<hstoreh<<<<<<<<<<gNjK\_KOR\ TO<<<<<<<<<<<<<R<htoh<<<<<<<<<<<<<gNjK\_KOR\ USYNV<<<<<<<<<<R<husingh<<<<<<<<<<gNjK\_KOR\ VYSUj9YZ\<<<<<<R<hvisualizeh<<<<<<gNjK\_KOR\ <<<<<<<<<<<<<<<?<hvisualiseh<<<<<<gNjK\_KOR\ WYTZ<<<<<<<<<<<R<hwithh<<<<<<<<<<<gNjK\_KOR\
  32. Mapping2rules Script Parser Abstract2syntax2tree Interpreter ... SCRIPT2EXECUTION Service22 Service21 Service2n

  33. SCRIPT EXECUTION exclude recent CityModel Trees Filter File A Feature

    extraction Tree classifier
  34. FURTHERwREAD Krämer1wMichelwandwSenner1wIvo Awmodularwsoftwarewarchitecturewfor processingwofwbigwgeospatialwdatawinwthewcloud ComputerswGwGraphics:wSpecialwIssuewonwProcessing LargewGeospatialwData1wElsevier1w2015wPtowbewpublishedL

  35. CONCLUSION Cloud is scalable! Current UIs are too complicated Cloud

    is easy! Current solutions are not scalable
  36. THANKS FOR YOUR ATTENTION! QUESTIONS?

  37. Michel3Krämer Fraunhofer3Institute3for Computer3Graphics3Research3IGD Fraunhoferstraße35 642833Darmstadt,3Germany michel.kraemer@igd.fraunhofer.de +493615131553415