Processing Smart City Data in the Cloud with Domain-Specific Languages

In this presentation that I gave at the Smart City Clouds Workshop co-located with the Utility and Cloud Computing Conference UCC 2014 I talked about a novel user interface that allows domain experts such as urban planners to harness the capabilities of Cloud Computing. The user interface is based on Domain-Specific Languages (DSLs) that are really readable and understandable even for users with no background in computer science.

My DSLs also hide the technical details of Cloud Computing and enable the users to specify what should be done instead of how it should be done.

In this talk I specifically focused on the modelling method that I use to specify new Domain-Specific Languages.

Transcript

1. PROCESSING SMART CITY DATA IN THE CLOUD WITH
   DOMAIN-SPECIFIC LANGUAGES
   MICHEL KRÄMER
   

   View Slide

2. View Slide

3. FP7dproject,d2012-2016 http://iqmulus.eu
   IQmulus
   PartiallydfundeddbydthedEuropeandCommission
   

   View Slide

4. As an urban planner I would like to update
   my existing 3D city model based on
   analysing recent LIDAR point clouds.
   "
   "
   -- Random IQmulus user
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. WAT?
   

   View Slide

10. PROBLEM STATEMENT
    

    View Slide

11. PROBLEM STATEMENT
    Current solutions
    are not scalable
    

    View Slide

12. PROBLEM STATEMENT
    Cloud is
    scalable!
    Current solutions
    are not scalable
    

    View Slide

13. PROBLEM STATEMENT
    Cloud is
    scalable!
    Current UIs are
    too complicated
    Current solutions
    are not scalable
    

    View Slide

14. PROBLEMySTATEMENT
    Cloudyis
    scalable!
    CurrentyUIsyare
    tooycomplicated
    Cloudyis
    easy!
    Currentysolutions
    areynotyscalable
    

    View Slide

15. THANKS
    FOR YOUR ATTENTION!
    QUESTIONS?
    

    View Slide

16. PROBLEMySTATEMENT
    Cloudyis
    scalable!
    CurrentyUIsyare
    tooycomplicated
    Cloudyis
    easy!
    Currentysolutions
    areynotyscalable
    

    View Slide

17. PROBLEM?STATEMENT
    Cloud?is
    scalable!
    Current?UIs?are
    too?complicated
    Current?solutions
    are?not?scalable
    ?
    

    View Slide

18. View Slide

19. HOW ABOUT THIS?
    

    View Slide

20. with recent PointCloud do
    exclude NonStaticObjects
    select added Trees and
    added FacadeElements
    add to CityModel
    end
    

    View Slide

21. DSL3MODELLING3METHOD
    1 Analyse3application3domain
    2 Create3user3stories
    3 Analyse3user3stories
    4 Create3domain3model
    

    View Slide

22. 5 Identify relevant verbs
    6 Build sample DSL scripts
    7 Derive grammar
    8 Review and reiterate
    

    View Slide

23. CREATEDUSERDSTORIES
    2
    AsDanDurbanDplannerDIDwouldDlikeDtoDupdate
    myDexistingD3DDcityDmodelDbasedDon
    analysingDrecentDLIDARDpointDclouds.
    L
    L
    --DExample
    

    View Slide

24. ANALYSECUSERCSTORIES
    3
    TopographicCobject
    UrbanCfurniture
    LIDARCpointCcloud
    Tree
    Car
    People
    Roof
    CableCnetwork
    TrafficClight
    UrbanCarea
    3DCcityCmodel
    RubbishCbin
    BusCstop
    Antenna
    StreetCedge
    Image
    Growth
    Object
    Bike
    FacadeCelement
    ...
    

    View Slide

25. DOMAINdMODEL
    4
    Topographicdobject
    NonStaticObject
    People
    Bike
    Car
    DataSet
    PointCloud
    CityModel
    Image
    ...
    StaticObject
    Facade Tree
    Roof
    Antenna ...
    

    View Slide

26. IDENTIFYzRELEVANTzVERBS
    5
    capture
    detect
    remove
    exclude
    create
    monitor
    characterize
    visualise
    update
    foresee
    include
    

    View Slide

27. BUILD SAMPLE SCRIPTS
    6
    with recent PointCloud do
    exclude NonStaticObjects
    select added Trees and
    added FacadeElements
    add to CityModel
    end
    

    View Slide

28. BUILD SAMPLE SCRIPTS
    6
    applyGSplineInterpolation
    GGGGwithG[PointCloud2010]
    GGGGusingGformat:GcRGBc
    GGGGGGGGandGtolerance:G0.3
    GGGGGGGGandGiterations:G6
    GGGGGGGGasGsurface2010
    applyGTrimSurfaceWithPoints
    GGGGwithG[PointCloud2010]GandGsurface2010
    GGGGusingGsensibility:G7
    GGGGasGtrimmedSurface2010
    

    View Slide

29. BUILD SAMPLE SCRIPTS
    6
    withD[SatelliteImage]Ddo
    DDDDapplyDImageEnhancement
    DDDDDDDDusingDfactor:D2.0DandDoffset:D0.0
    DDDDapplyDFloodedAreaDetection
    DDDDDDDDusingDupperThresholdWater:D10
    DDDDDDDDandDlowerThresholdWater:D0
    DDDDDDDDandDupperThresholdSoilHigh:D11
    DDDDDDDDandDupperThresholdSoilMedium:D13
    DDDDDDDDandDupperThresholdSoil:D15
    end
    

    View Slide

30. DERIVE GRAMMAR
    7
    start = SP* statements SP*
    statements = statement ( SP+ statement )*
    statement = block / process
    block = with SP+ statements SP+ "end"
    with = "with" SP+ dataset SP+ "do"
    dataset = "recent" SP+ ID / ID
    ...
    

    View Slide

31. DERIVE GRAMMAR (FINAL)
    7
    {
    <<<<<<<}
    <<<<<<<<<<<<<<<<}
    <<<<<}
    <<<<<<}
    }
    start
    <empty_workflow
    <<<<<<}
    workflow
    <<<<<<}
    statements
    <<<<<<}
    statement
    <<process_statement
    <<<<<<<<<}
    process_statement_with
    <<<<<<<<<}
    process_statement_using
    <<<<<<}
    process_statement_as
    <<<<<<}
    with
    <<<<<<<<<}
    dataset_expression
    <<<<<<}
    recent
    <<<<<<}
    latest
    <<<<<<}
    dataset
    <<<<params
    <<<<<<}
    param
    <<<<<<<}
    tuple_expression
    <expression
    <<<<tuple_ids
    <placeholder
    <<<<<<}
    ref
    <<name
    <<<<<<}
    string
    <<<<<<}
    <<<<<<}
    string_character
    <<store_process
    <<<<<<<<<<<<<}
    <<<<<<<<<<<<}
    <<<<<}
    visualize_process
    <<<<<<}
    apply_process
    <<<<<<}
    process
    <<<<<<add_process
    <<<<<<}
    <<<<<<}
    exclude_process
    <<<<<<}
    urban_process
    <<<<<<<urban_dataset_param
    <<<<-\RTjYN
    <marine_process
    <land_process
    <?SSNjK\
    <<<<<<}
    NjK\_KOR\
    <?SSReservedWord
    <Jeyword
    <<<<<<<<-j9-U9jT\
    <<-\RTjYN
    <<-O9ORYZ\
    <<<<\N!
    <<\X-9U!\
    <<<<9jT\ST
    <<<<<<<<?SSNUK'\R
    <<<<<<}
    \S-jP\_-ZjRj-T\R
    <SP
    <?SSj!!<<<<<<<<<<<j!!\!<<<<<<<<<j99<<<<<<<<<<<jN!<<<<<<<<<<<jPP9Y<<<<<<<<<jS<<<<<<<<<<<<-j9-U9jT\<<<<<-\RTjYN<<<<<<<-O9ORYZ\<<<<<<<<<<<<<<<<<<<<!\zORK\!<<<<<<!O<<<<<<<<<<<<\N!<<<<<<<<<<<\X-9U!\<<<<<<<V\N\RjT\<<<<<<XOYN<<<<<<<<<<9jT\ST<<<<<<<<Oz<<<<<<<<<<<<R\-\NT<<<<<<<<S\9\-T<<<<<<<<STOR\<<<<<<<<<TO<<<<<<<<<<<<USYNV<<<<<<<<<VYSUj9YZ\<<<<<<<<<<<<<<<<<<<WYTZ<<<<<<<<<<

    View Slide

32. Mapping2rules
    Script
    Parser
    Abstract2syntax2tree
    Interpreter
    ...
    SCRIPT2EXECUTION
    Service22
    Service21 Service2n
    

    View Slide

33. SCRIPT EXECUTION
    exclude
    recent
    CityModel
    Trees
    Filter
    File A
    Feature extraction
    Tree classifier
    

    View Slide

34. FURTHERwREAD
    Krämer1wMichelwandwSenner1wIvo
    Awmodularwsoftwarewarchitecturewfor
    processingwofwbigwgeospatialwdatawinwthewcloud
    ComputerswGwGraphics:wSpecialwIssuewonwProcessing
    LargewGeospatialwData1wElsevier1w2015wPtowbewpublishedL
    

    View Slide

35. CONCLUSION
    Cloud is
    scalable!
    Current UIs are
    too complicated
    Cloud is
    easy!
    Current solutions
    are not scalable
    

    View Slide

36. THANKS
    FOR YOUR ATTENTION!
    QUESTIONS?
    

    View Slide

37. Michel3Krämer
    Fraunhofer3Institute3for
    Computer3Graphics3Research3IGD
    Fraunhoferstraße35
    642833Darmstadt,3Germany
    michel.kraemer@igd.fraunhofer.de
    +493615131553415
    

    View Slide