Eywa: An Interoperable Fog Computing Infrastructure with RDF Stream Processing

Presentation at INSCI2017. Abstract. Fog computing is an emerging technology for the Internet of
Things (IoT) that aims to support processing on resource-constrained
distributed nodes in between the sensors and actuators on the ground
and compute clusters in the cloud. Fog Computing benefits from low
latency, location awareness, mobility, wide-spread deployment and geographical
distribution at the edge of the network. However, there is a need
to investigate, optimise for and measure the performance, scalability and
interoperability of resource-constrained Fog nodes running real-time applications
and queries on streaming IoT data before we can realise these
benefits. With Eywa, a novel Fog Computing infrastructure, we 1) formally
define and implement a means of distribution and control of query
workload with an inverse publish-subscribe and push mechanism, 2) show
how data can be integrated and made interoperable through organising
data as Linked Data in the Resource Description Format (RDF), 3) test
if we can improve RDF Stream Processing query performance and scalability
over state-of-the-art engines with our approach to query translation
and distribution for a published IoT benchmark on resource-constrained
nodes and 4) position Fog Computing within the Internet of the Future.

Transcript

1. Eywa Eugene Siow | Thanassis Tiropanis | Wendy Hall
2. Ryan Gosling / K Replicant Ana de Armas / Joi AI A movie about Fog Computing
3. Fog Computing Sci-Fi Blade Runner 2049’s Joi lives in the Fog Smart Home Hologram Joi lives on a console in K’s home rather than the cloud. She can control all actuators in the house. K’s Spinner Car Joi is connected to the car. When the spinner goes down, she loses the ability to project herself. Emanator Joi can reside on the portable emanator and move around with K. A Fog Computing device?
4. The Internet of Things Fog Computing An emerging technology that bridges the gap, deployed close to the source. Cloud Dynamic provisioning of scalable resources e.g. analytics on a huge volume of historical data. Things Connected sensors and actuators producing streams of time-series data.
5. Challenges for Fog Computing Stream Processing Performant and scalable processing of multiple streams in real-time 02 01 03Distribution Provisioning of resources and distribution of work load Interoperability Heterogeneity of device, platform and data
6. Eywa is like a huge biological internet; the trees are fog computing nodes that store and process information and sensors are connected flora and fauna Avatar By James Cameron
7. Eywa A Fog Computing Infrastructure Fog Node Stores and Processes IoT Sensors Produce Observations Streams of Time-series Data Interoperable Stream Processing Distribution of Query Workload
8. Semantic Interoperability in Eywa Using a Common RDF Graph Model Common Structure Widely-used, flexible model Data Integration Stores Rich Metadata Graph Querying Powerful SPARQL Graph Queries IoT Domains for Things and Apps Siow, E., Tiropanis, T., Hall, W. (2016). Interoperable and Efficient: Linked Data for the Internet of Things. The 3rd International Conference on Internet Science.
9. Process Stream Processing by Query Translation Distribute Workload Distribution by Projection Pushdown Deliver Inverse-publish- subscribe Eywa Network
10. Source Node Publishes Data Client Node Issues Queries Broker Node Facilitate Network Formation, Forwards Data τ Ъ ѕ Eywa Node
11. Deliver Queries Inverse-Publish-Subscribe A novel system for best-effort, co-operative query delivery in fog computing control-plane
12. Deliver Queries Inverse Publish-Subscribe (1) Ъ Broker Well-known Source 1 Produces Stream, μ1 S 1 S 2 Source 2 Produces Stream, μ2 Client Issues Query, q1 τ
13. Deliver Queries Inverse Publish-Subscribe (2) Ъ Broker Well-known Source 1 Produces Stream, μ1 S 1 S 2 Source 2 Produces Stream, μ2 Client Issues Query, q1 τ
14. Distribute Workload Projection Pushdown Efficient streaming of only projected data in the fog computing data-plane
15. Distribute Workload Graph Query in SPARQL Join Window :traffic Graph traffic Window :weather Graph weather Project ?v1,?v2,?v3 From CityBench (Smart City Streams) Query 2. Finding the traffic congestion level and weather conditions of my planned journey. Source 1 Processing q1 1 S 1 ?v1,?v2 ?v3 temp,hum congestionLevel Client Receives the Projection τ Project Streams temp,hum, congestionLevel No Extra Join Variables
16. Distribute Workload Efficient Mappings for RDF Stream Processing Siow, E., Tiropanis, T. and Hall, W. (2016) SPARQL-to-SQL on internet of things databases and streams. ISWC2016: The 15th International Semantic Web Conference Observation 1 Temperature property Time 1 at “stream.temp” value Aarhus location RDF Graph WeatherEvent temp hum wspd Stream
17. Window :traffic Graph traffic Distribute Workload Graph Matching to Projection (1) Join Window :weather Graph weather Project ?v1,?v2,?v3 Source 1 Processing q1 1 S 1 Graph Match Mapping
18. Window :traffic Graph traffic Distribute Workload Graph Matching to Projection (2) Join Window :weather Graph weather Project ?v1,?v2,?v3 Source 1 Processing q1 1 S 1 WeatherEvent temp hum Projection
19. Distribute Workload Projection Pushdown, Push Projection Client Processes Streams τ Source 1 Pushes Projection, π1 S 1 S 2 Source 2 Pushes Projection, π2
20. Process Query Translation Efficient stream processing of graphs for IoT time-series in the fog computing data-plane
21. Processing Streams Query Translation for Stream Processing Join Window :traffic Window :weather Project ?v1,?v2,?v3 Client Processes Query, q1 τ π 1 +π 2 Event Processing Language Query temp SELECT hum congestionLevel FROM weather(3s) congestionLevel(3s) v1 v2 v3 AS AS AS
22. Evaluation CityBench Smart City Benchmark Latency and Scalability
23. Evaluation on 3 Stream Processing Engines Smart City RDF Streams CITYBENCH C-SPARQL Barbieri et al. "C-SPARQL: SPARQL for continuous querying." WWW2009. CQELS Le-Phuoc et al. "A native and adaptive approach for unified processing of linked streams and linked data." ISWC2011 Eywa 01 02 03 Real-time streams (e.g. vehicle traffic, parking, weather, pollution) Based on smart city applications (e.g. parking space finder, admin console) Run on resource-constrained Raspberry Pis as Fog Nodes (~500mhz CPU, 512mb ram, SD CARD)
24. Latency Evaluation CityBench Query 1 (traffic congestion level on two roads) 0 2 4 6 8 10 12 14 16 1 8 15 C-SPARQL High Latency CQELS Some Fluctuations, Medium Latency Eywa Stable, low latency Latency (s) Experiment Time (min)
25. Scalability Evaluation CityBench Query 2 (traffic congestion level and weather) 0 50 100 150 200 250 1 8 15 C-SPARQL High Memory Consumption, Large Fluctuations CQELS Medium Memory Consumption Eywa Very Low Memory Consumption Experiment Time (min) Memory Consumed (MB)
26. Scalability Evaluation CityBench Query 5 (traffic congestion where event is happening) 0 20 40 60 80 100 120 140 160 180 1 8 15 C-SPARQL 20 Concurrent CQELS 20 Concurrent Eywa 20 Concurrent Experiment Time (min) Memory Consumed (MB) C-SPARQL CQELS Eywa
27. Scalability Evaluation CityBench Query 10 (most polluted area in the city in real-time) 0 50 100 150 200 250 1 8 15 C-SPARQL 5 Streams Experiment Time (min) Memory Consumed (MB) CQELS Eywa C-SPARQL 2 Streams
28. Fog Scalability Evaluation CityBench Query 10 (most polluted area in the city in real-time) 0 10 20 30 40 50 60 70 1 8 15 Single 8 Streams Experiment Time (min) Memory Consumed (MB) Fog 8 Streams Fog 5 Streams Single 5 Streams
29. Conclusions 02 Eywa Fog consumes less memory than other engines Scalability 04 Eywa is a means for distributed Fog Computing Distributed 05 Data locality, offline access, data ownership Utility for the IoT 03 From the RDF graph model for metadata and data Interoperability 01 Eywa is fast and performant at stream processing Latency