What is needed to solve these issues? Are some regions in the network hotspots? - Better load balancing How is user traffic moving in the network? - Better resource provisioning What are the popular handoff sequences? - Troubleshoot handoff related problems
Why Not Use a Graph Parallel Framework? �� �� ��� ��� ��� ��� ��� ��� ��� �������� ���������� ������ ����� ����������������������� ������������ Fails to produce results! Domain specific optimizations key for efficient analysis
CellIQ Implementation *Gonzales. et.al. “GraphX: Graph Processing in a Distributed Dataflow Framework”, OSDI 2014 Implemented as a layer on GraphX* Incorporates several domain specific optimizations GraphX Spark Pregel API PageRank Connected Comp. K-core Triangle Count LDA SVD++ CellIQ
3 Machine 3 Machine 4 B C B C D E A F Machine 1 Machine 2 A B C D E F Graph Partitioning G H G H Random (hashed) partitioning results in poor spatial locality
Machine 3 Machine 4 0 3 2 1 B C B C D E A F Machine 1 Machine 2 CellIQ Graph Partitioning G H Uses Hilbert space-filling curves Use curve’s distance as the 1-dimensional key
Machine 3 Machine 4 0 3 2 1 B C B C D E A F Machine 1 Machine 2 A B C D E F CellIQ Graph Partitioning G H G H Uses Hilbert space-filling curves Use curve’s distance as the 1-dimensional key Range partition the key space
0 1 2 3 4 7 6 5 8 11 10 9 14 15 12 13 Machine 3 Machine 4 B C B C D E A F Machine 1 Machine 2 A B C D E F CellIQ Graph Partitioning G H G H Uses Hilbert space-filling curves Use curve’s distance as the 1-dimensional key Range partition the key space
Part. 2 Part. 1 Vertex Table (RDD) B C A D F E A D Routing Table in GraphX enables Multicast D B C D E A A F Machine 1 Machine 2 Edge Table (RDD) A B A C C D B C A E A F E F E D B C D E A F Routing Table (RDD) B C D E A F 1
2
1
2
1
2
1
2
Slide courtesy: Joey Gonzales
Routing Table (RDD) B C D E A F 1
2
1
2
1
2
1
2
Part. 2 Part. 1 Vertex Table (RDD) B C A D F E A D D B C D E A A F Machine 1 Machine 2 Edge Table (RDD) A B A C C D B C A E A F E F E D B C D E A F Slide courtesy: Joey Gonzales Can compute destination partitions easily due to the use of geo-partitioner
Tracking Persistent Hotspots BS1 BS2 BS3 t1 t2 t3 W Combining graphs at the end of the window results in many join operations (inefficient) BS1 BS2 BS1 BS2
graphReduceByWindow
• Implemented using Spark’s cogroupedRDD
• Two default reduce functions: graph intersection and union • Further optimizations: – Co-partition graphs from multiple batches – Reuse indices and routing tables for graphs in the same window More details in the paper!
Evaluation Setup • LTE control plane data from a major cellular network operator • 1 million+ subscribers, live network • 2 TB data from 1 week – 1 file per minute, 750k records, 100s of fields/line – 10 collection points, 10 hours per day • Implemented several analysis tasks
Benefits of Geo-partitioning �� �� ��� ��� ��� ��� ��� ��� ��� �������� ���������� ������ ����� ����������������������� ������������ �������������������� ���������������� Small amount of data, movement not noticeable Default practitioner fails to produce results
CellIQ is a cellular network analytics system that uses domain-specific optimizations to achieve 2x to 5x improvements Ongoing Work: • Using techniques in CellIQ to perform root-cause analysis on operational LTE Networks • Generalized streaming graph analysis techniques
anandpiyer
yukizenimoto
kirrane
mkimura
yowata
hiroyaonoe
chokkan
haraduka
rpc
gmoriki
ya9do
tatsuropianooo
.png){kind=avatar}
elith
samanthasiow
michaelherold
zakiyama
jrom
eileencodes
frogandcode
eitanlees
ufuk
sferik
morganepeng
carmenintech
tanoku