Implementing Data Visualization Apps on iOS Devices

Slide 1

Slide 1 text

Elastic Image Software LLC Douglass Turner Elastic Image Software tweets: @dugla email: [email protected] Implementing Data Visualization Apps on iOS Devices

Slide 2

Slide 2 text

Elastic Image Software LLC iOS software development is done using Objective-C an object-oriented superset of C. It was developed in the spirit of Smalltalk. 2

Slide 3

Slide 3 text

Elastic Image Software LLC • Objective-C is simple, approachable, lightweight, and pragmatic. No frills. • Objective-C and C can be intermingled freely. • Think OOP for C hackers and Unix heads. 3

Slide 4

Slide 4 text

Elastic Image Software LLC • Class: Grouping of data + code. The type of an object. • Instance: A speciﬁc copy of a class. • Method: A message that an object can respond to. • Instance variable (ivar): A piece of data belonging to an object 4 Objective-C Supports

Slide 5

Slide 5 text

Elastic Image Software LLC 5 Interface - .h ﬁle @interface className : superclassName @property(nonatomic, retain) Type *propertyForType; +(return_type)classMethod; +(return_type)classMethodWithParam:(paramType) paramName; -(return_type)instanceMethodWithParam1:(param1Type)param1Name andParam2:(param2Type) param2Name; @end

Slide 6

Slide 6 text

Elastic Image Software LLC 6 @implementation classname @synthesize propertyForType; +(return_type)classMethod { // do stuff } +(return_type)classMethodWithParam:(paramType) paramName { // do stuff } -(return_type)instanceMethodWithParam1:(param1Type)param1Name andParam2:(param2Type) param2Name { // do stuff } @end Implementation - .m ﬁle

Slide 7

Slide 7 text

Elastic Image Software LLC Apple style tends towards longSelfDocumentingMethodNames. 7

Slide 8

Slide 8 text

Elastic Image Software LLC 8 Instantiation self.window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]]; Property setting self.window.backgroundColor = [UIColor whiteColor]; Message passing [self.window makeKeyAndVisible];

Slide 9

Slide 9 text

Elastic Image Software LLC 9 Objective-C Types

Slide 10

Slide 10 text

Elastic Image Software LLC 10 Dynamically-typed: id whoCaresWhatThisIs; Statically-typed: Thang* aThang;

Slide 11

Slide 11 text

Elastic Image Software LLC 11 Selectors identify methods by name @interface Observer : NSObject - (id)initWithTarget:(id)object action:(SEL)action; @property(nonatomic, strong) id targetObject; @property(nonatomic) SEL targetAction; @end

Slide 12

Slide 12 text

Elastic Image Software LLC 12 Selectors identify methods by name observer = [[Observer alloc] initWithTarget:self action:@selector(updateDisplay:)];

Slide 13

Slide 13 text

Elastic Image Software LLC 13 Objective Lifecycle • Create an instance. • Futz with it (Send messages. Pass it around.) • Discard it.

Slide 14

Slide 14 text

Elastic Image Software LLC iOS is designed around one foundational pattern: Model View Controller. Much of iOS development - excluding Model development - involves customization and extension of this pattern. 14

Slide 15

Slide 15 text

Elastic Image Software LLC 15 Controller Model View reference reference reference

Slide 16

Slide 16 text

Elastic Image Software LLC 16 Controller Model View Target - Action Key-value Observing or Notiﬁcation Key-value Observing or Notiﬁcation

Slide 17

Slide 17 text

Elastic Image Software LLC 17 iOS has rich support for a loose, ﬂat, decoupled style of programming • Notiﬁcation • Target - Action • Key-value Observing (KVO) • Block & Dispatch Queue • Delegation (Protocol)

Slide 18

Slide 18 text

Elastic Image Software LLC 18 Notiﬁcations

Slide 19

Slide 19 text

Elastic Image Software LLC 19 [[NSNotificationCenter defaultCenter] postNotificationName:MyNotification object:self]; Notiﬁcation Notiﬁer

Slide 20

Slide 20 text

Elastic Image Software LLC 20 [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(respondToMyNotification:) name:MyNotification object:nil]; Notiﬁcation - (void) respondToMyNotification:(NSNotification *)notification { // do stuff } Notiﬁcation respondent

Slide 21

Slide 21 text

Elastic Image Software LLC 21 Target - Action

Slide 22

Slide 22 text

Elastic Image Software LLC 22 Target - Action

Slide 23

Slide 23 text

Elastic Image Software LLC 23 Target - Action

Slide 24

Slide 24 text

Elastic Image Software LLC 24 Target - Action

Slide 25

Slide 25 text

Elastic Image Software LLC 25 Target - Action @interface Counter : NSObject @property(nonatomic, strong) NSNumber *count; - (IBAction)trackSlider:(UISlider *)slider; @end

Slide 26

Slide 26 text

Elastic Image Software LLC 26 Target - Action @implementation Counter -(IBAction) trackSlider:(UISlider *)slider; {! self.count = [NSNumber numberWithFloat:slider.value]; } @end

Slide 27

Slide 27 text

Elastic Image Software LLC 27 Target - Action @class Observer; @class Counter; @interface EIViewController : UIViewController @property(nonatomic, strong) IBOutlet UILabel *label; @property(nonatomic, strong) Observer *observer; @property(nonatomic, strong) IBOutlet Counter *counter; - (void)updateLabel:(NSNumber *)newValue; @end

Slide 28

Slide 28 text

Elastic Image Software LLC 28 Target - Action @implementation EIViewController - (void)viewDidLoad { self.observer = [[Observer alloc] initWithTarget:self action:@selector(updateLabel:)]; [self.counter addObserver:self.observer forKeyPath:@"count" options:NSKeyValueObservingOptionNew context:NULL]; } -(void) updateLabel:(NSNumber *)newValue { self.label.text = [NSString stringWithFormat:@"%.2f", [newValue floatValue]]; } @end

Slide 29

Slide 29 text

Elastic Image Software LLC 29 Key-value Observing (KVO)

Slide 30

Slide 30 text

Elastic Image Software LLC 30 Any property is by default “Key-value Compliant” allowing it to be observed.

Slide 31

Slide 31 text

Elastic Image Software LLC 31 Example: HelloSlider GitHub: http://bit.ly/XDxIvp

Slide 32

Slide 32 text

Elastic Image Software LLC 32 Example: HelloSlider GitHub: http://bit.ly/XDxIvp

Slide 33

Slide 33 text

Elastic Image Software LLC 33 Example: HelloSlider GitHub: http://bit.ly/XDxIvp

Slide 34

Slide 34 text

Elastic Image Software LLC 34 Example: HelloSlider GitHub: http://bit.ly/XDxIvp @interface Counter : NSObject @property(nonatomic, strong) NSNumber *count; - (IBAction)trackSlider:(UISlider *)slider; @end

Slide 35

Slide 35 text

Elastic Image Software LLC 35 Example: HelloSlider GitHub: http://bit.ly/XDxIvp @implementation Counter -(IBAction) trackSlider:(UISlider *)slider; {! self.count = [NSNumber numberWithFloat:slider.value]; } @end

Slide 36

Slide 36 text

Elastic Image Software LLC 36 @interface Observer : NSObject - (id)initWithTarget:(id)object action:(SEL)action; @property(nonatomic, strong) id targetObject; @property(nonatomic) SEL targetAction; @end Example: HelloSlider GitHub: http://bit.ly/XDxIvp

Slide 37

Slide 37 text

Elastic Image Software LLC 37 @implementation Observer - (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context { [self.targetObject performSelectorOnMainThread:self.targetAction withObject:[object valueForKeyPath:keyPath] waitUntilDone:NO]; } @end Example: HelloSlider GitHub: http://bit.ly/XDxIvp

Slide 38

Slide 38 text

Elastic Image Software LLC 38 Example: HelloSlider GitHub: http://bit.ly/XDxIvp @implementation EIViewController - (void)viewDidLoad { self.observer = [[Observer alloc] initWithTarget:self action:@selector(updateLabel:)]; [self.counter addObserver:self.observer forKeyPath:@"count" options:NSKeyValueObservingOptionNew context:NULL]; } -(void) updateLabel:(NSNumber *)newValue { self.label.text = [NSString stringWithFormat:@"%.2f", [newValue floatValue]]; } @end

Slide 39

Slide 39 text

Elastic Image Software LLC 39 Blocks & Dispatch Queues

Slide 40

Slide 40 text

Elastic Image Software LLC 40 ^{ NSLog(@"Doing something"); } Blocks & Dispatch Queues dispatch_queue_t queue = dispatch_get_global_queue(0,0); dispatch_async(queue, ^{ NSLog(@"Doing something"); }); Block Dispatch Queue

Slide 41

Slide 41 text

Elastic Image Software LLC 41 Blocks & Dispatch Queues - (void)updateFeaturesWithNotification:(NSNotification *)notification { dispatch_async([UIApplication sharedIGVAppDelegate].trackControllerQueue, ^{ [self updateFeatures]; dispatch_async(dispatch_get_main_queue(), ^{ [self.coverageTrack setNeedsDisplay]; [self.track setNeedsDisplay]; }); }); }

Slide 42

Slide 42 text

Elastic Image Software LLC 42 Delegation (Protocol)

Slide 43

Slide 43 text

Elastic Image Software LLC 43 A protocol is identical to an interface in Java. A collection of method signatures implemented by the object that “conforms” to the protocol. The delegate/protocol pattern is ubiquitous throughout iOS. Delegation (Protocol)

Slide 44

Slide 44 text

Elastic Image Software LLC 44 @interface UITableViewController:UIViewController UITableViewController inherits from UIViewController and conforms to the UITableViewDelegate and UITableViewDataSource protocols Delegation (Protocol)

Slide 45

Slide 45 text

Elastic Image Software LLC 45 UITableViewDelegate @protocol UITableViewDelegate @optional - (NSIndexPath *)tableView:willSelectRowAtIndexPath:; - (NSIndexPath *)tableView:willDeselectRowAtIndexPath:; @end

Slide 46

Slide 46 text

Elastic Image Software LLC 46 @protocol UITableViewDataSource @required - (NSInteger)tableView:numberOfRowsInSection:; @optional - (NSInteger)numberOfSectionsInTableView:; - (NSArray *)sectionIndexTitlesForTableView:; @end UITableViewDataSource

Slide 47

Slide 47 text

Elastic Image Software LLC 47 iOS devices combine mobility, gesture, and a powerful GPU. This makes iOS App development an entirely new form of software development.

Slide 48

Slide 48 text

Elastic Image Software LLC 48 Mobility introduces context as a driver for usage. New app types appear to meet user needs driven by context.

Slide 49

Slide 49 text

Elastic Image Software LLC 49 Gesture forces a “no interface”, approach to app design and U/X mindset. Data representation drives interaction and associated affordances.

Slide 50

Slide 50 text

Elastic Image Software LLC 50 GPU. That cute little iPhone in your hand is a graphics processing beast. It is a GPU device tamed for domestic use. The entire interface is GPU driven. That is why iOS apps feel the way they do. Light. Effortless. Friction free. Like butter.

Slide 51

Slide 51 text

Elastic Image Software LLC 51 Developers must discard many of their desktop assumptions when developing for iOS • No mouse • No interface • Minimal keyboard • Arms length interaction • One handed Interaction • Two handed Interaction • Untethered resources

Slide 52

Slide 52 text

Elastic Image Software LLC 52 Demos & Code

Slide 53

Slide 53 text

Elastic Image Software LLC 53 Multi-resolution Image •CATiledLayer •UIScrollView •Amazon Web Services (S3)

Slide 54

Slide 54 text

Elastic Image Software LLC 54 Multi-resolution Image • 8k x 8k image resolution. 101MB on disk. • Subsampled 4 successive times • Diced into 256 x 256 tiles • Stored on Amazon S3

Slide 55

Slide 55 text

Elastic Image Software LLC 55

Slide 56

Slide 56 text

Elastic Image Software LLC 56

Slide 57

Slide 57 text

Elastic Image Software LLC 57

Slide 58

Slide 58 text

Elastic Image Software LLC 58 IGV for iPad •CoreGraphics •Dispatch Queues •UIScrollView

Slide 59

Slide 59 text

Elastic Image Software LLC 59

Slide 60

Slide 60 text

Elastic Image Software LLC 60

Slide 61

Slide 61 text

Elastic Image Software LLC 61

Slide 62

Slide 62 text

Elastic Image Software LLC 62

Slide 63

Slide 63 text

Elastic Image Software LLC 63

Slide 64

Slide 64 text

Elastic Image Software LLC 64

Slide 65

Slide 65 text

Elastic Image Software LLC 65

Slide 66

Slide 66 text

Elastic Image Software LLC 66

Slide 67

Slide 67 text

Elastic Image Software LLC 67

Slide 68

Slide 68 text

Elastic Image Software LLC 68

Slide 69

Slide 69 text

Elastic Image Software LLC 69

Slide 70

Slide 70 text

Elastic Image Software LLC 70

Slide 71

Slide 71 text

Elastic Image Software LLC 71

Slide 72

Slide 72 text

Elastic Image Software LLC 72 The Elastic Image •OpenGL •GLSL - OpenGL Shading Language •Shader/Gesture plug-ability via plist •UISplitViewController

Slide 73

Slide 73 text

Elastic Image Software LLC 73

Slide 74

Slide 74 text

Elastic Image Software LLC 74

Slide 75

Slide 75 text

Elastic Image Software LLC 75

Slide 76

Slide 76 text

Elastic Image Software LLC 76

Slide 77

Slide 77 text

Elastic Image Software LLC 77

Slide 78

Slide 78 text

Elastic Image Software LLC 78

Slide 79

Slide 79 text

Elastic Image Software LLC 79

Slide 80

Slide 80 text

Elastic Image Software LLC 80

Slide 81

Slide 81 text

Elastic Image Software LLC 81

Slide 82

Slide 82 text

Elastic Image Software LLC 82

Slide 83

Slide 83 text

Elastic Image Software LLC 83

Slide 84

Slide 84 text

Elastic Image Software LLC 84

Slide 85

Slide 85 text

Elastic Image Software LLC 85

Slide 86

Slide 86 text

Elastic Image Software LLC 86

Slide 87

Slide 87 text

Elastic Image Software LLC 87

Slide 88

Slide 88 text

Elastic Image Software LLC 88 self.panGesture = [[UIPanGestureRecognizer alloc] initWithTarget:self action:@selector(handlePanGesture:)]; [self addGestureRecognizer:self.panGesture]; The Elastic Image Gestures are fundamental to iOS apps. A gesture is attached to a UIView. Gestures come in different ﬂavors. self.scaleGesture = [[UIPinchGestureRecognizer alloc] initWithTarget:self action:@selector(handleScaleGesture:)]; [self addGestureRecognizer:self. scaleGesture]; self.toggleGesture = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(handleToggleGesture:)]; self.toggleGesture.numberOfTapsRequired!! = 1; self.toggleGesture.numberOfTouchesRequired! = 1; [self addGestureRecognizer:self.toggleGesture]; Pan Pinch Tap

Slide 89

Slide 89 text

Elastic Image Software LLC 89 The Elastic Image Photo f/x are implemented in GLSL. The OpenGL Shading Language. Shaders are written in a C-like language and evaluated in a SIMD manner on the entire image in realtime.

Slide 90

Slide 90 text

Elastic Image Software LLC 90 The Elastic Image varying! vec2 v_st; uniform sampler2D hero; void main() { ! ! gl_FragColor = (heroChannels == 1) ? vec4(texture2D(hero, v_st).a) : texture2D(hero, v_st); ! } varying! vec2 v_st; uniform sampler2D hero; void main() { ! ! gl_FragColor = (heroChannels == 1) ? vec4(texture2D(hero, v_st).a) : texture2D(hero, v_st); ! } TextureMapShader - apply a texture (the photo) to a quad (the rendering surface).

Slide 91

Slide 91 text

Elastic Image Software LLC 91 The Elastic Image Property lists enable simple speciﬁcation of a shader gesture and its handler. The Objective-C runtimes enables easy conversion from string to class instance

Slide 92

Slide 92 text

Elastic Image Software LLC 92 - (UIGestureRecognizer *)createGestureWithDictionary:(NSDictionary *)gestureDictionary { NSString *gestureClassName = [gestureDictionary objectForKey:@"class"]; Class _gesture = NSClassFromString(gestureClassName); NSString *selectorName = [gestureDictionary objectForKey:@"selector"]; SEL _selector = NSSelectorFromString(selectorName); UIGestureRecognizer *shaderGesture = [[[_gesture alloc] initWithTarget:self action:_selector] autorelease]; shaderGesture.delegate = self.detailController; return shaderGesture; } The Elastic Image Property lists enable simple speciﬁcation of a shader gesture and its handler. The Objective-C runtimes enables easy conversion from string to class instance

Slide 93

Slide 93 text

Elastic Image Software LLC 93 •OpenGL •GLSL - OpenGL Shading Language •Proprietary Panorama Engine •UIPopoverController Beautiful Panoramas

Slide 94

Slide 94 text

Elastic Image Software LLC 94

Slide 95

Slide 95 text

Elastic Image Software LLC 95

Slide 96

Slide 96 text

Elastic Image Software LLC 96

Slide 97

Slide 97 text

Elastic Image Software LLC 97

Slide 98

Slide 98 text

Elastic Image Software LLC 98

Slide 99

Slide 99 text

Elastic Image Software LLC 99

Slide 100

Slide 100 text

Elastic Image Software LLC 100

Slide 101

Slide 101 text

Elastic Image Software LLC 101 •OpenGL •GLSL - OpenGL Shading Language •Proprietary Panorama Engine •3D Spatial Picking BMW Interior Panorama

Slide 102

Slide 102 text

Elastic Image Software LLC 102

Slide 103

Slide 103 text

Elastic Image Software LLC 103

Slide 104

Slide 104 text

Elastic Image Software LLC 104

Slide 105

Slide 105 text

Elastic Image Software LLC 105

Slide 106

Slide 106 text

Elastic Image Software LLC 106

Slide 107

Slide 107 text

Elastic Image Software LLC 107

Slide 108

Slide 108 text

Elastic Image Software LLC 108 RadPad •OpenGL •GLSL - OpenGL Shading Language •Wavelet Image Decompression •UISplitViewController •UIScrollViewController

Slide 109

Slide 109 text

Elastic Image Software LLC 109

Slide 110

Slide 110 text

Elastic Image Software LLC Thank You Douglass Turner Elastic Image Software tweets: @dugla email: [email protected]