Tweet
Tweet

Slide 1

Slide 1 text

Thinking in Immediate: ImGUI Zhihao Yuan

Slide 2

Slide 2 text

How do we understand GUI? 3

Slide 3

Slide 3 text

To break it down 4

Slide 4

Slide 4 text

What if there is no action…? class CellInspecter: i: int j: int ins = CellInspecter() ins.i = 99 5

Slide 5

Slide 5 text

“Encapsulation” ins = CellInspecter() ins.i = 99 ins.set_row(99) 6

Slide 6

Slide 6 text

Sending a signal is not an impl. detail • It is a requirement • It is equivalent to saying that the correct way of changing a state of an GUI object is to send a signal 7

Slide 7

Slide 7 text

Signal becomes the only abstraction 8

Slide 8

Slide 8 text

Cost of one-sided abstraction • A typical implementation is to represent the states as a matrix • A pair of (x, y) is good enough if not allowing multiple selection • Is a selectable cell a useful class? • If yes, it should have it own selected state • But the cost is having the same number of events 9

Slide 9

Slide 9 text

How to model shared states? 10

Slide 10

Slide 10 text

# of events grows faster than # of states 11

Slide 11

Slide 11 text

What is an abstraction? • An abstraction is a simplified view of an entity that omits unimportant details • Omitting details that are important leads to obscurity, creating a false abstraction Quote from “A Philosophy of Software Design” 12

Slide 12

Slide 12 text

GUI can use a different abstraction • Signal-slot/event-callback model hides states • States are important • Let users play with their states 13

Slide 13

Slide 13 text

Button has a binary state next_btn = QPushButton(widget) next_btn.setText('next') next_btn.clicked.connect(callback) 14

Slide 14

Slide 14 text

Binary state = boolean next_btn = QPushButton(widget) next_btn.setText('next') if next_btn.is_clicked: print("I'm clicked!") 15

Slide 15

Slide 15 text

What if we are running it in a loop? while True: next_btn = QPushButton(widget) next_btn.setText('next') if next_btn.is_clicked: print("I'm clicked!") 16

Slide 16

Slide 16 text

The loop runs too fast! while True: poll_events() process_inputs() next_btn = QPushButton(widget) next_btn.setText('next') if next_btn.is_clicked: print("I'm clicked!") 17

Slide 17

Slide 17 text

Render the whole thing 60 times per second while True: poll_events() process_inputs() next_btn = QPushButton(widget) next_btn.setText('next') if next_btn.is_clicked: print("I'm clicked!") render() 18

Slide 18

Slide 18 text

Basic structure of an ImGUI application while not window_should_close(): poll_events() process_inputs() update() render() 19

Slide 19

Slide 19 text

Get back to our GUI logic def update(): next_btn = QPushButton(widget) next_btn.setText('next') if next_btn.is_clicked: print("I'm clicked!") 20

Slide 20

Slide 20 text

Get back to our GUI logic def update(): next_btn = QPushButton() next_btn.setText('next') if next_btn.is_clicked: print("I'm clicked!") 21

Slide 21

Slide 21 text

Get back to our GUI logic def update(): if QPushButton().setText('next').is_clicked: print("I'm clicked!") 22

Slide 22

Slide 22 text

Get back to our GUI logic def update(): if QPushButton('next').is_clicked: print("I'm clicked!") 23

Slide 23

Slide 23 text

Get back to our GUI logic def update(): if QPushButton('next'): print("I'm clicked!") 24

Slide 24

Slide 24 text

pyimgui def update(): if imgui.button('next'): print("I'm clicked!") 25

Slide 25

Slide 25 text

Button returns binary state button(label: str) -> bool 26

Slide 26

Slide 26 text

Text returns no state button(label: str) -> bool text(text: str) -> None 27

Slide 27

Slide 27 text

Q: What does this update function do? def update(): if imgui.button('next'): imgui.text("I'm clicked!") 29

Slide 28

Slide 28 text

Stateless function → Stateless UI 30 if imgui.button('next'): imgui.text("I'm clicked!")

Slide 29

Slide 29 text

Let’s extend our first application a little bit 31

Slide 30

Slide 30 text

So, how to make a function stateful 32

Slide 31

Slide 31 text

Turn it into a closure def make_fib(): a, b = 0, 1 def update(): nonlocal a, b if imgui.button('next'): a, b = (b, (a + b)) imgui.text(str(a)) return update 33

Slide 32

Slide 32 text

Use a stateful widget fib = make_fib() while not window_should_close(): poll_events() process_inputs() fib() render() 35 † I omitted a detail here; you need to call imgui.new_frame() before drawing any items

Slide 33

Slide 33 text

A stateful pyimgui widget 36

Slide 34

Slide 34 text

How to write this in C++? def make_fib(): a, b = 0, 1 def update(): nonlocal a, b if imgui.button('next'): a, b = (b, (a + b)) imgui.text(str(a)) return update 37

Slide 35

Slide 35 text

Does this work? auto make_fib = [] { int a = 0, b = 1; return [&] { if (ImGui::Button("next")) tie(a, b) = tuple(b, a + b); ImGui::Text("%d", a); }; }; 38

Slide 36

Slide 36 text

A widget can be a generator of frames def make_fib(): a, b = 0, 1 while True: if imgui.button('next'): a, b = (b, (a + b)) imgui.text(str(a)) yield 39

Slide 37

Slide 37 text

The main loop for using a closure while not window_should_close(): poll_events() process_inputs() fib() render() 40

Slide 38

Slide 38 text

The main loop for using a frame generator while not window_should_close(): poll_events() process_inputs() next(fib) render() 41

Slide 39

Slide 39 text

Using generators is just a choice • Not necessarily the right way to model ImGUI • C++ co_yield does not generate void 42

Slide 40

Slide 40 text

Examples States as parameters, return values, and data structures 43

Slide 41

Slide 41 text

Time to challenge ourselves with this 44

Slide 42

Slide 42 text

Usage def app(): insp = cell_inspector() A = np.random.rand(30, 30) next(insp) while True: imgui.new_frame() insp.send(A) yield 45

Slide 43

Slide 43 text

First, we need to create a window • A window in code is a region between imgui.begin('Title’) # and imgui.end() • pyimgui implicitly creates a “Debug” window to contain widgets outside a window 46

Slide 44

Slide 44 text

Receive data before creating the window def cell_inspector(): while True: A = yield imgui.begin('Cell Inspector') # ... imgui.end() 47

Slide 45

Slide 45 text

Dragger items drag_int(label: str, value: int, change_speed: float = 1.0, min_value: int = 0, max_value: int = 0) -> Tuple[bool, int] 48

Slide 46

Slide 46 text

Let’s try to use it def cell_inspector(): i = 0 while True: A = yield imgui.begin('Cell Inspector') _, i = imgui.drag_int('i', value=i) imgui.end() 49

Slide 47

Slide 47 text

Hover and drag over the bar 50

Slide 48

Slide 48 text

Decompose the UI 51

Slide 49

Slide 49 text

The generator loop while True: A = yield imgui.begin('Cell Inspector') index_control(A) cell_info(A) imgui.end() 52

Slide 50

Slide 50 text

Bird’s-eye view def cell_inspector(): i, j = (0, 0) def index_control(A): ... def cell_info(A): ... while True: ... index_control(A) cell_info(A) 53

Slide 51

Slide 51 text

Breaking it down i, j = (0, 0) def index_control(A): nonlocal i, j M, N = A.shape _, i = imgui.drag_int('i', i, max_value=M-1) _, j = imgui.drag_int('j', j, max_value=N-1) 54

Slide 52

Slide 52 text

Breaking it down def cell_info(A): v = A[i, j] frac = Fraction(v).limit_denominator(10000) imgui.text('value: {}'.format(v)) imgui.text('fraction: {}'.format(frac)) 55

Slide 53

Slide 53 text

Demo 56

Slide 54

Slide 54 text

Data binding? • UI should follow data 57

Slide 55

Slide 55 text

Let’s create a widget with multiple selection 58

Slide 56

Slide 56 text

Usage grid = selection_grid() next(grid) while True: ... ls = grid.send(A) 59

Slide 57

Slide 57 text

Communicate its states back to its caller 60 yield []

Slide 58

Slide 58 text

Communicate its states back to its caller 61 yield [(1, 0), (0, 3), (2, 3)]

Slide 59

Slide 59 text

How to create a widget? • Like creating a window, but group the items in imgui.begin_group() # ... imgui.end_group() • Items in this region have their horizontal positions locked 62

Slide 60

Slide 60 text

Blueprint def selection_grid(m=5, n=6): selected = np.zeros((m, n), dtype=bool) while True: A = yield np.argwhere(selected) imgui.begin_group() clear_button() data_panel(A) imgui.end_group() 63

Slide 61

Slide 61 text

If you are not familiar with NumPy >>> x array([[False, False, False, False], [False, False, False, False], [False, False, False, False]]) >>> x[2, 3] = 1 >>> x[1, 1] = 1 >>> np.argwhere(x) array([[1, 1], [2, 3]], dtype=int64) 64

Slide 62

Slide 62 text

Button & text 65

Slide 63

Slide 63 text

How to fill items in a grid • Add items row-by-row with imgui.same_line(), or • Add items column-by-column using Columns API, or • Dear ImGui 1.8x Tables API (pyimgui WIP) 66

Slide 64

Slide 64 text

Declare number of columns ... imgui.begin_group() imgui.columns(n + 1) clear_button() data_panel(A) imgui.end_group() 67

Slide 65

Slide 65 text

We’ll make it auto-hide later def clear_button(): if imgui.button('clear'): selected[:, :] = False 68

Slide 66

Slide 66 text

Be aware when to go to next column def data_panel(A): for i in range(m): imgui.text(str(i)) imgui.next_column() for j in range(n): imgui.text(str(j)) imgui.next_column() 69

Slide 67

Slide 67 text

Demo 70

Slide 68

Slide 68 text

How to make a selectable cell? selectable( label: str, selected: bool = False, ) -> Tuple[bool, bool] 71

Slide 69

Slide 69 text

Fill the grid column-wise for j in range(n): imgui.text(str(j)) for i in range(m): _, selected[i, j] = imgui.selectable( str(A[i, j]), selected[i, j], ) imgui.next_column() 72

Slide 70

Slide 70 text

Demo 73

Slide 71

Slide 71 text

Button is bigger than a selectable item if imgui.small_button('clear'): selected[:, :] = False • Small_button is a button without frame padding • Like selectable, text, etc. 75

Slide 72

Slide 72 text

Auto-hide if selected.any(): if imgui.small_button('clear'): selected[:, :] = False else: imgui.text('') 77

Slide 73

Slide 73 text

Demo 78

Slide 74

Slide 74 text

Is that done? A = np.random.rand(200, 300) A = np.eye(200, 300) 79

Slide 75

Slide 75 text

80

Slide 76

Slide 76 text

81

Slide 77

Slide 77 text

ID • In Dear ImGui, items with states have IDs • text has no state, therefore no ID • Items with the same ID share states • By default, ID is derived from the item’s label 82

Slide 78

Slide 78 text

Quick fix • In a label, characters starting with “##” are hashed but not displayed • Use this to make the IDs unique _, selected[i, j] = imgui.selectable( f"{A[i, j]}##{i,j}", selected[i, j], ) 84

Slide 79

Slide 79 text

👍 85

Slide 80

Slide 80 text

Intermission Error handling 86

Slide 81

Slide 81 text

Why don’t use RAII? imgui.begin_group() # ... imgui.end_group() 87

Slide 82

Slide 82 text

Why don’t use RAII? @contextmanager def grouped(): imgui.begin_group() yield imgui.end_group() 88

Slide 83

Slide 83 text

Consider a widget that throws exceptions try: imgui.begin_group() navigation() for dir in os.scandir(cwd): if imgui.button(dir.name): chdir(Path(dir.path)) imgui.end_group() except OSError: 89

Slide 84

Slide 84 text

Throws without calling end_group() 90

Slide 85

Slide 85 text

Some problems to consider before throwing • Can I throw an exception every frame? • If we end the group/window/popup/etc. with RAII, what items we left on the screen, do they respond to meaningful interactions? • Should I bring users’ attention to the error? 91

Slide 86

Slide 86 text

Some facts • An ImGUI region cannot be reverted • Begun popups cannot disappear • Added items cannot be deleted • GUI must always be usable • GUI must not lose response • GUI must follow users’ expectation – unexpected must become expected 92

Slide 87

Slide 87 text

93

Slide 88

Slide 88 text

ImGUI region should be noexcept • An ImGUI region should be atomic – it either works fully or not displayed at all • Here are my tricks • If processing data can throw, do it before entering the region • Substitute in different regions to bring users’ attentions to errors and provide meaningful ways to act on 94

Slide 89

Slide 89 text

Substitute in alternative UI Normal Error 95

Slide 90

Slide 90 text

Implementation try: dirs, others, choices = ... except OSError as exc: error(exc) else: explorer(dirs, choices, others) finally: imgui.end_popup() 97

Slide 91

Slide 91 text

Back to Examples Model shared states 98

Slide 92

Slide 92 text

Let’s try to create this popup 99

Slide 93

Slide 93 text

Click a button to open a widget in a popup def click_to_popup(label, control): state, new_state = None, None while True: with imgui.extra.scoped(label): if imgui.button(label): imgui.open_popup(control.__name__) if imgui.begin_popup(control.__name__): new_state = next(control) imgui.end_popup() yield new_state != state, new_state state = new_state 100

Slide 94

Slide 94 text

A Quick Demo 101

Slide 95

Slide 95 text

Time to focus on creating the widget • Does this work? def volume_control(): l, r = 100, 100 while True: yield l, r 102

Slide 96

Slide 96 text

Selectable selectable( label: str, selected: bool = False, ) -> Tuple[bool, bool] 103

Slide 97

Slide 97 text

Checkbox checkbox( label: str, state: bool, ) -> Tuple[bool, bool] 104

Slide 98

Slide 98 text

Unlinked 105

Slide 99

Slide 99 text

Linked 106

Slide 100

Slide 100 text

Model relation between states l, r = 0, False vol = [100, 100] while True: yield vol[l], vol[r] 107

Slide 101

Slide 101 text

Blueprint while True: yield vol[l], vol[r] with grouped(): imgui.text('Volume') imgui.separator() link_toggle() volume_sliders() mute_toggle() 108

Slide 102

Slide 102 text

All the pieces def link_toggle(): nonlocal r clicked, _ = imgui.checkbox('Link', not r) if clicked: r = not r 109

Slide 103

Slide 103 text

Leave the mute toggle empty for now def mute_toggle(): pass 110

Slide 104

Slide 104 text

Default layout 111

Slide 105

Slide 105 text

imgui.same_line() 112

Slide 106

Slide 106 text

Dragger drag_int(label: str, value: int, change_speed: float = 1.0, min_value: int = 0, max_value: int = 0) -> Tuple[bool, int] 113

Slide 107

Slide 107 text

Slider v_slider_int(label: str, width: float, height: float, value: int, min_value: int = 0, max_value: int = 0) -> Tuple[bool, int] 114

Slide 108

Slide 108 text

Create a helper def slider(label, value): _, value = imgui.v_slider_int(label=label, width=w, height=h, value=value, min_value=0, max_value=100) return value 115

Slide 109

Slide 109 text

Where… def volume_sliders(): w, h = imgui.calc_text_size('100') h *= 10 116

Slide 110

Slide 110 text

Display volume sliders imgui.text('L') imgui.same_line() vol[l] = slider('##l', vol[l]) imgui.same_line() vol[r] = slider('##r', vol[r]) imgui.same_line() imgui.text('R') 117

Slide 111

Slide 111 text

Demo 118

Slide 112

Slide 112 text

Let’s add Mute l, r = 0, False vol = [100, 100] while True: yield vol[l], vol[r] 119

Slide 113

Slide 113 text

How about… l, r = 0, False vol = [100, 100] muted = False while True: if muted: yield 0, 0 else: yield vol[l], vol[r] 120

Slide 114

Slide 114 text

In that case def mute_toggle(): nonlocal muted _, muted = imgui.checkbox('Mute', muted) 121

Slide 115

Slide 115 text

Sliders do no reflect muted 122

Slide 116

Slide 116 text

Unmuted 123

Slide 117

Slide 117 text

Unmuted 124

Slide 118

Slide 118 text

Previous set of states l, r = 0, False vol = [100, 100] muted = False while True: if muted: yield 0, 0 else: yield vol[l], vol[r] 125

Slide 119

Slide 119 text

New set of states l, r = 0, False vol = [100, 100] vol_inactive = [0, 0] muted = False while True: yield vol[l], vol[r] 126

Slide 120

Slide 120 text

Toggle → swap def mute_toggle(): nonlocal muted _, muted = imgui.checkbox('Mute', muted) 127

Slide 121

Slide 121 text

Toggle → swap def mute_toggle(): nonlocal muted, vol, vol_inactive clicked, muted = imgui.checkbox('Mute', muted) if clicked: vol, vol_inactive = vol_inactive, vol 128

Slide 122

Slide 122 text

Sliding both channels to 0 implies muting # ... vol[r] = slider('##r', vol[r]) imgui.same_line() imgui.text('R') if vol[l] or vol[r]: muted = False vol_inactive = [0, 0] 129

Slide 123

Slide 123 text

Finished 130

Slide 124

Slide 124 text

Advanced Topics Timer and Animation 131

Slide 125

Slide 125 text

How to schedule a work in ImGUI? • Executor? • Polling? 132

Slide 126

Slide 126 text

ImGUI has a “Planck time” • Duration of a frame = 1s / fps • Elapsed N frames = elapsed N / imgui.get_io().framerate seconds 133

Slide 127

Slide 127 text

2D Graphics • Draw list draw_list = imgui.get_window_draw_list() draw_list = imgui.get_overlay_draw_list() • Methods add_line, add_rect, add_circle, add_text, add_image, … 134

Slide 128

Slide 128 text

Animation • If you refresh fast enough, anything can turn into animation • A matter of what to draw and when 135

Slide 129

Slide 129 text

Example of animation 136

Slide 130

Slide 130 text

Code fps = imgui.get_io().framerate elapsed = self.__count / fps if elapsed % .7 < .5: self.__draw_border() if elapsed > 2.: self.__init__() self.__count += 1 137

Slide 131

Slide 131 text

Summary • ImGUI exposes states rather than events • ImGUI is suitable for GUI that follows data • Design ImGUI widgets ≈ design data structures • Have fun 138

Slide 132

Slide 132 text

Questions? @lichray 139