Fancy Rich Text on Android - Using Roguso as an Example

Android as a mobile platform has proven much success over the past couple of years. However, compared to the web platform, which most of us are familiar with by means of desktop browsers, there are additional challenges. The same set of features that used to work well with link clicks and rich multimedia on a big screen now need to fit into a smaller screen. Providing the same rich text experience on Android is essential but also surprisingly difficult. In this talk, we will use an Android app Roguso to showcase and discuss how rich text can be implemented on Android.

Transcript

1. Fancy Rich Text on Android Using Roguso as an Example David Wu @wuman blog.wu-man.com Taipei Google Technology User Group January 18, 2012 1 Wednesday, January 18, 12
2. Desktop Experience 2 Wednesday, January 18, 12
3. Mobile Experience 3 Wednesday, January 18, 12
4. Some Examples 4 Wednesday, January 18, 12
5. http:/ /www.roguso.com 5 Wednesday, January 18, 12
6. Font size/color Typeface Style Images Qualifier Clickable links Fancy Markup 6 Wednesday, January 18, 12
7. Direct User Interaction 7 Wednesday, January 18, 12
8. Emoticons that finally work! Animated gif that’s supposed to animate 8 Wednesday, January 18, 12
9. URL shortening Emoticons Image upload progress updates 9 Wednesday, January 18, 12
10. Behind the scenes 10 Wednesday, January 18, 12
11. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() Dissecting the TextView and styled Spans MovementMethod Emoticons Bonus: TextWatcher and Tokenizer applications 11 Wednesday, January 18, 12
12. Text + Markup 12 Wednesday, January 18, 12
13. int getSpanStart(Object) int getSpanEnd(Object) T[] getSpans(int start, int end, Class<T>) void setSpan(Object, int, int, int flags) void removeSpan(Object) CharSequence Interface Implementation Spanned SpannedString Spannable SpannableString Editable SpannableStringBuilder Editable insert(int where, CharSequence) Editable replace(int, int, CharSequence) Editable delete(int, int) void clear() void clearSpans() immutable markup, immutable text mutable markup, immutable text immutable markup, immutable text 13 Wednesday, January 18, 12
14. TextView.setText(CharSequence, BufferType.SPANNABLE) EditText.setText(CharSequence, BufferType.EDITABLE) 14 Wednesday, January 18, 12
15. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() Dissecting the TextView and styled Spans MovementMethod Emoticons Bonus: TextWatcher and Tokenizer applications 15 Wednesday, January 18, 12
16. Linkify textView.setText(someContent) Linkify.addLinks(textView, Linkify.ALL) Linkify.ALL Linkify.EMAIL_ADDRESSES Linkify.MAP_ADDRESSES Linkify.PHONE_NUMBERS Linkify.WEB_URLS 16 Wednesday, January 18, 12
17. Linkify 17 Wednesday, January 18, 12
18. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() Dissecting the TextView and styled Spans MovementMethod Emoticons Bonus: TextWatcher and Tokenizer applications 18 Wednesday, January 18, 12
19. Spanned Html.fromHtml(String) Spanned Html.fromHtml(String, ImageGetter, TagHandler) 19 Wednesday, January 18, 12
20. ImageGetter Drawable getDrawable(String src) 20 Wednesday, January 18, 12
21. TagHandler void handleTag( boolean opening, String tag, Editable output, XMLReader xmlReader) 21 Wednesday, January 18, 12
22. Html.fromHtml() <br /> <p /> <div /> <strong />, <b /> <em />, <cite />, <dfn />, <i /> <big /> <small /> <font color=”” face=””/> <blockquote /> <tt /> / / monospace <a href=””/> <u /> <sup /> <sub /> <h1 /> .. <h6 /> <img /> 22 Wednesday, January 18, 12
23. Html.fromHtml() 23 Wednesday, January 18, 12
24. Problem #1 with Html.fromHtml() Unable to intercept or override handling of common tags 24 Wednesday, January 18, 12
25. Problem #1 with Html.fromHtml() Make a copy of the Html class and TagSoup library to include in your own application package 25 Wednesday, January 18, 12
26. Problem #2 with Html.fromHtml() Small objects created and recycled frequently may impact scrolling performance Use object pool Check out techniques introduced in my other presentation (http:/ /slidesha.re/andperf) HTML parsing is a bottleneck 26 Wednesday, January 18, 12
27. Problem #2 with Html.fromHtml() 27 Wednesday, January 18, 12
28. Problem #2 with Html.fromHtml() Pre-parse in a background thread Cache parsed results Stop the thread to avoid short-term memory leaks Careful about memory in Drawable caching as well: Drawable holds a reference to Activity 28 Wednesday, January 18, 12
29. Problem #3 with Html.fromHtml() Native implementation of <img /> handles only images with dimensions known ahead of time. Drawable drawable = imageGetter.getDrawable(src); text.append(“\uFFFC”); text.setSpan(new ImageSpan(drawable, src), len, text.length(), Spannable.SPAN_EXCLUSIVE_EXCLUSIVE); 29 Wednesday, January 18, 12
30. Problem #3 with Html.fromHtml() ImageGetter should return a Drawable which, upon remote image retrieval, triggers the containing TextView to relayout. Might need to be careful if you’re also caching the Drawables. Make sure the Drawable is referencing the correct TextView within ListView. Use AbsListView.RecyclerListener 30 Wednesday, January 18, 12
31. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() Dissecting the TextView and styled Spans MovementMethod Emoticons Bonus: TextWatcher and Tokenizer applications 31 Wednesday, January 18, 12
32. Spans Markup = changes the style or behavior of a segment of the text buffer. Span is Android’s implementation of markup 32 Wednesday, January 18, 12
33. View System Hierarchy of parent layout and child views Measure, layout, draw 33 Wednesday, January 18, 12
34. TextView CharSequence String Spanned Spannable Editable Layout StaticLayout DynamicLayout Paint Measure Layout Draw 34 Wednesday, January 18, 12
35. CharacterStyle MetricAffectingSpan ClickableSpan UnderlineSpan StrikethroughSpan BackgroundColorSpan ForegroundColorSpan MaskFilterSpan RasterizerSpan URLSpan TextAppearanceSpan TypefaceSpan StyleSpan AbsoluteSizeSpan RelativeSizeSpan SuperscriptSpan SubscriptSpan ReplacementSpan DynamicDrawableSpan ImageSpan 35 Wednesday, January 18, 12
36. CharacterStyle MetricAffectingSpan ClickableSpan UnderlineSpan StrikethroughSpan BackgroundColorSpan ForegroundColorSpan MaskFilterSpan RasterizerSpan URLSpan TextAppearanceSpan TypefaceSpan StyleSpan AbsoluteSizeSpan RelativeSizeSpan SuperscriptSpan SubscriptSpan ReplacementSpan DynamicDrawableSpan ImageSpan 36 Wednesday, January 18, 12
37. CharacterStyle MetricAffectingSpan ClickableSpan UnderlineSpan StrikethroughSpan BackgroundColorSpan ForegroundColorSpan MaskFilterSpan RasterizerSpan URLSpan TextAppearanceSpan TypefaceSpan StyleSpan AbsoluteSizeSpan RelativeSizeSpan SuperscriptSpan SubscriptSpan ReplacementSpan DynamicDrawableSpan ImageSpan UpdateLayout 37 Wednesday, January 18, 12
38. CharacterStyle MetricAffectingSpan ClickableSpan UnderlineSpan StrikethroughSpan BackgroundColorSpan ForegroundColorSpan MaskFilterSpan RasterizerSpan URLSpan TextAppearanceSpan TypefaceSpan StyleSpan AbsoluteSizeSpan RelativeSizeSpan SuperscriptSpan SubscriptSpan ReplacementSpan DynamicDrawableSpan ImageSpan UpdateAppearance 38 Wednesday, January 18, 12
39. Many more Span interfaces and classes ParcelableSpan ParagraphStyle AlignmentSpan LeadingMarginSpan LineBackgroundSpan TabStopSpan WrapTogetherSpan LineHeightSpan DrawableMarginSpan IconMarginSpan QuoteSpan BulletSpan EasyEditSpan 39 Wednesday, January 18, 12
40. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() Dissecting the TextView and styled Spans MovementMethod Emoticons Bonus: TextWatcher and Tokenizer applications 40 Wednesday, January 18, 12
41. View System View.onTouchEvent() and ViewGroup.onInterceptTouchEvent() View.onKey*() View.onTrackballEvent() View.onFocusChanged() 41 Wednesday, January 18, 12
42. MovementMethod TextView uses the MovementMethod helper class to help it handle user events on its content. TextView.setMovementMethod() Without a MovementMethod, TextView will not draw highlighted link selections nor handle user events. 42 Wednesday, January 18, 12
43. MovementMethod BaseMovementMethod ArrowKeyMovementMethod ScrollingMovementMethod LinkMovementMethod Interface Implementation onKeyDown() onKeyUp() onKeyOther() onTakeFocus() onTouchEvent() onTrackballEvent() up() down() left() right() home() bottom() leftWord() rightWord() lineStart() lineEnd() pageUp() pageDown() cursor movement and selection scroll text buffer traverses links in text buffer and scrolls if necessary 43 Wednesday, January 18, 12
44. LinkMovementMethod TextView.onTouchEvent() LinkMovementMethod.onTouchEvent() ClickableSpan.onClick() Selection.setSelection(buffer, start, end) 44 Wednesday, January 18, 12
45. Problem #1 with Clicks Clicking on the trailing white space of any line ending with a ClickableSpan will still invoke its ClickableSpan.onClick(). 45 Wednesday, January 18, 12
46. Problem #1 with Clicks public boolean onTouchEvent(...) { ... Layout layout = widget.getLayout(); int line = layout.getLineForVertical(y); int off = layout.getOffsetForHorizontal(line, x); ClickableSpan[] link = buffer.getSpans(off, off, ClickableSpan.class); .... if ( action == MotionEvent.ACTION_UP ) { link[0].onClick(widget); } ... } 46 Wednesday, January 18, 12
47. Layout TextView uses the Layout class to manage text layout. Layout is a base abstract class. TextView uses StaticLayout normally for immutable text. TextView uses DynamicLayout for mutable text such as Spannable and its derivatives. 47 Wednesday, January 18, 12
48. Layout getLine*() / / Width, Start, End, Top, Ascent, Bounds, etc. getOffset*() getParagraph*() getText() getWidth(), getHeight() getPaint() 48 Wednesday, January 18, 12
49. Fix for LinkMovementMethod public boolean onTouchEvent(...) { ... Layout layout = widget.getLayout(); int line = layout.getLineForVertical(y); int off = layout.getOffsetForHorizontal(line, x); int maxLineRight = layout.getLineWidth(line) if ( x <= maxLineRight + slop ) { ClickableSpan[] link = buffer.getSpans(off, off, ClickableSpan.class); } .... if ( action == MotionEvent.ACTION_UP ) { link[0].onClick(widget); } 49 Wednesday, January 18, 12
50. Problem #2 with Clicks LinkMovementMethod also handles highlighted link selection when the link is being clicked. There is a bug that makes the selection stay there even after the click event is consumed. Need to do Selection.removeSelection after consuming the touch event in LinkMovementMethod 50 Wednesday, January 18, 12
51. Problem #3 with Clicks ListActivity.onListItemClick() will not work if a MovementMethod is set. 51 Wednesday, January 18, 12
52. Problem #3 with Clicks MovementMethod is required to handle link clicks, but setMovementMethod will also do: setFocusable(true) setClickable(true) setLongClickable(true) TextView (View) will eat up the touch event even when no link is clicked 52 Wednesday, January 18, 12
53. Problem #3 with Clicks Workaround is to do the following: This will kill link highlights, so we need to do more hacks at onDraw() to draw the highlights ourselves. 53 Wednesday, January 18, 12
54. Problem #4 with Clicks Layout.getSelectionPath(int start, int end, Path dest) uses a naive way of calculating the path. 54 Wednesday, January 18, 12
55. In a Nutshell, Customizing the TextView in Android is surprisingly difficult. Make a copy of TextView rather than extend it? 55 Wednesday, January 18, 12
56. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() MovementMethod Dissecting the TextView and styled Spans Emoticons Bonus: TextWatcher and Tokenizer applications 56 Wednesday, January 18, 12
57. Inline Emoticons Best solution would be to modify the skia rendering library; Too much work Or to include another rendering library that already supports animated GIF; ࡴ㮦༻ڇ౛ Or to use Movie class as illustrated in API Demos Doesn’t work on merged animated GIFs 57 Wednesday, January 18, 12
58. TextWatcher and SpanWatcher They are themselves markup objects just like any other Spans void afterTextChanged(Editable) void beforeTextChanged(CharSequence, int, int, int) void onTextChanged(CharSequence, int, int, int) void onSpanAdded(Spannable, Object, int, int) void onSpanChanged(Spannable, Object, int, int, int, int) void onSpanRemoved(Spannable, Object, int, int) 58 Wednesday, January 18, 12
59. TextWatcher and SpanWatcher Users of TextView has access in two ways: Detected and invoked directly by Spannable or Editable TextView also allows external objects to add/remove TextWatchers as listeners: void TextView.addTextChangedListener(TextWatcher) This is accomplished by TextView itself setting a ChangeWatcher into the current Spannable upon setText(). 59 Wednesday, January 18, 12
60. How TextView itself uses TextWatcher and SpanWatcher In short, TextView itself listens for changes in both text and markup via its own ChangeWatcher object. It redirects text changes to external TextWatcher listeners. It does the following things (if applicable) upon a span change: Invalidate the cursor Notifies a selection change Invalidate() and checkForResize() Inform the IMS and the current extract editor Use a bookkeeping flag to remind itself later at onDraw() a new highlighted selection path should be used. 60 Wednesday, January 18, 12
61. Inline Emoticons Implement them as a subclass of DynamicDrawableSpan holding an AnimationDrawable Drawbacks (puns intended) Need to separate layers of an animated GIF yourself Only works for a pre-defined set of animated GIFs 61 Wednesday, January 18, 12
62. Inline Emoticons AnimationDrawables come with start() and stop() methods. They need to be stopped or else you will have a unstoppable Looper event. What’s worse is that it’s usually also going to be a memory leak because the Drawable holds a reference to a Context. You do that with a SpanWatcher. This way you can stop the animation at onSpanRemoved(). You might also want to stop the AnimationDrawable when the associated TextView is detached from the window. You can find out about this via the View.onDetachedFromWindow() callback. 62 Wednesday, January 18, 12
63. AnimationDrawable in Spans Applications other than just emoticons! 63 Wednesday, January 18, 12
64. Editing text Photo upload Post Editing text Photo upload Post AnimationDrawable in Spans 64 Wednesday, January 18, 12
65. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() MovementMethod Dissecting the TextView and styled Spans Emoticons Bonus: TextWatcher and Tokenizer applications 65 Wednesday, January 18, 12
66. TextWatcher application Use it to update the word count or the remaining character count 66 Wednesday, January 18, 12
67. MultiAutoCompleteTextView.Tokenizer 67 Wednesday, January 18, 12
68. MultiAutoCompleteTextView.Tokenizer int findTokenStart(CharSequence, int cursor) int findTokenEnd(CharSequence, int cursor) terminateToken(CharSequence) 68 Wednesday, January 18, 12
69. MultiAutoCompleteTextView.Tokenizer @Override public int findTokenStart(CharSequence text, int cursor) { int start = cursor; while (start > 0 && text.charAt(start - 1) != ' ') { start--; } while (start < cursor && text.charAt(start) == ' ') { start++; } if (start < cursor && text.charAt(start) == '@') { start++; } else { start = cursor; } return start; } 69 Wednesday, January 18, 12
70. MultiAutoCompleteTextView.Tokenizer @Override public CharSequence terminateToken(CharSequence text) { if (text instanceof Spanned) { SpannableString spanText = new SpannableString(text); TextUtils.copySpansFrom((Spanned) text, 0, text.length(), Object.class, spanText, 0); setAutoComplete(false); return spanText; } else { setAutoComplete(false); return text; } } 70 Wednesday, January 18, 12
71. MultiAutoCompleteTextView.Tokenizer private void setAutoComplete(boolean enabled) { int inputType = mEditbox.getInputType(); if (enabled) { inputType |= InputType.TYPE_TEXT_FLAG_AUTO_COMPLETE; } else { inputType &= (~InputType.TYPE_TEXT_FLAG_AUTO_COMPLETE); } mEditbox.setRawInputType(inputType); } 71 Wednesday, January 18, 12
72. Agenda Some examples Background knowledge about Spanned Linkify Html.fromHtml() MovementMethod Dissecting the TextView and styled Spans Emoticons Bonus: TextWatcher and Tokenizer applications 72 Wednesday, January 18, 12
73. Thank you! david@wu-man.com @wuman blog.wu-man.com http:/ /www.roguso.com 73 Wednesday, January 18, 12