Nobody loves the native search

You can search on the frontend and on the backend of WordPress, but this search is … simple.

It is just a full text SQL search. It lacks many features, like highlighting, synonyms, phrase search, etc. And many page builders, even the block editor, create more problems.

The search doesn’t work at all or there are too many “false positive” search results.

Let’s take a deep dive and find out what is broken and how to fix it!

Further details: https://torstenlandsiedel.de/wceu2024/

Lightning Talk at WordCamp Europe 2024 in Turino

Transcript

1. Nobody loves the native search Torsten Landsiedel @zodiac1978
2. Works for me?
3. Where and what does WordPress not search? ➔ Categories, Tags, Custom Taxonomies ➔ Custom Fields ➔ Comments ➔ Widgets ➔ Slugs ➔ Authors #WCEU
4. Where does WP search? The columns post_title, post_excerpt and post_content from the posts table are searched. #WCEU
5. Too many results … The story of looking for a syntax highlighting extension on a blog of a friend. #WCEU
6. False positives <!-- wp:syntaxhighlighter/code {"language":"php"} --> <pre class="wp-block-syntaxhighlighter-code"> function wp_example( ) { echo ‘Hello WC Europe!’; } add_filter( 'wp_footer', wp_example );</pre> <!-- /wp:syntaxhighlighter/code --> #WCEU
7. Content, not markup <!-- wp:syntaxhighlighter/code {"language":"php"} --> <pre class="wp-block-syntaxhighlighter-code"> function wp_example( ) { echo ‘Hello WC Europe!’; } add_filter( 'wp_footer', wp_example );</pre> <!-- /wp:syntaxhighlighter/code --> #WCEU
8. Polluted content Searching for “syntaxhighlighter”, “image”, “background”, “paragraph”, etc. is not possible without getting tons of incorrect search results. #WCEU
9. Dev feedback?
10. “Unfortunately, this is a known issue in WordPress […], if you search for "table", you'll get results including <table> tags. ” #WCEU
11. #WCEU “MySQL's string searching isn't capable of dealing with this kind of contextual parsing.”
12. “If you require 100% accurate search results, the best option is to use a dedicated search engine, like Elasticsearch. ” - Gary Pendergast #WCEU
13. “One opportunity is for [...] search backends to render blocks fully before sending post_content into the index. #WCEU
14. #WCEU “I don't have any great ideas for how to resolve this with MySQL. I'd love to hear of a solution if someone has one.” - Daniel Bachhuber
15. Solutions, please!
16. ✨ REGEXP_REPLACE ✨ Replacements with regular expressions Only for MySQL 8.0.4+ and MariaDB 10.0.5+ #WCEU
17. Ignore comment markup REGEXP_REPLACE({$wpdb->posts}.post_content, '<!--.+?-->', '') LIKE '%{$search_query}%'"; #WCEU
18. RegEx will have a performance impact. A better solution would be a separate index. Performance? #WCEU
19. Ignoring the complete markup? What about filenames or alternative text? What about shortcodes? What about backward compatibility? What about non encoded characters? What about JSON? #WCEU
20. Focus on the four phases
21. Missing maintenance The search is a fundamental feature, but it is lacking innovation and extensibility in core. #WCEU
22. Missing tools “The software should provide a framework for translation to make it globally accessible to speakers of all languages.“ (WordPress Foundation) - but we are missing safeguards for settings hidden in the translation #WCEU
23. Missing settings Sometimes we overrate “Decisions, not options” and should give users a choice. Basic settings for SEO, sitemaps or search would be helpful. #WCEU
24. Wishlist
25. ➔ Let’s add more features to the internal search ➔ Let’s add very basic search settings ➔ Let’s add safeguards for translation settings (like the stop word list) - check your locale! I need your help! #WCEU
26. https:/ /torstenlandsiedel.de/wceu2024/ You want to know more? #WCEU
27. Thank you! #WCEU