Getting the most out of Google Search Console API with RegEx

Slide 1

Slide 1 text

@SPEAKERNAME/#SMX Getting the most out of the Google Search Console API with RegEx

Slide 2

Slide 2 text

@SPEAKERNAME/#SMX stagnant

Slide 3

Slide 3 text

@SPEAKERNAME/#SMX declining

Slide 4

Slide 4 text

@SPEAKERNAME/#SMX core update drop 🤔

Slide 5

Slide 5 text

@SPEAKERNAME/#SMX

Slide 6

Slide 6 text

@SPEAKERNAME/#SMX

Slide 7

Slide 7 text

@SPEAKERNAME/#SMX Data Studio GSC Analysis in

Slide 8

Slide 8 text

@SPEAKERNAME/#SMX https://twitter.com/aleyda/status/1461358112745537545

Slide 9

Slide 9 text

@SPEAKERNAME/#SMX https://datastudio.google.com/u/0/reporting/1Fm7x1vc0vLokRhGf0WqaMd52mw7wjaSI/page/6zXD

Slide 10

Slide 10 text

@SPEAKERNAME/#SMX https://twitter.com/DataChaz/status/1509198629361303560

Slide 11

Slide 11 text

@SPEAKERNAME/#SMX https://twitter.com/HannahRampton/status/1513923100768935939

Slide 12

Slide 12 text

@SPEAKERNAME/#SMX https://www.hannahrampton.co.uk/v2-search-console-explorer-studio/

Slide 13

Slide 13 text

@SPEAKERNAME/#SMX https://twitter.com/HannahRampton/status/1226166365788233728

Slide 14

Slide 14 text

@SPEAKERNAME/#SMX https://www.hannahrampton.co.uk/search-console-explorer-sheet-free-google-sheet/

Slide 15

Slide 15 text

@SPEAKERNAME/#SMX 🐢

Slide 16

Slide 16 text

@SPEAKERNAME/#SMX https://twitter.com/anthonydnelson/status/967059108905078786

Slide 17

Slide 17 text

@SPEAKERNAME/#SMX Sampling Problem Overcoming the

Slide 18

Slide 18 text

@SPEAKERNAME/#SMX https://twitter.com/noahlearner/status/1339621958665682944

Slide 19

Slide 19 text

@SPEAKERNAME/#SMX https://twitter.com/Similar_ai/status/1511008285951991811

Slide 20

Slide 20 text

@SPEAKERNAME/#SMX https://twooctobers.com/two-octobers-explorer-for-search/

Slide 21

Slide 21 text

@SPEAKERNAME/#SMX https://twooctobers.com/two-octobers-explorer-for-search/

Slide 22

Slide 22 text

@SPEAKERNAME/#SMX https://similar.ai/blog/closing-google-search-console-sampling-gap/ • Adding 10 well-chosen sub-directories as GSC profiles can close the gap to almost 75% • The gap saturates towards the end because of longer tail sub-directories

Slide 23

Slide 23 text

@SPEAKERNAME/#SMX Regular Expressions Getting more with

Slide 24

Slide 24 text

@SPEAKERNAME/#SMX https://twitter.com/jackson_lo/status/1352997899806912513

Slide 25

Slide 25 text

@SPEAKERNAME/#SMX https://twitter.com/googlesearchc/status/1379775388193320962

Slide 26

Slide 26 text

@SPEAKERNAME/#SMX

Slide 27

Slide 27 text

@SPEAKERNAME/#SMX https://xkcd.com/208/

Slide 28

Slide 28 text

@SPEAKERNAME/#SMX https://xkcd.com/208/

Slide 29

Slide 29 text

@SPEAKERNAME/#SMX RE2! https://xkcd.com/208/

Slide 30

Slide 30 text

@SPEAKERNAME/#SMX https://www.reddit.com/r/ProgrammerHumor/comments/tdtdfn/id_like_you_to_meet_regex/ 🥲

Slide 31

Slide 31 text

@SPEAKERNAME/#SMX https://www.reddit.com/r/ProgrammerHumor/comments/tdtdfn/id_like_you_to_meet_regex/ 🤪

Slide 32

Slide 32 text

@SPEAKERNAME/#SMX https://github.com/google/re2/wiki/Syntax

Slide 33

Slide 33 text

@SPEAKERNAME/#SMX Informational Queries RegEx

Slide 34

Slide 34 text

@SPEAKERNAME/#SMX https://twitter.com/danielkcheung/status/1524314760346365954

Slide 35

Slide 35 text

@SPEAKERNAME/#SMX https://twitter.com/seo_notebook/status/1381792930197831687

Slide 36

Slide 36 text

@SPEAKERNAME/#SMX are, can, can't, could, couldn't, did, didn't, do, does, doesn't, how, if, is, isn't, should, shouldn't, was, wasn't, were, weren't, what, when, where, who, whom, whose, why, will, won't, would, wouldn't

Slide 37

Slide 37 text

@SPEAKERNAME/#SMX

Slide 38

Slide 38 text

@SPEAKERNAME/#SMX

Slide 39

Slide 39 text

@SPEAKERNAME/#SMX

Slide 40

Slide 40 text

@SPEAKERNAME/#SMX

Slide 41

Slide 41 text

@SPEAKERNAME/#SMX https://regexper.com/

Slide 42

Slide 42 text

@SPEAKERNAME/#SMX https://regexper.com/

Slide 43

Slide 43 text

@SPEAKERNAME/#SMX https://regexper.com/

Slide 44

Slide 44 text

@SPEAKERNAME/#SMX

Slide 45

Slide 45 text

@SPEAKERNAME/#SMX https://twitter.com/danielwaisberg/status/1402979440183939074

Slide 46

Slide 46 text

@SPEAKERNAME/#SMX https://twitter.com/lazarinastoy/status/1461302669172166661

Slide 47

Slide 47 text

@SPEAKERNAME/#SMX Branded Queries RegEx

Slide 48

Slide 48 text

@SPEAKERNAME/#SMX

Slide 49

Slide 49 text

@SPEAKERNAME/#SMX aamaung, damsung, mamsang, sam sung, samaung, samdung, samesung, sameung, samgsung, samgung, samsang, samsaung, samsgu, samshgg, samshng, samsing, samsnug, samssung, samsu, samsuag, samsubg, samsubng, samsug, samsumg, samsumng, samsun g, samsunb, samsund, samsund, samsunh, samsunt …

Slide 50

Slide 50 text

@SPEAKERNAME/#SMX (s+|a|d|z)[a-z\s]{1,4}m? [a-z\s]{1,6}(m|u|n|g|t|h|b|v) Consider: • Main letters • Consonants • Letters surrounding hard consonants

Slide 51

Slide 51 text

@SPEAKERNAME/#SMX samsung galaxy note galaxy samsung new samsung TV Consider: • Start of string • Surrounded by spaces • End of string 🔍 🔍 🔍

Slide 52

Slide 52 text

@SPEAKERNAME/#SMX (^|\s)(s+|a|d|z)[a-z\s]{1,4}m? [a-z\s]{1,6}(m|u|n|g|t|h|b|v)(\s|$) • Start of string = ^ • Surrounded by spaces = \s • End of string = $

Slide 53

Slide 53 text

@SPEAKERNAME/#SMX https://twitter.com/ChouinardJC/status/1405471189653360646

Slide 54

Slide 54 text

@SPEAKERNAME/#SMX With the API Going deeper

Slide 55

Slide 55 text

@SPEAKERNAME/#SMX https://twitter.com/GregBernhardt4/status/1462797592664887305

Slide 56

Slide 56 text

@SPEAKERNAME/#SMX https://importsem.com/calculate-gsc-ctr-stats-by-position-using-python-for-seo/

Slide 57

Slide 57 text

@SPEAKERNAME/#SMX https://www.oncrawl.com/technical-seo/extract-data-google-search-console-data-analysis-in-python/

Slide 58

Slide 58 text

@SPEAKERNAME/#SMX US Product URLs // ///p- /tvs/ /tvs/4k-led/ /tvs/4k-led/p-4k-hd-model-314 i18n Language & Country URLs // /-/ /fr/ /fr-br/ /ae-ar/ /fr/tvs/4k/ /fr-br/tvs/4k-led/p-4k-hd-model-314 /ae-ar/tvs/4k-led/p-4k-hd-model-314

Slide 59

Slide 59 text

@SPEAKERNAME/#SMX Include /([^/]+/){1,2}p? • Any character that’s not a slash = [^/]+ • 1 or 2 directories = /){1,2} • Sometimes followed by a product slug = p? Get All US PLPs + PDPs and NOT i18n pages Exclude /[a-zA-Z]{2}|[a-zA-Z]{2}-[a-zA-Z]{2}/ • Any 2 letter directory = [a-zA-Z]{2} • 2 letter + 2 letter lang-country combo = [a-zA-Z]{2}-[a-zA-Z]{2}

Slide 60

Slide 60 text

@SPEAKERNAME/#SMX

Slide 61

Slide 61 text

@SPEAKERNAME/#SMX

Slide 62

Slide 62 text

@SPEAKERNAME/#SMX https://twitter.com/eywu/status/1508992377372901376

Slide 63

Slide 63 text

@SPEAKERNAME/#SMX https://twitter.com/victorpan/status/629291612812746753

Slide 64

Slide 64 text

@SPEAKERNAME/#SMX https://developers.google.com/webmaster-tools/v1/api_reference_index

Slide 65

Slide 65 text

@SPEAKERNAME/#SMX https://developers.google.com/webmaster-tools/v1/api_reference_index

Slide 66

Slide 66 text

@SPEAKERNAME/#SMX https://developers.google.com/webmaster-tools/v1/api_reference_index

Slide 67

Slide 67 text

@SPEAKERNAME/#SMX https://www.postman.com/

Slide 68

Slide 68 text

@SPEAKERNAME/#SMX https://www.postman.com/

Slide 69

Slide 69 text

@SPEAKERNAME/#SMX

Slide 70

Slide 70 text

@SPEAKERNAME/#SMX https://www.jcchouinard.com/how-to-get-google-search-console-api-keys/

Slide 71

Slide 71 text

@SPEAKERNAME/#SMX

Slide 72

Slide 72 text

@SPEAKERNAME/#SMX

Slide 73

Slide 73 text

@SPEAKERNAME/#SMX

Slide 74

Slide 74 text

@SPEAKERNAME/#SMX

Slide 75

Slide 75 text

@SPEAKERNAME/#SMX

Slide 76

Slide 76 text

@SPEAKERNAME/#SMX

Slide 77

Slide 77 text

@SPEAKERNAME/#SMX { "rows": [ { "keys": ["2022-06-15"],"clicks": 359756,"impressions": 7294403,"ctr": 0.049319457671861563,"position": 9.128287263536166}],"responseAggregationTy pe": "byPage" } Export JSON to file: samsung-g11n.json

Slide 78

Slide 78 text

@SPEAKERNAME/#SMX https://stedolan.github.io/jq/

Slide 79

Slide 79 text

@SPEAKERNAME/#SMX { "rows": [ { "keys": ["2022-06-15"], "clicks": 359756, "impressions": 7294403, "ctr": 0.049319457671861563, "position": 9.128287263536166 } ], "responseAggregationType": "byPage" } jq . < samsung-g11n.json

Slide 80

Slide 80 text

@SPEAKERNAME/#SMX jq '.rows | [.[] | { _Date: .keys[0], Clicks: .clicks|tostring, Impressions: .impressions|tostring, CTR: .ctr, Position: .position }]' < samsung-g11n.json

Slide 81

Slide 81 text

@SPEAKERNAME/#SMX [ { "_Date": "2022-06-15", "Clicks": "359756", "Impressions": "7294403", "CTR": "0.049319457671861563", "Position": "9.128287263536166" } ]

Slide 82

Slide 82 text

@SPEAKERNAME/#SMX https://github.com/TomWright/dasel

Slide 83

Slide 83 text

@SPEAKERNAME/#SMX jq '.rows | [.[] | { _Date: .keys[0], Clicks: .clicks|tostring, Impressions: .impressions|tostring, CTR: .ctr, Position: .position }]' < samsung-g11n.json > samsung.json

Slide 84

Slide 84 text

@SPEAKERNAME/#SMX cat samsung.json | dasel -r json -w csv > data.csv

Slide 85

Slide 85 text

@SPEAKERNAME/#SMX jq '.rows | [.[] | { _Date: .keys[0], Clicks: .clicks|tostring, Impressions: .impressions|tostring, CTR: .ctr, Position: .position }]' < samsung-g11n.json | dasel -r json -w csv > data.csv

Slide 86

Slide 86 text

@SPEAKERNAME/#SMX

Slide 87

Slide 87 text

@SPEAKERNAME/#SMX https://en.ryte.com/

Slide 88

Slide 88 text

@SPEAKERNAME/#SMX

Slide 89

Slide 89 text

@SPEAKERNAME/#SMX @eywu

Slide 90

Slide 90 text

@SPEAKERNAME/#SMX APPENDIX

Slide 91

Slide 91 text

@SPEAKERNAME/#SMX After this session, you’ll be able to: • Extract the maximum amount of data from the GSC API • Utilize the GSC API in your SEO process with Spreadsheets, Data Studio, or python • Learn how to use regular expressions (regEx) to filter difficult URLs Getting the most out of the Google Search Console API Google Search Console is an amazing tool that provides invaluable search data by real users directly from Google. While the charts and tables are friendly to work with, a large part of the data is not accessible from the UI. The only way to get to this hidden data is to use the API and extract all that valuable search data that is available to you, only if you know how.

Slide 92

Slide 92 text

@SPEAKERNAME/#SMX https://twitter.com/garabatokid/status/1147063121678389253

Slide 93

Slide 93 text

@SPEAKERNAME/#SMX https://twitter.com/garabatokid/status/1147063121678389253

Slide 94

Slide 94 text

@SPEAKERNAME/#SMX

Slide 95

Slide 95 text

@SPEAKERNAME/#SMX