Using Awk: An old-school text processing tool with surprising versatility

Slide 1

Slide 1 text

Using Awk An old-school text processing tool with surprising versatility

Slide 2

Slide 2 text

Postcodes --------- Het koppelen van postcodes aan verschillende soorten publications (dag en weekbladen) gaat via het volgende tabel. Wij houden zelf een tabel bij van postcodes met plaatsnamen, gemeentenamen en grenzen (voor kaartjes.) ### `geospatial_areas` | **Veld** | **Type** | **Beschrijving** | **Voorbeeld** | |---------------|----------|------------------------------|---------------| | `postal_code` | integer | 4 postcode posities | 3775 | | `region_id` | string | Regio code (dag of weekblad) | STAP |

Slide 3

Slide 3 text

COMMENT ON COLUMN geospatial_areas.postal_code IS '4 postcode posities'; COMMENT ON COLUMN geospatial_areas.region_id IS 'Regio code (dag of weekblad)';

Slide 4

Slide 4 text

Awk processes records with ﬁelds [CONDITION] { [ACTIONS] }

Slide 5

Slide 5 text

Awk processes records with ﬁelds [CONDITION] { [ACTIONS] } [CONDITION] { [ACTIONS] }

Slide 6

Slide 6 text

Awk processes records with ﬁelds [CONDITION] { [ACTIONS] } [CONDITION] { [ACTIONS] } BEGIN { [ACTIONS] } END { [ACTIONS] }

Slide 7

Slide 7 text

% awk '{ print }' docs.md

Slide 8

Slide 8 text

% awk '{ print }' docs.md Postcodes --------- Het koppelen van postcodes aan verschillende soorten publications (dag en weekbladen) gaat via het volgende tabel. Wij houden zelf een tabel bij van postcodes met plaatsnamen, gemeentenamen en grenzen (voor kaartjes.) ### `geospatial_areas` | **Veld** | **Type** | **Beschrijving** | **Voorbeeld** | |---------------|----------|------------------------------|

Slide 9

Slide 9 text

% awk '{ print $1 }' docs.md

Slide 10

Slide 10 text

% awk '{ print $1 }' docs.md Postcodes --------- Het Wij ### | |---------------|----------|------------------------------| ---------------| | |

Slide 11

Slide 11 text

% awk -F '\|' '{ print $1 }' docs.md

Slide 12

Slide 12 text

% awk -F '\|' '{ print $1 }' docs.md Postcodes --------- Het koppelen van postcodes aan verschillende soorten publications (dag en weekbladen) gaat via het volgende tabel. Wij houden zelf een tabel bij van postcodes met plaatsnamen, gemeentenamen en grenzen (voor kaartjes.) ### `geospatial_areas`

Slide 13

Slide 13 text

% awk -F '\|' '{ print $2 }' docs.md

Slide 14

Slide 14 text

% awk -F '\|' '{ print $2 }' docs.md **Veld** --------------- `postal_code` `region_id`

Slide 15

Slide 15 text

% awk -F '\|' '/\|/ { print $2 }' docs.md

Slide 16

Slide 16 text

% awk -F '\|' '/\|/ { print $2 }' docs.md **Veld** --------------- `postal_code` `region_id`

Slide 17

Slide 17 text

% awk -F '\|' '/\|/ && !/-|\*/ { print $2 }' docs.md

Slide 18

Slide 18 text

% awk -F '\|' '/\|/ && !/-|\*/ { print $2 }' docs.md `postal_code` `region_id`

Slide 19

Slide 19 text

% awk -F ' *\| *' '/\|/ && !/-|\*/ { print $2 }' docs.md

Slide 20

Slide 20 text

% awk -F ' *\| *' '/\|/ && !/-|\*/ { print $2 }' docs.md `postal_code` `region_id`

Slide 21

Slide 21 text

% awk -F ' *\| *' '/\|/ && !/-|\*/ { print $2, $4 }' docs.md

Slide 22

Slide 22 text

% awk -F ' *\| *' '/\|/ && !/-|\*/ { print $2, $4 }' docs.md `postal_code` 4 postcode posities `region_id` Regio code (dag of weekblad)

Slide 23

Slide 23 text

#!/usr/bin/awk -f BEGIN { FS = " *\| *" } /\|/ && !/-|\*/ { print $2, $4 } `postal_code` 4 postcode posities `region_id` Regio code (dag of weekblad)

Slide 24

Slide 24 text

#!/usr/bin/awk -f BEGIN { FS = " *\| *" } /\|/ && !/-|\*/ && $4 != "" { print $2, $4 } `postal_code` 4 postcode posities `region_id` Regio code (dag of weekblad)

Slide 25

Slide 25 text

#!/usr/bin/awk -f BEGIN { FS = " *\| *" } { gsub("`", "") } /\|/ && !/-|\*/ && $4 != "" { print $2, $4 } postal_code 4 postcode posities region_id Regio code (dag of weekblad)

Slide 26

Slide 26 text

#!/usr/bin/awk -f BEGIN { FS = " *\| *" } { gsub("`", "") } /\|/ && !/-|\*/ && $4 != "" { printf "COMMENT ON COLUMN t.%s IS '%s';\n", $2, $4 } COMMENT ON COLUMN t.postal_code IS '4 postcode posities'; COMMENT ON COLUMN t.region_id IS 'Regio code (dag of weekblad)';

Slide 27

Slide 27 text

Slide 28

Slide 28 text

#!/usr/bin/awk -f BEGIN { FS = " *\| *" } { gsub("`", "") } /^### / { split($0, a, " ") } /\|/ && !/-|\*/ && $4 != "" { printf "COMMENT ON COLUMN %s.%s IS '%s';\n", a[2], $2, $4 }

Slide 29

Slide 29 text

#!/usr/bin/awk -f BEGIN { FS = " *\| *" } { gsub("`", "") } /^### / { split($0, a, " ") } /\|/ && !/-|\*/ && $4 != "" { printf "COMMENT ON COLUMN %s.%s IS '%s';\n", a[2], $2, $4 } COMMENT ON COLUMN geospatial_areas.postal_code IS '4 postcode posities'; COMMENT ON COLUMN geospatial_areas.region_id IS 'Regio code (dag of weekblad)';

Slide 30

Slide 30 text

ruby -F' *\| *' -a -l -n -e '$table = $_.split(" ")[1] if / ^### /; next if !/\|/ || /-|\*/; print gsub('`', '', format "COMMENT ON COLUMN %s.%s IS %s;\n", $table, $F[1], $F[3])' docs.md

Slide 31

Slide 31 text

#!/usr/bin/env ruby -F' *\| *' -a -l -n table = $_.split(" ")[1] if /^### / next if !/\|/ || /-|\*/ || $F[3] == "" print gsub('`', '', format( "COMMENT ON COLUMN %s.%s IS %s;\n", $table, $F[1], $F[3] ) )