Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to test an urban theory - Data Science Lecture

Marco De Nadai
November 12, 2020

How to test an urban theory - Data Science Lecture

Slides for the lecture "Testing an urban theory" @ University of Trento

Marco De Nadai

November 12, 2020
Tweet

More Decks by Marco De Nadai

Other Decks in Science

Transcript

  1. Testing an urban theory
    Marco De Nadai (http://www.marcodena.it)

    View Slide

  2. 2
    Cities have always been studied
    IDEAL CITY
    (XV century)

    View Slide

  3. 3
    Cities have always been studied
    IDEAL CITY
    (XV century)
    SYSTEM
    (XIX century)

    View Slide

  4. 4
    Cities have always been studied
    IDEAL CITY
    (XV century)
    SYSTEM
    (XIX century)
    LIVING ORGANISM
    (XX century)

    View Slide

  5. 5
    Understand cities

    View Slide

  6. 6
    Understand cities
    New data

    View Slide

  7. 7
    Understand cities
    New methods
    New data

    View Slide

  8. How can we test an urban theory?
    8
    1
    Take the
    theory
    “Operationalize”
    the theory
    Does it work?
    Is it still valid?
    Why does it
    matter?
    2 3 4

    View Slide

  9. 9
    Follow and ask please!
    How to test an urban
    theory
    1
    There are no stupid
    questions
    2

    View Slide

  10. Take the theory
    STEP 1

    View Slide

  11. The theory: Jane Jacobs
    One of the most influential books in
    city planning
    • planning models that dominated
    mid-century planning
    • American housing policy (HOPE VI)
    • Melbourne, Toronto etc.
    11
    1 2 3 4
    THEORY TEST
    Klemek, C. (2011) ‘Dead or Alive at Fifty? Reading Jane Jacobs on
    her Golden Anniversary’ Dissent, Vol. 58, No. 2, 75–79.

    View Slide

  12. The theory: not tested!
    • Not empirically tested until 2015
    • Tested in Seoul, from costly surveys
    collected in years
    • Theory from 1961!
    12
    1 2 3 4
    THEORY TEST
    Sung, Hyungun, Sugie Lee, and SangHyun Cheon. "Operationalizing Jane
    Jacobs’s Urban Design Theory Empirical Verification from the Great City of
    Seoul, Korea." Journal of Planning Education and Research (2015.

    View Slide

  13. The theory: Jane Jacobs
    One of the most influential books in
    city planning
    • Death: caused by the elimination
    of pedestrian activity
    • Life: created by a vital urban fabric
    at all times of the day
    13
    1 2 3 4
    THEORY TEST

    View Slide

  14. Jacobs’ diversity conditions
    Diversity => Urban vitality
    There are 4 diversity conditions
    To be ensured in each city’s district
    (10,000+ inhabitants)
    14
    1 2 3 4
    THEORY TEST
    SMALL
    BLOCKS
    LAND USE
    AGED
    BUILDINGS
    DENSITY

    View Slide

  15. Land Use Mix
    2+ primary uses (contemporarily)
    JACOBS’ VIEW: People come for different
    purposes, continuously
    EFFECT: “sidewalk ballet” and “eyes on
    the street”
    15
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    1 2 3 4
    THEORY TEST

    View Slide

  16. Small blocks
    City blocks should be small/short
    LAND USE
    SMALL
    BLOCKS
    16
    1 2 3 4
    THEORY TEST
    AGED
    BUILDINGS
    DENSITY
    JACOBS’ VIEW: improves walkability
    EFFECT: Increase face-to-face interactions

    View Slide

  17. Aged buildings
    Buildings mixed (age and types)
    17
    AGED
    BUILDINGS
    1 2 3 4
    THEORY TEST
    LAND USE
    DENSITY
    SMALL
    BLOCKS
    JACOBS’ VIEW: To ensure economic
    diversity
    EFFECT: high-/low-income residents
    new/small enterprises

    View Slide

  18. Density
    Concentration of people and enterprises
    JACOBS’ VIEW: People have a reason
    to live in a district
    EFFECT: Attract people
    18
    SMALL
    BLOCKS
    DENSITY
    1 2 3 4
    THEORY TEST
    LAND USE
    AGED
    BUILDINGS

    View Slide

  19. Necessary, diversity conditions
    All four factors are necessary
    19
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    1 2 3 4
    THEORY TEST

    View Slide

  20. Border Vacuums
    • Patches of land dedicated to one
    single use
    • They could be either bad and good:
    • Parks are good for pedestrian activity
    • But they are exposed to criminality and
    deprivation if not well managed (e.g. night)
    20
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 2 3 4
    THEORY TEST

    View Slide

  21. “Operationalize” the theory
    STEP 2

    View Slide

  22. The data:
    22
    1 3 4
    THEORY TEST 2

    View Slide

  23. The data:
    23
    1 3 4
    THEORY TEST 2
    Get the data at: https://developer.foursquare.com/

    View Slide

  24. The data:
    24
    1 3 4
    THEORY TEST 2

    View Slide

  25. The data:
    25
    1 3 4
    THEORY TEST 2

    View Slide

  26. The data:
    26
    1 3 4
    THEORY TEST 2
    Get the data at: https://overpass-turbo.eu/

    View Slide

  27. The data:
    27
    1 3 4
    THEORY TEST 2

    View Slide

  28. The data:
    28
    1 3 4
    THEORY TEST 2
    • Urban Atlas: https://land.copernicus.eu/local/urban-
    atlas/urban-atlas-2012

    View Slide

  29. The data:
    29
    1 3 4
    THEORY TEST 2
    • ISTAT (Census): https://www.istat.it/it/archivio/104317

    View Slide

  30. 30
    2
    1 3 4
    URBAN DESCRIPTION
    GIS data
    “Operationalize” the theory

    View Slide

  31. 31
    2
    1 3 4
    URBAN DESCRIPTION
    GIS data
    “Operationalize” the theory

    View Slide

  32. 32
    2
    1 3 4
    URBAN DESCRIPTION
    GIS data
    “Operationalize” the theory

    View Slide

  33. 33
    2
    1 3 4
    URBAN DESCRIPTION
    GIS data
    “Operationalize” the theory

    View Slide

  34. 34
    2
    1 3 4
    URBAN DESCRIPTION
    GIS data
    Does it work?
    Is it still valid?
    3
    “Operationalize” the theory

    View Slide

  35. “Operationalize” Land Use Mix
    For district :
    !
    = − '
    "∈$
    !,"
    log(!,"
    )
    log ||
    !,#: % square footage of land use
    : {residential, commercial, recreation}
    35
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 3 4
    THEORY TEST 2
    Ref: R. Cervero. Land-use mixing and suburban mobility. University of California Transportation
    Center, 1989
    EFFECT: The higher, the better.
    1
    0

    View Slide

  36. 36
    1 3 4
    THEORY TEST 2

    View Slide

  37. 37
    1 3 4
    THEORY TEST 2

    View Slide

  38. “Operationalize” Small blocks
    Block size is a proxy for an high number
    of peoples’ interactions
    For district :
    1
    | |
    *
    0∈023456(7)
    ()
    EFFECT: The lower, the better
    38
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 3 4
    THEORY TEST 2

    View Slide

  39. 39
    1 3 4
    THEORY TEST 2

    View Slide

  40. Aged buildings
    Aged buildings are supposed to be a
    proxy for new, small enterprises.
    For district :
    1
    |7
    |
    *
    8∈9!

    7
    : set of companies
    EFFECT: The higher, the worse
    40
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 3 4
    THEORY TEST 2

    View Slide

  41. Aged buildings
    For district the weighted standard
    deviation of buildings age.
    EFFECT: The higher, the better
    41
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 3 4
    THEORY TEST 2
    !"#
    = ∑!"#
    $ %!('!( ̅
    ')%
    +
    ('(#)
    '
    ∑!"#
    $ %!

    View Slide

  42. 42
    1 3 4
    THEORY TEST 2

    View Slide

  43. 43
    1 3 4
    THEORY TEST 2

    View Slide

  44. “Operationalize” Density
    For district :
    Employment density:
    |:;<23=>? <>3<2>!|
    @A>@!
    Population density:
    |B3@A>@!
    EFFECT: The higher, the better
    44
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 3 4
    THEORY TEST 2

    View Slide

  45. 45
    1 3 4
    THEORY TEST 2

    View Slide

  46. “Operationalize” Vacuums
    Distance to huge parks for district :
    1
    7
    *
    8∈F!
    ( , , )
    7
    : the set of the blocks
    : the set of parks
    EFFECT: The higher, the better
    46
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS
    1 3 4
    THEORY TEST 2

    View Slide

  47. 47
    1 3 4
    THEORY TEST 2

    View Slide

  48. Diversity Vitality

    View Slide

  49. Call Detail Records
    Data collected by mobile operators for billing
    reasons
    • Unique userID
    • Gender and age
    • Geographical location (Antenna)
    • Datetime
    49
    1 2 3 4
    THEORY TEST

    View Slide

  50. Call Detail Records
    50
    1 2 3 4
    THEORY TEST

    View Slide

  51. “Operationalize” Vitality
    • Mobile phone Internet activity as a proxy for
    urban vitality
    • We calculate the activity density in each
    district
    1
    ||
    %
    "∈$
    | %
    |
    : set of hours (180 days x 24h)
    • Six Italian cities with 100,000+ inhabitants
    (e.g. Rome, Milan…)
    • 6 months time span (in 2014)
    51
    0.2
    0.0
    0.2
    0.4
    0.6
    0.8
    1.0
    Land Use Mix
    0.8 1.2 1.8 2.7 4.1 6.1 9.3 14.0 21.1 31.9
    Activity density ⇥ 103
    ROME
    MILAN
    0.8 1.2 1.8 2.7 4.1 6.1 9.3 14.0 21.1 31.9
    Activity density 103
    1 2 3 4
    THEORY TEST
    MILAN

    View Slide

  52. 52
    2
    1 3 4
    URBAN DESCRIPTION
    Mobile data
    GIS data
    “Operationalize” the theory

    View Slide

  53. 53
    2
    1 3 4
    URBAN DESCRIPTION
    Mobile data
    GIS data
    “Operationalize” the theory

    View Slide

  54. 54
    2
    1 3 4
    URBAN DESCRIPTION
    Mobile data
    GIS data
    “Operationalize” the theory

    View Slide

  55. 55
    2
    1 3 4
    URBAN DESCRIPTION
    Mobile data
    GIS data
    “Operationalize” the theory

    View Slide

  56. 56
    2
    1 3 4
    URBAN DESCRIPTION
    Mobile data
    GIS data
    Does it work?
    Is it still valid?
    3
    “Operationalize” the theory

    View Slide

  57. Demo 1
    https://github.com/denadai2/test-the-
    theory-
    lesson/blob/master/step1_compute_fe
    atures.ipynb
    57

    View Slide

  58. Is the theory still valid?
    STEP 3

    View Slide

  59. −3 −2 −1 0 1 2 3
    −9
    −8
    −7
    −6
    −5
    −4
    −3
    −2
    .
    Is the theory still valid?
    59
    Intersections density (log + Z-score)
    Activity density
    (log)
    1 2 3 4
    THEORY TEST

    View Slide

  60. −3 −2 −1 0 1 2 3
    −9
    −8
    −7
    −6
    −5
    −4
    −3
    −2
    R2 : 0.63
    Is the theory still valid?
    60
    Intersections density (log + Z-score)
    Activity density
    (log)
    1 2 3 4
    THEORY TEST

    View Slide

  61. Demo 2
    https://github.com/denadai2/test-the-
    theory-
    lesson/blob/master/step2_exploration.
    ipynb
    61

    View Slide

  62. The log Linear Regression
    62
    = !
    !
    + "
    "
    + ⋯ + "#
    "#
    +
    1 2 3 4
    THEORY TEST

    View Slide

  63. The log Linear Regression
    63
    Activity density
    1 2 3 4
    THEORY TEST
    = !
    !
    + "
    "
    + ⋯ + "#
    "#
    +

    View Slide

  64. The log Linear Regression
    64
    Activity density
    Land Use Mix
    Employment density
    1 2 3 4
    THEORY TEST
    = !
    !
    + "
    "
    + ⋯ + "#
    "#
    +

    View Slide

  65. Demo 3
    http://setosa.io/ev/ordinary-least-
    squares-regression/ and
    https://github.com/denadai2/test-the-
    theory-
    lesson/blob/master/step3_Regression.i
    pynb
    65

    View Slide

  66. Jacobs’ theory holds and is still valid
    66
    1 2 3 4
    THEORY TEST
    Urban metric Beta coefficient
    Employment density 0.434***
    Intersections density 0.191***
    Housing types 0.1854***
    Closeness highways -0.102***
    3rd places x closeness highways 0.07**
    Closeness parks x closeness highways -0.07***
    − 0.77
    *** p-value < 0.001; ** p-value < 0.01;
    4-fold Cross-validation: 75% training – 25% testing, 1000 interactions

    View Slide

  67. Jacobs’ theory holds and is still valid
    67
    URBAN
    VITALITY
    1 2 3 4
    THEORY TEST
    URBAN
    METRICS
    Predict
    $: 0.77

    View Slide

  68. …But something is different
    68
    1 2 3 4
    THEORY TEST
    LAND USE
    SMALL
    BLOCKS
    AGED
    BUILDINGS
    DENSITY
    VACUUMS

    View Slide

  69. Why does it matter?
    STEP 4

    View Slide

  70. Web data and mobile phone records
    offer insights on how most urban
    dwellers experience entire cities
    1 2 3 4
    THEORY TEST

    View Slide

  71. Why does it matter?
    • Evaluate the districts vitality
    • Know in advance the best places for retails
    • Quantifying regulatory interventions
    • We created the recipe for city that works
    71
    1 2 3 4
    THEORY TEST

    View Slide

  72. Let’s test an urban theory
    72
    1
    The Jacobs’
    theory
    We created
    the metrics
    We tested
    the theory
    Framework
    for urban vitality
    2 3 4

    View Slide

  73. Test the un-tested
    And now?

    View Slide

  74. Broken windows theory
    • City mismanagement
    • Dirty places
    • Poor infrastructure
    Lead to misbehavior => Crime
    Q: Are people avoiding places where they
    feel unsafe?
    74
    Wilson, James Q., and George L. Kelling. "Broken windows."
    Critical issues in policing: Contemporary readings (1982): 395-
    407.

    View Slide

  75. 75
    Urban perception from Place Pulse
    Salesses, P., Schechtner, K., & Hidalgo, C. A. (2013). The collaborative image of the city: mapping the inequality of urban
    perception. PloS one

    View Slide

  76. 76

    1
    5
    Place Pulse
    • New York
    • Boston
    • Linz
    • Salzburg
    Place Pulse 2
    • Rome
    • Milan
    URBAN PERCEPTION
    Safety perception: MIT Place Pulse

    View Slide

  77. 77
    Security perception prediction
    * B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. “Learning Deep Features for Scene
    Recognition using Places Database.” NIPS, 2014.
    • Learning human security perception
    PERCEPTION
    SCORE
    [0-10]

    View Slide

  78. Demo 4
    https://github.com/denadai2/test-the-
    theory-
    lesson/blob/master/step_extra_vision.i
    pynb
    78

    View Slide

  79. 79
    Urban metric Standardized Beta coefficient
    Population density 0.155**
    Employees density 0.328**
    Deprivation -0.022
    Distance from the center -0.257**
    Security perception 0.105**
    adj − R$ 0.91
    ** p-value < 0.001; * p-value < 0.01;
    Security perception -> presence of people

    View Slide

  80. Take home
    Try to test your favorite theory!
    80

    View Slide

  81. Thanks
    @denadai2

    View Slide