Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data on the Web
Search
Will Farrington
June 08, 2011
Technology
3
180
Data on the Web
It's an intro to data on the web for some folks new to web development.
Will Farrington
June 08, 2011
Tweet
Share
More Decks by Will Farrington
See All by Will Farrington
test-queue makes your tests run fast
wfarr
0
390
Incident Response Done Right: From First Page to Postmortem
wfarr
0
420
Boxen: PuppetConf 2013
wfarr
6
820
Puppet at GitHub: PuppetConf 2013
wfarr
21
2k
Puppet at GitHub (PuppetCamp Raleigh 2013)
wfarr
1
400
Boxen: PuppetCamp SF 2013
wfarr
5
600
Boxen: MWRC
wfarr
5
190
Boxen: PuppetCamp ATL
wfarr
0
250
BOXEN
wfarr
43
5.3k
Other Decks in Technology
See All in Technology
KubeConにproposalを送りたい人へのアドバイス
sat
PRO
3
250
Além do else! Categorizando Pokemóns com Pattern Matching no JavaScript
wmsbill
0
620
FrontDoorとWebAppsを組み合わせた際のリダイレクト処理の注意点
kenichirokimura
1
520
Java EE/Jakarta EEの現状と将来―クラウドネイティブ時代にJava EEは対応できるのか?―
takakiyo
1
160
ExaDB-D dbaascli で出来ること
oracle4engineer
PRO
0
2.1k
EMとして2023年度に頑張ったこと / What we did well in FY2023 as a EM
pauli
1
170
DevOpsメトリクスとアウトカムの接続にトライ!開発プロセスを通して計測できるメトリクスの活用方法
ham0215
2
240
私が trocco を推す理由
__allllllllez__
1
220
ChatworkのSRE部って実は 半分くらいPlatform Engineering部かもしれない
saramune
0
160
サーバー間 GraphQL と webmock-graphql の話 / server-to-server graphql and webmock-graphql
qsona
2
190
ServiceNow Knowledge 24の歩き方 EYストラテジー・アンド・コンサルティング
manarobot
0
200
どうするコスト最適化のトレードオフ
tetsuyaooooo
1
520
Featured
See All Featured
Into the Great Unknown - MozCon
thekraken
10
990
Practical Orchestrator
shlominoach
182
9.7k
Why Our Code Smells
bkeepers
PRO
331
56k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
2
3.4k
For a Future-Friendly Web
brad_frost
172
9k
Building Better People: How to give real-time feedback that sticks.
wjessup
355
18k
Designing on Purpose - Digital PM Summit 2013
jponch
110
6.5k
Web development in the modern age
philhawksworth
202
10k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
Docker and Python
trallard
34
2.7k
Code Review Best Practice
trishagee
55
15k
Faster Mobile Websites
deanohume
299
30k
Transcript
Data on the Web Will Farrington
File I/O
File I/O • Minimal reusability • No "correct" format •
Hard to maintain • Prone to problems caused by encoding changes
Standardize
CSV
Comma Separated Values
CSV • Used for tabular data • Small footprint •
Widely recognized and supported format • Many different flavors • Support in database systems and spreadsheets
Example CSV Id,Name,Desc,Points,Due 1,Homework 1,Nothing special,15,6/7/2011 15,"Project, número uno",,100,6/21/2001
XML
Extensible Markup Language
XML • Open, standard specification • Unicode-friendly • Came to
prominence with Java and .NET • Widely used on the web • Good at representing tree-like data
Example XML <?xml version="1.0" encoding="UTF-8"?> <statuses type="array"> <status> <created_at>Tue Jun
07 21:30:50 +0000 2011</created_at> <id>78212343649140736</id> <text>@skalnik Looks good.</text> <source><a href="http://itunes.apple.com/us/app/twitter/id409789998?mt=12" rel="nofollow">Twitter for Mac</a></source> <truncated>false</truncated> <favorited>false</favorited> <in_reply_to_status_id>78211453777231872</in_reply_to_status_id> <in_reply_to_user_id>15878923</in_reply_to_user_id> <in_reply_to_screen_name>skalnik</in_reply_to_screen_name> <retweet_count>0</retweet_count> <retweeted>false</retweeted> <user> <id>10403812</id> </user> <geo/> <coordinates/> <place/> <contributors/> </status> </statuses>
Criticisms of XML • Very verbose • Parsers can be
extremely complicated • Does not map well to some type systems • Does not represent highly structured data well
JSON
JavaScript Object Notation
JSON • Based on a subset of JavaScript circa 2003
• Lightweight • Simple to parse • Designed to be human-readable • Well-suited to structured data as well as trees
Example JSON [{ "coordinates":null, "created_at":"Tue Jun 07 21:30:50 +0000 2011",
"truncated":false, "favorited":false, "contributors":null, "text":"@skalnik Looks good.", "id":78212343649140736, "retweet_count":0, "geo":null, "retweeted":false, "in_reply_to_user_id":15878923, "source":"<a href=\"http://itunes.apple.com/us/app/twitter/id409789998?mt=12\" rel= \"nofollow\">Twitter for Mac</a>", "place":null, "in_reply_to_screen_name":"skalnik", "user":{"id":10403812}, "in_reply_to_status_id":78211453777231872 }]
More on JSON • eval() (is bad) • JSON.parse() •
Built-in browser support • Popular for AJAX: both single-domain and cross-domain
JSONP • JSON with Padding • Used for cross-domain requests
• Alternative to Cross-Origin Resource Sharing • Only supports GET
BSON • Binary JSON • Superset of JSON • Used
by MongoDB for storage of binary data
YAML
YAML Ain't Markup Language
YAML • Not often used over the network • Popular
for configuration files • Human-readable • Data-oriented • No execution means no injection
Example YAML --- - coordinates: created_at: Tue Jun 07 21:30:50
+0000 2011 truncated: false favorited: false contributors: text: "@skalnik Looks good." id: 78212343649140736 retweet_count: 0 geo: retweeted: false in_reply_to_user_id: 15878923 source: <a href="http://itunes.apple.com/us/app/twitter/id409789998?mt=12" rel="nofollow">Twitter for Mac</a> place: in_reply_to_screen_name: skalnik user: id: 10403812 in_reply_to_status_id: 78211453777231872
What to do with all these formats?
APIs
Application Programming Interfaces
APIs • Websites tell you what formats they support •
Websites document their URL structure • Developers use these APIs to integrate products • You can even consume your own APIs
But...
Not everyone offers APIs
What do?
Screen-scraping
Screen-scraping • Requests the full HTML for a page •
Parses out the content you want • Slow • Website layout may change and break yours
Demo!
Questions?
Will Farrington
[email protected]
http://speakerdeck.com/u/wfarr