© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. Senior Cloud Operations Engineer 812-320-3840 elijah@joyent.com Text Text Fraud in the Joyent Public Cloud Elijah Wright 1 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . About me •SA / operations engineer / cloud ops guy •I love me some chef and puppet. •I prefer Ubuntu on my desk, illumos or Debian on servers •Also, monitoring •Performance is cool. Let’s build race cars! •Webops, devops, you know the drill. •Current capacity czar (self-mockery here) 2 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . What Joyent is.. •Small to medium cloud operator, in growth mode. •Software shop - SmartDatacenter, keepers of Dtrace, KVM on illumos, SmartOS, nodejs, stuff like that. •Very interested in performance - introsp. used to eval OS-level change to benefit our customers. SmartOS. •Heavy in engineering - ex-Sun folk from Fishworks and Dtrace and ZFS teams, CEO from Force10, a bunch of super classy guys who’ve been around a few X. •Max and Brendan are our secret weapons. :) Training. •Cloud provider with a different take than AMZN 3 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Why we hate fraud and fraudsters •Don’t hate the player hate the game? •Well.... hehe. •As we’ve grown this has become a big PITA •Not only do they do things like run DDoS attacks •They burn up capacity that we would like to sell. •The cloud costs real money, you know! •[Direct impact to the bottom line - only a few bux per fraudulent VM, but requires additional staff, effort, online capacity in reserve to absorb the fraud hit.] •You end up growing a BizOps team eventually.... 4 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Fraud along our growth trajectory •Free Facebook •A few zones here and there •crappy in-house customer tracking system •Telephone auth workarounds •commercial fraud detection tooling •PCI and two-factor auth • 5 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . The thigh bone’s connected to the leg bone... •Customer portal •Talks to Customer API, two-factor, and dialback provider (e.g. Tropo) •Then talks to the billing intermediary (think Aria, Zuora) •Who does validation via several other services (e.g. Moneris) •Who finally passes data to the credit card billing processor (e.g., Authorize.net) • 6 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Who is this customer, anyway? •Remember signing up for BBSes in the early 90s? •With dialback auth? •With a made-up name and address? •That game still works for a LOT of businesses. •It’s really pretty hard to fight back against without sig. investment in tools. 7 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Suspicious Shit •Vietnamese cable modem network •That particular block in Houston •Anybody who buys 64G Windows instances •Anybody who buys a BUNCH of CentOS 512m VMs •Certain blocks of phone nums @Toronto, Haifa, Brazil •Phone numbers that don’t match the CC billing address •Things that just make you curious •When corporate location (from WHOIS) and CC billing and phone # don’t match up at all 8 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Don’t be a soft target. •When fraudsters have web pages teaching each other how to exploit your weak business process.... •You are totally fucked. •This happened to us - “how to run a seedbox on Joyent” - a few years ago when we were much smaller. •Solution - improve business process. A lot. 9 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Where it can all go horribly wrong •Disposable cell phones - not just for the mob or murder-for-hire •WalMart - go buy a reloadable CC. Try it out (please, not on us). Or try Walgreens. •Find one that doesn’t want you to give it an SSN to sign up. Or, at least, not a real one / weak validation. •PreAuth (oh god) •In the middle of the night (Billy Joel ref) nobody is manually watching the provisioner_messages table for heavily suspect sh*t 10 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . It does get better. •With sane processes in place, people’s attempts to rob you blind are survivable. •Trust, but verify. •Faster response to funky events helps. •Listen to your NOC guys - they see odd things. • 11 Wednesday, April 3, 13

© 2012 Joyent, Inc. Proprietary & Confidential Information of Joyent, Inc. . Ask Me Anything - QVESTIONS! 12 Wednesday, April 3, 13