OptiCon-2017-"Pushing the boundaries of experimentation with Optimizely APIs"

Talk about using Optimizely JS + REST APIs to support our experimentation programs, from the perspective of:
- Dev Manager
- Experimenters
- Marketers

Talk link: https://www.optimizely.com/opticon/2017/sessions/pushing-the-boundaries-of-experimentation-with-optimizely-s-apis-and-extensions/

Transcript

1. SHREYAS | SR. GROWTH HACKER | @SEEKSHREYAS Using the Optimizely APIs The Atlassian story
2. Introduction
4. Agenda CHALLENGES SOLUTIONS Debugging (with JS API) Monitoring (with REST API) Automation (with JS + REST API)
5. Challenges .. of running a successful experimentation program
6. A SIMPLE EXPERIMENT ..
7. Multiple experiments Simultaneous multiple experiments,
8. Parallel experiments Simultaneous multiple experiments, JIRA Confluence Aug Sep
9. SCALING
 leads to challenges
10. Communication & Collaboration
11. Dev Manager Experimenters Marketers “Is the bug caused by an experiment?” “when will I have my results?” “biz impact of experimentation” “how can I not be repetitive”
12. Agenda SOLUTIONS CHALLENGES Debugging (with JS API) Monitoring (with REST API) Automation (with JS + REST API)
15. “Convention over Configuration” “Signup form experiment” “(WAT-XXX) Signup form experiment” +
16. “UPTIMIZELY” - a Chrome extension Developer Credit: Roger de la Fuente
19. Using the JS API For debugging & goals
20. > optimizely.activeExperiments Debugging > optimizely.data.experiments['3721640680'].variation_ids.length > optimizely.data.variations['3702600931'].code // Experiment Jira Id: WAT-244 // Optimizely Experiment Id: 3518000728 // Experiment Name: "(WAT-244) Summit Banner on WAC"; // Experiment URL: ["https://www.atlassian.com/*"] // ------------ // VARIATION:: Banner // ------------ // Author: Shreyas // Email: shreyas@atlassian.com /* _optimizely_evaluate=force */ console.log("DEV LOG:: with white background"); Dev Manager
21. > decodeURIComponent(GROW.getCookie('optimizelyBuckets')); Funnel Goals { “8784903299":"8783514119", “8784919100":"8782127665" } Experimenters
22. Using the REST APIs Experiment Analyses & Radar
23. REPORTING AUTOMATION Reduce repeatable tasks Alfred Experimenters Icon Credit: Matt Regan
24. { "experiment_metrics": [{ "db_nickname": "socrates_presto", "query_name": "dai_day0_raw", "query_location": "query/dai_day0_raw.sql", "query_params": { "{{experiment_id}}": "experiment_id", "{{start_timestamp}}": "start_timestamp", "{{end_timestamp}}": "end_timestamp" }, "metric_count_type": "raw", "metric_test": ["bayesian", "chi-squared"] }], "statistical_tests": { "t-test": { "confidence_interval": "0.95" }, "chi-squared": { "confidence_interval": "0.95" }, "bayesian": { "alpha": "0.05", "sample": "500", "cores": "1" } } } Alfred
29. MONITORING & RETRO ANALYSIS Experiment Radar
34. Radar MONITORING & RETRO ANALYSIS Experiment Radar
35. Dev Manager Experimenters Marketers JS snippets Radar Uptimizely Alfred
36. Dev Manager Experimenters Marketers
37. Thank you