11th Conreality Hackathon November 17-18, 2017

Overview 1. Practicalities 2. Concept & History 3. Schedule 4. Photography & Privacy 5. Catering & Dinner 6. Goals 7. Working Groups 8. Future 9. Questions & Answers

● Wi-Fi connectivity ○ Network name ITEA_LV_Conf, password on the wall ● Social media ○ https://www.facebook.com/conreality ○ https://www.instagram.com/conreality/ ○ https://twitter.com/ConrealityGame ○ #conreality hashtag on both ○ https://conreality.org is the project website ● Chat rooms ○ https://conreality.slack.com #hackathon ● T-shirts and stickers ● Logistics and physical security Practicalities

● Gather in one place for a weekend to work on proof-of-concepts and integrate individual work being done separately in different locations ● Our previous hackathons took place in Bratislava and Berlin (10 hackathons since late 2015) ● This is our first hackathon in Lviv, and the first fully public one Concept & History

Saturday 10:00 Coffee and snacks 10:30 Introduction 12:00 Presentation by Arto 13:00 Presentations by Dan and Alexander 14:00 Lunch 15:00 Hacking 20:00 End of day 20:30 Dinner (in the center of town) 23:00 Hacking and/or drinking Schedule for Day 1

Sunday 10:00 Coffee and snacks 10:30 Hacking 12:00 Presentations by working groups 14:00 Lunch 15:00 Kids' workshop & Hacking 19:00 Closing 20:00 End of day 20:30 Dinner (in the center of town) 23:00 Hacking and/or drinking Schedule for Day 2

Saturday 11:30-14:00 for presentations 21:00-22:00 at dinner Sunday 15:00-17:00 for kids' workshop & hacking Both days Occasional use of flying camera drone Photographs will be public and will be used in the promotion of the project on Facebook and elsewhere In case you don't wish to appear in photographs, let our photographer (Taras) know to leave you on the editing table Photography & Privacy

Catering & Dinner Saturday 14:00-15:00 Lunch at conference venue 20:30-23:00 Dinner in restaurant Mons Pius (in the center of town) Sunday 14:00-15:00 Lunch at conference venue 20:30-23:00 Dinner in restaurant Mon Chef (in the center of town) Both days 10:00-20:00 Coffee, tea, water, soft drinks, beer

● Learning & Teaching ○ Learn about virtual reality (VR), augmented reality (AR), computer vision (OpenCV), machine learning (TensorFlow, CUDA), electronics (Arduino, Raspberry Pi), robotics (drones), etc. ○ Teach a younger generation that magic is knowable ● HMD (Helmet-Mounted Device) R&D ○ Create a HUD overlay PoC on consumer VR headsets (HTC Vive, OSVR) ○ Get OSVR development environment set up with Linux in addition to Windows ● Android App ○ Begin developing a prototype of the Conreality Player app ● Design & Crafts ○ Finish the initial visual identity (logo in 3 variants, on light/dark backgrounds, in multiple sizes) ○ Begin 3D modeling of drone canopies with Vectary Goals for Hackathon

● HMD (Helmet-Mounted Device) ○ Dan, Samuel, (Arto) ● Android App ○ Arto, (Dan) ● Locationing Sensor Workshop ○ Alexander, (Arto) ● Design & Crafts ○ Mariya, Sonia Working Groups for Day 1

TBD Working Groups for Day 2

● Our next hackathon will be held in Lviv in January or February 2018 ○ Currently looking at the weekend of January 27-28 ○ …with backup options being the weekends of February 3-4 or February 10-11 ○ Please indicate your preference if you have one ● The main goal of the next hackathon will be to begin fielding our tech ○ We are arranging an indoors paintball game for the hackathon team Future

Questions?

Let’s hack!

No content

Introduction to Conreality Arto Bendiken

Summary 1. What is Conreality? 2. Reality is Boring 3. Cyberpunk Games 4. Live-Action Games 5. Augmented Reality 6. Tactical Software

What is Conreality? Consensus Reality (aka Conreality, conreality.org) is a live-action augmented-reality, tactical wargame platform currently in an early development phase.

Reality is Boring Let’s make it more interesting

Reality is Boring “Anyone who sees a hurricane coming should warn others. I see a hurricane coming. “Over the next generation or two, ever larger numbers of people, hundreds of millions, will become immersed in virtual worlds and online games. While we are playing, things we used to do on the outside, in “reality,” won’t be happening anymore, or won’t be happening in the same way. You can’t pull millions of person-hours out of a society without creating an atmospheric-level event. “If it happens in a generation, I think [this] century will see a social cataclysm larger than that caused by cars, radios, and TV, combined. . . . The exodus of these people from the real world, from our normal daily life, will create a [cataclysmic] change in social climate.” — Edward Castronova, Exodus to the Virtual World (2007)

Reality is Broken “Gamers want to know: Where, in the real world, is that gamer sense of being fully alive, focused, and engaged in every moment? Where is the gamer feeling of power, heroic purpose, and community? Where are the bursts of exhilarating and creative game accomplishment? Where is the heart-expanding thrill of success and team victory? “While gamers may experience these pleasures occasionally in their real lives, they experience them almost constantly when they’re playing their favorite games. The real world just doesn’t offer up as easily the carefully designed pleasures, the thrilling challenges, and the powerful social bonding afforded by virtual environments. . . . Reality, compared to games, is broken.” — Jane McGonigal, Reality is Broken: Why Games Make Us Better and How They Can Change the World (2011)

2016 and 2017: VR and AR, At Long Last Virtual Reality (VR) Johnny Mnemonic (1995) Augmented Reality (AR) Iron Man (2008)

Say No to VR

VR as a Solution to the Fermi Paradox? “I suggest a different, even darker solution to the Paradox. Basically, I think the aliens don’t blow themselves up; they just get addicted to computer games. “They forget to send radio signals or colonize space because they’re too busy with runaway consumerism and virtual-reality narcissism. They don’t need Sentinels to enslave them in a Matrix; they do it to themselves, just as we are doing today. “Once they turn inwards to chase their shiny pennies of pleasure, they lose the cosmic plot. They become like a self-stimulating rat, pressing a bar [which] feels…ever so good.” — Geoffrey Miller, Why We Haven’t Met Any Aliens (2006) http://seedmagazine.com/content/article/why_we_havent_met_any_aliens/

Cyberpunk Games Dystopian virtual reality at its best

Cyberpunk Games

Deus Ex: Mankind Divided (Prague 2029)

Deus Ex: Mankind Divided

Live-Action Games Bringing dystopia back home

Live-Action Games Today ● Historical reenactment ● LARPs ● Paintball (Пейнтбол) ● Airsoft (Страйкбол) ● Laser tag (Лазерний тег) ● Office games (e.g., nerf guns) ● Geocaching ● Rudimentary AR ○ Ingress ○ Pokemon Go

Airsoft pic w/o drone

Airsoft pic w/o drone

Paintball Battle Drone demo video

Augmented Reality It’s all about the man-machine I/O interface

Hyper-Reality AR demo video

Meta AR demo at TED 2016 https://www.ted.com/talks/meron_gribetz_a_glimpse_of_the_future_through_an_a ugmented_reality_headset

Physical reality You can't win. You can't break even. You can't even get out of the game. Levels of Reality Social reality Social norms and false reifications; the world that has been pulled over your eyes. Consensus reality Names and labels, encyclopedic general knowledge and reference works. Personal reality Reminders, relationship indicators, reputation scores, personal preferences.

Let’s hack!

No content

Overview of Conreality Arto Bendiken

1. Target Audiences 2. Technology Overview 3. Hardware Concepts 4. Software Clients 5. Software Kits 6. Software Components 7. Software Architecture 8. Planned Milestones 9. Project Team 10. Contribution Opportunities Summary

Target Audiences ● Gamers ● Coders ● Makers ● Educators ● Entrepreneurs

Target Audience: Gamers ● Create and script your own game scenarios and robot behaviors using our Game Development Kit (GDK) ○ Lua

Target Audience: Coders ● Integrate with game servers using our Software Development Kits (SDKs) for various popular programming languages ○ .NET, C++, Elixir, Go, Kotlin, Python, and Ruby

Target Audience: Makers ● Develop drivers for your favorite gear and peripherals using our Driver Development Kit (DDK) ○ C++, Python

Target Audience: Educators ● Teach students, young or older, about programming, electronics, robotics, and augmented reality

Target Audience: Entrepreneurs ● Make a million bucks leveraging our platform to create bestseller games for the real world ● Once network effects get going, they’ll drive adoption and concentrate revenue on the platform in the same way as for top-tier game engines such as Unreal and Unity ● No-strings-attached licensing (everything we do is 100% public domain)

Technology Overview A*, APM, BBB/BBG(W), BEC, BTLE, Buildroot, C/C++, CoAP, DMA, DW1000, ESC, Elixir, Erlang, FPV, GAP, GATT, GPIO, Gazebo, I2C, IRC, LIDAR, Lua, MATLAB, MQTT, MAVLink, Nerves, NFC, OPTO, OpenCV, PCB, POSIX, PPM, Prolog, PWM, PX4, PXFmini, QEMU, RDF, RK4, ROS, RPi, RTLS, SBEC, SLAM, SPI, TCP, TDOA, UART, UBEC, UDP, USB, UVC, UWB, V4L2, seL4, …

Technology Overview ● DIY electronics (Arduino, Raspberry Pi, BeagleBone) ● Robotics (drones, ground vehicles, turrets, ROS) ● Virtual reality (VR) and augmented reality (AR) ● Indoor/outdoor locationing (UWB TDoA, GPS L5/L1C) ● Computer vision (OpenCV) ● Simultaneous localization and mapping (SLAM) ● Machine learning (TensorFlow, CUDA) ● Physical simulation (linear algebra, Gazebo)

● Cube MkI ○ Master game server for hosting local games ○ Enough CPU and GPU power for machine learning and computer vision Hardware Concepts

● HMD MkI ○ Proof of concept (PoC) of poor-man’s AR headset ○ Combines commodity VR headset and HD camera Hardware Concepts

● Laser Turret MkI ● Scout Car MkI Hardware Concepts ● Recon Drone MkI ● Assault Drone MkI

Hardware Concepts

Hardware Concepts

Software Clients ● Console App (concon) ○ Command and control (C2) for administering and monitoring game scenarios ○ Built with C++ and Qt/QML, to run on tablets, laptops, and desktops ● Player App (conapp) ○ The pocket app for gamers to participate in a game ○ Open mic for team comms and real-time locations of teammates ○ Built with Kotlin, to run on Android smartphones with a Bluetooth headset ● Helmet App (HUD aka HMD) ○ The headset app for gamers, providing a tactical overlay ○ Built with C++, to run on eventual commodity AR headsets

Software Kits ● Conreality Game Development Kit (GDK) ○ For gamers and game developers, to script scenarios and robot behavior ○ Written in the Lua programming language, at a high level of abstraction ○ Currently at an early development stage ● Conreality Software Development Kits (SDKs) ○ For coders, to integrate with the master game server ○ Available for the .NET, C++, Elixir, Go, Kotlin, Python, and Ruby programming languages ○ Currently at varying levels of development and completeness ● Conreality Driver Development Kit (DDK) ○ For hackers and makers ○ Available for the C++ and Python programming languages ○ Currently at an early development stage

● Master Daemon (conmasterd) ○ The game server for a team, coordinates a game in progress ○ Fully isolated from other teams’ game servers (our AI can’t cheat) ○ Provides centralized intelligence services for slaves and clients ○ Built with Elixir, C++, Erlang/OTP, and PostgreSQL ● Slave Daemon (conslaved) ○ The brains for every robot (drone, vehicle, turret) and peripheral (e.g., smart camera) ○ Implements local intelligence (computer vision, machine learning, path planning, etc) to the extent the device’s hardware resources can support that, and relies on the master beyond that ○ Built with Elixir, C++, Erlang/OTP, and Nerves Software Components

● In the initial phase, a simple client/server network architecture ○ The master server coordinates games and provides intelligence resources (computer vision, machine learning, path planning) where needed ○ Robots and peripherals run tightly slaved to the master with limited local intelligence ○ Client software talks to the master for APIs, voice comms, and commands ● Future directions include a gradual move towards peer-to-peer (P2P) architecture and mesh networking Software Architecture

● Q4 2017 ○ Initial usable version of the Player app for Android ○ Initial R&D for the HMD hardware and software ● Q1 2018 ○ Field ready for supporting/upgrading airsoft & paintball games ● Q2 2018 ○ Initial field-test deployments at airsoft & paintball areas ○ Blueprints and alpha/beta software for first-generation (MkI) products: ■ Scout Car MkI ■ Laser Turret MkI ■ HMD MkI Planned Milestones

Project Team in Lviv, Bratislava, and Berlin ● @ab0032 (Alexander) ● @bendiken (Arto) ● @dankomorny (Dan) ● @gilcherry (Gilbert) ● @joecoin (Joerg) ● @mikegogulski (Mike) ● @mrijnyk (Mariya) ● @samuelsarle (Samuel)

Contribution Opportunities ● A chance to participate in an open-source project that is... ○ tangible (non-abstract, unlike so much software), ○ ambitious, ○ highly challenging, ○ and, most of all, plain old fun ● Designers, coders, hackers, makers, gamers, educators, entrepreneurs, and sponsors are all most welcome

Let’s hack!