A Brand-New Application of Elixir and Pelemay: Real-time, High Resolution Satellite Image Processing

Do you know there is a plan to launch artificial satellites by Japanese companies? Don’t be surprised, but it will have the potential to enable us to observe everywhere on the Earth within only 10 minutes! However, it has a serious issue: it requires too much volume of image processing, which may consume computing power more than a super computer! But don’t worry, we propose to apply amazing parallelism of Elixir and Pelemay to it, with reasonable resources! How do we realize it? Listen to this talk!

Talk Description:
This talk proposes to apply Elixir and Pelemay to real-time, high resolution satellite image processing, which has the wide potential to realize useful applications, such as disaster preparation and recovery, harvest predictions, economic predictions, natural environment protection and observation, physical distribution management, searching for mineral resources and so on. Elixir and Pelemay will play an important role in analysis of huge volumed image processing. This talk shows a concrete and reasonable application of such image processing, and approaches how we will realize it. We are confident that it will grow to be third major application of Elixir after web and IoT applications.

Transcript

1. A Brand-New Application of Elixir and Pelemay: Real-time, High Resolution Satellite Image Processing Susumu Yamazaki Univ. of Kitakyushu 1 © 2020 Susumu Yamazaki © 2012 Plataformatec © 2018 Susumu Yamazaki & Yuki Hisae
2. Susumu Yamazaki (ZACKY) • Twitter / GitHub / Gmail: ZACKY1972 • Associate Professor • The Leader of the Nudge Social Implementation Research Center • at Univ. of Kitakyushu • Co-author of Pelemay • Co-organizer of ElixirConf JP 2 © 2020 Susumu Yamazaki
3. © 2020 Susumu Yamazaki Japan USA Far, far away… I’m here!
4. © 2020 Susumu Yamazaki ʆ Honshu Kyushu Japan I’m here! Shikoku Hokkaido
5. © 2020 Susumu Yamazaki Honshu Kyushu I’m here! Kitakyushu
6. Prologue 6 © 2020 Susumu Yamazaki
7. A long time ago in a galaxy far, far away… 7 © 2020 Susumu Yamazaki
8. A long time ago in a galaxy far, far away… 8 © 2020 Susumu Yamazaki I’m sorry but BGM is turned off due to copyright.
9. Who is Mr. Takayama? 9 © 2020 Susumu Yamazaki
10. Hisanobu Takayama • He is the CEO of minsora, Inc., a company that has the philosophy, “Change structures of the space industries from the region. Make the outer space more fun and interesting!” • He has organized many projects on outer space, including ALMA telescope (see the upper figure), which is a radio telescope placed in Atacama, HTV (see the lower figure), which supplies resources to the space station, and promoting foundation of business on utilization of outer space. • Currently, he is conducting to create the Asia's first space port and city, and then a new market and ecosystem centered on the port, in Oita, Kyushu, Japan. • He has proposed joint research on utilizing SAR satellites to me. 10 © 2020 Susumu Yamazaki ©JAXA/NASA © NAO
11. What is SAR satellites? 11 © 2020 Susumu Yamazaki
12. What is SAR Satellites? • SAR: Synthetic Aperture Radar • They observe the Earth’s surface by microwave (see the right figure), not by radiant rays. • Because SAR is an active sensor unlike an optical passive sensor, they can observe the Earth’s surface even at nighttime when the sun doesn’t reach. • Microwave transmitted by them also passes through even cloud unlike radiant rays. 12 © 2020 Susumu Yamazaki © CRISP, 2001
13. How much observation power do SAR satellites have? 13 © 2020 Susumu Yamazaki
14. How much observation power do SAR satellites have? • SAR satellites can observe always the Earth’s surface even at nighttime and/or in bad weather due to above-mentioned their own principle. • Synspective, one of Japanese enterprise, will deploy 25 SAR satellites, which can identify a 1-3m (3-10 feet) square object on the Earth’s surface. • GeoEye-1 and WorldView-2, SAR satellites of USA, can identify a 50cm (1.6 feet) square object on the Earth’s surface. • iQPS, another of Japanese enterprise, will deploy 36 SAR satellites that can take photographs of almost any point in the world within 10 minutes, and conduct fixed-point observations at particular areas once every 10 minutes. 14 © 2020 Susumu Yamazaki
15. Amazing! What apps will be expected? 15 © 2020 Susumu Yamazaki
16. What apps will be expected? • Disaster preparation and recovery • Harvest predictions • Economic predictions • Natural environment protection and observation • Physical distribution management • Searching for mineral resources • And so on. 16 © 2020 Susumu Yamazaki
17. More detail! 17 © 2020 Susumu Yamazaki
18. What apps will be expected? • Disaster preparation and recovery • Harvest predictions • Economic predictions • Natural environment protection and observation • Physical distribution management • Searching for mineral resources • And so on. 18 © 2020 Susumu Yamazaki
19. App 1: Physical Distribution Management for Tourism 19 © 2020 Susumu Yamazaki
20. © 2020 Susumu Yamazaki ʆ Kanmon area Honshu Kyushu ↓ To Oita Prefecture (Space Port) I’m here!
21. Many Tourist Attractions in the Kanmon area 21 © 2020 Susumu Yamazaki
22. History 22 © 2020 Susumu Yamazaki
23. © 2020 Susumu Yamazaki The end battle of the Genpei War at Dan-no-ura
24. © 2020 Susumu Yamazaki The duel at Ganryujima
25. © 2020 Susumu Yamazaki © nimame Kokura-jo (Kokura Catsle), Kitakyushu, Fukuoka, Japan
26. © 2020 Susumu Yamazaki © nimame Nagasaki Kaido, a road across Kyushu from Kokura to Nagasaki
27. © 2020 Susumu Yamazaki World Heritage Site of Meiji Restoration
28. © 2020 Susumu Yamazaki Unknown, Courtesy of Meiji Seihanjo, 1922 Albert Einstein visited (1922)
29. Beauty 29 © 2020 Susumu Yamazaki
30. © 2020 Susumu Yamazaki Tsunoshima Bridge in Shimonoseki, Yamaguchi, Japan
31. © 2020 Susumu Yamazaki Hotel Nishinagato Resort near Tsunoshima Bridge
32. © 2020 Susumu Yamazaki Hotel Nishinagato Resort near Tsunoshima Bridge
33. © 2020 Susumu Yamazaki Hotel Nishinagato Resort near Tsunoshima Bridge
34. © 2020 Susumu Yamazaki Beautiful Blossoms
35. © 2020 Susumu Yamazaki Beautiful Blossoms
36. Culture 36 © 2020 Susumu Yamazaki
37. © 2020 Susumu Yamazaki Exciting Festivals
38. © 2020 Susumu Yamazaki Exciting Festivals
39. Venue of ElixirConf JP 39 © 2020 Susumu Yamazaki
40. © 2020 Susumu Yamazaki ElixirConf JP is held here Please attend and enjoy it if it will resume! I’m here!
41. But, after the outbreak of Coronavirus,… 41 © 2020 Susumu Yamazaki
42. After the outbreak of Coronavirus… • Tourism has been damaged seriously, because of preventing tourists from moving between areas as well as between countries, and from dinning at restaurant during the tour without any countermeasures against contagion. • It will be important to make tourism profitable by making it efficient, without reducing its attraction and safety. • I hypothesis that transportation during a tour is changed from public one to private one by vehicles. • Navigation for tourists smoothly is one of important measures. • But, gathering personal information by smartphones is not welcome. 42 © 2020 Susumu Yamazaki
43. Smooth navigation by satellites • SAR satellites has a potential to identify all of vehicles and obstacles on roads and parking lots observable from the outer space, once every 10 minutes, even nighttime and/or in bad weather. • So, it can be realized to implement a traffic control system that can identify congestion on every roads and parking lots using image recognition technology to images from satellites. • Of course, it can be realized without any personal information and additional sensors on the roads and parking lots! • I’d like to be going to make this system integrated with other navigation systems, such as Google Maps, in future. 43 © 2020 Susumu Yamazaki
44. App 2: Disaster Preparation and Recovery 44 © 2020 Susumu Yamazaki
45. Sediment disasters are more likely to occur in Japan See the right figure: • The x-axis of it is year. • The y-axis of it shows the total flooded area, the total amount of damage caused by flood, and the ratio. • The blue line shows the flooded area of Japan (square kilometer). • The green line shows the total amount of damage caused by flood of Japan (billion Yen, normalized with the value in 2000). • The gray lines shows the ratio of the total amount of damage to the total flooded area multiplied by 1,000 and the trend line of it, which represents seriousness of the flood damage. 45 © 2020 Susumu Yamazaki The trend line of seriousness of the flood damage has a positive slope!
46. How serious is a menace of flood damage? 46 © 2020 Susumu Yamazaki
47. A Menace of Flood Damage 47 © 2020 Susumu Yamazaki This is the street view of Toho small town, Fukuoka, Japan, when the flood damage of a very heavy outbreak of rain that poured in west Japan in 2017, which forced us to pay 536,000,000,000 Yen (about 5 billion dollars). You can see this at https://goo.gl/maps/eM6cpL4LpYBMyFig7
48. Why is flood damage so serious in Japan? 48 © 2020 Susumu Yamazaki
49. Why is flood damage so serious in Japan? Rivers in Japan are much more rapid than those in the US and Europe. See the right figure: • The x-axis of it is the distance from the estuary (kilometer). • The y-axis of it is the elevation difference from the estuary (meter). • This ellipse surrounds Japanese rivers. 49 © 2020 Susumu Yamazaki Japanese Rivers are like waterfalls! This is because the total area of Japanese mountains and hills is 70% of the whole country.
50. Why is flood damage so serious in Japan? In addition, heavy rainfall becomes more likely in recent years. See the right figure: • The x-axis of it is year. • The y-axis of it is the frequency of rainfall over 50 mm per hour in Japan. • The blue line shows the frequency for each year. • The red line shows the trend line of it. 50 © 2020 Susumu Yamazaki The frequency of heavy rainfall has been increasing! So, flooding has become more likely in Japan in recent years, though flood control has been developed.
51. What can satellites support for solving to flood issues? 51 © 2020 Susumu Yamazaki
52. What can satellites support for solving to flood issues? We can find out where there might have been a landslide by following the steps: 1. Take pictures of places with potential heavy rainfall according to the weather forecast, before the heavy rainfall. 2. Take pictures of the places again, during and after the heavy rainfall. 3. Find image differences of the photos before and after the heavy rainfall. Very easy! All we have to do is to do it in time. 52 © 2020 Susumu Yamazaki
53. What is outcome that is expected? 53 © 2020 Susumu Yamazaki
54. Potential outcome Such a system to detect sediment-related disasters every time even in nighttime and/or in bad weather • Can notify the damage caused by flood quickly to the disaster countermeasures office. • May find houses buried under sand and mud quickly to rescue people from it. • Enables quick and accurate recovery planning by the disaster countermeasures office. We can obtains many benefits by it, as a result. 54 © 2020 Susumu Yamazaki
55. Integration 55 © 2020 Susumu Yamazaki
56. Integration Usually: Physical Distribution Management for Tourism In Emergency: Disaster Preparation and Recovery 56 © 2020 Susumu Yamazaki Usually In Emergency
57. A Brand-New Application: Real-time, High Resolution Satellite Image Processing 57 © 2020 Susumu Yamazaki
58. A Brand-New Application: Real-time, High Resolution Satellite Image Processing 58 © 2020 Susumu Yamazaki I’ll build an ecosystem using satellites that enables the proposed apps as well as other useful apps, to be developed!
59. BTW, how will they be related to Elixir? 59 © 2020 Susumu Yamazaki © 2012 Plataformatec
60. We need more POWER! 60 © 2020 Susumu Yamazaki
61. We need more POWER! • Suppose to utilize full potential of SAR satellites. • Surface area of the Earth is 510,100,000 (five hundred ten million and one hundred thousand) square kilometer. • If the resolution of SAR satellites is supposed to be 50 centimeter, the number of the pixels of the image of the entire Earth will be 2,040,400,000,000,000 (two quadrillion, forty trillion and four hundred billion). • This equals to approximately 246,000,000 (two hundred forty six million) 4K images. • The memory size of raw data of it will be several PB, which equals to several thousands TB! • Such huge images may come every 10 minutes from satellites! • Too huge to be transmitted, processed and memorized, even if using supercomputers… • So, current available computational power in the entire world is not enough to utilize full potential of SAR satellites. 61 © 2020 Susumu Yamazaki
62. Limit the potential of satellites • Within limited local area when observing usually, for example, the Kanmon area; • Within some areas as large as the usual observing area, when observing in emergency; • In order to realize the system with limited available computer resources… Still it requires quite large computational power. 62 © 2020 Susumu Yamazaki ʆ Kammon area about 1,200 square kilometer
63. A NEW HOPE: Elixir and Pelemay 63 © 2020 Susumu Yamazaki © 2012 Plataformatec © 2018 Susumu Yamazaki & Yuki Hisae
64. What’s Pelemay? 64 © 2020 Susumu Yamazaki © 2018 Susumu Yamazaki & Yuki Hisae
65. What’s Pelemay? •Pelemay is a native compiler for Elixir, which generates SIMD instructions. It has a plan to generate for GPU code. https://github.com/zeam-vm/pelemay •Current Pelemay’s version is 0.0.13. •Supported Platforms: •x86_64 and ARM / macOS, Linux and Nerves •Elixir 1.9 to 1.10 / OTP 20 to 22 •See the upper right figure, which shows sample source code using Pelemay. •See the lower right figure, which shows an additional description to mix.exs. 65 © 2020 Susumu Yamazaki © 2018 Susumu Yamazaki & Yuki Hisae
66. How much effective is Pelemay? 66 © 2020 Susumu Yamazaki © 2018 Susumu Yamazaki & Yuki Hisae
67. How much effective is Pelemay? •Pelemay makes Elixir much faster! •See the right figure: •This shows speed up ratios of the benchmarks by Pelemay to Enum, which is measured by Benchfella. •The speed up ratios of integer calculation, float calculation and string replacement by Pelemay are 2.25x, 4.48x and 3.85x to Enum. •Furthermore, it has more potential to make Elixir much faster, if now-going research and development will be completed! 67 © 2020 Susumu Yamazaki © 2018 Susumu Yamazaki & Yuki Hisae
68. A NEW HOPE: Elixir and Pelemay 68 © 2020 Susumu Yamazaki © 2012 Plataformatec © 2018 Susumu Yamazaki & Yuki Hisae
69. A NEW HOPE: Elixir and Pelemay 69 © 2020 Susumu Yamazaki © 2012 Plataformatec © 2018 Susumu Yamazaki & Yuki Hisae • Elixir has enough potential to process concurrency for transmission. • Elixir and Pelemay will provide enough parallel computation for processing.
70. How will Elixir and Pelemay process and transmit such too huge images? 70 © 2020 Susumu Yamazaki © 2012 Plataformatec © 2018 Susumu Yamazaki & Yuki Hisae
71. TO BE CONTINUED…→ 71 © 2020 Susumu Yamazaki Look forward to my next presentation! Feel free to contact me @zacky1972