Energy-Efficient 360-Degree Video Rendering on FPGA via Algorithm- Architecture Co-Design

F0c4b39a71fc7c752d4e6c451f6f678b?s=47 HorizonLab
February 24, 2020

Energy-Efficient 360-Degree Video Rendering on FPGA via Algorithm- Architecture Co-Design

FPGA 2020 presentation. Presented by Qiuyue Sun.

F0c4b39a71fc7c752d4e6c451f6f678b?s=128

HorizonLab

February 24, 2020
Tweet

Transcript

  1. 3.
  2. 4.
  3. 10.

    360-Degree Video Delivery Pipeline Rendering Field of View (FOV) Consumes

    over 4 W power Exceeds TDP of typical mobile devices Original Frame
  4. 15.

    Rendering 4 Current Rendering Algorithm Mapping Perspective Update Filtering Matrix

    Multiplication Cartesian Coordinates Linear Interpolation
  5. 23.

    Challenges: Memory Accesses ▸ Irregular Access Pattern ▹Accesses are not

    sequential ▹Severely hurts the efficiency of hardware acceleration
  6. 24.

    Challenges: Memory Accesses ▸ Irregular Access Pattern ▹Accesses are not

    sequential ▹Severely hurts the efficiency of hardware acceleration ▸ Large Footprint
  7. 25.

    Challenges: Memory Accesses ▸ Irregular Access Pattern ▹Accesses are not

    sequential ▹Severely hurts the efficiency of hardware acceleration ▸ Large Footprint ▹1080P is ~5.9 MB and 4K is ~23.7 MB
  8. 26.

    Challenges: Memory Accesses ▸ Irregular Access Pattern ▹Accesses are not

    sequential ▹Severely hurts the efficiency of hardware acceleration ▸ Large Footprint ▹1080P is ~5.9 MB and 4K is ~23.7 MB ▹Cannot be fully captured by a typical on-chip memory
  9. 31.

    Our Design ▸ Enforce a streaming data access ▸ Reduce

    unnecessary computations (x’, y’) (x, y)
  10. 32.

    Our Design ▸ Enforce a streaming data access ▸ Reduce

    unnecessary computations ▹Perform boundary checking
  11. 33.

    Our Design ▸ Enforce a streaming data access ▸ Reduce

    unnecessary computations ▹Perform boundary checking ▸ Fully pipeline pixel rendering
  12. 36.

    Setup and Evaluation 8 ▸ Xilinx Zynq UltraScale+ MPSoC ZCU104

    ▸ Pascal GPU on the Nvidia Jetson TX2
  13. 37.

    Setup and Evaluation 8 ▸ Xilinx Zynq UltraScale+ MPSoC ZCU104

    ▸ Pascal GPU on the Nvidia Jetson TX2 ▸ Real User Trace Evaluation
  14. 38.

    Setup and Evaluation 8 ▸ Xilinx Zynq UltraScale+ MPSoC ZCU104

    ▸ Pascal GPU on the Nvidia Jetson TX2 ▸ Real User Trace Evaluation ▸ Baseline: Original algorithm implemented on GPU and FPGA
  15. 39.

    Setup and Evaluation 8 Energy Savings(%) 0 20 40 60

    RC Elephant NYC Rhino Paris Venice Saving over FPGA Savings over GPU ▸ Xilinx Zynq UltraScale+ MPSoC ZCU104 ▸ Pascal GPU on the Nvidia Jetson TX2 ▸ Real User Trace Evaluation ▸ Baseline: Original algorithm implemented on GPU and FPGA
  16. 40.

    Setup and Evaluation 8 Energy Savings(%) 0 20 40 60

    RC Elephant NYC Rhino Paris Venice Saving over FPGA Savings over GPU ▸ Xilinx Zynq UltraScale+ MPSoC ZCU104 ▸ Pascal GPU on the Nvidia Jetson TX2 ▸ Real User Trace Evaluation ▸ Baseline: Original algorithm implemented on GPU and FPGA
  17. 41.

    Setup and Evaluation 8 Energy Savings(%) 0 20 40 60

    RC Elephant NYC Rhino Paris Venice Saving over FPGA Savings over GPU ▸ Xilinx Zynq UltraScale+ MPSoC ZCU104 ▸ Pascal GPU on the Nvidia Jetson TX2 ▸ Real User Trace Evaluation ▸ Baseline: Original algorithm implemented on GPU and FPGA
  18. 44.

    Conclusion 9 ▸ 360-degree video rendering consumes excessive power ▸

    Virtual reality popularity is growing rapidly
  19. 45.

    Conclusion 9 ▸ 360-degree video rendering consumes excessive power ▸

    Our co-design achieves on average 26.4% and 40.0% energy savings over baselines ▸ Virtual reality popularity is growing rapidly