Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction of NII S. Koyama's Lab (AY2024)

Introduction of NII S. Koyama's Lab (AY2024)

NII S. Koyama's Lab

April 01, 2024
Tweet

More Decks by NII S. Koyama's Lab

Other Decks in Research

Transcript

  1. Introduction of Our Lab Audio Processing Research Group AY2024 Shoichi

    Koyama, Ph. D. Digital Content and Media Sciences Research Division, National Institute of Informatics
  2. April 1, 2024 2 Basic Technologies of Sound Field Analysis

    and Synthesis VR/AR audio Active noise control Local-field recording and reproduction Signal enhancement Visualization/auralization Room acoustic analysis Summary Sound field analysis/synthesis and its applications
  3. What is sound field analysis/synthesis? April 1, 2024 3 Estimating

    sound field inside target region using multiple mics Synthesizing desired sound field inside target region using multiple loudspeakers Wavefield-informed signal processing and machine learning for sound field analysis and synthesis Analysis Synthesis Microphone Loudspeaker
  4. Basic Technologies April 1, 2024 4 Analysis Synthesis Microphone Loudspeaker

    • Kernel interpolation with constraint of Helmholtz eq • Sparsity-based super-resolution • Analysis based on Reciprocity Gap Functional [Ueno+ IEEE SPL 2018, IEEE TSP 2021] [Murata+ IEEE TSP 2018, Koyama+ JASA 2018, IEEE JSTSP 2019] [Takida+ Signal Process 2019] Wavefield-informed signal processing and machine learning
  5. Analysis Synthesis Microphone Loudspeaker Basic Technologies April 1, 2024 5

    • Weighted pressure and mode matching for sound field control • Optimization of source and sensor placement • Amplitude matching for multizone control [Ueno+ IEEE/ACM TASLP 2019, Koyama+ JAES 2023] [Koyama+ IEEE/ACM TASLP 2020, Nishida+ IEEE TSP 2022] [Koyama+ IEEE ICASSP 2021, Abe+ IEEE/ACM TASLP 2023] Enhancing flexibility and scalability to make the range of applications broader Wavefield-informed signal processing and machine learning
  6. Kernel Interpolation of Sound Field Ø Kernel interpolation with constraint

    of Helmholtz eq – Estimated function should satisfy governing equation of acoustic field – Derived kernel function to constraint solution of kernel ridge regression satisfying Helmholtz eq April 1, 2024 7 Estimate continuous sound field from discrete mics Target region: Microphone <latexit sha1_base64="YqiyTflU53yE7Th8bfA2lgdHgBE=">AAACHXicbVDLSgNBEJyNrxhfUY9eBkMgQQm7IagXIagHjxHMA7IxzE46yZCZ3WVmVghLvsCP8Bu86tmbeBWP/omTx8EkFjQUVd10d3khZ0rb9reVWFldW99Ibqa2tnd299L7BzUVRJJClQY8kA2PKODMh6pmmkMjlECEx6HuDa7Hfv0RpGKBf6+HIbQE6fmsyyjRRmqnszns3gDXBJ/gwUMR53GUcz0Ry9GpGwjokTy+xHY7nbEL9gR4mTgzkkEzVNrpH7cT0EiAryknSjUdO9StmEjNKIdRyo0UhIQOSA+ahvpEgGrFk3dGOGuUDu4G0pSv8UT9OxETodRQeKZTEN1Xi95Y/M9rRrp70YqZH0YafDpd1I041gEeZ4M7TALVfGgIoZKZWzHtE0moNgnObfHEyGTiLCawTGrFgnNWKN2VMuWrWTpJdISOUQ456ByV0S2qoCqi6Am9oFf0Zj1b79aH9TltTVizmUM0B+vrF4oXn7I=</latexit> ( + k2)u(r, !) = 0 [Ueno+ IEEE SPL 2018, IEEE TSP 2021, Koyama+ IEEE ICASSP 2022 Tutorial] Kernel function: Helmholtz eq:
  7. Kernel Interpolation of Sound Field Ø Experimental results using real

    data from MeshRIR data set – Reconstructing pulse signal from single loudspeaker w/ 18 mic April 1, 2024 8 True Proposed Gaussian kernel (Black dots indicate mic positions) Impulse response measurement system [Koyama+ 2021]
  8. Sparse Sound Field Decomposition Ø Sound field inside region including

    sources satisfies inhomogeneous Helmholtz eq April 1, 2024 9 Sound field estimation inside region including sources Target region: Microphone <latexit sha1_base64="VXB56x8cV3M2DCu+b66chBSNCqQ=">AAADi3icdZJfaxNBEMC3Of/UaG2qj74sBiFFDXepoIiFUBF8KbRg2kIuDXObvWS53b1jd04Tj/g1/DS+6mfw27iXpmqay8DC8Jv9MbvDRJkUFn3/91bNu3X7zt3te/X7D3Ye7jb2Hp3ZNDeM91gqU3MRgeVSaN5DgZJfZIaDiiQ/j5L3Zf38MzdWpPoTzjI+UDDWIhYM0KFhI2iFGiIJl53nyWVnn+atUAFOorgw8xfJPj2kL09X0bDR9Nv+Iuh6EiyTJlnGyXCvdhSOUpYrrpFJsLYf+BkOCjAomOTzephbngFLYMz7LtWguB0Ui7/N6TNHRjROjTsa6YL+bxSgrJ2pyN0sX2lv1kpYVevnGL8ZFEJnOXLNrhrFuaSY0nJQdCQMZyhnLgFmhHsrZRMwwNCNc6VLpNwfNP/CUqVAj4oQzFjBdF4s5pZmRWgUdexbCUMplEBbYQhdYTi40bB5tm6UcKMhdLxulLDacN0d/spXnb90gwTTKuma/pMUJBzcwqIbaD1k3WOYHgMaMXVbaw+Dg+sbKU64cVsX3Nyx9eSs0w78dnD6qtk9Wu7fNnlCnpIWCchr0iUfyQnpEUa+kx/kJ/nl7XgH3lvv3dXV2tbSeUxWwvvwB0omMJU=</latexit> Unknown boundary condition on room surface Source distribution Prior information on is necessary to estimate <latexit sha1_base64="xQ9sjA9plcq9kqOcIDgKbT4eR7k=">AAACvHicfZFdSyMxFIbT0fWjrrbqpTeDZUGllJmlrHslol54IypYFTqlnEnP1NAkMyQZ2TrMz/DKW/1R+282U1vRVvdA4OE9b5LzESacaeN5f0vO3Py3hcWl5fLK99W1SnV941rHqaLYojGP1W0IGjmT2DLMcLxNFIIIOd6Eg+Mif3OPSrNYXplhgh0BfckiRsFYqVutpDuBAHMXRpnK64PdbrXmNbxRuLPgj6FGxnHRXS89Br2YpgKloRy0bvteYjoZKMMox7wcpBoToAPoY9uiBIG6k40qz90fVum5UazskcYdqe9vZCC0HorQOosq9XSuED/LtVMT/e5kTCapQUlfP4pS7prYLcbg9phCavjQAlDFbK0uvQMF1NhhlT98E4p68b7Skc5t5gRtmwrPrHSeoAITq70sANUX8Ce3bfeDekH/MzI5MVr6ymgfYYI9YJ690ZdWJifWCZXtIv3ptc3C9c+G/6vRvGzWDo/GK10iW2Sb7BCf7JNDckouSItQkpIn8kxenAOn5wwc8Wp1SuM7m+RDOPf/AJeG3Ys=</latexit> u(r, k) <latexit sha1_base64="Y7figz1V+fZdwrOdcUlH4QN0w5E=">AAACvHicfZFda9swFIYVdx9Zti7Jdrkbs1DoSgh2CduuSuh2sZuxBJamEIdwrBynwpJsJDk0M/4Zvert9qP2bybnY6zJ2gOCh/e8ks5HmHKmjef9rjgHjx4/eVp9Vnv+4vBlvdF8daGTTFEc0oQn6jIEjZxJHBpmOF6mCkGEHEdh/KnMjxaoNEvkd7NMcSJgLlnEKBgrTRv1wXEgwFyFUa6Kdvxu2mh5HW8V7j74G2iRTfSnzcpNMEtoJlAaykHrse+lZpKDMoxyLGpBpjEFGsMcxxYlCNSTfFV54R5ZZeZGibJHGnel/nsjB6H1UoTWWVapd3Ol+L/cODPRx0nOZJoZlHT9UZRx1yRuOQZ3xhRSw5cWgCpma3XpFSigxg6rduebULTL95WOdGEzn9G2qfCrlb6lqMAk6iQPQM0FXBe27XnQLukhI5Nbo6X7jPYRJtgPLPK/dK+Vya11SzW7SH93bftwcdrx33e6g26rd75ZaZW8IW/JMfHJB9IjX0ifDAklGbklP8kv58yZObEj1lansrnzmtwJZ/EHQ27dZw==</latexit> Q(r, k)
  9. Sparse Sound Field Decomposition Ø Represebtation by sum of particular

    and homogeneous solution Ø Approximation by linear eq April 1, 2024 10 [Koyama+ JASA 2018] Free-field Green s func <latexit sha1_base64="Bxhsq/TEXHZWpXBdZTZjErdTxDk=">AAADiHicdZLdahNBFICnWX9q/Ev10pvBIAhC2FWhelEI0QtvCi00bSEbwtnJ2WTI/Cwzs7rpsj6FT+OtvoRv42yaVNMkBxbOfmc+zszhJJng1oXhn71GcOfuvfv7D5oPHz1+8rR18Ozc6tww7DMttLlMwKLgCvuOO4GXmUGQicCLZPaprl98RWO5VmdunuFQwkTxlDNwHo1anViCmyZpOa/oEV39fK5WWVHRNzf4qhq12mEnXATdTKJl0ibLOBkdNHrxWLNconJMgLWDKMzcsATjOBNYNePcYgZsBhMc+FSBRDssFw+r6CtPxjTVxn/K0QX93yhBWjuXiT9ZX9HertVwW22Qu/TDsOQqyx0qdt0ozQV1mtZTomNukDkx9wkww/1dKZuCAeb8LNe6JNK/QeE3pqUENS5jMBMJRVUuhqazMjaSeva9hrHgkju7xeBqi+HhTsPm2aZRw50GV+mmUcPthu/u8RWuOzd0hwTFNmlF/0kSZgh+W50faDNm3WMojsEZXviVtUfRu9UJ7aZo/NZFt3dsMzl/24nCTnT6vt3tLfdvn7wgL8lrEpFD0iVfyAnpE0Z+kJ/kF/kdNIMwOAw+Xh9t7C2d52Qtgt5fS0ox9Q==</latexit> Direct component Reverberant component Dictionary matrix of Green s func Observation vector Direct component Reverberant component Grid point: <latexit sha1_base64="reMY8i1ewjRGRTHn33uoP6JqdRU=">AAADZHicdZLdahNREIBPs2prrNofvBJkMQhehd1aqDdCqDfeFCqYtpANYfZkNjnk/CznzNbEJX0Mb/W1fAGfw7NpqqbZDCwM35yPmR0mzaVwFEW/thrBg4ePtnceN5/sPn32fG//4MKZwnLsciONvUrBoRQauyRI4lVuEVQq8TKdfKzql9donTD6C81y7CsYaZEJDuRRkiigcZqVdj7Qg71W1I4WEa4n8TJpsWWcD/Ybp8nQ8EKhJi7BuV4c5dQvwZLgEufNpHCYA5/ACHs+1aDQ9cvF0PPwjSfDMDPWf5rCBf3fKEE5N1Opf1kN6e7XKlhX6xWUve+XQucFoea3jbJChmTCagPhUFjkJGc+AW6FnzXkY7DAye9ppUuq/D9o/MqNUqCHZQJ2pGA6LxdrM3mZWBV6dlPBRAolyNUYQtcYHm40XJGvGxXcaAidrRsVrDd8d4+/4arzl26QYFon3dF/koIJgr9E8gttJrxzBtMzICum/hzdh/jd3QtDY7T+6uL7N7aeXBy146gdfz5udU6X97fDXrLX7C2L2QnrsE/snHUZZzn7zn6wn43fwW5wGLy4fdrYWjqHbCWCV38AxlYkxQ==</latexit>
  10. Sparse Sound Field Decomposition Ø Assuming spatial sparsity of source

    distribution Ø Optimization algorithm considering multidimensional sparsity April 1, 2024 11 Sparse source signal in time-freq domain <latexit sha1_base64="CMp0bdMsbHJXrQaEDWLvT3oAJJo=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+f6ZrcTdaNlhJtJvEo6bBVnN3vN42RkeKFQE5fg3CCOchqWYElwiYt2UjjMgU9hjAOfalDohuXypovwrSejMDPWH03hkv5vlKCcm6vUdyqgiXtYq2BdbVBQ9mlYCp0XhJrfDcoKGZIJq2eHI2GRk5z7BLgV/q4hn4AFTv5z1qakyr9B41dulAI9KhOwYwWzRZlUU01eJlaFnn2vYCKFEuRqDKFrDA+3Gq7IN40KbjWEzjaNCtYbfrrH33Dd+Uu3SDCrk+7pP0nBFMGvH/kPbSe8dwqzUyArZn4H3VH84b7D0ASt37r44Y5tJpfvu3HUjc8PO73j1f612AF7w96xmH1kPXbCzlifcYbslv1gP5u/g0bQCtp3rc3GytlnaxHs/wFx4R/A</latexit> <latexit sha1_base64="f0AM1dFjbF0IvWOeeDQ4Zux15bE=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+d0s9uJutEyws0kXiUdtoqzm73mcTIyvFCoiUtwbhBHOQ1LsCS4xEU7KRzmwKcwxoFPNSh0w3J500X41pNRmBnrj6ZwSf83SlDOzVXqOxXQxD2sVbCuNigo+zQshc4LQs3vBmWFDMmE1bPDkbDISc59AtwKf9eQT8ACJ/85a1NS5d+g8Ss3SoEelQnYsYLZokyqqSYvE6tCz75XMJFCCXI1htA1hodbDVfkm0YFtxpCZ5tGBesNP93jb7ju/KVbJJjVSff0n6RgiuDXj/yHthPeO4XZKZAVM7+D7ij+cN9haILWb138cMc2k8v33TjqxueHnd7xav9a7IC9Ye9YzD6yHjthZ6zPOEN2y36wn83fQSNoBe271mZj5eyztQj2/wCDnx/G</latexit> <latexit sha1_base64="HW0NkVNdxtG8JSmuoj2JEWTs8bY=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+fZzW4n6kbLCDeTeJV02CrObvaax8nI8EKhJi7BuUEc5TQswZLgEhftpHCYA5/CGAc+1aDQDcvlTRfhW09GYWasP5rCJf3fKEE5N1ep71RAE/ewVsG62qCg7NOwFDovCDW/G5QVMiQTVs8OR8IiJzn3CXAr/F1DPgELnPznrE1JlX+Dxq/cKAV6VCZgxwpmizKpppq8TKwKPftewUQKJcjVGELXGB5uNVyRbxoV3GoInW0aFaw3/HSPv+G685dukWBWJ93Tf5KCKYJfP/If2k547xRmp0BWzPwOuqP4w32HoQlav3Xxwx3bTC7fd+OoG58fdnrHq/1rsQP2hr1jMfvIeuyEnbE+4wzZLfvBfjZ/B42gFbTvWpuNlbPP1iLY/wNaOR+4</latexit> <latexit sha1_base64="0uStd8PEwCtmctIFyVxU/PU2PFc=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+fqZrcTdaNlhJtJvEo6bBVnN3vN42RkeKFQE5fg3CCOchqWYElwiYt2UjjMgU9hjAOfalDohuXypovwrSejMDPWH03hkv5vlKCcm6vUdyqgiXtYq2BdbVBQ9mlYCp0XhJrfDcoKGZIJq2eHI2GRk5z7BLgV/q4hn4AFTv5z1qakyr9B41dulAI9KhOwYwWzRZlUU01eJlaFnn2vYCKFEuRqDKFrDA+3Gq7IN40KbjWEzjaNCtYbfrrH33Dd+Uu3SDCrk+7pP0nBFMGvH/kPbSe8dwqzUyArZn4H3VH84b7D0ASt37r44Y5tJpfvu3HUjc8PO73j1f612AF7w96xmH1kPXbCzlifcYbslv1gP5u/g0bQCtp3rc3GytlnaxHs/wFu7B+/</latexit> Activated grid [Koyama+ JASA 2018] <latexit sha1_base64="jt8UZmLAo4CM+kOHoDXBXXv/2js=">AAAD3HicdZPfihMxFMbTjn/WqmtXLxS8CRZhxW6ZqYLeCGX1QoSFFexuodMdzqSZNnSSmU0y2hrHC/FOvPXdfAMfw0zbce22DQx8+XJ+nDMnJ2EaM6Vd93el6ly5eu36zo3azVu3d+/U9+6eqCSThHZJEieyF4KiMRO0q5mOaS+VFHgY09Nw8ro4P/1IpWKJ+KBnKR1wGAkWMQLaWkH9m8+ZYJx9poHxOehxGJlenmM/kkCMl5u21SrjgdHNyMovZdAsxwe43LzJSzUtYoL2WRs/xX5s6xjCIopAbN7lgUmb583zfP8i15Og3nBb7nzhdeEtRQMt13GwVz30hwnJOBWaxKBU33NTPTAgNSMxzWt+pmgKZAIj2rdSAKdqYObNyvFj6wxxlEj7CY3n7v+EAa7UjIc2sqhRXT4rzE1n/UxHLweGiTTTVJBFoiiLsU5w0Xk8ZJISHc+sACKZrRWTMdg2a3s/K1lCbv9B0E8k4RzE0PggRxym+eKGktT4kmPrfS1MP7bXp9UGgokNhDW3EipL14nC3EowEa0ThbmZKGdtlfnnboFgugkq3QuIw4SCfQHaNrTmk84RTI9ASza1z0C98p6VEYkeU2mnzrs8Y+vipN3y3Jb3/nmjc7icvx30ED1C+8hDL1AHvUXHqIsI+lPZrdyvPHDOnO/OD+fnIrRaWTL30Mpyfv0FTtFRkg==</latexit> <latexit sha1_base64="XMTa05C+9ds+LrBKIVtxIUVcJmU=">AAADbXicdZLNihNBEIB7M/6s8S/r4kmRxiCulzCjgh4UwnrxsrCC2V3IhFDTqUma9J/dPZo4xFfxqo/kU/gK9mSzajaThhmKr+qjuovKjODOx/GvnUZ05eq167s3mjdv3b5zt7V378TpwjLsMS20PcvAoeAKe557gWfGIshM4Gk2fVflTz+jdVyrj35ucCBhrHjOGfiAhq39g5i+oYamAumn5T95Nmy14068PHQzSFZBm6zO8XCvcZiONCskKs8EONdPYuMHJVjPmcBFMy0cGmBTGGM/hAokukG5vP2CPglkRHNtw6c8XdL/jRKkc3OZhUoJfuIu5ypYl+sXPn89KLkyhUfFzhvlhaBe02oUdMQtMi/mIQBmebgrZROwwHwY2FqXTIY3KPzCtJSgRmUKdixhtijTqqs2ZWolDexbBVPBJfeuxuCqxghwq+EKs2lUcKvBVb5pVLDeCN0D/orrzl+6RYJZnXRB/0kSpghhJX0YaDNl3SOYHYG3fBb20r1NXlxUaD9BG7Yuubxjm8HJ804Sd5IPL9vdw9X+7ZIH5DE5IAl5RbrkPTkmPcLInHwnP8jPxu/ofvQwenRe2thZOftk7URP/wA0wyUg</latexit> <latexit sha1_base64="jNNevYceU70kmgc9X62mIHeNJOc=">AAADwHicdZLdbtMwFIC9hp9R/jq45MaiQmqlqkoACW4mVeUGIU0aEt0qNV114jqtVdtJbAfaeeEJeQFeg1u4wGk7RtfWUqST75xPJz45UcqZNr7/86Di3bl77/7hg+rDR4+fPK0dPTvTSa4I7ZGEJ6ofgaacSdozzHDaTxUFEXF6Hs0+lPnzr1RplsgvZpHSoYCJZDEjYBwa1eJQgJkS4PZTMbJpK2tlRWPJotj2iyY+xqHOxUjikNPYNFZv1rTiAl/d1DVky6Hm1UWGQ8UmU9O8sGGsgNi0sFlRjGp1v+0vD94OgnVQR+tzOjqqdMNxQnJBpSEctB4EfmqGFpRhhNOiGuaapkBmMKEDF0oQVA/tciAFfuXIGMeJco80eEn/NywIrRcicpXlFfTtXAl35Qa5id8PLZNpbqgkq0ZxzrFJcDldPGaKEsMXLgCimPtWTKbg5mDcP9joEgl3B0m/kUQIkGMbgpoImBd2OdQktaES2LHvJQw5E8zoHQaTOwwH9xo6T7eNEu41mIy3jRLuNlx3hy/ppvOP7pFgvku6pjeSgBkFt+XGDbQaks4JzE/AKDZ3q66PgzfXFYmZUuW2Lri9Y9vB2et24LeDz2/rne56/w7RC/QSNVCA3qEO+ohOUQ8R9AP9Qr/RH6/rTb3Ey1allYO18xxtHO/yL3fVSK4=</latexit> Multidimensional mixed-norm penalty ( -norm) <latexit sha1_base64="UO/CtIGxHZlCwhDll94dyV4o5hI=">AAADZHicdZJfaxNBEMC3Of/UWLW1+CTIYhB8KOGuCvVFCPXFl0IF0xZyIcxt5pIl+6+7ezXxiB/DV/1afgE/h3tpqqa5DBwMv5kfszdMZgR3Po5/bTWiO3fv3d9+0Hy48+jxk929p2dOF5Zhl2mh7UUGDgVX2PXcC7wwFkFmAs+zyYeqfn6F1nGtPvuZwb6EkeI5Z+ADSlMUYlCag8uDy/lgtxW340XQ9SRZJi2yjNPBXuM4HWpWSFSeCXCul8TG90uwnjOB82ZaODTAJjDCXkgVSHT9cvHoOX0VyJDm2oZPebqg/xslSOdmMgudEvzY3a5VsK7WK3z+rl9yZQqPil0PygtBvabVBuiQW2RezEICzPLwVsrGYIH5sKeVKZkM/6DwC9NSghqWKdiRhOm8TKup2pSplTSwbxVMBZfcuxqDqxojwI2GK8y6UcGNBlf5ulHBeiNMD/grrjp/6QYJpnXSDf0nSZgghEv0YaHNlHVOYHoC3vJpOEf3Pnlz06H9GG24uuT2ja0nZ4ftJG4nn962OsfL+9smz8lL8pok5Ih0yEdySrqEEUO+kx/kZ+N3tBPtR8+uWxtbS2efrET04g+IbSRa</latexit> [Murata+ IEEE TSP 2018]
  11. Application to Binaural Reproduction April 1, 2024 12 Conversion into

    binaural sounds Ø Binaural reproduction in real world is difficult, compared to binaural synthesis in VR space Ø Binaural reproduction from recordings of multiple small arrays instead of single spherical array Ø Broad listening area by using flexible and scalable recording system Binaural reproduction from mic array recordings for VR audio Recording Reproduction [Iijima+ JASA 2021]
  12. Application to Binaural Reproduction Ø Recording system using multiple Ambisonic

    mics and 360-degree cameras April 1, 2024 13 Small mic arrays (Ambisonic mics) 360-degree cameras Demo Proposed Single array [Iijima+ IEEE WASPAA 2021 (demo)] Error distribution
  13. Sound Field Synthesis Ø Optimization problem to obtain loudspeaker driving

    signals April 1, 2024 15 Synthesizing desired pressure field w/ multiple loudspeakers Loudspeaker Target region: Synthesized sound field Desired sound field Difficult to solve owing to regional integration Driving signal Transfer function
  14. Conventional Pressure Matching Ø Driving signals are obtained as simple

    least-squares solution April 1, 2024 16 : Target region Secondary source Control points : Target region Secondary source Discretization of target region into control points Discrete approxmation Transfer function matrix Driving signal vector Desired pressure vector <latexit sha1_base64="i9WfaOjLrhWO59r+nVt9MC7/SLM=">AAADEHicfZHLbhMxFIY90wIlXJrCko1FhFRu0QyqgA1SBUiUBaJIpK0Up5HHOTOxantGtgcRLL8AS56GHWLLmg2vwgrPJEH0xpEsff7Pf3xsn6wS3Ngk+RXFK6sXLl5au9y5cvXa9fXuxo09U9aawYCVotQHGTUguIKB5VbAQaWBykzAfnb0osnvfwBteKne21kFI0kLxXPOqA3SuKtIJt3E42eYCMjtJm72r/yhI5LaqcndjvdzCd/HBCxtDa89JpoXU3v30D1M/VlFrVYvNR2agPF+3O0l/aQNfBrSBfTQInbHG9FnMilZLUFZJqgxwzSp7MhRbTkT4DukNlBRdkQLGAZUVIIZufZjPL4TlAnOSx2WsrhV/61wVBozk1lwtjc/mWvEs3LD2uZPR46rqrag2LxRXgtsS9z8Mp5wDcyKWQDKNA93xWxKNWU2zOJYl0z6Toe8hPA4DW9Co7cVaGpLfc8RqgtJP/rw2II8aOh/Rq6WxkDnGcMhXPJP4N1fOtfK1dK6pDC99OSsTsPeo376uL/1bqu3/XwxxzV0C91GmyhFT9A22kG7aIAY+ol+RyvRavwl/hp/i7/PrXG0qLmJjkX84w9/cP+K</latexit> d = GHG + ⌘I 1 GHudes
  15. Weighted Pressure Matching Ø Original cost function is approximated as

    Ø Driving signals are obtained as weighted least squares solution April 1, 2024 17 Pressure matching for continuous region based on kernel interpolation of sound field <latexit sha1_base64="wBoAciZ2FK9Swr355ix7ob1QUvE=">AAADEnicfZHLbtNAFIYn5lbCLYUlmxERUkEosqsK2CBVLQs2qEVqmoo4RMeTY3dUz9iaGVdNR/MWPADPwQrElhfgBdjCIzB2EkR64UiWP/3nn9t/kjLn2oThj1Zw5eq16zdWbrZv3b5z915n9f6+LirFsM+KvFAHCWjMucS+4SbHg1IhiCTHQXK0XfcHx6g0L+SemZY4EpBJnnIGxkvjzvs4EXbg6Csac2nGNt4RmIGjtXzq1uqfck8+2Bi0uUAVYA51avec79WshJ3MbMqNO92wFzZFz0M0hy6Z1+54tfUpnhSsEigNy0HrYRSWZmRBGc5ydO240lgCO4IMhx4lCNQj24Tg6GOvTGhaKP9JQxv13xUWhNZTkXhnc+uzvVq8qDesTPpyZLksK4OSzQ5Kq5yagtaJ0glXyEw+9QBMcX9Xyg5BATM+96VTErH0BtskplPt2u34NfonK3zrpZ0SFZhCPfWpq0zAifMRZPGzmv5n5HJh9HSZ0W/CBT9FZ//SpVYuF9YF+ZlGZyd4HvbXe9Hz3sa7je7m1ny6K+QheUTWSERekE3yhuySPmHkC/lJfpHfwcfgc/A1+DazBq35mgdkqYLvfwAs/wN9</latexit> W = Z ⌦ z(r)⇤z(r)Tdr <latexit sha1_base64="ots8Wmdw3c6kEAqnCoYqjjAHV2E=">AAAD0HiclVJbb9MwFHYWLqNctsEjL9YqpgJb1UwT8II0ARIDhDbQuk6au8pxT1KrcRLZDmrJLMQjvPM7eIWfwr/BSdOKbhSJIyX+9J3v3HzspxFXutX65Sy5ly5fubp8rXb9xs1bK6trt49UkkkGbZZEiTz2qYKIx9DWXEdwnEqgwo+g4w+fF/7OB5CKJ/GhHqfQFTSMecAZ1ZbqrTnrr/EGoWkqkxEmPNa9nOwLCKnBJIJAn2Hii5wMrYKaRoGluX+aE0H1QAX5oal0jVL3xjwkI16gV5aXPBxoK97y5kQvTfHvm63iyEyVTFoKlJmFVefZ6Tae+c2kPiaktvG0SllmXJRs1sK0372igFV2zH+E91brrWarNHwReBWoo8oO7K1+I/2EZQJizSKq1InXSnU3p1JzFoGpkUxBStmQhnBiYUwFqG5ebtPge5bp4yCR9os1Ltk/I3IqlBoL3yrLqc77CvJvvpNMB0+6OY/TTEPMJoWCLMI6wcXTwH0ugelobAFlktteMRtQSZm2D6g2V8YXm+VFqUAZ63kBdkwJby21n4KkOpEPckJlKOjI2LFDslmgfwl5PBVatEhok3DBP4LJZ2ihlMdT6RTV7CK982u7CI62m96j5s67nfrus2qly+guWkcN5KHHaBftoQPURsz54nx3fjg/3ffuyP3kfp5Il5wq5g6aM/frbwuZR80=</latexit> J ⇡ Z ⌦ (r)T (K + ⇠I) 1 Gd udes 2 dr = Gd udes H W Gd udes <latexit sha1_base64="UqfxQe3UGsqqnmcvxH4VBmwhEN8=">AAAC/XicfZFdaxNBFIYn61eNH03rpTeDQYhaQ1aKFkEo6oUiYoSmLWRjODs5uxkyM7vMzErTZfBX+BO8Ey/1twje6u9wdpuIba0HBh7e952PMyfOBTe21/veCM6dv3Dx0srl5pWr166vttbWd01WaIYDlolM78dgUHCFA8utwP1cI8hY4F48e1b5e+9RG56pHTvPcSQhVTzhDKyXxq1+FMvy0NHHT2hF0QzyHFynYu3uvCsjCXZqknLHORoJTGynzr1y92h0wCt86Q3N06n16fuhG7favW6vLnoawgW0yaL647XGx2iSsUKiskyAMcOwl9tRCdpyJtA1o8JgDmwGKQ49KpBoRmXduqO3vTKhSab9UpbW6t87SpDGzGXsk3UnJ71K/Jc3LGyyNSq5yguLih1dlBSC2oxW/0gnXCOzYu4BmOb+rZRNQQOz/rebx66J5UZ1vjaJcd55jr5Nja+99CZHDTbTd8sIdCrhwPm202ijov8FuVoGPZ0V9IdwyQ/RlX/ozChXy+iSmn6Q4cmxnYbdB93wYXfz7WZ7++lipCvkJrlFOiQkj8g2eUH6ZEAY+Up+kJ/kV/Ah+BR8Dr4cRYPGYs8NcqyCb78Bcfj4Dw==</latexit> z := (r)T (K + ⇠I) 1 Kernel ridge regression <latexit sha1_base64="CRJvQ0OuNgbbSk0jTZw5dDlZAo8=">AAADxHiclVJbaxNBFJ7teqnx0ovgiy+DQanahqyU6otQrNgKihVMU8ikYXZyNhk6M7vMzIpxGf+Fv8BX/UH+G2e3G0nbRPDAwrff+c5tzokzwY1tt38HS+GVq9euL99o3Lx1+87K6tr6kUlzzaDDUpHq45gaEFxBx3Ir4DjTQGUsoBuf7pX+7mfQhqfqk51k0Jd0pHjCGbWeGqwF98iY2oLEshg6hx+9xITqkeRqUHOEK0wkteM4LvbcyTuHiYDEbuDSvT90eKtCuTspKpn2QWB8KqL5aGwf17RJioOS9NLu/6bAhDTKxmaC3Jy0+w4/xQQsLf/ezjSwFbnFUW5+7cFqs91qV4Yvg6gGTVTboX/J72SYslyCskxQY3pRO7P9gmrLmQDXILmBjLJTOoKeh4pKMP2i2qDDDz0zxEmq/acsrtjZiIJKYyYy9spqgIu+kpzn6+U2edEvuMpyC4qdFUpygW2Ky3PAQ66BWTHxgDLNfa+YjammzPqjaZwrE8vN6oVMYpz3vAY/pob3nvqQgaY21U+K6njoF+fHHpHNEv1LyNVU6NEioU/CJf8KrviLFkq5mkqnqOEXGV1c22Vw9KwV7bS2P243d1/VK11G99EDtIEi9BztogN0iDqIBS74EfwMfoVvQhGaMD+TLgV1zF10zsJvfwD8G0Dj</latexit> ˆ d = arg min d2CL Gd udes H W Gd udes = GHW G + ⌘I 1 GHW udes [Koyama+ JAES 2023]
  16. Weighted Pressure Matching Ø Comparison between Pressure Matching and Weighted

    Pressure Matching April 1, 2024 18 PM WPM (uniform) WPM (directional) Pressure Error [Koyama+ JAES 2023]
  17. Amplitude Matching for Multizone Sound Field Control Ø Problem to

    be solved in amplitude matching April 1, 2024 19 Target region Generating multiple personal sound zones by using loudspeakers Desired amplitude No closed form solution, but majorization minimization (MM) algorithm or alternating direction method of multipliers (ADMM) can be applied Element-wise absolute value [Abe+ IEEE/ACM TASLP 2023]
  18. Amplitude Matching for Multizone Sound Field Control April 1, 2024

    20 https://youtu.be/oYw7kmpZcY4 Full version:
  19. Application to Spatial Active Noise Control Ø Environmental noise is

    still unsolved problem Ø Active noise control (ANC) is aimed to cancel noise by loudspeaker signals, but its effect is limited to local region Ø ANC in 3D space based on sound field analysis/synthesis April 1, 2024 21 Noise suppression by loudspeaker signals Quiet zone
  20. Application to Spatial Active Noise Control Ø Cost function of

    regional noise power is estimated by kernel interpolation of sound field Ø Adaptive filtering algorithm based on kernel interpolation is also derived April 1, 2024 22 ANC in 3D space based on sound field interpolation Ø Conventional cost function Ø Proposed cost function <latexit sha1_base64="Pnlxe5THpW3gUCE+pMJvaUIbsmQ=">AAACFXicbZDLSsNAFIYnXmu9RV2KMFgUVyUpRd0IRTcuXFSwF2himUwn7dCZJMxMhBKz8iF8Bre6diduXbv0TZy0WdjWHwY+/nMO58zvRYxKZVnfxsLi0vLKamGtuL6xubVt7uw2ZRgLTBo4ZKFoe0gSRgPSUFQx0o4EQdxjpOUNr7J664EIScPgTo0i4nLUD6hPMVLa6poHDkdqgBFLblJ4fAGdR+h4PCGppvtK1yxZZWssOA92DiWQq941f5xeiGNOAoUZkrJjW5FyEyQUxYykRSeWJEJ4iPqkozFAnEg3GX8jhUfa6UE/FPoFCo7dvxMJ4lKOuKc7s6PlbC0z/6t1YuWfuwkNoliRAE8W+TGDKoRZJrBHBcGKjTQgLKi+FeIBEggrndzUFo+nOhN7NoF5aFbK9mm5elst1S7zdApgHxyCE2CDM1AD16AOGgCDJ/ACXsGb8Wy8Gx/G56R1wchn9sCUjK9f5U+ekQ==</latexit> L = kek2 : Power of error mics : Regional noise power <latexit sha1_base64="pm/hhsXKsYUZ/lWoZalEg8K5t2U=">AAACNHicbVDLSgMxFM34rPVVdekmWJS6KTOlqAuFohsXghXsAzq1ZNK0DU0yQ5IRynR+xo/wG9zqTnAj4tZvMNN2YVsPBA7nnMu9OV7AqNK2/W4tLC4tr6ym1tLrG5tb25md3aryQ4lJBfvMl3UPKcKoIBVNNSP1QBLEPUZqXv8q8WuPRCrqi3s9CEiTo66gHYqRNlIrc+5ypHsYsegmhkcX0KVCtyL3lpMuiuEwzLkej2R8PHwowFFU8qgdj0XYymTtvD0CnCfOhGTBBOVW5tNt+zjkRGjMkFINxw50M0JSU8xInHZDRQKE+6hLGoYKxIlqRqNfxvDQKG3Y8aV5QsOR+nciQlypAfdMMjlUzXqJ+J/XCHXnrBlREYSaCDxe1AkZ1D5MKoNtKgnWbGAIwpKaWyHuIYmwNsVObfF4bDpxZhuYJ9VC3jnJF++K2dLlpJ0U2AcHIAcccApK4BqUQQVg8ARewCt4s56tD+vL+h5HF6zJzB6YgvXzC9DNrBs=</latexit> L = Z ⌦ |u(r)|2dr [Ito+ IEEE ICASSP 2019 (Best Student Paper Award), Koyama+ IEEE/ACM TASLP 2021]
  21. Application to Spatial Active Noise Control Ø Band-limited noise (500-800Hz),

    T60 : 240ms April 1, 2024 23 Proposed: -10.5 dB MPC: -6.2 dB (dB) Regional noise reduction is achieved by the proposed method
  22. Summary Ø Recent research topics – Wavefield-informed DNN for sound

    field estimation, Spatial active noise control, Amplitude matching for generating personal sound zones, DNN-based HRTF interpolation, Source and sensor placement for sound field control, Region-to-region interpolation of acoustic transfer functions Ø Keywords – Kernel methods, Gaussian process, Reproducing kernel Hilbert space, Sparse modeling, Deep neural network, Physics-informed machine learning, Adaptive filter, Convex optimization, Physical acoustics, Partial differential equation April 1, 2024 24