Introduction of NII S. Koyama's Lab (AY2024)

Transcript

1. Introduction of Our Lab Audio Processing Research Group AY2024 Shoichi Koyama, Ph. D. Digital Content and Media Sciences Research Division, National Institute of Informatics
2. April 1, 2024 2 Basic Technologies of Sound Field Analysis and Synthesis VR/AR audio Active noise control Local-field recording and reproduction Signal enhancement Visualization/auralization Room acoustic analysis Summary Sound field analysis/synthesis and its applications
3. What is sound field analysis/synthesis? April 1, 2024 3 Estimating sound field inside target region using multiple mics Synthesizing desired sound field inside target region using multiple loudspeakers Wavefield-informed signal processing and machine learning for sound field analysis and synthesis Analysis Synthesis Microphone Loudspeaker
4. Basic Technologies April 1, 2024 4 Analysis Synthesis Microphone Loudspeaker • Kernel interpolation with constraint of Helmholtz eq • Sparsity-based super-resolution • Analysis based on Reciprocity Gap Functional [Ueno+ IEEE SPL 2018, IEEE TSP 2021] [Murata+ IEEE TSP 2018, Koyama+ JASA 2018, IEEE JSTSP 2019] [Takida+ Signal Process 2019] Wavefield-informed signal processing and machine learning
5. Analysis Synthesis Microphone Loudspeaker Basic Technologies April 1, 2024 5 • Weighted pressure and mode matching for sound field control • Optimization of source and sensor placement • Amplitude matching for multizone control [Ueno+ IEEE/ACM TASLP 2019, Koyama+ JAES 2023] [Koyama+ IEEE/ACM TASLP 2020, Nishida+ IEEE TSP 2022] [Koyama+ IEEE ICASSP 2021, Abe+ IEEE/ACM TASLP 2023] Enhancing flexibility and scalability to make the range of applications broader Wavefield-informed signal processing and machine learning
6. SOUND FIELD ANALYSIS April 1, 2024 6
7. Kernel Interpolation of Sound Field Ø Kernel interpolation with constraint of Helmholtz eq – Estimated function should satisfy governing equation of acoustic field – Derived kernel function to constraint solution of kernel ridge regression satisfying Helmholtz eq April 1, 2024 7 Estimate continuous sound field from discrete mics Target region: Microphone <latexit sha1_base64="YqiyTflU53yE7Th8bfA2lgdHgBE=">AAACHXicbVDLSgNBEJyNrxhfUY9eBkMgQQm7IagXIagHjxHMA7IxzE46yZCZ3WVmVghLvsCP8Bu86tmbeBWP/omTx8EkFjQUVd10d3khZ0rb9reVWFldW99Ibqa2tnd299L7BzUVRJJClQY8kA2PKODMh6pmmkMjlECEx6HuDa7Hfv0RpGKBf6+HIbQE6fmsyyjRRmqnszns3gDXBJ/gwUMR53GUcz0Ry9GpGwjokTy+xHY7nbEL9gR4mTgzkkEzVNrpH7cT0EiAryknSjUdO9StmEjNKIdRyo0UhIQOSA+ahvpEgGrFk3dGOGuUDu4G0pSv8UT9OxETodRQeKZTEN1Xi95Y/M9rRrp70YqZH0YafDpd1I041gEeZ4M7TALVfGgIoZKZWzHtE0moNgnObfHEyGTiLCawTGrFgnNWKN2VMuWrWTpJdISOUQ456ByV0S2qoCqi6Am9oFf0Zj1b79aH9TltTVizmUM0B+vrF4oXn7I=</latexit> ( + k2)u(r, !) = 0 [Ueno+ IEEE SPL 2018, IEEE TSP 2021, Koyama+ IEEE ICASSP 2022 Tutorial] Kernel function: Helmholtz eq:
8. Kernel Interpolation of Sound Field Ø Experimental results using real data from MeshRIR data set – Reconstructing pulse signal from single loudspeaker w/ 18 mic April 1, 2024 8 True Proposed Gaussian kernel (Black dots indicate mic positions) Impulse response measurement system [Koyama+ 2021]
9. Sparse Sound Field Decomposition Ø Sound field inside region including sources satisfies inhomogeneous Helmholtz eq April 1, 2024 9 Sound field estimation inside region including sources Target region: Microphone <latexit sha1_base64="VXB56x8cV3M2DCu+b66chBSNCqQ=">AAADi3icdZJfaxNBEMC3Of/UaG2qj74sBiFFDXepoIiFUBF8KbRg2kIuDXObvWS53b1jd04Tj/g1/DS+6mfw27iXpmqay8DC8Jv9MbvDRJkUFn3/91bNu3X7zt3te/X7D3Ye7jb2Hp3ZNDeM91gqU3MRgeVSaN5DgZJfZIaDiiQ/j5L3Zf38MzdWpPoTzjI+UDDWIhYM0KFhI2iFGiIJl53nyWVnn+atUAFOorgw8xfJPj2kL09X0bDR9Nv+Iuh6EiyTJlnGyXCvdhSOUpYrrpFJsLYf+BkOCjAomOTzephbngFLYMz7LtWguB0Ui7/N6TNHRjROjTsa6YL+bxSgrJ2pyN0sX2lv1kpYVevnGL8ZFEJnOXLNrhrFuaSY0nJQdCQMZyhnLgFmhHsrZRMwwNCNc6VLpNwfNP/CUqVAj4oQzFjBdF4s5pZmRWgUdexbCUMplEBbYQhdYTi40bB5tm6UcKMhdLxulLDacN0d/spXnb90gwTTKuma/pMUJBzcwqIbaD1k3WOYHgMaMXVbaw+Dg+sbKU64cVsX3Nyx9eSs0w78dnD6qtk9Wu7fNnlCnpIWCchr0iUfyQnpEUa+kx/kJ/nl7XgH3lvv3dXV2tbSeUxWwvvwB0omMJU=</latexit> Unknown boundary condition on room surface Source distribution Prior information on is necessary to estimate <latexit sha1_base64="xQ9sjA9plcq9kqOcIDgKbT4eR7k=">AAACvHicfZFdSyMxFIbT0fWjrrbqpTeDZUGllJmlrHslol54IypYFTqlnEnP1NAkMyQZ2TrMz/DKW/1R+282U1vRVvdA4OE9b5LzESacaeN5f0vO3Py3hcWl5fLK99W1SnV941rHqaLYojGP1W0IGjmT2DLMcLxNFIIIOd6Eg+Mif3OPSrNYXplhgh0BfckiRsFYqVutpDuBAHMXRpnK64PdbrXmNbxRuLPgj6FGxnHRXS89Br2YpgKloRy0bvteYjoZKMMox7wcpBoToAPoY9uiBIG6k40qz90fVum5UazskcYdqe9vZCC0HorQOosq9XSuED/LtVMT/e5kTCapQUlfP4pS7prYLcbg9phCavjQAlDFbK0uvQMF1NhhlT98E4p68b7Skc5t5gRtmwrPrHSeoAITq70sANUX8Ce3bfeDekH/MzI5MVr6ymgfYYI9YJ690ZdWJifWCZXtIv3ptc3C9c+G/6vRvGzWDo/GK10iW2Sb7BCf7JNDckouSItQkpIn8kxenAOn5wwc8Wp1SuM7m+RDOPf/AJeG3Ys=</latexit> u(r, k) <latexit sha1_base64="Y7figz1V+fZdwrOdcUlH4QN0w5E=">AAACvHicfZFda9swFIYVdx9Zti7Jdrkbs1DoSgh2CduuSuh2sZuxBJamEIdwrBynwpJsJDk0M/4Zvert9qP2bybnY6zJ2gOCh/e8ks5HmHKmjef9rjgHjx4/eVp9Vnv+4vBlvdF8daGTTFEc0oQn6jIEjZxJHBpmOF6mCkGEHEdh/KnMjxaoNEvkd7NMcSJgLlnEKBgrTRv1wXEgwFyFUa6Kdvxu2mh5HW8V7j74G2iRTfSnzcpNMEtoJlAaykHrse+lZpKDMoxyLGpBpjEFGsMcxxYlCNSTfFV54R5ZZeZGibJHGnel/nsjB6H1UoTWWVapd3Ol+L/cODPRx0nOZJoZlHT9UZRx1yRuOQZ3xhRSw5cWgCpma3XpFSigxg6rduebULTL95WOdGEzn9G2qfCrlb6lqMAk6iQPQM0FXBe27XnQLukhI5Nbo6X7jPYRJtgPLPK/dK+Vya11SzW7SH93bftwcdrx33e6g26rd75ZaZW8IW/JMfHJB9IjX0ifDAklGbklP8kv58yZObEj1lansrnzmtwJZ/EHQ27dZw==</latexit> Q(r, k)
10. Sparse Sound Field Decomposition Ø Represebtation by sum of particular and homogeneous solution Ø Approximation by linear eq April 1, 2024 10 [Koyama+ JASA 2018] Free-field Green s func <latexit sha1_base64="Bxhsq/TEXHZWpXBdZTZjErdTxDk=">AAADiHicdZLdahNBFICnWX9q/Ev10pvBIAhC2FWhelEI0QtvCi00bSEbwtnJ2WTI/Cwzs7rpsj6FT+OtvoRv42yaVNMkBxbOfmc+zszhJJng1oXhn71GcOfuvfv7D5oPHz1+8rR18Ozc6tww7DMttLlMwKLgCvuOO4GXmUGQicCLZPaprl98RWO5VmdunuFQwkTxlDNwHo1anViCmyZpOa/oEV39fK5WWVHRNzf4qhq12mEnXATdTKJl0ibLOBkdNHrxWLNconJMgLWDKMzcsATjOBNYNePcYgZsBhMc+FSBRDssFw+r6CtPxjTVxn/K0QX93yhBWjuXiT9ZX9HertVwW22Qu/TDsOQqyx0qdt0ozQV1mtZTomNukDkx9wkww/1dKZuCAeb8LNe6JNK/QeE3pqUENS5jMBMJRVUuhqazMjaSeva9hrHgkju7xeBqi+HhTsPm2aZRw50GV+mmUcPthu/u8RWuOzd0hwTFNmlF/0kSZgh+W50faDNm3WMojsEZXviVtUfRu9UJ7aZo/NZFt3dsMzl/24nCTnT6vt3tLfdvn7wgL8lrEpFD0iVfyAnpE0Z+kJ/kF/kdNIMwOAw+Xh9t7C2d52Qtgt5fS0ox9Q==</latexit> Direct component Reverberant component Dictionary matrix of Green s func Observation vector Direct component Reverberant component Grid point: <latexit sha1_base64="reMY8i1ewjRGRTHn33uoP6JqdRU=">AAADZHicdZLdahNREIBPs2prrNofvBJkMQhehd1aqDdCqDfeFCqYtpANYfZkNjnk/CznzNbEJX0Mb/W1fAGfw7NpqqbZDCwM35yPmR0mzaVwFEW/thrBg4ePtnceN5/sPn32fG//4MKZwnLsciONvUrBoRQauyRI4lVuEVQq8TKdfKzql9donTD6C81y7CsYaZEJDuRRkiigcZqVdj7Qg71W1I4WEa4n8TJpsWWcD/Ybp8nQ8EKhJi7BuV4c5dQvwZLgEufNpHCYA5/ACHs+1aDQ9cvF0PPwjSfDMDPWf5rCBf3fKEE5N1Opf1kN6e7XKlhX6xWUve+XQucFoea3jbJChmTCagPhUFjkJGc+AW6FnzXkY7DAye9ppUuq/D9o/MqNUqCHZQJ2pGA6LxdrM3mZWBV6dlPBRAolyNUYQtcYHm40XJGvGxXcaAidrRsVrDd8d4+/4arzl26QYFon3dF/koIJgr9E8gttJrxzBtMzICum/hzdh/jd3QtDY7T+6uL7N7aeXBy146gdfz5udU6X97fDXrLX7C2L2QnrsE/snHUZZzn7zn6wn43fwW5wGLy4fdrYWjqHbCWCV38AxlYkxQ==</latexit>
11. Sparse Sound Field Decomposition Ø Assuming spatial sparsity of source distribution Ø Optimization algorithm considering multidimensional sparsity April 1, 2024 11 Sparse source signal in time-freq domain <latexit sha1_base64="CMp0bdMsbHJXrQaEDWLvT3oAJJo=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+f6ZrcTdaNlhJtJvEo6bBVnN3vN42RkeKFQE5fg3CCOchqWYElwiYt2UjjMgU9hjAOfalDohuXypovwrSejMDPWH03hkv5vlKCcm6vUdyqgiXtYq2BdbVBQ9mlYCp0XhJrfDcoKGZIJq2eHI2GRk5z7BLgV/q4hn4AFTv5z1qakyr9B41dulAI9KhOwYwWzRZlUU01eJlaFnn2vYCKFEuRqDKFrDA+3Gq7IN40KbjWEzjaNCtYbfrrH33Dd+Uu3SDCrk+7pP0nBFMGvH/kPbSe8dwqzUyArZn4H3VH84b7D0ASt37r44Y5tJpfvu3HUjc8PO73j1f612AF7w96xmH1kPXbCzlifcYbslv1gP5u/g0bQCtp3rc3GytlnaxHs/wFx4R/A</latexit> <latexit sha1_base64="f0AM1dFjbF0IvWOeeDQ4Zux15bE=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+d0s9uJutEyws0kXiUdtoqzm73mcTIyvFCoiUtwbhBHOQ1LsCS4xEU7KRzmwKcwxoFPNSh0w3J500X41pNRmBnrj6ZwSf83SlDOzVXqOxXQxD2sVbCuNigo+zQshc4LQs3vBmWFDMmE1bPDkbDISc59AtwKf9eQT8ACJ/85a1NS5d+g8Ss3SoEelQnYsYLZokyqqSYvE6tCz75XMJFCCXI1htA1hodbDVfkm0YFtxpCZ5tGBesNP93jb7ju/KVbJJjVSff0n6RgiuDXj/yHthPeO4XZKZAVM7+D7ij+cN9haILWb138cMc2k8v33TjqxueHnd7xav9a7IC9Ye9YzD6yHjthZ6zPOEN2y36wn83fQSNoBe271mZj5eyztQj2/wCDnx/G</latexit> <latexit sha1_base64="HW0NkVNdxtG8JSmuoj2JEWTs8bY=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+fZzW4n6kbLCDeTeJV02CrObvaax8nI8EKhJi7BuUEc5TQswZLgEhftpHCYA5/CGAc+1aDQDcvlTRfhW09GYWasP5rCJf3fKEE5N1ep71RAE/ewVsG62qCg7NOwFDovCDW/G5QVMiQTVs8OR8IiJzn3CXAr/F1DPgELnPznrE1JlX+Dxq/cKAV6VCZgxwpmizKpppq8TKwKPftewUQKJcjVGELXGB5uNVyRbxoV3GoInW0aFaw3/HSPv+G685dukWBWJ93Tf5KCKYJfP/If2k547xRmp0BWzPwOuqP4w32HoQlav3Xxwx3bTC7fd+OoG58fdnrHq/1rsQP2hr1jMfvIeuyEnbE+4wzZLfvBfjZ/B42gFbTvWpuNlbPP1iLY/wNaOR+4</latexit> <latexit sha1_base64="0uStd8PEwCtmctIFyVxU/PU2PFc=">AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+fqZrcTdaNlhJtJvEo6bBVnN3vN42RkeKFQE5fg3CCOchqWYElwiYt2UjjMgU9hjAOfalDohuXypovwrSejMDPWH03hkv5vlKCcm6vUdyqgiXtYq2BdbVBQ9mlYCp0XhJrfDcoKGZIJq2eHI2GRk5z7BLgV/q4hn4AFTv5z1qakyr9B41dulAI9KhOwYwWzRZlUU01eJlaFnn2vYCKFEuRqDKFrDA+3Gq7IN40KbjWEzjaNCtYbfrrH33Dd+Uu3SDCrk+7pP0nBFMGvH/kPbSe8dwqzUyArZn4H3VH84b7D0ASt37r44Y5tJpfvu3HUjc8PO73j1f612AF7w96xmH1kPXbCzlifcYbslv1gP5u/g0bQCtp3rc3GytlnaxHs/wFu7B+/</latexit> Activated grid [Koyama+ JASA 2018] <latexit sha1_base64="jt8UZmLAo4CM+kOHoDXBXXv/2js=">AAAD3HicdZPfihMxFMbTjn/WqmtXLxS8CRZhxW6ZqYLeCGX1QoSFFexuodMdzqSZNnSSmU0y2hrHC/FOvPXdfAMfw0zbce22DQx8+XJ+nDMnJ2EaM6Vd93el6ly5eu36zo3azVu3d+/U9+6eqCSThHZJEieyF4KiMRO0q5mOaS+VFHgY09Nw8ro4P/1IpWKJ+KBnKR1wGAkWMQLaWkH9m8+ZYJx9poHxOehxGJlenmM/kkCMl5u21SrjgdHNyMovZdAsxwe43LzJSzUtYoL2WRs/xX5s6xjCIopAbN7lgUmb583zfP8i15Og3nBb7nzhdeEtRQMt13GwVz30hwnJOBWaxKBU33NTPTAgNSMxzWt+pmgKZAIj2rdSAKdqYObNyvFj6wxxlEj7CY3n7v+EAa7UjIc2sqhRXT4rzE1n/UxHLweGiTTTVJBFoiiLsU5w0Xk8ZJISHc+sACKZrRWTMdg2a3s/K1lCbv9B0E8k4RzE0PggRxym+eKGktT4kmPrfS1MP7bXp9UGgokNhDW3EipL14nC3EowEa0ThbmZKGdtlfnnboFgugkq3QuIw4SCfQHaNrTmk84RTI9ASza1z0C98p6VEYkeU2mnzrs8Y+vipN3y3Jb3/nmjc7icvx30ED1C+8hDL1AHvUXHqIsI+lPZrdyvPHDOnO/OD+fnIrRaWTL30Mpyfv0FTtFRkg==</latexit> <latexit sha1_base64="XMTa05C+9ds+LrBKIVtxIUVcJmU=">AAADbXicdZLNihNBEIB7M/6s8S/r4kmRxiCulzCjgh4UwnrxsrCC2V3IhFDTqUma9J/dPZo4xFfxqo/kU/gK9mSzajaThhmKr+qjuovKjODOx/GvnUZ05eq167s3mjdv3b5zt7V378TpwjLsMS20PcvAoeAKe557gWfGIshM4Gk2fVflTz+jdVyrj35ucCBhrHjOGfiAhq39g5i+oYamAumn5T95Nmy14068PHQzSFZBm6zO8XCvcZiONCskKs8EONdPYuMHJVjPmcBFMy0cGmBTGGM/hAokukG5vP2CPglkRHNtw6c8XdL/jRKkc3OZhUoJfuIu5ypYl+sXPn89KLkyhUfFzhvlhaBe02oUdMQtMi/mIQBmebgrZROwwHwY2FqXTIY3KPzCtJSgRmUKdixhtijTqqs2ZWolDexbBVPBJfeuxuCqxghwq+EKs2lUcKvBVb5pVLDeCN0D/orrzl+6RYJZnXRB/0kSpghhJX0YaDNl3SOYHYG3fBb20r1NXlxUaD9BG7Yuubxjm8HJ804Sd5IPL9vdw9X+7ZIH5DE5IAl5RbrkPTkmPcLInHwnP8jPxu/ofvQwenRe2thZOftk7URP/wA0wyUg</latexit> <latexit sha1_base64="jNNevYceU70kmgc9X62mIHeNJOc=">AAADwHicdZLdbtMwFIC9hp9R/jq45MaiQmqlqkoACW4mVeUGIU0aEt0qNV114jqtVdtJbAfaeeEJeQFeg1u4wGk7RtfWUqST75xPJz45UcqZNr7/86Di3bl77/7hg+rDR4+fPK0dPTvTSa4I7ZGEJ6ofgaacSdozzHDaTxUFEXF6Hs0+lPnzr1RplsgvZpHSoYCJZDEjYBwa1eJQgJkS4PZTMbJpK2tlRWPJotj2iyY+xqHOxUjikNPYNFZv1rTiAl/d1DVky6Hm1UWGQ8UmU9O8sGGsgNi0sFlRjGp1v+0vD94OgnVQR+tzOjqqdMNxQnJBpSEctB4EfmqGFpRhhNOiGuaapkBmMKEDF0oQVA/tciAFfuXIGMeJco80eEn/NywIrRcicpXlFfTtXAl35Qa5id8PLZNpbqgkq0ZxzrFJcDldPGaKEsMXLgCimPtWTKbg5mDcP9joEgl3B0m/kUQIkGMbgpoImBd2OdQktaES2LHvJQw5E8zoHQaTOwwH9xo6T7eNEu41mIy3jRLuNlx3hy/ppvOP7pFgvku6pjeSgBkFt+XGDbQaks4JzE/AKDZ3q66PgzfXFYmZUuW2Lri9Y9vB2et24LeDz2/rne56/w7RC/QSNVCA3qEO+ohOUQ8R9AP9Qr/RH6/rTb3Ey1allYO18xxtHO/yL3fVSK4=</latexit> Multidimensional mixed-norm penalty ( -norm) <latexit sha1_base64="UO/CtIGxHZlCwhDll94dyV4o5hI=">AAADZHicdZJfaxNBEMC3Of/UWLW1+CTIYhB8KOGuCvVFCPXFl0IF0xZyIcxt5pIl+6+7ezXxiB/DV/1afgE/h3tpqqa5DBwMv5kfszdMZgR3Po5/bTWiO3fv3d9+0Hy48+jxk929p2dOF5Zhl2mh7UUGDgVX2PXcC7wwFkFmAs+zyYeqfn6F1nGtPvuZwb6EkeI5Z+ADSlMUYlCag8uDy/lgtxW340XQ9SRZJi2yjNPBXuM4HWpWSFSeCXCul8TG90uwnjOB82ZaODTAJjDCXkgVSHT9cvHoOX0VyJDm2oZPebqg/xslSOdmMgudEvzY3a5VsK7WK3z+rl9yZQqPil0PygtBvabVBuiQW2RezEICzPLwVsrGYIH5sKeVKZkM/6DwC9NSghqWKdiRhOm8TKup2pSplTSwbxVMBZfcuxqDqxojwI2GK8y6UcGNBlf5ulHBeiNMD/grrjp/6QYJpnXSDf0nSZgghEv0YaHNlHVOYHoC3vJpOEf3Pnlz06H9GG24uuT2ja0nZ4ftJG4nn962OsfL+9smz8lL8pok5Ih0yEdySrqEEUO+kx/kZ+N3tBPtR8+uWxtbS2efrET04g+IbSRa</latexit> [Murata+ IEEE TSP 2018]
12. Application to Binaural Reproduction April 1, 2024 12 Conversion into binaural sounds Ø Binaural reproduction in real world is difficult, compared to binaural synthesis in VR space Ø Binaural reproduction from recordings of multiple small arrays instead of single spherical array Ø Broad listening area by using flexible and scalable recording system Binaural reproduction from mic array recordings for VR audio Recording Reproduction [Iijima+ JASA 2021]
13. Application to Binaural Reproduction Ø Recording system using multiple Ambisonic mics and 360-degree cameras April 1, 2024 13 Small mic arrays (Ambisonic mics) 360-degree cameras Demo Proposed Single array [Iijima+ IEEE WASPAA 2021 (demo)] Error distribution
14. SOUND FIELD SYNTHESIS April 1, 2024 14
15. Sound Field Synthesis Ø Optimization problem to obtain loudspeaker driving signals April 1, 2024 15 Synthesizing desired pressure field w/ multiple loudspeakers Loudspeaker Target region: Synthesized sound field Desired sound field Difficult to solve owing to regional integration Driving signal Transfer function
16. Conventional Pressure Matching Ø Driving signals are obtained as simple least-squares solution April 1, 2024 16 : Target region Secondary source Control points : Target region Secondary source Discretization of target region into control points Discrete approxmation Transfer function matrix Driving signal vector Desired pressure vector <latexit sha1_base64="i9WfaOjLrhWO59r+nVt9MC7/SLM=">AAADEHicfZHLbhMxFIY90wIlXJrCko1FhFRu0QyqgA1SBUiUBaJIpK0Up5HHOTOxantGtgcRLL8AS56GHWLLmg2vwgrPJEH0xpEsff7Pf3xsn6wS3Ngk+RXFK6sXLl5au9y5cvXa9fXuxo09U9aawYCVotQHGTUguIKB5VbAQaWBykzAfnb0osnvfwBteKne21kFI0kLxXPOqA3SuKtIJt3E42eYCMjtJm72r/yhI5LaqcndjvdzCd/HBCxtDa89JpoXU3v30D1M/VlFrVYvNR2agPF+3O0l/aQNfBrSBfTQInbHG9FnMilZLUFZJqgxwzSp7MhRbTkT4DukNlBRdkQLGAZUVIIZufZjPL4TlAnOSx2WsrhV/61wVBozk1lwtjc/mWvEs3LD2uZPR46rqrag2LxRXgtsS9z8Mp5wDcyKWQDKNA93xWxKNWU2zOJYl0z6Toe8hPA4DW9Co7cVaGpLfc8RqgtJP/rw2II8aOh/Rq6WxkDnGcMhXPJP4N1fOtfK1dK6pDC99OSsTsPeo376uL/1bqu3/XwxxzV0C91GmyhFT9A22kG7aIAY+ol+RyvRavwl/hp/i7/PrXG0qLmJjkX84w9/cP+K</latexit> d = GHG + ⌘I 1 GHudes
17. Weighted Pressure Matching Ø Original cost function is approximated as Ø Driving signals are obtained as weighted least squares solution April 1, 2024 17 Pressure matching for continuous region based on kernel interpolation of sound field <latexit sha1_base64="wBoAciZ2FK9Swr355ix7ob1QUvE=">AAADEnicfZHLbtNAFIYn5lbCLYUlmxERUkEosqsK2CBVLQs2qEVqmoo4RMeTY3dUz9iaGVdNR/MWPADPwQrElhfgBdjCIzB2EkR64UiWP/3nn9t/kjLn2oThj1Zw5eq16zdWbrZv3b5z915n9f6+LirFsM+KvFAHCWjMucS+4SbHg1IhiCTHQXK0XfcHx6g0L+SemZY4EpBJnnIGxkvjzvs4EXbg6Csac2nGNt4RmIGjtXzq1uqfck8+2Bi0uUAVYA51avec79WshJ3MbMqNO92wFzZFz0M0hy6Z1+54tfUpnhSsEigNy0HrYRSWZmRBGc5ydO240lgCO4IMhx4lCNQj24Tg6GOvTGhaKP9JQxv13xUWhNZTkXhnc+uzvVq8qDesTPpyZLksK4OSzQ5Kq5yagtaJ0glXyEw+9QBMcX9Xyg5BATM+96VTErH0BtskplPt2u34NfonK3zrpZ0SFZhCPfWpq0zAifMRZPGzmv5n5HJh9HSZ0W/CBT9FZ//SpVYuF9YF+ZlGZyd4HvbXe9Hz3sa7je7m1ny6K+QheUTWSERekE3yhuySPmHkC/lJfpHfwcfgc/A1+DazBq35mgdkqYLvfwAs/wN9</latexit> W = Z ⌦ z(r)⇤z(r)Tdr <latexit sha1_base64="ots8Wmdw3c6kEAqnCoYqjjAHV2E=">AAAD0HiclVJbb9MwFHYWLqNctsEjL9YqpgJb1UwT8II0ARIDhDbQuk6au8pxT1KrcRLZDmrJLMQjvPM7eIWfwr/BSdOKbhSJIyX+9J3v3HzspxFXutX65Sy5ly5fubp8rXb9xs1bK6trt49UkkkGbZZEiTz2qYKIx9DWXEdwnEqgwo+g4w+fF/7OB5CKJ/GhHqfQFTSMecAZ1ZbqrTnrr/EGoWkqkxEmPNa9nOwLCKnBJIJAn2Hii5wMrYKaRoGluX+aE0H1QAX5oal0jVL3xjwkI16gV5aXPBxoK97y5kQvTfHvm63iyEyVTFoKlJmFVefZ6Tae+c2kPiaktvG0SllmXJRs1sK0372igFV2zH+E91brrWarNHwReBWoo8oO7K1+I/2EZQJizSKq1InXSnU3p1JzFoGpkUxBStmQhnBiYUwFqG5ebtPge5bp4yCR9os1Ltk/I3IqlBoL3yrLqc77CvJvvpNMB0+6OY/TTEPMJoWCLMI6wcXTwH0ugelobAFlktteMRtQSZm2D6g2V8YXm+VFqUAZ63kBdkwJby21n4KkOpEPckJlKOjI2LFDslmgfwl5PBVatEhok3DBP4LJZ2ihlMdT6RTV7CK982u7CI62m96j5s67nfrus2qly+guWkcN5KHHaBftoQPURsz54nx3fjg/3ffuyP3kfp5Il5wq5g6aM/frbwuZR80=</latexit> J ⇡ Z ⌦ (r)T (K + ⇠I) 1 Gd udes 2 dr = Gd udes H W Gd udes <latexit sha1_base64="UqfxQe3UGsqqnmcvxH4VBmwhEN8=">AAAC/XicfZFdaxNBFIYn61eNH03rpTeDQYhaQ1aKFkEo6oUiYoSmLWRjODs5uxkyM7vMzErTZfBX+BO8Ey/1twje6u9wdpuIba0HBh7e952PMyfOBTe21/veCM6dv3Dx0srl5pWr166vttbWd01WaIYDlolM78dgUHCFA8utwP1cI8hY4F48e1b5e+9RG56pHTvPcSQhVTzhDKyXxq1+FMvy0NHHT2hF0QzyHFynYu3uvCsjCXZqknLHORoJTGynzr1y92h0wCt86Q3N06n16fuhG7favW6vLnoawgW0yaL647XGx2iSsUKiskyAMcOwl9tRCdpyJtA1o8JgDmwGKQ49KpBoRmXduqO3vTKhSab9UpbW6t87SpDGzGXsk3UnJ71K/Jc3LGyyNSq5yguLih1dlBSC2oxW/0gnXCOzYu4BmOb+rZRNQQOz/rebx66J5UZ1vjaJcd55jr5Nja+99CZHDTbTd8sIdCrhwPm202ijov8FuVoGPZ0V9IdwyQ/RlX/ozChXy+iSmn6Q4cmxnYbdB93wYXfz7WZ7++lipCvkJrlFOiQkj8g2eUH6ZEAY+Up+kJ/kV/Ah+BR8Dr4cRYPGYs8NcqyCb78Bcfj4Dw==</latexit> z := (r)T (K + ⇠I) 1 Kernel ridge regression <latexit sha1_base64="CRJvQ0OuNgbbSk0jTZw5dDlZAo8=">AAADxHiclVJbaxNBFJ7teqnx0ovgiy+DQanahqyU6otQrNgKihVMU8ikYXZyNhk6M7vMzIpxGf+Fv8BX/UH+G2e3G0nbRPDAwrff+c5tzokzwY1tt38HS+GVq9euL99o3Lx1+87K6tr6kUlzzaDDUpHq45gaEFxBx3Ir4DjTQGUsoBuf7pX+7mfQhqfqk51k0Jd0pHjCGbWeGqwF98iY2oLEshg6hx+9xITqkeRqUHOEK0wkteM4LvbcyTuHiYDEbuDSvT90eKtCuTspKpn2QWB8KqL5aGwf17RJioOS9NLu/6bAhDTKxmaC3Jy0+w4/xQQsLf/ezjSwFbnFUW5+7cFqs91qV4Yvg6gGTVTboX/J72SYslyCskxQY3pRO7P9gmrLmQDXILmBjLJTOoKeh4pKMP2i2qDDDz0zxEmq/acsrtjZiIJKYyYy9spqgIu+kpzn6+U2edEvuMpyC4qdFUpygW2Ky3PAQ66BWTHxgDLNfa+YjammzPqjaZwrE8vN6oVMYpz3vAY/pob3nvqQgaY21U+K6njoF+fHHpHNEv1LyNVU6NEioU/CJf8KrviLFkq5mkqnqOEXGV1c22Vw9KwV7bS2P243d1/VK11G99EDtIEi9BztogN0iDqIBS74EfwMfoVvQhGaMD+TLgV1zF10zsJvfwD8G0Dj</latexit> ˆ d = arg min d2CL Gd udes H W Gd udes = GHW G + ⌘I 1 GHW udes [Koyama+ JAES 2023]
18. Weighted Pressure Matching Ø Comparison between Pressure Matching and Weighted Pressure Matching April 1, 2024 18 PM WPM (uniform) WPM (directional) Pressure Error [Koyama+ JAES 2023]
19. Amplitude Matching for Multizone Sound Field Control Ø Problem to be solved in amplitude matching April 1, 2024 19 Target region Generating multiple personal sound zones by using loudspeakers Desired amplitude No closed form solution, but majorization minimization (MM) algorithm or alternating direction method of multipliers (ADMM) can be applied Element-wise absolute value [Abe+ IEEE/ACM TASLP 2023]
20. Amplitude Matching for Multizone Sound Field Control April 1, 2024 20 https://youtu.be/oYw7kmpZcY4 Full version:
21. Application to Spatial Active Noise Control Ø Environmental noise is still unsolved problem Ø Active noise control (ANC) is aimed to cancel noise by loudspeaker signals, but its effect is limited to local region Ø ANC in 3D space based on sound field analysis/synthesis April 1, 2024 21 Noise suppression by loudspeaker signals Quiet zone
22. Application to Spatial Active Noise Control Ø Cost function of regional noise power is estimated by kernel interpolation of sound field Ø Adaptive filtering algorithm based on kernel interpolation is also derived April 1, 2024 22 ANC in 3D space based on sound field interpolation Ø Conventional cost function Ø Proposed cost function <latexit sha1_base64="Pnlxe5THpW3gUCE+pMJvaUIbsmQ=">AAACFXicbZDLSsNAFIYnXmu9RV2KMFgUVyUpRd0IRTcuXFSwF2himUwn7dCZJMxMhBKz8iF8Bre6diduXbv0TZy0WdjWHwY+/nMO58zvRYxKZVnfxsLi0vLKamGtuL6xubVt7uw2ZRgLTBo4ZKFoe0gSRgPSUFQx0o4EQdxjpOUNr7J664EIScPgTo0i4nLUD6hPMVLa6poHDkdqgBFLblJ4fAGdR+h4PCGppvtK1yxZZWssOA92DiWQq941f5xeiGNOAoUZkrJjW5FyEyQUxYykRSeWJEJ4iPqkozFAnEg3GX8jhUfa6UE/FPoFCo7dvxMJ4lKOuKc7s6PlbC0z/6t1YuWfuwkNoliRAE8W+TGDKoRZJrBHBcGKjTQgLKi+FeIBEggrndzUFo+nOhN7NoF5aFbK9mm5elst1S7zdApgHxyCE2CDM1AD16AOGgCDJ/ACXsGb8Wy8Gx/G56R1wchn9sCUjK9f5U+ekQ==</latexit> L = kek2 : Power of error mics : Regional noise power <latexit sha1_base64="pm/hhsXKsYUZ/lWoZalEg8K5t2U=">AAACNHicbVDLSgMxFM34rPVVdekmWJS6KTOlqAuFohsXghXsAzq1ZNK0DU0yQ5IRynR+xo/wG9zqTnAj4tZvMNN2YVsPBA7nnMu9OV7AqNK2/W4tLC4tr6ym1tLrG5tb25md3aryQ4lJBfvMl3UPKcKoIBVNNSP1QBLEPUZqXv8q8WuPRCrqi3s9CEiTo66gHYqRNlIrc+5ypHsYsegmhkcX0KVCtyL3lpMuiuEwzLkej2R8PHwowFFU8qgdj0XYymTtvD0CnCfOhGTBBOVW5tNt+zjkRGjMkFINxw50M0JSU8xInHZDRQKE+6hLGoYKxIlqRqNfxvDQKG3Y8aV5QsOR+nciQlypAfdMMjlUzXqJ+J/XCHXnrBlREYSaCDxe1AkZ1D5MKoNtKgnWbGAIwpKaWyHuIYmwNsVObfF4bDpxZhuYJ9VC3jnJF++K2dLlpJ0U2AcHIAcccApK4BqUQQVg8ARewCt4s56tD+vL+h5HF6zJzB6YgvXzC9DNrBs=</latexit> L = Z ⌦ |u(r)|2dr [Ito+ IEEE ICASSP 2019 (Best Student Paper Award), Koyama+ IEEE/ACM TASLP 2021]
23. Application to Spatial Active Noise Control Ø Band-limited noise (500-800Hz), T60 : 240ms April 1, 2024 23 Proposed: -10.5 dB MPC: -6.2 dB (dB) Regional noise reduction is achieved by the proposed method
24. Summary Ø Recent research topics – Wavefield-informed DNN for sound field estimation, Spatial active noise control, Amplitude matching for generating personal sound zones, DNN-based HRTF interpolation, Source and sensor placement for sound field control, Region-to-region interpolation of acoustic transfer functions Ø Keywords – Kernel methods, Gaussian process, Reproducing kernel Hilbert space, Sparse modeling, Deep neural network, Physics-informed machine learning, Adaptive filter, Convex optimization, Physical acoustics, Partial differential equation April 1, 2024 24