Introduction of NII Shoichi Koyama's Lab (FY2023)

Transcript

1. Introduction of Our Lab
   Audio Processing Research Group
   FY2023
   Shoichi Koyama, Ph. D.
   Digital Content and Media Sciences Research Division,
   National Institute of Informatics
   

   View Slide

2. April 10, 2023 2
   Basic Technologies of
   Sound Field Analysis
   and Synthesis
   VR/AR audio
   Active noise control
   Local-field recording
   and reproduction
   Signal enhancement
   Visualization/auralization
   Room acoustic analysis
   Summary
   Sound field analysis/synthesis and its applications
   

   View Slide

3. What is sound field analysis/synthesis?
   April 10, 2023 3
   Estimating sound field inside
   target region using multiple mics
   Synthesizing desired sound field
   inside target region using multiple
   loudspeakers
   Wavefield-informed signal processing and machine learning
   for sound field analysis and synthesis
   Analysis Synthesis
   Microphone
   Loudspeaker
   

   View Slide

4. Basic Technologies
   April 10, 2023 4
   Analysis Synthesis
   Microphone Loudspeaker
   • Kernel interpolation with constraint of Helmholtz eq
   • Sparsity-based super-resolution
   • Analysis based on Reciprocity Gap Functional
   [Ueno+ IEEE SPL 2018, IEEE TSP 2021]
   [Murata+ IEEE TSP 2018, Koyama+ JASA 2018, IEEE JSTSP 2019]
   [Takida+ Signal Process 2019]
   Wavefield-informed signal processing and machine learning
   

   View Slide

5. Analysis Synthesis
   Microphone Loudspeaker
   Basic Technologies
   April 10, 2023 5
   • Weighted pressure and mode matching for sound field control
   • Optimization of source and sensor placement
   • Amplitude matching for multizone control
   [Ueno+ IEEE/ACM TASLP 2019, Koyama+ JAES 2023]
   [Koyama+ IEEE/ACM TASLP 2020, Nishida+ IEEE TSP 2022]
   [Koyama+ IEEE ICASSP 2021, Abe+ IEEE/ACM TASLP 2023]
   Enhancing flexibility and scalability to make
   the range of applications broader
   Wavefield-informed signal processing and machine learning
   

   View Slide

6. SOUND FIELD ANALYSIS
   April 10, 2023 6
   

   View Slide

7. Kernel Interpolation of Sound Field
   Ø Kernel interpolation with constraint of Helmholtz eq
   – Estimated function should satisfy governing equation of acoustic field
   – Derived kernel function to constraint solution of kernel ridge regression satisfying
   Helmholtz eq
   April 10, 2023 7
   Estimate continuous sound field from discrete mics
   Target region:
   Microphone
   AAACHXicbVDLSgNBEJyNrxhfUY9eBkMgQQm7IagXIagHjxHMA7IxzE46yZCZ3WVmVghLvsCP8Bu86tmbeBWP/omTx8EkFjQUVd10d3khZ0rb9reVWFldW99Ibqa2tnd299L7BzUVRJJClQY8kA2PKODMh6pmmkMjlECEx6HuDa7Hfv0RpGKBf6+HIbQE6fmsyyjRRmqnszns3gDXBJ/gwUMR53GUcz0Ry9GpGwjokTy+xHY7nbEL9gR4mTgzkkEzVNrpH7cT0EiAryknSjUdO9StmEjNKIdRyo0UhIQOSA+ahvpEgGrFk3dGOGuUDu4G0pSv8UT9OxETodRQeKZTEN1Xi95Y/M9rRrp70YqZH0YafDpd1I041gEeZ4M7TALVfGgIoZKZWzHtE0moNgnObfHEyGTiLCawTGrFgnNWKN2VMuWrWTpJdISOUQ456ByV0S2qoCqi6Am9oFf0Zj1b79aH9TltTVizmUM0B+vrF4oXn7I=
   ( + k2)u(r, !) = 0
   [Ueno+ IEEE SPL 2018, IEEE TSP 2021, Koyama+ IEEE ICASSP 2022 Tutorial]
   Kernel function:
   Helmholtz eq:
   

   View Slide

8. Kernel Interpolation of Sound Field
   Ø Experimental results using real data from MeshRIR data set
   – Reconstructing pulse signal from single loudspeaker w/ 18 mic
   April 10, 2023 8
   True Proposed Gaussian kernel
   (Black dots indicate mic positions)
   Impulse response measurement system
   [Koyama+ 2021]
   

   View Slide

9. Sparse Sound Field Decomposition
   Ø Sound field inside region including sources satisfies inhomogeneous
   Helmholtz eq
   April 10, 2023 9
   Sound field estimation inside region including sources
   Target region:
   Microphone
   AAADi3icdZJfaxNBEMC3Of/UaG2qj74sBiFFDXepoIiFUBF8KbRg2kIuDXObvWS53b1jd04Tj/g1/DS+6mfw27iXpmqay8DC8Jv9MbvDRJkUFn3/91bNu3X7zt3te/X7D3Ye7jb2Hp3ZNDeM91gqU3MRgeVSaN5DgZJfZIaDiiQ/j5L3Zf38MzdWpPoTzjI+UDDWIhYM0KFhI2iFGiIJl53nyWVnn+atUAFOorgw8xfJPj2kL09X0bDR9Nv+Iuh6EiyTJlnGyXCvdhSOUpYrrpFJsLYf+BkOCjAomOTzephbngFLYMz7LtWguB0Ui7/N6TNHRjROjTsa6YL+bxSgrJ2pyN0sX2lv1kpYVevnGL8ZFEJnOXLNrhrFuaSY0nJQdCQMZyhnLgFmhHsrZRMwwNCNc6VLpNwfNP/CUqVAj4oQzFjBdF4s5pZmRWgUdexbCUMplEBbYQhdYTi40bB5tm6UcKMhdLxulLDacN0d/spXnb90gwTTKuma/pMUJBzcwqIbaD1k3WOYHgMaMXVbaw+Dg+sbKU64cVsX3Nyx9eSs0w78dnD6qtk9Wu7fNnlCnpIWCchr0iUfyQnpEUa+kx/kJ/nl7XgH3lvv3dXV2tbSeUxWwvvwB0omMJU=
   Unknown boundary condition on room surface
   Source distribution
   Prior information on is necessary to estimate AAACvHicfZFdSyMxFIbT0fWjrrbqpTeDZUGllJmlrHslol54IypYFTqlnEnP1NAkMyQZ2TrMz/DKW/1R+282U1vRVvdA4OE9b5LzESacaeN5f0vO3Py3hcWl5fLK99W1SnV941rHqaLYojGP1W0IGjmT2DLMcLxNFIIIOd6Eg+Mif3OPSrNYXplhgh0BfckiRsFYqVutpDuBAHMXRpnK64PdbrXmNbxRuLPgj6FGxnHRXS89Br2YpgKloRy0bvteYjoZKMMox7wcpBoToAPoY9uiBIG6k40qz90fVum5UazskcYdqe9vZCC0HorQOosq9XSuED/LtVMT/e5kTCapQUlfP4pS7prYLcbg9phCavjQAlDFbK0uvQMF1NhhlT98E4p68b7Skc5t5gRtmwrPrHSeoAITq70sANUX8Ce3bfeDekH/MzI5MVr6ymgfYYI9YJ690ZdWJifWCZXtIv3ptc3C9c+G/6vRvGzWDo/GK10iW2Sb7BCf7JNDckouSItQkpIn8kxenAOn5wwc8Wp1SuM7m+RDOPf/AJeG3Ys=
   u(r, k)
   AAACvHicfZFda9swFIYVdx9Zti7Jdrkbs1DoSgh2CduuSuh2sZuxBJamEIdwrBynwpJsJDk0M/4Zvert9qP2bybnY6zJ2gOCh/e8ks5HmHKmjef9rjgHjx4/eVp9Vnv+4vBlvdF8daGTTFEc0oQn6jIEjZxJHBpmOF6mCkGEHEdh/KnMjxaoNEvkd7NMcSJgLlnEKBgrTRv1wXEgwFyFUa6Kdvxu2mh5HW8V7j74G2iRTfSnzcpNMEtoJlAaykHrse+lZpKDMoxyLGpBpjEFGsMcxxYlCNSTfFV54R5ZZeZGibJHGnel/nsjB6H1UoTWWVapd3Ol+L/cODPRx0nOZJoZlHT9UZRx1yRuOQZ3xhRSw5cWgCpma3XpFSigxg6rduebULTL95WOdGEzn9G2qfCrlb6lqMAk6iQPQM0FXBe27XnQLukhI5Nbo6X7jPYRJtgPLPK/dK+Vya11SzW7SH93bftwcdrx33e6g26rd75ZaZW8IW/JMfHJB9IjX0ifDAklGbklP8kv58yZObEj1lansrnzmtwJZ/EHQ27dZw==
   Q(r, k)
   

   View Slide

10. Sparse Sound Field Decomposition
    Ø Represebtation by sum of particular and homogeneous solution
    Ø Approximation by linear eq
    April 10, 2023 10
    [Koyama+ JASA 2018]
    Free-field Green s func
    AAADiHicdZLdahNBFICnWX9q/Ev10pvBIAhC2FWhelEI0QtvCi00bSEbwtnJ2WTI/Cwzs7rpsj6FT+OtvoRv42yaVNMkBxbOfmc+zszhJJng1oXhn71GcOfuvfv7D5oPHz1+8rR18Ozc6tww7DMttLlMwKLgCvuOO4GXmUGQicCLZPaprl98RWO5VmdunuFQwkTxlDNwHo1anViCmyZpOa/oEV39fK5WWVHRNzf4qhq12mEnXATdTKJl0ibLOBkdNHrxWLNconJMgLWDKMzcsATjOBNYNePcYgZsBhMc+FSBRDssFw+r6CtPxjTVxn/K0QX93yhBWjuXiT9ZX9HertVwW22Qu/TDsOQqyx0qdt0ozQV1mtZTomNukDkx9wkww/1dKZuCAeb8LNe6JNK/QeE3pqUENS5jMBMJRVUuhqazMjaSeva9hrHgkju7xeBqi+HhTsPm2aZRw50GV+mmUcPthu/u8RWuOzd0hwTFNmlF/0kSZgh+W50faDNm3WMojsEZXviVtUfRu9UJ7aZo/NZFt3dsMzl/24nCTnT6vt3tLfdvn7wgL8lrEpFD0iVfyAnpE0Z+kJ/kF/kdNIMwOAw+Xh9t7C2d52Qtgt5fS0ox9Q==
    Direct component Reverberant component
    Dictionary matrix of Green s func
    Observation vector
    Direct component
    Reverberant component
    Grid point:
    AAADZHicdZLdahNREIBPs2prrNofvBJkMQhehd1aqDdCqDfeFCqYtpANYfZkNjnk/CznzNbEJX0Mb/W1fAGfw7NpqqbZDCwM35yPmR0mzaVwFEW/thrBg4ePtnceN5/sPn32fG//4MKZwnLsciONvUrBoRQauyRI4lVuEVQq8TKdfKzql9donTD6C81y7CsYaZEJDuRRkiigcZqVdj7Qg71W1I4WEa4n8TJpsWWcD/Ybp8nQ8EKhJi7BuV4c5dQvwZLgEufNpHCYA5/ACHs+1aDQ9cvF0PPwjSfDMDPWf5rCBf3fKEE5N1Opf1kN6e7XKlhX6xWUve+XQucFoea3jbJChmTCagPhUFjkJGc+AW6FnzXkY7DAye9ppUuq/D9o/MqNUqCHZQJ2pGA6LxdrM3mZWBV6dlPBRAolyNUYQtcYHm40XJGvGxXcaAidrRsVrDd8d4+/4arzl26QYFon3dF/koIJgr9E8gttJrxzBtMzICum/hzdh/jd3QtDY7T+6uL7N7aeXBy146gdfz5udU6X97fDXrLX7C2L2QnrsE/snHUZZzn7zn6wn43fwW5wGLy4fdrYWjqHbCWCV38AxlYkxQ==
    

    View Slide

11. Sparse Sound Field Decomposition
    Ø Assuming spatial sparsity of source distribution
    Ø Optimization algorithm considering multidimensional sparsity
    April 10, 2023 11
    Sparse source signal in time-freq
    domain
    AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+f6ZrcTdaNlhJtJvEo6bBVnN3vN42RkeKFQE5fg3CCOchqWYElwiYt2UjjMgU9hjAOfalDohuXypovwrSejMDPWH03hkv5vlKCcm6vUdyqgiXtYq2BdbVBQ9mlYCp0XhJrfDcoKGZIJq2eHI2GRk5z7BLgV/q4hn4AFTv5z1qakyr9B41dulAI9KhOwYwWzRZlUU01eJlaFnn2vYCKFEuRqDKFrDA+3Gq7IN40KbjWEzjaNCtYbfrrH33Dd+Uu3SDCrk+7pP0nBFMGvH/kPbSe8dwqzUyArZn4H3VH84b7D0ASt37r44Y5tJpfvu3HUjc8PO73j1f612AF7w96xmH1kPXbCzlifcYbslv1gP5u/g0bQCtp3rc3GytlnaxHs/wFx4R/A
    AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+d0s9uJutEyws0kXiUdtoqzm73mcTIyvFCoiUtwbhBHOQ1LsCS4xEU7KRzmwKcwxoFPNSh0w3J500X41pNRmBnrj6ZwSf83SlDOzVXqOxXQxD2sVbCuNigo+zQshc4LQs3vBmWFDMmE1bPDkbDISc59AtwKf9eQT8ACJ/85a1NS5d+g8Ss3SoEelQnYsYLZokyqqSYvE6tCz75XMJFCCXI1htA1hodbDVfkm0YFtxpCZ5tGBesNP93jb7ju/KVbJJjVSff0n6RgiuDXj/yHthPeO4XZKZAVM7+D7ij+cN9haILWb138cMc2k8v33TjqxueHnd7xav9a7IC9Ye9YzD6yHjthZ6zPOEN2y36wn83fQSNoBe271mZj5eyztQj2/wCDnx/G
    AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+fZzW4n6kbLCDeTeJV02CrObvaax8nI8EKhJi7BuUEc5TQswZLgEhftpHCYA5/CGAc+1aDQDcvlTRfhW09GYWasP5rCJf3fKEE5N1ep71RAE/ewVsG62qCg7NOwFDovCDW/G5QVMiQTVs8OR8IiJzn3CXAr/F1DPgELnPznrE1JlX+Dxq/cKAV6VCZgxwpmizKpppq8TKwKPftewUQKJcjVGELXGB5uNVyRbxoV3GoInW0aFaw3/HSPv+G685dukWBWJ93Tf5KCKYJfP/If2k547xRmp0BWzPwOuqP4w32HoQlav3Xxwx3bTC7fd+OoG58fdnrHq/1rsQP2hr1jMfvIeuyEnbE+4wzZLfvBfjZ/B42gFbTvWpuNlbPP1iLY/wNaOR+4
    AAADWXicdZLNattAEIDXVtu47l/SHHMRNYWejNQG2kvApJdcAgnEScAyYbQe2Yv3R+yOWrvCfYFc24crfZmuHKetY3lgYfhmPmZ3mTSXwlEU/Wo0g0ePn+y0nrafPX/x8tXu3utLZwrLsc+NNPY6BYdSaOyTIInXuUVQqcSrdPq5ql99QeuE0Rc0z3GoYKxFJjiQR+fqZrcTdaNlhJtJvEo6bBVnN3vN42RkeKFQE5fg3CCOchqWYElwiYt2UjjMgU9hjAOfalDohuXypovwrSejMDPWH03hkv5vlKCcm6vUdyqgiXtYq2BdbVBQ9mlYCp0XhJrfDcoKGZIJq2eHI2GRk5z7BLgV/q4hn4AFTv5z1qakyr9B41dulAI9KhOwYwWzRZlUU01eJlaFnn2vYCKFEuRqDKFrDA+3Gq7IN40KbjWEzjaNCtYbfrrH33Dd+Uu3SDCrk+7pP0nBFMGvH/kPbSe8dwqzUyArZn4H3VH84b7D0ASt37r44Y5tJpfvu3HUjc8PO73j1f612AF7w96xmH1kPXbCzlifcYbslv1gP5u/g0bQCtp3rc3GytlnaxHs/wFu7B+/
    Activated grid
    [Koyama+ JASA 2018]
    AAAD3HicdZPfihMxFMbTjn/WqmtXLxS8CRZhxW6ZqYLeCGX1QoSFFexuodMdzqSZNnSSmU0y2hrHC/FOvPXdfAMfw0zbce22DQx8+XJ+nDMnJ2EaM6Vd93el6ly5eu36zo3azVu3d+/U9+6eqCSThHZJEieyF4KiMRO0q5mOaS+VFHgY09Nw8ro4P/1IpWKJ+KBnKR1wGAkWMQLaWkH9m8+ZYJx9poHxOehxGJlenmM/kkCMl5u21SrjgdHNyMovZdAsxwe43LzJSzUtYoL2WRs/xX5s6xjCIopAbN7lgUmb583zfP8i15Og3nBb7nzhdeEtRQMt13GwVz30hwnJOBWaxKBU33NTPTAgNSMxzWt+pmgKZAIj2rdSAKdqYObNyvFj6wxxlEj7CY3n7v+EAa7UjIc2sqhRXT4rzE1n/UxHLweGiTTTVJBFoiiLsU5w0Xk8ZJISHc+sACKZrRWTMdg2a3s/K1lCbv9B0E8k4RzE0PggRxym+eKGktT4kmPrfS1MP7bXp9UGgokNhDW3EipL14nC3EowEa0ThbmZKGdtlfnnboFgugkq3QuIw4SCfQHaNrTmk84RTI9ASza1z0C98p6VEYkeU2mnzrs8Y+vipN3y3Jb3/nmjc7icvx30ED1C+8hDL1AHvUXHqIsI+lPZrdyvPHDOnO/OD+fnIrRaWTL30Mpyfv0FTtFRkg==
    AAADbXicdZLNihNBEIB7M/6s8S/r4kmRxiCulzCjgh4UwnrxsrCC2V3IhFDTqUma9J/dPZo4xFfxqo/kU/gK9mSzajaThhmKr+qjuovKjODOx/GvnUZ05eq167s3mjdv3b5zt7V378TpwjLsMS20PcvAoeAKe557gWfGIshM4Gk2fVflTz+jdVyrj35ucCBhrHjOGfiAhq39g5i+oYamAumn5T95Nmy14068PHQzSFZBm6zO8XCvcZiONCskKs8EONdPYuMHJVjPmcBFMy0cGmBTGGM/hAokukG5vP2CPglkRHNtw6c8XdL/jRKkc3OZhUoJfuIu5ypYl+sXPn89KLkyhUfFzhvlhaBe02oUdMQtMi/mIQBmebgrZROwwHwY2FqXTIY3KPzCtJSgRmUKdixhtijTqqs2ZWolDexbBVPBJfeuxuCqxghwq+EKs2lUcKvBVb5pVLDeCN0D/orrzl+6RYJZnXRB/0kSpghhJX0YaDNl3SOYHYG3fBb20r1NXlxUaD9BG7Yuubxjm8HJ804Sd5IPL9vdw9X+7ZIH5DE5IAl5RbrkPTkmPcLInHwnP8jPxu/ofvQwenRe2thZOftk7URP/wA0wyUg
    AAADwHicdZLdbtMwFIC9hp9R/jq45MaiQmqlqkoACW4mVeUGIU0aEt0qNV114jqtVdtJbAfaeeEJeQFeg1u4wGk7RtfWUqST75xPJz45UcqZNr7/86Di3bl77/7hg+rDR4+fPK0dPTvTSa4I7ZGEJ6ofgaacSdozzHDaTxUFEXF6Hs0+lPnzr1RplsgvZpHSoYCJZDEjYBwa1eJQgJkS4PZTMbJpK2tlRWPJotj2iyY+xqHOxUjikNPYNFZv1rTiAl/d1DVky6Hm1UWGQ8UmU9O8sGGsgNi0sFlRjGp1v+0vD94OgnVQR+tzOjqqdMNxQnJBpSEctB4EfmqGFpRhhNOiGuaapkBmMKEDF0oQVA/tciAFfuXIGMeJco80eEn/NywIrRcicpXlFfTtXAl35Qa5id8PLZNpbqgkq0ZxzrFJcDldPGaKEsMXLgCimPtWTKbg5mDcP9joEgl3B0m/kUQIkGMbgpoImBd2OdQktaES2LHvJQw5E8zoHQaTOwwH9xo6T7eNEu41mIy3jRLuNlx3hy/ppvOP7pFgvku6pjeSgBkFt+XGDbQaks4JzE/AKDZ3q66PgzfXFYmZUuW2Lri9Y9vB2et24LeDz2/rne56/w7RC/QSNVCA3qEO+ohOUQ8R9AP9Qr/RH6/rTb3Ey1allYO18xxtHO/yL3fVSK4=
    Multidimensional mixed-norm penalty
    ( -norm)
    AAADZHicdZJfaxNBEMC3Of/UWLW1+CTIYhB8KOGuCvVFCPXFl0IF0xZyIcxt5pIl+6+7ezXxiB/DV/1afgE/h3tpqqa5DBwMv5kfszdMZgR3Po5/bTWiO3fv3d9+0Hy48+jxk929p2dOF5Zhl2mh7UUGDgVX2PXcC7wwFkFmAs+zyYeqfn6F1nGtPvuZwb6EkeI5Z+ADSlMUYlCag8uDy/lgtxW340XQ9SRZJi2yjNPBXuM4HWpWSFSeCXCul8TG90uwnjOB82ZaODTAJjDCXkgVSHT9cvHoOX0VyJDm2oZPebqg/xslSOdmMgudEvzY3a5VsK7WK3z+rl9yZQqPil0PygtBvabVBuiQW2RezEICzPLwVsrGYIH5sKeVKZkM/6DwC9NSghqWKdiRhOm8TKup2pSplTSwbxVMBZfcuxqDqxojwI2GK8y6UcGNBlf5ulHBeiNMD/grrjp/6QYJpnXSDf0nSZgghEv0YaHNlHVOYHoC3vJpOEf3Pnlz06H9GG24uuT2ja0nZ4ftJG4nn962OsfL+9smz8lL8pok5Ih0yEdySrqEEUO+kx/kZ+N3tBPtR8+uWxtbS2efrET04g+IbSRa
    [Murata+ IEEE TSP 2018]
    

    View Slide

12. Application to Binaural Reproduction
    April 10, 2023 12
    Conversion into
    binaural sounds
    Ø Binaural reproduction in real world is difficult, compared to binaural
    synthesis in VR space
    Ø Binaural reproduction from recordings of multiple small arrays instead of
    single spherical array
    Ø Broad listening area by using flexible and scalable recording system
    Binaural reproduction from mic array recordings for VR audio
    Recording Reproduction [Iijima+ JASA 2021]
    

    View Slide

13. Application to Binaural Reproduction
    Ø Recording system using multiple Ambisonic mics and 360-degree cameras
    April 10, 2023 13
    Small mic arrays
    (Ambisonic mics)
    360-degree cameras
    Demo
    Proposed Single array
    [Iijima+ IEEE WASPAA 2021 (demo)]
    Error distribution
    

    View Slide

14. SOUND FIELD SYNTHESIS
    April 10, 2023 14
    

    View Slide

15. Sound Field Synthesis
    Ø Optimization problem to obtain loudspeaker driving signals
    April 10, 2023 15
    Synthesizing desired pressure field w/ multiple loudspeakers
    Loudspeaker
    Target region:
    Synthesized sound field Desired sound field
    Difficult to solve owing to regional integration
    Driving signal
    Transfer function
    

    View Slide

16. Conventional Pressure Matching
    Ø Driving signals are obtained as simple least-squares solution
    April 10, 2023 16
    : Target region
    Secondary source
    Control points
    : Target region
    Secondary source
    Discretization of target region into control points
    Discrete
    approxmation
    Transfer function matrix
    Driving signal vector Desired pressure vector
    AAADEHicfZHLbhMxFIY90wIlXJrCko1FhFRu0QyqgA1SBUiUBaJIpK0Up5HHOTOxantGtgcRLL8AS56GHWLLmg2vwgrPJEH0xpEsff7Pf3xsn6wS3Ngk+RXFK6sXLl5au9y5cvXa9fXuxo09U9aawYCVotQHGTUguIKB5VbAQaWBykzAfnb0osnvfwBteKne21kFI0kLxXPOqA3SuKtIJt3E42eYCMjtJm72r/yhI5LaqcndjvdzCd/HBCxtDa89JpoXU3v30D1M/VlFrVYvNR2agPF+3O0l/aQNfBrSBfTQInbHG9FnMilZLUFZJqgxwzSp7MhRbTkT4DukNlBRdkQLGAZUVIIZufZjPL4TlAnOSx2WsrhV/61wVBozk1lwtjc/mWvEs3LD2uZPR46rqrag2LxRXgtsS9z8Mp5wDcyKWQDKNA93xWxKNWU2zOJYl0z6Toe8hPA4DW9Co7cVaGpLfc8RqgtJP/rw2II8aOh/Rq6WxkDnGcMhXPJP4N1fOtfK1dK6pDC99OSsTsPeo376uL/1bqu3/XwxxzV0C91GmyhFT9A22kG7aIAY+ol+RyvRavwl/hp/i7/PrXG0qLmJjkX84w9/cP+K
    d = GHG + ⌘I 1
    GHudes
    

    View Slide

17. Weighted Pressure Matching
    Ø Original cost function is approximated as
    Ø Driving signals are obtained as weighted least squares solution
    April 10, 2023 17
    Pressure matching for continuous region based on kernel interpolation
    of sound field
    AAADEnicfZHLbtNAFIYn5lbCLYUlmxERUkEosqsK2CBVLQs2qEVqmoo4RMeTY3dUz9iaGVdNR/MWPADPwQrElhfgBdjCIzB2EkR64UiWP/3nn9t/kjLn2oThj1Zw5eq16zdWbrZv3b5z915n9f6+LirFsM+KvFAHCWjMucS+4SbHg1IhiCTHQXK0XfcHx6g0L+SemZY4EpBJnnIGxkvjzvs4EXbg6Csac2nGNt4RmIGjtXzq1uqfck8+2Bi0uUAVYA51avec79WshJ3MbMqNO92wFzZFz0M0hy6Z1+54tfUpnhSsEigNy0HrYRSWZmRBGc5ydO240lgCO4IMhx4lCNQj24Tg6GOvTGhaKP9JQxv13xUWhNZTkXhnc+uzvVq8qDesTPpyZLksK4OSzQ5Kq5yagtaJ0glXyEw+9QBMcX9Xyg5BATM+96VTErH0BtskplPt2u34NfonK3zrpZ0SFZhCPfWpq0zAifMRZPGzmv5n5HJh9HSZ0W/CBT9FZ//SpVYuF9YF+ZlGZyd4HvbXe9Hz3sa7je7m1ny6K+QheUTWSERekE3yhuySPmHkC/lJfpHfwcfgc/A1+DazBq35mgdkqYLvfwAs/wN9
    W =
    Z
    ⌦
    z(r)⇤z(r)Tdr
    AAAD0HiclVJbb9MwFHYWLqNctsEjL9YqpgJb1UwT8II0ARIDhDbQuk6au8pxT1KrcRLZDmrJLMQjvPM7eIWfwr/BSdOKbhSJIyX+9J3v3HzspxFXutX65Sy5ly5fubp8rXb9xs1bK6trt49UkkkGbZZEiTz2qYKIx9DWXEdwnEqgwo+g4w+fF/7OB5CKJ/GhHqfQFTSMecAZ1ZbqrTnrr/EGoWkqkxEmPNa9nOwLCKnBJIJAn2Hii5wMrYKaRoGluX+aE0H1QAX5oal0jVL3xjwkI16gV5aXPBxoK97y5kQvTfHvm63iyEyVTFoKlJmFVefZ6Tae+c2kPiaktvG0SllmXJRs1sK0372igFV2zH+E91brrWarNHwReBWoo8oO7K1+I/2EZQJizSKq1InXSnU3p1JzFoGpkUxBStmQhnBiYUwFqG5ebtPge5bp4yCR9os1Ltk/I3IqlBoL3yrLqc77CvJvvpNMB0+6OY/TTEPMJoWCLMI6wcXTwH0ugelobAFlktteMRtQSZm2D6g2V8YXm+VFqUAZ63kBdkwJby21n4KkOpEPckJlKOjI2LFDslmgfwl5PBVatEhok3DBP4LJZ2ihlMdT6RTV7CK982u7CI62m96j5s67nfrus2qly+guWkcN5KHHaBftoQPURsz54nx3fjg/3ffuyP3kfp5Il5wq5g6aM/frbwuZR80=
    J ⇡
    Z
    ⌦
    (r)T (K + ⇠I) 1 Gd udes
    2
    dr
    = Gd udes H W Gd udes
    AAAC/XicfZFdaxNBFIYn61eNH03rpTeDQYhaQ1aKFkEo6oUiYoSmLWRjODs5uxkyM7vMzErTZfBX+BO8Ey/1twje6u9wdpuIba0HBh7e952PMyfOBTe21/veCM6dv3Dx0srl5pWr166vttbWd01WaIYDlolM78dgUHCFA8utwP1cI8hY4F48e1b5e+9RG56pHTvPcSQhVTzhDKyXxq1+FMvy0NHHT2hF0QzyHFynYu3uvCsjCXZqknLHORoJTGynzr1y92h0wCt86Q3N06n16fuhG7favW6vLnoawgW0yaL647XGx2iSsUKiskyAMcOwl9tRCdpyJtA1o8JgDmwGKQ49KpBoRmXduqO3vTKhSab9UpbW6t87SpDGzGXsk3UnJ71K/Jc3LGyyNSq5yguLih1dlBSC2oxW/0gnXCOzYu4BmOb+rZRNQQOz/rebx66J5UZ1vjaJcd55jr5Nja+99CZHDTbTd8sIdCrhwPm202ijov8FuVoGPZ0V9IdwyQ/RlX/ozChXy+iSmn6Q4cmxnYbdB93wYXfz7WZ7++lipCvkJrlFOiQkj8g2eUH6ZEAY+Up+kJ/kV/Ah+BR8Dr4cRYPGYs8NcqyCb78Bcfj4Dw==
    z := (r)T (K + ⇠I) 1
    Kernel ridge regression
    AAADxHiclVJbaxNBFJ7teqnx0ovgiy+DQanahqyU6otQrNgKihVMU8ikYXZyNhk6M7vMzIpxGf+Fv8BX/UH+G2e3G0nbRPDAwrff+c5tzokzwY1tt38HS+GVq9euL99o3Lx1+87K6tr6kUlzzaDDUpHq45gaEFxBx3Ir4DjTQGUsoBuf7pX+7mfQhqfqk51k0Jd0pHjCGbWeGqwF98iY2oLEshg6hx+9xITqkeRqUHOEK0wkteM4LvbcyTuHiYDEbuDSvT90eKtCuTspKpn2QWB8KqL5aGwf17RJioOS9NLu/6bAhDTKxmaC3Jy0+w4/xQQsLf/ezjSwFbnFUW5+7cFqs91qV4Yvg6gGTVTboX/J72SYslyCskxQY3pRO7P9gmrLmQDXILmBjLJTOoKeh4pKMP2i2qDDDz0zxEmq/acsrtjZiIJKYyYy9spqgIu+kpzn6+U2edEvuMpyC4qdFUpygW2Ky3PAQ66BWTHxgDLNfa+YjammzPqjaZwrE8vN6oVMYpz3vAY/pob3nvqQgaY21U+K6njoF+fHHpHNEv1LyNVU6NEioU/CJf8KrviLFkq5mkqnqOEXGV1c22Vw9KwV7bS2P243d1/VK11G99EDtIEi9BztogN0iDqIBS74EfwMfoVvQhGaMD+TLgV1zF10zsJvfwD8G0Dj
    ˆ
    d = arg min
    d2CL
    Gd udes H W Gd udes
    = GHW G + ⌘I 1
    GHW udes
    [Koyama+ JAES 2023]
    

    View Slide

18. Weighted Pressure Matching
    Ø Comparison between Pressure Matching and Weighted Pressure Matching
    April 10, 2023 18
    PM WPM (uniform) WPM (directional)
    Pressure
    Error
    [Koyama+ JAES 2023]
    

    View Slide

19. Amplitude Matching for Multizone Sound Field Control
    Ø Problem to be solved in amplitude matching
    April 10, 2023 19
    Target region
    Generating multiple personal sound zones by using loudspeakers
    Desired amplitude
    No closed form solution, but majorization minimization (MM) algorithm
    or alternating direction method of multipliers (ADMM) can be applied
    Element-wise absolute value
    [Abe+ IEEE/ACM TASLP 2023]
    

    View Slide

20. Amplitude Matching for Multizone Sound Field Control
    April 10, 2023 20
    https://youtu.be/oYw7kmpZcY4
    Full version:
    

    View Slide

21. Application to Spatial Active Noise Control
    Ø Environmental noise is still unsolved problem
    Ø Active noise control (ANC) is aimed to cancel noise by loudspeaker signals,
    but its effect is limited to local region
    Ø ANC in 3D space based on sound field analysis/synthesis
    April 10, 2023 21
    Noise suppression by loudspeaker signals
    Quiet zone
    

    View Slide

22. Application to Spatial Active Noise Control
    Ø Cost function of regional noise power is estimated by kernel interpolation of
    sound field
    Ø Adaptive filtering algorithm based on kernel interpolation is also derived
    April 10, 2023 22
    ANC in 3D space based on sound field interpolation
    Ø Conventional cost function
    Ø Proposed cost function
    AAACFXicbZDLSsNAFIYnXmu9RV2KMFgUVyUpRd0IRTcuXFSwF2himUwn7dCZJMxMhBKz8iF8Bre6diduXbv0TZy0WdjWHwY+/nMO58zvRYxKZVnfxsLi0vLKamGtuL6xubVt7uw2ZRgLTBo4ZKFoe0gSRgPSUFQx0o4EQdxjpOUNr7J664EIScPgTo0i4nLUD6hPMVLa6poHDkdqgBFLblJ4fAGdR+h4PCGppvtK1yxZZWssOA92DiWQq941f5xeiGNOAoUZkrJjW5FyEyQUxYykRSeWJEJ4iPqkozFAnEg3GX8jhUfa6UE/FPoFCo7dvxMJ4lKOuKc7s6PlbC0z/6t1YuWfuwkNoliRAE8W+TGDKoRZJrBHBcGKjTQgLKi+FeIBEggrndzUFo+nOhN7NoF5aFbK9mm5elst1S7zdApgHxyCE2CDM1AD16AOGgCDJ/ACXsGb8Wy8Gx/G56R1wchn9sCUjK9f5U+ekQ==
    L = kek2 : Power of error mics
    : Regional noise power
    AAACNHicbVDLSgMxFM34rPVVdekmWJS6KTOlqAuFohsXghXsAzq1ZNK0DU0yQ5IRynR+xo/wG9zqTnAj4tZvMNN2YVsPBA7nnMu9OV7AqNK2/W4tLC4tr6ym1tLrG5tb25md3aryQ4lJBfvMl3UPKcKoIBVNNSP1QBLEPUZqXv8q8WuPRCrqi3s9CEiTo66gHYqRNlIrc+5ypHsYsegmhkcX0KVCtyL3lpMuiuEwzLkej2R8PHwowFFU8qgdj0XYymTtvD0CnCfOhGTBBOVW5tNt+zjkRGjMkFINxw50M0JSU8xInHZDRQKE+6hLGoYKxIlqRqNfxvDQKG3Y8aV5QsOR+nciQlypAfdMMjlUzXqJ+J/XCHXnrBlREYSaCDxe1AkZ1D5MKoNtKgnWbGAIwpKaWyHuIYmwNsVObfF4bDpxZhuYJ9VC3jnJF++K2dLlpJ0U2AcHIAcccApK4BqUQQVg8ARewCt4s56tD+vL+h5HF6zJzB6YgvXzC9DNrBs=
    L =
    Z
    ⌦
    |u(r)|2dr
    [Ito+ IEEE ICASSP 2019 (Best Student Paper Award), Koyama+ IEEE/ACM TASLP 2021]
    

    View Slide

23. Application to Spatial Active Noise Control
    Ø Band-limited noise (500-800Hz), T60
    : 240ms
    April 10, 2023 23
    Proposed: -10.5 dB
    MPC: -6.2 dB
    (dB)
    Regional noise reduction is achieved by the proposed method
    

    View Slide

24. Summary
    Ø Recent research topics
    – Wavefield-informed DNN for sound field estimation, Spatial active noise control,
    Amplitude matching for generating personal sound zones, DNN-based HRTF
    interpolation, Source and sensor placement for sound field control, Region-to-region
    interpolation of acoustic transfer functions
    Ø Keywords
    – Kernel methods, Gaussian process, Reproducing kernel Hilbert space, Sparse modeling,
    Deep neural network, Physics-informed machine learning, Adaptive filter, Convex
    optimization, Physical acoustics, Partial differential equation
    April 10, 2023 24
    

    View Slide