The performance of a website affects the users of that website in ways that are hard to measure. The lack of responsiveness, lag, jank, and UI lock ups can cause perceptual dissonance that break the user’s flow of thought. A page not responding in the way the user expects, or as fast as the user expects it to, can result in frustration and anger.
From a business point of view, frustrated and angry users are unlikely to convert, and buy items from the site.
Apart from the psychological and emotional effects of bad performance, the factors that result in bad performance can also make sites expensive to use. How much bandwidth does your site require to be downloaded? How much does that bandwidth cost in different parts of the world? How expensive is your site in terms of battery usage?