You didn't list your distro, but I had the same issue on Pop!_OS. I fixed it by modifying the ALSA properties in /usr/share/wireplumber/main.lua.d/50-alsa-config.lua. I'd first try uncommenting headroom and changing it to 1024, then logging out and back in. If that doesn't work, go 2048. If that still doesn't work, then I'd maybe try adjusting period-size.

For me, a value in the ~1200 range for headroom fixed it completely. But it will entirely depend on your sound card/hardware. And note, these values will probably be overwritten with any wireplummer updates.

Also keep in mind, increasing headroom will increase audio latency, so try not to increase it too drastically.

I was running Bazzite on my Deck, which uses pipewire(-pulse?), and it had intermittent audio crackling. I swapped back to SteamOS some time ago, which I believe uses just PulseAudio, and the audio crackling is gone. I suspect either there's something funky going on with the default config, or there's something wrong with pipewire-pulse itself.

Either way, hopefully someone can corroborate or that can help point you in the right direction.