Hey, also having this issue here. Client M1 MacBook Pro 11.0.1 and Windows 10 Server.
@Nico M i'm not the person you asked but i changed the roles of my setup and the lag persisted.
@joebob disabling AirDrop worked for me. I noticed that hardwiring into my network worked very well, but that also disables airdrop (if wifi is off).
Disabling AirDrop is my suggestion for anyone searching this now.
AirDrop was the culprit for me. I noticed it got really bad when I opened my phone to send a photo to someone. Disabled AirDrop from Finder on the Mac side got rid of the repeating stickiness.