Years ago, I used the prime lens on my DSLR and never used to change it, thinking I would never get the dreaded dusty focus screen. And yes spots on the focus screen too. So it's not a zoom lens issue only or removing the lens issue only. I remove it every day now. What I started to do years ago was after every day of shooting, I do the vacuum trick. This is simply removing the lens and holding the camera body so it faces downward. I then use the vacuum tube and suck out the dust/debris. (Do not try to hold the vacuum tube so close to the body's lens mount you want a tiny gap so not to suck out the shutter blades. Pretty simple. I still get some tiny black spots that won't budge but I live with it, and do not try to Q-tip or rub on them. My focus screens seems to say a bit clearer.
Never, I repeat, never rub the focus screen with Q-tips or microfiber cloth. This is for a few reasons. Nikon for some strange reason uses some sort of micro fuzzy blackout in the black box, this micro fuzz detaches from the black box and ends up on the focus screen when using a microfiber cloth. The Q-tip leaves cotton on the focus screen and that's a huge pain trying to remove that as it gets pinched in the focus screens locking tab and frame. And the microfiber cloth will scratch the focus screen as it is soft plastic.
I learned the vacuum trick from Nikon. I never use any thing that blows in the body either, This just forces the debris onto the shutter blades and onto the sensor when the open and close. I shoot f/16–22 a lot so I don't risk it.
The dirty focus screen is easy to remove, but I also don't recommend this either because when reinstalled there will be debris in-between the focus screen and focus point screen (Yes there is two screens a focus screen and focus point screen before prism). I also extend my zoom lenses and vacuum out the lens too.