That happens to me frequently as well. I go slow and adapt to what they are doing. I have an Incredibell that walkers & hikers really appreciate. I think they like the sound of the bell rather than someone shouting out at them. I like to do a warm up on a loop on a trail that is heavily traveled (Peters Canyon). Since I'm warming up I usually am going fairly slowly anyway. I often talk to people who shoot left or panic when I approach. In a very friendly way I tell them that we are used to passing them on their left so just stay to the right or just stop and we'll move around you. Most seem to appreciate that.
What gets on my nerves are the packs of women on a mission to get their walk on. They fit a certain profile, white, middle aged, well to do and on a mission. They take the whole path often times walking 3 & 4 abreast and don't even think they will make room for a bike. One time on this Peters Canyon Loop I literally came to a stop as I would have hit 1 or 2 of them as they refused to budge. I stood there on the right of the trail and asked if they would be considerate enough to make room for another trail user. 3 completely ignored me, one shouted, WE ARE WALKING HERE!!!! Oh excuse the hell out of me I guess this is your trail. F'ing whatever was her response and they brushed by me.
Every other trail user I come across is either considerate or clueless which is OK with me, I can live with that. Just don't freaking act like you own the trail.