Bag on a Stick
Very simple gag but a great one, since it can be used multiple times over, even in the same adventure. Great for tribal natives gone restless and humanoids, but anyone can have set this up. Just what the header says, a simple bag over a stick stuck in the ground or floor.
As GM you can place the bag on a stick anywhere, in a floor crack the heroes have passed before, outdoors in a clearing or path, or at the edge of the PCs' encampment the following morning, what have you. Place anything on the stick - a coiled yellow viper angered by the bag removal, mini crossbow w/poison, transdermal hallucinatory drug dusted on the bag, yellow mold colony, an NPC ally's head, a weapon, scroll tube or satchel, what have you.
The idea is to build tension and/or stall for time/distract the party. Provided it's used properly, you'll be amazed at how paranoid players will get from this simple gag.