I read in a magazine a while ago that at the Marc Jacobs store in Greenwhich Village, they sell little knicknacks, like pencils and erasers that say March Jacobs and are really cute, for cheap. I hunted this place down, and what I found was amazing!
It was a whole store dedicated for little, cheap things. They had $35 rainboots in splashy colors and cashmere hats for, I think, under $25. They had $25 tee-shirts and $7 bracelets. They had bandaids and pencil sharpeners.
I got pencil sharpeners (really cute, they are like animals and they say MJ on the back, $1 each) for my friends. I also got a keychain and a portable bandaid carrier (part of me wants desperately to get hurt and part of me wants to save them forever and ever!).
While you are there, I recomend Magnolia Bakery a few doors down for the most worthwhile stomach ache ever. They have the richest, yummiest cupcakes ever. Make sure to get milk with it. I think it is on Bleeker St. and 11th.