My kids like to do crafts. I don't like to spend a ton of money on craft supplies though. I've begun over the years (like before my oldest was born), to collect craft supplies when I find them super cheap somewhere (free, on clearance, given to me from a friend, recycled from other things I purchased, etc.).
Now, I don't go crazy and I'm not a pack rat by any stretch of the imagination. I have one stacking bin with pull out drawers to fill. I fill it with little things: paper, markers, crayons, stamps, pads, pipe cleaners, buttons, etc. Whenever I happen to be somewhere and there's a really good deal on something, I pick them up and put them in the bin.
All the little things I picked up over the years are kept in that bin and now my kids help themselves to craft supplies and paper to make their masterpieces. I encourage them to use their imaginations with what I have on hand instead of making trips to the craft store for specific things.