Pretty big rockhounding nerd here.. got about 20 flats of various collected minerals from around my state.
Colorado is a great state for rockhounding, but I have to say for fossils WNY's Penn Dixie Quarry is about as fossil rich a locality as I could ever hope to imagine. It's a pay-to-dig site but I came back with an impressive trilobite, several well preserved brachiopods, and more horn coral than I know what to do with.
I guess back in the day the Walworth Quarry east of Rochester used to let the public collect gorgeous water-clear fluorites, but like so many other good things those days are long over.