It's actually pretty easy to get into it. I am a low level hobbyist and pretty much have used just kits so far. Come with the ingredients and instructions. Used Brewers Best and have had good luck. Also used a kit from Niagara Traditions and it worked well also. A beginners kit can run you a little more the $100, kits about $40. For bottles I use what I drink from and have purchased bottles from Consumers for the deposit cost - just be careful to grab the cleanest.
Hate to say because he is kinda goofy, but Craigtube on YouTube actually is very helpful. Corny, but informative for a beginning brewer. Watch his older stuff first. Also, a bunch of other good stuff on YouTube.