There's no one answer for everyone. Large schools work better for some, small schools for others. There are certainly more options in a larger school and many have an easy time creating a tight group of friends, often through where they live, what activities they do and/or where they work. Others cannot stand the sheer anonymity of it and prefer a tighter overall community that a small school can provide.
You certainly are unhappy. Sometimes the reason for your unhappiness is not what you initially think. Are there other things going on in your life that is making you unhappy? Did you feel unhappy at all before you went college? Once you are sure that it is Le Moyne and that UB does not suffer from the same problems, go for it. You will get a degree either way, but why be unhappy while doing so if you do not have to be?
A best friend of mine went to Union and hated it there. He eventually transferred to UR and flourished. Certainly such a change can pay off big time.