Couples who have been married only a few years often remark how there were so many people at their wedding, or even bridal party, who they no longer speak to. They have fallen off the face of the Earth since the wedding. Maybe life shifts, attentions divert, people move away. For the most part, included in this group are people with which your bonds may have been superficial.
Do not invite certain people to look good. Do not fear you will look less than if this or that person does not attend your wedding too. So what if they were in your fraternity - be the cool discerning outlier and do not invite everyone. There are other innovative people in your alumni association that will accomplish more and be a better contact for life, and they do not care about being invited to your wedding. Similarly, do not invite people that will attend your wedding just because they too have FOMO. Your wedding day is too meaningful for that.
Do not invite work friends that are not going to be friends after you change your job. Invite no one for work. If anyone asks say it was your in-laws rule. Just think how much less annoying it will be if you plan on keeping your job. No endless stories for years to come about your gross cake (joking) or the time Timmy decided to use the bushes to relieve himself and your grandma saw. Joking again or maybe not. Additionally, your parents should not be allowed to invite their work friends either, and that goes for both sets of parents.
Lastly, in the New York City and surrounding suburban areas it is really hard to find many attractive venues that have a guest count at or exceeding 250 people. There are enough venues that push their max capacity to be more attractive to more couples. If you doubt how comfortable their max capacity would be for you and your guests, then you may want to ask your coordinator to tell it like it is - is your guest count pushing it? Maybe ask for some photos of past weddings with a higher guest count or stalk instagram.
Having problems with your guest count? Ask yourselves who really will be there for you in five years. Those who are there for you for the long run do not need an event or milestone to take place in order to stay in touch and vice versa on your end as well. Plus, a smaller wedding makes for a more intimate wedding, which often times equates to a more relaxed atmosphere.
True friends = wedding guests