Legal moralists worry about the degradation of social norms and community connections. Their worry is that immorality tears at the “fabric of society” where that “fabric,” presumably, is the system of moral beliefs held in common by most people in the community. Legal moralists are thus happy to impose their own moral views on others with the power of government—they think that this must be done if the norms (and moral beliefs commonly held) are threatened.
In their willingness to use government power to impose their views of morality, moralists ignore the fact that when a government is empowered to force people to act in certain ways, that power crowds out the ability of individuals to interact freely with one another. That is a problem for their view because if individuals can’t freely choose to act in ways others (including the moralists) think is bad, they also can’t freely choose to act in ways others (again, including the moralists) think is good. The problem for the moralist, then, is that you can’t have a morally good community if people can’t choose freely—you could at best have a simulacrum of such, more like a collection of automatons than a community of persons. A morally good community is an association of moral beings—beings that choose for themselves—who (often) freely choose the good. Putting this a different way, the moralist has to believe you can have a community made top down, forced upon members who are free, but that is impossible. Community thus has to be made bottom-up; community is made by the individuals within it choosing to interact well together.
This applies, by the way, regardless of the level or size of community. A condo or homeowners association, for example, can’t be made into a genuine community by fiat—even if those trying to do so take themselves to know (or actually do know!) what is best for everyone. It simply cannot work—or rather cannot work unless everyone in the group agrees—in which case, it is not top down after all.
To be clear: if you want to start a genuine community, do so only with people who already agree with you. (Like, but not necessarily as rigid as, a cult.) I’d add that if you want the community to remain a community, you’ll need a way to guarantee that all who enter it agree with you in advance. (Again, like, but not necessarily as rigid as, a cult.) Otherwise, you’ll face opposition from some of the newcomers—different ideas about what the community should be. And those ideas from newcomers (at least those who enter justly), will have just as much claim to be legitimate as yours. Denying that entails not community, but moralist dictatorship.