It's a huge factor for the reasons I mentioned previously. Steering people into lower value neighborhoods is bad but it's the lost value, based on lower equity, that becomes the real problem. Remember, home equity is approximately 70% of the average American's net worth. Steering people into low value neighborhoods is steering them into a lower lifetime net worth and all of the issues that go with that.
Also, I think people overstate the general Section 8 issue. No one puts low income Section 8 housing into a good neighborhood. And people who complain about that possibility don't seem to understand the economics. If an individual buys a high value property then they're going to try and maximize rental value. Section 8 tenants don't maximize rental value so no one intelligent would do that. If the neighborhood is good, you'll have more competitive rent options or you can get an attractive sales price.
Now, the real debate happens when someone thinks they're in a better neighborhood than they really are. In those neighborhoods, an investor can acquire properties that are low enough in value to justify Section 8. But, again, they're still trying to maximize rental value. You don't put in Section 8 tenants unless the neighborhood can't support higher rents on its own or there's not much value in simply selling it and moving on. The issue is that the neighbors see the Section 8 tenant and think that the Seciton 8 tenant is driving down property values. But they're too close to the neighborhood, they don't understand that already dropping property value is what made the neighborhood attractive for a Section 8 landlord in the first place.
Think about it like a girl who used to be slim and gradually blows up in weight. When her boyfriend leaves her for a slim chick, the fat girl blames the slim chick for stealing her man, without realizing that her gradual weight increase is what made the new chick so appealing. But, of course, she's too close to the situation to realize she was letting herself go.
That's the mistake many of these people make. They see the Section 8 tenant and think it's the cause, when it's really the result of an already worsening neighborhood.