So, how many of them murdered someone? Should we extend this reasoning to, say, Muslims - when one rams a truck into a group of people, we round up the lot and send them to Guantanamo? What about the Bernie Bro who recently shot a Republican political figure? What about the black gay guy who murdered those white journalists? What do we do to that group of black kids who beat up that disabled boy on youtube? What do we do to BLM members who riot and become violent - and then the rest of the movement that fostered them? Does your blanket "Well, yeah, they didn't actually murder anyone, but they were making the same salutes as that dude and read the same books!" extend beyond the extreme Right and encompass any number of other groups we could peg with similar guilt by association and some shared beliefs? Careful, because as with many other dangerous tools, you use it for your ends, but the other side will pick it up and start using it when you're done with it - against groups you support.
Show me some sign that there was organization to commit murder, that this is supported by that community, and then we talk. Until then, by suggesting you round up the people chanting and saluting, you're literally suggesting treating people as murderers for shared political belief.