keep in mind, a HUGE difference is made by the common misconceptions of untrained people about fighting.
an untrained person has absolutely a different idea on what fighting is to a trained person. hence the untrained person doesnt value qualities that he THINKS are irrelevant.
two untrained people fight in the following manner usualy:
person A punches person B. person B is FORBIDDEN to block or dodge.("natural born runner", "coward" etc)
person B retaliates. "i punch you, you punch me, lets see who the bigger man is/who wants the most/other such crap". this is why the phrase "trade blows" came to existance.
two trained people fight by using jabs, footwork, cutting angles, etc. an untrained fighter has NEVER seen/felt/valued these things; he realy thinks Mighty Mouse' footwork is irrelevant in a fight because, like, "he cant strike & move, bro! he must just trade blows otherwise he violates some guy code!!!!"
i am an amateur MMA fighter, and i am not a badass, not even in my own gym; but when a friend of mine (not too bigger, but MUCH stronger than me and most everyone i know) insisted i'd have no chance, i realy gave him gloves for a bit and we sparred. NOT a full fight, just sparred for a bit to show him the difference. he couldnt do anything because he'd never understand things such as grappling, positioning, control, footwork, but only what Van Damme movies showcased "you punch me i kick you, but basicaly we just trade blows".
if what an average person thinks is that fighting is oly a fair trade of blows and that using tactics, control and fotwork is "cheating" or "cowardice", then yeah, i see why they think smaller fighters have no chance. they basicaly think technique is either "coward's way" or unimportant/irrelevant alltogether.