Yeah it's down to pattern recognition. If you've ever seen Floyd talking about defence he shows his tried and tested method of dealing with the most common combo in boxing, the 1-2 left hook. He parries the jab with his right, blocks the right with his left shoulder and brings his right hand back to his head to pick up the left hook. When that pattern changes to something outside the norm is when it can be hard to defend.
I remember one of the better boxers at my gym got caught by this young inexperienced kid and it was because the kid threw a left hook and then followed up with the left uppercut. The more experienced boxer wasn't expecting it so he got clocked hard. It was an unusual combo for him to deal with so he didn't recognise the pattern.
A punch takes roughly 50 ms from start to finish, but a human cannot react in less than 100 ms (well that's the limit that sprinters are allowed to react to the starters gun). So to defend a punch you have to see it coming before it's thrown. That's why reducing tells is so important in boxing. The tells are what allow the other fight to react and defend in time. Personally I think that some humans can react faster than 100ms.