Unfortunately, I'm often subject to colds, during this last autumn-winter I remember taking like 5-6 of them, luckily not that badly.
What I usually do is taking off AT LEAST 3-4 days, or sometimes even a week considering how I feel, to be sure to avoid fall backs or passing it to others.
But on the other hand colds are my kryptonite, others could recover a lot sooner, you should see how you personally feel after two days.