I don't believe that he is trying to trivialize your particular situation Nick. I think he is trying to say that sickeness happens when you get groups of people together. Obviously, your recent medical conditions, as you yourself pointed out, have greatly affected your ability to handle a cold. Not surprising really considering your body is trying to heal itself.
I also believe that he is trying to state that in public gatherings, its a numbers game. The more people you have together, the greater the risk of people being sick.
Heck, my wife/daughter had strep before and after the LAN. It mite have been me and not the daycare that got you sick.
To extend the analogy, to say since I got sick after going to school today, I am not going back may be a bit of a stretch.