I have told people the same thing when I get asked this question. "The best camera is the one you have on you". There is a huge market for smaller more portable cameras today and many times I find myself reaching for mine when I do not feel like lugging around a whole kit.
At the end of the day, it is as you say, it is the photographer, not the gear that matters.