First of all, let's review what the instrumental temperature record looks like:
explanation: the early (~1915-1940) warming trend has a different explanation to the ~1975-2010 warming trend, and the cooling in between has yet a third explanation. The earlier warming was due to "a combination of factors, including a lull in volcanic activity (therefore the absence of its cooling influence), a slight increase in solar output, and yes, an increase in greenhouse gases too (although not nearly so much as during more recent times)". The second warming period was caused largely by our industrial emissions: mostly CO2, but other greenhouse gases played a role. The lull in warming from 1940-1975, in turn, was due to anthropogenic cooling due to global dimming. The quick version is that aerosols such as sulfates increased very rapidly from about 1940 to about 1975 (spot the relationship?) and then declined again rapidly. In other words, the warming forcing of the greenhouse gases was masked by the cooling forcing of various aerosols, and the rapid decline of the latter after 1975 meant net radiative forcing suddenly turned positive - resulting in warming. While this is undoubtedly complicated (though, what would you expect?) and may even seem contrived, it is important to note that this version of events is consistent with the available data and no other scenario is consistent with the available data.
But how does someone who agrees that global warming has occurred but denies anthropogenic forcing is responsible, explain the observed temperature record? (And, importantly, the pattern of warming). To think both the above propositions true, you have to believe the darnedest things: (1) that some mechanism entirely unknown to science inhibits or obviates the warming effects of our greenhouse gas emissions and (2) that some other mechanism entirely unknown to science caused the warming that has been observed. We know beyond all reasonable doubt that carbon dioxide is a greenhouse gas - it's a basic finding of physics, established under the most rigorous laboratory conditions. We also know, beyond all reasonable doubt, that the atmospheric concentration of greenhouse gases has increased dramatically because of our industrial emissions since the Industrial Revolution. To think these two facts don't combine to produce warming is to think that there is something (no one knows what) that somehow (no one knows how) inhibits the warming that would normally occur. But worse, there is yet something else (no one knows what) that actually caused to warming we see. (Something must cause it! And it's not the sun, in case you were wondering).
To be a climate denialist, then, you need to believe at least one of the following: that there is a grand conspiracy among scientists to withhold the truth about the climate record or that, extraordinarily improbably, (1) and (2) are true. On the first, note that there is no precedent in all of history for the existence of a conspiracy on the scale climate scientists have been accused of. ("Climategate" was a non-event. No fewer than three independent inquiries concluded there was no serious misconduct). On the second, extraordinary claims require extraordinary evidence and, as far as I know at least, no one has yet provided convincing, let alone extraordinary, evidence for either (1) or (2).
My Zimbabwean two cents worth: Vegter is no idiot (I actually quite like him) and, unlike several other climate denialists, I think it highly unlikely he's in the pocket of Big Pollution. My view is that he suffers from immense confirmation bias given that, as a libertarian, he doesn't want global warming to be true. One thing is clear, though, and this should give Vegter some comfort: the truth of climate change is logically independent of what our political response to it should be. It is possible to think - as Bjorn Lomborg's new book demonstrates - that climate change is real and a serious problem, without thinking policies libertarians despise are the remedy. A final note: as George Monboit nicely illustrated recently, there is honor is saying "I was wrong". Ivo, it's time to change your mind.