I think the (anticipated) trouble with targeted ads is that they compete for your attention with the content that you were trying to view in the first place. Imagine you sit down to read a book and are first hit with an ad which, at that moment, actually seems more compelling than the book, because unlike the book, it has been perfectly refined by analyzing your actual behavior to attract your attention.

It depends what you were browsing when it picked up your interests and who is watching when it shows the ads!

A UK ISP got into trouble when it was (illegally) tracking all your web browsing and replacing each site's ads with it's own ads based on your browsing history. When it replaced the ads for a childrens charity on a national fundraising event with some targeted at 'gentlemen's special interest' sites there was something of an outcry