You probably already have considered this buy why not get an iPhone? Once you remove the music and video functionality from the iPod touch, the GPS and mobile internet functionality stand out more.
Anyway to answer your question, the 2nd generation uses a 620mhz CPU underclocked to 533mhz and the 3rd generation uses a newer CPU underclocked to 600mhz with a more powerful GPU as well. Compared to my iPhone 3g which has a 400mhz verison of the same CPU as the 2nd gen, the 2nd gen iPod feels marginally faster while the iPhone 3GS (same hardware as 3rd gen iPod) is significantly faster like you mentioned. I would go so far as to say my iPhone 3G feels sluggish with OS3.0 while it was obviously made for the iPhone 3Gs so the 3rd gen iPod Touch pairs nicely too. I think the 3rd gen also gets bluetooth support, if you need that.
Long story short, get the 3rd gen.