It is definitely more the second. There are plenty of very hot girls in Golok, but a lot of them really don't speak English at all. Malay and Mandarin are more common.
No ladyboys that I could find, though, at least not ones working in the sex trade, like any Thai town they are still there working in 7-11 or cafes. Hat Yai is however top notch for ladyboy freelancers if you are in the south, had many a foursome with three really hot ones out of the Hansa Plaza cabaret at only 300B each.