I get this a lot, as I am a foreigner in Japan, lol.. Some of the times, it's the insulting "eeek a weird white person!" (yes, a
white foreigner, at that) kind of looks, but I do sometimes get cute guys staring at me. As I am socially awkward, too, most of the time I do nothing

Sometimes, though, when I feel up for it, I wave or smile at the guy.
A few days ago, I was going home from Shibuya (I was alone) and there were three guys staring at me. They were trying to get my attention
so bad.. and I was trying to not laugh,
so bad! Being socially awkward, part II.
But, yeah, I guess smiling back at the person is the best way, also looking up from time to time (but not the whole time) to see if he/she is still looking. If you don't end up starting a conversation, at least you'll be left with a warm feeling, because someone smiled when seeing you.