That's not racist imo! It's just a neat little fact!
Anyways, no, I don't think you'll have to worry about those differences at all! When you like someone for who they really are as a person, you don't pay attention to things like looks as much as you'd think!
And just wait a couple years, and you won't be as tall compared to him
Good luck!! :D