I was two years younger than my classmates,” PEOPLE’s Sexiest Man Alive recalls of his socially awkward early years

These days the 40-year-old music superstar is having fun celebrating his latest distinction of EGOTSMA, a new acronym representing the fact that he’s won an EGOT (Emmy, Grammy, Oscar, Tony award) and now Sexiest Man Alive.

But the Voice coach, who just released the new deluxe edition of his holiday album A Legendary Christmas, admits he wasn’t always the confident, swoon-worthy guy fans have come to know love.

“Oh yes, I had an awkward phase,” says Legend of his early years. “I was two years younger than my classmates, so I get to high school, everyone else is around 14 and I’m 12. And I get to college, everyone else is around 18 and I’m 16,” recalls the star.

John Legend as a freshman in high school. SETH POPPEL/YEARBOOK LIBRARY

Though the Ivy-league educated musician had brains beyond his years, he says “I was always like less mature, less at ease socially. I had plenty of awkward times.”

A savant who picked up playing piano at age 3, Legend says he tried using his skills to win over the ladies at an early age. “I tried, I’d say around 12 or so to start writing songs for girls I liked. It wasn’t extremely effective at age 12 but I got better at it,” he says, adding, “I eventually wrote ‘All of Me’. It’s got about two billion views now.”

John Legend playing the piano as a child. COURTESY PATRICIA TYREE

And he landed the girl of his dreams, model-turned-cookbook chef and TV star Chrissy Teigen, 33, whom he married in 2013. “She always looks amazing,” says Legend. “And I get to stand next to her a lot and I don’t want to be a huge let down…so I try to dress well enough to be worthy of being next to my wife.”

Despite Legend becoming a local celebrity on the University of Pennsylvania’s campus for his performances in a popular A cappella group while in school, breaking into the music business wasn’t easy. “I was turned down by every record label in existence, including the one I’m signed to now,” says Legend, whose first album Get Lifted in 2004 earned him the Grammy for best new artist. “I kept going until I finally turned those no’s into a yes.”

As for what he thinks is his sexiest quality, “I mean, I’m a pretty good singer. That helps,” he says with a smirk. “Least sexy? I love to do crossword puzzles. I’m a nerd in a lot of ways.”

John Legend Is PEOPLE’s Sexiest Man Alive 2019: ‘I’m Excited but a Little Scared at the Same Time’

John Legend. DOUG INGLISH

And while he has little insecurities like everyone else (“I don’t like my legs that much truthfully, so it’s a good thing we didn’t have shorts in the shoot!”) Legend, who’s dad to Luna, 3½, and Miles, 18 months, says he couldn’t be more at ease with himself now.

“I think you grow into being comfortable in your own skin,” he says. “Most of your teenage years, into your 20s, you’re still figuring out who you are, how you present yourself to the world…eventually I think you get more at ease with yourself. I know who I am. Maturity helps with that.”

For more on the Sexiest Man Alive – and all the hottest guys in Hollywood – click here and pick up this week’s issue, on stands Friday, Nov. 15!