On Wednesday night's "Late Show with David Letterman," Paul McCartney opened up about befriending and later drifting apart from Michael Jackson.

Paul said Michael originally called him up in the early '80s asking if he wanted to make some hit songs together.

"You know, he came to my house and we got to know the family and stuff," the 67-year-old legend told Dave. "We had a really good time. We made a couple of records together ["Say Say Say" and "The Girl is Mine"], did a video and were very good friends."

The friendship was never quite the same, though, after Michael bought the Beatles music catalogue and decided not to sell it to Paul.

"He kept saying, 'That's just business, Paul, you know,' so I went, 'Yeah, it is,' and waited for a reply. But we never kind of got to it and I thought, 'Mmmmm,' so we kind of drifted apart. It was no big bust-up. We kind of drifted apart after that. But he was a lovely man, massively talented and we miss him," Paul said.

After the interview, the former Beatle and his band rocked atop the marquee of the Ed Sullivan Theater overlooking downtown NYC, playing "Get Back" and "Sing the Changes."