Let us assume for a moment that the EU countries collectively decide to define themselves by culture & history, so as to include settler states outside Europe, & that those settler states accept this definition & wish to join.
Any workable & internally consistent definition would imply that Europe rejects those people from outside Europe who have settled within it (note: that includes some of my family), & that the settler states (Australia, New Zealand, Canada, USA, Argentina, Uruguay, etc) do the same, & also reject their own indigenous populations. Sydney Chinatown, which I remember well from when I lived next to it, would suddenly become "un-Australian", as would much of Redfern, & the little Korean enclave near where I worked in Chatswood, while the cluster of Spanish restaurants around the social club near the western end of Liverpool Street would become the epitome of European Australianness. Maori wouldn't qualify: they'd be foreigners in their own country. Etc., etc.
Do you really want to live in that sort of society? I don't. Once you start down that road, where do you stop? There are real horrors at the end.
And that's probably far enough in this direction.