Make Ancestries Distinct By Giving Them Mastery of Their Domain

What do dwarves like? Brewing beer and mining gold out of the earth? Yeah, so what. Humans like those things too. 

What about elves? Music and magic? Humans. Love. That. Shit. Too.

If you want ancestries to stand out, don’t just make them the best at their thing. Give them exclusive dominion over it.

Dwarves 

Of the surface peoples, only dwarves can mine beneath the earth. They are not just the best at it, or the ones who do it most frequently. They are the only ones who can do it. The earth quite literally rebels against the efforts of surface folk to excavate its riches. Cave-ins, monster attacks, and even earthquakes greet those who try. But dwarves enjoy a mutualistic relationship with the earth, like plovers cleaning crocodile teeth. They can burrow into the ground and sense which riches can be removed without agitating the angry earth. Only dwarves can mine.

Elves

Of the intelligent creatures of the world, only elves can play music. They’re not just the most talented; only they can do it. Other creatures literally can’t carry a tune. Musical notation is gibberish to them. Instruments are like alien artifacts. Music is a force as much as magic is, and like magic, it can be a demanding master. Elves have some gift for it, reaching back into the prehistoric past, when the gods divided up their gifts to the peoples of the world. Only elves can make music.

Humans

Of the civilizations of the world, only humans can build cities. The dwarves have mountain strongholds and the elves have forest sanctuaries, but only humans build cosmopolitan gathering places defined by cultural exchange and mercantile trade. Only humans build cities.

Gnomes

Of the craftspeople of this land, only gnomes can build machines. For other ancestries, inherent process inconsistency means that ideals of engineering rigor and industrial production elude them. But only in the hands of gnomes can machinery work like it does in the real world. Only gnomes can build machines.

Tieflings

Of the peoples native to the prime material world, only tieflings can travel the planes. The otherworldly blood flowing through their veins is the ink on an invisible passport each of these plane-touched strangers carries with them at all times. Portals open only to their touch, and Plane Shift reliably sends them where they want to go. For other creatures, planar travel is an uncontrollable (and usually deadly) misadventure. Only tieflings can travel the planes.

The Player Rebellion

If you tell a group of players that things work in a certain way in the game world, at least one player is going to immediately want to create a character who breaks that rule. They may bristle at restrictions as a matter of principle, or find that ancestry-exclusive cultural or societal domains are too close to real-world stereotypes about nationalities or ethnic groups being "best" at something. So it’s a matter of when, not if, a player will want to break the rules.

And… that’s great! Let them! 

The first human in living memory who can perform music? A great hook for a character. A group of elves and humans and dwarves who embark on a quest to travel the planes, the metaphysical constants be damned? Those are some stakes for an adventure.