The Retired Gladiator is a new NPC in the City of Stone, added with the Pyramid of Prophecy expansion.
You can find him slightly northwest of the NPC used to start the arena.
When you talk to him an upgrade screen will display, giving the choice to upgrade from 4 stats: attack power, skill power, armor or resistance. Each upgrade will always cost 1 Sword Token, which is a new currency earned in the Arena, and some gold. The amount of gold starts off at 500, and increases by 500 for every level.
You can calculate the gold needed for any level by simply using this formula:
Gold Cost = 500 x (level of upgrade you want to purchase)
There is no limit to how far you can upgrade stats here, as long as you have adequate tokens and gold, you can always buy another upgrade.
Here is a table with some stats on the cost of putting N swords into a single skill.