It 100% does make it a Ponzi scheme, don't try to fool yourself. On the bright side - many Ponzi schemes have a period of growth where the early participants enjoy incredible returns. Just don't be too late to the party ;)

The rest of the crypto movement relies on the Greater Fool Theory (google it) - in that you can only make money on XYZ coin if you can find someone who will buy it from you for a higher price than what you paid to buy it yourself. Not exactly a Ponzi scheme but not really an Investment either. More likely a Speculation.

Take care and good luck!