So far IdentityServer4 only supported a single signing key at a time. There are historic reasons for that.
When we started with .NET Core, the only x-plat algorithm that really worked (without #ifdef hell) was RSA with SHA-256 (RS256) so we went with that. In the light of newer security requirements (e.g. FAPI), RS256 became discouraged, and newer algorithms like PSS or ES will take its place sooner or later.
A quick fix (one we could do without breaking changes) was to allow using a different signing algorithm than RS256. We added that in IdentityServer4 v3. But still we only allowed a single algorithm only, which makes migration from RS to something else, not really feasible.
Starting with v4, you can register multiple signing keys – or to be more precise, one for each algorithm you want to support, and clients and resources can express their preference if they want to. It’s a pure opt-in behaviour.
This way you can smoothly phase-out RS256 in favour of something else by switching your clients and APIs one by one when they are ready.
Check the updated docs – it’s really a small change, but pretty important going forward.