ScriptableObjects are actually an excellent example of a mechanism for eliminating Singletons (though a purist may argue that, like the PersistentObjectPrefab, they are really just another flavor of a Singleton).
There are a couple of caveats to this approach, however:
A ScriptableObject cannot contain components or UI Elements… In the RPG course, we use the PersistentObjectSpawner to bring in a Saving System and a Fader, the latter of which is a piece of UI to obscure the scene between changes. The SavingSystem could easily be adapted as a ScriptableObject, but the Fader and the Saving Wrapper (which we use to listen for keystrokes, and to automatically load the current save file on game start) cannot be adapted as ScriptableObjects.
SO’s also do not have Update loops, which limits some of their functionality. They can technically get around this by finding almost any GameObject and starting a coroutine, which they could treat as an Update… I don’t recommend this except for short one offs, as it’s very hard to debug these situations. Audio Managers and ScriptableObjects go together very well. An SO cannot have (or be) an AudioSource, however, so they have to use AudioSource.PlayClipAt() to make their sound.