I was having a hard time. I used the solution at the end. Even with the solution I was still having a hard time. I experimented more and this might help a lot of people.
I would follow all the steps Sam laid out but still no success. A yellow error would pop up in the terminal:
LogOnline: Warning: OSS: No game present to join for session (GameSession)
I did some research. I found out that the possible problem is that there is not enough spawn able points. In that second I realized maybe there is not enough players in the game.
I immediately opened Unreal and went to Play->Change Play Mode and Play Settings->Multiplayer Options->Number of Players
Once there I changed the number of players to two and it worked. I then changed it to three and I was able to spawn three clients into my server!
Here are some photos.
The Play Expanded Settings:
The Steps to find where to change number of players.
The End Result:
I hope this helps anyone! 