So finally managed to connect to the UE server. It was the port number that successfully got the client to connect. For some reason, my serverv is using port 7780. But I have also seen it use 7781.
"C:\Program Files\Epic Games\UE_4.26\Engine\Binaries\Win64\UE4Editor.exe" "C:\Users\kelvi\OneDrive\Documents\Unreal Projects\PuzzlePlatforms\PuzzlePlatforms.uproject" xxx.xxx.xxx.xxx:7780 -game -log