Game is still Joinable after launch

I started debugging like you showed and if I’m understanding this correctly if you are using presence based joining then you are still going to be accepting players unless you manually disable bAllowJoinInProgress. I’ using 4.24.3

bool FOnlineSessionNull::NeedsToAdvertise( FNamedOnlineSession& Session )
	// In Null, we have to imitate missing online service functionality, so we advertise:
	// a) LAN match with open public connections (same as usually)
	// b) Not started public LAN session (same as usually)
	// d) Joinable presence-enabled session that would be advertised with in an online service
	// (all of that only if we're server)
	return Session.SessionSettings.bShouldAdvertise && IsHost(Session) &&
			  Session.SessionSettings.bIsLANMatch && 			  
			  (Session.SessionState != EOnlineSessionState::InProgress || (Session.SessionSettings.bAllowJoinInProgress && Session.NumOpenPublicConnections > 0))
				Session.SessionSettings.bAllowJoinViaPresence || Session.SessionSettings.bAllowJoinViaPresenceFriendsOnly


That code shows whether the match should be advertised but yes, that is what the bool is doing, only advertise if bAllowJoinInProgress and it has open public connection.