I created a singular collider for the Rocket prefab, then called using
rocketCollider = GetComponent<Collider>();
and toggled the same way:
if (Input.GetKeyDown(KeyCode.C))
{
rocketCollider.enabled = !rocketCollider.enabled;
}
This was a full collision disable allowing me to actually fly through the walls.