Performance Considerations of Collider vs Trigger

Are there any significant differences between implementing taking damage with a trigger vs a collider?

No, the difference is in what happens when a collision occurs verses a trigger. In a collision, the physics system (RigidBody) will react, preventing an actual intersection by applying physics consequences to one or both of the colliders. Triggers will simply fire the OnTrigger events.