Instead of an arbitrary padding value, I think the best way to do this is to consider the size of the sprite itself. Here’s what I did:
float spriteSize = GetComponent().bounds.size.x / 2;
xMin = leftEdge.x + spriteSize;
xMax = rightEdge.x - spriteSize;
For example, my sprite is 112 pixels wide. GetComponent().bounds.size.x returns a value of 1.12 which is the pixels / pixels per unit (in this case 100). Then you divide by 2 to get half the sprite size, and adjust the xMin and xMax values accordingly.
Still works if you change pixels per unit or scale of the object.
Now your sprite can go right to the edge if you want!