The 4th Gen i7 may also be a little low-spec as well. I have a 10th Gen i7 10700k which can clock to 4.66GHz when using boost and use a 3060ti which is actually more than fast enough. The CPU struggles with VR as the frame rate is 120fps quite happily with that card.
There are still a number of bugs and I would hope that UE 5.1 will address them and boost performance somewhat as 5 is slow.
I previously had a 1060ti and it was amazingly fast with UE 4.27 and VR but could barely get 30fps in 5.
One last question - when you started the project, did you create it using the VR template or a blank template - I found using the VR Template and then a blank level gives better performance so there must be some settings within it which improves VR Performance.
See here for requirements. These I would say are minimum rather than required:
Hardware and Software Specifications for Unreal Engine | Unreal Engine 5.0 Documentation