You have to feel some sort of attraction before the date happens. If you two got chemistry keep on dating. You don’t have to move fast but as long as you’re interested in someone keep it up.
That being said, I wouldn’t want a person that I don’t know well entering my home. I wouldn’t want to go to their place either. Stay safe and follow your intuition.
In order to build a deeper connection you have to put the urges to the side for a little while. I’m definitely not saying to “wait till marriage” lol , just till you get to know the person adequately. Make sure he’s worth your time and energy.
Relationships are hard. Get to know yourself and your needs before approaching people. It’s much more enjoyable for both parts this way. Having standards and boundaries is the way to go. If something doesn’t work out life moves on.