Google "Day Use" hotels and see if there's any availability in your area. Or else just get the room for the night even if you will be using it for a few hours.
And sometimes you can get AirBNB for cheaper.

I just googled day use hotels for my nearest big city and got zero relevant results.  Zero!  There may well be some that offer day use rates, but they don't seem to advertise it in any way that google can recognize.  Personally, I rent a room in a cheaper motel, make sure to schedule my date after check in time, and then after she leaves I put the key card on the dresser or table and go home.  

I don't know if the following will be helpful to you since I don't know the OP's domestic situation, but as I've said here before, these days I use my RV for dating.  The girls seem to like it a lot.  I park in some convenient parking lot such as a shopping mall, she meets me there, we have a great time BCD, then she drives away.  Several who have become regulars and trusted friends prefer to have me drive the RV to their house and pick them up, then we go park, then I take them home again.  It works great and has saved me a ton of $$ on hotel rooms.

Some hotels and many motels near have day and hourly rates. It seems you have to call each one though, because they don't list them. You'll have better luck near airports.  

There's an app called recharge for hourly rooms but it's still in its infancy. Just starting up in LA so Atlanta is probably a no go.  

Also try hotelsbyday.com and dayuse.com which are also in their infancies but I've occasionally found rooms using them.

I seem to recall seeing some "sex news" clip on Playboy TV many years ago (no longer a subscriber now) that featured "love hotels" in Tokyo,  

These were relatively cheap, usually themed rooms available for partial day rental.  

Apparently in Japan, the incidence of singles living with parents is very high, so it's rather uncommon for a single (or married looking for strange) to have a "love shack" of their own.  

OP: If you decide to run with this and create a start up in Atlanta, I expect a 5% stake, please.

I've used this in Atlanta a few times, I can always get something under $100 in Perimeter or Downtown (no one wants to go to the airport). The key is you prepay, so don't use it until you're sure she doesn't flake out on you.

I've used different sites, but for me, Hotel Tonight is the one that has worked the best. You know which hotel you're getting (unlike Hotwire and Priceline), and the more you use it, the cheaper the rates get (for the same properties). I sometimes call the hotel first if I'm booking a morning meet to make sure they can accommodate the request, then book on the app once they say they can take early check-in.