You’re not alone.
There are a lot of good reasons to want to use non-hash routing for the publicly accessible URLs you want search engines to index.
I originally went the route of using iframes to mix Anvil contents with static site contents. It turned out to be a bit of a nightmare for my use case, and I instead opted for a static site with Javascript that calls out to HTTP endpoints in my Anvil app as needed, and a link on the static site that goes to the full app for the private account area (the parts that shouldn’t be indexed).
For me part of the benefit was being able to isolate the parts of the site that tire-kickers would be using frequently to the static only site, so that they wouldn’t be using compute resources from the Anvil app.
I know that others here have used iframes, though, and have had it work fine for their use cases. So I’d say experiment with some prototypes and see what works for what you’re trying to do.
And as Paul notes, you can do the entire site in Anvil, although at that point you’re using hash routing URLs. If you ever switch away from Anvil to another platform you’ll most likely break any links to pages on your site that exist on the web.