I have been indulging in speed tests myself lately. And unfortunately, you can only improve your score up to a certain limit. The loading size of your anvil app will increase with each line of code you write so it’s highly likely for your app to have a bad score.
But those tests don’t know that your site is a SPA. All Single Page Applications have bad scores on speed tests because they load everything at once. Nobody will count the fact that, unlike normal websites, there will be no loading after the initial loading.
So the only thing you can do is to try and fix as many problems as you can. All Speed Tests come with reasons and solutions attached to every problem.
And if you are interested in something advanced, you can try to implement something like Server Side Routing Attempt and no longer make your app behave like a SPA