Exactly right! Permalinks are dangerous – not all content is public, so those URLs contain authentication tokens, and it’s a bad idea for authentication tokens to be valid forever. (It would mean that anyone who has been allowed access to this media once would be able to access it forever, which is fine for your application but could be dangerous for other people’s.)