So I added the new M3 theme as a dependency to one of my apps (the app is using the Rally template).
This had undesirable effects on the style of components in the editor but not on the live/running app. See the below screenshot - the buttons in the editor have become completely different whilst the styling is unaffected when the app is running.
When inspecting these elements in the browser, the SETTINGS button has class="anvil-btn anvil-btn-default anvil-to-disable"
in the editor, while it has class="btn btn-default to-disable"
in the running instance.