Service Account Authentication via Secret Key JSON file (for Google Cloud Platform)

Anvil Q&A
1

I’m sure this is pretty simple, but can’t figure it out… I need to authenticate with Google Cloud Platform to access storage buckets using google-cloud-storage (which has been confirmed as installed).

I would typically use a service account and a key file (with appropriately designated permissions), stored e.g. locally for local application development. The code to authenticate and instantiate a GCS client would then be:

import os   
from google.cloud import storage

project_id = 'MY_PROJECT_ID'
os.environ["MY_GOOGLE_APPLICATION_CREDENTIALS"] = "MY_PATH_TO_CREDENTIALS/credentials_file.json"

gcs_client = storage.Client(project=project_id)

I know that I want to use the App Secrets service, but I’m not sure exactly what the best pattern would be… do I need to copy and paste the JSON string from the file into the service or should I upload the file to a specific location and give Anvil access to that?

1 Like
2

Off the top of my head, this should work:

import os.path

if not os.path.exists("/tmp/creds.json"):
  with open("/tmp/creds.json", "w") as f:
    f.write(anvil.secrets.get_secret("google_creds"))

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/tmp/creds.json"

…but the Google SDK almost certainly has a way to configure authentication more cleanly than writing it out to a file and then slurping it in. (I just had a look, but I didn’t find anything – Google developer documentation deeply painful to navigate, though, so I might have just missed something.)

3

Thanks Meredydd, I know what you mean about the Google Developer docs! This looks great, I’ll try this approach.

4

OK, I just got around to testing this and am getting the following error when I set the value of the google_creds secret to the JSON (which is valid JSON according to e.g. https://jsonlint.com/):

DefaultCredentialsError: ('File /tmp/creds.json is not a valid json file.', JSONDecodeError('Expecting value: line 1 column 1 (char 0)'))

Which seems to indicate that the first character of the value is not valid JSON. (Obfuscated) JSON is:

{
	"type": "service_account",
	"project_id": "XXXX",
	"private_key_id": "XXXX",
	"private_key": "-----BEGIN PRIVATE KEY-----XXXX",
	"client_email": "XXXX.iam.gserviceaccount.com",
	"client_id": "XXXX",
	"auth_uri": "https://accounts.google.com/o/oauth2/auth",
	"token_uri": "https://oauth2.googleapis.com/token",
	"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
	"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/XXXX"
}

Is there a certain way/format you should paste in a JSON string to use it as a secret, or any other thoughts? Thanks!

5

Two thoughts:

  1. Are you sure you pasted the JSON character-for-character correctly into the Secrets Service configuration?

  2. Could you try printing the contents of the JSON file from your app to the logs? Something like:

    with open("/tmp/creds.json", "rb") as f:
    print(f.read())

    (Or even using anvil.media.from_file('/tmp/creds.json') to get the file as a Media object, then making it downloadable as a Link, so you can check you wrote the file correctly?)

6

Just coming back to this and I don’t know what you did, but it now works perfectly without any additional updates from me! Thanks… we have lift off!

7

Hi there, this exact code has been working in production for a while but suddenly has started returning the following error:

DefaultCredentialsError: The file /tmp/creds.json does not have a valid type. Type is None, expected one of ('authorized_user', 'service_account').

Luckily it has not been noticed yet but I do need to fix this urgently and I’m not sure how! Thanks in advance.

8

Hmm…I wonder whether the file somehow got truncated and needs overwriting?

9

Thanks for getting back so quickly. I have printed this to the output and it doesn’t look truncated - it is the valid JSON file wrapped in b’{valid_json}’ as a bytes string