Codepage of text database columns

Anvil Q&A
1

What I’m trying to do:
I store a big JSON text in a DB column.
Then read it and feed it to a REST web service.
This way I get the error

status 400: {"ErrorId":"ERR0011","Message":"'$.Activities[0].Action.Sign.Elements.TextBoxes[51].Validation': Unable to translate bytes [EC] at index 654 from specified code page to Unicode.","TraceId":"0f738d19-38e3-42e9-9445-84026b1f492b"}

What I’ve tried and what’s not working:
The same JSON copy-pasted in Postman works fine.
Copy-pasting from the DB column into Notepad++, saving the JSON and searching for byte “EC” with a binary editor won’t even find “EC” in the whole file.

So I was wondering if text columns in the DB are subject to a particular codepage.

2

Nevermind, it has nothing to do with encoding.
Even if I force it bye stating:

string.encode().decode('utf-8')

It throws the same error.
It looks like the problem were the too many \n\t inside the JSON.
A simple “cleaning” strategy like:

import json
new_json = json.dumps(json.load(old_json))

solved the problem.
So the problem was the remote party’s ability to manage carriage returns and/or line feeds in the received JSON string.

2 Likes