Ways to speed up csv import to data table?

I have a nightly upload that uploads 1000 rows with 8 columns and takes less than 10 seconds.

In my case it’s not a python script, so instead of using the uplink, I created an http endpoint to which I pass a list of dictionaries in json.

Whether it’s uplink or http endpoint, the concept is the same described by @jshaffstall: minimize the number of round trips by sending one list of rows, then doing the bulk of database operations in the server.

1 Like