📢 New Feature: Easily Work with Large Datasets and ML Models!

Hi everyone!

We’re happy to announce that we’ve made it easier than ever to attach large files to your app and use them on the server side with the Data Files service. It’s especially useful for machine learning models, large datasets and data that stays constant.

Check out the full announcement blog post here:

Ready to get started?

8 Likes

That’s cool, @ryan!

One question: are these files only server side, or is this also a convenient way to mange asset files for the client to store locally?

Data Files are server side. At heart, they’re a bridge between “this data exists in my database” (easy to manage in a serverless environment, it’s obvious where everything is) and “I want to access this file using ordinary tools like pandas.read_csv()” (accessing files the way you’re used to in a local script or Jupyter notebook).

The client doesn’t have a filesystem to read files from, because it’s in a browser. Clients can, of course, use Media from data tables (which is what Data Files is built on), as well as app assets.

If you want to get a sense of where we really thing Data Files can smooth the experience of developing apps, check out the examples linked above!

3 Likes

I have to say that you guys are just really fantastic! I needed exactly this feature for a project earlier this week, to store some JSON files that are read in by server-side scripts, and the solution I settled on was to triple-quote the JSON inside Python script files. Bravo, keep up the (as always) excellent work!

4 Likes

Also see “SimpleObject” column type under Column types. This stores JSON-format data, making it appear to your code as native Python values (scalars, lists, or dicts).

2 Likes