Using large files in a OVHcloud AI notebook

I’m trying to train an artificial intelligence model on OVHCloud AI Notebooks with the Common Voice’s dataset from Mozilla.

The problem is that this database has a 70 GB size. I have tried to download it on my computer, and then to export the database to the OVHCloud Object Storage so I can use it, but this is excessively long.

Is it possible to download directly this file on the notebook ? (My file is a tar.gz file)

Topic cloud ai jupyter data python

Category Data Science

1
answered at
  1. You can directly download this file in your OVHcloud AI notebook by running the following command in a cell of your notebook:
!curl -L "link-to-my-file" > my-file.tar.gz

Don't forget to put the link-to-my-file of your .tar.gz file in quotes.

  1. If you want to store my-file.tar.gz in the OVHcloud Object Storage (from your notebook), don't forget to connect a volume (named data for example) with an object container to your notebook before starting it.

Use this command in a cell of your notebook:

!curl -L "link-to-my-file" > my-file.tar.gz /workspace/data

This way, you can have your my-file.tar.gz file stored in an object container in the Object Storage.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.