Faster Data Transfer?

I'm doing a project which involves a small dataset (4GB). I'm trying to upload it to Paperspace to do some analysis but it's taking an absurd amount of time.

  • Using Gradient, it would upload maybe 40MB in 10min then stop uploading. This was done using the "upload" button in the jupyter notebook.
  • Using a P400 hourly computer I'm transferring at 800MBPS over SSH (well, through Nautilus), but it's still taking hours to transfer this small dataset.

What am I doing wrong? Is there a faster way?

Topic performance

Category Data Science

1
answered at

Most cloud systems support the command line mv utility to move files or folders.

Taking a command line approach allows for more control and visibility into tasks.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.