How to download dynamic files created during work on Google Colab?

I have two different files and on the first, I tried to save data to file as:

np.save(open(Q1_TRAINING_DATA_FILE, 'wb'), q1_data)

On second file, i'm trying to load it the same way using:

q1_data = np.load(open(Q1_TRAINING_DATA_FILE, 'rb'))

I then get the error:

FileNotFoundError: [Errno 2] No such file or directory: 'q1_train.npy'

I searched my google drive but couldn't find this file.

Platform: https://research.google.com

Edit: I'm trying to run below Kaggle problem on Colab platform. The author has two files (Jupyter and nbs) - one to prepare and 2nd to train. The step on nb1 where it's creating some files - which later to be consumed by file 2 is where I'm struck.

https://github.com/bradleypallen/keras-quora-question-pairs/blob/master/quora-question-pairs-training.ipynb

Topic colab cloud-computing deep-learning neural-network machine-learning

Category Data Science


Based on what I've seen and experienced, the best way is to store and retrieve your data from your drive account. Actually your question is a bit unclear but first I say, try to use the following command to see the current files in your directory, although I guess each 12 hours they all would be deleted automatically.

!ls

Anyway I recommend the following instructions:

Use the following code for having permission to access to your drive account:

!pip install -U -q PyDrive

import tensorflow as tf
import timeit

config = tf.ConfigProto()
config.gpu_options.allow_growth = True

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials

# Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)

Use the following code to get the id of contents in your drive:

file_list = drive.ListFile({'q': "'root' in parents and trashed=false"}).GetList()
for file1 in file_list:
  print('title: %s, id: %s' % (file1['title'], file1['id']))

Put the id of the desired file, e.g. a typical text file, in the content of the following dictionary with id key:

downloaded = drive.CreateFile({'id': 'the id of typical text file'})
file = downloaded.GetContentString()
print('Downloaded content "{}"'.format(len(file)))

Till now you have copied the text file, then you have to write it in your Colab disk using the following code:

text_file = open("your desired name.txt", "w")
text_file.write(file)    
text_file.close()

Create & upload a file.

uploaded = drive.CreateFile({'title': 'filename.csv'})
uploaded.SetContentFile('filename.csv')
uploaded.Upload()
print('Uploaded file with ID {}'.format(uploaded.get('id')))

Downloading from Colab without Uploading to drive

from google.colab import files
files.download('your typical h5 file or what ever.h5')

For more information about transferring different data formats there are more explanations in the notebook provided with Colab.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.