Can anyone recommend an alternative to the big 3 cloud computing alternatives? I know they're the best but I found them overly complicated because they cater to massive enterprises. The amount of set up required just to get an instance running is too much. I am looking for a multi GPU cloud offering which offers RAPIDS pre-installed. I see that Blazing SQL will have an offering soon, does anyone know of anything else that I could use in the mean …

I am trying to run a keras model on vast.ai using multiple GPUs. For that I am using keras.utils.multi_gpu_model , however I keep having this error: if multi_GPU and n_GPUs > 1: model = multi_gpu_model(model) AttributeError: module 'tensorflow_core._api.v2.config' has no attribute 'experimental_list_devices') I am using this default docker : Official docker images for deep learning framework TensorFlow Successfully loaded tensorflow/tensorflow:nightly-gpu-py3 I have also checked the available GPUs and all the GPUs are detected correctly: Any ideas? cheers

I just graduated with a non-tech background. As a person from a non-IT Background is DevOps a good career choice? Is it a good time to enroll in devops training?

I am working concurrently with multiple very large datasets (10s-100s of GBs)). I signed up for Colab Pro+ thinking it is the best option. However, I face a significant bottleneck in getting data into Colab. My options all seem very bad: Downloading from AWS (where the data is located) - very slow. Uploading data to Google Drive and mounting Drive using below code. This is also surprisingly very slow. from google.colab import drive drive.mount('/content/drive') Paying for a persistent server. Something …

I have a big doubt. I see a lot of blog posts where they say that you can use the Colab front-end to edit a local Jupiter Notebook However, I don't see the point. the actual advantage would be to use something like DataSpell or some local IDE, on a remote Notebook on Colab, and use the Colab Resources to do the computations, so you have: IDE level of suggestions (Colab is pretty slow compared to local IDE) cloud computing …

Let's say I have a Juypter Notebook I am working on where I am analyzing, visualizing, testing, etc. various Machine Learning Models with different hyperparameters on some arbitrary data set or I am developing Machine Learning libraries and toolkits, all in Python. Let's also say that some of the computations I want to run using that data set need a powerful GPU in order to complete in a reasonable amount of time (I guess powerful is relative, I am talking …

I followed the instructions from this article about creating a code-free machine learning pipeline. I already had a working pipeline offline using the same data in TPOT (autoML). I uploaded my data to AWS, to try their autoML thing. I did the exact steps that were described in the article and uploaded my _train and _test csv files, both with a column named 'target' that contains the target value. The following error message was returned as a failure reason: AlgorithmError: …

I am working on a deep learning CNN project. The dataset contains more than 500 classes and the classes have different numbers of items (images). For example, some of the classes have 5 images and some of the classes have 10 and some of the classes have 20 images and some of the classes have more then 20 images. Can I use this dataset to create the CNN model? Should the number of the images in each class be the …

Say I have one server with 10 GPUs. I have a python program which detects available GPU and use all of them. I have a couple of users who will run python (Machine learning or data mining) programs and use GPU. I initially thought to use Hadoop, as I find Yarn is good at managing resources, including GPU, and YARN has certain scheduling strategies, like fair, FIFO, capacity. I don't like hard-coded rules, eg. user1 can only use gpu1, user2 …

My question there is a way to run the Orange, not on a local computer, but on a remote server, even in the cloud? My notebook has only 8GB memory and limited number of CPU core, which is insufficient for real-world data mining task, however I can use much powerful (remote) server machines.

I am trying to implement some deep learning models with large amount of data around 10gigabyte. Although, my Laptop and Collab-free crashes when it tries to load them. Do you think it worths to buy collab-pro? Do you suggest any other solutions? But my worries are mostly about buying collab-pro is only for US and Canada while I am from Europe. Thanks in advance.

I want to train a deep model with a large amount of training data, but my desktop does not have that power to train such a deep model with these abundant data. I'd like to know whether there are any free cloud services that can be used for training machine learning and deep learning models? I also would like to know if there is a cloud service, where I would be able to track the training results, and the training …

I am working in a company but I am new in the field. We have a central server which is faster than my laptop (of course). So my goal is that I still use my laptop to do python analysis for machine learning but actual computation is happening in the central server so that the computing power is stronger. I am not considering any cloud service as we have a fast server computer. But I am clueless in this yet. …

With Hadoop 2.0 and YARN Hadoop is supposedly no longer tied only map-reduce solutions. With that advancement, what are the use cases for Apache Spark vs Hadoop considering both sit atop of HDFS? I've read through the introduction documentation for Spark, but I'm curious if anyone has encountered a problem that was more efficient and easier to solve with Spark compared to Hadoop.

So I am starting a project that has some code in it that has to do with machine learning (for example recommender system). At the moment I am thinking that if I want to deploy this application, should I look into the VM´s that are dedicated to Machine Learning applications or can a regular VM run these kinds of operations as well? What are the differences between them? Thanks for answers!

I have built a model in a cloud machine(google cloud). It runs for a few hours to a day. I need to scan some parameters like learning rate and batch size. How do I duplicate my compute engine, run the model with different parameters, collect the results and turn them off? Edit: I have a neural network model, it runs for 24 hours. Usually without cloud setting I would do a grid search: learning rate in {0.001, 0.003, 0.1} and …

I'm trying to solve a simple problem and seeking advise/suggestions - I have lot of paper documents in my drawers which I want to store on cloud (S3) as PDFs. My idea blueprint is - 1) scanning all of them using printer 2) store it locally in organised folder structure 3) upload them to S3 cloud - (python code using Boto3 ?) Questions - 1) Does this sound good to start with? Please advise. 2) Is S3 the best for …

I've set up a project on GCP with a Compute Engine VM and Storage Bucket. Access Scopes set to Allow full access to all Cloud APIs Have set a default Region and Zone I believe I have completed SSH authorization but not 100% sure given the documentation I've read. I have ~100,000 photos on both my local machine that I managed to upload to a bucket and am now trying to run my algo using the VM connected jupyter lab. …

I have been trying to use python 3 for text mining on a 650 MB csv file, which my computer was not powerful enough to do. My second solution was to reach out to google cloud. I have set up my VMs and my jupyter notebook on google cloud, and it works perfectly well. The problem, however, is that I am in constant fear of getting disconnected. As a matter of fact, my connection with google server was lost a …

I have two different files and on the first, I tried to save data to file as: np.save(open(Q1_TRAINING_DATA_FILE, 'wb'), q1_data) On second file, i'm trying to load it the same way using: q1_data = np.load(open(Q1_TRAINING_DATA_FILE, 'rb')) I then get the error: FileNotFoundError: [Errno 2] No such file or directory: 'q1_train.npy' I searched my google drive but couldn't find this file. Platform: https://research.google.com Edit: I'm trying to run below Kaggle problem on Colab platform. The author has two files (Jupyter and …