Career switch to Big Data Analytics

I am a 35 year old IT professional who is purely technical. I am good at programming, learning new technologies, understanding them and implementing. I did not like mathematics at school, so I didn't score well in mathematics. I am very much interested in pursuing a career in Big Data analytics. I am more interested in Analytics rather than Big Data technologies (Hadoop etc.), though I do not dislike it. However, when I look around in the internet, I see that, people who are good in analytics (Data Scientists) are mainly Mathematics graduates who have done their PHds and sound like intelligent creatures, who are far far ahead of me. I get scared sometimes to think whether my decision is correct, because learning advance statistics on your own is very tough and requires a of hard work and time investment.

I would like to know whether my decision is correct, or should I leave this piece of work to only intellectuals who have spend their life in studying in prestigious colleges and earned their degrees and PHDs.

Topic career

Category Data Science


I haven't seen this mentioned, but it's important to keep in mind that you may see a decrease in salary. I say this without knowing how much you make, but moving from (I assume) an experienced IT professional to an entry level data scientist level may not earn you as much.

Here's a link to the a portion of the 2015 Burtch Works study on Data Science salaries:

http://www.burtchworks.com/files/2015/05/DS-2015_Changes-in-Base-Salaries.pdf

As you can see, the median salary for level 1 individual contributors is 90k (across the nation). The full report has the breakdown based on region but again, assuming you're an experienced IT professional, you're probably making more than that.

Anecdotal story with n=1: One of my classmates in my DS masters program was an experienced Java developer with a house, family, etc. Although he was very interested in data analytics (paid for the program out of pocket) his potential salary doing data analytics wouldn't be able to support the lifestyle he currently had as a Java developer. As a result he essentially "wasted" his degree and went back to development. I would really hate to see that happen to more people.


May be it will be a little offtopic, but I'd like to highly recommend you to go through this MOOC https://www.coursera.org/course/statistics. This is a very good and clear introduction to statistics. It give you a base principles about core field in data science. I hope it will be a good start point for beginning friendship between you and statistics.


In my experience to have a PhD doesn't mean necessarily be good in the enviroment of data science company, I work as data scientist and I'm just an engineer but I've known some universitary teachers who works in collaboration with my company and sometimes I've said them that Their point of view was not right because despite of their ideas and reasonings were right they are not applicables to the company activities, so we had to modify some data models to make them usefull for the company and the results lost their value so we had to seek new models. What I mean is that Data Science is a multidisciplinar area so many different people working together is needed so I think that your skills could be very useful in a data scientist team, you only have to find where you fit ;)


This is a really strange question in my opinion. Why you're going to move in a new direction if you are not sure that you love this new direction or at least find it very interesting? If you do love Big Data, why do you care about the PhD intelligent creatures that are already in the field? The same amount of PhD creatures are in every area of IT. Please have a quick read at this very nice article http://www.forbes.com/sites/louisefron/2013/09/13/why-you-cant-find-a-job-you-love/ and then ask yourself if you love Big Data enough and you are ready to add your grain of sand to the mountain of knowledge


Keep in mind that "big data" is an increasingly trendy thing for a company to say they're involved in. Higher ups might read an article about it in HBR, and say to themselves, "I've got to get me some of that" (not that they're necessarily wrong).

What this means for you is that the advanced analytics isn't as necessary for that company as just getting something up and running might be.

Luckily for you, most of the components said companies might need are free. Moreover, I believe both Hortonworks and Cloudera have free "sandbox" virtual machines, which you can run on your PC, to play around with and get your bearings.

Advanced analytics on big data platforms are valuable, to be sure, but many companies need to learn to crawl before they can run.


You should look more into the infrastructure side of things if you don't like maths. The lower you go in the software stack, the further away you get from maths (of the data science sort). In other words, you could build the foundation that others will use to create the tools that will serve analysts. Think of companies like Cloudera, MapR, Databricks, etc. Skills that will come in handy are distributed systems and database design. You are not going to be become a data scientist without maths; that's a ridiculous notion!


Due to high demand, it is possible to start a career in data science without a formal degree. My experience is that having a degree is often a 'requirement' in job descriptions, but if the employer is desperate enough, then that won't matter. In general, it's harder to get into large corporations with formalized job application processes than smaller companies without them. "Knowing people" can get you a long way, in either case.

Regardless of your education, no matter how high demand is, you must have the skills to do the job.

You are correct in noting that advanced statistics and other mathematics are very hard to learn independently. It is a matter of how badly you want to make the career change. While some people do have 'natural talent' in mathematics, everybody does have to do the work to learn. Some may learn more quickly, but everybody has to take the time to learn.

What it comes down to is your ability to show potential employers that you have a genuine interest in the field, and that you will be able to learn quickly on the job. The more knowledge you have, the more projects you can share in a portfolio, and the more work experience under your belt, the higher level jobs that will be available to you. You may have to start in an entry level position first.

I could suggest ways to study mathematics independently, but that isn't part of your question. For now, just know that it's hard, but possible if you are determined to make a career change. Strike while the iron is hot (while demand is high).

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.