Data Engineer is the person responsible for creating the infrastructure for collating data from various sources, cleaning it, transforming it and storing it so that it can be used by other people in the team. The work includes (but is not limited to) connecting various data sources, understanding how the information gets captured and whether the hardware and the software can support the business requirements. If you want to enter into data engineering, you can look out for Business Intelligence roles to start with. You can also upskill yourself in Hadoop, HDFS, Databases, SQL, NoSQL, Spark and you should be ready for an entry role as a data engineer.
Skills and tools: Hadoop, MapReduce, Hive, Pig, MySQL, MongoDB, Cassandra, Data streaming, NoSQL, SQL, programming.
Average Salary: requiresThe average pay for a Data Scientist / Engineer is Rs 510,000 per year. Most people move on to other jobs if they have more than 10 years’ experience in this field. The skills that increase pay for this job the most are Hadoop, SQL, and Apache Hadoop.
The work for data scientist typically starts where the job of data engineer ends. A data scientist is responsible for extracting insights from the data using various tools and techniques. You will need a combination of Mathematics / Statistics, Coding skills and business domain knowledge to be successful as a data scientist.
The problem-solving skills of a data scientist require an understanding of traditional and new data analysis methods to build statistical models or discover patterns in data. For example, creating a recommendation engine, predicting the stock market, diagnosing patients based on their similarity, or finding the patterns of fraudulent transactions.
Data Scientists may sometimes be presented with big data without a particular business problem in mind. In this case, the curious Data Scientist is expected to explore the data, come up with the right questions, and provide interesting findings! This is tricky because, in order to analyze the data, a strong Data Scientists should have a very broad knowledge of different techniques in machine learning, data mining, statistics and big data infrastructures.
Skills and tools: Python, R, Scala, Apache Spark, Hadoop, data mining tools and algorithms, machine learning, statistics.
Data Analysts are experienced data professionals in their organization who can query and process data, provide reports, summarize and visualize data. They have a strong understanding of how to leverage existing tools and methods to solve a problem and help people from across the company understand specific queries with ad-hoc reports and charts.
However, they are not expected to deal with analyzing big data, nor are they typically expected to have the mathematical or research background to develop new algorithms for specific problems.
The role is as much art as science. While you can learn the tools from the web and other resources, you will only know how good you are when you solve a real problem.
Skills and Tools: Data Analysts need to have a baseline understanding of some core skills: statistics, data munging, data visualization, exploratory data analysis, Microsoft Excel, SPSS, SPSS Modeler, SAS, SAS Miner, SQL, Microsoft Access, Tableau, SSAS.
Please write a comment if you need help to understand about it more in detail.