Can ChatGPT Solve My Data Science Project?

Should data scientists be afraid of AI taking over their jobs? Let’s test this by asking ChatGPT to solve a data science problem and see what happens.

We were told that AI would take over jobs and replace data science with machines. I don’t know about you, but I don’t want to be laid off. But before going into panic mode, let’s see if the current most popular machine is able to replace me as a data scientist.

Yes, I’m talking about the one and only ChatGPT! Let’s see how good it is and if it can replace me one day.

Say, can it handle a data science project from start to finish?

Project for Testing ChatGPT

I’ll use Pearson's Student Performance Analysis project to test ChatGPT. Pearson is an educational publisher that publishes and distributes academic books.

The project is a take-home assignment, and, as you know, these assignments are comprehensive tests of data science skills. Data scientists often have to solve them to get a job. So, the same criteria should apply to ChatGPT. We classified the project as an easy business analyst project. ChatGPT shouldn’t have problems when solving this, right? Let’s see!

Project Dataset

The data set consists of data collected from two Portuguese schools focusing on student achievement in mathematics and the Portuguese language. The purpose of this project is to uncover insights that could help understand the variables that contribute to and predict student academic success.

Data Science Project Fails by ChatGPT

Now, let's talk about the ChatGPT fails I encountered.

Fail #1: Legal Issues

Here, I'm talking about ChatGPT’s legality, not ability.

If you ask ChatGPT to write code and you approve that code to be pushed to your company's production platforms, you could be exposed to an ugly lawsuit. That's because ChatGPT freely pulls code snippets from data on which it was trained.

And the data it was trained on? Well, these were all external data sources, so you can’t tell where this code is coming from. That means you have no clue whether that code is licensed or copyrighted.

There’s also the opposite legal problem. If you copy and paste code owned by your company so that, say, ChatGPT can help you debug it, you're essentially non-compliant with your company's policy. Again!

Why? Because all companies hate it when their IP leaves the safety of the company's firewalls.

Do you know what companies also hate? You or anybody uploading a data set for ChatGPT to help you with a project. Again, you’re non-compliant if that data set actually belongs to the company you're working for. And, again, you're distributing company IP past their firewalls to be stored by OpenAi.

So, right off the bat, if you're working at a company, there's already a limited amount of things you can do with ChatGPT due to copyright and compliance regulations.

Fail #2: Statistical Test Mishaps

ChatGPT is not very good at anything involving statistical tests. This data project is a business analysis, so there will be many statistical tests run on the data.

I ask ChatGPT to write some Python code to run these tests.

ChatGPT draws a histogram showing a distribution of the final grades for the Portuguese language.

It also correctly chooses to perform a Chi-Square test, also used in the project’s official solution. So far, so good!

It calculates the Chi-Square of 11.439 and the p-value of 0.00072, which shows that there’s a statistically significant association between gender and grades.

But that’s entirely off the official solution’s values shown below!

We get the Chi-Square statistic of 21.91 and the p-value of 0.15. So, the conclusion is completely the opposite of what ChatGPT came up with: gender is not a statistically significant variable.

However, ChatGPT thinks that the gender variable significantly impacts grades.

The question is: Why? Why did it mess up this relatively simple calculation?

It seems that the answer is given by ChatGPT itself.

You can find more information about that answer in the article Why ChatGPT Can’t Do Basic Math by Rafael Moscatel on Medium.

Basically, ChatGPT admits that it doesn't use a calculator for mathematical computations. It generates responses based on patterns in natural language and the documents on which it was trained. They can be and, of course, are full of errors. As a result, ChatGPT's calculations can also be wrong.

Can you trust outputs if you don't think ChatGPT can perform these calculations correctly? I don't think so!

The additional benefit of a human component in this work is an extra layer of sanity checking. The data scientists can figure out what correct statistical test to use and then validate the assumptions to ensure that you're using the right test given the data set. This is something ChatGPT does not do nor explicitly mentions that it does.

As a data scientist, you can’t skip that! It's one of the most important pre-processing steps a data scientist must do to ensure their results are accurate, the critical human touch that AI forgets to do.

Fail #3: Understanding Stakeholder Priorities

Let’s say this project aims to build a tool that grades students fairly by adjusting test scores based on variables it analyzes in the data set, for example, adjusting scores based on a student's educational level, study time, gender, etc.

Does ChatGPT know how to incorporate these variables and these priorities correctly? In theory, it does.

It rightly includes the fairness and bias assessment stage in the model-building process, as you can see in its reply above.

But when we run the analysis using ChatGPT, it avoids direct answers regarding actual decision-making and incorporating the priorities of stakeholders.

I asked it to explain why female students have higher grades than males and to include the actual data set when explaining it. I also want to know how this impacts the fairness of a grading tool I want to build.

But ChatGPT gives me a generic answer that sounds like it's not looking at the data set when providing that answer.

It also includes some example code that is completely unusable.

Okay, I have to be more specific. No problem with that. I told ChatGPT to analyze the above contributing factors on the actual data that I provided.

It performs analysis. It attaches the Python code for that analysis. And it gives me implications for fairness that I can use for the tool.

But it expects me to write code incorporating all these variables together. Yes, after I asked ChatGPT to include the stakeholders' priorities, ChatGPT made me include them. We are getting a little bossy, aren’t we?

To be honest, ChatGPT does a great job of communicating these factors and how we may build a tool that fairly incorporates these issues. But it's not going to take my job because it doesn't write code to finish off the project. Instead, it's just acting as a mentor or a teammate for my project.

So, now I’m going to force ChatGPT to write the code for me. With this prompt, it seems that I've reached ChatGPT’s breaking point.

It does a good job of writing code to process the data and prepare it for the model-building step.

But when it comes to writing code for the fairness considerations it gives me nothing but comments.

In conclusion, you can't use ChatGPT to write code incorporating priorities as that involves actual decision-making. As you've seen, ChatGPT obviously wants to avoid making these decisions.

I'd say that's not a bug but a feature for OpenAI to avoid being sued by relying on ChatGPT’s decision.

Fail #4: Handling Novel Problems

ChatGPT is not creative! It just reshuffles what it's learned from the training data it has found on the Internet. Try to ask it something that it hasn't encountered before.

For example, let's think outside the box and include socioeconomic data to understand how variables like internet access, parental education, and family size affect the student's grades and performance.

When I introduced this variable, ChatGPT's response was limited to the data it had. Instead, it used proxy variables to vaguely indicate a student's economic status.

It was written at a high level and didn't even suggest some of the better indicators, like wages. So, I had to do it myself, as you can see below.

Once I did, it correctly realized that the existing data set did not have wage data and asked for that additional data.

Then, I uploaded the data set containing wages by ZIP codes and wanted ChatGPT to compare the test scores based on the wages. In reality, though, I expect ChatGPT to be able to find the data itself because it is publicly available—it's not hidden behind a paywall or anything like that.

So, what happened? It seems ChatGPT entered the loop it can't exit. It realized that this student data set doesn't have ZIP code information, so it can't be merged with the new data set I gave.

At the same time, its proposed solution is to merge the two datasets. The solution to the problem is the problem itself!?

Can ChatGPT try a different way? Can it use different data? Can it find a different data set online that it can work with? Nope. You have to tell ChatGPT exactly what to do.

At that point, what is the point? I can just do this work myself!

Conclusion

In conclusion, the success of the project relies entirely on you.

ChatGPT can help with fundamentals often required at the beginning of projects, such as performing Exploratory Data Analysis (EDA).

But anything further than that, you must tell it exactly what to do and provide all the information you need.

In that regard, ChatGPT’s success depends on how experienced you are!

It will never replace you, and because of that, your job is safe.