Issues while running the colab file for a modified dataset (modified Housing csv)

[Pranav110500]

Issues while running the colab file for a modified dataset (modified Housing csv)

[Pranav110500]

@askoshiyama @konstantinklemmer Hello there! Could either of you give some idea regarding the aforementioned issue that I'm facing? Any help would be appreciated to help me in understanding where I'm going wrong. Thank you.

[konstantinklemmer]

Hey! Sorry for the delay here; I will have a look into this issue in the coming week, thanks for pointing it out.

[konstantinklemmer]

Alright! So @Pranav110500, I can run the whole notebook no problem. The only thing I changed is to load the dataset directly from the github, so replacing

df = pd.read_csv("/content/drive/MyDrive/Houses1000.csv")

with

df = pd.read_csv("https://github.com/konstantinklemmer/spacegan/files/9315984/Houses1000.csv")

Can you check if this works for you? Were you running the notebook locally of via Google Colab?

[Pranav110500]

Hey there @konstantinklemmer, thanks a lot for the suggestion, as the code does execute and I am able to visualize it while using the dataset directly from github. My only query remaining is how is it that running the file locally with the csv data stored in my drive is displaying the slicing error? I want to know about this as there are some other csv files on which I'm trying to run the model, such as with the one given below (although this one has quite less the number of data points).

SoilValuesLatLong.csv

If you could let me know as to how to run the file without any errors locally with just my drive being mounted, it'll be really helpful. Unless if at all there is some other issue regarding it which I'd be glad to get to know of from your side.

[Pranav110500]

@konstantinklemmer Hey Sir, could you let me know how to resolve this problem I'm facing? I'd want to know how to locally run the files to execute them for any dataset, such as the one I've added in the trailing comment.

[konstantinklemmer]

I don't know what the problem is exactly as I don't have a version of your notebook that would contain the error so I currently cannot help. The above linked notebook does not have the error.

[Pranav110500]

So like, the main issue is shown in the colab file given below, with the other csv file that I'm trying to run. I am not able to understand how to have this issue clarified. It's similar to the one I originally shared with the houses1000 csv file. It seems like it is causing some problems while running the file locally with the csv mounted on my personal drive.

I'm attaching the csv and colab links for your reference here once, so that you could probably have a better idea of the main issue.

Colab link:- https://colab.research.google.com/drive/1i-nWyNUJarmdrZQfkYVtJtkd79QOaWhU?usp=sharing

SoilValuesLatLong.csv

[konstantinklemmer]

So it seems this line within the compute_metrics() function is the problem:

df_pred_agg_metrics.loc[i, pf] = perf_metrics_dict[pf](np.array(df_agg_pred["avg"][[i]].values, dtype="float64"), target, wdist)

We're not averaging over predictions here so we need to fix that. This should help:

df_pred_agg_metrics.loc[i, pf] = perf_metrics_dict[pf](np.array(df_agg_pred["avg"][[i]].values, dtype="float64").mean(1), target, wdist)

[Pranav110500]

Hey, so I've gone through the code I had attached in the previous comment, and went added the change to the compute_metrics function as per your suggestion. However, it gives an error of "ValueError: operands could not be broadcast together with shapes (11,) (11,4) ". Since this is an error related to broadcasting, I tried to rectify it by using expand_dims (in the rmse function), however I'm still stuck with it.
I'd appreciate if you could let me know by running the notebook on how to rectify this. Also, it could be something trivial, but I'm really not able to get around with it so sorry for disturbing with that.

[konstantinklemmer]

This seems like a simple broadcasting error, where you can't compute the metric (e.g. MSE) between an array of dimensionality [11] and one of dimensionality [11,4]. You should again be able to average over the second dimension of the second array array2.mean(1) to rectify this.