Simple question about pipelines

[ageron]

Simple question about pipelines

[ageron]

Hi @Ayeness ,
Thanks for your kind words! :)
Yes, you are correct: you do not have to build a pipeline, you can just apply the StandardScaler manually like this:

X_train_scaled = scaler.fit_transform(X_train)
X_valid_scaled = scaler.transform(X_valid)
X_test_scaled = scaler.transform(X_test)

lin_reg = LinearRegression()
lin_reg.fit(X_train_scaled, y_train)
score = lin_reg.score(X_valid_scaled, y_valid)

That said, you could also create a Pipeline containing the StandardScaler and the LinearRegression: it would simplify the code slightly:

pipeline = Pipeline([
  ("scaler", StandardScaler()),
  ("lin_reg", LinearRegression())
])
pipeline.fit(X_train, y_train)
score = pipeline.score(X_valid, y_valid)

Hope this helps!

[ageron]

Hi @Ayeness ,

Great work! You're almost there. The pipeline takes care of transforming the inputs, so you don't need to do it explicitly. You can replace these two lines:

some_data_prepared = pipeline.transform(some_data)
print('Prediction:', lin_reg.predict(some_data_prepared))

With this line:

print('Prediction:', lin_reg.predict(some_data))

Hope this helps!

[ssilverac]

Hi @Ayeness ,
Great work! You're almost there. The pipeline takes care of transforming the inputs, so you don't need to do it explicitly. You can replace these two lines:
some_data_prepared = pipeline.transform(some_data)
print('Prediction:', lin_reg.predict(some_data_prepared))
With this line:
print('Prediction:', lin_reg.predict(some_data))
Hope this helps!

thank you for the information @ageron
i just wanted to make this comment in case anyone in the future stumbles upon this thread, you have a small typo.

we should be replacing those first two lines with

print('Prediction:', pipeline.predict(some_data))

not
lin_reg.predict(some_data)

[ssilverac]

@ageron thanks for the advice.
I have tried to implement your suggestion with the following code

from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('lin_reg', LinearRegression())
])


pipeline.fit(boston_df, boston_df_target)

but then when I try to test this on some instances from the training set:

some_data = boston_df.iloc[:5]
some_targets = boston_df_target.iloc[:5]

some_data_prepared = pipeline.transform(some_data)
print('Prediction:', lin_reg.predict(some_data_prepared))
print('Labels:', list(some_targets))

I get the following error message

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-131-d8c9add2e0df> in <module>
      2 some_targets = boston_df_target.iloc[:5]
      3 
----> 4 some_data_prepared = pipeline.transform(some_data)
      5 print('Prediction:', lin_reg.predict(some_data_prepared))
      6 print('Labels:', list(some_targets))

~\Anaconda3\envs\ml_book\lib\site-packages\sklearn\pipeline.py in transform(self)
    532         # XXX: Handling the None case means we can't use if_delegate_has_method
    533         if self._final_estimator != 'passthrough':
--> 534             self._final_estimator.transform
    535         return self._transform
    536 

AttributeError: 'LinearRegression' object has no attribute 'transform'

do you know what a possible reason for this error could could be?

[ageron]

To be clear, under the hood:

• pipeline.fit(boston_df, boston_df_target) will do lin_reg.fit(scaler.fit_transform(boston_df), boston_df_target).
• and pipeline.predict(some_data) will do lin_reg.predict(scaler.transform(some_data)).

[ssilverac]

Thank you for the insight. I will try this out