sbadirli/grownet

grownet's Issues

Cerrective step described on paper

Hi Sarkhan,

I have a question about the implementation w.r.t what the paper describes. The paper says this on Section 6.2 (page 7)

Among all components of the model, the corrective step is presumably the most vital one. In this step, the parameters of all weak learners, that are added to the model, are updated by training the whole model on the original inputs without the penultimate layer features

If I understood correctly, the model shouldn't use the penultimate layer, that is, no concatenation should take place. But during the corrective step in the regression experiment, for instance, forward_grad is called, which uses the penultimate layer's output.

def forward_grad(self, x):
    if len(self.models) == 0:
        return None, self.c0
    # at least one model
    middle_feat_cum = None
    prediction = None
    for m in self.models:
        if middle_feat_cum is None:
            middle_feat_cum, prediction = m(x, middle_feat_cum)
        else:
            middle_feat_cum, pred = m(x, middle_feat_cum)
            prediction += pred
    return middle_feat_cum, self.c0 + self.boost_rate * prediction

It this correct? If it is, could you kindly point out what I am missing?

Cheers,
Darley

Support for Pytorch DataLoader

Is it working only with LibSVMdata?

Dear all, your data link is invalid now, can you provide a new link, thank you very much!

Dear all, your data link is invalid now, can you provide a new link, thank you very much!

Regarding NDCG Gain used in the results for ranking

Hi,

I was wondering what gains of NDCG you'd used for ranking results in your Arxiv version of the paper. Did you use identity gain, or exp2 gain? Wouldn't both have different effects on performance, and finally test results?

dataset not found

dataset 404
https://drive.google.com/open?id=1NnBpwvfSdqs-lRb5UFIC-q8P455o3vO3

Boosting Rate in Regression

Hi,
I was wondering why in line 148 of GrowNet/Regression/main_reg_cv.py
loss = loss_f1(net_ensemble.boost_rate * out, grad_direction) # T
the boosting rate needed to be multiplied with the outputs of the current model? With traditional boosting ideas it seems like training should be done directly on residuals. without the boosting rate. Is there something here that I'm missing? Thanks!

Why use splinear replace nn.Linear?

I do not know what is difference between splinear and nn.Linear?

About the first and second order gradient calculations in classification example

Hi there,

In the implementation of classification example in Classification/main_cls_cv.py, the first and second order gradient calculations are calculated with the logic below

h = 1/((1+torch.exp(y*out))*(1+torch.exp(-y*out))), grad_direction = y * (1.0 + torch.exp(-y * out))

However, in the paper, the formula to calculate those gradients are $h_i = \frac{ 4 y^2_i e^{2y_i \hat{y}^{(t-1)}_i}}{\left (1+e^{2y_i \hat{y}^{(t-1)}_i} \right )^2}$, and $\tilde{y}_i = \frac{ y_i (1 + e^{-2y_i \hat{y}^{(t-1)}_i}) } {2} $.

Is there anything I may miss? Thanks.

How is boost_rate used?

Hello,

Thanks for the interesting work. One question about the dynamic boost rate in each epoch. Is it really used in ensemble network forwarding?