enoche/mmrec

A Toolbox for MultiModal Recommendation. Integrating 10+ Models...

mmrec's Issues

About user_graph_dict_file

Hi, I meet some problems when running the codes caused by missing the file "user_graph_dict_file.npy", I wonder how to get this file, thank you!

why `calculate_loss` of ItemKNNCBF is return `torch.tensor(0.0)`

https://github.com/enoche/MMRec/blob/master/src/models/itemknncbf.py#L67-L69

    def calculate_loss(self, interaction):
        tmp_v = torch.tensor(0.0)
        return tmp_v

ItemKNNCBF	Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches

ModuleNotFoundError: No module named 'common.encoders', LightGCN_Encoder

when I try to run SELFCFED_LGN, i.e. SelfCF (?). the code raises an error whose message is as follows:

    from common.encoders import LightGCN_Encoder
ModuleNotFoundError: No module named 'common.encoders'

It means that LightGCN_Encoder is not existed.

Problem about dataset process

Thank you very much for your excellent work.

When I process the dataset I have a problem about 0rating2inter.ipynb, Block13
should we resort to this data by timestamp, before this code:

split_timestamps = list(np.quantile(df[ts_id], split_ratios))

Rely on this framework to add CF models.

Thank you very much for your excellent work.

Now I would like to rely on your framework to implement a couple of CF-based recommendation models, could you please tell me if there is any guideline to add the models?

Sincerely.

Product ID and userID for the Baby/Sports/Elec data

Hi, can I get the orignal product iD and user ID in your preprocessed Baby/Sports/Elec dataset in this page:
https://github.com/enoche/MMRec/tree/master/data

thanks

The dimension of features

Hello! Thanks for your contributions to multi-modal recommendation!

When I set a breakpoint in the forward function, the dimension of features not involve the batch_size. For example, in this picture, the dimension of [image_feats] and [item_embeds] is [7050, 64].

Is there any way to add the [batch_size] dimension before entering the training phase, i.e. during the forward function?（i.e. , [batch_size, 7050, 64]?

About user_graph_dict_file

Dear @enoche,

Thank you for updating this repo. Actually, I am encountering some difficulties with preprocessing user_graph_dict_file. Do you mind sharing it via google drive?

Best,
heejkoo

What is to be used as textual modality input when extracting the textual feature (of Amazon dataset)?

Hi, I am a bit confused about what to be used to extract the textual feature.

If as mentioned in your paper, the text description will be used:

The metadata of the amazon datasets contains the text description and the URL of images that could be used as textual and visual modalities.

but if as mentioned in the code comments, the title, brand, category, and description will be used:
# sentences: title + brand + category + description | All have title + description

Could you please tell me which one I should refer to?

Thanks for your contributions to the community of multimodal recommendation!

FileNotFoundError: [Errno 2] No such file or directory: 'data/baby/user_graph_dict.npy

I run the code:

python main.py --model DualGNN --dataset baby

and get the error:

 INFO =========1/25: Parameters:['aggr_mode', 'learning_rate', 'reg_weight', 'seed']=('add', 0.1, 0.1, 999)=======
Traceback (most recent call last):
  File "/data3/xx/workspacebb/MMRec/src/main.py", line 27, in <module>
    quick_start(model=args.model, dataset=args.dataset, config_dict=config_dict, save_model=True)
  File "/data3/xx/workspacebb/MMRec/src/utils/quick_start.py", line 74, in quick_start
    model = get_model(config['model'])(config, train_data).to(config['device'])
  File "/data3/xx/workspacebb/MMRec/src/models/dualgnn.py", line 53, in __init__
    self.user_graph_dict = np.load(os.path.join(dataset_path, config['user_graph_dict_file']), allow_pickle=True).item()
  File "/data3/xx/workspacebb/miniconda3/lib/python3.10/site-packages/numpy/lib/npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'data/baby/user_graph_dict.npy

About the result of MVGAE

Hi, recently I'm running MVGAE using the framework but got very low results, for example on the baby dataset, the recall is nearly zero, my settings are: learning rate=0.001, beta=1, reg_weight=0.001, epochs=200. Have you got the right results of MVGAE using the framework? Thank you!

About the implementation of saving and loading models

Hi, I just found the "saved" variable in the function Trainer.fit is not yet used when I wanted to save and load the best model parameters for further research.

    def fit(self, train_data, valid_data=None, test_data=None, saved=False, verbose=True):
        r"""Train the model based on the train data and the valid data.

        Args:
            train_data (DataLoader): the train data
            valid_data (DataLoader, optional): the valid data, default: None.
                                               If it's None, the early_stopping is invalid.
            test_data (DataLoader, optional): None
            verbose (bool, optional): whether to write training and evaluation information to logger, default: True
            saved (bool, optional): whether to save the model parameters, default: True

        Returns:
             (float, dict): best valid score and best valid result. If valid_data is None, it returns (-1, None)
        """

Could you please update the code for the saving and loading functions of the best model parameters? That would be very helpful for my current research.

Thanks again for your great contributions!

Quickstart script

Hi. Is there a sample or template I can use to jump start the experiments?

Why are they all based on GNN models?

RuntimeError: found negative index -562949566989948 for dim 1

when running DRAGON on Elec., knn_ind is abnormal whose value is shown below:

tensor([[                 0,                  0,                  0,
          ...,                  0,                  0,
                          0],
        [                 0,    133148282176512,                  0,
          ...,                  0, 544373700175071744,
           3978033069303296],
        [283817417514181632,   4470614405284095,   4470614278539776,
          ...,     96757024170496,   4472813428539647,
           4636747909554688],
        ...,
        [             62998,               1757,               3143,
          ...,              23576,              34277,
                      37654],
        [             62999,              13636,              20836,
          ...,              28260,              33679,
                      59455],
        [             63000,              61593,              54526,
          ...,              10903,              60347,
                      57793]], device='cuda:0')

The error message is below:

gpu_id=0
use_gpu=True
seed=[999]
data_path=../data/
inter_splitting_label=x_label
filter_out_cod_start_users=True
is_multimodal_model=True
checkpoint_dir=saved
save_recommended_topk=True
recommend_topk=recommend_topk/
embedding_size=64
weight_decay=0.0
req_training=True
epochs=1000
stopping_step=20
train_batch_size=2048
learner=adam
learning_rate=[0.001]
learning_rate_scheduler=[1.0, 50]
eval_step=1
training_neg_sample_num=1
use_neg_sampling=True
use_full_sampling=False
NEG_PREFIX=neg__
USER_ID_FIELD=userID
ITEM_ID_FIELD=itemID
TIME_FIELD=timestamp
field_separator=
metrics=['Recall', 'NDCG', 'Precision', 'MAP']
topk=[5, 10, 20, 50]
valid_metric=Recall@20
eval_batch_size=4096
use_raw_features=False
max_txt_len=32
max_img_size=256
vocab_size=30522
type_vocab_size=2
hidden_size=4
pad_token_id=0
max_position_embeddings=512
layer_norm_eps=1e-12
hidden_dropout_prob=0.1
end2end=False
hyper_parameters=['aggr_mode', 'reg_weight', 'learning_rate', 'seed']
inter_file_name=elec.inter
vision_feature_file=image_feat.npy
text_feature_file=text_feat.npy
user_graph_dict_file=user_graph_dict.npy
feat_embed_dim=64
n_mm_layers=1
n_layers=2
knn_k=10
mm_image_weight=0.1
aggr_mode=['add']
reg_weight=[0.0001]
info=normal-seed_None
model=DRAGON
dataset=elec
valid_metric_bigger=True
device=cuda


19 Jul 16:13    INFO elec
The number of users: 192403
Average actions of users: 8.779426516218562
The number of items: 63001
Average actions of items: 26.81208234789924
The number of inters: 1689188
The sparsity of the dataset: 99.98606462355167%
19 Jul 16:13    INFO 
====Training====
elec
The number of users: 192403
Average actions of users: 6.519861956414401
The number of items: 62989
Average actions of items: 19.91523916874375
The number of inters: 1254441
The sparsity of the dataset: 99.98964920548602%
19 Jul 16:13    INFO 
====Validation====
elec
The number of users: 192403
Average actions of users: 1.098194934590417
The number of items: 47923
Average actions of items: 4.4090728877574445
The number of inters: 211296
The sparsity of the dataset: 99.99770841780649%
19 Jul 16:13    INFO 
====Testing====
elec
The number of users: 192403
Average actions of users: 1.161369625213744
The number of items: 48794
Average actions of items: 4.57947698487519
The number of inters: 223451
The sparsity of the dataset: 99.99761985156943%
19 Jul 16:14    INFO 

=================================


19 Jul 16:14    INFO =========1/1: Parameters:['aggr_mode', 'reg_weight', 'learning_rate', 'seed']=('add', 0.0001, 0.001, 999)=======
Traceback (most recent call last):
  File "main.py", line 34, in <module>
    quick_start(model=args.model, dataset=args.dataset, config_dict=config_dict, save_model=True, seed=args.seed, alpha1=args.alpha1, alpha2=args.alpha2, eta=args.eta, estart=args.estart, alter=True if 'alter' in args.info else False)
  File "xxx/MMRec/src/utils/quick_start.py", line 77, in quick_start
    model = get_model(config['model'])(config, train_data).to(config['device'])
  File "xxx/MMRec/src/models/dragon.py", line 75, in __init__
    indices, image_adj = self.get_knn_adj_mat(self.image_embedding.weight.detach())
  File "xxx/MMRec/src/models/dragon.py", line 173, in get_knn_adj_mat
    return indices, self.compute_normalized_laplacian(indices, adj_size)
  File "xxx/MMRec/src/models/dragon.py", line 176, in compute_normalized_laplacian
    adj = torch.sparse.FloatTensor(indices, torch.ones_like(indices[0]), adj_size)
RuntimeError: found negative index -562949566989948 for dim 1

Request to Add "MicroLens" Dataset to MMRec Framework

Hi @enoche , thanks for your great contributions to the recommender system community !

I noticed that MMRec currently includes three datasets (Baby, Sports, Elec) in prepared formats. Could you add a newly multi-modal short-video dataset "MicroLens" to further promote the research?

I have already processed the dataset files (u_id_mapping.csv, i_id_mapping.csv, i_id_mapping.tsv, image_feat.npy, text_feat.npy, microlens.inter) to fit your framework. You can find them in this Google Drive link: https://drive.google.com/drive/folders/14UyTAh_YyDV8vzXteBJiy9jv8TBDK43w?usp=drive_link.

Thanks for your support !

Some problems about data-preprocessing module

Your work is impressive and helpful for beginners like me to understand the multi-modal recommendation system. But I encountered two problems when running your data-preprocessing module: 1. Your data-preprocessing module seems unable to handle the "complete review data"; I would like to know if you can release these codes; 2. During the remapping process in the "2reindex-feat.ipynb", I found that there would be a mismatch, so the map_dict would be empty. Except for processing the string "true" and "false" in the parse function and modifying the file path, I made no changes. Is there anything wrong?

Amazon product data

Hello author, I would like to know how the image_featuresSportsAnd-Outdoors.b file for the extracted image feature in the preprocessing section of the dataset, 3feature encoder ipynb, was obtained. Do you have any code to extract image features and save them as. b files

Converting movielens dataset into the format of amazon baby dataset

In the file baby.inter, please tell what does x_label represents.
@enoche

Library versions

Would it be possible to provide a requirements file? This would help prevent potential version conflicts. Thank you!

Lack of the necessary script tools/generate-u-u-matrix.py for running DRAGON

Hi, could you please upload the script mentioned in dragon.py?

# user-graph need to be generated by the following script
# tools/generate-u-u-matrix.py
self.user_graph_dict = np.load(os.path.join(dataset_path, config['user_graph_dict_file']), allow_pickle=True).item()

Thanks!

What was the first work that used not only text but images for recommendations?

Hi, I am working on multimodal recommendations now. I am a little bit confused about what is the first work that used not only text but images for review-based recommender systems? I saw many works just use reviews but not images.

I have some confusion about regarding user interacted items as positive samples

In the open-source datasets, such as the Baby dataset, I have observed that there are interaction records with very low ratings in the baby.inter (e.g., 6 5374 1.0 1374019200 2 has a rating of 1). But I see that in the metric calculation part of the codes, the user interacted items are considered positive samples and the items that have not interacted are considered negative samples. However, the current scenario will also consider some low-rated items as positive samples. Does this situation of treating low-rated items as positive samples introduce bias? Does it affect the model training process?

Data splitting

Hi,

Firstly thanks for your work for multi-modal recommendation.
I downloaded the Baby dataset and found that the trainsize:validsize:testsize is 118551:20559:21682,
which appears not the 7:1:2 ratios suggested in the 0rating2inter.ipynb.

Thanks!

problem about freedom.py

MMRec/src/models/freedom.py

Lines 81 to 83 in 3de1ff7

	sim = torch.mm(context_norm, context_norm.transpose(1, 0))
	_, knn_ind = torch.topk(sim, self.knn_k, dim=-1)
	adj_size = sim.size()

When we create this graph using GPU, it may cause a problem like this:

sim size: torch.Size([63001, 63001])
sim dtype: torch.float32
knn_ind: tensor([[7585306885142836339, 3991370661542720611, 3486688227326894121, ..., 140594981698912,
140594981698912, 4294967295],
[354038449242118, 1271310336001, 8387195064482211950, ..., 7957688336701796217,
8319958742476218724, 7795558767364238639],
[ 3304159867, 0, 1, ..., 140588979432688,
140594973553136, 140588979527472],
...,
[ 117, 117, 4473802540135219199, ..., 140594981698960,
140594981698960, 4294967295],
[ 0, 0, 0, ..., 0,
0, 0],
[ 0, 0, 0, ..., 0,
0, 0]], device='cuda:2')
knn_ind size: torch.Size([63001, 10])

But if we make this graph through cpu. This bug will disappear.

Some problem about layergcn

When I run layergcn，

MMRec/src/models/layergcn.py

Lines 56 to 61 in 58cbd75

	if self.pruning_random:
	# pruning randomly
	keep_idx = torch.tensor(random.sample(range(self.edge_values.size(0)), keep_len)).to(self.device)
	else:
	# pruning edges by pro
	keep_idx = torch.multinomial(self.edge_values, keep_len) # prune high-degree nodes

When we train the model using GPU, and enter line 58. An error will be reported.
The parameters are not on the same device, one GPU and one CPU.
When we remove .to(self.device), the problem is solved.

DualGNN: No such file or directory: 'data/baby/user_graph_dict.npy'

Hi, when I run DualGNN, there is no such a '.npy' file. Is there something wrong?