woshiawang/mmssl

[WWW'2023] Multi-Modal Self-Supervised Learning for Recommendation

MMSSL: Multi-Modal Self-Supervised Learning for Recommendation

PyTorch implementation for WWW 2023 paper Multi-Modal Self-Supervised Learning for Recommendation.

MMSSL is a new multimedia recommender system MMSSL which integrates the generative modality-aware collaborative self-augmentation and the contrastive cross-modality dependency encoding. It achieves better performance than existing SOTA multi-model recommenders.

Dependencies

• Python >= 3.9.13
• Pytorch >= 1.13.0+cu116
• dgl-cuda11.6 >= 0.9.1post1
• pyyaml == 5.4.1

Usage

Start training and inference as:

cd MMSSL
python main.py --dataset {DATASET}

Supported datasets: Amazon-Baby, Amazon-Sports, Tiktok, Allrecipes

Datasets

├─ MMSSL/ 
    ├── data/
      ├── tiktok/
      ...

Dataset		Amazon						Tiktok				Allrecipes
Modality		V	T		V	T		V	A	T		V	T
Embed Dim		4096	1024		4096	1024		128	128	768		2048	20
User		35598			19445			9319				19805
Item		18357			7050			6710				10067
Interactions		256308			139110			59541				58922
Sparsity		99.961%			99.899%			99.904%				99.970%

• We provide processed data at Google Drive. We spend a lot of time collecting datasets, if you want to use our datasets(especially Tiktok), please cite MMSSL in your paper.
• 2023.3.10 update: The official website of the Tiktok dataset has been closed. Thus, we also provide a 'large version' of the preprocessed Tiktok-70710 (Code: wiak). Again, we spent a lot of time pre-processing this dataset, so if you want to use our preprocessed Tiktok in your work please cite our paper.

# datapreprocess
  import json
  from scipy.sparse import csr_matrix
  n_user, n_item = 39387, 23033
  f = open(args.data_path + args.dataset + '/train.json', 'r')  
  train = json.load(f)
  row, col = [], []
  for index, value in enumerate(train.keys()):
      for i in range(len(train[value])):
          row.append(int(value))
          col.append(train[value][i])
  data = np.ones(len(row))
  self.ui_graph = self.ui_graph_raw = csr_matrix((data, (row, col)), shape=(n_user, n_item))

Experimental Results

Performance comparison of baselines on different datasets in terms of Recall@20, Precision@20 and NDCG@20:

Baseline	Tiktok				Amazon-Baby				Amazon-Sports				Allrecipes
	R@20	P@20	N@20		R@20	P@20	N@20		R@20	P@20	N@20		R@20	P@20	N@20
MF-BPR	0.0346	0.0017	0.0130		0.0440	0.0024	0.0200		0.0430	0.0023	0.0202		0.0137	0.0007	0.0053
NGCF	0.0604	0.0030	0.0238		0.0591	0.0032	0.0261		0.0695	0.0037	0.0318		0.0165	0.0008	0.0059
LightGCN	0.0653	0.0033	0.0282		0.0698	0.0037	0.0319		0.0782	0.0042	0.0369		0.0212	0.0010	0.0076
SGL	0.0603	0.0030	0.0238		0.0678	0.0036	0.0296		0.0779	0.0041	0.0361		0.0191	0.0010	0.0069
NCL	0.0658	0.0034	0.0269		0.0703	0.0038	0.0311		0.0765	0.0040	0.0349		0.0224	0.0010	0.0077
HCCF	0.0662	0.0029	0.0267		0.0705	0.0037	0.0308		0.0779	0.0041	0.0361		0.0225	0.0011	0.0082
VBPR	0.0380	0.0018	0.0134		0.0486	0.0026	0.0213		0.0582	0.0031	0.0265		0.0159	0.0008	0.0056
LightGCN-$M$	0.0679	0.0034	0.0273		0.0726	0.0038	0.0329		0.0705	0.0035	0.0324		0.0235	0.0011	0.0081
MMGCN	0.0730	0.0036	0.0307		0.0640	0.0032	0.0284		0.0638	0.0034	0.0279		0.0261	0.0013	0.0101
GRCN	0.0804	0.0036	0.0350		0.0754	0.0040	0.0336		0.0833	0.0044	0.0377		0.0299	0.0015	0.0110
LATTICE	0.0843	0.0042	0.0367		0.0829	0.0044	0.0368		0.0915	0.0048	0.0424		0.0268	0.0014	0.0103
CLCRec	0.0621	0.0032	0.0264		0.0610	0.0032	0.0284		0.0651	0.0035	0.0301		0.0231	0.0010	0.0093
MMGCL	0.0799	0.0037	0.0326		0.0758	0.0041	0.0331		0.0875	0.0046	0.0409		0.0272	0.0014	0.0102
SLMRec	0.0845	0.0042	0.0353		0.0765	0.0043	0.0325		0.0829	0.0043	0.0376		0.0317	0.0016	0.0118
MMSSL	0.0921	0.0046	0.0392		0.0962	0.0051	0.0422		0.0998	0.0052	0.0470		0.0367	0.0018	0.0135
p-value	1.28e-5	7.12e-6	6.55e-6		2.23e-6	7.69e-6	8.65e-7		7.75e-6	6.48e-6	6.78e-7		3.94e-4	5.06e-6	4.31e-5
Improv.	8.99%	9.52%	6.81%		16.04%	15.91%	14.67%		9.07%	8.33%	10.85%		15.77%	12.50%	14.40%

Ablation study on key components of MMSSL:

Data	Amazon-Baby		Allrecipes		Tiktok
Metrics	Recall	NDCG	Recall	NDCG	Recall	NDCG
w/o-ASL	0.0907	0.0396	0.0326	0.0124	0.0801	0.0358
w/o-CL	0.0924	0.0408	0.0328	0.0130	0.0821	0.0351
w/o-GT	0.0929	0.0405	0.0325	0.0121	0.0815	0.0353
r/p-GAE	0.0931	0.0411	0.0331	0.0126	0.0843	0.0364
MMSSL	0.0962	0.0422	0.0367	0.0135	0.0921	0.0392

Citing

If you find this work is helpful to your research, please kindely consider cite our paper:

@article{wei2023multi,
  title={Multi-Modal Self-Supervised Learning for Recommendation},
  author={Wei, Wei and Huang, Chao and Xia, Lianghao and Zhang, Chuxu},
  journal={arXiv preprint arXiv:2302.10632},
  year={2023}
}

Acknowledgement

The structure of this code is largely based on LATTICE. Thank for their work.