Fine tuning french BERT model on my own dataset for summarization task

[poloclub]

Fine tuning french BERT model on my own dataset for summarization task

[xiaohk]

Hello, thanks for your interest in Dodrio!

Sure thing, Dodrio can help you visualize the attention weights in a fine-tuned French BERT model. Dodrio is a web-based system and it uses pre-computed data files (JSON files) to populate the visualizations. You can see a list of all the data files Dodrio's public demo is using:

dodrio/src/Main.svelte

Lines 17 to 25 in 458c405

	const attentionDataDir = 'PUBLIC_URL/data/sst2-attention-data/';
	const dependencyDataFilepath = 'PUBLIC_URL/data/sst2-dependencies.json';
	const syntacticHeadDataFilepath = 'PUBLIC_URL/data/sst2-sorted-syntactic-heads.json';
	const semanticHeadDataFilepath = 'PUBLIC_URL/data/sst2-sorted-saliency-heads.json';
	const importantHeadDataFilepath = 'PUBLIC_URL/data/sst2-sorted-confidence-heads.json';
	const saliencyDataFilepath = 'PUBLIC_URL/data/sst2-saliency-list-grad-l1.json';
	const atlasDataFilepath = 'PUBLIC_URL/data/sst2-atlas.json';
	const embeddingDataFilepath = 'PUBLIC_URL/data/embedding-list-sst2.json';
	const tableDataFilepath = 'PUBLIC_URL/data/table-list-sst2.json';

These JSON files are included in the public/data folder.

Dodrio relies on three types of extracted data from your dataset and model. You can learn more about it from our paper:

1. Semantic information (human annotation or saliency score)
2. Syntactic information (human annotation or parsed syntactic dependencies)
3. Importance score (confidence score, sum of absolute attention gradients, or LRP, etc.)

Therefore, you want to extract and generate JSON files with the same data structure from your dataset and model. To make the process easier, we provide a Python script and its detailed documentation to demonstrate how to generate all required JSON files. You should definitely check them out :)

We are very excited to see how Dodrio can be used in more languages and tasks, and we are more than happy to help you visualize your model with Dodrio. Let us know if you have any questions along the way 🤗

[xiaohk]

Hopefully #1 (comment) helps you visualize your BERT model with French dataset! I will close the issue now, feel free to re-open it if you have more questions :)