glenjasper / remove-duplicates Goto Github PK

View Code? Open in Web Editor NEW

This script eliminates the duplicated records from formatted .xlsx files from Scopus, Web of Science, PubMed, PubMed Central, Dimensions or Google Scholar exported from Publish or Perish. Is mandatory that there be at least 2 different files from 2 different databases.

License: MIT License

Python 100.00%

remove-duplicates's Introduction

remove-duplicates

Pre-requisites

Python libraries

  $ sudo apt install -y python3-pip
  $ sudo pip3 install --upgrade pip

  $ sudo pip3 install argparse
  $ sudo pip3 install xlsxwriter
  $ sudo pip3 install numpy
  $ sudo pip3 install pandas
  $ sudo pip3 install crossrefapi
  $ sudo pip3 install tqdm
  $ sudo pip3 install colorama

Installation

Clone

To clone and run this application, you'll need Git installed on your computer. From your command line:

  # Clone this repository
  $ git clone https://github.com/glenjasper/remove-duplicates.git

  # Go into the repository
  $ cd remove-duplicates

  # Run the app
  $ python3 remove_duplicates.py --help

Download

You can download the latest installable version of remove-duplicates.

How To Use

$ python3 remove_duplicates.py --help
usage: remove_duplicates.py [-h] -f FILES [-o OUTPUT] [--version]

This script eliminates the duplicated records from formatted .xlsx files from Scopus,
Web of Science, PubMed, PubMed Central, Dimensions or Google Scholar (Publish or
Perish). Is mandatory that there be at least 2 different files from 2 different
databases.

optional arguments:
  -h, --help            show this help message and exit
  -f FILES, --files FILES
                        .xlsx files separated by comma
  -o OUTPUT, --output OUTPUT
                        Output folder
  --version             show program's version number and exit

Thank you!