kwirtz/novelpy

Compute novelty indicators

*** Novelpy***

The goal of this package is to help scientometrician work with novelty indicators.

novelpy's Issues

AttributeError when running the tutorial

Dear maintainers,

Thanks so much for your work on this package!

I am trying to reproduce your tutorial but this is running an error.

My setup is:

Python version       : 3.9.12
IPython version      : 8.4.0

novelpy: 1.1
tqdm   : 4.64.0

When I run:

for focal_year in tqdm.tqdm(range(2000,2011), desc = "Computing indicator for window of time"):
    Foster = novelpy.indicators.Foster2015(collection_name = "Ref_Journals_sample",
                                           id_variable = 'PMID',
                                           year_variable = 'year',
                                           variable = "c04_referencelist",
                                           sub_variable = "item",
                                           focal_year = focal_year,
                                           starting_year = 1995,
                                           community_algorithm = "Louvain")
    Foster.get_indicator()

I get:

start:   0%|                                                                                                 | 16/49872 [00:00<00:00, 229040.49it/s]
Computing indicator for window of time:   0%|                                                                                | 0/11 [00:26<?, ?it/s]
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [6], in <cell line: 1>()
      1 for focal_year in tqdm.tqdm(range(2000,2011), desc = "Computing indicator for window of time"):
      2     Foster = novelpy.indicators.Foster2015(collection_name = "Ref_Journals_sample",
      3                                            id_variable = 'PMID',
      4                                            year_variable = 'year',
   (...)
      8                                            starting_year = 1995,
      9                                            community_algorithm = "Louvain")
---> 10     Foster.get_indicator()

File /opt/anaconda3/envs/ai_diversity/lib/python3.9/site-packages/novelpy/indicators/Foster2015.py:172, in Foster2015.get_indicator(self)
    170 print("Saved ...")
    171 print('Getting score per paper ...')        
--> 172 self.update_paper_values()
    173 print("Done !")

File /opt/anaconda3/envs/ai_diversity/lib/python3.9/site-packages/novelpy/utils/run_indicator_tools.py:436, in create_output.update_paper_values(self, **kwargs)
    433         os.makedirs(self.path_output)
    435 if self.indicator in ['uzzi','lee','wang','foster']:
--> 436     self.populate_list()
    437 else:
    438     print('''indicator must be in 'uzzi', 'foster', 'lee', 'wang' ''')

File /opt/anaconda3/envs/ai_diversity/lib/python3.9/site-packages/novelpy/utils/run_indicator_tools.py:379, in create_output.populate_list(self)
    376         self.keep_diag=False
    378     # Use novelty score of combination + Matrix of combi of papers to have novelty score of the paper with id_variable = idx
--> 379     self.get_paper_score()
    380 else:
    381     continue

File /opt/anaconda3/envs/ai_diversity/lib/python3.9/site-packages/novelpy/utils/run_indicator_tools.py:286, in create_output.get_paper_score(self)
    284 scores_list = []
    285 for combi in combis:
--> 286     if self.list_of_items_restricted:
    287         if combi[0] in self.list_of_items_restricted and combi[1] in self.list_of_items_restricted: 
    288             combi = sorted( (self.name2index[combi[0]], self.name2index[combi[1]]) )

AttributeError: 'Foster2015' object has no attribute 'list_of_items_restricted'

Networkx 3.0 removed from_scipy_sparse_matrix

networkx 3.0 removed from_scipy_sparse_matrix (see Merged PR)

This causes the tutorial to fail.

Code:

import novelpy
import tqdm

for focal_year in tqdm.tqdm(range(2000,2011), desc = "Computing indicator for window of time"):
    Foster = novelpy.indicators.Foster2015(collection_name = "Ref_Journals_sample",
                                           id_variable = 'PMID',
                                           year_variable = 'year',
                                           variable = "c04_referencelist",
                                           sub_variable = "item",
                                           focal_year = focal_year,
                                           starting_year = 1995,
                                           community_algorithm = "Louvain",
                                           density = True)
    Foster.get_indicator()

Output:

Computing indicator for window of time:   0%|                                                            | 0/11 [00:00<?, ?it/s]

loading cooc for focal year 2000
cooc loaded !
loading items for papers in 2000


  0%|                                                                                                 | 0/49872 [00:00<?, ?it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 49872/49872 [00:00<00:00, 340925.07it/s]
Computing indicator for window of time:   0%|                                                            | 0/11 [00:00<?, ?it/s]

items_loaded !


---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[33], line 14
      4 for focal_year in tqdm.tqdm(range(2000,2011), desc = "Computing indicator for window of time"):
      5     Foster = novelpy.indicators.Foster2015(collection_name = "Ref_Journals_sample",
      6                                            id_variable = 'PMID',
      7                                            year_variable = 'year',
   (...)
     12                                            community_algorithm = "Louvain",
     13                                            density = True)
---> 14     Foster.get_indicator()

File ~/.local/lib/python3.10/site-packages/novelpy/indicators/Foster2015.py:161, in Foster2015.get_indicator(self)
    145 '''
    146 Description
    147 -----------
   (...)
    158 
    159 '''
    160 self.get_data()
--> 161 self.g = nx.from_scipy_sparse_matrix(self.current_adj, edge_attribute='weight')         
    162 print("Create empty df ...")
    163 self.generate_commu_adj_matrix()

AttributeError: module 'networkx' has no attribute 'from_scipy_sparse_matrix'

Could not build wheels for nmslib at install

Upon installation of novelpy on my Windows 10 and MacOS machines. Both errors pertain to the building of wheels for nmslib.

On Windows 10, I'm using python 3.11.3. Here is the output::

---

 error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.36.32532\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for nmslib
Failed to build nmslib
ERROR: Could not build wheels for nmslib, which is required to install pyproject.toml-based projects

I've already installed and included the path to cl.exe

On the mac (version 11.7.6) on Python 3.11.3, the error is clang: error: the clan compiler does not support '-march=native' followed by ERROR: Could not build wheels for nmslib, which is required to install pyproject.toml-based projects

Use of deprecated package 'sklearn' in novelpy

Hi there! I noticed that while installing novelpy, the package sklearn was listed as a requirement. However, it seems that the 'sklearn' package is now deprecated and has been replaced by scikit-learn (https://pypi.org/project/sklearn/). Just thought I'd give you a heads up in case you wanted to update the requirements in the future. Thank you!