LDA topics as feature selector

[amritbhanu]

LDA topics as feature selector

[amritbhanu]

• smallest no of terms, and stability which we will need and high f scores. (multi objective)
  • (maximized (fscore+Raw score))
  • Features (documents distributed over topics.) and smoting expecially for SE datasets.
    • RESULTS VERY BAD DUE TO very less column features. Values in 0.1
• topics distributed over words. Results
  • SE, not done tf-idf yet. No of cols are very high. (maximized (fscore+Raw score))
  • other can be just using hashing trick.
  • choose the top most topic and then use weights of words.
• tuned k=20, with 7 words each. untuned k=40-100, with 10 words.
  • Check for the set overlap and simple matching and unmatching.
• change the default parameter k in the steps of 20, 40, 80.
• Tune learners with tuned LDA. Objectives??
• Randomness Results