Giter Site home page Giter Site logo

media-centre / makenews Goto Github PK

View Code? Open in Web Editor NEW
53.0 14.0 22.0 60.46 MB

MakeNews is for journalists and newsrooms. It helps you track news from web and social media in real-time.

Home Page: https://makenews.in

License: GNU Affero General Public License v3.0

Shell 1.20% JavaScript 92.63% HTML 0.02% Ruby 0.16% SCSS 3.59% Jinja 2.41%
news-aggregator journalists news-feed twitter-feed facebook-feed webscraper

makenews's Introduction

Makenews

Makenews

MakeNews is for journalists and newsrooms. It helps you track news from web and social media in real-time.

Build Status Coverage Status

Pre Requisites:

  1. Install node(version >= 8.0) npm
  2. Install couchdb 2.0
  3. Create an admin user in couchdb with whom you can create, delete and change databases manually on the localhost:5984/_utils/#
    If your admin user in couchdb is not with the name 'admin' and password 'admin', change this data in the file " instal.config " from this very project.
  4. Install couchdb lucene 2.0
  5. Install gulp npm install gulp -g

Test Requisites:

  1. write npm --version in the console and it shall return the number of the certain installed npm version
  2. write 'localhost:5984' on your browser and there you should see the couchdb version and some additional information about couchdb.
  3. check whether you have configured correctly the couchdb-lucene. For instance, you can try querying a database(if you created database in couchdb and you have installed curl) like this:
	curl http://"your_database_ip":5984/database_name/_fti/design_doc/view_name?q=Query
  1. write gulp --version in the console and it shall return the number of the certain installed gulp version(local and cli)

1. Installation:

1.Clone the makenews application repository from git
2. All the configuration related data must be mentioned in the install.config file

  • Facebook appId and app secret key must be given in the file for being able to configure data from FB sources
  • Twitter consumer key and consumer secret must be mentioned under twitter configuration for being able to access twitter data from makenews
  • Refer install.config file for instructions

Comments: Facebook and Twitter appIds are not necessary to start the application, they are necessary for the actual connection between the app and facebook or twitter.
3. Start couchdb
4. Once the configurations are set in the install.config file, run the oneClickInstaller.sh file
5. Running this file installs the makenews application with the configurations set in the install.config
6. Towards the end of the execution, application user creation dialogue would be prompted on the console, enter the username and password for the intended user

2. Start the server:

Once the installation is successful, use the following command from the root folder, to start the server

node dist/server

The application would start running on the configured ip address and port, for the default configuration, http://127.0.0.1:5000

  • Additional Information: after running the server check both consoles(the console of the browser and the cmd prompt) for some errors.
  • In some cases, it is necessary to go to 127.0.0.1:5000 instead of localhost:5000, because of CORS, if you want to log in - it can give some errors in the console of the browser
  • In some other cases, after trying to login the process in the cmd prompt crashes because of the permissions. That's why you should sometimes start the server with sudo(just 'sudo node dist/server')

3. Creating Application users:

run createuser.sh file from dist folder

4. Application configurations:

After changing any configuration,  please run gulp build and restart the server

4.a Server configuration:

The server configuration file path is server/config/application.json

{

 "default": { //default configuration
   "serverIpAddress": "localhost", //ip address of the application server
   "serverPort": 5000, //Port on which the server runs
   "couchDbUrl": "http://localhost:5984", // couch db url
   "searchEngineUrl": "http://localhost:5986/_fti/local", // lucene url
   "userDbPrefix": "db_", //application user dbname prifix
   "adminDetails": { //application admin details
     "username": "admin",
     "password": "admin",
     "db": "common" //Common database for all users for storing web urls
   },
   "facebook": {
     "url": "https://graph.facebook.com/v2.8",
     "appSecretKey": "fb_appsecret", //facebook app appSecretKey
     "appId": "fb_appId", //App id of facebook app
     "timeOut": 2000, //timeout for request
     "limit": 500 //limit for data from fb for a source
   },
   "twitter": {
     "url": "https://api.twitter.com/1.1",
     "authenticateUrl": "https://api.twitter.com/oauth/authenticate",
     "consumerKey": "twitter_consumerKey", // twitter app consumer key
     "consumerSecret": "twitter_consumerSecret" // twitter app consumer secret
   }
 },
 "qa": { //environment specific, if we need diff config for qa environment you can add here

 }
}

4.b Client configuration:

The client config file is makenews/client/config/config.js

if(!window.mediaCenter) {
   window.mediaCenter = {};
}
window.mediaCenter.serverUrl = "";    //application server url
window.mediaCenter.facebookAppId = ""; //facebook appid
window.mediaCenter.autoRefreshSurfFeedsInterval = 300000; //time interval for refreshing feeds
window.mediaCenter.dbSessionInterval = 600000; //time interval for a login session
window.mediaCenter.storyAutoSaveTimeInterval = 300000; // time after which a story is auto saved

5. Troubleshoot:

  • If twitter authentication page shows empty - While creating the twitter app, mention the makenews app url under the callback url, to redirect to the app after authentication
  • If login fails and console shows CORS error - Ensure that the couchdb CORS is enabled
  • If login fails - Make sure Couchdb is running before trying to log into the application
  • In case of error with facebook authentication - Facebook app( whose id is mentioned in the configuration file) must be in active state, for the makenews app to be able to fetch data from Facebook
  • If the default Web URL suggestion are not seen - check lucene if setup is properly done
  • For Windows users, changing the server url to ‘localhost’ will not work, hence keep the server url to the default 127.0.0.1
  • Makenews is configured to work with facebook version of v2.10. There could be issues cropping up if a new version is released from Facebook, which might need fixes in the code

6. Developer options:

6.a Build and Tests:

The procedure to build the application, and get your changes working, are

  • gulp build
  • npm test
  • npm run lint //run static analyser
  • gulp clean //cleaning dist folder

check the gulpfile.js for more gulp tasks

6.b Migration:

If your change need data migration follow below steps:

  1. New migration files can be written in the makenews/server/src/migration folder
  2. Any migration, that has to change the common db, must be specified in the admin folder of migration
  3. A migration for the user db, must be specified inside the db folder of migration
  4. The migration data file should be named following the template of timestamp_filename
  5. The migration file must contain an up method, for the db upgradation
  6. A new condition must be added in the getObject method of the Migration.js file, for every new migration file created
  7. The following command should be used for running the new migration
    node dist/server/src/migration.js [--admin_user_name='username' --admin_password='password']
    

7. Tips:

  • If you have some problems to install couchdb or to connect it to couchdb-lucene, just to know that if you have or use mac os - the installation is very faster and easier. The mac users install only the application couchdb and there is almost no manual configurations between couchdb and couchdb-lucene(just 2-3 line in the console)

makenews's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

makenews's Issues

Previous results on configuration page

I have source result on facebook or twitter configuration and I switched to scanNews or writeStory and then I come back to configuration page that time it showing previous source results with webUlrs highlighted

AddURL

If URL i want to add already exists in the database. Its showing message as "Unable to add" instaed of "url already exists". No proper (warning/error) messages

Auto logout

Application is auto logged out after some time

AddURL POPUP

If the user wants to add URL without searching it on the search bar. I have to refresh a page.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.