hemanta212/nepali-news-portal-kbd

Online webapp that scrapes news from different new portals of Nepal and worldwide. Hosted at heroku.

Khabar-board

Khabar-board is an online webapp that scrapes news from diffrent new portals of Nepal and worldwide.

Currently, the news is Scraped for National nepali, International nepali, National english and International english sections from Kantipur, Nagarik, Ujyaalo and Kathmandu Post, himalayan times, nepal times and top headlines from International sources like bbc, cnn, new york times, etc

Features:

• Login, Logout, Remember login through cookies
• Reset and confirm email address
• National and International news in both nepali and english
• News collection(scraping, api) and serving
• Pagination, Collapsible sidebar, Responsive navigation
• Use proper structure with flask blueprinting
• Hosted Live at heroku @ kbd.herokuapp.com

Todos:

• Integrate various international news apis(top, general, sports, tech...)
• Rewrite manage.py and database management
• Experiment with docker deploys
• Add logging
• Add tests
• Extract the scraper to separate news API
• Redesign the dashboard
• Remove compulsory login and establish per person db record of preferences
• Give user choice to customize news topic, sources, language etc
• News search on keyword for custom category (like bitcoin, trump, etc)
• Integrate social media login (facebook, github)

Run with Docker

docker build -t kbd .
docker run -it --rm --net=host kbd

Installation and Usage

• Install python 3.6 or above
• Clone/download this repository and navigate to this repo through cmd

$ git clone https://github.com/hemanta212/nepali-news-portal-kbd

• Install either poetry or follow this guide for setting up virtual environment.
• Installing dependencies using poetry

$ poetry install

• Installing dependencies using pip.

$ python -m pip install -r requirements.txt

Setting up Database

You can setup any database supported here. This doc covers setting up sqlite and postgres db.

Setting up SQLite

Populate the following as your environment variables

SECRET_KEY="<your secret key here (random string sequence)"
SQLALCHEMY_DATABASE_URI="sqlite:///site.db"
MAIL_USERNAME="your email",  //optional
MAIL_PASSWORD="your password" //optional

NOTE

• optional: only required for 'forgot/reset password' functionality to work If you use gmail and 2 factor-auth, you can use app passwords without compromising to low security.

• To enable the international news section get an api key from News API and set environment variable 'NEWS_API_KEY' to that value.

Alternatively, you can populate these detail in 'template_secrets.json' file and rename it to secrets.json.

Setting up the Postgres Databse

The details of installing, configuring a postgres database is detailed here.

Once you have setup the db and have its URI, just set it as the value of 'SQLALCHEMY_DATABASE_URI' in above SQLite section and continue following this doc.

Initializing the database

To upgrade the newly created database to the project's structure use

$ python manage.py db upgrade

Similarly, if you need to start fresh, delete the migrations folder and now run,

$ python manage.py db init # create the migrations folder
$ python manage.py db migrate # create sql commands for the required db construction
$ python manage.py db upgrade # executes those changes in the db

KNOWN ISSUE

• While Using SQLite relative URI: Due to the mismatch in the folder level of manage.py and flask application only the sqlite:///site.db relative URI is supported other relative URIs wont work so consider using absolute sqlite URIs.

Running the Server

    python run.py // production environment (default)
    python run.py debug // for debug environment

NOTE: Running in Prod mode is unsupported with secrets.json file and will fallback to Debug.

Running from gunicorn (Not supported on windows)

Gunicorn is a production WSGI server that is essential for running flask project in production environments like with Procfile in Heroku.

After installing, you just provide the application instance of project to gunicorn.

eg.

gunicorn run:app  // prod env

gunicorn run:app debug

NOTE:

Gunicorn is not supported in windows operating system. However, you can run it using WSL.

Accessing all news

After successfully running the app, go to localhost:5000/signup and sign up with an account with this admin provisioned email '[email protected]' (yes this exact email only) and login. You should see news from all the sources.

nepali-news-portal-kbd's Issues

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Add footer in every page.

I think the footer currently in landing page should be implemented to every news page since landing page is now going. Also it looks little incomplete currently. The links like privacy policy, source code, go to top arrow are few things to add there. Although it would be nice to have go to top / bottom every time while scrolling as well

Dark mode is unstable.

The sitewide night mode is implemented using document cookies method. This results in site flashing every time page is loaded before the dark mode remembering cookies kicks in. I guess it can be troublesome in night while browsing.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Remove front landing page.

There is no need for a landing page since the availability of 'try without sign up feature'. It can be migrated to other URL or completely removed. The default try dashboard should be shown with login and signup at nav bar.

Why not try SQL?

I don't see why SQL implementation should be left since it is relatively easy to do it with sqlalchemy. I do see that:

• The manage.py issue is not fixed and still is in backlog.
  But seeing that it can handle postgres I don't see why SQL would be any difficult.
  Similarly,
  • There are no sites that I can run this on production wise but still it would be an experience with configuring windows for database. As I still haven't used any in windows.

live demo

can you put option of live demo of this site