kaikei-e/curionoah

CurioNoah is a set of applications that arouse curiosity and help you explore through intellectual pursuits.

CurioNoah

CurioNoah is a set of applications that arouse curiosity and help you explore through intellectual pursuits.

About

CurioNoah is composed of several applications that are based on RSS feeds. Some of the applications' roles have yet to be determined, and I am working on their implementation and ideas.

• Curiosity: This is the core application of CurioNoah. It will help curate and expand knowledge in a way that stimulates curiosity and encourages exploration. It may use some kind of external API to aid in curation.

• InfoIsland: This is a web-based application that serves as a user interface and front end. It will aggregate and list all services. It is intended to be minimalistic with an integrated, unified user interface.

• InsightStream: This is the main backend and will be responsible for registering, reading, and updating feeds. The update function may be implemented in Rust for greater speed and efficiency, but currently this app is written in Go, my main development language.

• CoreManager: This will be the core service. It will be written in Rust as mentioned above (to study Rust) and will be the behind-the-scenes help, doing batch processing, etc.

• FeedFlare: This will be an RSS feed search aid, and will focus on discovering new RSS feeds.

• SlateFlex: This is a desire, but it is envisioned as a native application, and we would like to make it note-taking capable. I have absolutely no experience in native app development, so this is the lowest priority.

Activation and termination. This is work around for development.

After you've completed the following Setting up steps, you can launch the program by running activation.sh, and terminate the Rust program running on port 5100 by using terminate_all.sh. Please note: Before proceeding, make sure to first execute the command

docker compose up -d

and then you should add the permission to both shell files.

sudo chmod +x EACH_FILE

Now, you can use it by accessing this

http://curionoah.com:4173/

Basic auth is implementing right now. You can bypass by using below account.

user > admin
password > admin

Decision

I have made a certain decision. It is to base this suite of applications on the use of partial cloud services. There will be no changes to the basic structure or functionality. Operation is assumed with Docker Compose. However, I have decided to adopt a policy of allowing high flexibility in external use by constantly running this application on my home machine, authenticated via the internet using Cloudflare Tunnel. This will make the setup more complex, but the needs for use cases that require complete local operation will be met by a desktop application I am developing with Rust and Tauri, named CardinalAura.

Therefore, the following setup, which was never complete, will now become even less relevant. However, I plan to update the setup method when I feel inclined to do so in the future.

Cloning and building the environment from scratch is too much work, so it will be improved.

Setting up

First, place the three .env files.

The variables are example. So you can configure it.

/.env

MYSQL_ROOT_PASSWORD="testthisisatest"
MYSQL_DATABASE="curio_noah"
MYSQL_USER="userhogehoge"
MYSQL_PASSWORD="testthisisatesttestestest"
POSTGRES_PASSWORD="testtesttestthisisatestmustmodify"
POSTGRES_USER="user1"
POSTGRES_DB="oath_keepers_db"

/InfoIsland/InfoIsland/.env

VITE_INSIGHT_STREAM="http://curionoah.com:9000/api/v1"
# VITE_ORIGIN="http://curionoah.com:5173"
VITE_ORIGIN="http://curionoah.com:4173"

/InsightStream/InsightStream/.env

MYSQL_USER="userhogehoge"
MYSQL_PASSWORD="testthisisatesttestestest"
MYSQL_DATABASE="curio_noah"
MYSQL_ADDR="192.168.100.10:3306"
NET_TYPE="tcp"

/FeedHarmony/feed_harmony/.env

DATABASE_URL="mysql://user1:testthisisatesttestestest@localhost:3306/curio_noah"

Then, edit the hosts.

127.0.0.1 curionoah.com

Once these preparations are complete, run the following command at project root

docker compose up -d

What is the motivation behind undertaking this project?

The primary objective is to enhance my comprehensive development skills encompassing various areas such as Docker, database management, backend and frontend development. Furthermore, this ambitious project strives to encompass quality assurance and performance optimization, acknowledging that these aspects may pose significant challenges even for someone like me who isn't a prodigy.

curionoah's Issues

TODO: Relaoding makes redirect to landing page issue

TODO: add logger to FeedHarmony

need to implement a logger for the FeedHarmony API

TODO: use yup or zod for the validation of searching functionality

I need to incorporate Zod or Yup for validation in the search API. In addition, I want to implement a more advanced search function, not just exact matches. In the future, I'm considering enhancing the search function using Language Model (LLM). However, at this stage, I want to introduce something light, like Elastic Search (I don't know the details), into the backend API.

Todo: Test-first implementation needs to be done.

I need to implement a hierarchically structured application with,h an awareness of onion architecture, etc. This is because it allows each implementation to be separated, and a test-first performance using unit tests can be realized.

I will refactor whole thing of this apps

Just regarding the title

TODO: click counter

I am considering implementing a functionality where clicking on a feed would notify the backend, allowing it to count the total number of times the feed has been read.

Todo: need to implement a logger in InsightStream

InsightStream needs a logger installed as soon as possible.

Designing the functionality of CoreManager

CoreManager is envisioned as an application that underpins Curiosity and handles curation, but I am still determining what it will look like.
The curation process requires high-speed Internet scanning and RSS feed discovery. However, I sense that these features also compete with the existing InsightStream.
Specifically, the ability to update the feed list every 30 minutes.
This applies to updates to a single DB, but the data structure must be reviewed. Due to a lack of domain knowledge, I have yet to be able to build an appropriate model.

I should use UUIDv7 NOT v4

Todo: making the local pipeline for unit tests

Starting with Go.
Use the git hooks and pre-commit.

Fix the style of timeline cards

The Card over wraps when description of the card is long.
I should consider that situation to fit the width of the mobile timeline.

TODO: implement basic auth and user switching functionality.

Until now, I had only considered this for personal use. However, if it is to be a comprehensive product quality application, it is essential to have a feature for managing RSS feeds for each user. Therefore, I plan to implement user authentication (since I only anticipate usage on local machines at this point, I'll use Basic authentication for convenience) and manage feeds for each user. I also plan to control and manage situations where multiple users have registered the same RSS feed, including the count of favorites and registrations.

Todo: Write unit tests. Coverage will be 80%.

IsightStreamでGoプログラムのソースコードのユニットテストカバレッジを、全体の平均を80％まで引き上げる。

TODO: Offer users the ability to mark RSS feeds as "read"

To offer users the ability to mark RSS feeds as "read", I need to address a gap in my current system: the RSS feed timeline isn't linked to individual users. This design choice was because I initially developed the platform for my personal use. My plan now is to craft an RSS feed follow-list tailored for each user. By introducing management on a user basis, I will be able to implement both "read" and "favorite" features.