left-on-read/leftonread

Left on Read

leftonread's Issues

Enable consistent eslint rules for app

As discussed in #53, let's turn on no-explicit-any and no-default-export in app. We should try to make the rules as consistent as possible with the top-level .eslintrc so when we actually do a migration, we won't end up with a ton of errors.

Populate example text data

We need to populate the stack of texts with real data and additionally add profile pictures.

Enable workflows for web

As title suggests, we should set up workflows for web similar to app.

Add Google Analytics

https://github.com/react-ga/react-ga

If possible, let's also set up our goal as users clicking the "Notify me" CTA when they sign up their email.

Add state to Notify Me CTA

There's no loading or error indicators on the Notify Me CTA

Discuss and Investigate Group Chat Filtering

Note: it's unfortunate, but there's no known easy way to do ONLY_GROUP_CHATS (where expected behavior is sent and received between people only in group chats). It's only "Both" or "Only Individual Convos". We need to dig deeper but I (Alex) think this is because of message.cache_names IS NOT NULL gives you all group chats in which you never sent a message. is_from_me = 1 AND message.cache_names IS NOT NULL does not exist.

Add Date to All Existing Queries/Tables

Every query should be able to filter on date. We do not currently keep track of date.

Prompt user asking for full disk access

This is part of the user flow. We need to figure out how to prompt the user for full disk access. We need full disk access because we copy the chat.db file from ~/Library/Messages and move it to ~/.leftonread.

In the long-term, we'll allow an option for the user to plop their chat.db on their Desktop. See: #21

The work for this issue should also update the docs (if anything changed for local development.)

Join contacts to all core tables

If there is a .vcf file on the user's computer, it should parse the contacts and join them to the core tables.

Add Github Logo to Bottom of Footer

I think a Github logo prompting users to start would look nice at bottom of footer.

Options:

1. We can mimic the Github.com footer itself. We can get the github logo here https://github.com/logos. It should link to this repo.
2. We use this package, and use the stargazers button: https://www.npmjs.com/package/react-github-button

I vote option 2, but it won't have typescript support I think. There might be other packages.

Add Husky and Lint-Staged pre-commit hook to entire project

Goal: anyone who commits to this project should have their files auto-prettified no matter their editor.

Impact: Without this, we will get contributions from people who do not have prettified code and that is annoying and makes the code less readable and the pr diffs confusing.

We used to do a workflow, but @Teddarific didn't like it because then you have to pull the remote changes.

Top Friends Chart v2 — should be a stacked bar chart with sent vs received for each person

Splitting this issue from #24

Will need to update the query, the types, and the chart.

---

Stacked bar chart could be done via an optional prop:

https://stackoverflow.com/questions/37249439/chartjs-v2-0-stacked-bar-chart

Fix Web Responsiveness

Shrink to half vert screen size on a Macbook 13" (no monitor).

Set up mock chat.db for tests within App

Options:

1. in memory sql db? beforeEach afterEach to create and delete tables every time

PROS:

• unit test friendly <3
• don't have to worry about conflicts. Setup/teardown straight forward
• don't have to worry about pushing a fake chat.db to github

CONS:

• will be extremely cumbersome

2. create or find a real fake chat.db with minimal data

PROS:

• more like the real thing
• populated data ready-to-go; not cumbersome

CONS:

• will be hard to find one to push to the internet
• massive file? need to include from build. I guess you put in tests directory?
• when we write to it, we need to delete those tables so future tests aren't messed up

Add githooks

As a developer,
I want the code to be pretty
So that it is standardized

Acceptance Criteria:

• githook runs on checkout: yarn/yarn install
• githook runs on commit: prettier

Delete Navbar links

At the moment, we don't nearly have enough navbar links to make a navbar worth it IMO. Let's just delete it, so that its only the Logo in the top left.

Top Emoji Table

Creating the "Top Emoji Table" should take ~10 minutes now that everything is setup

Will wait to does this until #15 done to see how that changes the overall architecture/types layout.

How to read emojis

Update 2022:

Went with https://github.com/Julien-Marcou/Unicode-Emoji in PR #187

---

Currently, we read the emojis from a hard-coded list in a .js file. But emojis can be found in/System/Library/Input Methods/CharacterPalette.app/Contents/Resources/CharacterDB.sqlite3. Instead of a hard-coded list we could read from this CharacterDB.sqlite3 file. However, it seems that Apple does not keep that db in sync with the latest emoji updates delivered in new iOSes. (An emoji from iOS 14 does not yet appear in the db stored on my machine.)

Options:

1. We get the majority of the emojis from this DB file and continue hardcode the new ones delivered in iOS 14 and so on.
2. We continue to hardcode all of them and keep it simple.

Ideally, we would get all emojis through an API. But we also want this app to work without an Internet connection.

Going with option 2 by continuing to hard code. But leaving this issue up for future discussion.

Should we add an "About Us" section to marketing website?

Doesn't have to be its own page... but maybe that's best.

I was thinking that we should make it clear that this is the same Left on Read from reddit, from Dartmouth, and the same Left on Read that previously made you upload your chat.db on the Internet.

Thoughts? @Teddarific

Fix Safari Desktop Compatibility

Unreal in 2021 we still have this issue. 😆

Website has a few bugs on Safari:

• SVG on footer is messed up
• margins are super large (on Safari and Firefox)
• while doing this issue, would be good to checkout Firefox as well (see comment below)

Production URL: https://leftonread.me/

Prompt for full disk permissions

Kind of blocked by design, but we should set up the main framework of the user flow. We can copy over the old copy / flow from the original site for now.

Investigate permissions on chat.db and .vcf

It's going to important to know what the technical limitations are in terms of accessing a chat.db and .vcf

This will help address what the user flow will look like:

1. Can we directly query their chat.db file? (aka nothing is needed from the user beyond asking for permissions?)
2. If not, what are the minimal number of steps the user needs to take in order for us to get an accessible chat.db?

Set up deploying for web

I typed out the below, but I realized this is a Nextjs app with server side rendering, so we almost 100% will go with Vercel.

What done looks like for this issue:

• website is deployed to our URL: leftonread.me
• github action to trigger a deploy on merge to master when web/ changes

---

Two options I see right now to deploy web:

1. firebase 🔥

pros:

• we are familiar with it. We know it's easy to do.
• I've also just learned that as of Nov 2020, there's a github action, so we could set up auto deploys and 'staging' (preview deploys)

cons:

• cost. Not an issue right now, but when we hit massive traffic, we did get charged around $5. Theoretically, this could have gotten bad.

2. vercel for git 🦄

pros:

• pricing model seems to be more based on companies (not traffic... at least so it seems)
• something new to learn and seems to be the hot new thing : https://vercel.com/docs/git
• looks like better monorepo support: https://vercel.com/docs/ (monorepos and firebase was bad in 2019 when I used it)

RESOURCES:

• https://www.reddit.com/r/nextjs/comments/hta4iz/firebase_hosting_vs_vercel/
• https://betterprogramming.pub/firebase-vs-vercel-aka-zeit-now-2020-7f5ab18d865f

Add Filtering Functions to TopFriends Query

Add the logic to filter on the TopFriends query just like the WordOrEmoji query.

Create "Core" Table Once and Then Never Again

We only need to create the SPLIT_TEXT_TABLE once and then never again. This should be a "Core" table that is created on initialize. This will greatly improve performance because then the WordCount and EmojiCount (and I imagine a word by day chart) tables can just read from this core table, which is created only once.

The core table could also already exclude stopWords, unicode characters, etc, so we don't have to run those WHERE clauses multiple times.

Add "All" as default to Contact Filtering

• Need to add the ability to filter back to "all contacts". Currently, once you select an individual, you can't go back.

Discussion: we will REQUIRE full disk access or implement a 2nd manual upload workflow

PROS:

• if we require full disk access, then we only have to worry about 1 technical implementation, i.e., what we currently have. (copying dbs to the Left on Read application folder.)
• no need for user to mess around in Library applications
• no need for user to read instructions
• super fast (less than 1 second on my machine). It copies, creates the tables, and queries. It would be fast if user dragged to Desktop as well... but would be "slower" in the sense that the user would have to step through a lot of prerequisites to get it working

CONS:

• the user may not trust us, and may exit the app, because there is no manual alternative

@Teddarific thoughts? @ayc19 and I have been discussing this and Alex wants your thoughts

Flatten app directory

Not a high priority.

It'd be nice to flatten the app directory so that the electron folder is basically the top level (but we can keep it named app). Is this possible?

RE: the .vscode folder, I don't much about this stuff, but if we want to keep it would be nice to hoist to top level (don't know implications of applying electron settings to other projects), or just remove it if it's not that useful.

Top Friends Chart

Create a top friends chart.

Initial PR:
#23

Ability to manually add a contact name

Is your feature request related to a problem? Please describe.
When I don't get a match on my contact name (because I don't have it on my computer), I'd like to add it manually.

Describe the solution you'd like
Text input box to map phone numbers to contacts?

Additional context
Implementation-wise, I think this means we would have to write to the db (update the contact_name) field for that phone number.

Implement one filter

Implement filtering - we can start with just a single filter, such as by date.

This should be redux-driven.

Fix Contact Filtering Bugs

• put getContactOptions in its own file and get it a types.ts
• Also, fix the existing bugs that when you filter to contact with low messages it breaks

Enable workflows for server

As title suggests, we should set up workflows for server similar to app.

Rethink how we match contacts to phone numbers

We need some way to JOIN between the addressbook.db tables and the chat.db tables (so that we can get the contact name, which is only stored in the addressbook.db).

We currently JOIN on phone number, as I don't know of any other shared field.

Unfortunately, the addressbook.db and chat.db store the phone number in different formats.

• the chat.db always lists the country code.
• the addressbook.db, however,only sometimes includes the country code.

As a result, we need to 'normalize' the two. We do this by stripping the country code if the phone_number is longer than 10 digits and ONLY matching when the phone numbers are both 10 digits.

This is bad because then contact names will not work for users outside the U.S. and Canada (the only countries where phone numbers are 10 digits).

Current implementation: normalization.ts

    THEN SUBSTR(
      replace(
        ${column}, "${COUNTRY_CODE_SYMBOL}", ""
      ),
    -${PHONE_NUMBER_LENGTH},
    ${PHONE_NUMBER_LENGTH}

Sign & notarize a production build of the app

See #9

Ability for custom stop words

As a user,
I want the ability to put in my own stop words,
So that I have a better sense of what 'meaningful' words I send

EDIT: This was closed because we won't do this. We'll just allow user to paginate through

Replace Default Icon with Left On Read Icon

We need to replace all the icons in here: https://github.com/Left-on-Read/app/tree/main/electron/resources (aka the resources folder with our icon. This is what appears in production.

Assigning to @Teddarific

Standardize Logging

We use electron logger, which is great, but we should still standardize logging across the entire app, i.e., every log should be formatted the same way.

Hoist baseline eslint and prettier config for app

We have a baseline eslint and prettier config for the monorepo now.

app will still probably need it's own eslint config to apply specific erb rules it looks like, but we should work towards making the rules consistent with that in server & web. For example, let's work towards turning on no-explicit-any and prefer-default-export etc

Prettier should be more straight forward. There'll be a big change of changing tabWidth from 4 to 2

Adding Contact Us Functionality

We'll have a link to Contact Us in the Footer. I'm open to how we implement this. A few options off the top of my head:

• Natively implementing a modal ourselves with a form that submits it to our server and stores in Mongo
• Integrating a Typeform
• If we're really lazy, just link to an email

Fix Mobile Browser Compatibility

In the mobile browser (at least on iPhone8 both Safari and Chrome mobile):

• the svg in the footer seems to be stretched (probably same issue as #112)
• the text animation seems to be cut off slightly

Add Emoji Chart to "Unique analytics" section

Currently in the "Unique analytics" section we only have the "Top Received Words" chart.

Let's add in a "Top Emojis" chart. We should add it so it's a carousel with left / right arrows to scroll through each different chart. You'll also probably have to add another hard-coded data field to the Texts. Don't worry too much about the actual content since I plan on going through and handling that near the end.

Add TOS and Privacy Policy Pages and then Link Them in Footer

Didn't see this anywhere in the code. Is React Router still your preferred way of implementing this in this way @Teddarific ?

Add Global "Contact" Filter

I want to search by a Friend aka Contact aka phone number.

Outline what still needs to be done with web and server

@Teddarific could we get issues... or if it's easier, just comment here what still needs to be done with the marketing site, i.e., web/ and server/

Document how to easily add a chart

By the time #24 and #15 are done, any developer should be able to easily add a chart within a few minutes (if they know the query). (Many of the queries are already roughly outlined here: https://gist.github.com/alexdanilowicz/56c7124d7b55ce76a3706bd5d96add01)

We should document how to do this. As a first pass, we can create a CONTRIBUTING.md (maybe make it live on the wiki?). It should outline how to add a chart, which will help document the overall architecture.

Set up deploying for server

Filter out words with punctuation, such as "hello!"

We should strip punctuation from texts.

PROS:

• "hello" and "hello!" should count toward the "hello" word count. Right now they are separate words.
• Looks weird to see "hello!" in the top word count

CONS:

• The only way I can think of doing this right now is removing punctuation entirely for every word.

Look at how https://github.com/Left-on-Read/leftonread/blob/main/app/src/chatBro/constants/punctuation.ts is currently done and make it more robust and clean.

Fix that workflows don't run on forked copies

See #124 as an example.

I believe the open-source standard is for contributors without push access to make forked repos. But if that's not the case, perhaps we could simplify this by giving everyone push access?

Otherwise, we should get workflows running on forked copies as well.

Remove Husky from app

Lets use workflows instead.

Feel free to re-assign to @ayc19