pepyatka/pepyatka-server

Server for Pepyatka

Pepyatka

Pepyatka is an open source FriendFeed clone. Basically, this is a social real-time feed aggregator that allows you to share cute kitty photos, coordinate upcoming events, discuss any other cool stuff on the Internet or setup a private Pepyatka instance in your company.

How to install

1. Build and run Server. Your server should be running on http://localhost:3000/ now.
2. Build and run HTML Client. Open http://localhost:3333/ in your browser.
3. (optional) Build and run iOS Client

Deployment

1. Check out instructions

pepyatka-server's Issues

Bookmarklet

Let browser save password on sign in form

via http://pepyatka.com/posts/34686235-6960-4dda-aab9-79c2d009682d

Slide down only new posts.

It's annoying to show posts with a slide down effect when a user loads a page for the first time (e.g. her homepage or public steam) we must be rendering already expanded posts for the first time and apply slide down effect only for new posts on that page.

RequireJS and cache

Right now requirejs (r.js -o build.js) generates two minimized files: main.js and common.js the issue is requirejs (build.js) is not configured in a way to prevent browser cache, so sometimes when we deploy updated scripts, users will still see an old pepyatka.

This article http://stackoverflow.com/questions/8315088/prevent-requirejs-from-caching-required-scripts explains how to update requirejs configuration.

Side notes:

• main.js is a pepyatka brain, it's updated every single time.
• common.js is an artel with emberjs, jquery, handlebars and stuff. so this script must hit user's cache.

Server creates too many connection to Redis server

From time to time especially long running server creates too many connections to a Redis server which is not CPU safe:

$ netstat -tn|grep 6379|wc -l
2054

We need to review who and when creates them, definitely there is a defect(s).

Groups property serialization issue?

There's a strange issue with groups property of the post object. Sometimes it is empty (e.g. http://pepyatka.com/v1/posts/b895e4e9-49d9-4154-a29a-9241bbbb4d87) and has array type, in other cases (http://pepyatka.com/v1/posts/a4e8178c-8cdb-4036-8ffc-f0e534f7111d) it is just an object.

I think it should be always an array, even when there's only one group. Otherwise its name should be changed to group.

And how comes post might be not present in any group?

Import a service

User can add RSS feed in his settings section. Then all posts published in that RSS will automatically fly into his Posts feed.

Specification to be defined later.

Make styles consistent between file and audio uploads

Right now there are too much whitespace between audio player and post body. Let's narrow it down to 8px or so.

Serializer class to generate JSON

Currently json serialisation is implemented as separate functions and this is not a scallable solution if we get more and more complex structures or want to optimize way we generate json. This issue is to implement serializator class (as an example: https://github.com/rails-api/active_model_serializers). Connected to issue #22.

User loses session from time to time

User loses session from time to time which is really annoying. I'm not sure what causes this yet however I'm assuming it might either has something to do with redis key expiration (sessions are stored in the redis) or cookie expires after a short time despite we set max-age to 86400000.

Places to start digging from:

• server.js and redis-store middleware
• routes.js and passport module.

Link to a post is missing in rss streams

When we generate rss stream for a user, e.g. http://pepyatka.com/users/anonymous.rss we prepare content for items in a following way:

<item>
  <title><![CDATA[Is there any english version of this site?]]></title>
  <description><![CDATA[lol, what?]]></description>
  <guid isPermaLink="false">60614666-9835-4042-bc00-bbe8040d415d</guid>
  <pubDate>Sun, 24 Nov 2013 19:33:01 GMT</pubDate>
</item>

we must add a link to a post as well, e.g.

  <link>http://pepyatka.com/posts/60614666-9835-4042-bc00-bbe8040d415d</link>

so you can return back to a post and read/post comments if there are.

User can change her password

On a settings screen we can introduce a new link "Change password" that will open a new page with three fields: Current password, New password, Confirm password.

Do not initialize express on envrionment level

Right now environment.js initialises environment like database connections, development/production modes and then selects right database and so on. However in addition to this it also requires express framework and creates express application. The issue is express application does not terminate process until user clicks ctrl+c or script explicitly says exit. As a result all migration scripts (those must be using environment to initialise database) after they done they do not kill a their process.

Server should use PORT from ENV first

Simply put: I need to setup nginx load balancer with a farm of pepyatka instances so every instance must be running on a separate port. Currently we do not have such option as config allows setting just one port. Server should be able to overwrite configuration port by environment variable PORT.

PORT=3000 node ./server.js

Terms page

Prepare page that describes "Usage license" and "No warranty and limitation of liability".

Add service reference to a post/comment models

When post/comment is published by a service to a user feed we must set which service created it, e.g.

• 3 hours ago from Advanced Tweets
• 1 hour ago from Bookmarklet
• 1 hour ago from iPhone
• 6 hours ago from LiveJournal

Migrate to Redis 2.8.x

Redis 2.8.1 has been released recently and it introduces SCAN command that allows (as well as SSCAN, HSCAN and ZSCAN: http://redis.io/commands/scan) incrementally iterate through a collection of elements. We mainly use iterators in migrations, when we need to scan all users or all posts or something and update them.

HTTPS bookmarklet doesn't work

https://www.archlinux.org/
Try to use bookmarklet there, you'll get

[19:45:08.332] Blocked loading mixed active content "http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"

Disable pointer events on scroll

It might be useful to disable pointer events on scroll to decrease load as described here: http://habrahabr.ru/post/204238/
At least in the brave new browsers.

Limit number of tags to 50

Limit number of tags returned from /v1/tags to 50.

Send email notification wher user sign-ups

Pepyatka must be sending Welcome email notifications to a user once she sign-ups in the application. Text and subject are TBD.

Thank you for using Pepyatka. To complete the registration process, you need to confirm your email address (<emailAddress>) by clicking the link below:

http://pepyatka.com/verify?code=abc123

If you did not register for Pepyatka, then someone probably mis-typed their email address. You can ignore this message, and we apologize for the inconvenience.

If you have any problems verifying your email address, you can get help from us and other Pepyatka users in the <Pepyatka feedback group>.

Happy Pepyatking,
The Pepyatka team 

Pre-render page on the server side

initial contents of page should be rendered on the side of server, so that contents could be shown immediately, without additional roundtrip.

also requested here: http://pepyatka.com/posts/c04d12c7-1b62-459a-9855-32e860ba27bb

Send email notification on a new subscriber

When someone adds a user to his subscriptions list Pepyatka should notify that this user has been added to a friend list:

<screenName> (<username>) has subscribed to your FriendFeed.

Subscribe back

Here are some of the things <screenName> has recently shared on Pepyatka:

<list of user's latest posts>

Multiprocessing and Node cluster

Server should be able to start several processes to be able to fully utilize several CPU cores.

The official way of doing this is http://nodejs.org/api/cluster.html

(originally proposed in #40)

Expander helper is broken (prettifyText function)

This commit d2ee9e3 extracts text style functions to a boundHelper, however this introduced a regression: expander is a javascript library that trims text to X symbols/words and injects hellip that allows user to expand this text if she wants. The problem is bound helper returns just html without javascript associated with it, so if we apply expand fn there user won't be able to run javascript and see entire text.

Shows groups name near username

If post has been puslibed to a group or two and more streams we should show up to two steams like in attached screenshot. This feature has more sense once we develop #5

Nonetheless we need to improve this following the way:

• if user has publised post to a group then it should look like: "<username>: <groupname>"
• if two or more groups or streams: "<username> to <groupname1>, <groupname2>, <feed's name> feed"
• drop parentheses

Second example shows if user adds a post to a feed, then its name should be changed to "username's feed", where "username" is a first word in user's screenname.

First and last comments are not updated by updateComment event

If there are 4+ comments then first and last comments are not updated by updateComment pubsub event (most likely other events also do not hit them) unless comments are expanded. #52 is related to this defect.

Separate AMD for vendor libraries and application.

FriendFeed remote key

Мне кажется, пора при регистрации просить FriendFeed remote key (http://friendfeed.com/account/api) во избежание зохвата ников.

Leaderboard

Replace deprecated CollectionView to {{each}} helpers

While ago Emberjs has deprecated CollectionView and it has been replaced to {{each}} helpers. We should review templates before the moment CollectionView will be removed from the library.

Make audio player FrF-like

I know it is going to be challenging however let's try to optimize look and feel of audio player. For the comparison I'm attaching this screenshot.

Items we can try to address:

1. player top margin is too big (separate issue: #29);
2. player is collapsed by default;
3. light blue color scheme (grey is too depressive :-);
4. file size at the right;
5. player at the top, below is a link to download a file;
6. if there is an image attached it should be at the very top, then audio player.

Replace console.log to winston logger where applicable

There a couple of places where we use console.log to output debug messages, we must be using proper logger like Winston to do this:

• db.js
• pubsub.js

Error messages and notifications on signup/signin

via http://pepyatka.com/posts/6ae4428b-1509-400b-8c48-10c9dd66fcda

Comments block with first and last comments

We should consider doing refactoring of comments block that renders comments, collapsed comments, first and last items. There are already a couple of issues connected to its bad design, e.g. http://pepyatka.com/posts/c810daff-25b8-4f2f-98df-0a0c8e3ff3d6 (or see attached image). Another one is: when user click on "X more comments" link we'll: a) hide all comments, b) apply slide down effect for all of them. Ideally we need to render just X-2 comments and do not touch the first and the last ones.

Adds an option to select which groups user can use to publish a post

Allows user to pre-select groups he wants to publish post to like in this screen.

Notify user if post has been deleted

If user A is on a post page and owner B of that post deleted it we need to either redirect the first user A to a home page or let her know that post has been deleted and no longer exist.

Is this correct output of `npm test`

Sorry for this silly question, I don't know much about js ecosystem.

Here is the screenshot what I see after npm test. Is it the correct output? If it is not, can you please show me the screenshot how it looks at your computer.

Settings support for Groups

Use Ember.js with AMD/Require.js

Enter key adds \n to a comment or post

via http://pepyatka.com/posts/b23a5d95-2f5a-40dd-8cc4-892341469eab

JSON serialisation is slow

This is the main performance killer so far.

Below is an example of timeline json:

$ curl http://localhost:3000/v1/timeline/anonymous
{
  "id": "0f8320aa-a31a-4edc-9bab-3643fd92c1ef",
  "name": "Posts",
  "user": {
    "id": "0b1bd0dc-9f10-4002-bcff-3cc7e6e8c02e",
    "username": "anonymous",
    "type": "user",
    "info": {
      "screenName": "anonymous"
    },
    "statistics": {
      "userId": "0b1bd0dc-9f10-4002-bcff-3cc7e6e8c02e",
      "posts": "1",
      "likes": "0",
      "discussions": "0",
      "subscribers": "1",
      "subscriptions": "1"
    },
    "subscribers": [
      {
        "id": "fa7d253f-7a70-452f-924b-bda7953a5321",
        "username": "test"
      }
    ],
    "subscriptions": [
      {
        "id": "e959db46-c2a5-4fee-95be-5f0aed0134a9",
        "name": "Posts",
        "user": {
          "id": "fa7d253f-7a70-452f-924b-bda7953a5321",
          "username": "test"
        }
      },
      {
        "id": "9b884493-8ec0-4fc6-8432-946e676aa660",
        "name": "Likes",
        "user": {
          "id": "fa7d253f-7a70-452f-924b-bda7953a5321",
          "username": "test"
        }
      },
      {
        "id": "4cd94d9e-4f2e-4682-b17f-3b04b6737133",
        "name": "Comments",
        "user": {
          "id": "fa7d253f-7a70-452f-924b-bda7953a5321",
          "username": "test"
        }
      }
    ]
  },
  "posts": [
    {
      "id": "88699b63-48b9-4ca5-b4f8-02909b3f4507",
      "body": "Hello world",
      "createdAt": 1380398266250,
      "updatedAt": 1380398266250,
      "comments": [],
      "attachments": [],
      "likes": [],
      "createdBy": {
        "id": "0b1bd0dc-9f10-4002-bcff-3cc7e6e8c02e",
        "username": "anonymous",
        "info": {
          "screenName": "anonymous"
        }
      },
      "groups": [
        {
          "id": "0b1bd0dc-9f10-4002-bcff-3cc7e6e8c02e",
          "username": "anonymous",
          "info": {
            "screenName": "anonymous"
          }
        }
      ]
    }
  ],
  "subscribers": [
    {
      "id": "fa7d253f-7a70-452f-924b-bda7953a5321",
      "username": "test"
    }
  ]
}

The main thing to point out is anonymous user block that is duplicated across this json, furthermore I must say that the backend selects all of them separately casing massive performance degradation when there are lot of same users/objects on a page.

I'm proposing two improvements how to generate json:
a) supply an array of ids, e.g.

{
  "post": {
    "id": 1,
    "title": "New post",
    "body": "A body!",
    "comment_ids": [ 1, 2 ]
  }
}

b) include nested object into the same json:

{
  "post": {
    "id": 1,
    "title": "New post",
    "body": "A body!",
    "comment_ids": [ 1, 2 ]
  },
  "comments": [
    { "id": 1, "body": "what a dumb post", "tag_ids": [ 1, 2 ] },
    { "id": 2, "body": "i liked it", "tag_ids": [ 1, 3 ] },
  ],
  "tags": [
    { "id": 1, "name": "short" },
    { "id": 2, "name": "whiny" },
    { "id": 3, "name": "happy" }
  ]
}

this will kill two birds with kind of one stone: reduce network traffic and reduce number of selects to db.

Show users avatar from gravatar.

Gravatar API documentation https://ru.gravatar.com/site/implement/hash/

Here is the js lib that can create md5 hash — https://github.com/blueimp/JavaScript-MD5

Favicon

For the ref: http://pepyatka.com/posts/a2c47157-aa33-4299-bdcc-52d2904b40eb and http://pepyatka.com/files/35c0cde1-1c3e-4105-9a6a-17f12c927cad.png

Improve youtube styles

At this moment youtube player is embedded into a page without a small preview image like it's done in FrF. This issue is to improve styles and add a preview button that opens an embedded youtube player on a first click and on the second it closes that player.

First section: current behavour; second and third: closed and opened FrF player.

Render 404 page if a user follows a post that doesn't exist

We need to handle situations when a user opens a page with a post that no longer exists (or just wrong url). Keep in mind that 404 pages are required for other objects as well like unknown user and stuff. However let's start from posts 404 page.

As a side note: this was already implemented for prerelease version of ember, however deprecated after migration to stable and latest one.

Bookmarklet doesn't allow user selecting group(s) to post to.

via http://pepyatka.com/posts/cb6f315b-d387-438e-a3fe-92400e82f684

Prepare Howto advanced search page

Though it's said nowhere, current search implementation allows doing advanced queries, like:

• intitle:hello AND world
• incomments:world from:epicmonkey
• ABC OR CNN

This issues is to document this as a static page.

Custom profile pictures

Following up on #20 we need to allow users to upload custom profiles images. In the settings there will be a new link: "Change picture" that opens a page with a file field, submit button and if there is already an uploaded image, user can remove it.

Images are automatically resized to (a) 96x96, (b) 64x64 and (c) 32x32.

(a) is used on settings and a post pages
(b) is a generic and most used profile picture, e.g. posts page, email notifications, etc
(c) reserved for a future

Losing comma in the likes list on un-like and re-like

Note tle lack of comma between epicmonkey and anonymous