theconnman/ideas

Ideas repository backed by GitHub

Ideas

Ideas repo for TheConnMan based on GitHub issues

ideas's Issues

Smallbites - Sharing meals with neighbors

Business Case

Some people (not me) enjoy trying new foods. Trying multiple dishes at a new resturant is best done by ordering many smaller dishes and sharing them with others. Even better is sharing them with someone knowledgeable about the food, resturant, or culture.

Smallbites is a network to bring people together to share new or their favorite foods with others from their area. Smallbites is 100% not a dating app.

Workflow

• Fill out profile
  • Location
  • Age
  • Interests (for conversation topics)
  • Types of food you want and know a lot about
  • Types of food you want and don’t know a lot about
  • Times you’re available to eat
  • Price range
• Smallbites sends recommendation of match, conversation topics, options for location, options for time
• You can start a conversation with the other person (no profile pic, no names to prevent discrimination)
• Once both parties agree then the reservation is booked through something like OpenTable

Searchable, Tagged Notes Repository

Business Case

From the articles I read there are many which either change my mind on a subject (to which I want to refer back to later), reinforce with data something I've suspected, or is a generally good point on something I haven't thought about. These are usually around topics like:

• Git
• Culture
• DevOps
• Career Path
• Development Languages
• Etc

I'd like a reverse index of these articles to refer back to. Search is required (though I have free text search from Pocket), but the reverse index is the main driver for quick reference to an article.

MVP Features

• CRUD on saving articles
  • Link
  • Tags
  • Full article contents?
  • CLI (easier than a UI)

Additional Features

• Multi-tenancy
• UI
• Optional encryption

Differentiators

• Easy to use
• Low cost to build and maintain

Monitization Strategy

None for now, just a useful index.

Challenges

• Free text search fully within this index would be a bonus and would be the largest challenge. Getting only the text from the webpage is key. Ideally Pocket's parsing engine could be used.

Conspiracy - The Inside Job Runner

Stateless, AWS backed job proxy.

Business Case

SaaS companies often have a publicly accessible API and a number of internal, private APIs for internal services. These services might have endpoints for running cleanup jobs, system remediation, service scaling, or other operational tasks. Monitoring to discover issues which need these API, however, might be provided by a third party. This means notifications from a third party need to securely get inside the VPN to hit the private APIs.

Conspiracy is a job runner which uses AWS as a secure exchange between a third party and internal APIs. The third party sends a job request through SQS which Conspiracy, running inside your VPN, pulls the job and hits an internal API. Conspiracy is purposefully kept simple so it stays a small job proxy; all logic of the job should be behind the internal API.

MVP

• Pull SQS jobs
• Jobs include:
  • URL
  • HTTP method
  • Params object
  • Body for POST/PUT
• Basic logging

Additional Features

• URL template with env var resolution
• "http://job.{{env}}.theconnman.com", ENV=dev
• StatsD
• Dead letter queue
• Timeouts

Slack Slash Command for ECS

The ECS console is a bit unwieldy, especially with many different services. The following questions would be helpful if they were answered by Slash Commands:

• What ECR image is service X running?
• How many containers are running for service X?
  • Has it finished deploying?
• What is the current CPU usage cluster-wide fro service X?
• How many EC2s are in the cluster?

Plus some actions:

• Scale the EC2 cluster up/down
• Scale service X up/down

Releasable - GitHub Organization Differ

Having trouble remembering which repos in your organization have code which is ready for release? Now you can easily check all repo for code diffs between their latest release and default branch.

MVP Features

• UI to specify a GitHub organization and display all repos, if they are up to date with the latest release, and links to compare

Additional Features

• Specify different branches to diff against

Relevant Documentation

• https://developer.github.com/v3/repos/#list-tags
• https://developer.github.com/v3/repos/commits/#compare-two-commits

Auxy - Authenticated Proxy

Business Case

Microservice architectures require a large number of public and private APIs serviced by many individual services. Internal services may directly service public APIs, but don't need the added responsibility of handling multi-tenancy or authentications. These still need to be authenticated, though the endpoints should not be duplicated through an authenticated public API.

Auxy handles authentication and multi-tenancy as an authenticated internal API proxy. It handles authentication at the edge, but only proxies requests internally instead of duplicating them reducing memory pressure, common object dependencies, serialization/deserialization, and development complexity.

Workflow

• Requests come in to the public API fronted by Auxy
• Auxy handles an existing authentication token or credentials
• Auxy checks the validity of these credentials or token
  • If they are invalid requests are denied at the edge
  • If they are valid Auxy adds a tenant or user identifier header to the request
• Auxy checks the request path and proxies the request to the appropriate service

MVP

• Handle basic auth
• Handle adding and validating an auth token
• Basic DB support with migrations for a user and/or tenant table for authentication
• File based config for proxy paths to services
  • Instead of proxy paths this proxy could be just middleware. If any multi-target proxying is needed routing could be done behind this proxy.

Additional Features

• OAuth support from Google, Twitter
• Additional DB support (DynamoDB?)

Pocket Read/Unread Icon Chrome Extension

A small Chrome extension to embed a read or unread Pocket icon next to links if you've already read them or have added it for further reading.

Static S3 Site Bootstrapper

CLI for setting up a static HTTPS site hosted in S3 fronted by CloudFront.

Inputs:

• Subdomain for the site (e.g. ideas.theconnman.com)
• Flag for if a user with push access to the new bucket should be created (useful for automated deployments)

Workflow:

• Create an S3 bucket
• Wire up CloudFront to use the bucket
• Create an HTTPS cert with DNS validation
  • Create the associated DNS entry in Route 53
• Attach the cert to CloudFront once validated
• (Optional) Create a user with push access to the new bucket

Miss.d - Notifications for when things don't happen

Notifications for when things don't happen

Business Case

Developers and operations teams have notification overload. Many times alerts are set up to fire when events (e.g. backups, cleanup tasks, nightly jobs) happen, but soon become numb to the notifications. It's then even tougher to know when something doesn't happen. Miss.d solves that problem by only sending you an alert when your event doesn't happen. Set up notifications and a job timeout so when you job doesn't happen we alert you.

MVP Features

• Notification types
  • Email
  • Slack notification
• Small UI to:
  • Alert CRUD
    • Set up timeout before alert is sent
    • Set up notifications
    • Provides ping URL (included API key)
  • Event list
• Supporting API and distributed timers

Additional Features

• Notification types
  • SMS
  • SNS
  • Hipchat
  • Etc
• More complicated timers
• Track all job times for enhanced reporting

Differentiators

• Very simple and scalable concept
  • Notification types grow horizontally
  • Jobs will most likely not be high frequency, so scaling is easier due to lower system throughput

Monitization Strategy

Free tier limiting alert number, paid tiers afterwards

Challenges

• Distributed timers are doable and worth building for the MVP even though simple, non-distributed timers are far easier

Name

Name is a play on Linux distributed services (system.d, etc.d)

Khronos - Timers as a Service

Business Case

Timers and cron schedules are a critical piece of many production environments. Nightly emails, batch job kickoffs, internal service TTLs, etc. Although cron is a well known problem (Quartz), durable timers can be trickier. Neither are difficult problems to solve, but are not core business for many companies. Much like websockets-as-a-service (pusher.com) Timers-as-a-Service can be used to offload operational complexity.

MVP Features

• Durable timers
• Expiration as webhook events
• Basic UI
  • Manually create and expire timers

Additional Features

• Additional notification types
  • SNS
  • SQS
• Cron support
• Websocket updates to the UI
• Basic CLI for CRUD of timers
• Billing

Differentiators

• Simple, easy solution for a commonly required, but non-core business function
• A real-time UI can provide a lot of visibility into timers and cron, lots that can be added around it for usability
• Cheap

Monitization Strategy

Free tier limiting concurrent timers and cron. Paid tiers for additional timers and cron.

Challenges

• Ease of use is critical. Timers and cron are solved problems, so TaaS has to be so cheap and easy to use that there's no reason for companies to build the solution themselves.

Zeit-like Shell Authentication

Ever since first using https://zeit.co/now I've been impressed by their CLI authentication workflow. I was able to download the CLI, run a command, get a confirmation email, and freely use the CLI. No UI needed. This workflow is an attempt to replicate that.

• Check locally for a . folder in the home directory
• If none, start the authentication workflow
• Send an API call to /auth with no timeout, a random UUID, and the user's input email (machine hostname too for later revoking?) (polls for up to 3? 5? min)
  • This endpoint holds and polls the API key table for an entry with the UUID waiting for the confirmation token to be removed
• /auth creates an API key DB entry with the UUID, the user's email, a timestamp, and a confirmation token then sends an email to the input email with a URL tied to the confirmation token
• User clicks on the email
• GET request to the URL confirms the session
• The confirmation token is removed and the user generated if not already
• The held /auth API call sees the confirmation token has been removed and returns to the CLI with a 200
• The CLI stores the token as local credentials for future API calls