trotto/go-links

The open source go links app at the core of Trotto.

Trotto Go Links

(See the guide here to deploy to Heroku in minutes.)

Reliable and battle-tested continuously in production since 2017, this go links solution is used by companies all around the world daily as the core of Trotto.

Try out the latest build at latest-master.trotto.dev and also check out the matching open-source browser extension.

If you'd rather use the fully-managed instance of Trotto, visit www.trot.to.

---

Don't hesitate to reach out to us through [email protected], a GitHub issue, or live chat at https://www.trot.to.

---

Features

Trotto provides all the baseline functionality you'd expect from a go links solution, including the ability to create, use, and modify go links that are shared with your organization—and with nobody else.

Trotto also includes powerful features above and beyond a basic go links implementation, including a searchable go links directory, a browser extension that makes go links work instantly, and programmatic links, which are go links containing placeholders that map onto the destination URL (ex: go/gh/%s pointing to https://github.com/my_org/%s).

For a full list of Trotto's features, check out https://www.trot.to/how-it-works.

Authentication

At the moment, the open-source app solely supports authentication using a Google account, but we plan to add support for additional identity providers upon request. So if you need support for another provider, submit a GitHub issue, and we'll help you out!

Deploy the app

See our deployment docs:

• Deploy to Heroku
• Deploy with Docker

Local development

You can bring up a local instance of Trotto within a few minutes.

Clone this repository

git clone [email protected]:trotto/go-links.git
cd go-links

Create a virtualenv

Inside the go-links/server directory, create and enter a Python 3.8 virtualenv and install dependencies:

cd go-links/server
python3.11 -m venv .virtualenv
source .virtualenv/bin/activate
pip install -r src/requirements.txt

You can use pyenv as well.

Add an app.yml file

Add a file at server/src/config/app.yml with this format:

sessions_secret: any_secret
postgres:
  url: "postgresql://username:password@host/database"

postgres.url should be the connection string for a Postgres 12 database. The server startup script will add the tables Trotto needs.

Start the backend server

From the server/ directory, run:

./run_local.sh

Start the frontend development server

In a separate terminal, from the frontend/ directory, run:

yarn install
yarn dev

Now, you can access the local instance at http://localhost:3000.

Making changes

Most server-side and frontend changes should be picked up automatically, thanks to the Flask dev server and React Hot Loader.

Database migrations

• update a table model file, e.g., server/src/modules/data/implementations/postgres/links.py
• created a revision file, e.g., ./scripts/create_revision.sh "Add unlisted column"
• review the generated file in server/src/migrations/versions
• reload the app to apply the migration

Running tests

Python

All tests:

./run_tests/run.sh

Subset of tests:

./run_tests/run.sh modules.links.tests.handlers_tests:TestUnlistedLinks

go-links's Issues

Docker run error: "../scripts/upgrade_db.sh: 1: cd: can't cd to server/src"

hello there,
First of all. Thanks for sharing this tool with us!!!

I get an error when I build the docker.
sudo docker run -p 8443:8000 trotto ../scripts/upgrade_db.sh: 1: cd: can't cd to server/src INFO [alembic.runtime.migration] Context impl PostgresqlImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. [2022-10-10 15:20:14 +0000] [16] [INFO] Starting gunicorn 20.0.4 [2022-10-10 15:20:14 +0000] [16] [INFO] Listening at: http://0.0.0.0:8000 (16) [2022-10-10 15:20:14 +0000] [16] [INFO] Using worker: sync [2022-10-10 15:20:14 +0000] [19] [INFO] Booting worker with pid: 19 [2022-10-10 15:20:14 +0000] [20] [INFO] Booting worker with pid: 20 [2022-10-10 15:20:14 +0000] [21] [INFO] Booting worker with pid: 21 [2022-10-10 15:20:14 +0000] [22] [INFO] Booting worker with pid: 22

The original, non-docker file (upgrade_db.sh) located in the cloned repo: server/scripts/ points to a relative path:

server/scripts/upgrade_db.sh:

cd server/src
export FLASK_APP=main.py
flask db upgrade

The command cd server/src could be replaced by the full docker folder path: cd /usr/src/app/server/src/

I resolved it by replacing the cd relative path in server/scripts/upgrade_db.sh by a cd to the absolute docker folder path (cd /usr/src/app/server/src/) and by rebuilding the docker after.

server/scripts/upgrade_db.sh:

cd /usr/src/app/server/src/
export FLASK_APP=main.py

Thanks for everything!

Feature request: Support for Azure SSO?

I see that Trotto only currently supports Google login - is there any chance of support for Microsoft's Azure SSO? (or at least some pointers on how to add it?)

Replica support - multiple k8s pods

I haven't tested this, but I am not sure if the app is known to work with multiple instances, in say a k8s load balanced scenario using PG for the DB.
If not, what do you think some of the areas of code would prevent this.
I am looking to balance the load along with handling cases with a node is drained

Thanks!

dev_appserver.py: command not found

When running locally

./run_local.sh: line 5: dev_appserver.py: command not found

go-links can not register Punycode-ed URL

Legitimate URL such as https://xn--wmq0m700b.jp/h30 is considered to be invaild URL.

This happens because underlying library "validators (0.12.0)" does not recognize the URL above as "correct" URL.

solution:
Use the latest version of library such as validators 0.18.1

Make it easy to add aliases to a go link

For instance, add the aliases go/2020 and go/okrs to the go link go/2020okrs. We're getting more and more requests for this type of functionality.

The current workaround is to chain go links, but that's non-obvious and time-consuming.

Support explicit setting of upstream host via header

When running an instance of Trotto in an environment like App Engine that uses the request host to resolve to the correct application, a reverse proxy like an nginx server needs a way to tell the app to use a different upstream host when redirecting to an authentication system.

Any plan to allow self-hosted domains in Chrome extension?

I love that this is now open source, it's really helpful for projects that have compliance constraints (e.g., an org I work with has to stay CJIS-compliant, so would prefer to self-host).

That said, the magic really comes from the go/linkness of it. The Chrome extension currently only handles go/ and trot.to, which means it wouldn't work with the self-hosted option today (if I'm understanding correctly). Do you have any plans to enable it to take other domains? Maybe using optional permissions (https://developer.chrome.com/extensions/permission_warnings#optional_events)?

I'm sure it's not on your critical path as a company, so understood if this is not planned.

go-links can no longer deploy to Heroku due to sqlalchemy dropping support to postgres:// URI

Deploying to Heroku today produces:

2021-03-28T10:39:46.332245+00:00 app[web.1]: "Can't load plugin: %s:%s" % (self.group, name)
2021-03-28T10:39:46.332354+00:00 app[web.1]: sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres

This is apparently caused by sqlalchemy 1.4.x dropping support for postgres:// URI (keeping only postgresql://):
sqlalchemy/sqlalchemy#6083
https://help.heroku.com/ZKNTJQSK/why-is-sqlalchemy-1-4-x-not-connecting-to-heroku-postgres

Options to fix include:

• pinning SQLAlchemy at 1.3.x (1.3.23)
• implementing URI substitution from postgres:// to postgresql://

Allow double hyphens in destination urls

I'm trying to add a go-link in trot.to, of the form "https://double--hyphen.example.com/some/path",
however I get "You must provide a valid destination URL" message. The problem apparently is the "--" infix.

According to RFC1035 a valid domain can have "--" infix (unless RFC 5891 4.2.3.1 applies, which is not the case for my example).
Please fix.

Validate config on startup

Should include:

• Validating the shape and types
• Validating that all admins pulled from a single app.yml config file are in the same org

Allow a placeholder in programmatic go links to include slashes

Right now in a programmatic go link, a %s will only include text in between slashes.

For example:
If go/cs/%s -> https://cs.github.com/?q=%s
then go/cs/blah will redirect me to https://cs.github.com/?q=blah
but go/cs/foo/bar will not redirect to https://cs.github.com/?q=foo/bar. (This is particularly relevant when searching for file paths.)

We'd probably need another placeholder token (maybe%q?) as to not conflict with allowing multiple placeholders.

For now one verbose workaround is to do something like:
go/cs/%s -> https://cs.github.com/?q=%s
go/cs/%s/%s -> https://cs.github.com/?q=%s/%s
go/cs/%s/%s/%s -> https://cs.github.com/?q=%s/%s/%s
go/cs/%s/%s/%s/%s -> https://cs.github.com/?q=%s/%s/%s/%s
... until you get to the max depth that you care about.

Detect deactivated owners (i.e. those who have left the company) and allow anyone to edit

I don't know if this is possible - would it be possible to fetch and detect if a user exists - if they don't, allow anyone to edit a link?

Issue with python3.7

Was getting this error running the Docker container:

File "/usr/src/app/server/src/modules/links/handlers.py", line 98, in <module>
links: list[dict[str, Any]],
TypeError: 'type' object is not subscriptable

Updated the Dockerfile from FROM python:3.7-buster to FROM python:3.9-buster helped resolve this issue. Not sure what this will impact, however.

Infinite Google Auth Redirect

When running the local dev server, the Google OAuth flow is stuck in an infinite loop, never logging in but redirecting back to the auth screen.

Open-source the Chrome extension

We're in the process of open-sourcing the Trotto Chrome extension as well. If you're interested in contributing to that project, or if you'd just like to be notified when it's open-sourced, comment on or subscribe to this issue.

Too many steps to delete a go/link

Either allow 1 click delete with a toaster with a timeout to undelete

or

two click confirmation Delete row item -> Delete in Modal

Don't make me type the link out, its just a pain.

Allow unauthenticated users to view links

My links aren't private.

I'd like anyone at my organization to be able to use them without logging in.

   """Returns (shortlink_object, actual_destination)."""
   perfect_match = models.ShortLink.query(
-      models.ShortLink.organization == organization,
       models.ShortLink.shortpath == shortpath).get()

-      self.render_login_selector_page(redirect_to_after_oauth=self.request.path_url)
-      return
+      #self.render_login_selector_page(redirect_to_after_oauth=self.request.path_url)
+      #return

Deploy to DPLYR Docs

This project can be deployed to DPLYR easily, why not add docs for this. I can help you with that

subscribe to daily emails of changes (probably only self hosted)

In a self hosted deployment, such as an internal go links for a company, it might be nice to find out about adds/changes/deletes to the links. A daily email seems about the right cadence.

AttributeError: module 'sqlalchemy' has no attribute '__all__'

I can't figure out how to solve this issue, it was working fine until it wasn't.

2023-01-26 21:06:40   File "/usr/src/app/server/src/main.py", line 106, in <module>
2023-01-26 21:06:40     app = init_app_without_routes()
2023-01-26 21:06:40   File "/usr/src/app/server/src/main.py", line 54, in init_app_without_routes
2023-01-26 21:06:40     db = SQLAlchemy(app)
2023-01-26 21:06:40   File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 720, in __init__
2023-01-26 21:06:40     _include_sqlalchemy(self, query_class)
2023-01-26 21:06:40   File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 74, in _include_sqlalchemy
2023-01-26 21:06:40     for key in module.__all__:
2023-01-26 21:06:40 AttributeError: module 'sqlalchemy' has no attribute '__all__'

Initial db migration failure causes trotto app to return 500 internal server error

The symptom and steps to reproduce:

1. Launch trotto instance.
2. Launch postgres instance.
3. Trotto instance keeps returning 500 internal server error.

Root cause:
When trotto app and postgres instance are written in one docker-compose file,
both are launched almost simultaneously.

Solution:
a: add health check to trotto instance, so it will eventually restarted, and DB migration after restart should success.
b: add retry capability to the initial database migration process.
c: inform users to make sure to launch database prior to trotto instance.

Are there plans to provide a Dockerfile?

I think it would be great to have an alternative cloud-agnostic way to deploy this app.

And if not - it would also be great to know that there aren't any plans :)

• Dominik

Switch to psycopg2-binary

Ideally python deps would be wheel installs where possible. psycopg2-binary is a package that resolves that. It avoids needing headers installed for python/postgres client (python3-dev, libpq-dev) to install the bindings.

Self Hosting on Raspberry Pi

Hello!

I'd like to self-host this on my Raspberry Pi 4 (arm64), however it does not look like the current Dockerfile is compatible. Any chance this is on the roadmap? Alternatively, am I a dumb dumb and just did something wrong?

Support for deployment outside of GCP

Hi! I love Trotto, thank you for making it open source!

Today I installed Trotto on a Raspberry Pi, but it was a bit complicated, mainly because of the tight coupling the current code has with GCP. I had to work around a handful of issues until I managed to get it to work.

I wonder if that kind of support is something that interests you. If so, I can try to put some effort into polishing those workarounds and make some Pull Requests.

If not, I completely understand! I'll keep the hacks to myself 😝

SQLAlchemy Error running Web and Postgres in different pods

Thanks for making this project, it's awesome.

We're running this service in k8s on GKE using a postgres:12 image as the database container in a seperate pod. We're seeing a lot of issues in the logs like below:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/src/app/go-links/server/src/main.py", line 77, in home
    if not current_user.is_authenticated:
  File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 306, in _get_current_object
    return self.__local()
  File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 26, in <lambda>
    current_user = LocalProxy(lambda: _get_user())
  File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 346, in _get_user
    current_app.login_manager._load_user()
  File "/usr/local/lib/python3.7/site-packages/flask_login/login_manager.py", line 318, in _load_user
    user = self._user_callback(user_id)
  File "/usr/src/app/go-links/server/src/main.py", line 54, in load_user
    return get_user_by_id(user_id)
  File "/usr/src/app/go-links/server/src/modules/users/helpers.py", line 34, in get_user_by_id
    return models.User.get_by_id(user_id)
  File "/usr/src/app/go-links/server/src/modules/data/implementations/postgres/users.py", line 27, in get_by_id
    return User.query.get(int(id))
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1018, in get
    return self._get_impl(ident, loading.load_on_pk_identity)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1135, in _get_impl
    return db_load_fn(self, primary_key_identity)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 286, in load_on_pk_identity
    return q.one()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3463, in one
    ret = self.one_or_none()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3432, in one_or_none
    ret = list(self)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3508, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3533, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
    distilled_params,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

[SQL: SELECT "user".id AS user_id, "user".created AS user_created, "user".modified AS user_modified, "user".email AS user_email, "user".organization AS user_organization, "user".role AS user_role, "user".accepted_terms_at AS user_accepted_terms_at, "user".domain_type AS user_domain_type, "user".notifications AS user_notifications 
FROM "user" 
WHERE "user".id = %(param_1)s]
[parameters: {'param_1': 2}]

I'm testing the workaround listed here now: https://stackoverflow.com/questions/55457069/how-to-fix-operationalerror-psycopg2-operationalerror-server-closed-the-conn

It's been up and running for a few hours with no issues so far so looking good. Here's my patch on server/src/main.py

if get_database() == 'postgres':
    app.config['SQLALCHEMY_DATABASE_URI'] = get_config()['postgres']['url']
    app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False

    global db
    global migrate
    class SQLAlchemy(_BaseSQLAlchemy):
      def apply_pool_defaults(self, app, options):
          super(SQLAlchemy, self).apply_pool_defaults(app, options)
          options["pool_pre_ping"] = True
    db = SQLAlchemy(app)
    migrate = Migrate(app, db)

Python Version for Heroku Install

Heroku says the python version specified is not compatible.

Is it possible to use self host trotto and disable authentication?

I want to self host trotto to use go links on my local network, I don't need Google Oauth. Is it possible to disable it and have my trotto go links publicly available?

Allow underscores in URLs (Diff in comments, can I have permissions to push a feature branch to make a PR with?)

so instead of go/projecttemplate, have go/project_template?

The https://trot.to/#/ link generator strips underscores when making links.

Delete links

I don't see a way to delete links.

I mispelled the go link and would like to declutter the list.

Bad 'A conflicting go link already exists' Message

When trying to add a few search links to the same domain is says they're conflicting.

Examples that say they conflict:

go link	full link
https://go/dd/apm/%s	https://app.datadoghq.com/apm/home?search=%s
https://go/dd/dash/%s	https://app.datadoghq.com/dashboard/lists?q=%s
https://go/dd/logs/%s	https://app.datadoghq.com/logs?query=service%3A%2A%s%2A

Example Screenshot:

Slashes for Hierarchy Style Links

How hard would it be to add in the functionality of allowing / ? The reason I’m asking is that it would be cool to be able to create our own little structures like go/kb/zoom would go to the zoom documentation in notion etc

Same with the ability for all of Client X’s stuff or any client in general. I feel like if we could somehow map out a hierarchy structure using slashes I could see a lot of potentials there maybe.

Like Client X’s weekly meeting would be go/clientx/weekly or something and the daily standups would be go/clientx/daily and go/clientx/notion or go/clientx/kb or something idk just a thought

Expand environment configurability

Would love to be able to configure more basics via an environment variable, including:

• Admins
• Sentry DSN

This makes it fairly easy to deploy to Google Cloud Run / Kubernetes without ever managing a custom configuration file or custom docker image.

I will throw up a PR if yall are in favor. Will take a look at other relevant non-layout settings that might make sense as well.

The "go link already exists" message should add a link to the target

Sometimes I mistype a go/ link and land on the page to create one.
But then I correct the link in the go/[ text form ].
And I get a message like:

That go link already exists. go/foo points to https://foo.bar/baz

I would love it if https://foo.bar/baz was an actual link!

[Feature request] Ability to categorize go/ links

When multiple teams of people are all using the same Trotto instance, browsing go/ links on the admin dashboard very quickly becomes cumbersome and overwhelming. I sure wish Trotto provided the ability to categorize the links we create.

A simple hashtag system would allow us to categorize links by team or subject, providing the ability to find links much more easily. For example, seeing all the links directly relevant to our support team would make Trotto significantly more useful for our team.

Right now we're swimming in irrelevant go/ links created by other teams necessitating the creation of a 3rd party resource to track all the links our team is creating. Keeping this external documentation up-to-date with our go/ links is proving to be quite a chore.

missing file for local dev setup

appears as if there is a missing file at this time;
https://github.com/trotto/go-links/blob/master/server/run_local.sh#L5

dev_appserver.py does not exist in master branch.

Link fallback

It'd be great if there was a way to define a link fallback that matches when no other go link is found. That way we can specify more custom directory search/other search behaviour.

Auto generate keywords

How about an option to auto-generate the keyword?
Sometimes I just need a shorter url, not a memorable one.

Migration of older data

Is there any documented way of migrating old data from some other golinks solution (given base data such as creator, shortened url and target) to a new db ?

Maybe a dev API I am missing?

Container size

Hi, are there any plans to work on smaller container size? Currently it's almost 500mb while node containers are ofter ~80mb and go even 5mb.
Maybe there should be used alpine images?

Search Feature

Would like to see a search functionality so that when I type in go/ that i can just search the list to try to find a specific go link

LDAP authentication support

Are there any plans to add LDAP authentication support in addition to Google?

Most companies use LDAP for authentication internally so it would be great to have support for the same.

Docker file does not compile

The dockerfile will not compile for me, it seems related to a version conflict introduced maybe as recent as yesterday between Cython and PYYaml:
https://stackoverflow.com/questions/76708329/docker-compose-no-longer-building-image-attributeerror-cython-sources

The fix is to simply update PyYAML to 6.0.1 in both places (I'm assuming this does not cause breaking changes with the way it's used), which fixed the build and allowed me to progress with setup.

Top nav links in Trot.to/#/ are unresponsive in Chrome / ChromeOS 84.0.4147.110

The Directory How it Works and Sign out links are unclickable in ChromeOS Chrome browser, you can navigate to them with tab button and they are functional, but you can't click on them with the cursor. The same issue happens with the keyword and link fields below, but inconsistently reproducible, whereas the top nave fails 100% of the time.

False positive on punycode -ed URL

Legitimate URL such as https://xn--wmq0m700b.jp/h30 is considered to be invaild URL.

This happens because underlying library "validators (0.12.0)" does not recognize the URL above as "correct" URL.

solution:
Use the latest version of library such as validators 0.18.1

Pin python-version

Dockerfile is using 3.7, readme says 3.8. Is there a suggested version and should it also be pinned via pyenv (.python-version)?