trotto / go-links Goto Github PK
View Code? Open in Web Editor NEWThe open source go links app at the core of Trotto.
Home Page: https://www.trot.to
License: Apache License 2.0
The open source go links app at the core of Trotto.
Home Page: https://www.trot.to
License: Apache License 2.0
Are there any plans to add LDAP authentication support in addition to Google?
Most companies use LDAP for authentication internally so it would be great to have support for the same.
When running an instance of Trotto in an environment like App Engine that uses the request host to resolve to the correct application, a reverse proxy like an nginx server needs a way to tell the app to use a different upstream host when redirecting to an authentication system.
Right now in a programmatic go link, a %s
will only include text in between slashes.
For example:
If go/cs/%s
-> https://cs.github.com/?q=%s
then go/cs/blah
will redirect me to https://cs.github.com/?q=blah
but go/cs/foo/bar
will not redirect to https://cs.github.com/?q=foo/bar
. (This is particularly relevant when searching for file paths.)
We'd probably need another placeholder token (maybe%q
?) as to not conflict with allowing multiple placeholders.
For now one verbose workaround is to do something like:
go/cs/%s
-> https://cs.github.com/?q=%s
go/cs/%s/%s
-> https://cs.github.com/?q=%s/%s
go/cs/%s/%s/%s
-> https://cs.github.com/?q=%s/%s/%s
go/cs/%s/%s/%s/%s
-> https://cs.github.com/?q=%s/%s/%s/%s
... until you get to the max depth that you care about.
I haven't tested this, but I am not sure if the app is known to work with multiple instances, in say a k8s load balanced scenario using PG for the DB.
If not, what do you think some of the areas of code would prevent this.
I am looking to balance the load along with handling cases with a node is drained
Thanks!
Legitimate URL such as https://xn--wmq0m700b.jp/h30 is considered to be invaild URL.
This happens because underlying library "validators (0.12.0)" does not recognize the URL above as "correct" URL.
solution:
Use the latest version of library such as validators 0.18.1
Sometimes I mistype a go/ link and land on the page to create one.
But then I correct the link in the go/[ text form ].
And I get a message like:
That go link already exists. go/foo points to https://foo.bar/baz
I would love it if https://foo.bar/baz was an actual link!
I see that Trotto only currently supports Google login - is there any chance of support for Microsoft's Azure SSO? (or at least some pointers on how to add it?)
When multiple teams of people are all using the same Trotto instance, browsing go/ links on the admin dashboard very quickly becomes cumbersome and overwhelming. I sure wish Trotto provided the ability to categorize the links we create.
A simple hashtag system would allow us to categorize links by team or subject, providing the ability to find links much more easily. For example, seeing all the links directly relevant to our support team would make Trotto significantly more useful for our team.
Right now we're swimming in irrelevant go/ links created by other teams necessitating the creation of a 3rd party resource to track all the links our team is creating. Keeping this external documentation up-to-date with our go/ links is proving to be quite a chore.
How about an option to auto-generate the keyword?
Sometimes I just need a shorter url, not a memorable one.
Hi! I love Trotto, thank you for making it open source!
Today I installed Trotto on a Raspberry Pi, but it was a bit complicated, mainly because of the tight coupling the current code has with GCP. I had to work around a handful of issues until I managed to get it to work.
I wonder if that kind of support is something that interests you. If so, I can try to put some effort into polishing those workarounds and make some Pull Requests.
If not, I completely understand! I'll keep the hacks to myself 😝
Legitimate URL such as https://xn--wmq0m700b.jp/h30 is considered to be invaild URL.
This happens because underlying library "validators (0.12.0)" does not recognize the URL above as "correct" URL.
solution:
Use the latest version of library such as validators 0.18.1
When running locally
./run_local.sh: line 5: dev_appserver.py: command not found
hello there,
First of all. Thanks for sharing this tool with us!!!
I get an error when I build the docker.
sudo docker run -p 8443:8000 trotto ../scripts/upgrade_db.sh: 1: cd: can't cd to server/src INFO [alembic.runtime.migration] Context impl PostgresqlImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. [2022-10-10 15:20:14 +0000] [16] [INFO] Starting gunicorn 20.0.4 [2022-10-10 15:20:14 +0000] [16] [INFO] Listening at: http://0.0.0.0:8000 (16) [2022-10-10 15:20:14 +0000] [16] [INFO] Using worker: sync [2022-10-10 15:20:14 +0000] [19] [INFO] Booting worker with pid: 19 [2022-10-10 15:20:14 +0000] [20] [INFO] Booting worker with pid: 20 [2022-10-10 15:20:14 +0000] [21] [INFO] Booting worker with pid: 21 [2022-10-10 15:20:14 +0000] [22] [INFO] Booting worker with pid: 22
The original, non-docker file (upgrade_db.sh) located in the cloned repo: server/scripts/ points to a relative path:
server/scripts/upgrade_db.sh:
cd server/src
export FLASK_APP=main.py
flask db upgrade
The command cd server/src
could be replaced by the full docker folder path: cd /usr/src/app/server/src/
I resolved it by replacing the cd relative path in server/scripts/upgrade_db.sh by a cd to the absolute docker folder path (cd /usr/src/app/server/src/) and by rebuilding the docker after.
server/scripts/upgrade_db.sh:
cd /usr/src/app/server/src/
export FLASK_APP=main.py
Thanks for everything!
Thanks for making this project, it's awesome.
We're running this service in k8s on GKE using a postgres:12 image as the database container in a seperate pod. We're seeing a lot of issues in the logs like below:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/src/app/go-links/server/src/main.py", line 77, in home
if not current_user.is_authenticated:
File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__
return getattr(self._get_current_object(), name)
File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 306, in _get_current_object
return self.__local()
File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 26, in <lambda>
current_user = LocalProxy(lambda: _get_user())
File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 346, in _get_user
current_app.login_manager._load_user()
File "/usr/local/lib/python3.7/site-packages/flask_login/login_manager.py", line 318, in _load_user
user = self._user_callback(user_id)
File "/usr/src/app/go-links/server/src/main.py", line 54, in load_user
return get_user_by_id(user_id)
File "/usr/src/app/go-links/server/src/modules/users/helpers.py", line 34, in get_user_by_id
return models.User.get_by_id(user_id)
File "/usr/src/app/go-links/server/src/modules/data/implementations/postgres/users.py", line 27, in get_by_id
return User.query.get(int(id))
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1018, in get
return self._get_impl(ident, loading.load_on_pk_identity)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1135, in _get_impl
return db_load_fn(self, primary_key_identity)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 286, in load_on_pk_identity
return q.one()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3463, in one
ret = self.one_or_none()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3432, in one_or_none
ret = list(self)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3508, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3533, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: SELECT "user".id AS user_id, "user".created AS user_created, "user".modified AS user_modified, "user".email AS user_email, "user".organization AS user_organization, "user".role AS user_role, "user".accepted_terms_at AS user_accepted_terms_at, "user".domain_type AS user_domain_type, "user".notifications AS user_notifications
FROM "user"
WHERE "user".id = %(param_1)s]
[parameters: {'param_1': 2}]
I'm testing the workaround listed here now: https://stackoverflow.com/questions/55457069/how-to-fix-operationalerror-psycopg2-operationalerror-server-closed-the-conn
It's been up and running for a few hours with no issues so far so looking good. Here's my patch on server/src/main.py
if get_database() == 'postgres':
app.config['SQLALCHEMY_DATABASE_URI'] = get_config()['postgres']['url']
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
global db
global migrate
class SQLAlchemy(_BaseSQLAlchemy):
def apply_pool_defaults(self, app, options):
super(SQLAlchemy, self).apply_pool_defaults(app, options)
options["pool_pre_ping"] = True
db = SQLAlchemy(app)
migrate = Migrate(app, db)
I'm trying to add a go-link in trot.to, of the form "https://double--hyphen.example.com/some/path",
however I get "You must provide a valid destination URL" message. The problem apparently is the "--" infix.
According to RFC1035 a valid domain can have "--" infix (unless RFC 5891 4.2.3.1 applies, which is not the case for my example).
Please fix.
I think it would be great to have an alternative cloud-agnostic way to deploy this app.
And if not - it would also be great to know that there aren't any plans :)
When running the local dev server, the Google OAuth flow is stuck in an infinite loop, never logging in but redirecting back to the auth screen.
Hello!
I'd like to self-host this on my Raspberry Pi 4 (arm64), however it does not look like the current Dockerfile is compatible. Any chance this is on the roadmap? Alternatively, am I a dumb dumb and just did something wrong?
Would love to be able to configure more basics via an environment variable, including:
This makes it fairly easy to deploy to Google Cloud Run / Kubernetes without ever managing a custom configuration file or custom docker image.
I will throw up a PR if yall are in favor. Will take a look at other relevant non-layout settings that might make sense as well.
In a self hosted deployment, such as an internal go links for a company, it might be nice to find out about adds/changes/deletes to the links. A daily email seems about the right cadence.
Would like to see a search functionality so that when I type in go/ that i can just search the list to try to find a specific go link
Ideally python deps would be wheel installs where possible. psycopg2-binary is a package that resolves that. It avoids needing headers installed for python/postgres client (python3-dev, libpq-dev) to install the bindings.
I love that this is now open source, it's really helpful for projects that have compliance constraints (e.g., an org I work with has to stay CJIS-compliant, so would prefer to self-host).
That said, the magic really comes from the go/linkness of it. The Chrome extension currently only handles go/ and trot.to, which means it wouldn't work with the self-hosted option today (if I'm understanding correctly). Do you have any plans to enable it to take other domains? Maybe using optional permissions (https://developer.chrome.com/extensions/permission_warnings#optional_events)?
I'm sure it's not on your critical path as a company, so understood if this is not planned.
Heroku says the python version specified is not compatible.
I want to self host trotto to use go links on my local network, I don't need Google Oauth. Is it possible to disable it and have my trotto go links publicly available?
The dockerfile will not compile for me, it seems related to a version conflict introduced maybe as recent as yesterday between Cython and PYYaml:
https://stackoverflow.com/questions/76708329/docker-compose-no-longer-building-image-attributeerror-cython-sources
The fix is to simply update PyYAML to 6.0.1 in both places (I'm assuming this does not cause breaking changes with the way it's used), which fixed the build and allowed me to progress with setup.
I don't see a way to delete links.
I mispelled the go link and would like to declutter the list.
Was getting this error running the Docker container:
File "/usr/src/app/server/src/modules/links/handlers.py", line 98, in <module>
links: list[dict[str, Any]],
TypeError: 'type' object is not subscriptable
Updated the Dockerfile from FROM python:3.7-buster
to FROM python:3.9-buster
helped resolve this issue. Not sure what this will impact, however.
Should include:
app.yml
config file are in the same orgDeploying to Heroku today produces:
2021-03-28T10:39:46.332245+00:00 app[web.1]: "Can't load plugin: %s:%s" % (self.group, name)
2021-03-28T10:39:46.332354+00:00 app[web.1]: sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres
This is apparently caused by sqlalchemy 1.4.x dropping support for postgres:// URI (keeping only postgresql://):
sqlalchemy/sqlalchemy#6083
https://help.heroku.com/ZKNTJQSK/why-is-sqlalchemy-1-4-x-not-connecting-to-heroku-postgres
Options to fix include:
For instance, add the aliases go/2020
and go/okrs
to the go link go/2020okrs
. We're getting more and more requests for this type of functionality.
The current workaround is to chain go links, but that's non-obvious and time-consuming.
appears as if there is a missing file at this time;
https://github.com/trotto/go-links/blob/master/server/run_local.sh#L5
dev_appserver.py
does not exist in master branch.
The Directory How it Works and Sign out links are unclickable in ChromeOS Chrome browser, you can navigate to them with tab button and they are functional, but you can't click on them with the cursor. The same issue happens with the keyword and link fields below, but inconsistently reproducible, whereas the top nave fails 100% of the time.
We're in the process of open-sourcing the Trotto Chrome extension as well. If you're interested in contributing to that project, or if you'd just like to be notified when it's open-sourced, comment on or subscribe to this issue.
so instead of go/projecttemplate, have go/project_template?
The https://trot.to/#/ link generator strips underscores when making links.
Hi, are there any plans to work on smaller container size? Currently it's almost 500mb while node containers are ofter ~80mb and go even 5mb.
Maybe there should be used alpine images?
The symptom and steps to reproduce:
Root cause:
When trotto app and postgres instance are written in one docker-compose file,
both are launched almost simultaneously.
Solution:
a: add health check to trotto instance, so it will eventually restarted, and DB migration after restart should success.
b: add retry capability to the initial database migration process.
c: inform users to make sure to launch database prior to trotto instance.
When trying to add a few search links to the same domain is says they're conflicting.
Examples that say they conflict:
I don't know if this is possible - would it be possible to fetch and detect if a user exists - if they don't, allow anyone to edit a link?
It'd be great if there was a way to define a link fallback that matches when no other go link is found. That way we can specify more custom directory search/other search behaviour.
Either allow 1 click delete with a toaster with a timeout to undelete
or
two click confirmation Delete row item -> Delete in Modal
Don't make me type the link out, its just a pain.
This project can be deployed to DPLYR easily, why not add docs for this. I can help you with that
Is there any documented way of migrating old data from some other golinks solution (given base data such as creator, shortened url and target) to a new db ?
Maybe a dev API I am missing?
I can't figure out how to solve this issue, it was working fine until it wasn't.
2023-01-26 21:06:40 File "/usr/src/app/server/src/main.py", line 106, in <module>
2023-01-26 21:06:40 app = init_app_without_routes()
2023-01-26 21:06:40 File "/usr/src/app/server/src/main.py", line 54, in init_app_without_routes
2023-01-26 21:06:40 db = SQLAlchemy(app)
2023-01-26 21:06:40 File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 720, in __init__
2023-01-26 21:06:40 _include_sqlalchemy(self, query_class)
2023-01-26 21:06:40 File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 74, in _include_sqlalchemy
2023-01-26 21:06:40 for key in module.__all__:
2023-01-26 21:06:40 AttributeError: module 'sqlalchemy' has no attribute '__all__'
My links aren't private.
I'd like anyone at my organization to be able to use them without logging in.
"""Returns (shortlink_object, actual_destination)."""
perfect_match = models.ShortLink.query(
- models.ShortLink.organization == organization,
models.ShortLink.shortpath == shortpath).get()
- self.render_login_selector_page(redirect_to_after_oauth=self.request.path_url)
- return
+ #self.render_login_selector_page(redirect_to_after_oauth=self.request.path_url)
+ #return
How hard would it be to add in the functionality of allowing / ? The reason I’m asking is that it would be cool to be able to create our own little structures like go/kb/zoom would go to the zoom documentation in notion etc
Same with the ability for all of Client X’s stuff or any client in general. I feel like if we could somehow map out a hierarchy structure using slashes I could see a lot of potentials there maybe.
Like Client X’s weekly meeting would be go/clientx/weekly or something and the daily standups would be go/clientx/daily and go/clientx/notion or go/clientx/kb or something idk just a thought
Dockerfile is using 3.7, readme says 3.8. Is there a suggested version and should it also be pinned via pyenv (.python-version
)?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.