discord/access

Access, a centralized portal for employees to transparently discover, request, and manage their access for all internal systems needed to do their jobs

access's Issues

Exceptional work! 🤩

As a fellow security engineer who works on GitHub's open source IAM system (entitlements), this is really awesome work!

use Okta's response headers for rate limits

transcript of (Gabriel's half of) a conversation with @somethingnew2-0

why do u use exponential backoff when Okta tells you when u can retry the api call?

access/api/services/okta_service.py

Lines 41 to 66 in 40d8c0b

	async def _retry(func: Callable[[Any], Any], *args: Any, **kwargs: Any) -> Any:
	"""Retry Okta API requests with specific status codes using exponential backoff."""
	for attempt in range(1 + REQUEST_MAX_RETRIES):
	result = await func(*args, **kwargs)
	if len(result) == 2:
	response, error = result
	elif len(result) == 3:
	_, response, error = result
	else:
	raise Exception("Unexpected result structure from Okta client.")
	if (attempt == REQUEST_MAX_RETRIES or
	error is None or
	response is None or
	(response is not None and response.get_status() not in RETRIABLE_STATUS_CODES)):
	return result
	if response is None:
	logger.warning('Got None response from Okta resource. Retrying...')
	else:
	logger.warning(f'Got {response.get_status()} response from Okta resource {response._url}, with error:'
	f' {error}. Retrying...'
	)
	await asyncio.sleep(RETRY_BACKOFF_FACTOR * (2**attempt))

if u get a 429 error that tells u when to retry, why not look at those headers?
eg (and this isn't perfect, but...)
https://github.com/gabrielsroka/gabrielsroka.github.io/blob/master/console/index.html#L169-L187

ie, if u reach the rate limit at 10:00:00 and it tells u to retry at 10:01:00, there's no point in retrying at 10:00:01.2, 10:00:02.4, 10:00:04.8. ur just gonna get more errors

Okta provides three headers in each response to report on both concurrent and org-wide rate limits.
For org-wide rate limits, the three headers show the limit that is being enforced, when it resets, and how close you are to hitting the limit:
X-Rate-Limit-Limit - the rate limit ceiling that is applicable for the current request.
X-Rate-Limit-Remaining - the number of requests left for the current rate-limit window.
X-Rate-Limit-Reset - the time at which the rate limit resets, specified in UTC epoch time (in seconds).

https://developer.okta.com/docs/reference/rl-best-practices/#check-your-rate-limits-with-okta-s-rate-limit-headers

retry on resp.next()

more converstaions with @somethingnew2-0

i noticed this "pattern" 4 times (paginating thru a given api)

access/api/services/okta_service.py

Lines 96 to 105 in 7fda498

	users, resp, error = await OktaService._retry(self.okta_client.list_users)
	if error is not None:
	raise Exception(error)
	assert users is not None and resp is not None
	while resp.has_next():
	more_users, _ = await resp.next()
	users.extend(more_users)
	return list(map(lambda user: User(user), users))

don't u need to retry the next(), too?

i use async generators for it. write it once, factor it out, reuse it everywhere. something like

import asyncio
from okta.client import Client as OktaClient

async def get_applications():
    async with OktaClient() as okta_client:
        filter = {'filter': 'status eq "ACTIVE"'}
        async for app in get_objects(okta_client.list_applications(filter)):
            print(f"Application Name: {app.name}")
            print(f"Application ID: {app.id}")
            async for app_user in get_objects(okta_client.list_application_users(app.id)):
                print(app_user.id)
            print()

async def get_objects(coro):
    objects, resp, _ = await coro
    while objects:
        for object in objects:
            yield object
        objects, _ = await resp.next() if resp.has_next() else (None, None)

asyncio.run(get_applications())

but that might not work here. i wrote it a while ago. maybe i should rewrite it

Argument 'postgresql_where' is not accepted by dialect 'postgresql' on behalf of <class 'sqlalchemy.sql.schema.UniqueConstraint'>

👋🏻 Hey guys, I saw your new blog and thought I should check it out.
🔎 Upon setting up, while running the flask db upgrade command to seed the DB, it turns out that it throws up an error.
⚠️ Error:

Traceback (most recent call last):
  File "/home/scott/access/venv/bin/flask", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/flask/cli.py", line 1107, in main
    cli.main()
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/flask/cli.py", line 388, in decorator
    return ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/flask_migrate/cli.py", line 154, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/home/scott/access/venv/lib/python3.11/site-packages/flask_migrate/__init__.py", line 111, in wrapped
    f(*args, **kwargs)
  File "/home/scott/access/venv/lib/python3.11/site-packages/flask_migrate/__init__.py", line 200, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/home/scott/access/venv/lib/python3.11/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/home/scott/access/venv/lib/python3.11/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/scott/access/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/scott/access/migrations/env.py", line 92, in <module>
    run_migrations_online()
  File "/home/scott/access/migrations/env.py", line 86, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/home/scott/access/venv/lib/python3.11/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/scott/access/venv/lib/python3.11/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
    step.migration_fn(**kw)
  File "/home/scott/access/migrations/versions/d6db40b0805d_initial_migration.py", line 79, in upgrade
    sa.UniqueConstraint("email", name=op.f('idx_email'), postgresql_where=sa.text("deleted_at IS NULL")),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/scott/access/venv/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 4340, in __init__
    Constraint.__init__(
  File "/home/scott/access/venv/lib/python3.11/site-packages/sqlalchemy/sql/schema.py", line 4117, in __init__
    self._validate_dialect_kwargs(dialect_kw)
  File "/home/scott/access/venv/lib/python3.11/site-packages/sqlalchemy/sql/base.py", line 618, in _validate_dialect_kwargs
    raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: Argument 'postgresql_where' is not accepted by dialect 'postgresql' on behalf of <class 'sqlalchemy.sql.schema.UniqueConstraint'>

💡 Keep in mind that I'm not very familiar with Python and databases. So I might be dumb at some times.