thundra-io/thundra-agent-python

Thundra Lambda Python Agent

thundra-agent-python's Issues

module 'botocore.vendored.requests' has no attribute 'Session'

Thundra setup is causing a following error on AWS Lambda:

module initialization error
module 'botocore.vendored.requests' has no attribute 'Session'

The recent updates in the botocore could be a reason to that: https://aws.amazon.com/blogs/developer/removing-the-vendored-version-of-requests-from-botocore/.

Might be related to the following line: https://github.com/thundra-io/thundra-lambda-agent-python/blob/master/thundra/integrations/modules/botocore.py#L40, couldn't find any other mentions of botocore.vendored.requests in the source code.

Broken link in the README.md

The link setup async monitoring
https://docs.thundra.io/docs/how-to-setup-async-monitoring
is broken

Django server crashes on start

After adding thundra to my django settings, my container crashes on start.

From my settings:

import thundra

thundra.configure(
    options={"config": {"thundra.apikey": "xxxx",}}
)

The command it fails on:

python /code/manage.py rundebugserver 0.0.0.0:80

Trace:

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", line 53, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 121, in inner_run
    self.check_migrations()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 459, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/loader.py", line 53, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/loader.py", line 216, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 55, in has_table
    with self.connection.cursor() as cursor:
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 259, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 235, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 206, in get_new_connection
    psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)
  File "/usr/local/lib/python3.7/site-packages/psycopg2/_json.py", line 156, in register_default_jsonb
    loads=loads, oid=JSONB_OID, array_oid=JSONBARRAY_OID, name='jsonb')
  File "/usr/local/lib/python3.7/site-packages/psycopg2/_json.py", line 125, in register_json
    register_type(JSON, not globally and conn_or_curs or None)
TypeError: argument 2 must be a connection, cursor or None

My dependencies:

python = "3.7.7"
argon2-cffi = "^20.1.0"
asgiref = "^3.2.10"
boto3 = "^1.14.45"
dj-database-url = "^0.5.0"
django = "^3.1.4"
django-axes = "^5.4.3"
django-memoize = "^2.2.1"
django-storages = "^1.9.1"
django-enhanced-emails = "^0.0.7"
django-ses = "^1.0.2"
djangorestframework = "^3.12.2"
djangorestframework_simplejwt = "^4.4.0"
djoser = "^2.0.3"
psycopg2-binary = "^2.8.6"
pytz = "^2020.1"
requests = "^2.24.0"
sentry-sdk = "^0.19.5"
sqlparse = "^0.3.1"
watchtower = "^0.8.0"
django-phonenumber-field = {version = "^4.0.0", extras = ["phonenumbers"]}
sodapy = "^2.1.0"
httpx = "^0.13.1"
pandas = "^1.1.1"
xlrd = "^1.2.0"
django-filter = "^2.2.0"
django-import-export = "^2.2.0"
scrapy = "^2.1.0"
scrapy_djangoitem = "^1.1.1"
django-dirtyfields = "^1.4.1"
thundra = "^2.5.7"

It also fails, with the same error, on a production gunicorn start.

Time of availability ?

Hi guys,

It is not an issue, but I have a lot of serverless functions written in python, and I would like to know when do you think the python agent could be available ?

Thanks !
Best

Thundra causes exception in lambda that uses SSM

SSM is important for secrets management.
Thundra injects itself into botocore and tries to wrap functions. But if it doesn't have a subclass for the required function it just crashes.

File "/var/runtime/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/task/thundra/integrations/modules/botocore.py", line 22, in _wrapper
response = INTEGRATIONS[integration_name].create_span(
KeyError: 'ssm'

The issue didn't exist before adding thundra, and removing thundra makes the issue go away.

Note: this happens in a call to boto3 that occurs before thundra agent is loaded:

        ssm_client = boto3.client('ssm', region_name='us-east-1')
        response = ssm_client.get_parameter(parameter_name)

AutoTracing Support For Lambda -> Lambda Asynchronous Invocation

👋 I have 2 simple lambdas setup using Asynchronous invocation, but they are not showing up as connected in the thundra UI.

I looked at the docs but I think I am missing something. Could you please advise on the best practice for propagating a trace to the asynchronously invoked lambda using thundra python client??

Thank you

Lambda 1 AWS Config Events

Lambda 1

from thundra.thundra_agent import Thundra

thundra = Thundra()

@thundra
def handler(event, context):
    print(event, context)
    return {'message': 'hello'}

Lambda 2

from thundra.thundra_agent import Thundra

thundra = Thundra()

@thundra
def handler(event, context):
    print('received: {} {}'.format(event, context))
    return {'hi': 'hi'}

Architecture