ajay9022/google_cloud_function_log_extractor

this is a simple webserver which extracts the GCP cloud function logs from the GCP stackdriver(Logging) module. The user selects the desired registry, Module Name, State Machine Name and based on that the webserver reads the config details of that specific registry to extract the logs of the specific cloud-functions. This can be known from the config.ini file.

Google_Cloud_Function_Log_Extractor

Description of Python files:

      ret.py - contains the logic of API calls and stores whichever functions have completed, running, did not run. (Remove this logic from ret.py and instead implement it inside the index.html)

      test.py - contains the logic of extracting the config details and host the server.

      /templates/index.html - Front end for the user.

      temp.py - You can try playing with the code with temp.py. It is a temporary code that makes the API calls.

Few functionalities that are remaining to be implemented in the future are:

      1. Create FILTER for tag, module_name, sm_name for extracting only specific logs. Also extract logs that have "Function execution started" and "function took x milliseconds".

      2. Auto Refresh functionality - Extract logs after every "frequency" number of seconds.

      3. CSV Export - I have partially implemented this functionality but facing some issues. Hope this can be resolved easily. :D

      4. Try logging_v2 API for extracting logs to check if it is fast enough to retrieve the logs.

      5. Support Multiple user - The same URL should be accessible to multiple user and should aid multiple users the same way as it supports a single one without any interference to other users.

      6. Colourise the Function_names buttons based on reading the logs present inside the textField rather than the logic written inside the API call list_entries() inside ret.py.

References:

      https://googleapis.dev/python/logging/latest/index.html
      https://googleapis.dev/python/logging/latest/client.html
      https://github.com/googleapis/python-logging/tree/master/samples/snippets
      https://cloud.google.com/logging/docs/view/advanced-queries
      https://googleapis.dev/python/logging/latest/handlers.html
      https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry
      https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry
      and ofcourse https://stackoverflow.com/ :D