Giter Site home page Giter Site logo

hossain-khan / kotlin-google-hangouts-parser Goto Github PK

View Code? Open in Web Editor NEW
4.0 1.0 1.0 188 KB

Google Hangouts takeout JSON file parser. https://hossain-khan.github.io/kotlin-google-hangouts-parser/

Home Page: https://takeout.google.com

License: MIT License

Kotlin 100.00%
google-hangouts hangouts google-hangouts-chat parser google-takeout google-hangouts-json google-hangouts-json-parser hangouts-json-parser kotlin kotlin-library

kotlin-google-hangouts-parser's Introduction

Kotlin-hangouts-parser

Google Hangouts JSON Parser

Contains JSON model classes using Moshi JSON parser. See JSON model classes sources.

The Google Hangouts JSON file can be optained via Google Takeout at https://takeout.google.com/settings/takeout

See example below:
Google Takeout - Hangouts

Usage

I am still experimenting with the takeout JSON data. However, here is how to use the library to parse the Hangouts.json file (use JitPack.io to add gradle dependency)

// import dev.hossain.hangouts.Parser (also see `HangoutsJsonProcessor.kt` for sample usage)

    val file = File("/path/to/hangouts.json")

    val source: BufferedSource = Okio.buffer(Okio.source(file)) // Use buffer to read large file
    
    val hangoutsDocument: HangoutsDocument = Parser.parse(source)
    println("Completed processing - got ${hangoutsDocument.conversations.size} conversations.")

Data Snapshot

Here is a data snapshot taken from IntelliJ IDEA debugger.

SQLite Database

The example project creates a SQLite database using SQLDelight found in data module. See HangoutsJsonProcessor.kt for details.

Here is sample snapshot of output from the HangoutsJsonProcessor.kt.

┌────────────────┬─────────────────────────┬──────────┐
│                │                  Total  │   417    │
│                ├─────────────────────────┼──────────┤
│  Conversation  │     Group Conversation  │    41    │
│                ├─────────────────────────┼──────────┤
│                │      One-to-one Thread  │   376    │
├────────────────┼─────────────────────────┼──────────┤
│                │                  Total  │   435    │
│                ├─────────────────────────┼──────────┤
│  Participants  │           Google Users  │   112    │
│                ├─────────────────────────┼──────────┤
│                │       Non-Google (SMS)  │   323    │
├────────────────┼─────────────────────────┼──────────┤
│                │                  Total  │  176932  │
│                ├─────────────────────────┼──────────┤
│                │          Text Messages  │  168733  │
│  Chat Message  ├─────────────────────────┼──────────┤
│                │       Web URL Messages  │   3741   │
│                ├─────────────────────────┼──────────┤
│                │  Longest Message Chars  │   1446   │
└────────────────┴─────────────────────────┴──────────┘

Google Sheets Generated Charts

JSON Data Reference

Here is a simlified snapshot of JSON dump with conversation and events containing chat message.

{
  "conversations": [
    {
      "conversation": {
        "conversation_id": {
          "id": "cid-12345678"
        },
        "conversation": {
          "id": {
            "id": "cid-12345678"
          },
          "type": "GROUP",
          "self_conversation_state": {
            "self_read_state": {
              "participant_id": {
                "gaia_id": "users-google-user-id-123",
                "chat_id": "users-google-user-id-123"
              },
              "latest_read_timestamp": "1369968040768557"
            },
            "status": "ACTIVE",
            "notification_level": "RING",
            "view": [
              "ARCHIVED_VIEW"
            ],
            "inviter_id": {
              "gaia_id": "users-google-user-id-123",
              "chat_id": "users-google-user-id-123"
            },
            "invite_timestamp": "1369539123372000",
            "sort_timestamp": "1369968040768557",
            "active_timestamp": "1369539123372000",
            "delivery_medium_option": [
              {
                "delivery_medium": {
                  "medium_type": "BABEL_MEDIUM"
                },
                "current_default": true
              }
            ],
            "is_guest": false
          },
          "read_state": [
            {
              "participant_id": {
                "gaia_id": "users-google-user-id-567",
                "chat_id": "users-google-user-id-567"
              },
              "latest_read_timestamp": "0"
            },
            {
              "participant_id": {
                "gaia_id": "users-google-user-id-123",
                "chat_id": "users-google-user-id-123"
              },
              "latest_read_timestamp": "1369968040768557"
            },
            {
              "participant_id": {
                "gaia_id": "users-google-user-id-789",
                "chat_id": "users-google-user-id-789"
              },
              "latest_read_timestamp": "0"
            }
          ],
          "has_active_hangout": false,
          "otr_status": "ON_THE_RECORD",
          "otr_toggle": "ENABLED",
          "current_participant": [
            {
              "gaia_id": "users-google-user-id-567",
              "chat_id": "users-google-user-id-567"
            },
            {
              "gaia_id": "users-google-user-id-789",
              "chat_id": "users-google-user-id-789"
            },
            {
              "gaia_id": "users-google-user-id-123",
              "chat_id": "users-google-user-id-123"
            }
          ],
          "participant_data": [
            {
              "id": {
                "gaia_id": "users-google-user-id-567",
                "chat_id": "users-google-user-id-567"
              },
              "fallback_name": "yasir usama",
              "invitation_status": "ACCEPTED_INVITATION",
              "participant_type": "GAIA",
              "new_invitation_status": "ACCEPTED_INVITATION",
              "in_different_customer_as_requester": false,
              "domain_id": "domain-id-123"
            },
            {
              "id": {
                "gaia_id": "users-google-user-id-123",
                "chat_id": "users-google-user-id-123"
              },
              "fallback_name": "Hossain Khan",
              "invitation_status": "ACCEPTED_INVITATION",
              "participant_type": "GAIA",
              "new_invitation_status": "ACCEPTED_INVITATION",
              "in_different_customer_as_requester": false,
              "domain_id": "domain-id-123"
            },
            {
              "id": {
                "gaia_id": "users-google-user-id-789",
                "chat_id": "users-google-user-id-789"
              },
              "fallback_name": "Istiaque Ahmed",
              "invitation_status": "ACCEPTED_INVITATION",
              "participant_type": "GAIA",
              "new_invitation_status": "ACCEPTED_INVITATION",
              "in_different_customer_as_requester": false,
              "domain_id": "domain-id-123"
            }
          ],
          "fork_on_external_invite": false,
          "network_type": [
            "BABEL"
          ],
          "force_history_state": "NO_FORCE",
          "group_link_sharing_status": "LINK_SHARING_OFF"
        }
      },
      "events": [
        {
          "conversation_id": {
            "id": "cid-12345678"
          },
          "sender_id": {
            "gaia_id": "users-google-user-id-567",
            "chat_id": "users-google-user-id-567"
          },
          "timestamp": "1369539622561784",
          "self_event_state": {
            "user_id": {
              "gaia_id": "users-google-user-id-123",
              "chat_id": "users-google-user-id-123"
            },
            "notification_level": "RING"
          },
          "hangout_event": {
            "event_type": "END_HANGOUT",
            "participant_id": [
              {
                "gaia_id": "users-google-user-id-567",
                "chat_id": "users-google-user-id-567"
              },
              {
                "gaia_id": "users-google-user-id-123",
                "chat_id": "users-google-user-id-123"
              },
              {
                "gaia_id": "users-google-user-id-789",
                "chat_id": "users-google-user-id-789"
              }
            ],
            "hangout_duration_secs": "102"
          },
          "event_id": "7-H0Z7-Uqf17-H1W1MUbr7",
          "advances_sort_timestamp": true,
          "event_otr": "ON_THE_RECORD",
          "delivery_medium": {
            "medium_type": "BABEL_MEDIUM"
          },
          "event_type": "HANGOUT_EVENT",
          "event_version": "1369539622561784"
        },
        {
          "conversation_id": {
            "id": "cid-12345678"
          },
          "sender_id": {
            "gaia_id": "users-google-user-id-123",
            "chat_id": "users-google-user-id-123"
          },
          "timestamp": "1369548258628038",
          "self_event_state": {
            "user_id": {
              "gaia_id": "users-google-user-id-123",
              "chat_id": "users-google-user-id-123"
            },
            "client_generated_id": "1369548257038",
            "notification_level": "RING"
          },
          "chat_message": {
            "message_content": {
              "segment": [
                {
                  "type": "TEXT",
                  "text": "Chat message",
                  "formatting": {
                    "bold": false,
                    "italics": false,
                    "strikethrough": false,
                    "underline": false
                  }
                }
              ]
            }
          },
          "event_id": "7-H0Z7-Uqf17-HHzEcNqeT",
          "advances_sort_timestamp": true,
          "event_otr": "ON_THE_RECORD",
          "delivery_medium": {
            "medium_type": "BABEL_MEDIUM"
          },
          "event_type": "REGULAR_CHAT_MESSAGE",
          "event_version": "1369548258628038"
        }
      ]
    }
  ]
}

kotlin-google-hangouts-parser's People

Contributors

hossain-khan avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

kimhines

kotlin-google-hangouts-parser's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/gradle.yml
  • actions/checkout v4
  • actions/setup-java v1
gradle
gradle.properties
settings.gradle
build.gradle
  • com.squareup.sqldelight:gradle-plugin 1.4.0
  • org.jetbrains.kotlin.jvm 1.3.72
  • org.jetbrains.kotlin.kapt 1.3.72
  • com.squareup.moshi:moshi 1.15.1
  • com.squareup.moshi:moshi-kotlin-codegen 1.15.1
  • junit:junit 4.13.2
  • com.squareup.moshi:moshi 1.9.3
  • com.squareup.moshi:moshi-kotlin-codegen 1.9.3
data/build.gradle
  • com.squareup.sqldelight:sqlite-driver 1.4.0
example/build.gradle
  • com.squareup.sqldelight:sqlite-driver 1.4.0
  • com.jakewharton.picnic:picnic 0.3.1
  • junit:junit 4.13.2
library/build.gradle
gradle-wrapper
gradle/wrapper/gradle-wrapper.properties
  • gradle 6.9.4

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.