parafoxia / analytix Goto Github PK
View Code? Open in Web Editor NEWA simple yet powerful wrapper for the YouTube Analytics API.
Home Page: https://parafoxia.github.io/analytix/
License: BSD 3-Clause "New" or "Revised" License
A simple yet powerful wrapper for the YouTube Analytics API.
Home Page: https://parafoxia.github.io/analytix/
License: BSD 3-Clause "New" or "Revised" License
The valid values for the gender
dimension are due to change on 11 August. analytix will need to update its own list of valid values to accommodate the new "user_specified" option.
Lines 703 to 706 in 5bf39e5
conftest.py files provide a space to define shared fixtures.
Better practise than the current methods.
All shared fixtures go into the conftest.py file.
All mock creator methods would still need to exist, so maybe include them in a file of their own.
No response
Summary
Twitch analytics, similar to YouTube ones. There are seemingly a few options for this, but I'm not sure any of them contain channel analytics. Either some third-party thing will be needed, or this may not be possible.
The idea
More stats!
The ideal implementation
As close to the YouTube analytics systems as possible for maximum consistency.
Contribution
Do you want to contribute this feature yourself?
Yes
Currently all logging uses f-strings, however that is not the best way to do it, as strings are always resolved regardless of whether they're actually needed or not. The Python logging HOWTO touches on this, and various SO answers agree..
Speeeeeeed!
No response
No response
Polars is an alternative to Pandas which claims to be significantly faster.
Other dataframe alternatives are included, such as PyArrow.
report.to_polars
or report.to_polars_dataframe
perhaps? report.to_dataframe
may need renaming to report.to_pandas
.
No response
Hello,
I am facing an issue with "Error 403: Forbidden" with the default code from this projects home page.
Perhaps there is a chance I have messed up somewhere in the Google Developer Cloud, however I was successfully able to create both the secrets.json and tokens.json and ensure that the file paths are correctly set.
I enabled the "YouTube Analytics API" and set the scope for it for the last 2 options on the scope list as instructed in the YouTube video here.
It's set to use external with testing enabled and my email address was added to the test list which was the same email I authorized in the OAuth stage when I ran the application for the first time. Now when I attempt to run it again with the added tokens.json now in my project directory, the code throws "Error: 403". The OAuth Client ID is also currently set to Desktop.
Unlike the video however, I have updated the initialization code to reflect what's on this project page and I have enabled logging as you can see in the screenshot I have provided.
Summary
YouTubeAnalyticsReport
class is not documented.authorise
method no longer raises an IncompleteRequest
exception.include_historical_data
is incomplete.Reproduction steps
N/A
Error traceback
N/A
Checklist
Summary
analytix can currently export reports to JSON and CSV/TSV formats, though being able to export as a spreadsheet (XSLX by default) would be useful in cases where data will be analysed in Excel.
The ideal implementation
A Report.as_spreadsheet
or Report.as_excel
method. Potentially allow for different spreadsheet extensions to be allowed aside from XSLX.
Contribution
Do you want to contribute this feature yourself?
Yes/No
For some reason, docs render perfectly when built locally, but break completely when loaded onto ReadTheDocs.
Apache Arrow is a commonly used data processing platform. While I'm not particularly smitten with the platform itself, the Feather format that comes with it is seriously nice, and definitely worth including natively.
Conversions should be made to:
Having these methods allows developers to utilise these tools without having to install pandas.
Report.to_arrow_table
Report.to_feather
Report.to_parquet
No response
They should either pass or fail depending on whether the min version is stil supported.
They always fail as they can't find any tests.
This is based in CI, not an analytix bug per se.
The remaining three manageable resources after groups (#30) have been implemented.
Mainly convenience, as users would have a consistent interface to access additional information without the need for an additional library.
Video
, Channel
, and Playlist
classes that inherit from the Resource
ABC created as part of #30.
No response
Webserver authorisation (or loopback IP addresses) are the recommended way to handle authorisation for desktop projects.
Manual code entry has been removed, and though v3.3.2 introduces a temporary solution (see #33), there's no telling how long that solution will work for.
Probably something within the Analytics
and AsyncAnalytics
classes, but unsure beyond that.
No response
with Client("secrets.json") as client:
report = client.retrieve_report(
dimensions=("day",),
metrics=("likes",),
sort_options=("-views",),
)
analytix handles this, and throws an InvalidRequest
error.
An APIError
is thrown.
4.1.1
Information:
• Python version: 3.11.1 (CPython)
• Operating system: Darwin (22.3.0)
• Installed in: /Users/ethanhenderson/Programs/Projects/analytix/analytix
No response
Summary
A more basic form of the Analytics API more or less, complete with a less complex auth flow. This could also be used to get info on any channel, albeit nowhere near as detailed as the Analytics API.
The idea
Moar support! Plus, the ability to compare basic info for channels.
The ideal implementation
This wouldn't work the same as the Analytics of Reporting stuff at all, but would be simpler, so that works.
Contribution
Do you want to contribute this feature yourself?
Instead of having multiple tests for each report types, have a single test that is run multple times using different parameters.
Using pytest.mark.parametrize
.
No response
Remove all deprecated functionality related to the deprecation of the "isCurated" filter.
The API is removing it -- deprecated reports won't work anymore.
Just remove them, probably. Make sure to remove the tests as well.
The ability to get the OAuth2 URL instead of showing it in the console and a function to process Google's response without using the server.
I am creating a web app and showing the OAuth2 URL in the console is not useful for me as the final users don't see the console. Also, in the case of web apps, you are already running a server, and temporarily running the internal server this library use for authorization may cause many complications for web apps. Some examples are:
To have a way to not use the internal server and just show an error message if the user is not authorized yet.
Example:
try:
client = Analytics.with_secrets("secrets.json", use_server=False)
except NotAuthorized as e:
# Handle the error however you prefer.
# `e` contains the error message "You need to authorize analytix; to do so, visit this URL: https://[...]"
Also, as the server is turned-off, I'll need a way to process the received data. As this is a webapp, I'll need get the query strings and/or headers however my framework allows me to do it and use a function to save the token.
Example:
Analytics.save_token(
# ...
)
No response
I can contribute by answering questions, providing ideas, and testing. But I don't think I can code it by myself as I don't fully understand how the code works.
I am running the below code every day. However after a week or so the token expires and I had to renew the access token. It may be basic thing, However I am struggling with it. Can you please help me ?
def get_report():
client = Client("secret.json")
report = client.retrieve_report(
dimensions=("video",),
max_results=200,
sort_options=("-views",),
end_date= date.today(),
currency="EUR",
start_date=(date.today()- timedelta(days=365))
)
file_name = "analytics.csv"
report.to_csv(file_name)
return file_name
I think this is because the token expiry is set to a less time
No response
No response
While error messages were improved in version 4, they're still not 100% clear.
Firstly, the raising of InvalidRequest
or any of it's subclasses should be stopped in favour of only ever raising InvalidRequest
for invalid requests (though the subclasses should still exists in case anyone is checking for those explicitly).
Secondly, the wording could be improved, i.e.: "the 'day' and 'month' dimesions cannot be used together".
Errors would be easier to resolve.
No response
No response
Summary
All video reports are supported, so the next step is to get all the playlist ones good to go.
The idea
More comprehensive support.
The ideal implementation
Same as video reports, just with the playlists. May need to add a boolean allowing the user to specify whether they want a video or playlist report.
Contribution
Do you want to contribute this feature yourself?
Summary
Factory methods can be used to simplify the process of creating more common reports. For example, the current method of getting the daily performance for a specific video is:
client = YouTubeAnalytics("secrets.json")
client.retrieve(
dt.date(2021, 1, 1),
dt.date(2021, 6, 30),
dimensions=("day",),
filters={"video": "videoID"},
)
The proposed method would look (something) like this:
factory = YouTubeAnalyticsFactory("secrets.json")
factory.video_performance("videoID", by="day", since=dt.date(2021, 1, 1))
A better example would be getting the daily performance channel wide:
factory = YouTubeAnalyticsFactory("secrets.json")
factory.daily_performance(since=dt.date(2021, 1, 1))
The idea
It will make it quicker and easier for users to make more common requests.
The ideal implementation
A separate YouTubeAnalyticsFactory
class.
Contribution
Do you want to contribute this feature yourself?
Yes
Summary
Issue forms provide a much better way to issues to be created, forcing issue creators to provided all necessary information.
As demonstrated, I can just remove all the other categories at will.
There should be a form for:
Contribution
Do you want to contribute this improvement yourself?
Yes/No
A method in the AnalyticsReport class than can plot reoprts.
Having the ability to quickly look at data directly (as well as save plots) could be helpful for those who aren't necessarily interested in creating detailed plots themselves, but just want a quick look at what the data looks like. In theory, this could incentivise users to create more detailed plots for their needs.
Some simple stuff, just to enable quick plotting of data.
For time series reports, a line plot should be created. For anything else, either a pie or bar chart should be created.
No response
New stuff was added:
city
dimensioncreatorContentType
dimensionAs per https://developers.google.com/youtube/analytics/revision_history#december-15,-2022.
.
No response
No response
Summary
Metrics are stored as sets which are unordered, meaning the columns are also unordered.
Reproduction steps
Pull any particularly complicated report.
Error traceback
N/A
Checklist
import analytix
from analytix import Client
analytix.enable_logging()
if name == "main":
with Client("secrets_youtube_data.json") as client:
report = client.retrieve_report(
dimensions=("video",),
metrics=("estimatedMinutesWatched", "views", "likes", "subscribersGained"),
max_results=10,
sort_options=("-estimatedMinutesWatched",),
)
report.to_csv("top_10_watched_videos.csv")
list of 10 video's sorted by views
blank csv file with only headers.
4.1.3
Information:
• Python version: 3.9.12 (CPython)
• Operating system: Windows (10)
• Installed in: C:\Users\s.piroe\Anaconda3\lib\site-packages\analytix
1,748,694ms [ INFO ] analytix.oidc: Secrets loaded (type: installed)!
1,748,703ms [ INFO ] analytix.oidc: Tokens loaded!
1,748,765ms [ INFO ] analytix.shard: Access token does not need refreshing
1,748,766ms [ INFO ] analytix.client: Authorisation complete!
1,748,766ms [ INFO ] analytix.queries: Validating request
1,748,769ms [ INFO ] analytix.queries: Getting data between 2023-01-22 and 2023-02-19
1,748,769ms [ INFO ] analytix.queries: Report type determined as 'Top videos by region'
1,748,770ms [ INFO ] analytix.queries: Request OK!
1,748,981ms [ INFO ] analytix.shard: Created report of shape (0, 5)!
How can I pull data for 1 year ?
The current fetched report provides data for 28 days
No response
No response
Summary
Currently analytix only supports pandas
for DataFrame conversion, but modin
and sharks
are alternatives some users may prefer,
The idea
analytix will have extra compatibility.
The ideal implementation
Detect whether any library is installed (in order -- modin
, pandas
, sharks
), import the relevant library, and create the DataFrame using it.
modin
needs extra setup to get working, though this should ideally be handled by the user where possible.
Contribution
Do you want to contribute this feature yourself?
Yes/No
Summary
When a report type that has a required dimension and optional dimensions is verified, it errors when optional dimensions are passed.
Reproduction steps
N/A (already fixed)
Error traceback
N/A (already fixed)
Checklist
May render #54 redundant (at least for now).
The removal of Python 3.7 support, alongside other things, opened the door for more types to be used. However, the list of "other things" is very small, and largely consists of typing changes anyway, so a better option could be to simply import new types from typing_extensions
where necessary and take advantage of types included all the way in 3.11.
Whether a new dependency would be needed in all situations, or whether it would only need to be installed when using analytix[types]
is currently unknown. I can think of a way to do it, but it might be too messy to be really valid.
Also, since PEP 563 is no longer happening, it might be worth removing support for future annotations.
Better typing = better code!
No response
No response
Re-exporting only specific classes, functions, etc. can help avoid undesirable behaviour:
Only things from analytix should be importable by analytix, ideally.
Adding an __all__
to all files that need to re-export specific things (which is probably all files).
No response
Attrs provides increased functionality for more Python versions than dataclasses.
Attrs is already required by AIOHTTP, so there would be no extra dependencies.
No response
No response
Currently, classes have their initialisation args and kwargs documented, but not their attributes.
For completeness.
No response
No response
The following resources should be implemented:
These resources are part of the Analytics API, but have been missed thus far.
More than likely Group
and GroupItem
classes which are constructed from the data. These should both inherit from an ABC (maybe called Resource
?) to allow for easier potential implementation of Data API elements.
No response
Currently in a single go data can be fetched for 200 videos. However if there are more than 200 videos then indexing needs to be done, which is okay to do. However, there is no way to find max number of videos present in the channel so that indexing can be done.
If a channel has more than 200 videos it is difficult to fetch the data.
No response
No response
PyPy 3.8 is no longer supported (even though Python 3.8 is). Things are already breaking (though granted only in third-party libs, for now), and it may be time to let go of it as an officially supported version.
^^
No response
No response
Summary
Add the ability to refresh OAuth tokens automatically.
The idea
Refreshing tokens when needed (or just every time the application is started as there's no usage limit), will mean the user only needs to reauthorise after 200 days, rather than 1 hour.
The ideal implementation
Refresh token on startup would be easiest, however, this would not work for scheduled things. With that being said, analytix is not natively designed for this purpose anyways, so it will probably be fine. Version 3 will handle this.
Contribution
Do you want to contribute this feature yourself?
If someone doesn't PR something before I get the chance to work on it lmao.
Currently in order to write a report to json or to csv asynchronously you must use a separate method.
While it's not the biggest deal (considering most libs that impl sync and async work this way currently),it would be nice to just be able to have a single method that can be used in both contexts.
report = analytix.Report(...) # Some generated report
report.to_csv("sync.csv") # works
await report.to_csv("async.csv") # also works and uses aiofiles
report.to_json("sync.json") # works
await report.to_json("async.json") # also works and uses aiofiles
No response
This would remove support for Python 3.7, which reaches end of life on 27 June 2023.
It would allow the use of additional features, such as the walrus operator and TypedDict.
No response
No response
This is a tracking issue for v4 tests.
Add a page in the documentation covering the most common issues and ways to fix them.
The API is complicated, and analytix can only do so much to make the experience easier, and even then it can be a little difficult to get used to, especially when the causes of issues are unclear.
A guide in the documentation.
No response
Add the column headers information found in the response from the API.
It would provide a simpler interface to perform certain tasks, as well as provide the user with additional information provided by the API.
Probably a ColumnHeaders
dataclass that has name
, data_type
, and column_type
attributes. A Report.column_info
attribute would be a list of these objects. The columns
attribute would become a property extracting data from this, and a rows
property would also be created to match.
No response
JWTs contain user identification information, which is useful when applications wish to use Google's systems when authenticating (for example, they don't want to store passwords etc. in their databases). This would allow those building web apps to have a single authentication process, rather than binding a website account with the authentication tokens.
analytix does not currently have a mechanism with which to do this, and if a developer wanted to authenticate and fetch JWTs, they wouldn't be able to use analytix's utilities to do so. Those using the scripting client that wish to use them would also need to essentially rewrite part of the client.
Additional JWT scopes should be added to the Scopes
enum in a way that isn't breaking. There should also be a system (probably in the client) that can fetch the decode keys from the Google OAuth API and decode the JWTs using those keys.
Work on this has already begun.
Summary
Support for the YouTube Reporting API as well as the Analytics API.
The idea
Support for more things!
The ideal implementation
New file in the youtube
submodule that handles it all the same. The Reporting API uses the same service so that doesn't have to change, thankfully.
Contribution
Do you want to contribute this feature yourself?
Install Griffe 39 and run the docs.
The docs render as expected.
The admonitions break (well, when the version isn't pinned like it currently is).
5.1.0
Information:
• Python version: 3.12.1 (CPython)
• Operating system: Darwin (23.2.0)
• Installed in: /Users/ethanhenderson/Programs/Projects/analytix/analytix
No response
3.11-dev
should be changed to 3.11
in the CI workflow.
This is currently blocked by actions/runner-images#6459. The Darwin build artefact is now available.
Python 3.11 final has been released.
No response
No response
Remove support for the isCurated fitler and add support for the new playlistViews metric.
https://developers.google.com/youtube/analytics/revision_history#january-19,-2024
https://developers.google.com/youtube/analytics/dimensions#Playlist_Dimensions
There are plenty of other changes too that need to be addressed.
The isCurated filter is being deprecated in favour of a new playlistViews metric. analytix currently doesn't support this and will need to by 30 Jun 2024.
Still working that out.
No response
The client is authorised.
3.3.1
Information:
• Python version: 3.10.4 (CPython)
• Operating system: Linux (5.10.102.1-microsoft-standard-WSL2)
• Installed in: /home/parafoxia/Programs/Projects/analytix/analytix
This is caused by the first redirect URL being removed from downloaded API secrets.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.