privacydevel / nitter Goto Github PK
View Code? Open in Web Editor NEWThis project forked from zedeus/nitter
Alternative Twitter front-end
Home Page: https://nitter.net
License: GNU Affero General Public License v3.0
This project forked from zedeus/nitter
Alternative Twitter front-end
Home Page: https://nitter.net
License: GNU Affero General Public License v3.0
One of my instances uses this fork, and they have changed likes endpoint (as far as I can tell) upstream sometime around 1/19.
I'm not finding the issue looking at the code, but I've created a python script to get likes for a user. I'm not seeing this here only 'favoriters'?
Also the user I used to test to see if hidden likes are truly hidden, are now hidden.
For likes I've been using
https://twitter.com/i/api/graphql/G_zHbTiwSqLm0TAK_3sNWQ/Likes
My script utilizes the same method of copying over x-csrf-token and 2 cookie elements for now.
Again I'm probably way off base using 'likes' but things are broken in this area now.
ETA https://gist.github.com/cmj/adfd541dde30585d861d28fd58bec9f0
The instance/server is not usable after the service went up again.
Hope it can be fixed.
Hey!
Thanks for your effort in re-enabling 18+ Posts.
I would like to be independent of public instances and host my own, as I already do with the official nitter docker image.
Could you perhaps create a docker image for your fork as well?
I can see that entire privacydev website with services went off? even tried to sent an email, the email didnt reach because the gateway down as well?..
ThX!
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
syncio.nim(855) readFile
Error: unhandled exception: cannot open: ./guest_accounts.json [IOError]
Recently (starting only one or two days ago?), profile pages are not correctly showing the most recent tweets for the account.
The tweets that are shown also depend on how the profile name is capitalized. For example, nitter.privacydev.net/elonmusk gives a different result than nitter.privacydev.net/Elonmusk, and neither result is accurately showing recent tweets.
tried to view this tweet-
... MapsUkraine/status/1674250481848266753
get
"@MapsUkraine's account has been withheld in Germany based on local law(s). Learn more."
get the same when I even try to see his TL.
does happen on other nitter instances (ofc the 18+ don't display but all others do..)
*edit
I assume your instance is housed in Germany.. I guess that's it then,,
Not sure if this is specific to this fork, or Nitter in general, but it only occurs when on the privacydev.net instance.
When using the /with_replies page, sometimes people who replied to the user in question get presented alongside that user's own replies, retweets, and normal tweets. This also shows up in the RSS feeds of said page, leading to an RSS feed full of random people. This is not desirable.
First off, I'd like to thank you for making this fork! It's been an incredibly valuable resource to me for searching Twitter after search was disabled for guests.
Here's my request. Since this fork uses credentials to allow search and show blocked posts, then theoretically it could also show the names of users who like and retweet a post (like on Twitter when you click the "(#) Likes" and "(#) Retweets" texts). If you could add that functionality, it would make this fork even more valuable.
I have no experience with the programming that goes into this, so I don't know how hard it would be to implement, but it would make for a great feature.
Hi
I want to start using this fork, sice it's in active development unlike zedeus/nitter.
I use Nitter mainly for RSS Feeds and could I request that you consider adding these two improvements from my posts:
IMHO these make RSS feed much better.
Thanks.
After your last update .tokens has dismissed and .accounts and .health has been hinerited from nitter branch but using the web entry point
https://nitter.privacydev.net/.health
the system produce the error messages:
An error has occured in one of your routes.
Detail: tables.nim(232) raiseKeyError
asyncfutures.nim(389) read
asyncfutures.nim(389) read
key not found: favoriters
the .accounts works correctly.
since (afaik) this fork uses regular accounts, could it be possible to fix this?
Can you give the Privacydev instance the option to switch from SFW to N4 or O2?
Hello. I encountered some problems when using docker-compose to build this project. As shown in the picture above, I exposed the 8080 port to 9003 in docker-compose.yml, and also filled in the domain name and the name shown in the nitter.conf. port, but after enabling it, I encountered the error in Figure 1. I would like to ask you how to enable it normally (use Google Translate for conversation)
Thanks so much for your fork, the 'following' feature you added is super helpful! I'm really keen to set it up for myself, but I've run into a bunch of issues trying to get it deployed. Any chance you could update a Docker image? That would be awesome and super appreciated!
Hi, Do you know something about the rate limit? It's amazing you manage to keep a public instance running with 1 account. My accounts get blocked often with:
"status_code: 200, msg: Twitter responded with an error: AuthorizationError: Authorization: Denied by access control: To protect our users from spam and other malicious activity, this account is temporarily locked. Please log in to https://twitter.com to unlock your account."
How do you handle this?
Thank you
(I realise this is not really an 'Issue', let me know if there is a better way to contact you to ask a question)
The problem Nitter is currently facing with NSFW content is that you can only fetch it with authenticated requests, which are rate-limited. However, some of the rate limit is currently being wasted on requests that don't contain any NSFW content at all.
My proposal is to always fetch as a guest first, and only retry with authentication if there was NSFW content in the first response. The idea is that since guest requests are essentially free, the rate limit will be used up less quickly and instances will hopefully become more usable.
In pseudocode:
result = fetch(auth=false)
if auth token is set and result contains "Age-restricted adult content. This content...":
result = fetch(auth=true)
return result
Whenever I try to generate RSS feeds for the likes tab, they always come up completely empty.
After investigating, turns out that the tweet.user.id
parameter in src/views/rss.nimf
in line 64 is completely empty, causing the check to fail and exit the if statement.
Changing it to this fixes the issue
# if tweet.user.id == "":
# if userId.len > 0 and tweet.user.id == userId: continue
# end if
# elif userId.len > 0 and tweet.user.id != userId: continue
Here's the commit in my own repo
rotorot0@2969af1
RSS feed generation works normally for all other tabs too.
Any ideas to have an actual fix for this or is this good enough?
To explain, in brief, it seems that no matter what I do, the configuration isn't being taken properly. I was suspecting that it was something that I missed but after comparing config with a buddy who's also hosting the same fork, I'm not quite sure anymore.
I'll recount my steps here to show you my process of building and deploying the fork to see if anything comes up to you here.
I began with cloning the repo with a git clone
so I could build the image. docker build -t privacydevel/nitter:latest .
I let the image build and then copy/pasted the dockercompose.yml
while replacing the image
parameter with the image that I built from your fork. The only other things that I edit in the .yml are the ports and the volume path (to point to the nitter.conf
file).
I copy over the nitter.example.conf
and verify that it's being read by pointing the container to it in the docker compose file. Changing values like "title" is reflected in my nitter instance so the container is reading config just fine.
I have a twitter account that I've set to allow for the searching and viewing of sensitive content.
Accessing https://twitter.com/settings/search gives me this:
Likewise, Settings and Support > Settings and Privacy > Privacy and Safety > Content you see:
With an inspect element (chromium browser), I grab the values needed for ct0
, auth_token
and xCsrfToken
. ct0
and auth_token
I got from the application tab while xCsrfToken
I grab from a request header in the network tab.
Double-checking myself, I read through the issue that was posted here earlier where someone had whitespace in their configuration, which caused their search to not work (as well as R18 content to not be displayed). Their symptoms are similar to mine, though there wasn't any whitespace in my configuration like how the other person's was.
This is what the tokens look like in my config (the X'es are placeholders):
cookieHeader = "ct0=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX; auth_token=XXXXXXXXXXXXXXX"
xCsrfToken = "XXXXXXXXXXXXXXXXXXXXXX"
Outside of this, I've double-checked for any issues with firewalling and file permissions. Between myself and my friend who's running the same fork, we compared configs, set-up, etc and they're exactly the same. The main differences between me and him that I could find (if significant at all) are that:
On that last one, I've checked with a non-private account to see if that'd make any difference (it didn't). Other details that might be relevant is that I'm proxying the host with NIGNX, though symptoms still persist whether by connecting via the proxy or directly through the IP and port of the machine.
I hope I'm not missing something obvious. I've been trying to see if it's something that I missed but I think I've exhausted my options with about as much as I know to do, currently.
...after only maybe 4 clicks.
nm
I guess it was just timing...
I thought everything had come back but it was only a momentary blip
Hi, I'm trying to build this fork to set up my instance, and am getting the following output when running nimble build -d:release --verbose
:
Verifying dependencies for [email protected]
Reading official package list
Checking for jester@#baca3f
Info: Dependency on jester@#baca3f already satisfied
Verifying dependencies for jester@#baca3f
Reading official package list
Checking for httpbeast@>= 0.3.0
Info: Dependency on httpbeast@>= 0.3.0 already satisfied
Verifying dependencies for [email protected]
Reading official package list
Checking for asynctools@#0e6bdc3ed5bae8c7cc9
Info: Dependency on asynctools@#0e6bdc3ed5bae8c7cc9 already satisfied
Verifying dependencies for asynctools@#0e6bdc3ed5bae8c7cc9
Reading official package list
Checking for https://github.com/timotheecour/asynctools@#pr_fix_compilation
Info: Dependency on https://github.com/timotheecour/asynctools@#pr_fix_compilation already satisfied
Verifying dependencies for asynctools@#pr_fix_compilation
Reading official package list
Checking for karax@#5cf360c
Info: Dependency on karax@#5cf360c already satisfied
Verifying dependencies for karax@#5cf360c
Reading official package list
Checking for ws@any version
Info: Dependency on ws@any version already satisfied
Verifying dependencies for [email protected]
Reading official package list
Checking for dotenv@>= 2.0.0
Info: Dependency on dotenv@>= 2.0.0 already satisfied
Verifying dependencies for [email protected]
Reading official package list
Checking for sass@#7dfdd03
Info: Dependency on sass@#7dfdd03 already satisfied
Verifying dependencies for sass@#7dfdd03
Reading official package list
Checking for nimcrypto@#4014ef9
Info: Dependency on nimcrypto@#4014ef9 already satisfied
Verifying dependencies for nimcrypto@#4014ef9
Reading official package list
Checking for markdown@#158efe3
Info: Dependency on markdown@#158efe3 already satisfied
Verifying dependencies for markdown@#158efe3
Reading official package list
Checking for packedjson@#9e6fbb6
Info: Dependency on packedjson@#9e6fbb6 already satisfied
Verifying dependencies for packedjson@#9e6fbb6
Reading official package list
Checking for supersnappy@#6c94198
Info: Dependency on supersnappy@#6c94198 already satisfied
Verifying dependencies for supersnappy@#6c94198
Reading official package list
Checking for redpool@#8b7c1db
Info: Dependency on redpool@#8b7c1db already satisfied
Verifying dependencies for redpool@#8b7c1db
Reading official package list
Checking for https://github.com/zedeus/redis@#d0a0e6f
Info: Dependency on https://github.com/zedeus/redis@#d0a0e6f already satisfied
Verifying dependencies for redis@#d0a0e6f
Reading official package list
Checking for https://github.com/zedeus/redis@#d0a0e6f
Info: Dependency on https://github.com/zedeus/redis@#d0a0e6f already satisfied
Verifying dependencies for redis@#d0a0e6f
Reading official package list
Checking for zippy@#ca5989a
Info: Dependency on zippy@#ca5989a already satisfied
Verifying dependencies for zippy@#ca5989a
Reading official package list
Checking for flatty@#e668085
Info: Dependency on flatty@#e668085 already satisfied
Verifying dependencies for flatty@#e668085
Reading official package list
Checking for jsony@#ea811be
Info: Dependency on jsony@#ea811be already satisfied
Verifying dependencies for jsony@#ea811be
Building nitter/nitter using c backend
Executing /home/nitter/.nimble/bin/nim c --colors:on --noNimblePath -d:release -d:NimblePkgVersion=0.1.0 --path:'/home/nitter/.nimble/pkgs/jester-#baca3f' --path:/home/nitter/.nimble/pkgs/httpbeast-0.4.1 --path:'/home/nitter/.nimble/pkgs/asynctools-#0e6bdc3ed5bae8c7cc9' --path:'/home/nitter/.nimble/pkgs/asynctools-#pr_fix_compilation' --path:'/home/nitter/.nimble/pkgs/karax-#5cf360c' --path:/home/nitter/.nimble/pkgs/ws-0.5.0 --path:/home/nitter/.nimble/pkgs/dotenv-2.0.1 --path:'/home/nitter/.nimble/pkgs/sass-#7dfdd03' --path:'/home/nitter/.nimble/pkgs/nimcrypto-#4014ef9' --path:'/home/nitter/.nimble/pkgs/markdown-#158efe3' --path:'/home/nitter/.nimble/pkgs/packedjson-#9e6fbb6' --path:'/home/nitter/.nimble/pkgs/supersnappy-#6c94198' --path:'/home/nitter/.nimble/pkgs/redpool-#8b7c1db' --path:'/home/nitter/.nimble/pkgs/redis-#d0a0e6f' --path:'/home/nitter/.nimble/pkgs/redis-#d0a0e6f' --path:'/home/nitter/.nimble/pkgs/zippy-#ca5989a' --path:'/home/nitter/.nimble/pkgs/flatty-#e668085' --path:'/home/nitter/.nimble/pkgs/jsony-#ea811be' -o:/home/nitter/nitter/nitter /home/nitter/nitter/src/nitter.nim
Hint: used config file '/home/nitter/.choosenim/toolchains/nim-1.6.14/config/nim.cfg' [Conf]
Hint: used config file '/home/nitter/.choosenim/toolchains/nim-1.6.14/config/config.nims' [Conf]
Hint: used config file '/home/nitter/nitter/config.nims' [Conf]
............................................................................................................................................................................................................................................................. Error: Build failed for package: nitter
... Execution failed with exit code 137
... Command: /home/nitter/.nimble/bin/nim c --colors:on --noNimblePath -d:release -d:NimblePkgVersion=0.1.0 --path:'/home/nitter/.nimble/pkgs/jester-#baca3f' --path:/home/nitter/.nimble/pkgs/httpbeast-0.4.1 --path:'/home/nitter/.nimble/pkgs/asynctools-#0e6bdc3ed5bae8c7cc9' --path:'/home/nitter/.nimble/pkgs/asynctools-#pr_fix_compilation' --path:'/home/nitter/.nimble/pkgs/karax-#5cf360c' --path:/home/nitter/.nimble/pkgs/ws-0.5.0 --path:/home/nitter/.nimble/pkgs/dotenv-2.0.1 --path:'/home/nitter/.nimble/pkgs/sass-#7dfdd03' --path:'/home/nitter/.nimble/pkgs/nimcrypto-#4014ef9' --path:'/home/nitter/.nimble/pkgs/markdown-#158efe3' --path:'/home/nitter/.nimble/pkgs/packedjson-#9e6fbb6' --path:'/home/nitter/.nimble/pkgs/supersnappy-#6c94198' --path:'/home/nitter/.nimble/pkgs/redpool-#8b7c1db' --path:'/home/nitter/.nimble/pkgs/redis-#d0a0e6f' --path:'/home/nitter/.nimble/pkgs/redis-#d0a0e6f' --path:'/home/nitter/.nimble/pkgs/zippy-#ca5989a' --path:'/home/nitter/.nimble/pkgs/flatty-#e668085' --path:'/home/nitter/.nimble/pkgs/jsony-#ea811be' -o:/home/nitter/nitter/nitter /home/nitter/nitter/src/nitter.nim
Is there something I'm missing? I followed the instructions in the README, substituting the repo URL for this fork.
@PrivacyDevel
If the token pool is stale nitter will report that it is throttled instead of first attempting to discard all stale tokens and refill the pool as much as it is able, and then try to serve the request.
This seems to happen intermittently, so I'm not sure if it's when a single token or several are stale resulting in a request failure.
It definitely happens when restarting nitter after a while as the tokens still live in redis but are all stale.
Some instances still do work, but have to refresh a few times ti pull the tweet..
Thanks for hosting this for so long btw.
edit
I guess not since nothing is working now anyway
Do I need to use proxy if I am using multiple twitter accounts to faciliate a private instance? Or I can just continue without using proxy?
Two days ago, I was browsing the privacydev instance to access Twitter accounts without restrictions. However, it got to a point where I was hit with a “Instance has been rate limited”, and assuming it was a temporary message, I stopped using said instance and went to bed, thinking it was going to be resolved, but it has not the next day I woke up. The instance remains “rate limited” as of recently today. Should I keep waiting?
Basically what the title asks.
How do i use the OAuth script when Nitter is being ran as a docker container? my homelab consists of docker services and i ve set up things around things being docker.
Hi, not sure is it a known issue, but all searches don't work in pre_guest_accounts
branch. I would like to continue using my own account instead of guest accounts since GIFs don't load on guest accounts for me.
It shows the following message on console whenever I try to search, and the result is always empty.
200 OK: --- url: https://api.twitter.com/1.1/search/universal.json?cards_platform=Web-13&tweet_mode=extended&ui_lang=en-US&send_error_codes=1&simple_quoted_tweet=1&skip_status=1&include_blocked_by=0&include_blocking=0&include_can_dm=0&include_can_media_tag=1&include_cards=1&include_composer_source=0&include_entities=1&include_ext_is_blue_verified=1&include_ext_media_color=0&include_followed_by=0&include_mute_edge=0&include_profile_interstitial_type=0&include_quote_count=1&include_reply_count=1&include_user_entities=1&include_want_retweets=0&q=from%3Ashirakamifubuki+filter%3Aself_threads+OR-filter%3Areplies+include%3Anativeretweets&modules=status&result_type=recent&ext=mediaStats%2CisBlueVerified%2CisVerified%2Cblue%2CblueVerified&include_ext_alt_text=1&include_ext_media_availability=1&count=20
If I switch to the older nsfw
branch, searching for tweets work well but not when searching for users.
Is there anything I need to do?
I self-host a private Nitter instance just for me.
I just switched from the pre_guest_accounts branch to master, and everything is mostly working, except RSS feeds are broken. The port (:8080) is no longer included in any of the nitter URLs. So image thumbnails won't display, and if I try to open a feed item in my browser from my rss reader, it won't load. But if I manually edit the URL in the browser address bar to add :8080, then it loads.
Is this an issue with this fork, or should I post this in the main zedeus/nitter Issues?
to alleviate hitting the rate limit can you pull tweets using https://platform.twitter.com/embed/Tweet.html?id=TWEET_ID ?
then pull age restricted tweets using user account info?
When I went to Nitter's PrivacyDev instance to look over various accounts without a twitter account, I ran into some issues.
Images and videos are broken, I get a 500 internal server error when I check the media or replies of some accounts, the instance is rate limited message is also involved as well, replies are not sometimes shown, etc.
There could be more issues than the ones I am recalling at the time of writing; it happened last night.
Instead of showing the tweet with "x retweeted" above, it says "RT @xyz: abcde...". It also doesn't display attached media.
Example: https://nitter.privacydev.net/ditzy_bunny/status/1634535003597033472
What would be an example of the config needed to run this repo with a user account?
Hey! Thanks for the workaround, you're doing a great job on this thing. 👍
I'm having a small problem with rate limit errors on build 2023.06.06-25b7884.
Apparently I'm running out of tokens even when the instance didn't see any traffic aside from me opening the page and searching for something. Visiting a profile straight from search also causes rate limit errors. It goes away immediately after refreshing the page. I can confirm that this doesn't happen on the official docker image.
Container logs give me this:
19/06/2023 15:05:18
fetch error: (errors: @[(code: badToken, message: "Bad guest token")])
19/06/2023 15:05:18
error: RateLimitError,
msg: rate limited,
token: (tok: "1670757847372640256", init: (seconds: 1687174686, nanosecond: 818120247), lastUse: (seconds: 1687179377, nanosecond: 778415499), pending: 1, apis: (data: @[(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: -2374071652988301375, key: userMedia, val: (remaining: 499, reset: 1687177533)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: -955892393542437747, key: favorites, val: (remaining: 498, reset: 1687176337)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: -5940405725068231575, key: tweetDetail, val:(remaining: 497, reset: 1687179500)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: -2114734800836552661, key: userTweets, val: (remaining: 498, reset: 1687177527)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0)),
(hcode: 0, key: tweetDetail, val: (remaining: 0, reset: 0))], counter: 4)), url: https://api.twitter.com/1.1/users/search.json?q=test&skip_status=1&count=20&page
I did see an issue about something similar in the main branch but I'm not sure if it's connected.
FYI getting a date of certificate invalid from privacydev.net...
Hey,
I see that on your nitter instance you've enabled links to quote tweets and retweets in the tweet's footer. However, using your fork it's not straightforward as to how to enable this feature on my own instance.
After rebuilding the program,
# git reset --hard privacydevel/master
HEAD is now at e4e7fe5 Merge remote-tracking branch 'upstream/guest_accounts'
# (set -e; cd /opt/nitter; for i in build scss md; do nimble $i -d:release; done; service nitter restart)
the footer turns from gray to orange, but clicking on individual items inside it doesn't work.
Is there an additional configuration step to be made, or perhaps some code still remains to be made public?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.