mesozoic / gmail-yaml-filters Goto Github PK
View Code? Open in Web Editor NEWA quick tool for generating Gmail filters from YAML rules.
License: MIT License
A quick tool for generating Gmail filters from YAML rules.
License: MIT License
gmail-yaml-filters looks very interesting. Conversion of yaml into xml seems to work fine.
For the upload feature the program fails because of missing files (client_secret.json
). I couldn't find any instructions how to create this file properly. I tried to create credentials in Google console but don't know the correct redirect URI.
How to enable these features?
Thanks!
I ask because an operation says "sync" - I don't want to lose my existing rules.
Thanks, Martin.
Basically subj.
When updating filters, I have to go to the web UI and reapply them. Would be great to do it automatically on sync.
I was expecting that the rules were generated in order of the yaml file. I noticed that if my first three rules were "for_each" rules, the fourth rule, the first non-for_each rule was first in the output followed by the for_each rules. Is this expected behavior?
Encountered in versions: 0.6.1, 0.7.1.
Steps to reproduce:
- { label: Bug demo, from: [email protected] }
gmail-yaml-filters --sync filters.yaml
gmail-yaml-filters --sync filters.yaml
Resulting stacktrace:
Traceback (most recent call last):
File "/usr/local/google/home/marain/hack/gmail-filters/venv/bin/gmail-yaml-filters", line 11, in <module>
sys.exit(main())
File "/usr/local/google/home/marain/hack/gmail-filters/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/main.py", line 606, in main
upload_ruleset(ruleset, service=gmail, dry_run=args.dry_run)
File "/usr/local/google/home/marain/hack/gmail-filters/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 179, in upload_ruleset
known_filters = GmailFilters(service)
File "/usr/local/google/home/marain/hack/gmail-filters/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 146, in __init__
self.reload()
File "/usr/local/google/home/marain/hack/gmail-filters/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 150, in reload
self.matchable_filters = [_simplify_filter(existing) for existing in self.filters]
File "/usr/local/google/home/marain/hack/gmail-filters/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 139, in _simplify_filter
'action': {key: set(values) for key, values in filter_dict['action'].items()},
KeyError: 'action'
Workaround: manually delete the rule, and re-sync.
gmail-yaml-filters 0.61.
To reproduce:
Delete all filters from GMail settings. Then:
% gmail-yaml-filters --upload filters.yaml
Traceback (most recent call last):
File "/tmp/hack/venv/bin/gmail-yaml-filters", line 11, in
sys.exit(main())
File "/tmp/hack/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/main.py", line 651, in main
upload_ruleset(ruleset, dry_run=args.dry_run)
File "/tmp/hack/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 183, in upload_ruleset
known_filters = GmailFilters(service)
File "/tmp/hack/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 146, in init
self.reload()
File "/tmp/hack/venv/local/lib/python2.7/site-packages/gmail_yaml_filters/upload.py", line 149, in reload
self.filters = self.gmail.users().settings().filters().list(userId='me').execute()['filter']
KeyError: 'filter'
Reason: when the user has no filters, the GMail API behaves illogically: instead of returning "{filter: []}", it returns just "{}":
% curl -H 'Authorization: Bearer ya29.GltEBGQ8__restoftokenelided__' https://www.googleapis.com/gmail/v1/users/me/settings/filters
{}
I generated a client secret json file and I get this error when trying to use it:
❮ gmail-yaml-filters --dry-run --sync my-filters.yaml
Traceback (most recent call last):
File "/home/mark/.local/bin/gmail-yaml-filters", line 33, in <module>
sys.exit(load_entry_point('gmail-yaml-filters==0.9.5', 'console_scripts', 'gmail-yaml-filters')())
File "/home/mark/.local/lib/python3.10/site-packages/gmail_yaml_filters/main.py", line 99, in main
credentials = get_gmail_credentials(client_secret_path=args.client_secret)
File "/home/mark/.local/lib/python3.10/site-packages/gmail_yaml_filters/upload.py", line 273, in get_gmail_credentials
credentials = store.get()
File "/home/mark/.local/lib/python3.10/site-packages/oauth2client/client.py", line 407, in get
return self.locked_get()
File "/home/mark/.local/lib/python3.10/site-packages/oauth2client/file.py", line 54, in locked_get
credentials = client.Credentials.new_from_json(content)
File "/home/mark/.local/lib/python3.10/site-packages/oauth2client/client.py", line 302, in new_from_json
module_name = data['_module']
KeyError: '_module'
Is it possible to add some new supported actions for Categorize
?
The export from Gmail generates this:
# for Primary
<apps:property name='smartLabelToApply' value='^smartlabel_personal'/>
# for Social
<apps:property name='smartLabelToApply' value='^smartlabel_social'/>
# for Updates
<apps:property name='smartLabelToApply' value='^smartlabel_notification'/>
# for Forums
<apps:property name='smartLabelToApply' value='^smartlabel_group'/>
# for Promotions
<apps:property name='smartLabelToApply' value='^smartlabel_promo'/>
I'd like to apply multiple labels, is there a syntax for this? It appears I need to use a foreach loop to do it now.
I was wondering if there is a way to reverse the nesting structure you have. So for example, if any of the following three are true:
more:
-
from:
not:
all:
- [email protected]
- [email protected]
[email protected]
subject:
any:
- "TEST"
- "Test Test"
- "TESTESTEST"
more:
-
from:
any:
- "[email protected]"
- "[email protected]"
- "[email protected]"
- "[email protected]"
- "[email protected]"
- "[email protected]"
more:
-
from:
not:
all:
- [email protected]
- [email protected]
[email protected]
to:
any:
- "[email protected]"
- "[email protected]"
- "[email protected]"
- "[email protected]"
- "[email protected]"
- "[email protected]"
So if any of the above three are true, do the following:
important: true
label: TEST
gmail-yaml-filters --sync
will crash when it tries to prune a rule with an empty action.
Reproducible by:
from: [email protected]
label: FromJim
gmail-yaml-filters --sync
to create the rule in Gmail.FromJim
.FromJim
and delete the label.Settings->All Settings->Filters and Blocked Addresses
. Notice that the rule has no action.gmail-yaml-filters --sync
Expected Results:
No errors. Ideally, the label would be recreated, the rule would be redefined, and the old rule (the one with the empty action) would be deleted.
Actual Results:
An error with a traceback:
Traceback (most recent call last):
File "/home/user/.local/bin/gmail-yaml-filters", line 33, in <module>
sys.exit(load_entry_point('gmail-yaml-filters==0.9.4', 'console_scripts', 'gmail-yaml-filters')())
File "/home/user/.local/lib/python3.9/site-packages/gmail_yaml_filters/main.py", line 108, in main
prune_filters_not_in_ruleset(ruleset, service=gmail, dry_run=args.dry_run)
File "/home/user/.local/lib/python3.9/site-packages/gmail_yaml_filters/upload.py", line 215, in prune_filters_not_in_ruleset
print('Deleting', prunable_filter['id'], prunable_filter['criteria'], prunable_filter['action'], file=sys.stderr)
KeyError: 'action'
gmail-yaml-filters
is failing for me, with any options, with the samples included in the documentation.
$ python3 -V
Python 3.6.5
$ pip3 list | grep yaml
gmail-yaml-filters 0.7.2
$ $ cat test-filters.yaml
# Simple example
-
from: [email protected]
label: news
not_important: true
$ gmail-yaml-filters test-filters.yaml
Traceback (most recent call last):
File "/usr/local/bin/gmail-yaml-filters", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 595, in main
ruleset = RuleSet.from_object(rule for rule in data if not rule.get('ignore'))
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 487, in from_object
return cls.from_iterable(obj, base_rule=base_rule)
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 516, in from_iterable
ruleset.update(cls.from_object(data, base_rule=base_rule))
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 485, in from_object
return cls.from_dict(obj, base_rule=base_rule)
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 503, in from_dict
new_rule = Rule(data, base_rule=base_rule)
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 337, in __init__
self.update(data)
File "/usr/local/lib/python3.6/site-packages/gmail_yaml_filters/main.py", line 353, in update
for key, value in dict(data).iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'
A few of my rules rely on ordering, which I understand it not guaranteed, however it seems to work. The specific requirement is that they test for the presence (or absence) or labels from other rules. This allows me to avoid giant nested structures. I can do this with 'has:userlabels' or 'has:nouserlabels'
Is there any potential for supporting this as a condition?
When we upload or sync a configuration, the filters are created but not applied to all old emails.
Is it possible to apply the filters to all conversations?
I read this but I'm not sure that is to do what I explain: https://developers.google.com/gmail/api/guides/labels#applying_labels_to_messages_and_threads
In the gmail filter UI, when I specify that a filter categorize something as "social", the resulting XML is:
<apps:property name='smartLabelToApply' value='^smartlabel_social'/>
gmail-yaml-filters wants to do:
<apps:property name="label" value="CATEGORY_SOCIAL"/>
And when used literally creates a label called CATEGORY_SOCIAL and does not place the email in the social tab. Seems that the smartlabel syntax is the appropriate one.
Right now, the XML output that's generated contains two different things that differ from invocation to invocation: the ID (due to the use of hash
) and the timestamp. I like to diff my XML against the previous version to verify the changes, and this adds a lot of diff noise.
It would be great if we could have a way to produce reproducible output. I have a branch that replaces the invocation of hash
with CRC32 and allows the user to set the SOURCE_DATE_EPOCH
environment variable used in reproducible builds to get a fixed timestamp.
Assuming this is okay with you, I'd like to submit that branch, unless you're aware of a different way to go about it.
Is it possible to use the sync capability with multiple gmail accounts at the same time (e.g. work account and personal account)? For example, is there a way to pass the gmail account as a CLI parameter?
Hi, It would be great to have support for forwarding filter syntax.
I noticed that gmail-britta has this feature:
https://github.com/antifuchs/gmail-britta#better-detection-of-your-own-email-address
Does this project have anything equivalent? If not, maybe I could hack that in if it was easy - any suggestions on the format of the YAML configuration for it? Thanks!!
With v0.9.3, gmail-yaml-filters seems to expect client credentials even though I'm just using it to convert YAML to XML:
> pip3 install -U gmail-yaml-filters
> gmail-yaml-filters filters.yaml > filters.xml
/home/mark/.local/lib/python3.8/site-packages/oauth2client/_helpers.py:255: UserWarning: Cannot access /home/mark/.credentials/gmail_yaml_filters.json: No such file or directory
warnings.warn(_MISSING_FILE_MESSAGE.format(filename))
Traceback (most recent call last):
File "/home/mark/.local/lib/python3.8/site-packages/oauth2client/clientsecrets.py", line 121, in _loadfile
with open(filename, 'r') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'client_secret.json'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mark/.local/bin/gmail-yaml-filters", line 8, in <module>
sys.exit(main())
File "/home/mark/.local/lib/python3.8/site-packages/gmail_yaml_filters/main.py", line 93, in main
credentials = get_gmail_credentials(client_secret_path=args.client_secret)
File "/home/mark/.local/lib/python3.8/site-packages/gmail_yaml_filters/upload.py", line 275, in get_gmail_credentials
flow = oauth2client.client.flow_from_clientsecrets(client_secret_path, scopes)
File "/home/mark/.local/lib/python3.8/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/mark/.local/lib/python3.8/site-packages/oauth2client/client.py", line 2134, in flow_from_clientsecrets
client_type, client_info = clientsecrets.loadfile(filename,
File "/home/mark/.local/lib/python3.8/site-packages/oauth2client/clientsecrets.py", line 165, in loadfile
return _loadfile(filename)
File "/home/mark/.local/lib/python3.8/site-packages/oauth2client/clientsecrets.py", line 124, in _loadfile
raise InvalidClientSecretsError('Error opening file', exc.filename,
oauth2client.clientsecrets.InvalidClientSecretsError: ('Error opening file', 'client_secret.json', 'No such file or directory', 2)
The previous version (pip3 install -U gmail-yaml-filters==0.9.2
) works fine.
Can we add a test for CC similarly to TO?
The generated XML file is missing the XML declaration:
<?xml version='1.0' encoding='UTF-8'?>
This results in an error, where gmail won't allow you to import the file. Make sure the generated file includes the XML declaration.
"deliveredto" is mentioned but "to" is not mentioned. It is shown in the examples and it seems to work.
Tried reinstalling this with both Python 2.x and Python 3.x. Both fail with:
ImportError: No module named oauth2client.client
Per https://github.com/google/oauth2client, oauth2client is deprecated anyways.
Hi,
Great work with this app.
I was trying to mark something as SPAM, but adding label SPAM does not work and marking not_spam: false also doesn't. Am I missing something or it is not available yet?
Thanks,
C.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.