anticompositenumber / signatures Goto Github PK
View Code? Open in Web Editor NEWValidates user signatures, checking for technical and policy issues
Home Page: https://signatures.toolforge.org
License: GNU Affero General Public License v3.0
Validates user signatures, checking for technical and policy issues
Home Page: https://signatures.toolforge.org
License: GNU Affero General Public License v3.0
It broke because of the Great WMCS DB migration. It should be fixed.
Could wikitech (and other special wikis) please be added so that they can be checked?
JSON is easy, but NFS is bad. And it's hard to combine data from different sites.
I'm thinking of doing two tables -- one to keep the signatures as-scanned, and one to keep the errors.
sig_id | timestamp | signature |
---|---|---|
1 | 20210320200805 | foo bar |
username | wiki | sig_id | timestamp | error |
---|---|---|---|---|
foo | enwiki | 1 | 20210320200805 | no-user-links |
foo | enwiki | 1 | 20210320200805 | plain-fancy-sig |
Many large wikis have policies and guidelines for signatures that go above and beyond the proposal for technical requirements. These requirements should be supported in an extended validation mode, possibly with granular test selection.
For example: https://signatures.toolforge.org/check/eo.wikipedia.org/WMFOffice
Also trying to check a wiki that doesn't exist: https://signatures.toolforge.org/check/abcde.wikipedia.org/WMFOffice
Currently, only signatures stored in the replica database can be accessed.
A MediaWiki site with a public API would still be required for i18n and template markup checks. That site would have to expose the Linter API in the same manner as public WMF sites to do lint checks.
Searching a username with a lowercase letter would return a "User does not exist" message. Since MediaWiki does not allow the creation of usernames start with lowercase letters, it is possible to add an automatic conversion.
Hi,
Can this be added to translatewiki for localization?
The instructions are here: https://translatewiki.net/wiki/Translating:New_project
Underscore symbol should be parsed to whitespace, as they have the equivalent effect for Mediawiki username.
>>> p.sort_stats(pstats.SortKey.TIME).print_stats(30)
Mon Mar 23 02:27:58 2020 sigprobs_profile
5200087 function calls (5155274 primitive calls) in 495.606 seconds
Ordered by: internal time
List reduced from 1999 to 30 due to restriction <30>
ncalls tottime percall cumtime percall filename:lineno(function)
104 402.013 3.866 402.013 3.866 {method 'recv_into' of '_socket.socket' objects}
2036 85.032 0.042 85.032 0.042 {method 'read' of '_ssl._SSLSocket' objects}
5979 0.262 0.000 0.262 0.000 {built-in method posix.stat}
4189 0.183 0.000 0.264 0.000 url.py:210(_encode_invalid_chars)
1 0.167 0.167 495.195 495.195 sigprobs.py:378(main)
205 0.150 0.001 0.172 0.001 <frozen importlib._bootstrap_external>:914(get_data)
91358 0.149 0.000 0.364 0.000 os.py:673(__getitem__)
108346 0.141 0.000 0.647 0.000 _collections_abc.py:742(__iter__)
1901 0.141 0.000 0.141 0.000 {method 'write' of '_ssl._SSLSocket' objects}
110 0.135 0.001 0.137 0.001 {built-in method io.open}
401706 0.129 0.000 0.171 0.000 {built-in method builtins.isinstance}
534880 0.127 0.000 0.127 0.000 {method 'lower' of 'str' objects}
1015 0.116 0.000 0.308 0.000 feedparser.py:471(_parse_headers)
21322 0.104 0.000 0.312 0.000 parse.py:361(urlparse)
311422 0.102 0.000 0.102 0.000 {method 'decode' of 'bytes' objects}
91358 0.088 0.000 0.138 0.000 os.py:751(encode)
48898 0.086 0.000 0.086 0.000 {method 'match' of 're.Pattern' objects}
2030 0.085 0.000 0.667 0.000 request.py:2456(getproxies_environment)
1015 0.084 0.000 0.890 0.001 client.py:203(parse_headers)
166462 0.083 0.000 0.148 0.000 os.py:755(decode)
49753 0.080 0.000 0.092 0.000 parse.py:109(_coerce_args)
24371 0.080 0.000 0.140 0.000 parse.py:412(urlsplit)
9135 0.077 0.000 0.142 0.000 message.py:462(get)
5077 0.076 0.000 0.220 0.000 _collections_abc.py:824(update)
31322 0.076 0.000 0.128 0.000 _policybase.py:293(header_source_parse)
2030 0.070 0.000 0.577 0.000 feedparser.py:218(_parsegen)
85260 0.068 0.000 0.140 0.000 os.py:696(__iter__)
166141/166140 0.067 0.000 0.067 0.000 {method 'encode' of 'str' objects}
6877/985 0.059 0.000 0.496 0.001 utils.py:36(parse_anything)
31440/11160 0.059 0.000 0.109 0.000 cookiejar.py:1214(deepvalues)
>>> p.sort_stats(pstats.SortKey.CUMULATIVE).print_stats(30)
Mon Mar 23 02:27:58 2020 sigprobs_profile
5200087 function calls (5155274 primitive calls) in 495.606 seconds
Ordered by: cumulative time
List reduced from 1999 to 30 due to restriction <30>
ncalls tottime percall cumtime percall filename:lineno(function)
219/1 0.002 0.000 495.638 495.638 {built-in method builtins.exec}
1 0.000 0.000 495.638 495.638 sigprobs.py:21(<module>)
1 0.167 0.167 495.195 495.195 sigprobs.py:378(main)
2140 0.016 0.000 487.096 0.228 socket.py:575(readinto)
902 0.008 0.000 402.222 0.446 sigprobs.py:60(iter_active_user_sigs)
1410 0.021 0.000 402.091 0.285 connections.py:648(_read_packet)
100 0.001 0.000 402.068 4.021 cursors.py:151(execute)
100 0.001 0.000 402.061 4.021 cursors.py:451(_query)
100 0.001 0.000 402.058 4.021 connections.py:508(query)
100 0.001 0.000 402.047 4.020 connections.py:720(_read_query_result)
100 0.002 0.000 402.046 4.020 connections.py:1086(init_unbuffered_query)
2820 0.012 0.000 402.045 0.143 connections.py:687(_read_bytes)
2861 0.006 0.000 402.021 0.141 {method 'read' of '_io.BufferedReader' objects}
104 402.013 3.866 402.013 3.866 {method 'recv_into' of '_socket.socket' objects}
886 0.039 0.000 92.566 0.104 sigprobs.py:197(check_sig)
1015 0.027 0.000 91.829 0.090 sessions.py:463(request)
1015 0.041 0.000 89.084 0.088 sessions.py:614(send)
1015 0.034 0.000 88.646 0.087 adapters.py:394(send)
1015 0.048 0.000 87.597 0.086 connectionpool.py:494(urlopen)
1015 0.046 0.000 87.008 0.086 connectionpool.py:351(_make_request)
886 0.012 0.000 86.469 0.098 sigprobs.py:218(get_lint_errors)
886 0.009 0.000 86.334 0.097 sessions.py:567(post)
1015 0.012 0.000 86.136 0.085 client.py:1292(getresponse)
1015 0.031 0.000 86.081 0.085 client.py:299(begin)
1015 0.035 0.000 85.076 0.084 client.py:266(_read_status)
33404 0.041 0.000 85.059 0.003 {method 'readline' of '_io.BufferedReader' objects}
2036 0.012 0.000 85.055 0.042 ssl.py:1041(recv_into)
2036 0.010 0.000 85.042 0.042 ssl.py:902(read)
2036 85.032 0.042 85.032 0.042 {method 'read' of '_ssl._SSLSocket' objects}
129 0.001 0.000 5.505 0.043 sessions.py:534(get)
>>> p.strip_dirs().sort_stats(pstats.SortKey.FILENAME, pstats.SortKey.CUMULATIVE).print_stats()
Mon Mar 23 02:27:58 2020 sigprobs_profile
5200087 function calls (5155274 primitive calls) in 495.606 seconds
Ordered by: file name, cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
(...)
1 0.000 0.000 495.638 495.638 sigprobs.py:21(<module>)
1 0.167 0.167 495.195 495.195 sigprobs.py:378(main)
902 0.008 0.000 402.222 0.446 sigprobs.py:60(iter_active_user_sigs)
886 0.039 0.000 92.566 0.104 sigprobs.py:197(check_sig)
886 0.012 0.000 86.469 0.098 sigprobs.py:218(get_lint_errors)
128 0.002 0.000 5.436 0.042 sigprobs.py:317(evaluate_subst)
886 0.011 0.000 4.599 0.005 sigprobs.py:236(check_links)
886 0.002 0.000 1.447 0.002 sigprobs.py:345(check_tildes)
985 0.028 0.000 0.596 0.001 sigprobs.py:255(compare_links)
1 0.001 0.001 0.085 0.085 sigprobs.py:122(get_site_data)
2724 0.010 0.000 0.014 0.000 sigprobs.py:189(normal_name)
886 0.003 0.000 0.003 0.000 sigprobs.py:333(check_fanciness)
1 0.000 0.000 0.003 0.003 sigprobs.py:39(load_config)
886 0.001 0.000 0.002 0.000 sigprobs.py:370(check_length)
1 0.000 0.000 0.000 0.000 sigprobs.py:159(<dictcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:163(<dictcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:168(<setcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:173(<listcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:174(<listcomp>)
(...)
>>> p.strip_dirs().sort_stats(pstats.SortKey.FILENAME, pstats.SortKey.TIME).print_stats()
Mon Mar 23 02:27:58 2020 sigprobs_profile
5200087 function calls (5155274 primitive calls) in 495.606 seconds
Ordered by: file name, internal time
ncalls tottime percall cumtime percall filename:lineno(function)
(...)
1 0.167 0.167 495.195 495.195 sigprobs.py:378(main)
886 0.039 0.000 92.566 0.104 sigprobs.py:197(check_sig)
985 0.028 0.000 0.596 0.001 sigprobs.py:255(compare_links)
886 0.012 0.000 86.469 0.098 sigprobs.py:218(get_lint_errors)
886 0.011 0.000 4.599 0.005 sigprobs.py:236(check_links)
2724 0.010 0.000 0.014 0.000 sigprobs.py:189(normal_name)
902 0.008 0.000 402.222 0.446 sigprobs.py:60(iter_active_user_sigs)
886 0.003 0.000 0.003 0.000 sigprobs.py:333(check_fanciness)
128 0.002 0.000 5.436 0.042 sigprobs.py:317(evaluate_subst)
886 0.002 0.000 1.447 0.002 sigprobs.py:345(check_tildes)
886 0.001 0.000 0.002 0.000 sigprobs.py:370(check_length)
1 0.001 0.001 0.085 0.085 sigprobs.py:122(get_site_data)
1 0.000 0.000 0.003 0.003 sigprobs.py:39(load_config)
1 0.000 0.000 0.000 0.000 sigprobs.py:159(<dictcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:163(<dictcomp>)
1 0.000 0.000 495.638 495.638 sigprobs.py:21(<module>)
1 0.000 0.000 0.000 0.000 sigprobs.py:168(<setcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:173(<listcomp>)
1 0.000 0.000 0.000 0.000 sigprobs.py:174(<listcomp>)
(...)
[00:00:05] https://github.com/AntiCompositeNumber/signatures/blob/master/src/sigprobs.py#L214
[00:03:15] I think action=parse&onlypst=1&text=... is a better API call to use here
[00:05:35] interestingly The Earwig's sig defaults to empty string, so when you expand it that way you get nothing.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.