shaarli/python-shaarli-client

Python3 CLI to interact with a Shaarli instance

The personal, minimalist, super fast, database-free, bookmarking service.

Do you want to share the links you discover? Shaarli is a minimalist link sharing service that you can install on your own server. It is designed to be personal (single-user), fast and handy.

Quickstart

• Documentation
• Change log
• Bugs/Feature requests/Discussion

Demo

You can use this public demo instance of Shaarli. It runs the latest development version of Shaarli and is updated/reset daily.

Login: demo; Password: demo

License

Shaarli is Free Software. See COPYING for a detail of the contributors and licenses for each individual component.

python-shaarli-client's Issues

sharli post-link URL dies with "expecting value" message

Tonight I I installed shaarli v0.97.2 and v0.10.2 side by side on a server configured with nginx, letsencrypt cert and all (except ntp, I will later), just to compare them.

Then I installed shaarli client with python 3.4, following the procedure described here, with pip and a virtualenv:

https://github.com/shaarli/python-shaarli-client/blob/master/docs/user/installation.rst

I sourced .virtualenvs/shaarli/bin/activate then created this .shaarli/shaarli.conf file:
[shaarli]
url = https://example.com (my real URL, of course)
secret = secret taken by shaarli config panel

Now if I run the shaarli client as specified here https://python-shaarli-client.readthedocs.io/en/latest/user/usage.html I get:

(shaarli) [marco@myserver ~]$ shaarli -c .shaarli/shaarli.conf post-link --url https://w3c.github.io/activitypub/
Traceback (most recent call last):
File "/home/pilatort4/.virtualenvs/shaarli/bin/shaarli", line 11, in
sys.exit(main())
File "/home/pilatort4/.virtualenvs/shaarli/lib64/python3.4/site-packages/shaarli_client/main.py", line 65, in main
print(json.dumps(response.json(), sort_keys=True, indent=4))
File "/home/pilatort4/.virtualenvs/shaarli/lib64/python3.4/site-packages/requests/models.py", line 896, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib64/python3.4/json/init.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.4/json/decoder.py", line 361, in raw_decode
raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)
(shaarli) [marco@myserver ~]$

what is happening? Should I give more/different options, or what?

Thanks

Add support for "204 No content" return when DELETE-ing data

Issue

See shaarli/Shaarli#1141 (comment)

When consuming an endpoint with an HTTP DELETE request, a 204 No content response is returned, with an empty body.

The client attempts to JSON-decode the response, which fails (unless the --format text flag is used).

Fix

Suggestion:

• check the HTTP return code
• expect a 204 response with an empty body for DELETE endpoints

broken CI builds

https://github.com/shaarli/python-shaarli-client/commits/master
https://github.com/shaarli/python-shaarli-client/runs/4808361548

Tests started failing after an unrelated/documentation update on all tox environments with:

FAILED tests/client/test_v1.py::test_get_info_invalid_uri[shaarli-MissingSchema-No schema supplied]

Define command line usage/workflow

This issue aims at defining command-line usage of the Shaarli API client.

• Command line syntax (available parameters/arguments)
• Output formatting (success/error messages/shaare listings/export to other formats)

Example CLI usage that comes to mind. Feel free to edit it with your suggested improvements:

    USAGE: shaarli-client.py SHAARLI_URL METHOD URI [--param1='value' --param2='value' ...]:
        available methods: GET, POST, PUT
        GET parameters: limit, searchterms, searchtags, offset, visibility, format
        POST parameters: url, tags, description
        PUT parameters: ...
    shaarli-client.py https://my.links.example/ GET /links/349
    shaarli-client.py https://my.links.example/ /links --limit=10 --searchterm=dankmemes
    shaarli-client.py https://my.links.example/ /links --visibility=private
    shaarli-client.py https://my.links.example/ POST /links --url="https://github.com/shaarli/python-shaarli-client" --tags="software,python" --description='A Python 3 Command-Line Interface to interact with a Shaarli instance.'

I'm unsure about how output format selection should be implemented/made available.
Should --format be an available parameter for GET, or can it be more loosely coupled? The client would output raw JSON, and extra actions could be ran:

--actions=output_prettyprint,output_html,download_media,download_pages,send_archiveorg,...

It should also be possible to stash configuration values in a configuration file (eg. API token, default shaarli_url,...)

Missing Commands after installing from pip

I only see:

usage: shaarli [-h] [-c CONFIG] [-i INSTANCE] [-u URL] [-s SECRET]
               [--format {json,pprint,text}]
               {get-info,get-links,post-link} ...

positional arguments:
  {get-info,get-links,post-link}
                        REST API endpoint
    get-info            Get information about this instance
    get-links           Get a collection of links ordered by creation date
    post-link           Create a new link or note

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Configuration file
  -i INSTANCE, --instance INSTANCE
                        Shaarli instance (configuration alias)
  -u URL, --url URL     Shaarli instance URL
  -s SECRET, --secret SECRET
                        API secret
  --format {json,pprint,text}
                        Output formatting

not (as per https://python-shaarli-client.readthedocs.io/en/latest/user/usage.html#getting-help):

$ shaarli -h

usage: shaarli [-h] [-c CONFIG] [-i INSTANCE] [-u URL] [-s SECRET]
               [--format {json,pprint,text}]
               {get-info,get-links,post-link,put-link,get-tags,get-tag,put-tag,delete-tag}
               ...

positional arguments:
  {get-info,get-links,post-link,put-link,get-tags,get-tag,put-tag,delete-tag}
                        REST API endpoint
    get-info            Get information about this instance
    get-links           Get a collection of links ordered by creation date
    post-link           Create a new link or note
    put-link            Update an existing link or note
    get-tags            Get all tags
    get-tag             Get a single tag
    put-tag             Rename an existing tag
    delete-tag          Delete a tag from every link where it is used

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Configuration file
  -i INSTANCE, --instance INSTANCE
                        Shaarli instance (configuration alias)
  -u URL, --url URL     Shaarli instance URL
  -s SECRET, --secret SECRET
                        API secret
  -f --format {json,pprint,text}
                        Output formatting
  -o --outfile FILENAME
                        File to save the program output to

versions:

$ pip freeze
certifi==2018.11.29
chardet==3.0.4
idna==2.8
PyJWT==1.7.1
requests==2.21.0
requests-jwt==0.5.3
shaarli-client==0.2.0
urllib3==1.24.1

Add an option to bypass SSL certificate verification (self-signed certs)

My test environment has Shaarli sitting behind a webserver configured to use a self-signed certificate. This causes python-shaarli-client client to fail with a requests.exceptions.SSLError ([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)).

I have worked around it by adding verify=Falseto parameters of the requests.request() call but having a --insecure command-line option to bypass SSL checks on-demand would be a nice improvement.

Post link with newline in the description

How can post a link from the command line that has a description that should contain a newline when displayed in the shaarli online instance?

When trying something like (see the newlines in the description)

shaarli post-link --url 'http://test.url/' --title 'Test' --description 'One\nword\nper\nline.'

The shaarli script (v 0.3.0) returns

{
    "created": "2019-08-04T09:56:11+02:00",
    "description": "One\\nword\\nper\\nline.",
    "id": 2786,
    "private": false,
    "shorturl": "r-KS6g",
    "tags": [],
    "title": "Test",
    "updated": "",
    "url": "http://test.url/"
}

The newlines are escaped, and the text in the online shaarli instance contains the '\n' text verbatim instead of the actual newline.

Using '\r\n' instead of '\n' does not work either.

May this be a shaarli API issue?

add media/page archiving capabilities

Moved from #22

add a subcommand parser dedicated to data archival:

• should be implemented in a library form, with a console entrypoint that may serve as a Minimal Working Example

these operations should be performed separately:
query a Shaarli instance to get a list of links
parse a list of links and retrieve/archive corresponding media

Archival preferences could be specified in a config file

• local archive directories
• multimedia preferences, e.g. audio & video formats

Suggestions:

• move API commands to an api subparser, and declare other subparsers for specific actions:

   $ shaarli api <params>
   $ shaarli archive <params>
   $ shaarli <action> <params>

there's also the possibility of providing an interactive CLI entrypoint using the click library (possibly overkill but potentially quite fun to write :) )

Documentation: provide a complete example

Currently the Usage documentation page does not provide a complete example, with all field being used.

For example, I misused the --tags flag, as it is supposed to be an array of values instead of a string. See shaarli/Shaarli#1651

get-links: excluding tags from results (using `-` operator) does not work

https://shaarli.example.org/?searchterm=&searchtags=video+-idontwantthistag will return all links tagged video but NOT tagged idontwantthistag. Excluding tags in this way using the python client does not appear to work.

(.venv) $ shaarli get-links --searchtags video -idontwantthistag
shaarli: error: unrecognized arguments: -idontwantthistag

(.venv) $ shaarli get-links --searchtags video '-idontwantthistag'
shaarli: error: unrecognized arguments: -idontwantthistag

(.venv) $ shaarli get-links --searchtags video '\-idontwantthistag'
[]

This might be a limitation of the API (?), I have not checked using curl since I don't know exactly how to generate a JWT token to use with curl --header 'Autorization: Bearer ...

Configuration file generation

Mentioned in #15 (review)

Some ideas:

• add a setup subcommand
• automate configuration file generation (maybe first-time setup is enough?)

adding get_link() method and get-link endpoint

Hi all

I forked the repo and added a get_link() method and get-link endpoint to fetch a single link by id:

master...andreworg:python-shaarli-client:andrew-get_link

I needed this for a project of mine (see shaarli/Shaarli#1943); I think it also serves as partial workaround for #34 (you can use get_link() to fetch all data for an existing record, then use put_link() to update the record).

I'm writing this here before submitting a PR because I have not updated the test suite yet - I'm having trouble with that - and I was wondering what to do.

Thanks

Usage in a pelican plugin

Just FYI, I used this package to build a pelican plugin : getpelican/pelican-plugins#1167

Thank you for writing it and releasing it !

[sphinx] make singlehtml fails

$ pip install --user --pre -r requirements/docs.txt
...
$ export PATH="$PATH:$HOME/.local/bin"
$ cd docs; make singlehtml
Running Sphinx v1.5.1
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [singlehtml]: all documents
updating environment: 3 added, 0 changed, 0 removed
reading sources... [100%] user                                                                      
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
assembling single document... user developer 
writing... 
Exception occurred:
  File "/home/user/.local/lib/python2.7/site-packages/sphinx_rtd_theme/breadcrumbs.html", line 9, in top-level template code
    {% if meta is defined and 'github_url' in meta %}
TypeError: argument of type 'NoneType' is not iterable
The full traceback has been saved in /tmp/sphinx-err-9fJXsV.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
Makefile:20 : la recette pour la cible « singlehtml » a échouée
make: *** [singlehtml] Erreur 1

Better error message if the configuration isn't specified

Currently I get ERROR:root:'shaarli' with the help.

ModuleNotFoundError: No module named 'requests_jwt'

(update: setting requests-jwt == 0.4 in setup.py (git install) fixes this)

Ubuntu 18.04, followed these instructions.

shaarli --help
Traceback (most recent call last):
  File "/home/tbrown/.virtualenvs/shaarli/bin/shaarli", line 7, in <module>
    from shaarli_client.main import main
  File "/home/tbrown/.virtualenvs/shaarli/lib/python3.6/site-packages/shaarli_client/main.py", line 7, in <module>
    from .client import ShaarliV1Client
  File "/home/tbrown/.virtualenvs/shaarli/lib/python3.6/site-packages/shaarli_client/client/__init__.py", line 2, in <module>
    from .v1 import InvalidEndpointParameters, ShaarliV1Client
  File "/home/tbrown/.virtualenvs/shaarli/lib/python3.6/site-packages/shaarli_client/client/v1.py", line 7, in <module>
    from requests_jwt import JWTAuth
ModuleNotFoundError: No module named 'requests_jwt'

pip freeze
certifi==2018.10.15
chardet==3.0.4
idna==2.7
pkg-resources==0.0.0
PyJWT==1.6.4
requests==2.20.0
requests-jwt==0.5.1
shaarli-client==0.2.0
urllib3==1.24

(shaarli) tb:ot: /mnt/edata/edata/shaarli
shaarli:0> python
Python 3.6.6 (default, Sep 12 2018, 18:26:19) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from requests_jwt import JWTAuth
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'requests_jwt'

Something upstream broken maybe?

post-link with a tag

Hello all,
after reading the doc , is it possible to add a link with his tag in one line ?

Traceback error ( AttributeError: 'str' object has no attribute 'decode')

Python 3.7.3

When I enter shaarli get-info (or really any valid command) I get this:

Traceback (most recent call last):
File "/home/pi/.virtualenvs/shaarli/bin/shaarli", line 10, in
sys.exit(main())
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/shaarli_client/main.py", line 58, in main
response = ShaarliV1Client(url, secret).request(args)
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/shaarli_client/client/v1.py", line 267, in request
return self._request(* self._retrieve_http_params(args))
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/shaarli_client/client/v1.py", line 261, in _request
params=params
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/requests/sessions.py", line 528, in request
prep = self.prepare_request(req)
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/requests/sessions.py", line 466, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/requests/models.py", line 320, in prepare
self.prepare_auth(auth, url)
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/requests/models.py", line 551, in prepare_auth
r = auth(self)
File "/home/pi/.virtualenvs/shaarli/lib/python3.7/site-packages/requests_jwt/init.py", line 165, in call
request.headers['Authorization'] = self._header_format % token.decode('ascii')
AttributeError: 'str' object has no attribute 'decode'

I do have the correct secret and URL of my instance.

Support creation and update dates

For POST and PUT link endpoint, the REST API will support new flags created and updated. They're dates in ISO 8601 format.

Related to:

• shaarli/Shaarli#1223
• shaarli/Shaarli#1525
• shaarli/api-documentation#36

Use Pipenv for packaging/distribution/installation

https://packaging.python.org/tutorials/managing-dependencies/#managing-dependencies now recommends Pipenv as the preferred packaging/installation method for Python projects. There is a short usage example at https://www.ostechnix.com/pipenv-officially-recommended-python-packaging-tool/.

It would be interesting to improve the virtualenv-based setup described at https://python-shaarli-client.readthedocs.io/en/latest/user/installation.html by using Pipenv which seems to abstarct away virtualenv management for end users, while retaining advantages of a venv setup.

Create the project layout

• #1 - changelog, readme, base packaging info
• #3 Sphinx documentation stub
• #3 RTFD integration
• #5 Tox testing automation
• #5 Travis CI integration
• #6 Static analysis tools (Pylint, Pydocstyle, Pycodestyle, isort)
• #8 setuptools entrypoint and api/v1/info client
• #12 #13 Base usage & development documentation
• #13 PyPI release & documentation

endpoints KeyError

Got a KeyError from .post_link() fixed with the following edit:

diff --git a/shaarli_client/client/v1.py b/shaarli_client/client/v1.py
index 5b029ec..5c296b6 100644
--- a/shaarli_client/client/v1.py
+++ b/shaarli_client/client/v1.py
@@ -277,7 +277,7 @@ class ShaarliV1Client:

     def post_link(self, params):
         """Create a new link or note"""
-        self._check_endpoint_params('post-links', params)
+        self._check_endpoint_params('post-link', params)
         return self._request('POST', 'links', params)

     def put_link(self, resource, params):

Didn't check the rest of the self._check_endpoint_params() calls. After fixing this, python-shaarli-client got all my bookmarks pushed into shaarli, thanks!

put-link should update only specified fields instead of overwriting all fields

The Usage section of the documentation says that "put-link" "Updates an existing link or note". This is not true. What it does is to transform a link into a private note with a different title. Even the example ("shaarli put-link --private 3252") in the documentation itself shows this.

Also, the inline help says:

(shaarli) [root@fima-01-hetzner ~]# shaarli -c /root/scripts/shaarli.conf   put-link --help
usage: shaarli put-link [-h] [--description DESCRIPTION [DESCRIPTION ...]]
                        [--private] [--tags TAGS [TAGS ...]]
                        [--title TITLE [TITLE ...]] [--url URL]
                        resource

positional arguments:
  resource              Link ID

optional arguments:
  -h, --help            show this help message and exit
  --description DESCRIPTION [DESCRIPTION ...]
                        Link description
  --private             Link visibility
  --tags TAGS [TAGS ...]
                        List of tags associated with the link
  --title TITLE [TITLE ...]
                        Link title
  --url URL             Link URL

but if add and then I try to update a link, it gets changed title, and URL, just like in the docs. But why should they change, if one only sets status to private?:

(shaarli) [marco ~]# shaarli -c shaarli.conf  post-link --url https://www.linux.com --tags "freeSoftware testing" --description "go penguins" --title 'A nice operating system'
{
    "created": "2018-09-04T18:54:08+02:00",
    "description": "go penguins",
    "id": 4,
    "private": false,
    "shorturl": "Gxqflg",
    "tags": [
        "freeSoftware",
        "testing"
    ],
    "title": "A nice operating system",
    "updated": "",
    "url": "https://www.linux.com"
}
(shaarli) [marco ~]# shaarli -c shaarli.conf  put-link --private 4
{
    "created": "2018-09-04T18:54:08+02:00",
    "description": "",
    "id": 4,
    "private": true,
    "shorturl": "Gxqflg",
    "tags": [],
    "title": "?Gxqflg",
    "updated": "2018-09-04T18:54:36+02:00",
    "url": "https://localhost/?Gxqflg"

Besides: "put-link --private true 2" , or e.g "put-link --title 'new title' does not work at all:

(shaarli) [marco ~]# shaarli -c shaarli.conf  put-link --private true 4
usage: shaarli put-link [-h] [--description DESCRIPTION [DESCRIPTION ...]]
                        [--private] [--tags TAGS [TAGS ...]]
                        [--title TITLE [TITLE ...]] [--url URL]
                        resource
shaarli put-link: error: argument resource: true is not a positive integer
(shaarli) [marco ~]# shaarli -c shaarli.conf  put-link --title 'A new title' 4
usage: shaarli put-link [-h] [--description DESCRIPTION [DESCRIPTION ...]]
                        [--private] [--tags TAGS [TAGS ...]]
                        [--title TITLE [TITLE ...]] [--url URL]
                        resource
shaarli put-link: error: argument resource: A new title is not a positive integer