Coordination of packaging inside deb-based distribution

[hosiet]

Coordination of packaging inside deb-based distribution

[robert7]

Would it be possible if you provide with a fallback path to use system's libtidy5 if tidy5.6 is not found under /opt/? That will help ease the packaging for sure.

Hmm... but the "system version" would almost for sure be older then 5.6... E.g. Ubuntu 18.04 has 5.2 and Ubuntu 16.04 has "0.99" (really really OLD). My idea was to avoid complexity by using X versions of the tidy. I thinks this was also problem in original v2.0 and thus Randy switched from using tidy library to using tidy binary (which I thinks was greatly suboptimal - as e.g. Ubuntu 16.04 had version 0.99).

Note: I have almost no experience with linux packaging :) I had none 2 weeks ago and I really struggled with it :) wanted to give up few times - but then someway succeeded to bring out something that works (and I only succeeded because I could use the existing version from @artmg which was based on your version I thins).

So the existing packaging can be for sure improved - I open to any suggestions.

Isn't possible to add extra package which will ship tidy 5.6 (with new name? so it doesn't conflict with existing system default tidy-html5?)

Other idea (of beginner) .. we could statically link the library..

[robert7]

I intend to update the old nixnote2 package in deb-based distributions (starting from Debian, down to Ubuntu, Linux Mint, Deepin etc) using the new codebase as a Debian Developer

btw ... this would be really great!!
One of the main current problems is, that the my fork is basically invisible. So I think currently there are just few users. If we can update the debian packages, this would make the update more visible. And my motivation to stay with the project would be much greater.

Another issue: I changed the package name to nixnote21 to allow parallel installation of v2.0 and v2.1 - but this can be temporary and I can later change the package name back to "nixnote2".

[hosiet]

It's totally okay if you want to use some libraries that is not provided by the distribution; it happens from time to time. However, the packagers will have a headache if you choose to hardcode dependency at files in /opt or similar paths and that will be patched out by us :-(

I know that different distribution, especially old distributions, are providing old libtidy. I will try to deal with libtidy in deb-based distributions by pushing new version from Debian side... things will improve in the upcoming years.

For now, it should be much better to assume that a feasible libtidy is present in some standard location, e.g., /usr/local/ or even /usr/. Please search for them first before trying to look up some nonstandard paths. I believe QMake has already provided wrappers to search in standard path. After that (if the search returns nothing), you may consider using libtidy embedded in your nixnote2 repository -- code embedding is not good news to (distribution) packagers but that is still much saner than using code from /opt/tidy56/ since other users who want to build from source will not be forced to search for libtidy code on internet and extract a tidy 5.6 into /opt/ before building; all they need is your code repository. git submodule or hard copy are two widely accepted ways and I think they are both viable.

[robert7]

I'm wondering about the use of /opt/tidy56/ in Nixnote21.pro file

The current version is kind of first workaround with which I managed that, the packaging worked. As I said, I had no experience.

OK hardcoded dependency on /opt is not needed.I could do either submodule of copy. I will think about.

[artmg]

Another issue: I changed the package name to nixnote21 to allow parallel installation of v2.0 and v2.1 - but this can be temporary and I can later change the package name back to "nixnote2".

Perhaps @hosiet can comment from a deb upstream point of view, but from a source code, PPA release management, and general user understand standpoint I would suggest it makes sense to have a 'permanent change of identity' to nixnote21 (or perhaps even something slightly more drastic). It's not like this was one dev on a team handing over to another existing dev, but you had to fork away and now (you have a fleshy bot ;) telling users to come find your new project.

The new name would indicate a new era of feature improvement and bug fixing under new management, and long may it continue. Thank you for your effort and dedication, @robert7

[robert7]

@artmg althought I understand, what you are saying, I thought yesterday about the name change to "nixnote21" and I think this was not a good idea.
One point is, that it will be probably v2.2 one day.
Then I did not really change so much, that it would deserve an "new name".

Then for the future "roadmap": the plan is bugfixing, answering questions to users, keeping the project alive, adding some more likely smaller features and minor improvements, keeping up with the distribution library upgrades.., coordinating pull requests - if there will be any.. Unfortunately with all the respect, the source code is not really a show case for "clean code" and with the resources available, its impossible for me to do refactoring on a greater scale. And on the other side, it works quite well and for me, almost all important features are already there. I personally use it daily a lot. And it works very well. So I basically satisfied with it, how it is now.

But I don't really have time to do really much new development (I already spent like 10 times more time, that I originally intended). I didn't plan to become "maintainer" - I just wanted to contribute few new features.
So I don't plan to leave the project, but also at the same time, my time for this project is quite limited. For me its kind of personal "open source experiment" - I never did open source development before. I always wanted to try, just never had time. I don't have time now.. :) but I never will have more in the future. So I decided to try it out :) And so far it was quite fun..

With that said - I think I will change the name back to "nixnote2".
Of course anyone is free to stay with the v2.0, if they like it better. From my viewpoint the new version is at least a little bit better, then the previous one - but anyone can decide for them selves. Also I will not touch the your PPA, so v2.0 will stay available.

[robert7]

@hosiet:
Hi,

... as a starting point, I'm wondering about the use of /opt/tidy56/ in Nixnote21.pro file ...
... the packagers will have a headache if you choose to hardcode dependency at files in /opt or similar paths and that will be patched out by us ..

As a first change I removed from project the hardcoded dependency to "tidy" library directory.
The project now expects, that "tidy" can be found using pkg-config.
E.g. on Ubuntu 18.04, it now will build fine against system default "tidy", which is version 5.2.

Currently there is no version restriction in the project. I'm not sure, which version is "feasible" - I did some tests recently, it seems to work OK with version 5.2.
But as Evernote will reject invalid markup, the html cleaning is really critical for nixnote.
Therefore I would prefer to build against newest version.
I can't estimate, how much difference for our use case, is between 5.2 and 5.6.
So linking against 5.2 could be suboptimal, but should work fine in principe.

On Ubuntu 18.04 there is minor problem - for some reason the "libtidy-dev" package doesn't install the "tidy.pc" file (which is config for pkg-config), although the library itself does provide one. So I provided one in the project.

Now nixnote on Ubuntu 18.04 can be build using system libtidy using e.g.:

cd $PROJECTDIR
env PKG_CONFIG_PATH=$PROJECTDIR/development/pkgconfig qmake CONFIG+=release PREFIX=appdir/usr && make && make install

My debian packaging for PPA still uses the helper package "tidy-html56", which installs in /opt/tidy56, but this is now external to the project and doesn't need to bother you.

Also I changed the app name back to original "nixnote2".

So you could try the packaging for systems which already have libtidy 5.2+.

As next step I can think about embedding the "tidy" (as a solution for older systems and improvement for the ones which have tidy < 5.6).

[hosiet]

Just FYI, I've started pushing tidy >= 5.6 (in fact it is tidy 5.7.16) downto Debian. I believe it shall be available in next Debian stable release (Debian 10 "Buster") as well as Ubuntu 19.04 and later deb-based distributions. The tidy.pc problem is also solved there.

[robert7]

OK. Perfect.
I wonder from where the version 5.7.16 is coming - I checked https://github.com/htacg/tidy-html5 and the last tag is 5.6.0.. ?

Btw as an update to my the last message in this thread:

1. the dependency on tidy is now outside of project and need to be provided by pkg-config
2. it seems that all works with tidy 5.2 (but the html cleaning may be not as good as in 5.6). So I suggest using the newest tidy (5.6.0 or newer).
3. For my debian packaging in PPA, I created 2 packages
   • nixnote2-tidy - will build original tidy with tag 5.6.0 and install in /usr/lib/nixnote2/tidy
   • nixnote2 which then depends on nixnote2-tidy
   • this style could be used for the older distributions - but you can also arrange differently, if you have better idea

The current version "2.1.0-beta4l" is already a kind of "release candidate". The plan is to fix one more minor bug, then I will mark it as "2.1.0-beta4".. then wait week or two if something of importance pops out, then I'l tag is as "2.1.0".

[stevenpusser]

I think that Debian and Ubuntu policy would be much better satisfied if you just backported libtidy-dev 5.6.0 in your PPA and built against that. Much less Byzantine workarounds that way. I've managed to backport it to Debian Jessie, which is roughly equivalent to Ubuntu 16.04.

[hosiet]

Okay. During the last 3 weeks I've been dealing with libtidy in Debian and now Debian unstable & testing has tidy 5.6 with some of the 5.7+ patches. It was a mess but finally settled down. You can expect next Debian release (Debian 10) and Ubuntu 19.04 shipping new tidy.

As for backporting: I can backport libtidy for Debian Stretch (Debian 9) in official backports repository if you feel like it.

With shiny new nixnote2 2.1.0 tagged, I will probably start packaging it in Debian recently.

[stevenpusser]

Well, we had a request to add Nix Note 2 2.1.0 to the Jessie and Stretch-based MX Linux repos, so I already backported libtidy 5.6.0 for those platforms and was going to send those up to our test repos along with nixnote2, but I can't figure out what syntax to change the rules file to in order to find the 'tidy.h" file:

TIDY_LIB_DIR=/usr/lib/nixnote2/tidy

I tried

DEB_HOST_MULTIARCH ?= $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
TIDY_LIB_DIR=/usr/lib/$(DEB_HOST_MULTIARCH)/tidy

but it still couldn't find the header with libtidy-dev 5.6.0 added to build-deps and the pbuilder repo instead of nixnote-tidy..

[robert7]

@hosiet

With shiny new nixnote2 2.1.0 tagged, I will probably start packaging it in Debian recently.

great!

@stevenpusser

I think that Debian and Ubuntu policy would be much better satisfied if you just backported libtidy-dev 5.6.0 in your PPA and built against that. ...

Sorry, I have only very basic understanding about the packaging. So how I did, is probably best I could, with the time available. So I'm basically happy with the PPA config how it is now, but you are welcome to do the "official" packaging better.

Well, we had a request to add Nix Note 2 2.1.0 to the Jessie and Stretch-based MX Linux repos, so I already backported libtidy 5.6.0 for those platforms and was going to send those up to our test repos along with nixnote2, but I can't figure out what syntax to change the rules file to in order to find the 'tidy.h" file

I don't exactly understand, what you are trying to do. But basically the qmake project expects, that tidy can be found using pkg-config. That means tidy.pc needs to be present either in standard paths where pkg-config is searching or you need to set PKG_CONFIG_PATH to the non-standard path where tidy.pc is found. tidy.pcist then pkg-config config file, where the location of the library is defined.

Note that there is nothing special, how tidyis to be found - exactly same procedure is applied for other required libraries: poppler-qt5 libcurl hunspell.

Example:

=> that means that you don't need to set TIDY_LIB_DIR (I'm not sure in which context you trying to do that..)

If you have further difficulties, please write me how I can reproduce the problem you are facing.
Bear in mind, that beside my lame attempt to package nixnote in the PPA I hat never had anything to do with linux packaging, also never used Debian.
I'm on Ubuntu 18.04, but I can install Debian Jessie or Stretch in a VM and try to reproduce the steps you are trying, if you provide me those.

[robert7]

Another example - how to build with system default tidy located on system default location (provided on Ubuntu 18.04 by package "libtidy-dev").

make install
/usr/lib/qt5/bin/qmake -install qinstall -exe /d/dev/nixnote2/qmake-build-debug/nixnote2
....

robert7@LP-RSPIEGEL:/d/dev/nixnote2$ ldd appdir/usr/bin/nixnote2 |grep tidy
	libtidy.so.5 => /usr/lib/libtidy.so.5 (0x00007ff2f23a4000)

[hosiet]

@robert7 Tidy.pc was only introduced by tidy-html5 upstream back in May 2017 (!), which was pretty late (only 18 months earlier). You can't expect systems like Ubuntu 16.04 or Debian 9 to ship that file. Maybe you can try to search libtidy without using pkg-config...

Anyway, I have prepared an upload of nixnote2 2.1.0-1 into Debian Experimental; you may find my packaging scripts here: https://salsa.debian.org/debian/nixnote2/tree/experimental/debian . It is an initial version with many legacy files not cleaned up and I will surely make improvements later.

[robert7]

I have prepared an upload of nixnote2 2.1.0-1 into Debian Experimental

Thank you very much! Great!

Maybe you can try to search libtidy without using pkg-config...

As far I know, "pkg-config" is the standard way, how to find the libraries with qmake. Have no idea, how to do it differently. If you have any specific suggestion for better solution then I'm open to it :)

Regarding: https://salsa.debian.org/debian/nixnote2/blob/experimental/debian/README.Debian
The plugins are not used anymore. I removed one (camera plugin) as it was useless a and make the other one "spellchecker" integrated (as the plugin version had many problems). So there are no plugins anymore.

[robert7]

I looked at the patches you did; all should be integrated by now in develop branch. i'll merge in master soon and then create new tag (2.1.1).

[hosiet]

Looks great. I have prepared an upload onto Debian Unstable using git snapshot on 20181202; this basically means that Debian officially switched to this fork of nixnote2 and this version will be available in Debian 10 and Ubuntu 19.04. More updates will follow your future releases (2.1.1, etc).

I guess I will close this issue here since the major switch has finished. Will make further discussions in other separated issues. Thanks for all your work!

[robert7]

ok. great!

[robert7]

@hosiet
FYI: I integrated your last patch https://salsa.debian.org/debian/nixnote2/blob/master/debian/patches/0001-nixnote2.pro-Install-correct-upstream-changelog.patch
and created new tag 2.1.1 with all the changes and new icons