Comments (37)
Hi!
So... are we any closer to a 3.3+ Matomo Debian package than we were in April?... 🙂 (sorry)
from matomo-package.
@to-go
Hi Torsten,
Yes, I'm well aware of this as you can imagine. I just spoke at length with @mattab and we have a plan that I need to action. We're targeting around mid-feb (18/02) if everything goes according to plan.
To share more with you, I lack of personal free time and we also want to make sure the work we put in the package will be useful in the long term.
I will use this issue to track my commit so you should be aware of the progresses made.
Thanks
ref:
from matomo-package.
Hi @mackuba,
Don't be sorry, I'm equally impatient as you are. I'm hoping to have version 3.3.0 out in the next 10 days or so. Maybe less but I can't guarantee due to work commitments (I'm presenting at the AWS Sydney Summit). But definitely keep an eye on this issue as I'll update it as soon as it's available.
from matomo-package.
Status report:
- Installed piwik transitional package successfully
- Installed matomo 3.3.0 successfully, including migrating
config.ini.php
(file copy)
I had to:
- restart apache2 service bit it should only be a warning, or a
reload
- remove the link in
/etc/apache2/conf.d/piwik.conf
(but unsure it belongs to our package)
from matomo-package.
@mackuba I hope so. Aiming for the end of this weekend.
from matomo-package.
I spoke with @mattab and we both agreed that 2b is the best solution of all. We should be rather close to have a transition package and having Matomo 3.3.0 released at an internal test version. If all goes well then I'll publish version 3.4.0 soon after.
from matomo-package.
Status report:
master
containsmatomo
package's main branch.piwik-package
contains only the transitional package.- no new package has been pushed yet to the Debian/Ubuntu repository yet
Both branches have been pushed to Github.
The next step is to test the upgrade process as much as possible.
from matomo-package.
Status report:
- published
piwik
(9.9.1-1) - published
matomo
(3.3.0-1)
I'm going to publish the other missing versions very soon.
from matomo-package.
thank you!!! works as advertised. couple of steps necessary, but nothing too difficult.
$ sudo a2disconf piwik
removing dangling symlink /etc/apache2/conf-enabled/piwik.conf
vi /etc/apache2/sites-enabled/000-default.conf
and replace most instances of piwik with matomo
then service apache2 reload
and bob's your uncle! again, great job aureq, and thank you VERY much! :)
EDIT: there were a couple of minor issues I discovered after deploying this. firstly, make sure you edit the cron job to point to the new files (/usr/share/matomo in place of /usr/share/piwik)...secondly, I had a weird issue with a custom plugin that I needed to cp -Rp over from the piwik dir to the new matomo dir. once I did those things, it appears that all is well.
from matomo-package.
Perhaps this is just waiting for a fix upstream but since it is likely to require users to do a second rework of their sites tracking code for optimal deployment it is worth special mention
fyi indeed we haven't yet fixed it into core: matomo-org/matomo#12785
once it will be matomo.js and matomo.php we will still keep backward compatibility with piwik.js|php so you won't have to change your tracking codes.
from matomo-package.
Ok, for the people who are following this thread, I'll be brain-dumping as I progress through.
There's a lot of left-over from piwik
and matomo
is not everywhere yet. To name a few:
- https://builds.matomo.org/ contains only archives named
piwik-*
piwik-3.3.0.(tar.gz|zip)
has an initial folder namedpiwik
Here's an incomplete list of things to do :
- Merge
origin/matomo
intomaster
- Update
scripts/build-package.sh
to support Matomo and Piwik at the same time. Then @mattab to republish version3.3.0
and the following beta version. - Create a transition package for Debian and Ubuntu without breaking everything
- Make the transition package coexist alongside with Matomo in this repository
- Remove as many traces of
piwik
from the Matomo Debian package (where sensible to do so) - Move
/etc/piwik
to/etc/matomo
due to the symlink namedconfig/
. I think the Piwik transitional package should own/etc/piwik
and Matomo package should own/etc/matomo
. But both pointing at the same location. - Similar to
/etc/piwik
, the web server alias/piwik
should belong to Piwik package and/matomo
should belong to Matomo package - Update content for https://debian.matomo.org/ with up to date instructions or alternatively have a proper installation documentation on https://matomo.org
from matomo-package.
@mattab I noticed that in scripts/build-package.sh#L439 the script recommends sending an email to Microsoft App Store, but the filename in the message body is piwik-X.X.X-WAG.zip
.
Should the script use instead matomo-X.X.X-WAG.zip
instead?
from matomo-package.
@mattab the latest build script has a few features. It publishes and archive for piwik
and has piwik/
as its main folder, and for the matomo
archive the main folder is matomo/
. This should give users who download these archives to transition smoothly.
On your side, if you continue to execute the script as usual, you will publish both matomo
and piwik
archives on the main website. However, only matomo
is published for the Web App folder. I noticed that https://webgallery.microsoft.com/apps/Piwik is quite outdated and I don't know if this is intentional.
Last, if you could review my code changes, that would be great.
Cheers
from matomo-package.
Hi @aureq
Thanks for the update! Feedback:
- Looks like Microsoft is not updating the app anymore, unfortunately. It's not so important for now so we can still leave the Webgallery package generation. It's OK to have only the Matomo package for the webgallery 👍
- Would you mind doing future changes to the script in a pull request? this way I can review the changes
- fyi: I will review the changes next time I deploy a beta, next week
from matomo-package.
@mattab Have you had a chance to test the latest script version? Would it be possible to publish the matomo package alongside with the piwik package please?
from matomo-package.
In order to create a transitional package from piwik
to matomo
, we have 2 options
1 - update debian/control
and add a second Package:
section to reference Matomo.
2 - duplicate the current repository back into piwik-package
(or create a new one) and:
2a - update the new piwik-package
to include only transitional information (symlinks, configuration migration and so on)
2b - update matomo-package
so it contains the Matomo application, docs and configuration files
If we do 1 each time we generate a Matomo release, a Piwik release will be generated at the same time and both with share the same version number. This is kind of annoying as it will be confusing to many users, but also because if the transitional package has to be updated, then Matomo will receive a new version at the same time (and vice versa).
If we do 2, it create a bit more work and duplicates some work, however we have control over each package independently which is more desirable.
Last question is to understand whether an entirely new/forked repository is more appropriate (better visibility) rather than a branch in this repository instead (less management overhead).
from matomo-package.
- I can't publish the package because one of my command is missing (reprepro) which hasn't been reinstalled with the server upgrade.
- I finally have something that seems to be working fine, but I haven't tested it through a proper Debian repository.
from matomo-package.
Hi!
Sorry for asking that kind of question, but do you have any approximate timeline for when 3.3/3.4 Debian packages might be available on http://debian.matomo.org? Is it like probably this/next week if things go well, or rather further out?
from matomo-package.
@mattab If it could be possible to have inoticoming installed on the server that would be great.
from matomo-package.
inoticoming is installed @aureq 👍
from matomo-package.
Status update for those following this issue.
- my Vmware refuses to provide me with network connectivity which makes my work useless right now (VM boots but no network)
- I need to create an EC2 instance and have my lab in it rather than in Vmware
- find a way to extra any work that hasn't been committed into GIT and move it to my lab EC2 instance
- do some basic checks between Debian 8 (Vmware) and Debian 9 (EC2)
An important aspect to this is securely transferring my GPG signing keys to my lab EC2 instance.
from matomo-package.
Hi @aureq - would it be possible for you to run all the code from the SSH account provided for debian.matomo.org ? This way it's fully backed up, etc. maybe it could make things simpler?
from matomo-package.
Hi @mattab and all
TL;DR - Remove the human component and automate everything.
Long version:
@mattab technically yes, but probably not a good idea. It's a shared box, ideally the environment should be tightly controlled bu us only due to sensitive material like GPG signing keys. I also remember it's tricky to get all the packages required installed in a timely manner (I don't have any list either yet).
While I'm super time poor at the moment, that Vmware event shows that I need to take the bullet, step up and make the process of creating a .deb
fully automated (or as much as it is possible to).
I need to do the following:
- Manually generate an EC2 instance from a vanilla AMI
- Record the list of installed packages and specific commands
- Record the deployment process for Matomo's package
- Secure and automate the GPG keys handling, SSH private key and remote server public keys (using S3 and KMS)
Once the steps above are done:
- Create a bootstrap script to include all the above
- Create a CloudFormation template to deploy an EC2 instance(*) and inject the bootstrap script
- Push the resulting
.deb
intodebian.matomo.org
for publication
From a more strategic and long term perspective...
I would say this is probably the best initial steps we should be taking. As additional improvements we should consider the following:
- Use AWS CodeBuild with a
buildspec.yml
instead of EC2 (Better/longer AWS free-tier coverage) - Automatically trigger a build when a new release is published
- Move the Debian/Ubuntu repository outside of our dedicated hosting
- Automate the Debian/Ubuntu repository and publish it to CloudFront and S3(*).
Benefits would be:
- Near real-time package publication for Debian/Ubuntu
- Ability to publish
beta
andrc
packages (great for early adopters and testers) - No web hosting infrastructure due to S3 and CloudFront for a possibly small fee
- A very secure environment to generate, publish and host all packages due to architecture and AWS services like KMS and Server Side Encryption (SSE)
- Portability of such environment into any AWS account (Infrastructure as code)
- No/Minimal human intervention required
- Happy community users
(*) Cost should be considered on the following items.
from matomo-package.
People interested in this issue should follow #64 as well.
from matomo-package.
@aureq Sounds all good if you can maintain the EC2 setup so it keeps running for the years ahead....! 👍
I've just release the 3.5.0-b3 using the updated script so you now have both matomo and piwik packages available.
All the best for this project and your upcoming event... 🥇
from matomo-package.
Status report:
I was hoping to keep the transitional package and the main package together inside the same branch, but it doesn't appear to be possible. I'm going to create a separate branch piwik-transition
that will only contain the dummy/transitional package while master
will hold the real package.
Doing this allows me to:
- bump
piwik
version to 9.99.9 to signal death of this package, while pullingmatomo
as a required dependency. - not have to update the
piwik
package as frequently asmatomo
- keep code separated and easier to maintain over time
from matomo-package.
Status report:
- published
matomo
(3.4.0-1) - published
matomo
(3.5.0-1) - published
matomo
(3.5.1-1)
I used the official package repository to upgrade my own Matomo installation. The only hurdle is switching to newer webserver configuration but everything else should be mostly smooth.
from matomo-package.
thank you, working!
The directories in apaches matamo.conf point to "/usr/share/piwik". Is that intended?
from matomo-package.
@ptr1120 Not sure what you are referring to. Could you please clarify ?
If you are talking about /etc/apache2/conf-available/matomo.conf
, then it should point to /etc/matomo/apache.conf
. This link is created during the installation process (see)
from matomo-package.
@jawrat thanks for reporting all the details:
a2disconf
: I'll look into that./etc/apache2/sites-enabled/000-default.conf
: I don't think this is related to the Debian package.- Plugins: I'll look what could be done regarding Matomo plugins (Had a similar problem)
- Cron: there should be a new cron file named
/etc/cron.d/matomo-archive
. But I guess the old cron should be removed for sure.
from matomo-package.
@aureq: sorry, I talk about the directory directives inside /etc/apache2/conf-available/matomo.conf
that point to /usr/share/piwik
(see). Shouldn't they point to matomo folder?
from matomo-package.
@ptr1120 You're absolutely right. Let me fix that quickly.
from matomo-package.
@ptr1120 I pushed the updated package. Let me know if this is all good for you. Cheers
from matomo-package.
Thanks, installed 3.5.1-2, looks perfect.
from matomo-package.
Thank you so much for this development! When updating from the existing package, I just had to do @jawrat's indication to sudo a2disconf piwik
, edit the virtualhost aliases in conf-enabled/matomo.conf
and everything's going smooth.
from matomo-package.
Some feedback from someone who has just completed their first migration from Piwik to Matamo.
I too had the dangling symlink /etc/apache2/conf-available/piwik.conf
which for such a small issue results in a hard fail of Apache startup.
I also seen the issue with addons. All the native addons migrated successfully but my extra addon installed from the store (free) QueuedTracking failed to transfer. Matomo expected this addon to be there based on the stored config and complained loudly about it. I can only assume it also broke the install but after consulting this thread I moved directly to the cp -Rp /usr/share/piwik/plugins/QueuedTracking/ /usr/share/matomo/plugins/QueuedTracking/
fix which work immediately as as expected.
On an upgrade ideally both the Piwik and Matomo Aliases in /etc/apache2/conf-available/matomo.conf
should be uncommented as I dont think it is normal behavior for apt update to turn daemons off. It also helps clear up potential confusion and loss of tracking after the apt upgrade is complete but before the users have reworked their sites to use the new Matomo tracking.
On that note I expected the new tracking code to be fully rebranded and was surprised to see a mentions of Piwik
<!-- Matomo -->
...
var u="https://example.com/matomo/";
_paq.push(['setTrackerUrl', u+'piwik.php']);
...
g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'piwik.js'; s.parentNode.insertBefore(g,s);
...
<!-- End Matomo Code -->
Perhaps this is just waiting for a fix upstream but since it is likely to require users to do a second rework of their sites tracking code for optimal deployment it is worth special mention
It has been suggested that /etc/cron.d/piwik-archive
should be deleted but this file can often user crafted configuration. Better would be to rename the file to new /etc/cron.d/matomo-archive
and add the new sample cron entry for reference. After all the currently upgrade leaves the old one running anyway.
Lastly I am still a bit vague about owners and permissions. This is an area Matomo does quite badly at itself with vague advice and quite a bit of trial and error debates in their forums. I would say that it feels odd to have web assets owned by root:root by default after this upgrade. I think this is something given we have a relatively fixed environment we can do better with.
Let this not take away from the excellent work done here. I cannot begin to imagine how much time you have saved me. Kudos
Update: I have completed 5 more updates with no new issues.
from matomo-package.
question: am I missing something or have packages for the latest builds not been published to the repo? I was going to update to 3.6 but it doesn't appear that the repo has been updated to that....
@aureq @mattab ?
from matomo-package.
Related Issues (20)
- Smaller release file (remove not needed files from release) HOT 2
- Do not prepend autoloader HOT 1
- Package for 3.15.5 HOT 3
- On upgrade "The Matomo JavaScript tracker file "/matomo.js" & "/piwik.js" is not writable" HOT 2
- No debian package since 3.13.6 HOT 17
- Automate the packaging of Matomo beta/rc/stable releases via Github actions HOT 6
- Remove the WAG build from the script
- Releasing Matomo 4.0 for Debian HOT 13
- Don't include JS test files in the release
- Exclude phpcs.xml from release package
- Is the Matomo Debian package dead? HOT 72
- Working on debian package generation: .eslintrc ? HOT 1
- Working on debian package generation: autoload.php HOT 1
- Manifest / Integrity check has issues with symlinks HOT 2
- Changes in website format prevents pulling changelog when generating Debian package HOT 8
- Matomo 3.8.1 update? HOT 9
- libmaxminddb recommendations HOT 22
- matomo.php and matomo.js are missing from debian package HOT 1
- Security / Policy breach /usr/share/matomo/misc/user HOT 6
- Debian stretch (Matomo will stop supporting PHP 7.0 in the next major version.) HOT 17
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from matomo-package.