kyranb/footprints

:feet: A simple registration attribution tracking solution for Laravel (UTM Parameters and Referrers)

footprints's Issues

UTM POST issue

Can you add config parameter where can be set HTTP methods? Because now it use only GET method

Laravel 5.4 error

Hey,

I have just upgraded to Laravel 5.4 and seem to be getting an issue. The error is Closure object cannot have properties. The issue seems to be related to asyncTrackVisit in CaptureAttributionDataMiddleware.

Anyone else seen this or knows how to fix it?

UTM not being stored when in production at aws

Hi Kyranb,

I installed correctly the Footprints in my 5.4 laravel project and it worked well in my local environment using valet. But when running in production(AWS - Elastic Beanstalk) the table visits don't record a single visit with utm_* attribute, even if I force it in the URI.

Did you have any problem regarding this environment or any other during your development?

Multiple models

It would be nice if we could track multiple models instead of just one. Using a morphables, you could have model_type and model_id columns. This is useful when we have registration for several different models.

Scheduled deletion of old unclaimed Visits

Hey,

This package looks great!

Quick question, I noticed the attribution_duration config. Does this just amend how long the config is stored? Is there any built in way of deleting old Visit rows that weren't claimed by a user_id?

I know it can easily be done as a scheduled artisan command in Laravel but just wondering if there was any consideration to this/anything built in?

Thanks :)

using this in a RESTful project

how can i use this package in a RESTful project, (Models are in API and web site connect to database via API)

Error in cookie

Hi guys. I think I found a problem with the Footprinter.php file.

on the line 21 we have this function

public function footprint(Request $request): string
{
        $this->request = $request;

        if ($request->hasCookie(config('footprints.cookie_name'))) {
            return $request->cookie(config('footprints.cookie_name'));
        }

        // This will add the cookie to the response
        Cookie::queue(
            config('footprints.cookie_name'),
            $footprint = substr($this->fingerprint(), 0, 255),
            config('footprints.attribution_duration'),
            null,
            config('footprints.cookie_domain')
        );

        return $footprint;
}

the problem is in the line 32 because laravel make a cookie encryptation and then the first request returns the value stored in the variable $footprint

but the second request returns the cookie value encrypted

my solution was to add the cookie name in the EncryptCookies middleware and so we prevent laravel from changing the value.

INSTALLATION FAILS

- Installation request for laravel/framework (locked at v7.2.2, required as ^7.0) -> satisfiable by laravel/framework[v7.2.2].

when i try to 'composer require kyranb/footprints' i got this error

Class Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware does not exist

L.S.,

I got the following error:
ReflectionException in Container.php line 734: Class Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware does not exist in Container.php line 734 at ReflectionClass->__construct('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware') in Container.php line 734 at Container->build('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware', array()) in Container.php line 629 at Container->make('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware', array()) in Application.php line 697 at Application->make('Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware') in Pipeline.php line 126 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in VerifyCsrfToken.php line 64 at VerifyCsrfToken->handle(object(Request), object(Closure)) at call_user_func_array(array(object(VerifyCsrfToken), 'handle'), array(object(Request), object(Closure))) in Pipeline.php line 136 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32 at Pipeline->Illuminate\Routing\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 102 at Pipeline->then(object(Closure)) in Router.php line 726 at Router->runRouteWithinStack(object(Route), object(Request)) in Router.php line 699 at Router->dispatchToRoute(object(Request)) in Router.php line 675 at Router->dispatch(object(Request)) in Kernel.php line 246 at Kernel->Illuminate\Foundation\Http\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 52 at Pipeline->Illuminate\Routing\{closure}(object(Request)) in InputTrimmer.php line 20 at InputTrimmer->handle(object(Request), object(Closure)) at call_user_func_array(array(object(InputTrimmer), 'handle'), array(object(Request), object(Closure))) in Pipeline.php line 136 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32

One way to solve it is using php artisan optimize --force

Do you have an idea about what could go wrong?

`
protected $middleware = [
\Kyranb\Footprints\FootprintsServiceProvider::class,
];

protected $routeMiddleware = [
...
'capture.attribution' => \Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware::class,
...
];
`

Installation failed

Here is my composer require kyranb/footprints result:

Using version ^0.2.4@beta for kyranb/footprints
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

Problem 1
- Installation request for kyranb/footprints ^0.2.4@beta -> satisfiable by kyranb/footprints[0.2.4-beta].
- Conclusion: remove laravel/framework v5.1.45
- Conclusion: don't install laravel/framework v5.1.45
- kyranb/footprints 0.2.4-beta requires illuminate/support ~5.3 -> satisfiable by illuminate/support[5.3.x-dev, 5.4.x-dev, v5.3.0, v5.3.16, v5.3.23, v5.3.4].
- don't install illuminate/support 5.3.x-dev|don't install laravel/framework v5.1.45
- don't install illuminate/support v5.3.0|don't install laravel/framework v5.1.45
- don't install illuminate/support v5.3.16|don't install laravel/framework v5.1.45
- don't install illuminate/support v5.3.23|don't install laravel/framework v5.1.45
- don't install illuminate/support v5.3.4|don't install laravel/framework v5.1.45
- don't install illuminate/support 5.4.x-dev|don't install laravel/framework v5.1.45
- Installation request for laravel/framework (locked at v5.1.45, required as 5.1.*) -> satisfiable by laravel/framework[v5.1.45].

Installation failed, reverting ./composer.json to its original content.

and here is my installed packages:

anhskohbo/no-captcha 2.1.2 No CAPTCHA reCAPTCHA For Laravel. barryvdh/laravel-debugbar v2.3.0 PHP Debugbar integration for Laravel bosnadev/repositories 0.12 Laravel Repositories classpreloader/classpreloader 3.0.0 Helps class loading performance by generating a single PHP file containing all of the autoloaded files for a specific use case danielstjules/stringy 1.10.0 A string manipulation library with multibyte support dnoegel/php-xdg-base-dir 0.1 implementation of xdg base directory specification for php doctrine/annotations v1.3.0 Docblock Annotations Parser doctrine/cache v1.6.1 Caching library offering an object-oriented API for many cache backends doctrine/collections v1.3.0 Collections Abstraction library doctrine/common v2.6.1 Common Library for Doctrine projects doctrine/dbal v2.5.5 Database Abstraction Layer doctrine/inflector v1.1.0 Common String Manipulations with regard to casing and singular/plural rules. doctrine/instantiator 1.0.5 A small, lightweight utility to instantiate objects in PHP without invoking their constructors doctrine/lexer v1.0.1 Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers. fzaninotto/faker v1.6.0 Faker is a PHP library that generates fake data for you. guzzlehttp/guzzle 6.2.2 Guzzle is a PHP HTTP client library guzzlehttp/promises 1.2.0 Guzzle promises library guzzlehttp/psr7 1.3.1 PSR-7 message implementation hamcrest/hamcrest-php v1.2.2 This is the PHP port of Hamcrest Matchers intervention/image 2.3.8 Image handling and manipulation library with support for Laravel integration jakub-onderka/php-console-color 0.1 jakub-onderka/php-console-highlighter v0.3.2 jeremeamia/SuperClosure 2.2.0 Serialize Closure objects, including their context and binding laravel/framework v5.1.45 The Laravel Framework. laravel/socialite dev-master 2979451 Laravel wrapper around OAuth 1 & OAuth 2 libraries. laravelcollective/html v5.1.9 league/flysystem 1.0.32 Filesystem abstraction: Many filesystems, one API. league/oauth1-client 1.7.0 OAuth 1.0 Client Library maximebf/debugbar v1.13.0 Debug bar in the browser for php application mockery/mockery 0.9.5 Mockery is a simple yet flexible PHP mock object framework for use in unit testing with PHPUnit, PHPSpec or any other testing framework. Its core goal is to offer a test double framework with a ... monolog/monolog 1.21.0 Sends your logs to files, sockets, inboxes, databases and various web services mtdowling/cron-expression v1.1.0 CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due nesbot/carbon 1.21.0 A simple API extension for DateTime. nikic/php-parser v2.1.1 A PHP parser written in PHP paragonie/random_compat v1.4.1 PHP 5.x polyfill for random_bytes() and random_int() from PHP 7 phpdocumentor/reflection-common 1.0 Common reflection classes used by phpdocumentor to reflect the code structure phpdocumentor/reflection-docblock 3.1.1 With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a DocBlock. phpdocumentor/type-resolver 0.2 phpspec/php-diff v1.0.2 A comprehensive library for generating differences between two hashable objects (strings or arrays). phpspec/phpspec 2.5.3 Specification-oriented BDD framework for PHP 5.3+ phpspec/prophecy v1.6.1 Highly opinionated mocking framework for PHP 5.3+ phpunit/php-code-coverage 2.2.4 Library that provides collection, processing, and rendering functionality for PHP code coverage information. phpunit/php-file-iterator 1.4.1 FilterIterator implementation that filters files based on a list of suffixes. phpunit/php-text-template 1.2.1 Simple template engine. phpunit/php-timer 1.0.8 Utility class for timing phpunit/php-token-stream 1.4.8 Wrapper around PHP's tokenizer extension. phpunit/phpunit 4.8.27 The PHP Unit Testing framework. phpunit/phpunit-mock-objects 2.3.8 Mock Object library for PHPUnit predis/predis v1.1.1 Flexible and feature-complete Redis client for PHP and HHVM psr/http-message 1.0.1 Common interface for HTTP messages psr/log 1.0.2 Common interface for logging libraries psy/psysh v0.7.2 An interactive shell for modern PHP. sebastian/comparator 1.2.0 Provides the functionality to compare PHP values for equality sebastian/diff 1.4.1 Diff implementation sebastian/environment 1.3.8 Provides functionality to handle HHVM/PHP environments sebastian/exporter 1.2.2 Provides the functionality to export PHP variables for visualization sebastian/global-state 1.1.1 Snapshotting of global state sebastian/recursion-context 1.0.2 Provides functionality to recursively process PHP variables sebastian/version 1.0.6 Library that helps with managing the version number of Git-hosted PHP projects segmentio/analytics-php dev-master 2025040 Segment Analytics PHP Library swiftmailer/swiftmailer v5.4.3 Swiftmailer, free feature-rich PHP mailer symfony/console v2.7.20 Symfony Console Component symfony/css-selector v2.8.13 Symfony CssSelector Component symfony/debug v2.7.20 Symfony Debug Component symfony/dom-crawler v2.7.20 Symfony DomCrawler Component symfony/event-dispatcher v2.8.13 Symfony EventDispatcher Component symfony/finder v2.7.20 Symfony Finder Component symfony/http-foundation v2.7.20 Symfony HttpFoundation Component symfony/http-kernel v2.7.20 Symfony HttpKernel Component symfony/polyfill-mbstring v1.2.0 Symfony polyfill for the Mbstring extension symfony/polyfill-php56 v1.2.0 Symfony polyfill backporting some PHP 5.6+ features to lower PHP versions symfony/polyfill-util v1.2.0 Symfony utilities for portability of PHP codes symfony/process v2.7.20 Symfony Process Component symfony/routing v2.7.20 Symfony Routing Component symfony/translation v2.7.20 Symfony Translation Component symfony/var-dumper v2.7.20 Symfony mechanism for exploring and dumping PHP variables symfony/yaml v3.1.6 Symfony Yaml Component vlucas/phpdotenv v1.1.1 Loads environment variables from .envtogetenv(), $_ENVand$_SERVERautomagically. webmozart/assert 1.1.0 Assertions to validate method input/output with nice error messages.

I've tried to remove all packages and reinstall etc.. still same. Do you have any idea what can be wrong?

Packagist auto update disabled

@kyranb Wanted to let you know that I just refreshed this package on Packagist because it does not auto-updates. This can be enabled using this guide: https://packagist.org/about#how-to-update-packages

Async write

Please make tag with last update async write

TrackVisit using authenticated user id

The TrackUser function is using the authenticated user id even if we are using a different model:

$user[config('footprints.column_name')] = Auth::user() ? Auth::user()->id : null;

Tracking occurs after controller and not before

An issue with the current middleware implementation is that the tracking is done after the request has been handled by the controller:

// src/Middleware/CaptureAttributionDataMiddleware.php

    /**
     * Handle an incoming request.
     *
     * @param \Illuminate\Http\Request $request
     * @param \Closure $next
     *
     * @return mixed
     */
    public function handle(Request $request, Closure $next)
    {
        // Anything done here is BEFORE the controller
 
        $response = $next($request);

        // Anything done here is AFTER the controller

        if ($this->filter->shouldTrack($request, $response)) {
            return $this->logger->track($request, $response);
        }

        return $response;
    }

A problem with this approach is if the very first request is the one creating a new model. In this case then the cookie would not be present when the new model is created and the system would (when handling async) log a visit AFTER the model was created.

Manually Creating a User Fails

Hi,

I think there might be an issue when manually creating a User through Eloquent (e.g. User::create($data)). In TrackRegistrationAttribution.php there is this boot method:

 public static function bootTrackRegistrationAttribution()
    {
        // Add an observer that upon registration will automatically sync up prior visits.
        static::created(function (Model $model) {
            $model->trackRegistration();
        });
    }

Which calls this:

    /**
     * Assign earlier visits using current request.
     */
    public function trackRegistration(Request $request): void
    {
        $job = new AssignPreviousVisits($request->footprint(), $this);

        if (config('footprints.async') == true) {
            dispatch($job);
        } else {
            $job->handle();
        }
    }

But there is no Request so it fails with error:

Too few arguments to function App\Models\User::trackRegistration(), 0 passed

Testing on localhost (no cookie/visit being recorded)

Should this work on localhost? It seems the cookie isn't being set when I manually input the query string parameters.

SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'referrer_url'

This SQL error

PDOException: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'referrer_url' at row 1
  File "/public/index.php", line 52
    $request = Request::capture()
...
(64 additional frame(s) were not displayed)

occurs when the referrer_url is too long for the database column which is per default set in the migration to string('field') which creates a varchar(255).

This is causing problems for us for some email client where the referrer is a really long string with some unique ids auto generated.

Basically there are two options:

1. Change the default migration to some longer strings
2. Add truncation before storing to the database

I personally lean towards truncating the fields since these long url's will not provided any useful data.

Case-sensitive PSR-4 autoloading: "Class 'Kyranb\Footprints\Jobs\AssignPreviousVisits' not found"

It looks like the lower-cased folder names prevent PSR-4 autoloading from being able to find the classes in this package.

I get this error:

In TrackRegistrationAttribution.php line 43:
                                                                 
  Class 'Kyranb\Footprints\Jobs\AssignPreviousVisits' not found

I experimentally renamed the vendor/kyranb/footprints/src/jobs directory to vendor/kyranb/footprints/src/Jobs to follow PSR-4 autoloading conventions, and the autoloader is then able to find that class.

This is with Composer version 1.10.1 2020-03-13 20:34:27, PHP 7.4.3 and Laravel v6.18.3.

Logging user IP and much more

@kyranb would you be willing to consider a PR implementing implementing what is described below?

Description

Currently there is a concept of "Custom Parameters" (defined in the configuration: footprints.custom_parameters) which should make it possible to log extra stuff that might be relevant. However we are limited to request inputs since this is implemented like so:

// https://github.com/kyranb/Footprints/blob/master/src/Middleware/CaptureAttributionDataMiddleware.php
    /**
     * @return array
     */
    protected function getCustomParameter()
    {
        $arr = [];

        if (config('footprints.custom_parameters')) {
            foreach (config('footprints.custom_parameters') as $parameter) {
                $arr[$parameter] = $this->request->input($parameter); // <--- Only inputs
            }
        }

        return $arr;
    }

Cons

Now we all know that cookies are not always the best way to track users. In many applications where it was not possible to track a cookie then I assume that it would be sufficient to match previous visits based on the IP address of the user (for many applications you will never see multiple users from the same IP address) or slightly better would be a combination of IP address and the User-agent provided in the headers. Now none of these approaches are possible using the package right now.

Suggestion

These suggestions would each constitute a breaking change, so I am proposing that all of them are implemented in the same go:

Adding IP

I suggest adding the IP of the request ($request->ip()) in CaptureAttributionDataMiddleware since this is simple and could be used for matching in many cases. This would also need to be added to the Visit model and to the migration.

Note that I am not proposing to save the headers as described as an option in the initial description.

Introducing CaptureHandler

I suggest modifying the CaptureAttributionDataMiddleware so that everything except the handle method is moved to a separate class called CaptureHandler which implements a new interface called CaptureHandlerInterface.

The point being that a developer can easily swap out the Handler for another concrete implementation and hence save the request in any way or shape they find useful. This would also solve #21.

Moving authenticated user

Since we are already introducing breaking changes then I suggest moving the TrackVisit.php#L26:

    public function handle()
    {
        $user = [];
        $user[config('footprints.column_name')] = Auth::user() ? Auth::user()->id : null; // <-- Moving to CaptureHandler enable modifications

        $visit = Visit::create(array_merge([
        ....

to the new CaptureHandler (previously CaptureAttributionDataMiddleware.php#L106:

which would partially solve #30

Problem identifying users when queued

Hello !

There is a bug in the src/Jobs/TrackVisit.php file.

Visit::create(array_merge([ config('footprints.column_name') => Auth::user() ? Auth::user()->id : null, ], $this->attributionData));

A queued job should not make a call to the Auth::user function (since it won't be available in async mode).

The Auth::user() should be passed when dispatching the job.

I made a PR here #59

Correct ordering problem for initial and final attribution data

When calling initialAttributionData or finalAttributionData, it calls the visits() method which is already ordered.

In the current state, the initialAttributionData doesn't work because it will call the visit method which has already ordered the results by created_at descending.

I created a PR #61

Exclude robots

Hi there ! Great package, thank you very much for this.
Just wondering if robots were excluded from footprints or not ?
I fell like it is not the case.
Thank you

user_id is always blank

I'm noticing that Visits "user_id" is never being set in my system. My registration is unconventional (occurs when another resource is created), and I'm wondering if this might be the cause.

Are we listening for an event on User created in this package?

Any help is greatly appreciated!

Could not find package kyranb/footprints

When I try running "composer require kyranb/footprints" it always returns the following error

[InvalidArgumentException]
Could not find package kyranb/footprints at any version for your minimum-st
ability (stable). Check the package spelling or your minimum-stability

I don't know how I can run this without this error. Any help would be great

P.S I am on Laravel 5.2 and I have PHP 7

AssignPreviousVisits job fails when run asynchronously

The TrackRegistrationAttribution trait has an assignPreviousVisits() method, which gets called when a TrackableInterface model's created event fires. This method dispatches the AssignPreviousVisits job — which is supposed to run on the queue if config('footprints.async') === true.

Unfortunately queuing the job fails every time, because:

Serialization of 'Closure' is not allowed

Laravel can't serialize the protected Request $request property on the class, as the Request class uses closures.

The way AssignPreviousVisits is implemented requires backwards-incompatible changes to become queuable again. This is because the Request class has been type-hinted in several places. It's possible to work around this issue, but it's not very elegant... (cc @bilfeldt, in case I'm missing anything?).

1. Create a custom function on your TrackableInterface model implementation. A custom function is required, as you can't just override the existing trackRegistration() function without the model becoming incompatible with TrackableInterface:

/**
 * Assign earlier visits using current request.
 *
 * @see \Kyranb\Footprints\TrackRegistrationAttribution::trackRegistration()
 */
public function trackRegistrationViaFootprint(string $footprint): void
{
    Visit::unassignedPreviousVisits($footprint)->update([
        config('footprints.column_name') => $this->id,
    ]);
}

2. Create a custom copy of the AssignPreviousVisits job, changing the Request $request property for string $footprint, and calling the new trackRegistrationViaFootprint() method:

/**
 * @see \Kyranb\Footprints\Jobs\AssignPreviousVisits
 */
class AssignPreviousVisitsViaFootprint implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected string $footprint;
    protected TrackableInterface $trackable;

    public function __construct(string $footprint, TrackableInterface $trackable)
    {
        $this->footprint = $footprint;
        $this->trackable = $trackable;
    }

    public function handle()
    {
        $this->trackable->trackRegistrationViaFootprint($this->footprint);

        event(new RegistrationTracked($this->trackable));
    }
}

3. Back on the TrackableInterface model, override the assignPreviousVisits() function from the TrackRegistrationAttribution trait, to call the custom job with a string $footprint (which can be serialized):

/**
 * Sync visits from the logged in user before they registered.
 *
 * @see \Kyranb\Footprints\TrackRegistrationAttribution::assignPreviousVisits()
 */
public function assignPreviousVisits(): void
{
    $job = new AssignPreviousVisitsViaFootprint(request()->footprint(), $this);

    if (config('footprints.async') === true) {
        dispatch($job);
    } else {
        $job->handle();
    }
}

UTM parameters as array = bug

Hi,

When UTM parameters are an array, the package will generate a full 500 errors which is not acceptable for a production app.

Example URL:
https://website.com/?utm_campaign[0]=p1&utm_campaign[1]=p2&utm_medium[0]=p3&utm_medium[1]=p4

Stack trace:

TypeError Kyranb\Footprints\TrackingLogger::Kyranb\Footprints\{closure}(): Argument #1 ($item) must be of type ?string, array given vendor/kyranb/footprints/src/TrackingLogger.php:58 Kyranb\Footprints\TrackingLogger::Kyranb\Footprints\{closure} [internal] array_map vendor/kyranb/footprints/src/TrackingLogger.php:58 Kyranb\Footprints\TrackingLogger::captureAttributionData vendor/kyranb/footprints/src/TrackingLogger.php:28 Kyranb\Footprints\TrackingLogger::track vendor/kyranb/footprints/src/Middleware/CaptureAttributionDataMiddleware.php:34 Kyranb\Footprints\Middleware\CaptureAttributionDataMiddleware::handle