heroku/heroku-buildpack-ci-redis

An experimental buildpack to provide Redis running inside CI dynos

Warning this is an experimental buildpack and is provided as-is without any promise of support.

Heroku CI buildpack: Redis

This experimental Heroku buildpack vendors Redis into the dyno. It is intended for use with Heroku CI or any other environment where data retention is not important.

Please note that Redis will loose all data each time a dyno restarts.

Usage

The first run of this buildpack will take a while as Redis is downloaded and compiled. Thereafter the compiled version will be cached. Redis will start locally and be available on redis://127.0.0.1:6379 available in the REDIS_URL environment variable.

By default Redis 7 is used, however you can specify a REDIS_VERSION in the env section of your app.json to use a different major (e.g. "4" or "5") or exact (e.g. "6.0.16") version. This feature is experimental and subject to change.

Releasing a new version

Make sure you publish this buildpack in the buildpack registry

heroku buildpacks:publish heroku/ci-redis master

heroku-buildpack-ci-redis's Issues

Username "h" in connection string results in WRONGPASS errors for Redis 6+

In the past, Heroku was adding the h username on Redis connection strings (see: https://devcenter.heroku.com/changelog-items/1932). With the release of Redis 6 and ACLs (https://redis.io/docs/manual/security/acl/), Heroku stopped adding the h username for Redis 6 instances (and new Redis add-ons using older Redis versions).

However, this buildpack is still hardcoding the h username: https://github.com/heroku/heroku-buildpack-ci-redis/blob/master/bin/compile#L68 in the connection string config var.

This is resulting in WRONGPASS invalid username-password pair or user is disabled. errors with Redis 6+ if the Redis clients are trying to authenticate using the h username and the given password.

With Redis 6+, using requirepass will configure that pass for the default user. The h user wouldn't exist, neither it's configured with that password, so connections using username=h, password=xxxx would fail. (Although one could use username=default, password=xxx and that'd work). We run requirepass here: https://github.com/heroku/heroku-buildpack-ci-redis/blob/master/bin/compile#L71.

We've seen reports of this issue after 6.2 was made the default version as part of #29. I'm admittedly confused as of why we didn't get reports of the same problem before, when 6.0.9 was the default version, which should have worked similarly in terms of users/ACLs.

As a result, this is resulting in test failures in Heroku CI when the Redis client tries to connect to the locally running instance and authenticate as the h user. We've seen this happen with redis-py (3.5.3, and also tested with 4.3.1 through a console directly - outside of Heroku CI).

Some other clients like redis-rb are handling this scenario specifically but raising a warning:

[redis-rb] The Redis connection was configured with username "h", but the provided password was for the default user. This will start failing in redis-rb 5.0.0.

(https://github.com/redis/redis-rb/blob/4724c1bfde2d5679128ced8ec7b46bac66605b7b/lib/redis/client.rb#L124-L134)

To sort this out, the REDIS_URL config var shouldn't include the h username at least if the version is 6 or higher.

heroku-buildpack-ci-redis does not invalidate cache when switching stack

The cached Redis build is used even if $STACK has changed.

To fix this, the cached build directory could be made to include not only the Redis version string, but also the stack.

However the buildpack should also probably clean up old build directories (currently no cleanup is performed at all, so the cache will gradually become bloated even for version changes). To do that it's likely best to pick a directory namespace under the cache dir (eg .indyno/redis or ci-redis/) that way cleanup could remove that directory entirely rather than need to use globbing.

Tracked internally at W-7501650.

App Shows: `WRONGPASS invalid username-password pair or user is disabled` with `Redis 7`

Not sure how to fix it. I am using the following lines of code to connect to the Redis instance. Didn't work.
https://github.com/touhidurrr/UncivServer.xyz/blob/a933ae223f8c8e4949600535904861ea3fb458f7/src/plugins/Redis.ts#L2-L6

[Note: It is working on another hosting, but not on Heroku.]

Remove testing against Heroku-16 in CI

The Heroku-16 stack reached end-of-life on May 1st, 2021, and in a few days (June 1st, 2021) the Heroku-16 build system will be decommissioned, meaning no further builds will be possible:
https://help.heroku.com/0S5P41DC/heroku-16-end-of-life-faq

As such, testing in CI against Heroku-16 can be removed:

heroku-buildpack-ci-redis/.circleci/config.yml

Lines 28 to 30 in b75acc4

	matrix:
	parameters:
	stack: [heroku-16, heroku-18, heroku-20]

Does not export REDIS_URL

The buildpack starts Redis with a predefined, hard-coded configuration. It currently requires buildpack consumers to also make those assumptions. For flexibility and consistency with the Postgres CI buildpack, this buildpack should export a REDIS_URL.