New releases on hex.pm

[tsloughter]

New releases on hex.pm

[lpgauth]

shackle 0.4.2 is now on hex.pm: https://hex.pm/packages/shackle

@weixiyen I'll publish marina on hex.pm next week.

[tokenrove]

A few of the problems I had: (which can at least be attributed to my own ignorance or carelessness, but might be nice to protect against)

Since priv/ is in DEFAULT_FILES, the compiled .so was included on my first attempt. I guess that rebar3 hex doesn't require you to work from a clean checkout because you might want to publish generated files like parsers, but for most purposes this is surprising. My assumption would have been to only use a clean worktree or obey .gitignore, and for the case of including generated outputs, an explicit generated_files key or similar.

Attempts to submit packages from temporary branches (instead of having to tag and retag for experiments) resulted in unclear 400 errors, perhaps because the generated (hairy) version number wasn't valid. I meant to submit a bug about this to clarify the error. Better still would be documenting a workflow for testing a package before submitting it.

From the documentation on hex.pm, it's not obvious how files works. Even after the first couple of surprising uses, my eyeballing of the included files listed by publish failed to notice one or another missing files. At the very least, the documentation should be explicit about what files you do and do not want to include in a package, because it wasn't clear to me (perhaps because I don't currently use hex.pm packages myself).

A documented workflow for locally testing a packaging before publishing it would have at least allowed me to make all these mistakes privately. Even better might be some Travis CI magic I can include in my own travis.yml that builds the package locally so I don't lose when some future PR adds a new file. But as I mentioned, best of all would be the OPAM-style workflow where a bunch of automated testing happens with each package submission so I don't have to worry about publishing broken packages.

In general all the documentation could use work (how do I discover that magic revert option to publish, and actually make it work?), but I get that that's an ongoing concern and that ideally I'd submit patches for that. Hopefully these suggestions help anyway.

Thanks.

[tsloughter]

@tokenrove thank you, this is really great. I hope I can address these issues soon. These will certainly be issues for everyone publishing c_src. My original design goal of rebar3 was that all generated code would be in _build, this ran into issues with c code since builds to priv which we want to symlink to from _build. I doubt I'll be able to solve that core issue anytime soon, but providing better documentation around a hex package workflow and docs on files I can and will soon.

@lpgauth any update on publishing the latest shackles? :)

Also, thanks for shackle! I've been using for https://github.com/gofactory/vonnegut

[lpgauth]

I would love to, but it depends on a NIF (granderl) which is not on hex.pm... And last time we tried pushing a NIF on hex.pm it didn't work. I'll try again and fill issues this time.

[tsloughter]

Oh, didn't realize it used a NIF now. However, NIFs like it should publish to hex.pm the same as any other project.

Also, looks like granderl is on hex, just not the latest version. If @tokenrove is having issues publishing the latest I am happy to help :)

[lpgauth]

@tsloughter I'll have to check with @tokenrove, but if I remember correctly the hex.pm tool was pushing the .so files...

[tokenrove]

There were several problems around what files were or were not included in the package. I need to get back to it and do a proper attempt again for granderl, but I really suggest having a facility like opam has where every package has to pass a bunch of CI tests before it gets accepted.

This would have prevented me from pushing the (currently broken) granderl package and not being able to remove it, thus leading to me throwing my hands up in despair and walking away.

[tokenrove]

This should do it: https://hex.pm/packages/granderl/0.1.5

(Only about three failed publishing attempts and tag rewrites this time.)

[tsloughter]

Great! Sorry for the pain with rebar3's hex publishing. Support for testing the package would definitely be a great feature to have, I'll have to think about that.

Can you give more detail on what issues you had? Is there something I can do besides the test framework to improve the experience? Like do you need an "ignore files" config option?

[weixiyen]

seconded, would love to see shackle (and also marina) get updated with granderl 0.1.5.

[lpgauth]

Once granderl is there I'll publish...

./bin/rebar3 hex publish
===> Verifying dependencies...
===> Can not publish package because the following deps are not available in hex: granderl

[tokenrove]

You'll need to point rebar.config at granderl 0.1.5. 0.1.4 isn't on hex.pm. (Actually, now that I think of it, I have no idea if it's the tag in rebar.config, since that probably doesn't matter. But granderl 0.1.5 does seem to be in hex.)

[lpgauth]

Ok, I updated granderl to 0.1.5 but I'm still getting the same error message.

[tokenrove]

Pointing rebar.config at the package seems to work:

diff --git a/rebar.config b/rebar.config
index 50a563c..2b18386 100644
--- a/rebar.config
+++ b/rebar.config
@@ -15,8 +15,7 @@
 {coveralls_service_name, "travis-ci"}.
 
 {deps, [
-  {granderl, ".*",
-    {git, "https://github.com/tokenrove/granderl.git", {tag, "v0.1.5"}}}
+  {granderl, "0.1.5"}
 ]}.
 
 {edoc_opts, [
diff --git a/rebar.lock b/rebar.lock
index be2fb24..9643ec5 100644
--- a/rebar.lock
+++ b/rebar.lock
@@ -1,4 +1,6 @@
-[{<<"granderl">>,
-  {git,"https://github.com/tokenrove/granderl.git",
-       {ref,"cbc9f0cc66e2da706fff37bfd0000f3f3f2bab80"}},
-  0}].
+{"1.1.0",
+[{<<"granderl">>,{pkg,<<"granderl">>,<<"0.1.5">>},0}]}.
+[
+{pkg_hash,[
+ {<<"granderl">>, <<"F20077A68BD80B8D8783BD15A052813C6483771DEC1A5B837D307CBE92F14122">>}]}
+].

[lpgauth]

But then I loose rebar2 compatibility... @tsloughter is there any way to support rebar2, rebar3 and hex.pm?

[lpgauth]

Ok, I found a way 6a6c198... not sexy but I guess that's life 😞

[tokenrove]

Ugh, you're right.

The following seems to "work" (with rebar2 and 3, and hex publish doesn't complain) but seems like a hack:

  {granderl, ".*",
    {git, "https://github.com/tokenrove/granderl.git", {tag, "v0.1.5"}},
    [{pkg, granderl}]
  }

[tokenrove]

Oh, never mind. It causes rebar3 upgrade to barf, as one might expect.

[tsloughter]

To support rebar2 you have to use a rebar.config.script to replace the deps with git deps if it is rebar2 running it. Since rebar2 doesn't support hex packages at all there is no other way.

[tsloughter]

Another alternative is to only replace deps with hex deps to publish the package and not commit that change. But probably best to go with the script if you really want to keep rebar2 support for the foreseeable future.

[tsloughter]

@lpgauth the package is broken. It thinks there are 0 dependencies, so it misses granderl and fails during runtime.

Be sure you are using the latest rebar3 and hex plugin (rebar3 as global plugins upgrade rebar3_hex) and check when you run rebar3 hex publish that the deps listed before it asks if it should continue Y/n are correct.

[lpgauth]

@weixiyen FYI, lpgauth/marina#29

[expelledboy]

Thanks, abit neater perhaps: expelledboy/miffy@fea430c