nerves-project/nerves

Craft and deploy bulletproof embedded software in Elixir

nerves's Issues

mix nerves.new inside umbrellas do not generate umbrella options

Since Elixir v1.2, Elixir generates the following configuration for apps inside umbrellas:

     build_path: "../../_build",
     config_path: "../../config/config.exs",
     deps_path: "../../deps",
     lockfile: "../../mix.lock",

While nerves generates:

    deps_path: "deps/#{@target}",
    build_path: "_build/#{@target}",

It is understandable that Nerves does not want to share the deps and build paths with the umbrella due to target configurations (and that's fine IMO) but I would recommend sharing the config and lockfile paths to avoid configuration and dependency conflicts respectivelly.

As an example, here is how the check is implemented in Phoenix:

• https://github.com/phoenixframework/phoenix/blob/master/installer/lib/phoenix_new.ex#L484-L495
• https://github.com/phoenixframework/phoenix/blob/master/installer/templates/new/mix.exs#L6-L10

Support for Erlang/OTP 20

When building firmware on a host that has Erlang/OTP 20 installed, the following error message appears:

** (Mix) Major version mismatch between host and target Erlang/OTP versions
  Host version: 20
  Target version: 19
This will likely cause Erlang code compiled for the target to fail in
unexpected ways. Install an Erlang OTP release that matches the target
version before continuing.

This is because Nerves requires that your host OTP version (running on the machine that's building the firmware) match the version that will run on the target (embedded system). Since the currently-released versions of the supported Nerves Systems are using OTP 19, you need to use OTP 19 to build your firmware.

We plan to support OTP 20 in upcoming releases of the supported Systems, but the constraint of matching your host OTP version will continue to exist for the foreseeable future. So at that point, OTP 19 will no longer be supported on the host for System versions beyond that point.

To make this process easier to manage and avoid having to uninstall/reinstall Erlang system-wide, we recommend using a version-management tool like asdf to easily switch between versions of your build tools, as described here: http://embedded-elixir.com/post/2017-05-23-using-asdf-vm/.

Provide progress when downloading systems and toolchains

On slow networks, the downloads complete, but they take long enough that some progress would be really nice. Here's an example:

==> nerves_system_rpi
Generated nerves_system_rpi app
[nerves_system][compile]
[nerves_system][http] Downloading System

Don't build Nerves systems when artifact downloads fail

In training meetings, it's not uncommon for someone to have the artifact download fail when building for the first time. It's probably due to us overloading the network. This triggers a Docker build of the system. Since they've never used Nerves before, they don't know that this isn't part of the normal process. When it works, it takes a long time, but usually something isn't installed for the Docker build and the error messages get to be pretty confusing.

I'd really like it if we could download failures error out and let the user fix their network, wait, etc, and try again.

What does everyone think of marking official system releases as "never build"? Obviously, it would be good to have a way to turn this off, but it seems like would be a separate step.

Or, how about making system building "opt-in"? I.e., systems never get locally built until the user enables it in their config.exs. I sort of like this since it seems like we could make the artifact download failure error message say something really helpful and instruct them that building locally is an option. When it does this, the "opt-in" part is the user acknowledging that a really big operation that involves Docket, etc. is ok with them. That way there won't be any surprises.

Format `/root` partition as EXT4

As discussed in #30, we would prefer for the /root partition to be EXT4, which will require coordinated changes in several places, including "reformat-on-corruption-or-first-use" in nerves_runtime and some changes to fwup.

rootfs_additions -> rootfs_overlay

The rootfs_additions config variable and directory name need to be updated to rootfs_overlay now that we're following the Buildroot naming convention. A git grep rootfs_additions looks like it finds everything that needs to change. The only trick is to make sure that the rootfs_additions key still works so as to not break existing projects.

Add a Mix task to return the full path to the firmware file

The goal is to eliminate the need for things like this: https://github.com/fhunleth/nerves_firmware_ssh/blob/master/upload.sh#L25

Possible solutions include:

• New flags to mix firmware (e.g. mix firmware --quiet --path) that would ensure the firmware file exists and return its full path
• New Mix task (either in bootstrap or nerves) that would simply return where the firmware should be, but not actually build it (e.g. mix nerves.info --firmware-path)

[documentation] 'targets' link is broken

The getting started guide has several links to
https://github.com/nerves-project/nerves/blob/master/docs/targets.html#target-dependencies

which appear to be broken - at least they are when viewing on Github. Maybe when they are published they are correct?

Support for Elixir 1.3.0-dev

I tried building nerves-examples/blinky -- and bootstrapping a new project -- with Elixir 1.3.0-dev (HEAD).

When building blinky, I got the following error (trying both rpi2 and rpi3):

DEBUG=1 mix firmware
warning: the archive nerves_bootstrap requires Elixir "~> 1.2.4 or ~> 1.3" but you are running on v1.3.0-dev
Nerves Precompile Start
Nerves Env current
Nerves Env loaded
Nerves Precompile End
===> Expanded command sequence to be run: []
===> Expanded command sequence to be run: [{default,app_discovery},
                                                  {bare,compile}]
===> Evaluating config script "/Users/bruce/Development/nerves-examples/blinky/deps/rpi3/erlware_commons/rebar.config.script"
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace
===> Uncaught error: {badmatch,
                             {error,
                              {19,file,
                               {error,
                                {badmatch,
                                 ["3","0","0+build","272","ref7d29b74"]},
                                [{erl_eval,expr,3,[]}]}}}}
===> Stack trace to the error location: [{rebar_config,consult_file_,
                                                 1,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_config.erl"},
                                                  {line,106}]},
                                                {rebar_config,consult_file,1,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_config.erl"},
                                                  {line,90}]},
                                                {rebar_app_discover,
                                                 project_app_config,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_app_discover.erl"},
                                                  {line,127}]},
                                                {rebar_app_discover,
                                                 merge_deps,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_app_discover.erl"},
                                                  {line,81}]},
                                                {rebar_app_discover,
                                                 '-do/2-fun-1-',5,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_app_discover.erl"},
                                                  {line,46}]},
                                                {lists,foldl,3,
                                                 [{file,"lists.erl"},
                                                  {line,1262}]},
                                                {rebar_prv_app_discovery,do,
                                                 1,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_prv_app_discovery.erl"},
                                                  {line,38}]},
                                                {rebar_core,do,2,
                                                 [{file,
                                                   "/home/travis/build/rebar/rebar3/_build/default/lib/rebar/src/rebar_core.erl"},
                                                  {line,125}]}]

When attempting to bootstrap a new project with mix nerves.new nerves_test --target rpi3, I see:

warning: the archive nerves_bootstrap requires Elixir "~> 1.2.4 or ~> 1.3" but you are running on v1.3.0-dev
** (Mix) Nerves v0.1.2 requires at least Elixir v1.2.4.
 You have 1.3.0-dev. Please update accordingly

I'd say the ~> 1.3 requirement was just too strict, but the error in building blinky (fixed by using 1.2.5) seems to point out a deeper issue supporting 1.3.0-dev.

Can't install dependencies.

Environment

• Erlang version (erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' -noshell):
• Elixir version (elixir -v): 1.4.0
• Operating system: Ubuntu 16.04
• Nerves Environment Info:
  MIX_TARGET: rpi
  MIX_ENV: dev

Current behavior

When I want to install my dependencies this happens:

$ mix deps.get
Env
  MIX_TARGET:   rpi
  MIX_ENV:      dev

Failed to fetch record for 'nerves_runtime' from registry (using cache)
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>0CD6022F4C66038E</RequestId><HostId>H50ep+f7DZmj3WdJf4RGN35emOP2QPod+/CiJB46usSo1TW/u6je8ITTfp82f5Gx5SiqFFPbzuw=</HostId></Error>
** (Mix) No package with name nerves_runtime (from: mix.exs) in registry

Expected behavior

I expected to have my dependencies installed.

rootfs additions in deps?

It would be handy when deps can specify rootfs additions as well. Currently only the toplevel project can do this, but libraries may want to add things to the root filesystem as well, e.g. libraries (in /lib), executables or config files.

How this would be configured is not really clear to me yet, maybe every application can expose a :nerves_rootfs_additions config key; the mix task would then need to scan all application configuration for this key:

config :my_library, :nerves_rootfs_additions, "my-rootfs-extra/"

Which the mix firmware task would then translate into multiple -a <path> arguments to the rel2fw.sh script.

cc @fhunleth

Support NERVES_COMPILER_CACHE=local

Is there perhaps an undocumented equivalent of NERVES_SYSTEM_CACHE=local for the compiler toolchain (nerves_toolchain) download?

Right now, if I do the following:

export NERVES_SYSTEM_CACHE=local
NERVES_TARGET=qemu_arm mix do deps.get, deps.precompile
NERVES_TARGET=rpi mix do deps.get, deps.precompile
NERVES_TARGET=rpi3 mix do deps.get, deps.precompile

...the actual systems get cached fine, but the toolchain always gets re-downloaded for each target in each Nerves project:

==> nerves_toolchain
Compiling 2 files (.ex)
Generated nerves_toolchain app
==> nerves_toolchain_arm_unknown_linux_gnueabihf
Compiling 1 file (.ex)
Generated nerves_toolchain_arm_unknown_linux_gnueabihf app
[nerves_toolchain][compile]
[nerves_toolchain][http] Downloading Toolchain
[nerves_toolchain][http] Toolchain Downloaded
[nerves_toolchain][http] Unpacking Toolchain
==> nerves_system_rpi3
Generated nerves_system_rpi3 app
[nerves_system][compile]
[nerves_system][local] Checking Cache for nerves_system_rpi3-0.6.0
nerves_system_rpi3-0.6.0 was not found in your cache.
cache dir: /Users/arto/.nerves/cache/system
Would you like to download the system to your cache? [Yn] 
[nerves_system][http] Downloading System
[nerves_system][http] System Downloaded
[nerves_system][http] Unpacking System

I'd like to be able to work on new Nerves projects offline and/or on data-limited mobile connectivity. It seems like caching everything, one way or another, would be a good idea to enable such a work modality.

Timestamps on files burned to SD card are off by 1 month

I noticed that after mix firmware.burn, the timestamps on the files on the SD card are showing in Finder as April 30, 2016 at 12:59 AM when it was burned on May 29, 2016 at 8:59 PM Eastern.

The day and the time (in UTC) are correct, but the month is off by one.

The timestamp on the .fw file under _images is correct:

$ ls -l _images/rpi2/
-rw-r--r--  1 wsmoak  staff  19045757 May 29 20:59 delete_me7.fw

Elixir 1.2.5
Nerves 0.3.2
fwup 0.6.0 (apparently there is an 0.6.1 but I haven't been able to get it from homebrew)

Dockerfile fails running locale-gen stage: /bin/sh: 1: locale-gen: not found

The Dockerfile template fails to build:

erik@sparrow:nerves_system_rpi0 (master=) $ mix 
==> distillery
Compiling 18 files (.ex)
Generated distillery app
==> nerves
Compiling 25 files (.ex)
Generated nerves app
==> nerves_system_br
Generated nerves_system_br app
==> nerves_toolchain_ctng
Compiling 2 files (.ex)
Generated nerves_toolchain_ctng app
==> nerves_toolchain_armv6_rpi_linux_gnueabi
Compiling 1 file (.ex)
Generated nerves_toolchain_armv6_rpi_linux_gnueabi app
==> nerves_system_rpi0
Generated nerves_system_rpi0 app
|Elixir.Nerves.Package.Providers.HTTP| Downloading Artifact nerves_system_rpi0-0.12.0-dev.armv6_rpi_linux_musl.tar.gz
  From Location:
    https://github.com/tmecklem/nerves_system_rpi0/releases/download/v0.12.0-dev/nerves_system_rpi0-v0.12.0-dev.tar.gz

|Elixir.Nerves.Package.Providers.HTTP| No Available Locations

|Elixir.Nerves.Package.Providers.Docker| Create Image

The Nerves Docker provider needs to create the image.
Proceed?  [Yn] 
Sending build context to Docker daemon 3.584 kB
Step 1/10 : FROM ubuntu:xenial
 ---> 6a2f32de169d
Step 2/10 : MAINTAINER Justin Schneck
 ---> Using cache
 ---> 9d5dadcbd004
Step 3/10 : ENV NERVES_BR_DL_DIR /nerves/cache/buildroot
 ---> Using cache
 ---> fe23d0f005e0
Step 4/10 : ENV LANG en_US.UTF-8
 ---> Using cache
 ---> 2795082da495
Step 5/10 : ENV LANGUAGE en_US:en
 ---> Using cache
 ---> f95777e4edb0
Step 6/10 : ENV LC_ALL en_US.UTF-8
 ---> Using cache
 ---> b86b9624a355
Step 7/10 : RUN locale-gen en_US.UTF-8
 ---> Running in 5c45f6ee4b58
/bin/sh: 1: locale-gen: not found
The command '/bin/sh -c locale-gen en_US.UTF-8' returned a non-zero code: 127
** (Mix) Nerves Docker provider could not create docker volume nerves_cache

locale-gen is no longer part of the ubuntu:xenial image. Commenting out Dockerfile line 16 bypasses the issue and the build continues as normal.

Info:
Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Elixir 1.4.2
macOS
docker Version 17.03.1-ce-mac5 (16048)
rpi* targets

Documentation: Getting Started section mix firmware generates error message about missing release config file

Environment

• Erlang version: 19
• Elixir version (elixir -v): 1.3.4
• Operating system: Ubuntu Linux 15.10 x64 with Kernel 4.4.8-040408-generic
• Nerves Environment Info:

$ mix nerves.env --info
==> distillery
Compiling 18 files (.ex)
Generated distillery app
==> nerves
Compiling 25 files (.ex)
Generated nerves app
==> nerves_system_br
Generated nerves_system_br app
==> nerves_toolchain_ctng
Compiling 2 files (.ex)
Generated nerves_toolchain_ctng app
==> nerves_toolchain_arm_unknown_linux_gnueabihf
Compiling 1 file (.ex)
Generated nerves_toolchain_arm_unknown_linux_gnueabihf app
==> nerves_system_rpi2
Generated nerves_system_rpi2 app
==> hello_nerves
---------------------------
Nerves Environment Packages
---------------------------

Pkg:      nerves_system_br
Vsn:      0.9.4
Type:     system_platform
Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Local, []}]

Pkg:      nerves_system_rpi2
Vsn:      0.10.0
Type:     system
Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Local, []}]

Pkg:      nerves_toolchain_arm_unknown_linux_gnueabihf
Vsn:      0.9.0
Type:     toolchain
Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.HTTP, []}]

Pkg:      nerves_toolchain_ctng
Vsn:      0.8.0
Type:     toolchain_platform
Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Local, []}]

Nerves Loadpaths Start
Nerves Env Start
Nerves Env End
----------------------------
Nerves Environment Variables
----------------------------
target:     rpi2
toolchain:  /home/inactivist/.nerves/artifacts/nerves_toolchain_arm_unknown_linux_gnueabihf-0.9.0.linux-x86_64
system:     /home/inactivist/.nerves/artifacts/nerves_system_rpi2-0.10.0.arm_unknown_linux_gnueabihf
app:        /home/inactivist/m/devel/mine/nerves-tests/hello_nerves

Nerves Loadpaths End

Current behavior

Following Getting Started instructions should work as described with no error messages.

Expected behavior

mix firmware generates the following error message:

$ mix firmware
** (Mix)   You are missing a release config file. Run  nerves.release.init task first

fwup version check goes wrong with elixir 1.4.4

fwup requirement check in nerves/lib/mix/nerves/utils.ex goes wrong with elixir 1.4.4. Version.requirements() function does not accept version 0.8 as a parameter. Please change it to 0.8.0. And the operator should be >= instead of ~> according to nerves description.

In nerves/lib/mix/nerves/utils.ex

Environment

• Erlang version 19.3
• Elixir version 1.4.4
• Operating system: Linux Mint 18.1

Support `mix firmware.burn` with WSL

On Slack, @jmerryweather reports to that mix firmware.burn can be made to work in the Windows Subsystem for Linux by dropping the call to sudo. This also requires that users run Bash on Ubuntu as an Administrator.

Jonathan has a patch is progress to make this work. Also, see fwup-home/fwup#67.

This issue is intended to track progress on WSL support and any necessary documentation.

Failing to download artifact from URL results in CHECKSUM Error

0.4.0-rc.0

Current behavior

[Nerves.Package.Providers.HTTP]   Downloading Artifact:
      nerves_toolchain_aarch64_unknown_linux_gnueabi-0.8.0-rc.0.darwin-x86_64.tar.xz
  Location
      https://github.com/nerves-project/toolchains/releases/download/v0.8.0-rc.0/nerves_toolchain_aarch64_unknown_linux_gnueabi-0.8.0-rc.0.darwin-x86_64.tar.xz

[Nerves.Package.Providers.HTTP] No Available Locations
** (File.Error) could not write to file "/Users/jschneck/Developer/nerves/toolchains/nerves_toolchain_aarch64_unknown_linux_gnueabi/.nerves/artifacts/nerves_toolchain_aarch64_unknown_linux_gnueabi-0.8.0-rc.0.darwin-x86_64/CHECKSUM": no such file or directory
    (elixir) lib/file.ex:716: File.write!/3
    lib/mix/tasks/compile.nerves_package.ex:23: Mix.Tasks.Compile.NervesPackage.run/1
    (mix) lib/mix/task.ex:294: Mix.Task.run_task/3
    (elixir) lib/enum.ex:1231: Enum."-map/2-lists^map/1-0-"/2
    (elixir) lib/enum.ex:1231: Enum."-map/2-lists^map/1-0-"/2
    (mix) lib/mix/tasks/compile.all.ex:19: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:37: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:294: Mix.Task.run_task/3

Expected behavior

The error should be handled and stop before trying to initiate the CHECKSUM

limited busybox

Doing:

iex(32)> System.cmd("/bin/busybox", ["--list"])
{"ash\ncat\ncp\ndate\ndmesg\nfatattr\nhalt\ni2cdetect\ni2cdump\ni2cget\ni2cset\nid\nip\nipaddr\niplink\niproute\niprule\niptunnel\nkill\nkillall\nls\nlsmod\nmodprobe\nmount\nntpd\npoweroff\npwd\nreboot\nrm\nrmmod\nsh\nsha256sum\nudhcpc\nuevent\numount\nunzip\n", 0}

A much larger list of busybox commands are here.
How would I get these missing commands: ps, df, du, find, free, grep ?
I assume nerves is being minimalistic/secure, so this is not critical, but it's always nice to poke around, and knowing how to add/change the default busybox would be nice.
. . .

referred here from nerves_system_qemu_arm:
bendiken commented 9 minutes ago
@cleesmith As this isn't QEMU-specific, this would probably be better directed at https://github.com/nerves-project/nerves or on the #nerves Slack channel.

Create API for garbage collecting global artifacts

Keeping the system clean is important. Especially when the artifacts stored globally could easily become version bumped and then just sit there taking up a lot of space.

We should expose some mix commands that let you list / clean the artifacts that are in the global cache. To assist in this, every time an artifact is used in a project build, we should bump the timestamp on a hidden file in the artifact dir. This will let use display some useful information about when the artifact was last used and possibly even which project used it last.

Discuss erlang.cookie error when trying to write to read only FS

On the FAQ's we should add a blurb discussing what the obscure error received when Erlang tries to write an erlang.cookie file on the RO FS.

How to reproduce
unset -name and -sname and -setcookie in vm.args
Then, boot into the vm and try to start distributed erlang.

:os.cmd 'epmd -daemon'
:net_kernel.start([:"[email protected]"])

Solution
Ensure that a cookie is set in the vm.args even if the vm is starting in no distribution.

mix.firmware fails when there is a '+' sign in directory name

Environment

• Erlang version (erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' -noshell): 19
• Elixir version (elixir -v): 1.4.2
• Operating system: macOS 10.12.4
• Nerves Environment Info:

|nerves_bootstrap| Environment Variable List
  target:     rpi3
  toolchain:  /Users/maxmouchet/.nerves/artifacts/nerves_toolchain_arm_unknown_linux_gnueabihf-0.10.0.darwin-x86_64
  system:     /Users/maxmouchet/.nerves/artifacts/nerves_system_rpi3-0.11.0.arm_unknown_linux_gnueabihf
  app:        /private/tmp/test +/hello_nerves

Current behavior

/tmp/test +/hello_nerves % pwd
/tmp/test +/hello_nerves

/tmp/test +/hello_nerves % mix firmware
Env
  MIX_TARGET:   rpi3
  MIX_ENV:      dev

|nerves_bootstrap| Building OTP Release...

==> Cleaning all releases..
==> Clean successful!
usage: dirname path
usage: mkdir [-pv] [-m mode] directory ...
** (Mix) Nerves encountered an error. %IO.Stream{device: :standard_io, line_or_bytes: :line, raw: true}

Expected behavior

Should build firmware successfully. It works when there is no '+' sign in the directory name.

Question: how to autorun something

hi ,

is there an official way to auto run something when system boot.
run a elixir program or a script file.

i checked the online document, but i did't find a section say this.

thanks!

NERVES_SYSTEM isn't documented

I noticed that neither NERVES_SYSTEM nor NERVES_TOOLCHAIN were documented or maybe I couldn't find it? This is the main way that I use Nerves, so I was wondering if these should be documented.

Checking host requirements before compiling a package

Environment

OTP 19, Elixir 1.3.4
nerves 0.4.0-rc.0

Current behavior

Unable to build custom system which requires the presence of additional host packages installed.

Expected behavior

Add to the Nerves package config a key for required host tools and take the following action:
Providers:
local: Check for the presence in a preflight and recommend they install the missing packages
docker: Construct args to pass to image to make sure container has additional host packages.

mix firmware.burn broken on Linux

$ mix firmware.burn

Nerves Env loaded
Nerves Firmware Burn
usage: sudo -h | -K | -k | -V
usage: sudo -v [-AknS] [-g group] [-h host] [-p prompt] [-u user]
usage: sudo -l [-AknS] [-g group] [-h host] [-p prompt] [-U user] [-u user]
           [command]
usage: sudo [-AbEHknPS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p
           prompt] [-u user] [VAR=value] [-i|-s] [<command>]
usage: sudo -e [-AknS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p
           prompt] [-u user] file ...

Stock mix nerves.new task presents warning about applications/included_applications

Every nerves newbie has to get over the following error message when building firmware, which appears to indicate something is wrong and needs to be fixed.

I'm not sure the proper fix to this. The builds work regardless, but seems like there should be a way to not get this warning.

targhee:testpi3 garth$ mix firmware
[Nerves] Building OTP Release...
==> Building release testpi3:0.1.0 using environment dev
==> One or more direct or transitive dependencies are missing from
    :applications or :included_applications, they will not be included
    in the release:

    :nerves
    :nerves_system_rpi3

    This can cause your application to fail at runtime. If you are sure
    that this is not an issue, you may ignore this warning.

Low timeout for Nerves.Package.Providers.HTTP

Environment

• Erlang version: 19
• Elixir version: 1.4.0
• Operating system: macOS 10.12.3

Current behaviour

I'm working with not so fast internet connection (~5mbit/s) and cannot install nerves_system_rpi. It always fails after 120 seconds.

mix deps.compile nerves_system_rpi
==> nerves_system_rpi
Generated nerves_system_rpi app
[Nerves.Package.Providers.HTTP]   Downloading Artifact:
  nerves_system_rpi-0.9.1.armv6_rpi_linux_musl.tar.gz
  From Location:
  https://github.com/nerves-project/nerves_system_rpi/releases/download/v0.9.1/nerves_system_rpi-v0.9.1.tar.gz

|=======================                           | 46% (47 / 102) MBcould not compile dependency :nerves_system_rpi, "mix compile" failed. You can recompile this dependency with "mix deps.compile nerves_system_rpi", update it with "mix deps.update nerves_system_rpi" or clean it with "mix deps.clean nerves_system_rpi"
** (exit) exited in: GenServer.call(#PID<0.224.0>, {:get, "https://github.com/nerves-project/nerves_system_rpi/releases/download/v0.9.1/nerves_system_rpi-v0.9.1.tar.gz"}, 120000)
    ** (EXIT) time out
    (elixir) lib/gen_server.ex:737: GenServer.call/3
    (nerves) lib/nerves/package/providers/http.ex:50: Nerves.Package.Providers.HTTP.download/2
    (nerves) lib/nerves/package/providers/http.ex:20: Nerves.Package.Providers.HTTP.artifact/3
    (elixir) lib/enum.ex:1755: Enum."-reduce/3-lists^foldl/2-0-"/3
    (nerves) lib/nerves/package.ex:99: Nerves.Package.artifact/2
    (mix) lib/mix/task.ex:294: Mix.Task.run_task/3
    (elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2
    (elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2

Expected behavior

mix deps.compile nerves_system_rpi will compile with any internet connection speed

Replace automatic nerves_system_ dep naming with multiple function heads

The currently generated mix.exs pulls in the system dependency as follows:

{:"nerves_system_#{target}", "~> 0.10", runtime: false},

This has a few problems:

1. The version numbers have to be roughly the same between systems for it to work.
2. It obfuscates which system is used to new users
3. It couples MIX_TARGET to the system. (e.g., if this were the intent, then why not specify the MIX_TARGET=nerves_system_rpi3, etc.)

It seems like an improvement to have nerves.new generate a template with an explicit system dep in the mix.exs. This would encourage users to create a separate dep function for each target. Some of us are doing this already due to the version issue, but I also think that it would improve understanding by decoupling MIX_TARGET more from the system image and having an obvious system image reference to see.

Thoughts?

Minor change to custom system triggers full rebuild

Environment

• Brand new nerves and elixir 1.4 installation
• Updated (rolling release) Arch Linux

Current behavior

I have a custom nerves system (modified nerves_system_rpi3). A minor change (adding a file to rootfs-additions) to the system and running mix firmware from my dependent app (nerves-examples/hello_phoenix with just system path changed as specified in the docs) triggers a full rebuild of buildroot, Linux and all packages.

Expected behavior

Is this expected behaviour? My memory of using buildroot directly is that it's smarter than that.
__

Add update mechanism to nerves_bootstrap

Nerves Bootstrap needs to be able to inform the user that there is an update available for their version. we should create a mix task which installs the latest bootstrap version.

Document ssh-askpass dependency on Linux

On @DanKomorny's fresh Lubuntu 16.04 installation:

$ NERVES_TARGET=rpi mix firmware.burn
Use 7.4 GiB memory card found at /dev/sdb? [Yn] y
sudo: unable to run /usr/bin/ssh-askpass: No such file or directory

The relevant bit of code is at: lib/mix/tasks/firmware.burn.ex

Resolved by sudo apt-get install ssh-askpass.

The Nerves installation guide should probably document this requirement on Linux?

Application partition documentation is out of date

The current documentation talks about the FAT filesystem for persisting data. It's now EXT4. More information is at https://github.com/nerves-project/nerves_runtime#filesystem-initialization. Some of that could be reused or modified to make it more accessible to a new Nerves user.

"MIX_TARGET=rpi3 mix" should give a nicer error message

Currently, you get a wall of error messages from the system trying to load cross-compiled binaries. It seems like this should be intercepted to error out with something like:

"Cannot run build intended for rpi3 on the host. Try running 'mix compile' or 'mix firmware'"

Or something similar. It seems like something that people will accidentally type before they come to appreciate the cross-compiled nature of Nerves. Or, if they're like me, and are running on Linux with binfmt magic automatically redirecting foreign architecture binaries to qemu, they'll get something even more surprising.

Invalid generated docker id

Environment

• Erlang version (erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' -noshell): 19
• Elixir version (elixir -v): 1.4
• Operating system: MacOs Darwin Kernel Version 15.6.0
• Nerves Environment Info:
  (mix nerves.env --info) or
  General info about target / system / toolchain

|nerves_bootstrap| Environment Package List

  Pkg:      nerves_system_br
  Vsn:      0.10.0
  Type:     system_platform
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Docker, []}]

  Pkg:      nerves_toolchain_armv6_rpi_linux_gnueabi
  Vsn:      0.10.0
  Type:     toolchain
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.HTTP, []}]

  Pkg:      nerves_toolchain_ctng
  Vsn:      0.9.0
  Type:     toolchain_platform
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Docker, []}]

|nerves_bootstrap| Loadpaths Start

|nerves_bootstrap| Env Start

|nerves_bootstrap| Env End

NERVES_SYSTEM is unset

Current behavior

I forked the rpi0 nerves system from Tim (tmecklem/nerves_system_rpi0), cloned it and did a mix deps.get. The following mix job failed:

 mix
|Elixir.Nerves.Package.Providers.HTTP| Downloading Artifact nerves_system_rpi0-0.12.0-dev.armv6_rpi_linux_musl.tar.gz
  From Location:
    https://github.com/tmecklem/nerves_system_rpi0/releases/download/v0.12.0-dev/nerves_system_rpi0-v0.12.0-dev.tar.gz

|Elixir.Nerves.Package.Providers.HTTP| No Available Locations

|Elixir.Nerves.Package.Providers.Docker| Creating Docker container -KbCw3XbD-X-sYJ5-nerves_system_rpi0-FA897B0EEB99FDD29C9711F868394CEACD25D190C643CFE4E419B0806C9F8236

** (Mix) Nerves Docker provider encountered error: Error response from daemon: Invalid container name (-KbCw3XbD-X-sYJ5-nerves_system_rpi0-FA897B0EEB99FDD29C9711F868394CEACD25D190C643CFE4E419B0806C9F8236), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed

Expected behavior

The generated container must be valid. The leading - is not allowed. It seems that Base.url_encode64 in line 121 of docker.ex does not prevent this.

mix firmware doesn't properly rebuild release

Repro:

• clone a simple nerves project
• mix firmware
• change the mixfile to have a different version for the app
• mix firmware again

BUG: an "upgrade release" is created but the OLD release is included in the firmware bundle.

Here is an example log from my build. I had just changed the version stamp in my app's mix.exs from '0.2.1-pre5' to '0.2.1-pre-6a'

gmb:stem garth$ mix firmware
Compiling 1 file (.ex)
Generated stem app
Building release with MIX_ENV=dev.

You have dependencies (direct/transitive) which are not in :applications!
The following apps should be added to :applications in mix.exs:

        nerves_system_rpi3 -> nerves_system -> porcelain                                       => porcelain is missing from nerves_system
        nerves_system_rpi3 -> nerves_system_br                                                 => nerves_system_br is missing from nerves_system_rpi3
        nerves_system_rpi3 -> nerves_toolchain_arm_unknown_linux_gnueabihf -> nerves_toolchain => nerves_toolchain is missing from nerves_toolchain_arm_unknown_linux_gnueabihf
        logger_multicast_backend                                                               => logger_multicast_backend is missing from stem

==> Modifying release for Nerves System rpi3
This is an upgrade, verifying appups exist for updated dependencies..
==> All dependencies have appups ready for release!
==> Generated .appup for stem 0.2.1-pre5 -> 0.2.1-pre-6a
==> The release for stem-0.2.1-pre-6a is ready!
==> You can boot a console running your release with `$ rel/stem/bin/stem console`
Updating base firmware image with Erlang release...
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on /Users/garth/nerves/stem/_nerves-tmp/combined.squashfs, block size 131072.

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
        compressed data, compressed metadata, compressed fragments, no xattrs
        duplicates are removed
Filesystem size 14587.15 Kbytes (14.25 Mbytes)
        54.78% of uncompressed filesystem size (26630.20 Kbytes)
Inode table size 10441 bytes (10.20 Kbytes)
        29.62% of uncompressed inode table size (35248 bytes)
Directory table size 11471 bytes (11.20 Kbytes)
        43.03% of uncompressed directory table size (26659 bytes)
Number of duplicate files found 20
Number of inodes 1064
Number of files 836
Number of fragments 112
Number of symbolic links  106
Number of device nodes 1
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 121
Number of ids (unique uids + gids) 4
Number of uids 3
        root (0)
        unknown (33)
        garth (501)
Number of gids 3
        wheel (0)
        _appstore (33)
        staff (20)
Building _images/rpi3/stem.fw...

NOTE: above it says it's an UPGRADE release. At this point the stem.fw still boots to 0.2.1-pre5 :(

Now to fix it, I do:

gmb:stem garth$ rm -rf rel
gmb:stem garth$ mix firmware
Building release with MIX_ENV=dev.

You have dependencies (direct/transitive) which are not in :applications!
The following apps should be added to :applications in mix.exs:

        nerves_system_rpi3 -> nerves_system -> porcelain                                       => porcelain is missing from nerves_system
        nerves_system_rpi3 -> nerves_system_br                                                 => nerves_system_br is missing from nerves_system_rpi3
        nerves_system_rpi3 -> nerves_toolchain_arm_unknown_linux_gnueabihf -> nerves_toolchain => nerves_toolchain is missing from nerves_toolchain_arm_unknown_linux_gnueabihf
        logger_multicast_backend                                                               => logger_multicast_backend is missing from stem

==> Modifying release for Nerves System rpi3
==> The release for stem-0.2.1-pre-6a is ready!
==> You can boot a console running your release with `$ rel/stem/bin/stem console`
Updating base firmware image with Erlang release...
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on /Users/garth/nerves/stem/_nerves-tmp/combined.squashfs, block size 131072.

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
        compressed data, compressed metadata, compressed fragments, no xattrs
        duplicates are removed
Filesystem size 14570.72 Kbytes (14.23 Mbytes)
        54.95% of uncompressed filesystem size (26515.12 Kbytes)
Inode table size 10321 bytes (10.08 Kbytes)
        29.85% of uncompressed inode table size (34576 bytes)
Directory table size 11374 bytes (11.11 Kbytes)
        43.48% of uncompressed directory table size (26159 bytes)
Number of duplicate files found 11
Number of inodes 1043
Number of files 819
Number of fragments 111
Number of symbolic links  106
Number of device nodes 1
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 117
Number of ids (unique uids + gids) 4
Number of uids 3
        root (0)
        unknown (33)
        garth (501)
Number of gids 3
        wheel (0)
        _appstore (33)
        staff (20)
Building _images/rpi3/stem.fw...
gmb:stem garth$

stem.fw is now the proper 0.2.1-pre-6a :)

Add a Mix task for packaging a Nerves system artifact

Currently, it's kind of complicated to create a .tar.gz file for a custom system that can be uploaded to an HTTP cache location.

I propose that we add a mix system.artifact or mix nerves.system.artifact task that would build and package the artifact, then tell you the output location where it was placed, so you can upload it.

I think this would live in the Docker and Local providers (not the bootstrap archive), so the nerves. prefix on the task shouldn't be necessary unless we want it to be consistent with mix nerves.system.shell.

The documentation for UI is not complete

The documentation for how to crete a UI is somewhat out of date. At the moment sqlite does not work and is not needed neither is brunch. Configuration for the ui app in the fw app is missing and also configuration for the OTP application is missing. fw.ex must also have a child worker that sets up networking.

I have created a pr that fixes this #31

Kernel panic while booting Nerves EV3 example

Environment

• Erlang version (erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' -noshell):
  19
• Elixir version (elixir -v):
  1.3.1
• Operating system:
  Arch Linux
• Nerves Environment Info:
  nerves_system 0.1.6
  nerves_system_br 0.7.0
  nerves_system_ev3 0.7.0; more below

Current behavior

A kernel panic occurs, complaining about no working init found:
https://gist.github.com/arjan/c4011f7aa6c81fc1d9581ac97be9f467

Expected behavior

It boots..

cc @danielwillemse

Full mix.lock:

~/tmp/nerves_ev3_example> cat mix.lock 
%{"bbmustache": {:hex, :bbmustache, "1.0.4", "7ba94f971c5afd7b6617918a4bb74705e36cab36eb84b19b6a1b7ee06427aa38", [:rebar], []},
  "cf": {:hex, :cf, "0.2.1", "69d0b1349fd4d7d4dc55b7f407d29d7a840bf9a1ef5af529f1ebe0ce153fc2ab", [:rebar3], []},
  "elixir_make": {:hex, :elixir_make, "0.4.0", "992f38fabe705bb45821a728f20914c554b276838433349d4f2341f7a687cddf", [:mix], []},
  "erlware_commons": {:hex, :erlware_commons, "0.21.0", "a04433071ad7d112edefc75ac77719dd3e6753e697ac09428fc83d7564b80b15", [:rebar3], [{:cf, "0.2.1", [hex: :cf, optional: false]}]},
  "ex_ncurses": {:git, "https://github.com/fhunleth/ex_ncurses.git", "85cca394d1e638486fd34744494ce023b16420df", [branch: "bump_deps"]},
  "exrm": {:hex, :exrm, "1.0.8", "5aa8990cdfe300282828b02cefdc339e235f7916388ce99f9a1f926a9271a45d", [:mix], [{:relx, "~> 3.5", [hex: :relx, optional: false]}]},
  "getopt": {:hex, :getopt, "0.8.2", "b17556db683000ba50370b16c0619df1337e7af7ecbf7d64fbf8d1d6bce3109b", [:rebar], []},
  "logger_file_backend": {:hex, :logger_file_backend, "0.0.9", "5c2f7d4a28431e695cdf94d191523dbafe609321a67bb654254897f546c393db", [:mix], []},
  "nerves": {:hex, :nerves, "0.3.4", "7de99513f2775633e2db00acbf7437c55bd132f19b7406814f50818702b381b5", [:mix], [{:exrm, "~> 1.0.6", [hex: :exrm, optional: false]}]},
  "nerves_interim_wifi": {:hex, :nerves_interim_wifi, "0.0.2", "51c51578fc0a5d929a8ea26d54e6f470bcca24762742473cb44be3a622458a98", [:make, :mix], [{:elixir_make, "~> 0.3", [hex: :elixir_make, optional: false]}, {:nerves_network_interface, "~> 0.3.1", [hex: :nerves_network_interface, optional: false]}, {:nerves_wpa_supplicant, "~> 0.2.2", [hex: :nerves_wpa_supplicant, optional: false]}]},
  "nerves_network_interface": {:hex, :nerves_network_interface, "0.3.2", "e7450f7eccd7387ceebee6096a0646bcdc0dd4aa77077e5fa81f6767da9b77af", [:make, :mix], [{:elixir_make, "~> 0.3", [hex: :elixir_make, optional: false]}]},
  "nerves_system": {:hex, :nerves_system, "0.1.6", "d3cff2ae40d3764c75b187cbbdb09985ae169489c26178d765edc9b66bc4e355", [:mix], []},
  "nerves_system_br": {:hex, :nerves_system_br, "0.7.0", "e838dbec1079fa326d958a3aef4ee14f22d8ffbc91ce7f6b28fc7d9f3197dfa7", [:mix], []},
  "nerves_system_ev3": {:hex, :nerves_system_ev3, "0.7.0", "dc2ff4d479ce1c81f45bfa7771e10777da4383286acc87f6f3fdc65dc4347eb0", [:mix], [{:nerves_system, "~> 0.1.4", [hex: :nerves_system, optional: false]}, {:nerves_system_br, "~> 0.7.0", [hex: :nerves_system_br, optional: false]}, {:nerves_toolchain_armv5tejl_unknown_linux_musleabi, "~> 0.7.0", [hex: :nerves_toolchain_armv5tejl_unknown_linux_musleabi, optional: false]}]},
  "nerves_toolchain": {:hex, :nerves_toolchain, "0.7.1", "164f5fcf07f717d5e8f918965ea043c860b75065768c93a726f763498a7d8792", [:mix], []},
  "nerves_toolchain_arm_unknown_linux_gnueabi": {:hex, :nerves_toolchain_arm_unknown_linux_gnueabi, "0.6.2", "a8bbfb30bb11b9c0115643feb6d1980335e267c3ea8533d8918863fae6797295", [:mix], [{:nerves_toolchain, "~> 0.6.2", [hex: :nerves_toolchain, optional: false]}]},
  "nerves_toolchain_armv5tejl_unknown_linux_musleabi": {:hex, :nerves_toolchain_armv5tejl_unknown_linux_musleabi, "0.7.1", "43909a7aeab85e4d0a81c8b8f97b053b2975accdc0b04096ce5f8781bfbb63f0", [:mix], [{:nerves_toolchain, "~> 0.7", [hex: :nerves_toolchain, optional: false]}, {:nerves_toolchain_ctng, "~> 0.7", [hex: :nerves_toolchain_ctng, optional: false]}]},
  "nerves_toolchain_ctng": {:hex, :nerves_toolchain_ctng, "0.7.1", "6f46b1591b23d1a017347f8cd9962c24775b2a6fde4e2f2b5251439efef8d6a4", [:mix], []},
  "nerves_wpa_supplicant": {:hex, :nerves_wpa_supplicant, "0.2.2", "da829baec94a8afa7da3a6beaaf1725b9ca7f5f58779ba50b73568b7d37d3d98", [:make, :mix], [{:elixir_make, "~> 0.3", [hex: :elixir_make, optional: false]}]},
  "porcelain": {:hex, :porcelain, "2.0.1", "9c3db2b47d8cf6879c0d9ac79db8657333974a88faff09e856569e00c1b5e119", [:mix], []},
  "providers": {:hex, :providers, "1.6.0", "db0e2f9043ae60c0155205fcd238d68516331d0e5146155e33d1e79dc452964a", [:rebar3], [{:getopt, "0.8.2", [hex: :getopt, optional: false]}]},
  "relx": {:hex, :relx, "3.21.1", "f989dc520730efd9075e9f4debcb8ba1d7d1e86b018b0bcf45a2eb80270b4ad6", [:rebar3], [{:bbmustache, "1.0.4", [hex: :bbmustache, optional: false]}, {:cf, "0.2.1", [hex: :cf, optional: false]}, {:erlware_commons, "0.21.0", [hex: :erlware_commons, optional: false]}, {:getopt, "0.8.2", [hex: :getopt, optional: false]}, {:providers, "1.6.0", [hex: :providers, optional: false]}]}}

mix firmware produces "no case clause matching: :eacces" error

Environment

• Elixir version (elixir -v):
  Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
  Elixir 1.3.4

• Operating system:
  mac OS Sierra 10.12.1

• Nerves Environment Info:
  (mix nerves.env --info) or
  General info about target / system / toolchain

Current behavior

$  mix nerves.new tmp --target rpi
* creating tmp/config/config.exs
* creating tmp/lib/tmp.ex
* creating tmp/test/test_helper.exs
* creating tmp/test/tmp_test.exs
* creating tmp/rel/vm.args
* creating tmp/rel/.gitignore
* creating tmp/.gitignore
* creating tmp/mix.exs
* creating tmp/README.md

$ mix deps.get
Running dependency resolution
Dependency resolution completed
  bbmustache: 1.0.4
  cf: 0.2.2
  erlware_commons: 0.22.0
  exrm: 1.0.8
  getopt: 0.8.2
  nerves: 0.3.4
  nerves_system: 0.1.8
  nerves_system_br: 0.7.0
  nerves_system_rpi: 0.7.0
  nerves_toolchain: 0.7.1
  nerves_toolchain_armv6_rpi_linux_gnueabi: 0.7.2
  nerves_toolchain_ctng: 0.7.1
  providers: 1.6.0
  relx: 3.22.0
* Getting nerves (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves-0.3.4.tar)
  Using locally cached package
* Getting nerves_system_rpi (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_system_rpi-0.7.0.tar)
  Using locally cached package
* Getting nerves_system (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_system-0.1.8.tar)
  Using locally cached package
* Getting nerves_system_br (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_system_br-0.7.0.tar)
  Using locally cached package
* Getting nerves_toolchain_armv6_rpi_linux_gnueabi (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_toolchain_armv6_rpi_linux_gnueabi-0.7.2.tar)
  Using locally cached package
* Getting nerves_toolchain (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_toolchain-0.7.1.tar)
  Using locally cached package
* Getting nerves_toolchain_ctng (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_toolchain_ctng-0.7.1.tar)
  Using locally cached package
* Getting exrm (Hex package)
  Checking package (https://repo.hex.pm/tarballs/exrm-1.0.8.tar)
  Using locally cached package
* Getting relx (Hex package)
  Checking package (https://repo.hex.pm/tarballs/relx-3.22.0.tar)
  Using locally cached package
* Getting bbmustache (Hex package)
  Checking package (https://repo.hex.pm/tarballs/bbmustache-1.0.4.tar)
  Using locally cached package
* Getting cf (Hex package)
  Checking package (https://repo.hex.pm/tarballs/cf-0.2.2.tar)
  Using locally cached package
* Getting erlware_commons (Hex package)
  Checking package (https://repo.hex.pm/tarballs/erlware_commons-0.22.0.tar)
  Using locally cached package
* Getting getopt (Hex package)
  Checking package (https://repo.hex.pm/tarballs/getopt-0.8.2.tar)
  Using locally cached package
* Getting providers (Hex package)
  Checking package (https://repo.hex.pm/tarballs/providers-1.6.0.tar)
  Using locally cached package

$ mix firmware
Building release with MIX_ENV=dev.

You have dependencies (direct/transitive) which are not in :applications!
The following apps should be added to :applications in mix.exs:

        nerves_system_rpi -> nerves_system                                                     => nerves_system is missing from nerves_system_rpi
        nerves_system_rpi -> nerves_system_br                                                  => nerves_system_br is missing from nerves_system_rpi
        nerves_system_rpi -> nerves_toolchain_armv6_rpi_linux_gnueabi -> nerves_toolchain      => nerves_toolchain is missing from nerves_toolchain_armv6_rpi_linux_gnueabi
        nerves_system_rpi -> nerves_toolchain_armv6_rpi_linux_gnueabi -> nerves_toolchain_ctng => nerves_toolchain_ctng is missing from nerves_toolchain_armv6_rpi_linux_gnueabi
        nerves                                                                                 => nerves is missing from tmp

==> Modifying release for Nerves System rpi
** (CaseClauseError) no case clause matching: :eacces
    lib/exrm/utils/utils.ex:110: ReleaseManager.Utils.relx/5
    lib/mix/tasks/release.ex:339: anonymous fn/4 in Mix.Tasks.Release.do_release/1
    lib/ex_unit/capture_io.ex:146: ExUnit.CaptureIO.do_capture_io/2
    lib/ex_unit/capture_io.ex:119: ExUnit.CaptureIO.do_capture_io/3
    lib/mix/tasks/release.ex:338: Mix.Tasks.Release.do_release/1
    lib/mix/tasks/release.ex:78: Mix.Tasks.Release.do_run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    lib/mix/tasks/firmware.ex:27: Mix.Tasks.Firmware.run/1

Better handling of clean vs. incremental builds for custom systems

mix nerves.system.shell upon initialization calls make clean, which prevents introspection into the build. make clean should be called only by mix nerves.clean

OS X Key Chain prompts for access to unrelated item during custom system build.

Environment

• Erlang version: 19
• Elixir version: 1.4.0
• Operating system: OS X 10.11
• Nerves Environment Info:
  I cloned nerves_system_linkit, renamed it and customized the Metadata but didn't make any changes to the system. I just followed the steps for Customizing Your Own Nerves System.

Current behavior

When building the custom system. A terminal prompt appears as follows:

The Nerves Docker provider needs to create the image.
Proceed?  [Yn]

After hitting y to allow Docker to create an image, OSX requests access to a keychain item that's unrelated to Nerves or to my Nerves project. I selected "Deny" and the build appears to continue normally.

Expected behavior

After hitting y to allow Docker to create an image, if I were prompted to allow access to a Keychain item it would be an item related to Nerves or to the current project.

Note: I don't understand the problem well enough to know if this is an issue with Docker, OS X Keychain, or if it's just something I'm not understanding. Also note I was still able to build the system after clicking "Deny". It shouldn't block anyone's work but could be confusing.

mix firmware fails with poor error message when NERVES_TOOLCHAIN points to a directory that no longer exists

I did the mistake of removing the directory that NERVES_TOOLCHAIN points to and got a bad error message.

My config:

NERVES_TOOLCHAIN=/Users/jose/OSS/badge/pen/nerves-mipsel-unknown-linux-musl-darwin-x86_64-v0.6.3
NERVES_SYSTEM=/Users/jose/OSS/badge/pen/nerves_system_linkit

The error:

$ mix firmware
** (FunctionClauseError) no function clause matching in String.replace_suffix/3
    (nerves_bootstrap) lib/mix/tasks/nerves.loadpaths.ex:20: Mix.Tasks.Nerves.Loadpaths.run/1
    (nerves_bootstrap) lib/mix/tasks/nerves.precompile.ex:19: Mix.Tasks.Nerves.Precompile.run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:328: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:261: Mix.Task.run/2
    (mix) lib/mix/tasks/deps.compile.ex:62: Mix.Tasks.Deps.Compile.compile/2
    (mix) lib/mix/tasks/deps.loadpaths.ex:83: Mix.Tasks.Deps.Loadpaths.deps_check/2
    (mix) lib/mix/tasks/deps.loadpaths.ex:27: Mix.Tasks.Deps.Loadpaths.run/1

It may be worth checking if the directory in the environment variable actually exists before proceeding or something of sorts. :D

'mix firmware.burn --device myimage' creates a file as root

When creating image files, mix firmware.burn still wants root permissions on Linux. It shouldn't need those.

Another thought is to add mix firmware.image <filename> to create a raw image file.

Artifact download fails from https://raw.githubusercontent.com

This issue was reported by um404 in the Slack channel.

It appears that, in this case, the problem is a missing CONTENT-DISPOSITION header, but this error message also isn't helpful. We should improve the error-handling as well as solving this issue with a missing header.

Current behavior

$ MIX_TARGET=zynq mix firmware

Mix environment
  MIX_TARGET:   zynq
  MIX_ENV:      dev

warning: variable "target" is unused
  mix.exs:59

==> nerves_system_zynq
Generated nerves_system_zynq app
|Elixir.Nerves.Package.Providers.HTTP| Downloading Artifact nerves_system_zynq-0.1.0.arm_unknown_linux_gnueabihf.tar.gz
  From Location:
    https://github.com/uldza/nerves_system_zynq/raw/master/nerves_system_zynq.tar.gz

** (EXIT from #PID<0.70.0>) an exception was raised:
    ** (MatchError) no match of right hand side value: nil
        (nerves) lib/nerves/utils/http_client.ex:59: Nerves.Utils.HTTPClient.handle_info/2
        (stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
        (stdlib) gen_server.erl:667: :gen_server.handle_msg/5
        (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

10:48:18.598 [error] GenServer #PID<0.186.0> terminating
** (MatchError) no match of right hand side value: nil
    (nerves) lib/nerves/utils/http_client.ex:59: Nerves.Utils.HTTPClient.handle_info/2
    (stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:667: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: {:http, {#Reference<0.0.5.965>, :stream_start, [{'cache-control', 'max-age=300'}, {'connection', 'keep-alive'}, {'date', 'Mon, 03 Jul 2017 07:48:19 GMT'}, {'via', '1.1 varnish'}, {'accept-ranges', 'bytes'}, {'etag', '"e148ead325b500bb7d369cdba5da8e109b4f443b"'}, {'vary', 'Authorization,Accept-Encoding'}, {'content-length', '75989986'}, {'content-type', 'application/octet-stream'}, {'expires', 'Mon, 03 Jul 2017 07:53:19 GMT'}, {'content-security-policy', 'default-src \'none\'; style-src \'unsafe-inline\''}, {'strict-transport-security', 'max-age=31536000'}, {'x-content-type-options', 'nosniff'}, {'x-frame-options', 'deny'}, {'x-xss-protection', '1; mode=block'}, {'x-geo-block-list', []}, {'x-github-request-id', '5BDA:2020:35D119:385214:5959F6C1'}, {'x-served-by', 'cache-ams4420-AMS'}, {'x-cache', 'MISS'}, {'x-cache-hits', '0'}, {'x-timer', 'S1499068097.199341,VS0,VE2692'}, {'access-control-allow-origin', '*'}, {'x-fastly-request-id', '60fc2a44492072e249eff033259dfdeb6c67cace'}]}}
State: %{buffer: "", buffer_size: 0, caller: {#PID<0.70.0>, #Reference<0.0.5.907>}, content_length: 0, filename: "", number_of_redirects: 1, url: "https://raw.githubusercontent.com/uldza/nerves_system_zynq/master/nerves_system_zynq.tar.gz"}

Expected behavior

See the relevant code here: https://github.com/nerves-project/nerves/blob/master/lib/nerves/utils/http_client.ex#L49

• The download should succeed if the requested file exists.
• The file name should be inferred from the URL if the server does not indicate it in any way we can predict.
• Improve error-handling to display error or warning messages when things don't match what we expect, instead of (or at least in addition to) crashing the process.

nerves_bootstrap archive should be publicly versioned

I experienced a problem where nerves_bootstrap was out of date, but I had no way of diagnosing it. Updating it via mix local.nerves wasn't working for some reason, so I had to uninstall/reinstall. I don't know what that bug is, but that's not the point of this issue.

All of this would have been clearer if the nerves_bootstrap archive was versioned like hex is...

targhee:boom garth$ mix archive
* hex-0.15.0
* nerves_bootstrap
Archives installed at: /Users/garth/.mix/archives
targhee:boom garth$

Suggest adding a version stamp to the nerves_bootstrap archive that shows up in mix's archive list.

Artifact Downloading Fails on Github Redirect

Environment

• Erlang version (erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' -noshell): 19
• Elixir version (elixir -v): 1.4.2
• Operating system: Mac OS 10.12.3
• Nerves Environment Info: Trying to build the blinky example app for rpi target

|nerves_bootstrap| Environment Package List

  Pkg:      nerves_system_br
  Vsn:      0.9.4
  Type:     system_platform
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Docker, []}]

  Pkg:      nerves_system_rpi
  Vsn:      0.11.0
  Type:     system
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Docker, []}]

  Pkg:      nerves_toolchain_armv6_rpi_linux_gnueabi
  Vsn:      0.10.0
  Type:     toolchain
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.HTTP, []}]

  Pkg:      nerves_toolchain_ctng
  Vsn:      0.9.0
  Type:     toolchain_platform
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Docker, []}]

|nerves_bootstrap| Loadpaths Start

|nerves_bootstrap| Env Start

|nerves_bootstrap| Env End

|nerves_bootstrap| Environment Variable List
  target:     rpi
  toolchain:  /Users/hqmq/.nerves/artifacts/nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64
  system:     /Users/hqmq/.nerves/artifacts/nerves_system_rpi-0.11.0.armv6_rpi_linux_musl
  app:        /Users/hqmq/code/tmp/nerves-examples/blinky

|nerves_bootstrap| Loadpaths End

Current behavior

mmmries-pro:blinky$ MIX_TARGET=rpi mix do deps.get, firmware, firmware.burn
Env
  MIX_TARGET:   rpi
  MIX_ENV:      dev

Running dependency resolution...
Dependency resolution completed:
  nerves_system_rpi 0.11.0
  nerves_toolchain_armv6_rpi_linux_gnueabi 0.10.0
* Getting nerves (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves-0.5.0.tar)
  Fetched package
* Getting nerves_runtime (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_runtime-0.1.1.tar)
  Fetched package
* Getting nerves_system_rpi (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_system_rpi-0.11.0.tar)
  Fetched package
* Getting nerves_leds (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_leds-0.7.0.tar)
  Fetched package
* Getting nerves_system_br (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_system_br-0.9.4.tar)
  Fetched package
* Getting nerves_toolchain_armv6_rpi_linux_gnueabi (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.tar)
  Fetched package
* Getting nerves_toolchain_ctng (Hex package)
  Checking package (https://repo.hex.pm/tarballs/nerves_toolchain_ctng-0.9.0.tar)
  Fetched package
* Getting elixir_make (Hex package)
  Checking package (https://repo.hex.pm/tarballs/elixir_make-0.4.0.tar)
  Fetched package
* Getting gen_stage (Hex package)
  Checking package (https://repo.hex.pm/tarballs/gen_stage-0.11.0.tar)
  Fetched package
* Getting distillery (Hex package)
  Checking package (https://repo.hex.pm/tarballs/distillery-1.2.2.tar)
  Fetched package
==> distillery
Compiling 18 files (.ex)
Generated distillery app
==> nerves
Compiling 25 files (.ex)
Generated nerves app
==> nerves_system_br
Generated nerves_system_br app
==> nerves_toolchain_ctng
Compiling 2 files (.ex)
Generated nerves_toolchain_ctng app
==> nerves_toolchain_armv6_rpi_linux_gnueabi
Compiling 1 file (.ex)
Generated nerves_toolchain_armv6_rpi_linux_gnueabi app
|Elixir.Nerves.Package.Providers.HTTP| Downloading Artifact nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz
  From Location:
    https://github.com/nerves-project/toolchains/releases/download/v0.10.0/nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz

|Elixir.Nerves.Package.Providers.HTTP| No Available Locations

|Elixir.Nerves.Package.Providers.HTTP| Downloading Artifact nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz
  From Location:
    https://github.com/nerves-project/toolchains/releases/download/v0.10.0/nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz

|Elixir.Nerves.Package.Providers.HTTP| No Available Locations

==> nerves_system_rpi
Generated nerves_system_rpi app
|Elixir.Nerves.Package.Providers.HTTP| Downloading Artifact nerves_system_rpi-0.11.0.armv6_rpi_linux_musl.tar.gz
  From Location:
    https://github.com/nerves-project/nerves_system_rpi/releases/download/v0.11.0/nerves_system_rpi-v0.11.0.tar.gz

|Elixir.Nerves.Package.Providers.HTTP| No Available Locations

could not compile dependency :nerves_system_rpi, "mix compile" failed. You can recompile this dependency with "mix deps.compile nerves_system_rpi", update it with "mix deps.update nerves_system_rpi" or clean it with "mix deps.clean nerves_system_rpi"

Expected behavior

The downloads should succeed so that my firmware bundle can be built correctly

Improve error message with toolchain can't be downloaded

The problem here is that the toolchain doesn't exist in the GitHub release. Here's the current error message:

pi@raspberrypi:~/nerves-examples/hello_gpio $ mix compile
==> nerves_toolchain_arm_unknown_linux_gnueabihf
Compiling 1 file (.ex)
Generated nerves_toolchain_arm_unknown_linux_gnueabihf app
[nerves_toolchain][compile]
[nerves_toolchain][http] Downloading Toolchain
could not compile dependency :nerves_toolchain_arm_unknown_linux_gnueabihf, "mix compile" failed. You can recompile this dependency with "mix deps.compile nerves_toolchain_arm_unknown_linux_gnueabihf", update it with "mix deps.update nerves_toolchain_arm_unknown_linux_gnueabihf" or clean it with "mix deps.clean nerves_toolchain_arm_unknown_linux_gnueabihf"
** (RuntimeError) Nerves Toolchain Github cache returned error: "httpc request failed with: {:bad_status_code, 404}"
    (nerves_toolchain) lib/mix/tasks/compile.nerves_toolchain.ex:70: Mix.Tasks.Compile.NervesToolchain.cache/2
    (nerves_toolchain) lib/mix/tasks/compile.nerves_toolchain.ex:38: Mix.Tasks.Compile.NervesToolchain.run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
    (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
    (mix) lib/mix/tasks/compile.all.ex:19: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:37: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3

pi@raspberrypi:~/nerves-examples/hello_gpio $ 

It would be nice if it at least said what it was trying to download.

allow mix firmware to pass args through to fwup

mix firmware and mix firmware.burn should allow additional args to be passed through to fwup. This is going to be useful for signing firmware as part of the build of a project. We could support this by either passing "un-captured" args through to fwup
for example:
mix firmware --sign --private-key 12345

or by capturing something like fwup_args and passing its value through
For Example:
mix firmware --fwup-args "--sign --private-key 12345"

This currently goes through the script rel2fw.sh which makes the call to fwup so there could be a bit of work here. I will reference a separate issue which tracks the progress on converting this part of the process to Elixir.

mix firmware.burn TTY/PTY issue

Environment

• Erlang version (erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().' -noshell): 20
• Elixir version (elixir -v): 1.4.5
• Operating system: Ubuntu Server 16.04
• Nerves Environment Info:

Env
  MIX_TARGET:   kiosk_system_rpi3
  MIX_ENV:      dev

|nerves_bootstrap| Environment Package List

  Pkg:      kiosk_system_rpi3
  Vsn:      0.11.0
  Type:     system
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.Local, []}]

  Pkg:      nerves_system_br
  Vsn:      0.13.2
  Type:     system_platform
  Provider: []

  Pkg:      nerves_toolchain_arm_unknown_linux_gnueabihf
  Vsn:      0.10.0
  Type:     toolchain
  Provider: [{Nerves.Package.Providers.HTTP, []}, {Nerves.Package.Providers.HTTP, []}]

  Pkg:      nerves_toolchain_ctng
  Vsn:      0.9.0
  Type:     toolchain_platform
  Provider: []

|nerves_bootstrap| Loadpaths Start

|nerves_bootstrap| Environment Variable List
  target:     kiosk_system_rpi3
  toolchain:  /home/orestis/dev/kiosk_system_rpi3/build_br/host
  system:     /home/orestis/dev/kiosk_system_rpi3/build_br
  app:        /home/orestis/dev/nerves-examples/hello_phoenix/apps/fw

|nerves_bootstrap| Loadpaths End

Current behavior

I have setup a virtual box environment in order to build a custom system. When finally all is built, trying to run mix firmware.burn resulted in this error:

$ mix firmware.burn
Env
  MIX_TARGET:   kiosk_system_rpi3
  MIX_ENV:      dev

Discovered devices:
0) 32.0 GiB found at /dev/sda
1) 14.84 GiB found at /dev/sdb
Which device do you want to burn to? 1
sudo: no tty present and no askpass program specified

After much searching online, I realised the culprit is that when running System.cmd there's no pty/tty session attached.

When running via bash:

$ tty
/dev/pts/0

When running via iex:

iex(1)> System.cmd "tty", []
{"not a tty\n", 1}

So essentially when running "sudo fwup", sudo complains that there's no TTY attached and cannot ask for a password. A bit more bafflingly, the SUDO_ASKPASS environment variable doesn't do anything, since System.cmd doesn't inherit the environment in any case (I think). If I try to force the default SUDO_ASKPASS behaviour by doing sudo --askpass fwup, the x11-ssh-askpass is indeed invoked and (rightfully) complains there's no display.

Expected behavior

The sudo fwup command should be invoked with the correct PTY passed in so sudo can just prompt for a password.

Workaround

I had to setup my user to have passwordless sudo in order to keep working.