davhau/drv-parts

Configure packages like NixOS systems

drv-parts's Issues

Could we somehow import packages from nixpkgs?

I'm wondering if we could somehow import existing package definitions from nixpkgs.
Maybe we could create another drv-parts backend that can wrap any default.nix file from nixpkgs.
Then would make porting existing packages to drv-parts much easier.

Split out a package interface for non-single-drv packages

Packages that aren't a single derivation are experimental, but enable new use cases.

• NixOS/rfcs#92
• multi-derivation packages, combining outputs of more than one derivation
  • Example: eliminate doc dependencies when docs aren't used
  • Example: don't rebuild program when doc tooling updates
  • NixOS/nix#6507

Shouldnt this be all builtins.derivation options?

drv-parts/modules/drv-parts/builtins-derivation/interface.nix

Line 7 in 38f59f9

builtin-derivation-options = {

The actual interface of builtins.derivation

builtins.derivation :: {
        name :: String;
        outputs :: [ output @ String ] ? [ "out" ];
        builder :: String;
        system :: String;
        args :: [ String ] ? [ ];
        ${additionalArg} :: a;
    } -> Derivation

source: https://www.werkbank.cloud/post/nix-derivation-type (from my blog under construction)

builtins.derivation will accept anything that it can implicitly coerce into string.

such as:

• list
• path (is a syntactic wrapper around string)
• string
• number
• etc

Side note:

even the outputs list is coerced into a single string. internally this is also why a list like
outputs = ["my app" "pkg" ]
would yield 3 outputs. Also this would fail because one item of the list MUST be "out"

Please say this can be used in non-flake settings?

I haven't had time for the shiny new flake stuff yet but your work is very useful and ergonomic. Is it possible yet to use it if I don't know anything about flakes?

passtrhu support in public?

Currently, to access a derivation's passthru, you need pkg.public.config.package-func.result.passthru.

I think passthru is something often used for gluing packages, so it would be good to have it under pkg.public.passthru directly.

Idea: declare pkg-config dependencies but only override Nixpkgs-style package names

Context

• NixOS/nixpkgs#212282
• NixOS/cabal2nix#594 (comment)

If we have a (json) mapping from pkg-config names to package attribute names, a module could easily add the appropriate deps, without sacrificing the consistency with Nixpkgs-style names when it comes to overriding.
Without a special integration for the mapping, it'd be hard to override things, because pkg-config names ("modules") don't map 1:1 to package attribute names; not even in quantity. For example, to override nix in a program that links it, you'd have to override all of nix-expr, nix-store, nix-main, and make sure not to miss a module if it grows a new one (it has). This is problem is hard to detect. hidapi has different pkg-config names depending on the platform.

Beauty

The design systems for Nix you've been cooking up recently are looking great.

I haven't tried out flake parts or std or similar systems yet; but the style you have here with interfaces and implementations split with modules to typecheck is really appealing.

In the future I'm hoping to carve out the time to explore these because it seems like it would save me from a lot of bespoke patterns I keep iterating through.

Idea: a `pkg-config` module

A package that expose pkg-config "modules" (.pc files) should do a couple of things.
In the current nixpkgs that would be

• add validatePkgConfig hook

In a future nixpkgs that may include

• declare the exposed pkg-config modules in a meta variable
• add a test to make sure that the declared pkg-config modules are present and valid in the output

Problems with `deps`

After collecting some experience using drv-parts in dream2nix, there are some observations with the current deps pattern that I'd like to discuss and find a solution for:

Observation 1: clashing names

Package can have clashing names. For example there can be a python package shadowing the name of a dependency from nixpkgs.

Observation 2: mixed concerns

Dependencies of very different concerns are currently mixed under deps., for example:

• nix library dependencies
• build dependencies (packages)
• other builders (like pkgs.writeText)
• stdenv

Observation 3: untyped

Due to the multi use described above, deps currently cannot be strictly typed.
It would be hard to apply a generic transformation on all dependencies (example: change stdenv for all deps), as not all dependencies are packages.

Observation 4: current deps aren't modules

In a lang2nix auto-generated dependency tree we'd like our dependencies to be modules, and we'd like to have a standard way of handling such dependencies, which is currently missing.
Since deps currently is a freeform module we could declare the type of some attributes to be a drv-parts submodule, but it feels odd mixing modules with arbitrary other types.

---

Ideas

It seems to me that we might want to categorize dependencies somehow. Dependencies can be different things, and have different capabilities.

Idea 1: Extra layer of nesting

Re-define deps to allow one extra layer of nesting, so it can be used like this:

{
  deps.generic = {nixpkgs, ...}: {
    inherit (nixpkgs) hello;
    python = nixpkgs.python310;
  }

  # language specific dependencies
  # allow the module maintainer to define the type of deps.python
  deps.python = {...}: {
    requests = ...;
    hello = ...;
  };
  
  # separate library deps
  deps.lib = ...
}

Idea 2: offer only a type or helper

Do not change/extend deps, but offer a pre-defined type or helper via drv-parts, so module maintainers can easily create new options similar to deps, like for example pythonDeps or nodejsDeps. This allows to assign a custom type to each dependency group which is useful.

Idea 3: Prefix the attribute names of individual dependencies

Example:

{
  deps = {nixpkgs, ...}: {
    inherit (nixpkgs) hello;
    python = nixpkgs.python310;
    
    # python deps
    python_hello = ...;
    python_requests = ...;

    # nodejs deps
    nodejs_hello = ...;
  }
}

I don't really like Idea 3.

---

Conclusion

Using Idea 2 seems like to most straight forward way to go as of now as it doesn't require changing drv-parts.
Still I'm wondering if we should implement a more sophisticated way of dealing with module dependencies and how such a solution would look like.

@roberth @phaer

Can `specialArgs` be avoided?

drv-parts/lib.nix

Line 10 in 26eceeb

specialArgs = {inherit dependencySets;};

This line parts ways with a "simplified" module interface by introducing higher order arguments to the module evaluation.

These aren't free. I'm not experienced enough to come up with a complete list of bullet points of why that is, but it's evident that two interfaces add additional implications and concerns over just a single interface.

Especially when they break up the module system DSL and revert "back" to the functional style (for any particular reason), as evidenced by the additional {} argument to the evaluator.

This may turn out to be a significant cost in the future (thinking about things like the proposed simplified module system for packages). Systems may be devised that are otherwise compatible with drv-parts' API but deny secondary interfaces such as specialArgs for good reasons.

Is there no way to get along without specialArgs and confine the design space to the module system DSL and don't mix it with the functional style of injection (i.e. specialArgs)?