Comments (5)
Let me quote some of my comment here:
The possibility of having multiple outputs was added to buck2 because we thought it was generally a good idea. However, based on something I heard from cjhopman , we never really used it internally because it turned out that too much stuff broke if --show-output returned more than one output. Instead, we've mostly gone with subtargets to make any additional outputs available.
The result of that is that the "multiple outputs" support is only partially complete, and I don't think we ever fully figured out what we want from it. If you want to stay on the well-trodden path, not using it would probably be your best bet. On the other hand, the fact that this isn't really figured out yet also means that we'd be happy to take PRs that build this support out and clarify how things should work
It's possible that someone on the team has stronger opinions on this then I do, but at least for me, this case is an example of it not being entirely clear what we should do.
from buck2.
Thanks for the context. If you were to add a function like default_outputs()
as I proposed in the first post, that would leave all existing behavior unchanged, while also being flexible enough to enable a bunch of new usage simplification scenarios. Right now I have to do something like this
srcs=[":{}[out_files][generated_srcs][{}]".format(internal_name, src) for src in generated_srcs],
But with the suggested change, I could change bit to this:
srcs=default_outputs(":" + internal_name),
which is a huge improvement not just in that line of code, but in the implementation of the rule itself, since I wouldn't even have to create any subtargets. Even beyond this use case though, just being able to access the default_outputs
of a target via a global function like this could be useful for other scenarios I'm not even thinking of. And it works just as well for single output providers too. You could even deprecate usages like srcs = [":foo"]
and say that the correct way going forward is srcs = default_outputs(":foo")
since that works everywhere and is composable (e.g. srcs = glob("*.cpp") + default_outputs(":foo")
)
from buck2.
but in the implementation of the rule itself, since I wouldn't even have to create any subtargets. Even beyond this use case though, just being able to access the
default_outputs
of a target via a global function like this could be useful for other scenarios I'm not even thinking of
So one of the difficulties with this kind of default_outputs
thing is that you can't actually know what the outputs of analysis are at loading time (or even how many they will be) - that's just an ordering problem, loading happens strictly before analysis. That means that you couldn't actually return a list from that function, you'd have to return some kind of a proxy object. The ergonomics of that quickly get pretty grim, so that's probably not a great direction to go in
The best thing that I can think of right now is that maybe we should just try and make multiple default outputs on a single item in an attr.list(attr.source())
magically work (and error when it's not in a list I suppose?). That's really not ideal, but I can't think of anything better
from buck2.
I trust you when you say the ergonomics of the proxy object would be grim, but it helps me understand what's going on under the hood, so apologies if I keep digging :)
Are you talking about ergonomics from the user side, from the buck2 core runtime side, from the rule implementation side, or something else? I'm imaging that if you did do the proxy object, then all of the complexity of reaching into this proxy object could be silently handled by attrs.list()
. For example, maybe attrs.list()
is taught to handle a proxy object of AnalysisList
(a list whose length isn't known until analysis time), and which resolves to a collection of Artifact
objects. So on the rule side, it doesn't have to be updated and ctx.attrs.srcs
just magically contains all the right stuff.
Does this not work?
from buck2.
I'm more worried about the macro side, not the rule side. You still need to make this compose with other things that people do in macros. For example, someone is going to write srcs = default_outputs(":foo") + ["x.txt", "y.txt"]
. Meaning all the standard list operations have to keep working. But of course they have to return proxy objects too. Now some macro that does len(srcs)
which previously worked will break in weird and surprising ways. (also, maybe a macro wants to inspect the insides of that default_outputs
? Might have to support that)
Something like this exists already for another function widely used in macros: select
, which also supports +
for combining, map
, and a bunch of other stuff. The amount of complexity that handling selects introduces on both macro authors and on the core implementation is extremely high. My instinct is that this use case doesn't meet the bar for introducing a second copy of that
from buck2.
Related Issues (20)
- Errant octal escape strings in prelude docs HOT 1
- RE: Error, message length too large for `BatchReadBlobs` HOT 1
- buck2 fails to build on m3 mac
- Treat
- Test
- Does not prebuilt_cxx_library.static_lib support $(location //:target)? HOT 3
- Conflicting inputs on erlang build of Opentelemetry HOT 1
- error: Variable `typing` not found HOT 2
- Excluding rules from certain platforms HOT 6
- Unable to `buck2 clean` a repo which uses `git_fetch()` on Windows
- `configured_alias` and configuration modifiers HOT 2
- review Go analysis.Pass.Module proposal
- Creating symbolic links to toolchains HOT 10
- Early-building some parts of graph HOT 7
- How to use "buck2 clean" or other commands to delete only the generated files (cache) without killing the daemon? HOT 2
- Question: how to pass dependencies that change state forward across a non-output-changing rule HOT 4
- buck2 : Is java supported as a part of buck2 HOT 2
- Share work across configurations by setting working directory for actions then canonicalizing that working directory in RE HOT 5
- How to use multiple execution platforms HOT 1
- Distributed project.ignore
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from buck2.