Move to user/library space with decorators?

[tc39]

Move to user/library space with decorators?

[bakkot]

@claytongulick, have you seen the FAQ for private fields? In particular, Why isn't access this.x? addresses a wide variety of proposals for other syntax, including I think this one.

I'm not sure I understand what you mean about function/arguments.caller.

[claytongulick]

@bakkot thanks for pointing me to that article, no I hadn't see it yet and it raises some great points. I'm not sure I'm convinced that using decorators as a more generic approach to access modifiers is the wrong approach though. Thinking about the current proposal, it seems like JavaScript is playing catch-up to other languages for feature parity with classes. Walking back from this and using a generic approach like decorators, I think, makes JS more advanced than other languages - we're not limited to existing concepts of access modifiers.

The best argument I saw in that FAQ against a run-time approach like decorators is the performance impact. That's non-trivial, and we'd be killing any ability for the compiler to do it's job with privates. Also, it'd be a run-time error rather than a syntax error as defined in the current proposal, so both of those are negatives for sure. The question is whether the power of the decorator approach outweighs those negatives?

What I meant about function/arguments.caller is that in order to determine whether calling code should be allowed to access a property/method, you need to know something about the calling code - i.e. Is this function a member of my class? Is it defined elsewhere? Is it a member of a subclass? Or maybe a class that I'd like to consider a friend? For the decorator approach to work, you'd need to be able to figure out who's accessing the method/property, and what their context is. Maybe instead of beefing up Function.caller, it'd make more sense to add a param to the decorator signature that would take a caller_info data structure of some type?

[n8stowell82]

I think I agree with @claytongulick. JavasScript is very powerful today because of this built in flexibility. Adopting a more restrictive system just because it is what other languages are doing would in my opinion be a step backward.

[blargism]

I think there is merit here. Doing this.#private_thing is simple, but will box the language in. It doesn't provide any means for more nuanced access rules. The # approach also introduces code composition problems.

class SimpleThing {
  #x = 0;
  y = 1;

  doSomething(change_x) {
    this.#x = change_x + this.#x + this.y;
  }
}

In contrast the above proposal could meet the goal of private variables in classes without forever limiting how and to whom variable access is given in the future. Consider this:

class AnotherSimpleThing {
  @Private
  x = 0;

  @Protected
  y = 1;

  doSomething(change_x) {
    this.x = change_x + this.x + this.y;
  }
}

In my view it is a simpler solution to just make anything decorated with @Private as private always. No ambiguity. With this approach a developer can't have a private x and a public x in the same class. In general naming two things x, albeit with the # prepended, isn't really something I would let pass in a code review. While this is a semantic argument, I do think it has merit from the perspective of code quality. How the semantics are defined will directly affect how developers will use the constructs.

Composition problems aside, # provides no means for anything like protected access, and we are running out of special characters. I know we CAN use emoji's (UTF-8 characters), but I don't think something as simple and fundamental as variable and function access should be represented with something that's not on the average keyboard. We could do something like ## for protected access by an extending class, but that doesn't provide any userspace options.

Using decorators opens the future possibility to define access in very granular ways. @Protected would expose a variable to an extending class. Or... let's get extra crazy and say that anything in the module can access the variable using a decorator like @Modular. I personally like that idea. The language only has to define what is considered essential, such as @Private. If userspace tools were made available, developers could use this in ways that we have not yet conceived.

I understand and acknowledge the problems with this approach as outlined in the linked article. However, creating a solution that has the outlined composition problems isn't a great idea either. I think what @claytongulick has proposed is worth consideration.

[jkrems]

With this approach a developer can't have a private x and a public x in the same class.

They can still have this.x and this._x. Or this.x and this.$x. The difference is that with this.#x you can immediately tell that it's talking about a private variable while reading the method, without having to scan a different part of the class. Anyhow, I don't think it's the language's job to ensure that similar identifiers cannot be used in the same class.

[bakkot]

While it's fun to speculate about alternate approaches, the FAQ entry on Why isn't access this.x? addresses this fairly solidly, I think.

Absent a static type system, any proposal which allows this.x to refer to a private field isn't going to work, unless we've overlooked something there. Such proposals need to start by addressing the points made in the FAQ.

@blargism, I'm not sure I understand your point about composition. Can you clarify?

Edit to add:

In general naming two things x, albeit with the # prepended, isn't really something I would let pass in a code review.

Sure. But the main reason this matters is when the fields are split across a superclass and subclass: that is, a superclass has a field this.#x and a subclass this.x. It's important from a language design perspective to allow this because the superclass may be written by a different person than the person who wrote the subclass, and consumer shouldn't need to know about implementation details of the superclass - including e.g. the names of private fields.

[littledan]

It seems pretty clear that @private couldn't work. We could drop the feature if it's more trouble than it's worth, though. I'll bring this up in TC39 again to be sure; last time I brought up these objections, though, the committee opted to continue the work.

These objections have been discussed in many threads on the private state repo; I'm not sure what new ground there is to cover. I believe @protected should be possible as a decorator, as I documented in the private state repo.

[claytongulick]

@littledan I know I came to the conversation late, and I apologize for that - I've been watching other language features a lot more closely and for some reason this one was under my radar. I know you've done a ton of work on this feature and it's incredibly well thought out, and well discussed.

I suppose I'm just trying to make sure that no stone has been left unturned, since there seems to be consensus about the distastefulness of '#'.

I'm still scratching my head about your comment on how @private can't work - I'm probably being dense. On the surface, it seems to me like a trick similar to your example with @protected would work, with one small(?) addition - somehow getting information on the caller and the caller's context.

I think the whole basis of my thoughts around @private really hinge on that, so if it's not possible for a variety of reasons, I'll close this issue out with a big "nevermind!".

So, that you'd intercept the private property with a decorator and prevent it from actually being added to the class, then use a Proxy to observe access to the property. If we had caller info and context, whether that be via an addition to the decorator spec or a beefed up Function.caller, we could check to see if the function that's accessing the private is a member of the same class. I must be missing how this wouldn't satisfy the points you laid out in https://github.com/tc39/proposal-private-fields/blob/master/FAQ.md#why-isnt-access-thisx .

Performance wise, of course, this would be horrendous - so that's a really strong argument against the decorator concept - however, we already do all sorts of tricks in tight code to work around quirks in js performance, like bringing a closure variable into the local scope to prevent scope chain navigation. High performance loops accessing a private seem to be less of a use case to me than the ability to have a more generic system for access modifiers.

I really like your point about decorators providing an escape hatch for privates, and I agree - they totally do. It does sort of reinforce in my mind that access modifiers are a meta programming construct though. Does is make sense to have two ways of defining access? One baked into the language and one in user/library space? That seems like something that we'll take a knock on from folks coming from other languages.

Also, thanks to you and @bakkot for taking the time to rehash all this again, you've done a ton of great work here and it's really cool of you to take the time to review this. Beers are on me if you're ever in DFW.

[bakkot]

@claytongulick Ah, I understand your point about arguments.caller now; thank you for expanding. Even using such a mechanism, though, it would be difficult to arrange that everything worked as expected: e.g. that a class and subclass can share a private field of the same name, and that code outside the class can't observe the field, and that a method operating on private fields can't be tricked into accepting a non-instance of the class as if were an instance. It's probably doable, but painful, and it would involve a lot of proxy traps and at least one WeakMap. There's "some performance impact is acceptable" and then there's "if you have a private field, all code in your class is an order of magnitude slower".

Separately, I think we really don't want to add arguments.caller back into the language, or anything similar. That would not be a small addition to the language.

[claytongulick]

@bakkot right, the entire concept hinges on the ability to get the caller info, perhaps arguments.caller isn't the best way, but what would you think about expanding the decorator signature to pass in a caller_info structure?

Also, I agree on the performance problem and how unreasonable it is, but I guess my thought is that with the decorator approach, we give the power and decision making ability to the user/library for balancing performance v/s protection. For example, for someone who doesn't care about super strict checking and potential leaking, there could be a @fast_private - one that's "good enough" but doesn't handle all the cases you mentioned.

[littledan]

A nice thing about function.caller is that it skips strict functions, so it will skip everything in class bodies. V8's stack trace introspection lets you get the function names, but not the function identity.

[bakkot]

@claytongulick For this to work, proxy traps would need to know something about the code which invoked the trap: in particular, they'd need to know... all of the classes syntactically containing that code, I guess, and maybe something about eval, and maybe whether properties were accessed with a.foo or a['foo']. It's hard for me to see how we'd expose that information to proxy traps in a reasonable way.

Re: performance vs protection: the approach currently in the proposal gets both, at the cost of configurability. But the impression I get is that private fields are overwhelmingly the majority use case for language-supported access modifiers, so I don't know we ought to sacrifice either performance or protection to achieve more configurability.

[claytongulick]

@bakkot that's a head-scratcher for sure, but a good problem - I think. It's the same problem that arguments.caller and Function.caller were trying to solve, but to @littledan 's point, fell down on. Solving the "who called me, and how?" question via Proxys is a really interesting concept, that I hadn't thought of.

Until now I was thinking in terms of possibly expanding decorators to pass in a caller_info structure of some sort but with your Proxy approach it solves the problem a lot more generically.

Would it be necessary to pass more than the this context of the calling code?

[littledan]

@claytongulick I think @bakkot was explaining something that's rather the opposite--it's intractable to square using ordinary property keys for private state because it would require getting weird context information, which is probably a bad idea to provide.

I think the whole basis of my thoughts around @Private really hinge on that, so if it's not possible for a variety of reasons, I'll close this issue out with a big "nevermind!".

Unfortunately, I think it's time to do that--it's been explained here and elsewhere (e.g., the FAQ) why this would be infeasible.

[claytongulick]

@littledan I'm surprised that you closed this issue given the volume of feedback you've received on this feature over the past year. I'm pretty far from the first person who's raised an objection to changing the language as proposed here.

@bakkot was indeed conjecturing on why he thought passing caller context via Proxy traps would be difficult, and raised fair points about performance implications, which I agree with and address. Difficult doesn't mean impossible, and I think the idea of using Proxys to solve the problem that arguments.caller and Function.caller are clearly trying to address is an interesting notion.

I haven't seen anything in the FAQ or your other articles that would contradict the use of decorators as access modifiers at all, quite the opposite - they tend to reinforce the point.

Whether or not decorators end up being the correct solution for access modification, I'm confused why we're pushing forward with a proposal that's received voluminous negative feedback, consumes one of our only "free" characters, and doesn't completely solve the access modifier problem - i.e. it just punts on questions about future modifiers like protected - or in fact recommends using decorators.

While I'm sure it's exhausting to have yet another objection raised at Stage 2, it's premature to close down discussion on this issue and proposal.

[bakkot]

@claytongulick, to be clear, I think the issues I raise above are fatal.

The FAQ doesn't address the question of how decorators as spec'd could be used to implement private fields because they cannot, as far as I'm aware. If you have a coherent design in mind for what that might look like, we could discuss it further. Though also I don't think decorators are the right approach for private fields, given the strength of the encapsulation that private fields are intended to provide.

(Also, by bringing up proxy traps I thought I was addressing your proposal, since decorators are executed once at class instantiation, whereas access can be done at a much later point - that is, decorators do not run at the appropriate time to decide if external code should have access to a field: "exposing caller info to decorators" is not possible. For something like what you propose to work we would have to expose it to proxy traps: which, like I say, I don't see any way to do that reasonably, and separately I think we should not.)

[claytongulick]

@bakkot I was afraid that'd be the case with expanding the signature of decorators, I wasn't really sure how it would work to pass a caller_info structure to those shy of replicating Proxy type functionality via a callback - so yeah, that leaves Proxy traps, or xxxx.caller.

Honestly, I'm more intrigued by the idea that @lifaon74 mentioned in this issue about adding modifiers to the descriptor. I haven't dug deeply into it yet to bang it up against your and @littledan 's points in the FAQ and elsewhere, but on the surface it's an exciting possibility, sort of a middle course between a pure decorator approach and the new # language construct, that could be trivially managed via decorators - while also providing for expansion in the future.

I do think there could be merit in the idea of expanding Proxy to include caller info. I'm going to noodle on that a bit and perhaps draw up a proposal if it makes sense, but I think that's an independent issue.

I think that @lifaon74 's approach makes more sense than my proposal for a pure decorator approach with runtime traps, so I think I'll follow that thread to see where it goes.

Thanks again for your time and attention and thoughtful responses here!