Unable to compile with nested classes, referenced from different files

[vidma]

Unable to compile with nested classes, referenced from different files

[julianpeeters]

I'm sorry that you ran into this weird one!

The problem is likely that UserEx hasn't been expanded/had it's schema generated before it gets referenced. What's the name of the file that references UserEx? Does it start with a letter that is alphabetically before 'U'?

If you've ever wondered about the #DontUseMacros hashtag, we have here an example where code behavior depends on what one names the file... :)

There currently is no way to specify the order of macro expansion (perhaps that will change with scala.meta, still some time from now). I'll update the error and the docs to warn about this.

The workaround is to rename your files(!).

Alternatively, there's sbt-avrohugger. There are subtle pros and cons to each code gen strategy, but it is a pretty seamless way to get the same functionality as the annotation without the expansion order issues (currently generates code from schemas only, but you or I could add a feature to implement SpecificRecord from plain case classes).

[vidma]

All avro-annotated classes were in the same file, with the reused classes on the top. Playing with case-class-names didn't solve the issue.

surely this must be related to the lexicographic naming of other files which import/use the annotated classes.

so taking your example: https://github.com/julianpeeters/avro-scala-macro-annotations/tree/419e5696f6ff41c35dc6b38fb57d8642f60346c6/AvroRecordTest/src/main/scala/tutorial
We'd get: Cannot support yet: A.
renaming the models.scala to come before A.scala (which only import/use annotated class) solves the issue.
Happily moving models.scala into a lexicographically earlier package name also solves the issue! but it would be nice to have a proper solution...

[vidma]

Emgh, o you have a clear idea how is this lexicographical order defined?
The order looks slightly unpredictable, and thus rather demotivating...

[julianpeeters]

Hi vidma,

I wasn't able to compile after changing the file name of models.scala, nor
by changing the package name. Is it possible that the success that you
reported is due to an old class file that's hanging around from a
previously successful compilation?

Doing "sbt clean " before reach run reveals a consistent pattern for me:

Essentially it appears that the typer of scalac is expanding the macros
lazily, and whichever class is used first is the class that is expanded
first.

And "first" in this case means with respect to file names, as well as with
respect to definition order within a file.

If the nesting/nested classes are defined but not imported and used
anywhere, then they will compile as long as the nested class is defined
first (since the expansion of the nesting class depends on the nested class
having already been expanded).

Adding a file with a reference to the nested class is ok too.

It's only when adding a reference to the nesting class that things get
weird.

If the nesting class is referenced first, it will expand first. The nested
class will not have expanded yet, and hence the error (no schema gets
stored and critical field data is lost).

Does this description jive with your experience? Am I missing anything?

I'm guessing that file names play a role here because macros are expanded
in the scalac typer phase, and that's just how the typed works? Marches
through files alphabetically? Making symbols and adding type trees, but
doing god-knows-what exactly. I'd love to understand how the compiler
solves its dependency problems.

So, death by state, yet again. Alternatives to storing schemas are:

1. make them on demand, but that means halting compilation in order to
   inspect the required class and that seems exceedingly challenging.
2. automatic/implicit referencing of the classes in the right order would
   be nicer than the current workaround of manually doing so in an early file.
   I tried adding an implicit class to the case class's companion object, but
   that still required some manual intervention so I never got it to work.
3. Scala reflection? Hate to resort to reflection when using macros, but
   all we need is field data from a class definition that exists at macro
   runtime. There's possibly a roadblock getting a class from a string when
   operating across the chasm of a macro?
4. possibly an alternate @AvroRecord api? There may be a way to expand all
   macros at once. I couldn't do so with the current api. But maybe if we
   define our classes in a trait, annotate the trait, then have a package
   object extend the trait?

My tests passed because they all reference the nested class first, but I
had missed this valid use-case where nesting classes are the first/only
class used/referenced.

Any other suggestions?

I'll attempt 3 and 4 this weekend.
On Mar 6, 2015 5:46 AM, "vidma" [email protected] wrote:

Emgh, o you have a clear idea how is this lexicographical order defined?
The order looks slightly unpredictable, and thus rather demotivating...

—
Reply to this email directly or view it on GitHub
#6 (comment)
.

[julianpeeters]

Tried 3, led to an easy solution more along the lines of 1. Now expanding case classes if their type is encountered in the macro.

Does 0.4-SNAPSHOT resolve your issues too?

[vidma]

I get:

clean
:compileJava
:compileScala
no-symbol does not have an owner
.../Schemas.scala:5: exception during macro expansion:
scala.reflect.internal.FatalError:
     while compiling:FileThatUsesSchemasAndPossiblyLexicographicallyEarlier.scala
        during phase: global=typer, atPhase=namer
     library version: version 2.10.4
    compiler version: version 2.10.4

any ideas?

[julianpeeters]

Ah, I had only tested with Scala 2.11. 2.10 seems to choke on the very c.typeCheck that seemingly solved the problem!

Apparently that's a compiler bug that was fixed in Scala 2.10.5.

I'm afraid that if we require Scala 2.10.4 or less, we're stuck with version 0.3 of this project and one of the workarounds.

Upgrading to Scala 2.10.5, and using a compatible version of macro paradise ("2.1.0-M5") should solve this issue, however. Note that we still must *define our annotated classes with the most-nested classes defined first (no longer a limitation in Scala 2.11.5), but at least now we can *use them in an arbitrary order.

[julianpeeters]

I stumbled across a solution for Scala 2.10.4 and published it as version 0.4.

It's up at sonatype, and the source is in this branch.