stanford-ppl/forge

A prototype meta DSL that generates Delite DSL implementations from a specification-like program.

Forge

A prototype meta DSL that generates Delite DSL implementations from a specification-like program.

Structure

A Forge DSL consists of a Forge specification (see src/examples) and optional applications (apps) and external code (extern). Forge generates DSL implementations from the specification, which are combined with the apps and external components to produce a complete DSL with examples. The publish script (bin/publish) automates this process for apps and external components located in apps/{dslName} and extern/{dslName}, by convention.

Alternatively, a user can simply copy the results of the Forge generation to another location to combine it with existing components.

Installation

Forge currently works with and requires Lightweight Modular Staging (https://github.com/TiarkRompf/virtualization-lms-core) branch delite-develop and Delite (https://github.com/stanford-ppl/Delite) branch develop.

1. Download and build both of these repositories on the corresponding branch. Make sure that you run sbt publish-local on each of them (in Delite, only the runtime and framework projects need to be published). Set the DELITE_HOME environment variable to the root of the Delite repository and the FORGE_HOME environment variable to the root of the Forge repository.

2. Compile Forge using sbt compile from the root of the Forge repository. The SimpleVectorDSL example can then be run using:

    bin/update ppl.dsl.forge.examples.SimpleVectorDSLRunner SimpleVector
    cd published/simplevector/
   

3. Run the sample application (apps/simplevector/src/HelloWorld.scala)

   Interpreter (library mode):

    bin/delitec HelloSimpleInterpreter
   

   Compiler (Delite mode):

    bin/delitec HelloSimpleCompiler
    bin/delite HelloSimpleCompiler
   

Using an existing Forge DSL

To use a Forge DSL that comes packaged with the Forge repository (e.g. OptiML, OptiQL), you can generate the DSL and then cd into the published folder (or move it to another location) and use it as normal, or you can put your applications in the <FORGE_HOME>/apps/<DSL>/src/ folder, which is automatically copied to the generated DSL directory during publishing.

To generate a DSL and move it to an external location (e.g. OptiML):

    bin/update ppl.dsl.forge.dsls.optiml.OptiMLDSLRunner OptiML
    mv published/OptiML <destination>

The destination will now contain the generated DSL, and you can add applications directly in the apps folder, and compile and run them using

    sbt compile
    bin/delitec <fully.qualified.appObject>
    bin/delite <fully.qualified.appObject>

You can also enter interactive mode using

    sbt "; console"

To run a DSL application located in <FORGE_HOME>/apps/<DSL>/src/, (e.g. Logistic Regression):

• Interpreter:

    bin/update -r "LogRegInterpreter <args>" ppl.dsl.forge.dsls.optiml.OptiMLDSLRunner OptiML
  

• Compiler:

    bin/update -d -r "LogRegCompiler <args>" ppl.dsl.forge.dsls.optiml.OptiMLDSLRunner OptiML
  

• Interactive:

    bin/update -i ppl.dsl.forge.dsls.optiml.OptiMLDSLRunner OptiML
  

If the DSL has already been generated, you can supply the -s 3 parameter to update to skip immediately to the publishing step.

forge's Issues

Block type not exposed to Delite Ops

Currently no support for calling functional arguments (MThunk and MFunction) from within the body of a Delite Op.

Consider this example:

val ptimes: Rep[DSLOp] = op (Profile) ("times", infix, List(), List(Profile, MThunk(MAny)), Profile, map ((MDouble, MDouble, Profile), 0, "x => {" + stream.printLines(
    "  val start = System.currentTimeMillis()",
    quotedArg(1),
    "  val end = System.currentTimeMillis()",
    "  (end - start)/1000d }"
)))

args not accessible inside of DSL application

As a side effect of arrays not being lifted =>

args(0) =>

[error] /afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/apps/src/Count.scala:12: Count.this.Rep[Array[String]] does not take parameters
[error] println(args(0))
[error] ^

implicit parameters for overloaded methods

On branch: owtemp
commit: 1e18709

incorrect number of implicit parameters generated for the second drop() implementation (1 instead of 2). Curiously other overloaded methods do not experience this problem.

Reproduce:

cd $FORGE_HOME ; sbt compile ; bin/forge ppl.dsl.forge.examples.OptiWrangleRunner ; bin/publish optiwrangle ; cd published/optiwrangle/ ; sbt compile

var type in application code produces huge error on compilation

This is a fraction of it, for example:

[error] HelloWrangler.this."overloaded1" // implicit val overloaded1: OverloadHack.this.Overloaded1 in trait OverloadHack, tree.tpe=HelloWrangler.this.Overloaded1
[error] )
[error] )
[error] Apply( // def println(__arg0: MiscOps.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext,implicit __imp1: MiscOps.this.Overloaded2): MiscOps.this.Rep[Unit] in trait MiscOps, tree.tpe=HelloWrangler.this.Rep[Unit]
[error] Apply( // def println(__arg0: MiscOps.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext,implicit __imp1: MiscOps.this.Overloaded2): MiscOps.this.Rep[Unit] in trait MiscOps, tree.tpe=(implicit __pos: scala.reflect.SourceContext, implicit __imp1: HelloWrangler.this.Overloaded2)HelloWrangler.this.Rep[Unit]
[error] HelloWrangler.this."println" // def println(__arg0: MiscOps.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext,implicit __imp1: MiscOps.this.Overloaded2): MiscOps.this.Rep[Unit] in trait MiscOps, tree.tpe=(__arg0: HelloWrangler.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext, implicit __imp1: HelloWrangler.this.Overloaded2)HelloWrangler.this.Rep[Unit]
[error] Apply( // implicit def StringStringToRep(x: String): LiftString.this.Rep[String] in trait LiftString, tree.tpe=HelloWrangler.this.Rep[String]
[error] HelloWrangler.this."StringStringToRep" // implicit def StringStringToRep(x: String): LiftString.this.Rep[String] in trait LiftString, tree.tpe=(x: String)HelloWrangler.this.Rep[String]
[error] Apply( // final def +(x$1: Any): String in class String, tree.tpe=String
[error] "Cut : "."$plus" // final def +(x$1: Any): String in class String, tree.tpe=(x$1: Any)String
[error] Apply( // tree.tpe=
[error] System.nanoTime.$minus(now)."$div"
[error] 1000000.0
[error] )
[error] )
[error] )
[error] )
[error] // 2 arguments
[error] Apply( // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=scala.reflect.SourceContext
[error] reflect.this."SourceContext"."apply" // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=(fileName: String, name: String, receiver: String, sourceInfo: List[(String, Int)])scala.reflect.SourceContext
[error] // 4 arguments
[error] "/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/apps/src/HelloWorld.scala"
[error] "println"
[error] "HelloWrangler"
[error] Apply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=List[(Null, Int)]
[error] TypeApply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=(xs: (Null, Int)_)List[(Null, Int)]
[error] immutable.this."List"."apply" // override def apply[A](xs: A*): List[A] in object List, tree.tpe=[A](xs: A*)List[A]
[error] // tree.tpe=(Null, Int)
[error] )
[error] Apply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(Null, Int)
[error] TypeApply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(_1: Null, _2: Int)(Null, Int)
[error] scala.this."Tuple2"."apply" // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=[T1, T2](_1: T1, 2: T2)(T1, T2)
[error] // 2 type arguments
[error] // tree.tpe=Null
[error] // tree.tpe=Int
[error] )
[error] // 2 arguments
[error] null
[error] 21
[error] )
[error] )
[error] )
[error] HelloWrangler.this."overloaded2" // implicit val overloaded2: OverloadHack.this.Overloaded2 in trait OverloadHack, tree.tpe=HelloWrangler.this.Overloaded2
[error] )
[error] Apply( // def __assign[T](lhs: Variables.this.Var[T],rhs: T)(implicit evidence$10: Manifest[T],implicit pos: scala.reflect.SourceContext): Variables.this.Rep[Unit] in trait Variables, tree.tpe=HelloWrangler.this.Rep[Unit]
[error] Apply( // def __assign[T](lhs: Variables.this.Var[T],rhs: T)(implicit evidence$10: Manifest[T],implicit pos: scala.reflect.SourceContext): Variables.this.Rep[Unit] in trait Variables, tree.tpe=(implicit evidence$10: Manifest[Long], implicit pos: scala.reflect.SourceContext)HelloWrangler.this.Rep[Unit]
[error] TypeApply( // def __assign[T](lhs: Variables.this.Var[T],rhs: T)(implicit evidence$10: Manifest[T],implicit pos: scala.reflect.SourceContext): Variables.this.Rep[Unit] in trait Variables, tree.tpe=(lhs: HelloWrangler.this.Var[Long], rhs: Long)(implicit evidence$10: Manifest[Long], implicit pos: scala.reflect.SourceContext)HelloWrangler.this.Rep[Unit]
[error] HelloWrangler.this."__assign" // def __assign[T](lhs: Variables.this.Var[T],rhs: T)(implicit evidence$10: Manifest[T],implicit pos: scala.reflect.SourceContext): Variables.this.Rep[Unit] in trait Variables, tree.tpe=[T](lhs: HelloWrangler.this.Var[T], rhs: T)(implicit evidence$10: Manifest[T], implicit pos: scala.reflect.SourceContext)HelloWrangler.this.Rep[Unit]
[error] // tree.tpe=Long
[error] )
[error] // 2 arguments
[error] "now" // var now: HelloWrangler.this.Var[Long], tree.tpe=HelloWrangler.this.Var[Long]
[error] Apply( // def nanoTime(): Long in object System, tree.tpe=Long
[error] java.this."lang"."System"."nanoTime" // def nanoTime(): Long in object System, tree.tpe=()Long
[error] Nil
[error] )
[error] )
[error] // 2 arguments
[error] reflect.this."ManifestFactory"."Long" // val Long: scala.reflect.AnyValManifest[Long] in object ManifestFactory, tree.tpe=scala.reflect.AnyValManifest[Long]
[error] Apply( // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=scala.reflect.SourceContext
[error] reflect.this."SourceContext"."apply" // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=(fileName: String, name: String, receiver: String, sourceInfo: List[(String, Int)])scala.reflect.SourceContext
[error] // 4 arguments
[error] "/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/apps/src/HelloWorld.scala"
[error] "__assign"
[error] "HelloWrangler"
[error] Apply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=List[(Null, Int)]
[error] TypeApply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=(xs: (Null, Int))List[(Null, Int)]
[error] immutable.this."List"."apply" // override def apply[A](xs: A*): List[A] in object List, tree.tpe=[A](xs: A*)List[A]
[error] // tree.tpe=(Null, Int)
[error] )
[error] Apply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(Null, Int)
[error] TypeApply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(_1: Null, _2: Int)(Null, Int)
[error] scala.this."Tuple2"."apply" // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=[T1, T2](_1: T1, 2: T2)(T1, T2)
[error] // 2 type arguments
[error] // tree.tpe=Null
[error] // tree.tpe=Int
[error] )
[error] // 2 arguments
[error] null
[error] 23
[error] )
[error] )
[error] )
[error] )
[error] Apply( // def infix_tableToFile(self: TableOps.this.Rep[TableOps.this.Table],__arg1: TableOps.this.Rep[String])(implicit __pos: scala.reflect.SourceContext): TableOps.this.Rep[Unit] in trait TableOps, tree.tpe=HelloWrangler.this.Rep[Unit]
[error] Apply( // def infix_tableToFile(self: TableOps.this.Rep[TableOps.this.Table],__arg1: TableOps.this.Rep[String])(implicit __pos: scala.reflect.SourceContext): TableOps.this.Rep[Unit] in trait TableOps, tree.tpe=(implicit __pos: scala.reflect.SourceContext)HelloWrangler.this.Rep[Unit]
[error] HelloWrangler.this."infix_tableToFile" // def infix_tableToFile(self: TableOps.this.Rep[TableOps.this.Table],__arg1: TableOps.this.Rep[String])(implicit __pos: scala.reflect.SourceContext): TableOps.this.Rep[Unit] in trait TableOps, tree.tpe=(self: HelloWrangler.this.Rep[HelloWrangler.this.Table], __arg1: HelloWrangler.this.Rep[String])(implicit __pos: scala.reflect.SourceContext)HelloWrangler.this.Rep[Unit]
[error] // 2 arguments
[error] "x" // val x: HelloWrangler.this.Rep[HelloWrangler.this.Table], tree.tpe=x.type
[error] Apply( // implicit def StringStringToRep(x: String): LiftString.this.Rep[String] in trait LiftString, tree.tpe=HelloWrangler.this.Rep[String]
[error] HelloWrangler.this."StringStringToRep" // implicit def StringStringToRep(x: String): LiftString.this.Rep[String] in trait LiftString, tree.tpe=(x: String)HelloWrangler.this.Rep[String]
[error] "/afs/cs.stanford.edu/u/gibbons4/data/"
[error] )
[error] )
[error] Apply( // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=scala.reflect.SourceContext
[error] reflect.this."SourceContext"."apply" // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=(fileName: String, name: String, receiver: String, sourceInfo: List[(String, Int)])scala.reflect.SourceContext
[error] // 4 arguments
[error] ""
[error] "infix_tableToFile"
[error] "HelloWrangler"
[error] Apply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=List[(Null, Int)]
[error] TypeApply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=(xs: (Null, Int))List[(Null, Int)]
[error] immutable.this."List"."apply" // override def apply[A](xs: A*): List[A] in object List, tree.tpe=[A](xs: A*)List[A]
[error] // tree.tpe=(Null, Int)
[error] )
[error] Apply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(Null, Int)
[error] TypeApply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(_1: Null, _2: Int)(Null, Int)
[error] scala.this."Tuple2"."apply" // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=[T1, T2](_1: T1, 2: T2)(T1, T2)
[error] // 2 type arguments
[error] // tree.tpe=Null
[error] // tree.tpe=Int
[error] )
[error] // 2 arguments
[error] null
[error] 0
[error] )
[error] )
[error] )
[error] )
[error] Block( // tree.tpe=Unit
[error] Apply( // def println(__arg0: MiscOps.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext,implicit __imp1: MiscOps.this.Overloaded2): MiscOps.this.Rep[Unit] in trait MiscOps, tree.tpe=HelloWrangler.this.Rep[Unit]
[error] Apply( // def println(__arg0: MiscOps.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext,implicit __imp1: MiscOps.this.Overloaded2): MiscOps.this.Rep[Unit] in trait MiscOps, tree.tpe=(implicit __pos: scala.reflect.SourceContext, implicit __imp1: HelloWrangler.this.Overloaded2)HelloWrangler.this.Rep[Unit]
[error] HelloWrangler.this."println" // def println(__arg0: MiscOps.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext,implicit __imp1: MiscOps.this.Overloaded2): MiscOps.this.Rep[Unit] in trait MiscOps, tree.tpe=(__arg0: HelloWrangler.this.Rep[Any])(implicit __pos: scala.reflect.SourceContext, implicit __imp1: HelloWrangler.this.Overloaded2)HelloWrangler.this.Rep[Unit]
[error] Apply( // implicit def StringStringToRep(x: String): LiftString.this.Rep[String] in trait LiftString, tree.tpe=HelloWrangler.this.Rep[String]
[error] HelloWrangler.this."StringStringToRep" // implicit def StringStringToRep(x: String): LiftString.this.Rep[String] in trait LiftString, tree.tpe=(x: String)HelloWrangler.this.Rep[String]
[error] Apply( // final def +(x$1: Any): String in class String, tree.tpe=String
[error] "Written from file : "."$plus" // final def +(x$1: Any): String in class String, tree.tpe=(x$1: Any)String
[error] Apply( // tree.tpe=
[error] System.nanoTime.$minus(now)."$div"
[error] 1000000.0
[error] )
[error] )
[error] )
[error] )
[error] // 2 arguments
[error] Apply( // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=scala.reflect.SourceContext
[error] reflect.this."SourceContext"."apply" // def apply(fileName: String,name: String,receiver: String,sourceInfo: List[(String, Int)]): scala.reflect.SourceContext in object SourceContext, tree.tpe=(fileName: String, name: String, receiver: String, sourceInfo: List[(String, Int)])scala.reflect.SourceContext
[error] // 4 arguments
[error] "/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/apps/src/HelloWorld.scala"
[error] "println"
[error] "HelloWrangler"
[error] Apply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=List[(Null, Int)]
[error] TypeApply( // override def apply[A](xs: A*): List[A] in object List, tree.tpe=(xs: (Null, Int))List[(Null, Int)]
[error] immutable.this."List"."apply" // override def apply[A](xs: A*): List[A] in object List, tree.tpe=[A](xs: A*)List[A]
[error] // tree.tpe=(Null, Int)
[error] )
[error] Apply( // case def apply[T1, T2](_1: T1,2: T2): (T1, T2) in object Tuple2, tree.timport optiwrangler.compiler.
pe=(Null, Int)
[error] TypeApply( // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=(_1: Null, _2: Int)(Null, Int)
[error] scala.this."Tuple2"."apply" // case def apply[T1, T2](_1: T1,_2: T2): (T1, T2) in object Tuple2, tree.tpe=[T1, T2](1: T1, 2: T2)(T1, T2)
[error] // 2 type arguments
[error] // tree.tpe=Null
[error] // tree.tpe=Int
[error] )
import optiwrangler.compiler.
[error] // 2 arguments
[error] null
[error] 25
[error] )
[error] )
[error] )
[error] HelloWrangler.this."overloaded2" // implicit val overloaded2: OverloadHack.this.Overloaded2 in trait OverloadHack, tree.tpe=HelloWrangler.this.Overloaded2
[error] )
import optiwrangler.compiler.
error
[error] )
[error] )
[error] )
[error]
[error] == Expanded type of tree ==
[error]
[error] TypeRef(TypeSymbol(final abstract class Unit extends AnyVal))
[error]
[error] System.nanoTime.$minus(now).$div(1000000.0)

Record lifting isn't functioning properly

Constructing a new Record { ... } from application code fails mysteriously in many cases. Possibly related to missing implicits or ambiguous implicits in scope.

Some scala string method names cannot be used as method names

As an example -

val split = op (Str) ("split", infix, List(), List(MString, MString) :: MArray(MString))
impl (split) (codegen($cala, quotedArg(0)+".split("+quotedArg(1)+")"))

generates (at DSL compilation time, ie cd published/OptiWrangler ; sbt compile)

/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/compiler/src/optiwrangler/    ops/StringOpsExp.scala:118: ambiguous reference to overloaded definition,
[error] both method string_split in trait StringOpsExp of type (__arg0:     StringOpsExp.this.Rep[String], __arg1: StringOpsExp.this.Rep[String])(implicit __imp0:     scala.reflect.SourceContext, implicit __imp1:     StringOpsExp.this.Overloaded4)StringOpsExp.this.Exp[StringOpsExp.this.ForgeArray[String    ]]
[error] and  method string_split in trait StringOpsExp of type (s: StringOpsExp.this.Exp[String], separators: StringOpsExp.this.Exp[String])(implicit pos: scala.reflect.SourceContext)StringOpsExp.this.Rep[Array[String]]
[error] match argument types (f.IR.Exp[String],f.IR.Exp[String])
[error]     case mn@String4_Split(__arg0,__arg1) => string_split(f(__arg0),f(__arg1))(mn.__imp0,implicitly[Overloaded4])
[error]                                             ^
[error]     /afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/compiler/src/optiwrangler/ops/TableOpsImpl.scala:69: value xsplit is not a member of TableOpsImpl.this.Rep[String]
[error]     self.flatMap(_.xsplit(__arg1), __arg2)
[error]                    ^
[error] one error found
[error] {file:/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/}OptiWrangler-lib/compile:compile: Compilation failed
[error] {file:/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/}OptiWrangler-comp/compile:compile: Compilation failed

(but renaming the method xsplit does not have this problem)

val xsplit = op (Str) ("xsplit", infix, List(), List(MString, MString) :: MArray(MString))
impl (xsplit) (codegen($cala, quotedArg(0)+".xsplit("+quotedArg(1)+")"))

Extra parenthesis around function in codegenerated code

$4(arg0, arg1)

generates

val b___arg4 = reifyEffects(__arg4((f___arg4___arg0,f___arg4___arg1)))

(should be?)

val b___arg4 = reifyEffects(__arg4(f___arg4___arg0,f___arg4___arg1))

my inability to create github flavored markdown aside, note __arg4 (( ... ))

Eliminate having the first argument being the calling object

Rather than having List(Vector, MInt) as the arguments, just have List(MInt). Instead of accessing calling object through quotedArg(0), make it this() (or something else?)

Overloaded methods with functional arguments not generated correctly

  infix ("cut") ((MString ==> MString, MAny) :: Table) implements composite ${...}

does not compile, but renaming it "cutF" does compile.

There is also the uniquely defined
infix ("split") ((MString ==> MString, MAny) :: Table) implements composite ${...} which does not compile if someone else is also named split

[error] /afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/library/src/optiwrangler/OptiWrangler.scala:29: method infix_cut overrides nothing
[error] override def infix_cut(self: Rep[Table],__arg1: (Rep[String]) => Rep[String],__arg2: Rep[Any])(implicit __pos: SourceContext,__imp1: Overloaded1) = table_cut(self,__arg1,__arg2)(__pos,__imp1)
[error]

ParallelCollectionBuffer does not work with non-Generic type

as seen running : bin/update ppl.dsl.forge.examples.IntVectorDSLRunner IntVector
from Austin-OW-Preprocess-Dev

Effects generated even if function is not called

I'm not sure that this ever gets called, but it came up looking at #29

=========== Forge code ===============

direct (Table) ("map_body", Nil, (MArray(MString), MArray(MInt), MArray(MInt), MString ==> MString, (MArray(MInt), MInt) ==> MBoolean) :: MArray(MString)) implements codegen ($cala, ${
  $0.zip($1).map{case(cell, index) =>
    //if($4($2, index)) $3(cell)
    //else cell
    cell
  }
})

=============generates=============

def table_map_body(__arg0: Rep[ForgeArray[String]],__arg1: Rep[ForgeArray[Int]],__arg2: Rep[ForgeArray[Int]],__arg3: (Rep[String]) => Rep[String],__arg4: (Rep[ForgeArray[Int]],Rep[Int]) => Rep[Boolean])(implicit __pos: SourceContext) = {

val f___arg3___arg0 = fresh[String]
val b___arg3 = reifyEffects(__arg3((f___arg3___arg0)))
val sb___arg3 = summarizeEffects(b___arg3)

val f___arg4___arg0 = fresh[ForgeArray[Int]]
val f___arg4___arg1 = fresh[Int]
val b___arg4 = reifyEffects(__arg4((f___arg4___arg0,f___arg4___arg1)))
val sb___arg4 = summarizeEffects(b___arg4)
reflectEffect(Table_Map_body(__arg0,__arg1,__arg2,b___arg3,f___arg3___arg0,b___arg4,f___arg4___arg0,f___arg4___arg1)(__pos), sb___arg3 andThen ((sb___arg4 andThen sb___arg3).star))

}

bin/publish uses sed in non compatible ways

BSD and GNU have different sed commands.

This may fix the problem:
sed -i'' -e 's/old_text/new_text/g' *

quoted newline character is read as a newline

direct (Table) ("toFile", Nil, (MString, MString, SSArray, MSI) :: MUnit) implements codegen ($cala, ${
  val of = new java.io.PrintStream(new java.io.FileOutputStream($0 + $1))
  of.println($2.map(x => x.mkString(",")).mkString("\n"))
  of.close()
})

[error] /afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/library/src/optiwrangler/classes/Table.scala:326: unclosed string literal
[error] of.println(__arg2.map(x => x.mkString(",")).mkString("
[error]

Separate types for type constructors and type instances

Currently, both Vector and Vector[Int] are Rep[DSLType], which leads to unchecked behavior (e.g., when doing a 'tpeInst'). These types should be more specific to make this saner.

bin/publish not compatible with case sensitive file system.

bin/publish simplevector
error: could not find generated DSL simplevector. (did you forget to run forge?)

bin/publish SimpleVector
== Publishing SimpleVector to /afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published//SimpleVector/
Traceback (most recent call last):
File "bin/publish", line 150, in
main()
File "bin/publish", line 32, in main
publish(args[0], opts)
File "bin/publish", line 117, in publish
os.mkdir(dest)
OSError: [Errno 2] No such file or directory: '/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published//SimpleVector//shared/src/SimpleVector/extern/'

======== actual directory structure ==========
published/SimpleVector/shared/src/simplevector/

Type checking DSL method parameters

A function which passes an incorrectly typed parameter generates an method not found error -

[error] /home/gibbons4/ppl/Forge/published/MapTest/apps/src/HelloWorld.scala:16: value ms_add is not a member of HelloWorld.this.Rep[HelloWorld.this.MultiSet]
[error] ms.ms_add(1.0)
[error] ^
[error] one error found

In DSL user code (first one is bad, second one is okay) -

ms.ms_add(1.0)
ms.ms_add(1.0f)

Lib impl makes assumptions about type par of infix methods

The library implementation currently assumes the name of the type parameter in the enclosing class and the name the of the type parameter in infix op arguments will be the same, when it is not necessary for them to be so to be semantically correct.

This is related to #12, since the declaration of a type constructor should not require specifying a concrete type parameter.

compiler DSL requires type parameter

From :

if (m.erasure == classOf[Table]) Some((classTag(m), collection.immutable.List(("_data",darrayManifest(m.typeArguments(0))),("_width",manifest[Int]),("_name",manifest[String]))))

(m.typeArguments is empty)

$ bin/delitec HelloWranglerCompiler
== Generating DEG file with options: {'profile': False, 'xdebug': False, 'verbose': False, 'xnocache': False, 'debug': False, 'opencl': False, 'no_fusion': False, 'build_dir': '/afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler', 'cluster': False, 'dump_exception': False, 'no_soa': False, 'cpp': False, 'no_blas': False, 'optimize': 0, 'cuda': False}
ERROR: ld.so: object '/usr/java/latest//jre/lib/amd64/libjsig.so' from LD_PRELOAD cannt be preloaded: ignored.
ERROR: ld.so: object '/usr/java/latest//jre/lib/amd64/libjsig.so' from LD_PRELOAD cannot be preloaded: ignored.
Delite Application Being Staged:[HelloWranglerCompiler$]
_Generating the program_
error: illegal sharing of mutable objects Sym(7)
at Sym(8)=Reflect(NewVar(Sym(7)),Summary(false,false,false,false,true,List(Sym(7)),List(Sym(7)),List(),List()),List(Sym(7)))
error: illegal sharing of mutable objects Sym(13)
at Sym(17)=Reflect(NewVar(Sym(13)),Summary(false,false,false,false,true,List(Sym(13)),List(Sym(13)),List(),List()),List(Sym(13)))
error: illegal sharing of mutable objects Sym(31)
at Sym(32)=Reflect(NewVar(Sym(31)),Summary(false,false,false,false,true,List(Sym(31)),List(Sym(31)),List(),List()),List(Sym(31)))
error: write to non-mutable Sym(100) -> Some(TP(Sym(100),Reflect(Table_Map_mapper(Sym(19),Sym(21),Sym(21),,),Summary(false,false,false,false,false,List(Sym(19)),List(),List(),List()),List(Sym(19)))))
at Sym(102)=Reflect(NestedFieldUpdate(Sym(100),List(_name),Sym(101)),Summary(false,false,false,false,false,List(),List(),List(Sym(100)),List(Sym(100))),List(Sym(100)))
error: write to non-mutable Sym(100) -> Some(TP(Sym(100),Reflect(Table_Map_mapper(Sym(19),Sym(21),Sym(21),,),Summary(false,false,false,false,false,List(Sym(19)),List(),List(),List()),List(Sym(19)))))
at Sym(104)=Reflect(NestedFieldUpdate(Sym(100),List(_header),Sym(103)),Summary(false,false,false,false,false,List(),List(),List(Sym(100)),List(Sym(100))),List(Sym(100), Sym(102)))
error: illegal sharing of mutable objects Sym(116)
at Sym(117)=Reflect(NewVar(Sym(116)),Summary(false,false,false,false,true,List(Sym(116)),List(Sym(116)),List(),List()),List(Sym(116)))
error: write to non-mutable Sym(183) -> Some(TP(Sym(183),Table_Map_mapper(Sym(100),Sym(106),Sym(106),,)))
at Sym(185)=Reflect(NestedFieldUpdate(Sym(183),List(_name),Sym(184)),Summary(false,false,false,false,false,List(),List(),List(Sym(183)),List(Sym(183))),List())
error: write to non-mutable Sym(183) -> Some(TP(Sym(183),Table_Map_mapper(Sym(100),Sym(106),Sym(106),,)))
at Sym(187)=Reflect(NestedFieldUpdate(Sym(183),List(_header),Sym(186)),Summary(false,false,false,false,false,List(),List(),List(Sym(183)),List(Sym(183))),List(Sym(185)))
Exception in thread "main" java.lang.IndexOutOfBoundsException: 0
at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:51)
at scala.collection.immutable.List.apply(List.scala:78)
at optiwrangler.compiler.ops.TableOpsExp$$anonfun$unapplyStructType$2.apply(TableOpsExp.scala:331)
at optiwrangler.compiler.ops.TableOpsExp$$anonfun$unapplyStructType$2.apply(TableOpsExp.scala:331)
at scala.virtualization.lms.common.IfThenElse$class.__ifThenElse(IfThenElse.scala:13)
at HelloWranglerCompiler$.__ifThenElse(HelloWorld.scala:5)
at optiwrangler.compiler.ops.TableOpsExp$class.unapplyStructType(TableOpsExp.scala:331)
at HelloWranglerCompiler$.ppl$delite$framework$datastructures$DeliteArrayOpsExpOpt$$super$unapplyStructType(HelloWorld.scala:5)
at ppl.delite.framework.datastructures.DeliteArrayOpsExpOpt$class.unapplyStructType(DeliteArray.scala:475)
at HelloWranglerCompiler$.unapplyStructType(HelloWorld.scala:5)
at ppl.delite.framework.datastructures.DeliteStructsExp$StructType$.unapply(DeliteStruct.scala:158)
at ppl.delite.framework.datastructures.ScalaGenDeliteStruct$class.remap(DeliteStruct.scala:197)
at optiwrangler.compiler.OptiWranglerExp$$anon$4.ppl$delite$framework$datastructures$ScalaGenDeliteArrayOps$$super$remap(OptiWrangler.scala:68)
at ppl.delite.framework.datastructures.ScalaGenDeliteArrayOps$class.remap(DeliteArray.scala:574)
at optiwrangler.compiler.OptiWranglerExp$$anon$4.ppl$delite$framework$codegen$delite$overrides$DeliteScalaGenAllOverrides$$super$remap(OptiWrangler.scala:68)
at ppl.delite.framework.codegen.delite.overrides.DeliteScalaGenAllOverrides$class.remap(DeliteOverrides.scala:22)
at optiwrangler.compiler.OptiWranglerExp$$anon$4.optiwrangler$compiler$OptiWranglerCodegenScala$$super$remap(OptiWrangler.scala:68)
at optiwrangler.compiler.OptiWranglerCodegenScala$class.remap(OptiWrangler.scala:127)
at optiwrangler.compiler.OptiWranglerExp$$anon$4.remap(OptiWrangler.scala:68)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$$anonfun$emitAnyNode$4.apply(DeliteGenTaskGraph.scala:188)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$$anonfun$emitAnyNode$4.apply(DeliteGenTaskGraph.scala:127)
at scala.collection.immutable.List.foreach(List.scala:309)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$class.emitAnyNode(DeliteGenTaskGraph.scala:127)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$class.emitNode(DeliteGenTaskGraph.scala:38)
at ppl.delite.framework.DeliteApplication$$anon$1.emitNode(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.GenericNestedCodegen$class.emitNode(GenericCodegen.scala:213)
at ppl.delite.framework.DeliteApplication$$anon$1.scala$virtualization$lms$internal$GenericFatCodegen$$super$emitNode(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.GenericFatCodegen$class.emitNode(FatCodegen.scala:19)
at ppl.delite.framework.DeliteApplication$$anon$1.ppl$delite$framework$codegen$delite$generators$DeliteGenTaskGraph$$super$emitNode(DeliteApplication.scala:52)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$$anonfun$emitAnyNode$1.apply$mcV$sp(DeliteGenTaskGraph.scala:65)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$class.withEffectWrapper(DeliteGenTaskGraph.scala:44)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$class.emitAnyNode(DeliteGenTaskGraph.scala:64)
at ppl.delite.framework.codegen.delite.generators.DeliteGenTaskGraph$class.emitNode(DeliteGenTaskGraph.scala:38)
at ppl.delite.framework.DeliteApplication$$anon$1.emitNode(DeliteApplication.scala:52)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$traverseBlockFocused$3$$anonfun$apply$22.emitAnyNode$1(DeliteCodegen.scala:234)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$traverseBlockFocused$3$$anonfun$apply$22.apply(DeliteCodegen.scala:239)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$traverseBlockFocused$3$$anonfun$apply$22.apply(DeliteCodegen.scala:215)
at scala.collection.immutable.List.foreach(List.scala:309)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$traverseBlockFocused$3.apply(DeliteCodegen.scala:215)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$traverseBlockFocused$3.apply(DeliteCodegen.scala:204)
at scala.virtualization.lms.internal.NestedGraphTraversal$$anonfun$focusExactScopeSubGraph$1.apply(Traversal.scala:167)
at scala.virtualization.lms.internal.NestedGraphTraversal$class.withInnerScope(Traversal.scala:55)
at ppl.delite.framework.DeliteApplication$$anon$1.withInnerScope(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedGraphTraversal$class.focusExactScopeSubGraph(Traversal.scala:164)
at ppl.delite.framework.DeliteApplication$$anon$1.focusExactScopeSubGraph(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedBlockTraversal$class.focusExactScopeFat(BlockTraversal.scala:62)
at ppl.delite.framework.DeliteApplication$$anon$1.scala$virtualization$lms$common$LoopFusionOpt$$super$focusExactScopeFat(DeliteApplication.scala:52)
at scala.virtualization.lms.common.LoopFusionOpt$class.focusExactScopeFat(LoopFusionOpt.scala:207)
at ppl.delite.framework.DeliteApplication$$anon$1.ppl$delite$framework$transform$LoopSoAOpt$$super$focusExactScopeFat(DeliteApplication.scala:52)
at ppl.delite.framework.transform.LoopSoAOpt$class.focusExactScopeFat(MultiLoopSoA.scala:355)
at ppl.delite.framework.DeliteApplication$$anon$1.focusExactScopeFat(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedBlockTraversal$class.focusExactScope(BlockTraversal.scala:59)
at ppl.delite.framework.DeliteApplication$$anon$1.focusExactScope(DeliteApplication.scala:52)
at ppl.delite.framework.codegen.delite.DeliteCodegen$class.traverseBlockFocused(DeliteCodegen.scala:204)
at ppl.delite.framework.DeliteApplication$$anon$1.traverseBlockFocused(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedBlockTraversal$$anonfun$traverseBlock$1.apply$mcV$sp(BlockTraversal.scala:106)
at scala.virtualization.lms.internal.NestedBlockTraversal$$anonfun$traverseBlock$1.apply(BlockTraversal.scala:106)
at scala.virtualization.lms.internal.NestedBlockTraversal$$anonfun$traverseBlock$1.apply(BlockTraversal.scala:106)
at scala.virtualization.lms.internal.FatBlockTraversal$$anonfun$focusFatBlock$1.apply(FatTraversal.scala:17)
at scala.virtualization.lms.internal.NestedGraphTraversal$class.withInnerScope(Traversal.scala:55)
at ppl.delite.framework.DeliteApplication$$anon$1.withInnerScope(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedGraphTraversal$class.focusSubGraph(Traversal.scala:75)
at ppl.delite.framework.DeliteApplication$$anon$1.focusSubGraph(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedBlockTraversal$class.focusFatBlock(BlockTraversal.scala:55)
at ppl.delite.framework.DeliteApplication$$anon$1.scala$virtualization$lms$internal$FatBlockTraversal$$super$focusFatBlock(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.FatBlockTraversal$class.focusFatBlock(FatTraversal.scala:15)
at ppl.delite.framework.DeliteApplication$$anon$1.focusFatBlock(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedBlockTraversal$class.focusBlock(BlockTraversal.scala:52)
at ppl.delite.framework.DeliteApplication$$anon$1.focusBlock(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.NestedBlockTraversal$class.traverseBlock(BlockTraversal.scala:105)
at ppl.delite.framework.DeliteApplication$$anon$1.traverseBlock(DeliteApplication.scala:52)
at ppl.delite.framework.DeliteApplication$$anon$1.traverseBlock(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.GenericCodegen$class.emitBlock(GenericCodegen.scala:105)
at ppl.delite.framework.DeliteApplication$$anon$1.emitBlock(DeliteApplication.scala:52)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$emitSource$1.apply$mcV$sp(DeliteCodegen.scala:173)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$emitSource$1.apply(DeliteCodegen.scala:173)
at ppl.delite.framework.codegen.delite.DeliteCodegen$$anonfun$emitSource$1.apply(DeliteCodegen.scala:173)
at scala.virtualization.lms.internal.GenericCodegen$class.withStream(GenericCodegen.scala:95)
at ppl.delite.framework.DeliteApplication$$anon$1.withStream(DeliteApplication.scala:52)
at ppl.delite.framework.codegen.delite.DeliteCodegen$class.emitSource(DeliteCodegen.scala:173)
at ppl.delite.framework.DeliteApplication$$anon$1.emitSource(DeliteApplication.scala:52)
at ppl.delite.framework.DeliteApplication$$anon$1.emitSource(DeliteApplication.scala:52)
at scala.virtualization.lms.internal.GenericCodegen$class.emitSource(GenericCodegen.scala:116)
at ppl.delite.framework.DeliteApplication$$anon$1.emitSource(DeliteApplication.scala:52)
at ppl.delite.framework.DeliteApplication$class.main(DeliteApplication.scala:142)
at HelloWranglerCompiler$.main(HelloWorld.scala:5)
at HelloWranglerCompiler.main(HelloWorld.scala)
error: Delite compilation failed

unapplyStruct assumes type parameters (on parallel data?)

data is trying to create with m.typeArgumnets(0), even though the tpe() does not take type arguments.

override def unapplyStructType[T:Manifest]: Option[(StructTag[T], List[(String,Manifest[_])])] = {
val m = manifest[T]
if (m.erasure == classOf[Table]) Some((classTag(m), collection.immutable.List(("_data",darrayManifest(m.typeArguments(0))),("_width",manifest[Int]),("_header",manifest[SmallMap[String,Int]]),("_name",manifest[String]))))
else super.unapplyStructType(m)
}

Austin-Codegen-Dev pegged for deletion

If you don't need the changes any further then I think it is safe to delete this branch.

add MapReduce pattern

need a mapreduce pattern in forge to enable pattern matching on the domain op (rather than using a composite of map and reduce)

IR has incompatible type for non-parallel types

860c0b4 does not compile
1830937 does compile

(as in, the published dsl does not compile)
(bin/update ppl.dsl.forge.examples.OptiWranglerDSLRunner OptiWrangler)

1839037

// the difference is the parallelize as call
[error] /afs/cs.stanford.edu/u/gibbons4/ppl/Forge/published/OptiWrangler/compiler/src/optiwrangler/ops/TableOpsExp.scala:195: overriding value IR in trait ScalaFatCodegen of type scala.virtualization.lms.internal.Expressions with scala.virtualization.lms.internal.Effects with scala.virtualization.lms.internal.FatExpressions;
[error] value IR has incompatible type
[error] val IR: TableOpsExp
[error] ^
[error] one error found

bin/initialize out of date

• uses old extern format
• does not use bin/update
• uses old OpType
• uses old type parameterization in data
• does not use new implementation syntax

add DeliteOpHashFilterReduce

need DeliteOpHashFilterReduce pattern in forge for fused groupBy ops
should take a keyFunc, valFunc, reduceFunc, and condFunc

Consolidate Forge parallel pattern generators

Forge parallel patterns should use the language of generators (collect, reduce, hashCollect, hashReduce) to simplify the code generation and error checking required in Forge, similar to Delite. The various front-end instances (map, zip, filter, etc.) should be different parameterizations of the generators. We should not need to use the simpler DeliteOp interfaces at all (like DeliteOpMap), and instead should be able to generate all DeliteOpLoop[_] instances with the correct loop bodies.

Mirror methods get generated incorrectly if withBound is not used

If we pass a context bound as an implicit parameter in the implicitArgs list, e.g. TNumeric(T), then the argument in the generated mirror method is not properly wrapped in mtype/ntype/etc. The correct behavior occurs when context bounds are used explicitly, (e.g. T withBound TNumeric withBound ...).

We should either fix the code generation or disallow type class instances to be passed as implicit args (without using withBound).

DSL methods crash with the same name as library methods (name clash) after erasure

I don't think manifests can peek inside Rep[], causing this issue. Maybe TypeTags address this issue?

From MapTest.scala in - a0fd9bd
bin/update -c ppl.dsl.forge.examples.MapTestDSLRunner MapTest

[error] /home/gibbons4/ppl/Forge/published/MapTest/compiler/src/maptest/ops/MultiSetOpsExp.scala:22: name clash between inherited members:
[error] method infix_contains:(self: MultiSetOpsExp.this.Rep[MultiSetOpsExp.this.MultiSet], __arg1: MultiSetOpsExp.this.Rep[Float])(implicit __pos: scala.reflect.SourceContext)MultiSetOpsExp.this.Rep[Boolean] in trait MultiSetOps and
[error] method infix_contains:(s1: MultiSetOpsExp.this.Rep[String], s2: MultiSetOpsExp.this.Rep[String])(implicit pos: scala.reflect.SourceContext)MultiSetOpsExp.this.Rep[Boolean] in trait StringOps
[error] have same type after erasure: (self: Object, __arg1: Object, implicit __pos: reflect.SourceContext)Object
[error] trait MultiSetOpsExp extends MultiSetCompilerOps with BaseFatExp with DeliteStructsExp {
[error]       ^

Methods silently not being called.

With

val testMessage = "\"Accessed Test\""
val test = op (Table) ("test", infix, List(), List(Table), MUnit, codegenerated)
codegen (test) ($cala, "println("+testMessage+")")

and

my_table.test()

The Interpreted version prints the message and the compiled version does not. I believe this problem stems from IO methods that get mirror'd / reflected (only thing I noticed as common to the problem - codegenerated methods doing printing / writing to file)
Note that the only symptom (in this example) is that the message is not printed to screen - everything else still compiles and runs normally.

Reproduce:

cd $FORGE_HOME ; sbt compile ; bin/forge ppl.dsl.forge.examples.OptiWrangleRunner ; bin/publish optiwrangle ; cd published/optiwrangle/ ; sbt compile ; bin/delitec HelloSimpleInterpreter ; bin/delitec HelloSimpleCompiler ; bin/delite HelloSimpleCompiler

Implicits not in scope for parallel pattern functions

In the compiler version, implicit conversions are not in scope for Delite op functions (e.g. map, reduce, etc.), because these functions are generated in Exp traits, rather than Impl traits. This causes non-trivial op functions to be difficult to write, since implicits have to be called explicitly.

We should create wrappers for these in Impl files and call them from the Exp files. A workaround to this issue is to do this manually; e.g., create a single task that wraps the function body, and invoke the single task from the parallel pattern function implementation.

Type generation incorrect if data() not called

val Vega = tpe("Vega") generates type Vega = scala.Vega unless I include (the otherwise useless) data(Vega, List()), which creates class Vega() { as expected.

Is this by design?

In line comments lose their newline

does not compile :
val indices = array_range(0, 1) //getColumns($2)
set_data($self, array_mapForgeArray[String], ForgeArray[String]

does compile :
val indices = array_range(0, 1) //getColumns($2)
// this line intentionally left blank
set_data($self, array_map[ForgeArray[String], ForgeArray[String]](

(as in, the published dsl does not compile)
(bin/update ppl.dsl.forge.examples.OptiWranglerDSLRunner OptiWrangler)

(the dollar sign didn't make a difference)

implicit overloads can become ambiguous in the library implementation

inside the implementation of allocates, getter, and/or setter, (which are inlined in the library), overloaded implicits can become ambiguous by chance (depending on what overloaded values are assigned to which methods).

we can't remove the overload from signature since the method needs to correctly implement the interface, and we can't pass the possibly ambiguous implicits explicitly because those calls come from user (DSL author) code. so we need another way to prioritize the local implicit to resolve this.

Generating return types on DSL methods

I'm having some problems with addition/subtraction

First, I was inlining curr+1 and forge_int_plus(curr, 1), and this both result in type Int(0) but should be type String

Now I assign it to a variable and refer to that variable, and this generates a slew of error: not found (one for each kernel) - ie

/home/gibbons4/ppl/Delite/generatedCache/scala/src/runtime/Executable0.scala:8: error: not found: value kernel_x1
val xx1 : Unit = kernel_x1()
^
/home/gibbons4/ppl/Delite/generatedCache/scala/src/runtime/Executable0.scala:9: error: not found: value kernel_x2
val xx2 : scala.collection.mutable.HashMap[Float, Int] = kernel_x2()

This was from testing in Maps-Dev - https://github.com/stanford-ppl/Forge/blob/ee166f464d14ee5addb3bd6abfb900afcece388e/src/examples/MapTest.scala