chris-twiner/scalesxml

Alternate Scala XML library

scalesxml's Issues

Scalaz Streams usage as replacement for IterV

Mutable iterate function - 40%+ speedup

Use a mutable approach inside the Iterator as the iteratee overhead is 40 odd %. This make sense in the onDone(List(onQName side, but for iterate it isn't required.

Project load fails due to undefined setting (scaladocOptions in Compile in doc)

[error] Reference to undefined setting:
[error]
[error] site/compile:doc::scaladoc-options from site/compile:doc::scaladoc-options
[error]

Thanks in advance :-)

Split out iteratees into their own library

IterV to match the naming convention of Scalaz

0.6.0 M2 Release

0.5.0 fixes and changes additionally against 2.11. Last build with current Scalaz dependencies on core and layouts.

Update build - diagrams in docs, sbt and coverage

sbt 0.13.6, diagrams working again, coverage attempts with jacoco again.

Smaller packages

The pre 0.5 package structure was still fairly flat, which can make both code navigation and discovery of the api less straightforward.

0.5 refactors a large chunk of the api into smaller more obvious packages.

There is a risk of source code incompatibility but the majority of applications should be untouched.

XPath DSL Documentation Improvements

Document the axe via a proper table:

XPath 1 Axe, Explanation, Scales DSL, example?

Also a how to understand XPath DSL page:

• construction of xpaths
• chaining predicates
• context (how does . keep within the context - what is it etc)
• predicates and steps with predicates (* vs *)

Add more content to scala docs and link the traits for attributes, elements etc directly

LazyTreeOptimisation one child bug

Silly refactoring bug

Scalaz Iteratee / Enumeratee based on Id is doomed to fail

Stackoverflows abound in actual usage for enumIterator or the custom version forcing Id. IO Trampoline's it seems. The M4 impl of iteratorEnumerator is also flawed now due to memory leaks. These are covered by various stack entries.

2.13 support

requires changing all the CC types and proxying them as 2.13 removes the repr arg from indexedseqlike and friends

0.6.0 M4 Release

allow relative initial contexts in string xpaths

. And .. or ./.. etc should be possible. Child axis and parent axis should be able to accept paths, currently only getDocumentNode wraps the paths

scala.collection.immutable is sealed, so MutableVectorLike can't be loaded

java.lang.SecurityException: sealing violation: can't seal package scala.collection.immutable: already loaded
    at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:395)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:417)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.scala$tools$nsc$util$ScalaClassLoader$$super$findClass(ScalaClassLoader.scala:86)
    at scala.tools.nsc.util.ScalaClassLoader$class.findClass(ScalaClassLoader.scala:51)
    at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.findClass(ScalaClassLoader.scala:86)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.scala$tools$nsc$util$ScalaClassLoader$$super$loadClass(ScalaClassLoader.scala:86)
    at scala.tools.nsc.util.ScalaClassLoader$class.loadClass(ScalaClassLoader.scala:57)
    at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.loadClass(ScalaClassLoader.scala:86)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at scales.xml.MutableVectorLikeStrategy$class.beginSubTree(OptimisingStrategies.scala:189)
    at scales.xml.QNameAndSpeedierStrategy$.beginSubTree(OptimisingStrategies.scala:202)
    at scales.xml.XmlParser$Handler.startElement(XmlParser.scala:147)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:506)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:376)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:602)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3063)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:881)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:607)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:116)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:488)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:835)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1210)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:568)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:302)
    at scales.xml.XmlParser$$anonfun$loadXml$1.apply(XmlParser.scala:29)
    at scales.xml.XmlParser$$anonfun$loadXml$1.apply(XmlParser.scala:27)
    at scales.xml.XmlFactories$DefaultSAXParserFactoryPool$$anon$1$$anonfun$loan$1.apply(XmlFactories.scala:40)
    at scales.xml.XmlFactories$DefaultSAXParserFactoryPool$$anon$1$$anonfun$loan$1.apply(XmlFactories.scala:40)
    at scales.utils.SimpleUnboundedPool$class.loan(Resources.scala:163)
    at scales.xml.XmlFactories$DefaultSAXParserFactoryPool$.loan(XmlFactories.scala:28)
    at scales.xml.XmlFactories$DefaultSAXParserFactoryPool$$anon$1.loan(XmlFactories.scala:40)
    at scales.xml.XmlParser$class.loadXml(XmlParser.scala:26)
    at scales.xml.package$.loadXml(package.scala:3)

etc.

sbt.ResolveException: download failed: maven-plugins#maven-cobertura-plugin

Hi, I have build.sbt:

name := "example"

version := "0.1"

scalaVersion := "2.9.2"

resolvers ++= Seq(
  "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases",
  "Typesafe repo" at "http://repo.typesafe.com/typesafe/repo",
  "Sonatype Snapshot" at "https://oss.sonatype.org/content/repositories/snapshots"
)


libraryDependencies ++= Seq(
    "com.github.scala-incubator.io" %% "scala-io-core" % "0.4.1-seq",
    "com.github.scala-incubator.io" %% "scala-io-file" % "0.4.1-seq",
    "org.scalesxml" %% "scales-xml" % "0.3.1",
    "org.scalesxml" %% "scales-jaxen" % "0.3.1"
)

but sbt can't resolve some libraries:

[info] Resolving findbugs#findbugsGUI;1.0.0 ...
[warn]  [NOT FOUND  ] maven-plugins#maven-cobertura-plugin;1.3!maven-cobertura-plugin.plugin (176ms)
[warn] ==== Typesafe Repository: tried
[warn]   http://repo.typesafe.com/typesafe/releases/maven-plugins/maven-cobertura-plugin/1.3/maven-cobertura-plugin-1.3.plugin
[warn]  [NOT FOUND  ] maven-plugins#maven-findbugs-plugin;1.3.1!maven-findbugs-plugin.plugin (172ms)
[warn] ==== Typesafe Repository: tried
[warn]   http://repo.typesafe.com/typesafe/releases/maven-plugins/maven-findbugs-plugin/1.3.1/maven-findbugs-plugin-1.3.1.plugin
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::              FAILED DOWNLOADS            ::
[warn]  :: ^ see resolution messages for details  ^ ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: maven-plugins#maven-cobertura-plugin;1.3!maven-cobertura-plugin.plugin
[warn]  :: maven-plugins#maven-findbugs-plugin;1.3.1!maven-findbugs-plugin.plugin
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[error] {file:/home/timothy/example/}default-307a69/*:update: sbt.ResolveException: download failed: maven-plugins#maven-cobertura-plugin;1.3!maven-cobertura-plugin.plugin
[error] download failed: maven-plugins#maven-findbugs-plugin;1.3.1!maven-findbugs-plugin.plugin

string() does not include CDATA

string() does not include content of CDATA in result.

Here it is example

import org.scalatest.FunSuite
import java.io.StringReader
import org.xml.sax.InputSource
import scales.xml._
import scales.xml.ScalesXml._
import scales.xml.Functions._

class StringAndCDagta extends FunSuite {
  test("Minimum items") {
    val input = """<root>
                  | <nested1>1</nested1>
                  | <nested2><![CDATA[1]]></nested2>
                  |    </root>
                  | """.stripMargin('|')
    val xml = pullXml(new InputSource(new StringReader(input)))
    val root = iterate(List("root" l), xml).next()
    assert( "1" === string(root \* "nested1"))
    assert( "1" === string(root \* "nested2"))
  }
}

It fails on second assert. I suppose CDATA text should be included in result of string()

Project load fails due to missing ensime-related dependency.

[error] /Users/arya/git/scalesXml/project/FullDocProject.scala:5: object ensime is not a member of package org
[error] import org.ensime.sbt.Plugin.Settings.ensimeConfig
[error]            ^
[error] /Users/arya/git/scalesXml/project/FullDocProject.scala:6: object ensime is not a member of package org
[error] import org.ensime.sbt.util.SExp._
[error]            ^

iteratee migration

allow 2.13 builds and more recent scalaz

0.5.0 release

Async Xml Pull

Scaladoc improvements

Revisit each file for current status - make sure 0.5 scaladocs are up-to-date / more complete.

0.5 M1 release

place holder for release

Saxon, validation and openjdk6 don't, by default, mix

The transformer impl is identified as saxon as such it does not serialize to stream by default. The rest of the jaxp interfaces seem not to be affected however.

The workaround for calling validate is to use asStreamSource first.

Replace should be allowed for root elems in foldPositions

The value of attribute "id" should be modified from "moduleCoreId" to "123".

val folded= foldPositions( one(root) ){
p => {
Replace(p.tree /@("id" ->"123"))
}
}

should yield a changed tree not an AddedBeforeOrAfterRoot fold error.

Historical code that only mattered in forward iterating folds, in reverse not only doesn't it matter but its counter-intuitive.

Upgrade Saxon Libs - attempt to fix / workaround saxon on openjdk6 build issues

Puzzle regarding multiple xpath usage

Hi there, I am having a bit of trouble when attempting to apply an xpath to the results of a previous xpath. In the code below, I should be able to select the level3 node returned by the first xpath, but no match. If I use // instead, that wrongly picks up both level3 nodes. Thanks for any ideas on this!

package com.test

import scales.utils._
import ScalesUtils._
import scales.xml._
import ScalesXml._
import Functions._
import org.xml.sax.XMLReader
import org.xml.sax.helpers.XMLReaderFactory
import scales.xml.jaxen._
import java.io.StringReader

object LocalFactoryPool extends SimpleUnboundedPool[XMLReader] with DefaultSaxSupport {

  // doesn't support xml version retrieval
  override def getXmlVersion(reader: XMLReader): AnyRef =
    null

  def create = {
    val reader: XMLReader = XMLReaderFactory.createXMLReader
    reader
  }

}

object TestNestedXpath {

  def xpathExpr(expr: String) =
    ScalesXPath(expr).withNameConversion(ScalesXPath.localOnly)

  def test = {
    val xmlText = """
      <level1>
        <level2>
          <level3>contents of level 3</level3>
          <shouldHideChildren>
            <level3>should not see</level3>
          </shouldHideChildren>
        </level2>
      </level1>      
    """

   val doc = loadXmlReader(new StringReader(xmlText), NoOptimisation, LocalFactoryPool)
   val root = top(doc)

   val level2 = xpathExpr("/level1/level2").xmlPaths(root).head
   println("found level2: " + string(level2))

   val level3matches = xpathExpr("/level3").xmlPaths(level2).toList
   println(level3matches.size match {
     case 1 => "pass"
     case x => "unexpected size: " + x
   })
  }

  def main(args: Array[String]): Unit = {
    test
  }

}

async pull not working

7 tests failing under 0.6, possibly scalaz, version wise probably not aalto. The aalto inject definitely has the correct content present, but the serializing iter does not.

0.6 release

Merge in 0.5, up Scalaz version, prepare merge in for #32

Pull : XmlCBF not used from addAndFocus/addChild

two imports added cleans up the performance - 50% impact.

0.5.0 Release

0.6.0 M3 Release

DSL ~> String should also allow Option[String]

currently only is possible with "" as the value, but it should be . I hadn't added as /( can accept Nil or one("Val"), but doesn't really fit. An npe for null isn't super either.

~> String should call ~> Option(str) .

AsyncDataChunkerEnumerator can no longer suspend under IO/Trampoline

testRandomAmounts works under Id, where the correct continuation is returned. Under IO/Trampoline it effectively restarts the iteratee's from their original continuation - likely due to an incorrect bind in enumToMany if it's at all possible to fix (*).

Swapping out the empty handling on pushXmlIter to use "theStatus" will allow the code to work under IO/Trampoline but clearly this heap based state management is not the way.

• this position assumes the iter returned is itself "correct" but not yet evaluated - as enumerators.apply do not get called within a monad we can only use point. This may imply the value we know is being returned correctly under Id is actually not evaluated correctly under Trampoline and can never be.

Xerces 2.9.1 Support

Sub project for test coverage

ReplaceWith bug (0.3-RC7)

I'm having some difficulty with ReplaceWith. I tried running one of the examples and ran into a similar issue, so I'm assuming something has broken in 0.3.

The example in the folding section under the "ReplaceWith - Nested" heading has code like such:

// for every child element add a text child that contains the qname of the elem
  def addTextNodes( op : XmlPath ) =
    foldPositions( op.\* ) { 
      p => Replace( p.tree / qname(p) ) 
    }

val nodes = top(builder). \\*(ns("Child2"))

 val res = foldPositions( nodes  ){
   _ => ReplaceWith(addTextNodes _)
 }

 asString(res.left.get.tree)

Which is supposed to yield:

<?xml version="1.0" encoding="UTF-8"?>
<Elem xmlns="test:uri" xmlns:pre="test:uri:attribs" pre:attr3="val3" attr2="val2" pre:attr1="val1">
  <Child/>
  Mixed Content
  <Child2>
    <Subchild>textSubchild</Subchild>
  </Child2>
</Elem>

But instead actually yields:

<?xml version="1.0" encoding="UTF-8"?>
<Elem xmlns="uri:test" xmlns:pre="test:uri:attribs" pre:attr3="val3" attr2="val2" pre:attr1="val1">
    <Child/>
    Mixed Content
    <Elem pre:attr3="val3" attr2="val2" pre:attr1="val1">
        <Child/>
            Mixed Content
        <Child2>
            <Subchild>textSubchild</Subchild>
        </Child2>
    </Elem>
</Elem>

Documentation links all point to 0.4.4

encoding issues

2.11 jdk 1.8/1.11 doesn't encode as expected (probably correctly though):

in BaseFunctionalityTest

testText
testTextOnly

XmlMarshallingTest

testAttrEncoding

move to maven

simplify, drop profiling etc. to get a clean build again

Remove internal onDone for iterate

2% performance increase for iterate users - use onQName directly.

fix interface for pulliteratees

the pulliteratees are by default imported so to use a monad specific onqname you need to alias the import which isn't good. The same "functions._" approach should be made

Project Status

What is the status of this project? I am looking into what java/scala solution to use to query relatively small (< 5mb) xml files, but this project doesn't look like it has had much movement as of late. I just don't want to choose something that doesn't have an upgrade path in the future.
Thanks.

Add empty and isNil functions

simple functions to get isNil out and check empty attributes and children

Can't build (repo not found)

I can't build since the repository in plugins.sbt can't be found:

https://scala-scales.googlecode.com/svn/repo

Are the sources of the dependencies available somewhere?

Scalaz Seven support

I am planning on starting the refactoring required for scalaz seven series support and wanted to check if there previous work or thoughts on it. Thanks.

XHTML empty tags - only 10 of them

From: http://justinsomnia.org/2005/12/there-are-only-10-legal-xhtml-empty-tags/ (thanks to Alex C. for the find)

only short form the following tags (all xmlns="http://www.w3.org/1999/xhtml" of course)

<area/>
<base/>
<br/>
<col/>
<hr/>
<img/>
<input/>
<link/>
<meta/>
<param/>

Attributes to be less memory intensive

swap implicit val for def - extend and drop this for each new attribute (ArraySet via ImmutableArrayProxy)

Optional trees in DSL

Currently the optional xml focusses on text and children items, a cascading approach is also needed, such that

root /( fred ?~> None )

will generate

<root/> 

only, not additionally

<root><fred/></root> 

as currently with the optional text child. This should cascade throughout the tree so that, if there are no children below there should be a None returned for the ItemOrElem.