Giter Site home page Giter Site logo

aminya / acuteml.jl Goto Github PK

View Code? Open in Web Editor NEW
6.0 6.0 4.0 1.56 MB

Acute Markup Language - HTML/XML in Julia

Home Page: https://aminya.github.io/AcuteML.jl/dev

License: MIT License

Julia 100.00%
dom html julia templating-engine web-framework web-library xml xml-files xpath

acuteml.jl's Introduction

Made more than 28,000 contributions on GitHub. Take a look at my CV here.

C++

Authored and contributed to several C++ projects that try to improve the C++ ecosystem:

Atom

I have been the head of @atom-community organization, in which we bring an integrated development environment to Atom. I have made more than 850 pull requests and issues for the Atom projects. Some of these projects are:

Julia

I was involved with many projects in the Julia ecosystem. I have made nearly 600 pull requests and issues for the Julia projects.

acuteml.jl's People

Contributors

alhirzel avatar aminya avatar github-actions[bot] avatar juliatagbot avatar

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

kristofferc alhirzel

acuteml.jl's Issues

Support for date and time types

Dates and Time:

  • Date
    Is covered under defined types, and it automatically is coneverted to the correct format
# Defining
Date(2013,7,1)

# Methods Check
string(Date(2013,7,1))
Date("2013-7-1")
<some-date>YYYY-MM-DD</some-date>
<some-date>2013-07-01</some-date>
  • Time
    Is covered under defined types, and it automatically is coneverted to the correct format
# Defining
Time(12,53,40)

# Methods Check
string(Time(12,53,40))
Time("12:53:40")
<some-time>hh:mm:ss</some-time>
<some-time>12:53:40</some-time>
  • DateTime
    Is covered under defined types, and it automatically is coneverted to the correct format
# Defining
DateTime(2013,5,1,12,53,40)

# Methods Check
# check
string(DateTime(2013,5,1,12,53,40))
DateTime("2013-05-01T12:53:40")
<some-datatime>YYYY-MM-DDThh:mm:ss</some-datatime> 
<some-datatime>2013-05-01T12:53:40</some-datatime> 
using Pkg 
Pkg.add("TimeZones")

using TimeZones

ZonedDateTime(DateTime(2014, 1, 1), tz"Europe/Warsaw")
# 2014-01-01T00:00:00+01:00

Full Example:

@aml struct event
date::Date
dateTime::DateTime
time::Time
end

https://www.w3schools.com/xml/schema_dtypes_date.asp

Extractor: ERROR: MethodError: Cannot `convert` an object of type Nothing to an object of type Array{

Hi,
I have this simple MWE:

using AcuteML

@aml mutable struct inner "~"
    id1::Int, att"~"
    desc1::String, "~"
end
@aml mutable struct outer doc"outer"
    id2::Int, att"~"
    innerList::Vector{inner}, "~"
end
i1=inner(id1=1, desc1="desc1")
i2=inner(id1=2, desc1="desc2")
o1=outer(id2=3, innerList=[i1, i2])
pprint(o1)

pprint("c:/tmp/o1.xml",o1)

Written xml file looks correct. However extracting results in error:

xml=parsexml(read("c:/tmp/o1.xml"))

o2=outer(xml)

ERROR: MethodError: Cannot `convert` an object of type Nothing to an object of type Array{inner,1}
Closest candidates are:
  convert(::Type{T}, ::AbstractArray) where T<:Array at array.jl:554
  convert(::Type{T}, ::T) where T<:AbstractArray at abstractarray.jl:14
  convert(::Type{T}, ::LinearAlgebra.Factorization) where T<:AbstractArray at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\LinearAlgebra\src\factorization.jl:55
  ...
Stacktrace:
 [1] outer(::Int64, ::Nothing, ::Document) at .\REPL[3]:1
 [2] outer(::Document) at C:\Users\znidar\.julia\packages\AcuteML\bfP7i\src\@aml\@aml_create\get_struct_xml_.jl:53
 [3] top-level scope at REPL[15]:1`

I checked example in documentation (it works ), however this one not. No idea what else to check.

Simon

Support for mutable struct

Now once an instance of a struct is built using some values, the aml field value remains the same, no matter what the fields value are.

Automatic xml/html name from variable name

Implement some logic, such that not providing a name in front of the variable will use the variable name as the xml/html name:

@aml struct People "people"
GPA, "~"
end

"~" means that use the variable name which is "GPA" as the html/xml name.

[Parsing] Automatic field creation for the remaining data

After a type is defined using @aml and we parse a xml/html file, there may be some parts of that xml/html string that are missing in the @aml. We should provide either

  1. automatic creation of fields for these parts of the string that are missing.
  2. allowing the user to xpath or addelm later.

Support for the defined types in argument list

Adding support for the types that are defined by the user in the argument list of the struct.

This problem is caused by the scoping issue of the macros.

@aml mutable struct Person "person"
    age::UInt, "age"
    field::String, "study-field"
    GPA::Float64, "GPA"
    courses::Vector{String}, "taken courses"
end

@aml struct University "university"
    name, "university-name"
    people::Vector{Person}, "students"
end

The definition for the University gives an error that Person is not defined.

New Document creation API

Changing

@aml mutable struct Doc xd""
    university::University, "~"
end

to

@aml mutable struct Doc "xml" 
    university::University, "~"
end

Updater bug when default is nothing

Updater returns an error that the aml field does not exist when it had been set to nothing during the first declaration. So updater should add the field in this case.

Keeping track of added elements 

addelement!(node, name, value)

adds an element to the node. Is there a way to keep track of this added element to access it later?(without the need to search again through the whole XML file).

We can use the returned value of addelement! like below:

julia> parent = ElementNode("parent")
EzXML.Node(<ELEMENT_NODE[parent]@0x00007f7face01c70>)

julia> child = addelement!(parent, "child")
EzXML.Node(<ELEMENT_NODE[child]@0x00007f7facb00140>)

julia> println(parent)
<parent><child/></parent>

julia> println(child)
<child/>

If extraction is done first, we should store the data somewhere to update it later.

Support for custom constructors

Required by MusicXML.jl.

In the Note:

    if pitch != nothing
        addelementOne!(aml, "pitch", pitch)
    elseif rest != nothing
        addelementOne!(aml, "rest", rest)
    elseif unpitched != nothing
        addelementOne!(aml, "unpitched", unpitched)
    else
        error("one of the pitch, rest or unpitched should be given")
    end

Support for specifying default values

Although the code is implemented it gives this error

@aml mutable struct Person "person"
    age::UInt, "age"
    field::String, "study-field"
    GPA::Float64=2, "GPA"
    courses::Vector{String}, "taken courses"
end
syntax "GPA::Float64=2, "GPA" " inside type definition is reserved

Allow direct attributes giving on a field

It helps to avoid creating new struct if the field is just a text and some attributes.
Attributes can be given in a Tuple with the xml name of them:

@aml mutable struct tr "~"
  th::String, "~", missing, ("class", "style")
end

Two commas were used because the first one is the place holder for the value checking function.

For accessing the Dict after creation, one should be able to do this:

tr.th # regular content
tr.th["class"] # attribute

Support for choice elements

Reference1
Reference2
Reference3
XML Schema choice element allows only one of the elements contained in the declaration to be present within the containing element.

<xs:element name="person">
  <xs:complexType>
    <xs:choice>
      <xs:element name="employee" type="employee"/>
      <xs:element name="member" type="member"/>
    </xs:choice>
  </xs:complexType>
</xs:element>

Text node support

Text nodes are not found using findfirst. They are only visible using nodes iterator (not elements).

Aml extract without namespace

In the case of the missing document, findfirst/findall(node) returns nothing, and so aml extractor method doesn't return anything.
Related to:
#6

Idea: use MacroTools.jl?

I know at this point it could be a heroic effort, but after looking at the parser for @aml, I thought it would be appropriate to suggest usage of MacroTools.jl and their @capture macro. It might be the biggest help around here:

AcuteML.jl/src/@aml/@aml_parse.jl

Lines 51 to 389 in 6b87580

########################
# Single struct name - "aml name"
if isa(ei, String)
# struct_function[1]=missing # function
# Self-name checker
if ei == "~"
if T isa Symbol
struct_name = string(T)
elseif T isa Expr && T.head == :curly
struct_name = string(T.args[1]) # S
end
else
struct_name = ei # Type aml name
end
argsexpr.args[i] = nothing # removing "aml name" from expr args
struct_nodetype = AbsNormal
################################################################
# Literal Struct name - empty"aml name"
elseif isa(ei, Tuple)
################################################################
# Struct aml
########################
# Literal only empty"aml name"
if isa(ei, Tuple{Type,String})
# struct_function[1]=missing # function
# Self-name checker
if ei[2] == "~"
if T isa Symbol
struct_name = string(T)
elseif T isa Expr && T.head == :curly
struct_name = string(T.args[1]) # S
end
else
struct_name = ei[2] # Type aml name
end
struct_nodetype = ei[1]
struct_nodetype = aml_dispatch(struct_nodetype, struct_name)
argsexpr.args[i] = nothing # removing "aml name" from expr args
# Custom Code
elseif isa(ei, Tuple{Symbol, Expr})
iMacro += 1
# Row for code insertion (insert before iArg-th argument)
if ei[1] == :creator
args_custom_creator[iArg] = ei[2]
elseif ei[1] == :extractor
args_custom_extractor[iArg] = ei[2]
elseif ei[1] == :updater
args_custom_updater[iArg] = ei[2]
end
argsexpr.args[i] = nothing # removing custom code macro from expr args
end
elseif ei isa Expr && ei.head == :tuple
########################
# Struct Function - "aml name", F
if isa(ei.args[1], String) && isa(ei.args[2], Union{Symbol,Function}) # "aml name", F
struct_function[1]=ei.args[2] # function
# Self-name checker
if ei.args[1] == "~"
if T isa Symbol
struct_name = string(T)
elseif T isa Expr && T.head == :curly
struct_name = string(T.args[1]) # S
end
else
struct_name = ei.args[1] # Type aml name
end
struct_nodetype = AbsNormal
argsexpr.args[i] = nothing # removing "aml name" from expr args
########################
# Literal and Struct Function - empty"aml name", F
elseif isa(ei.args[1], Tuple) && isa(ei.args[2], Union{Symbol,Function})
struct_function[1]=ei.args[2] # function
# Self-name checker
if ei.args[1][2] == "~"
if T isa Symbol
struct_name = string(T)
elseif T isa Expr && T.head == :curly
struct_name = string(T.args[1]) # S
end
else
struct_name = ei.args[1][2] # Type aml name
end
struct_nodetype = ei.args[1][1]
struct_nodetype = aml_dispatch(struct_nodetype, struct_name)
argsexpr.args[i] = nothing # removing "aml name" from expr args
################################################################
# Arguments
########################
# No Def Value
elseif ei.args[1] isa Union{Symbol,Expr} # var/var::T, "name"
# Def Value
# args_defaultvalue[iArg] = missing
# Type Checker
lhs = ei.args[1]
if lhs isa Symbol # var, "name"
var = ei.args[1]
args_type[iArg] = String # consider String as the type
args_param[iArg] = var
args_var[iArg] = var
argsexpr.args[i] = var # removing "name",...
elseif lhs isa Expr && lhs.head == :(::) && lhs.args[1] isa Symbol # var::T, "name"
var = lhs.args[1]
varType = lhs.args[2] # Type
args_type[iArg] = varType
args_param[iArg] = var
args_var[iArg] = var
argsexpr.args[i] = lhs # removing "name",...
end
# Literal Checker
if length(ei.args[2]) == 2 # literal
argAmlType = ei.args[2][1]
args_literaltype[iArg] = argAmlType # literal type
ni = ei.args[2][2]
# Self-name checker
if ni == "~"
args_name[iArg] = string(var)
else
args_name[iArg] = ni
end
else
args_literaltype[iArg] = AbsNormal # non-literal
ni = ei.args[2]
# Self-name checker
if ni == "~"
args_name[iArg] = string(var)
else
args_name[iArg] = ni
end
end
# Function Checker
if length(ei.args) == 3 && isa(ei.args[3], Union{Function, Symbol}) # var/var::T, "name", f
fun = ei.args[3] # function
args_function[iArg] = fun
# else # function name isn't given
# args_function[iArg] = missing
end
end # end Tuple sub possibilities
################################################################
# Def Value
elseif ei isa Expr && ei.head == :(=) # def value provided
# aml name Checker
if ei.args[2] isa Expr && ei.args[2].head == :tuple # var/var::T = defVal, "name"
# Def Value
defVal = ei.args[2].args[1]
args_defaultvalue[iArg] = defVal
lhs = ei.args[1]
argsexpr.args[i] = lhs # remove =defVal for type definition
# Type Checker
if lhs isa Symbol # var = defVal, "name"
var = ei.args[1]
args_type[iArg] = String # consider String as the type
args_param[iArg] = Expr(:kw, var, defVal)
args_var[iArg] = var
argsexpr.args[i] = var # removing "name",...
elseif lhs isa Expr && lhs.head == :(::) && lhs.args[1] isa Symbol # var::T = defVal, "name"
var = lhs.args[1]
varType = lhs.args[2] # Type
args_type[iArg] = varType
args_param[iArg] = Expr(:kw, var, defVal) # TODO also put type expression
args_var[iArg] = var
argsexpr.args[i] = lhs # removing "name",...
end
# Literal Checker
if length(ei.args[2].args[2]) == 2 # literal
argAmlType = ei.args[2].args[2][1]
args_literaltype[iArg] = argAmlType # literal type
ni = ei.args[2].args[2][2]
# Self-name checker
if ni == "~"
args_name[iArg] = string(var)
else
args_name[iArg] = ni
end
else
args_literaltype[iArg] = AbsNormal # non-literal
ni = ei.args[2].args[2]
# Self-name checker
if ni == "~"
args_name[iArg] = string(var)
else
args_name[iArg] = ni
end
end
# Function Checker
if length(ei.args[2].args) == 3 && isa(ei.args[2].args[3], Union{Function, Symbol}) # var/var::T = defVal, "name", f
fun = ei.args[2].args[3] # function
args_function[iArg] = fun
# else # function name isn't given
# args_function[iArg] = missing
end
########################
# No aml Name - But defVal
else # var/var::T = defVal # ignored for creating aml
# Type Checker
lhs = ei.args[1]
if lhs isa Symbol # var = defVal
defVal = ei.args[2]
args_defaultvalue[iArg] = defVal
# args_name[iArg] = missing # ignored for creating aml
# args_function[iArg] = missing # ignored for creating aml
var = ei.args[1]
args_type[iArg] = Any
args_param[iArg] = Expr(:kw, var, defVal)
args_var[iArg] = var
argsexpr.args[i] = var # remove =defVal for type definition
elseif lhs isa Expr && lhs.head == :(::) && lhs.args[1] isa Symbol # var::T = defVal
defVal = ei.args[2]
args_defaultvalue[iArg] = defVal
# args_name[iArg] = missing # ignored for creating aml
# args_function[iArg] = missing # ignored for creating aml
var = lhs.args[1]
varType = lhs.args[2] # Type
args_type[iArg] = varType
args_param[iArg] = Expr(:kw, var, defVal) # TODO also put type expression
args_var[iArg] = var
argsexpr.args[i] = lhs # remove =defVal for type definition
else
# something else, e.g. inline inner constructor
# F(...) = ...
continue
end
end
################################################################
# No aml name - No defVal
else # var/var::T # ignored for creating aml
# Type Checker
if ei isa Symbol # var
# args_name[iArg] = missing # argument ignored for aml
# args_function[iArg] = missing # ignored for creating aml
args_type[iArg] = String
var = ei
args_param[iArg] = var
args_var[iArg] = var
elseif ei.head == :(::) && ei.args[1] isa Symbol # var::T
# args_name[iArg] = missing # argument ignored for aml
# args_function[iArg] = missing # ignored for creating aml
var = ei.args[1]
varType = ei.args[2] # Type
args_type[iArg] = varType
args_param[iArg] = var
args_var[iArg] = var
elseif ei.head == :block # anything else should be evaluated again
# can arise with use of @static inside type decl
argsexpr, args_param, args_defaultvalue, args_type, args_var, args_name, args_function, args_literaltype, struct_name, struct_nodetype, struct_function, is_struct_mutable, args_custom_creator, args_custom_extractor, args_custom_updater, T = aml_parse(expr)
else
continue
end

Dict data support

  • Creation
  • Extraction/Updating

We should allow a field to be a Dict. Having a Dict as a filed means that user can use Dict instead of a struct to define/parse their html/xml data.

Getting custom code for defining methods

It is very useful to be able to get custom code from user and use them later for defining the extractor, creator, etc methods.

Two ways to do that:

  • have a field in each @aml structure, that is a Dict, and can store different codes
@aml mutable struct Foo
a::Int64, "~"
b::String, "~"
code = Dict(:creator_start => :(b=a), :extractor_end => :(if a>b a = b else b = a end))
end
  • have extra macros in each @aml
@aml mutable struct Foo
a::Int64, "~"
b::String, "~"
@aml_creator_start b = a
end

Self-name for the root name

Similar to what is done for the fields:

@aml struct Person "~"
# fields
end

Disadvantage:
In Julia, it is common to define a type with first letetr captalized, but in xml usually lower case is used.

Better Support for Self-closing (empty) fields

Now to define a self-closing field you should define a separate type. It would be nice if we can just specify its self-closing in the parent type definition.

 @aml mutable struct Scoreinstrument "score-instrument"
    name::String, "instrument-name"
    ensemble::UN{Int64} = nothing, sc"ensemble", positive
    solo::UN{Int64} = nothing, sc"solo", positive
end

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.